DefinIT Because if IT were easy, everyone would do it…

6Mar/1336

PowerCLI Script to set RDM LUNs to Perennially Reserved – Fixes Slow Boot of ESXi 5.1 with MSCS RDMs

vmware logoI've previously posted around this topic as part of another problem but having had to figure out the process again I think it's worth re-posting a proper script for this. VMware KB 1016106 is snappily titled "ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS nodes with RDMs may take a long time to boot or during LUN rescan" and describes the situation where booting ESXi (5.1 in my case) takes a huge amount of time to boot because it's attempting to gain a SCSI reservation on an RDM disk used by MS Clustering Services. It also details the fix.

The process is fairly simple, but a bit labour intensive if you're doing it manually on a large cluster.

  1. Retrieve the ScsiCanonicalName for each RDM
  2. Set the configuration for each RDM on each Host to "PerenniallyReserved"

Step 1 - Retrieve the ScsiCanonicalName for each RDM

This PowerCLI command lists the RDM disks attached to any VMs in a particular cluster. What we need here specifically is the SCSI canonical name (often known as the naa or eui identifier), but for the sake of sanity I suggest running it manually and examining the disks to ensure you're happy with the ones the script will set to PerenniallyReserved:

Get-VM -Location "DefinIT Lab Cluster" | Get-HardDisk -DiskType "RawPhysical","RawVirtual" | Select Parent,Name,DiskType,ScsiCanonicalName
List RDM Disks in PowerCLI

List RDM Disks in PowerCLI

Step 2 - Set the configuration for each RDM on each Host to "PerenniallyReserved"

Once we have the list of RDM disks, we can then set the reservation via Get-EsxCli

Connect-VIServer "host01.definit.local" -Credential (Get-Credential)
$esxcli = Get-EsxCli
$esxcli.storage.core.device.setconfig($false, "naa.6005076307ffc7930000000000000109", $true)

Building Steps 1 and 2 into a script

Now it's just a simple case of constructing a script to loop through the hosts in a cluster and the RDMs on each host:

param(
	[string]	$TargetCluster,
	$RootCredentials = (Get-Credential)
)

# Create a connection object to all hosts in the Target Cluster
Get-Cluster $TargetCluster | Get-VMHost | % { Connect-VIServer $_ -Credential $RootCredentials | Out-Null }

# Find the ScsiCanonicalName for all RDM Disks attached to VMs in the Target Cluster
$RDMDisks = Get-VM -Location $TargetCluster | Get-HardDisk -DiskType "RawPhysical","RawVirtual" | Select ScsiCanonicalName

# Retrieve and EsxCli instance for each connection
foreach($esxcli in Get-EsxCli) {
	# And for each RDM Disk
	foreach($RDMDisk in $RDMDisks) {
		# Set the configuration to "PereniallyReserved".
		# setconfig method: void setconfig(boolean detached, string device, boolean perenniallyreserved)
		$esxcli.storage.core.device.setconfig($false, ($RDMDisk.ScsiCanonicalName), $true)
	}
}

# Disconnect the connection objects created for the Target Cluster
Disconnect-VIServer * -Confirm:$false | Out-Null

The script connects to each ESXi instance in the target cluster, queries all VMs on the target cluster and returns the RDM disks and then sets the PerenniallyReserved flag for each of the cluster hosts and RDM disks.

As usual, post any comments or improvements!

Edit: After my conversations with Mike, I've modified the script a little and attached as a zip file to download here: Set-RDMReservations

 

 

Creative Commons License
PowerCLI Script to set RDM LUNs to Perennially Reserved – Fixes Slow Boot of ESXi 5.1 with MSCS RDMs by DefinIT, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Posted by Sam McGeown

Comments (36) Trackbacks (0)
  1. Hey Sam!

    This is exactly what I was looking for. However if I use the script you created I get a error:

    = : Missing expression after ‘=’.
    + CategoryInfo : ParserError: (=:String) [], ParseException
    + FullyQualifiedErrorId : MissingExpressionAfterToken

    • Hi – is that the full error message? Does it prompt you for credentials or not get that far? Can you try putting brackets around Get-Credential at the beginning? Line 3

      Sam

      • Here is the full error – no prompt for creds.

        At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:3 char:35
        + [PSCredential] $RootCredentials = Get-Credential)
        + ~
        Missing expression after ‘=’.
        At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:3 char:33
        + [PSCredential] $RootCredentials = Get-Credential)
        + ~
        Missing ‘)’ in function parameter list.
        At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:3 char:50
        + [PSCredential] $RootCredentials = Get-Credential)
        + ~
        Unexpected token ‘)’ in expression or statement.
        + CategoryInfo : ParserError: (:) [], ParseException
        + FullyQualifiedErrorId : MissingExpressionAfterToken

        • Ok, that’s definitely the brackets around the get-credential, I’ll update the script! Must have lost them in the paste as they’re in my original file! Let me know if that solves it for you?

          • Yeah – when I originally copied the code the formatting didn’t come over clean and was pasted onto one line. I’ve tested the copy on Powershell ISE and PowerGUI with the same results. After updating the brackets I do get further however it gets stuck on the $TargetCluster in line 7. See error below:

            Get-Cluster : Cannot validate argument on parameter ‘Name’. The argument is null or empty. Supply an argument that is not null or empty and then try the command again.
            At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:7 char:13
            + Get-Cluster $TargetCluster | Get-VMHost | % { Connect-VIServer $_ -Credential $R …
            + ~~~~~~~~~~~
            + CategoryInfo : InvalidData: (:) [Get-Cluster], ParameterBindingValidationException
            + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetCluster

            Get-VM : 5/8/2013 2:02:13 PM Get-VM VIContainer parameter: Could not find any of the objects specified by name.
            At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:10 char:13
            + $RDMDisks = Get-VM -Location $TargetCluster | Get-HardDisk -DiskType “RawPhysica …
            + ~~~~~~~~~~~~~~~~~~~~~~~~~~
            + CategoryInfo : ObjectNotFound: (VMware.VimAutom…iner[] Location:RuntimePropertyInfo) [Get-VM], ObnRecordProcessingFailedException
            + FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ObjectNotFoundCritical,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetVM

            Get-EsxCli : 5/8/2013 2:02:13 PM Get-EsxCli Value cannot be found for the mandatory parameter VMHost
            At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:13 char:20
            + foreach($esxcli in Get-EsxCli) {
            + ~~~~~~~~
            + CategoryInfo : NotSpecified: (:) [Get-EsxCli], VimException
            + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxCli

            Disconnect-VIServer : 5/8/2013 2:02:13 PM Disconnect-VIServer Could not find any of the servers specified by name.
            At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:23 char:2
            + Disconnect-VIServer * -Confirm:$false | Out-Null
            + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            + CategoryInfo : ObjectNotFound: (VMware.VimAutom…Server[] Server:RuntimePropertyInfo) [Disconnect-VIServer], ServerObnFailureException
            + FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ServerSpecifiedButNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.DisconnectVIServer

          • That seems to indicate that you’ve not put the $TargetCluster parameter into the script when you called it? I’ve not made it a mandatory parameter so perhaps it’s not too clear, call it using the name of the script, then the target cluster, e.g:

            .\Set-RDMReservation.ps1 "Production Cluster 1"

          • Hey Sam – thanks for the help. I’ve tried that too however when I call out the cluster when running the script it errors out on Get-Cluster with “you are not currently connected to any servers” even though I’ve when opening PowerCLI and getting prompted from the execution of the script.

          • Are you connected to a vCenter Server? Connect-VIServer “servername” first!

          • Of course!

          • Hmmm, not sure what to suggest here – I have just run the script without any issue:

            Connect-VIserver "virtualcenter"
            .\Set-RDMReservations.ps1 "Production Cluster 1"

            It prompts me for credentials and then processes the RDMs:

            Set-RDMReservations.ps1

          • I noticed you were running Release 2 so I just upgraded my version. Got a slightly different error however its still credential related. Any way you can send me the original code so I can compare it to what im using?

            Connect-VIServer : 5/9/2013 10:57:34 AM Connect-VIServer Cannot complete login due to an incorrect user name or
            password.
            At H:\vmware_scripts\RDM_Perennially\RDM.ps1:7 char:47
            + Get-Cluster $TargetCluster | Get-VMHost | % { Connect-VIServer $_ -Credential $R …
            + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            + CategoryInfo : NotSpecified: (:) [Connect-VIServer], InvalidLogin
            + FullyQualifiedErrorId : Client20_ConnectivityServiceImpl_Reconnect_Exception,VMware.VimAutomation.ViCore.Cmdlets.Comm
            ands.ConnectVIServer

          • The code is now linked above, that’s exactly the code I am using. The new error suggests that you aren’t supplying the root credential to log into the ESXi host? The credentials for vCenter won’t do here as it needs to get an ESXCLI instance.

          • Seems to work now! The code from the link above worked and it appears to bit slightly different from what I originally copied. Thanks again for all the help! This is gonna save me tons of time on my 5.1 upgrades. My hosts are take 3+ hrs. to boot. I’ll report back on timings after I put these changes in.

          • Great! I think a re-write of how to use the script might help a bit :-)

          • Seems like you have to reconnect to the VC before running this script regardless if you have done so when first launching PowerCLI. Ran this on my Cluster and my reboots are running about 7mins, down from 2.5-3hrs! Thanks again Sam!!

          • That’s great news, glad to be of help! The script disconnects all the sessions at the end because it creates one for each host, so that probably explains the need to connect. I can probably sort that fairly easily, but it’s not the sort of script I envisage needing to run too often!

  2. Any suggestions on how to write a script that would display the “Perennially reserved =” status for each RDM visible to each host?

    By the time you get an ESXi cluster of 9 hosts, with a dozen or so MSCS clusters, each with 3 or 4 RDMs each, it becomes cubersome to make sure you have created a reservation for each one. And if they don’t have reservations, then the maitre d’ can make you wait an hour for a table, and you end up waiting in the lounge drinking Manhattans on an empty stomach and getting into a debate about how long the Euro will last with a hapless bystander.

    So you see why I need to have a script like that.

    Thanks for any help you might be able to provide.

  3. love the script, thank you!! However how could I make the script run on just one host vs. the entire cluster?

    • Yep – should be able to change the lines:

      Get-Cluster $TargetCluster | Get-VMHost | % { Connect-VIServer $_ -Credential $RootCredentials | Out-Null }
      $RDMDisks = Get-VM -Location $TargetCluster | Get-HardDisk -DiskType “RawPhysical”,”RawVirtual” | Select ScsiCanonicalName

      to

      Get-VMHost $TargetCluster | % { Connect-VIServer $_ -Credential $RootCredentials | Out-Null }
      $RDMDisks = Get-VMHost $TargetCluster | Get-VM | Get-HardDisk -DiskType “RawPhysical”,”RawVirtual” | Select ScsiCanonicalName

      and then supply the VMHost name as the $TargetCluster parameter when you run the script.

  4. Are you supposed to get some sort of output in the powercli window when you run the script? It prompts for credentials, gives me the certificate errors, then it just bumps me back to the command prompt. But if I use esxcli I can see the luns reserved.

  5. Forgot to mention, was getting this too: Unable to find type [PSCredential}: make sure that the assembly containing this type is loaded. I got past it by removing PSCredentials from the script and just making it $RootCredentials = (get-credential).

    • Hi – thanks for this, I’ve seen this when I am running the script, I think it might be the difference between Powershell 2 and 3, it used to work fine for me. I’ll remove it from the script :)

  6. Hi Sam, thanks for this script.. immensely usefull! I’m stuck with an error :Could not find VIContainer with name ‘Clusters VFarm’. I tried to pass the cluster name right after the script name in the powercli console. I also tried to pass the argument in the script $TargetCluster = “Clusters VFarm” and then leave the rest of script with $TargetCluster. I also tried this:

    param(
    #[string] $TargetCluster = “Clusters VFarm”,
    $RootCredentials = (Get-Credential)
    )

    # Create a connection object to all hosts in the Target Cluster
    Get-Cluster “Clusters VFarm” | Get-VMHost | % { Connect-VIServer $_ -Credential $RootCredentials | Out-Null }

    # Find the ScsiCanonicalName for all RDM Disks attached to VMs in the Target Cluster
    $RDMDisks = Get-VM -Location “Clusters VFarm” | Get-HardDisk -DiskType “RawPhysical”,”RawVirtual” | Select ScsiCanonicalName

    # Retrieve and EsxCli instance for each connection
    foreach($esxcli in Get-EsxCli) {
    # And for each RDM Disk
    foreach($RDMDisk in $RDMDisks) {
    # Set the configuration to “PereniallyReserved”.
    # setconfig method: void setconfig(boolean detached, string device, boolean perenniallyreserved)
    $esxcli.storage.core.device.setconfig($false, ($RDMDisk.ScsiCanonicalName), $true)
    }
    }

    but nothing. We am I wrong?

    • Are you connected to your vCenter? What is the output of “Get-Cluster” on it’s own?

      • thanks for replying!

        it’s correct.. it says:

        PowerCLI C:\Scripts> Get-Cluster “Clusters VFarm”

        Name HAEnabled HAFailover DrsEnabled DrsAutomationLevel
        Level
        —- ——— ———- ———- ——————
        Clusters VFarm False 1 True FullyAutomated

        I connect to the vcenter before executing the script.. I’m prompted for pw and then I launch the script..

        • we have several datacenters in our vsphere client.. but I don’t think it should make a difference.. else the Get-Cluster command on its own should fail..

          • Hi Sam
            I’ve changed the script to do a single ESXi host of that vCluster and it works. It failed with the same error at first, then instead of just the host name I entered the fqdn and it worked.. I’ll run the script four times.. not too bad anyway!! Still better than entering 600 naa identifiers in the Host profile!!

          • OK – glad it can save you some time – not sure why it’s throwing that error though! I have multiple datacenters and it seems to work fine? Does it make any difference if you use the -Name parameter in the script when doing a Get-Cluster?

  7. well.. actually thinking about it twice.. I’ve just reserved those naa that belongs to those VMs on that particular ESXi host. Unfortunately each ESXi host when booted see all the other datastores and RDMs inside that belong to the other VMs on the other ESXi hosts. Therefore…if I’m not mistaking I need to get the original script working. I’ll use the name paramater and let you know.. thanks Sam!

  8. well.. the Get-Cluster -Name $TargetCluster didn’t work and it disconnected the ESXi nodes from vcenter. It stayed hours without succeeding or throwing errors.. I had to stop it.. I’ll see to find a script that passes all those naa.identifiers to each host.. thanks anyway!

  9. Here is a simple effective way to get it done also. Simply create a text file with the naa.xxx numbers. Point this script to it run and your done. The output file is great as it confirms your change worked. You MUST run it on all hosts in the cluster!

    $HostName = “YourHostName”
    Connect-VIServer -Server “$HostName.YourDomain.com” -User YourUserHere -Pass YourPWHere
    $myesxcli= get-esxcli -VMHost “$HostName.YourDomain.com”
    Get-Content c:\PowerCLI\naaList.txt | Foreach-Object {
    $myesxcli.storage.core.device.setconfig($false,$_, $true)
    $myesxcli.storage.core.device.list($_) > C:\PowerCLI\$HostName\$_.txt
    }

  10. The VMware guide mentions to set this on the esxi hosts that are serving the passive MSCS nodes. Did you run this on all hosts in your cluster including the hosts with active MSCS nodes? If yes can you safely run the command on the host serving the active node? I’ve got 4 of 5 hosts upgraded from 5.0 to 5.5 and I’m trying to resolve this prior to upgrading the last host with the active node.

    http://pubs.vmware.com/vsphere-55/index.jsp?topic=%2Fcom.vmware.vsphere.mscs.doc%2FGUID-E794B860-E9AB-43B9-A6D0-F7DE222695A1.html

    • Hi Tom,

      I’ve had no problems running this on the host with the active node, I’ve done it several times. I set the reservation on any host that the LUN is presented to regardless of active/passive node – if you fail over you don’t want to have to then set the reservation on the new passive node.

      Hope that helps,

      Sam


Leave a comment

No trackbacks yet.