PowerCLI Script to set RDM LUNs to Perennially Reserved – Fixes Slow Boot of ESXi 5.1 with MSCS RDMs

vmware logoI’ve previously posted around this topic as part of another problem but having had to figure out the process again I think it’s worth re-posting a proper script for this. VMware KB 1016106 is snappily titled “ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS nodes with RDMs may take a long time to boot or during LUN rescan” and describes the situation where booting ESXi (5.1 in my case) takes a huge amount of time to boot because it’s attempting to gain a SCSI reservation on an RDM disk used by MS Clustering Services. It also details the fix.

The process is fairly simple, but a bit labour intensive if you’re doing it manually on a large cluster.

  1. Retrieve the ScsiCanonicalName for each RDM
  2. Set the configuration for each RDM on each Host to “PerenniallyReserved”

Step 1 – Retrieve the ScsiCanonicalName for each RDM

This PowerCLI command lists the RDM disks attached to any VMs in a particular cluster. What we need here specifically is the SCSI canonical name (often known as the naa or eui identifier), but for the sake of sanity I suggest running it manually and examining the disks to ensure you’re happy with the ones the script will set to PerenniallyReserved:

Get-VM -Location "DefinIT Lab Cluster" | Get-HardDisk -DiskType "RawPhysical","RawVirtual" | Select Parent,Name,DiskType,ScsiCanonicalName
List RDM Disks in PowerCLI

List RDM Disks in PowerCLI

Step 2 – Set the configuration for each RDM on each Host to “PerenniallyReserved”

Once we have the list of RDM disks, we can then set the reservation via Get-EsxCli

Connect-VIServer "host01.definit.local" -Credential (Get-Credential)
$esxcli = Get-EsxCli
$esxcli.storage.core.device.setconfig($false, "naa.6005076307ffc7930000000000000109", $true)

Building Steps 1 and 2 into a script

Now it’s just a simple case of constructing a script to loop through the hosts in a cluster and the RDMs on each host:

param(
	[string]	$TargetCluster,
	$RootCredentials = (Get-Credential)
)

# Create a connection object to all hosts in the Target Cluster
Get-Cluster $TargetCluster | Get-VMHost | % { Connect-VIServer $_ -Credential $RootCredentials | Out-Null }

# Find the ScsiCanonicalName for all RDM Disks attached to VMs in the Target Cluster
$RDMDisks = Get-VM -Location $TargetCluster | Get-HardDisk -DiskType "RawPhysical","RawVirtual" | Select ScsiCanonicalName

# Retrieve and EsxCli instance for each connection
foreach($esxcli in Get-EsxCli) {
	# And for each RDM Disk
	foreach($RDMDisk in $RDMDisks) {
		# Set the configuration to "PereniallyReserved".
		# setconfig method: void setconfig(boolean detached, string device, boolean perenniallyreserved)
		$esxcli.storage.core.device.setconfig($false, ($RDMDisk.ScsiCanonicalName), $true)
	}
}

# Disconnect the connection objects created for the Target Cluster
Disconnect-VIServer * -Confirm:$false | Out-Null

The script connects to each ESXi instance in the target cluster, queries all VMs on the target cluster and returns the RDM disks and then sets the PerenniallyReserved flag for each of the cluster hosts and RDM disks.

As usual, post any comments or improvements!

Edit: After my conversations with Mike, I’ve modified the script a little and attached as a zip file to download here: Set-RDMReservations

 

 

CC BY-SA 4.0
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Comments

  1. Mike says

    Hey Sam!

    This is exactly what I was looking for. However if I use the script you created I get a error:

    = : Missing expression after ‘=’.
    + CategoryInfo : ParserError: (=:String) [], ParseException
    + FullyQualifiedErrorId : MissingExpressionAfterToken

    • says

      Hi – is that the full error message? Does it prompt you for credentials or not get that far? Can you try putting brackets around Get-Credential at the beginning? Line 3

      Sam

      • Mike says

        Here is the full error – no prompt for creds.

        At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:3 char:35
        + [PSCredential] $RootCredentials = Get-Credential)
        + ~
        Missing expression after ‘=’.
        At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:3 char:33
        + [PSCredential] $RootCredentials = Get-Credential)
        + ~
        Missing ‘)’ in function parameter list.
        At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:3 char:50
        + [PSCredential] $RootCredentials = Get-Credential)
        + ~
        Unexpected token ‘)’ in expression or statement.
        + CategoryInfo : ParserError: (:) [], ParseException
        + FullyQualifiedErrorId : MissingExpressionAfterToken

        • says

          Ok, that’s definitely the brackets around the get-credential, I’ll update the script! Must have lost them in the paste as they’re in my original file! Let me know if that solves it for you?

          • Mike says

            Yeah – when I originally copied the code the formatting didn’t come over clean and was pasted onto one line. I’ve tested the copy on Powershell ISE and PowerGUI with the same results. After updating the brackets I do get further however it gets stuck on the $TargetCluster in line 7. See error below:

            Get-Cluster : Cannot validate argument on parameter ‘Name’. The argument is null or empty. Supply an argument that is not null or empty and then try the command again.
            At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:7 char:13
            + Get-Cluster $TargetCluster | Get-VMHost | % { Connect-VIServer $_ -Credential $R …
            + ~~~~~~~~~~~
            + CategoryInfo : InvalidData: (:) [Get-Cluster], ParameterBindingValidationException
            + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetCluster

            Get-VM : 5/8/2013 2:02:13 PM Get-VM VIContainer parameter: Could not find any of the objects specified by name.
            At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:10 char:13
            + $RDMDisks = Get-VM -Location $TargetCluster | Get-HardDisk -DiskType “RawPhysica …
            + ~~~~~~~~~~~~~~~~~~~~~~~~~~
            + CategoryInfo : ObjectNotFound: (VMware.VimAutom…iner[] Location:RuntimePropertyInfo) [Get-VM], ObnRecordProcessingFailedException
            + FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ObjectNotFoundCritical,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetVM

            Get-EsxCli : 5/8/2013 2:02:13 PM Get-EsxCli Value cannot be found for the mandatory parameter VMHost
            At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:13 char:20
            + foreach($esxcli in Get-EsxCli) {
            + ~~~~~~~~
            + CategoryInfo : NotSpecified: (:) [Get-EsxCli], VimException
            + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxCli

            Disconnect-VIServer : 5/8/2013 2:02:13 PM Disconnect-VIServer Could not find any of the servers specified by name.
            At H:\VMware_Scripts\RDM_Perennially\RDM.ps1:23 char:2
            + Disconnect-VIServer * -Confirm:$false | Out-Null
            + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            + CategoryInfo : ObjectNotFound: (VMware.VimAutom…Server[] Server:RuntimePropertyInfo) [Disconnect-VIServer], ServerObnFailureException
            + FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ServerSpecifiedButNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.DisconnectVIServer

          • says

            That seems to indicate that you’ve not put the $TargetCluster parameter into the script when you called it? I’ve not made it a mandatory parameter so perhaps it’s not too clear, call it using the name of the script, then the target cluster, e.g:

            .\Set-RDMReservation.ps1 "Production Cluster 1"

          • Mike says

            Hey Sam – thanks for the help. I’ve tried that too however when I call out the cluster when running the script it errors out on Get-Cluster with “you are not currently connected to any servers” even though I’ve when opening PowerCLI and getting prompted from the execution of the script.

          • says

            Hmmm, not sure what to suggest here – I have just run the script without any issue:

            Connect-VIserver "virtualcenter"
            .\Set-RDMReservations.ps1 "Production Cluster 1"

            It prompts me for credentials and then processes the RDMs:

            Set-RDMReservations.ps1

          • Mike says

            I noticed you were running Release 2 so I just upgraded my version. Got a slightly different error however its still credential related. Any way you can send me the original code so I can compare it to what im using?

            Connect-VIServer : 5/9/2013 10:57:34 AM Connect-VIServer Cannot complete login due to an incorrect user name or
            password.
            At H:\vmware_scripts\RDM_Perennially\RDM.ps1:7 char:47
            + Get-Cluster $TargetCluster | Get-VMHost | % { Connect-VIServer $_ -Credential $R …
            + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            + CategoryInfo : NotSpecified: (:) [Connect-VIServer], InvalidLogin
            + FullyQualifiedErrorId : Client20_ConnectivityServiceImpl_Reconnect_Exception,VMware.VimAutomation.ViCore.Cmdlets.Comm
            ands.ConnectVIServer

          • says

            The code is now linked above, that’s exactly the code I am using. The new error suggests that you aren’t supplying the root credential to log into the ESXi host? The credentials for vCenter won’t do here as it needs to get an ESXCLI instance.

          • Mike says

            Seems to work now! The code from the link above worked and it appears to bit slightly different from what I originally copied. Thanks again for all the help! This is gonna save me tons of time on my 5.1 upgrades. My hosts are take 3+ hrs. to boot. I’ll report back on timings after I put these changes in.

          • Mike says

            Seems like you have to reconnect to the VC before running this script regardless if you have done so when first launching PowerCLI. Ran this on my Cluster and my reboots are running about 7mins, down from 2.5-3hrs! Thanks again Sam!!

          • says

            That’s great news, glad to be of help! The script disconnects all the sessions at the end because it creates one for each host, so that probably explains the need to connect. I can probably sort that fairly easily, but it’s not the sort of script I envisage needing to run too often!

  2. John Wiersma says

    Any suggestions on how to write a script that would display the “Perennially reserved =” status for each RDM visible to each host?

    By the time you get an ESXi cluster of 9 hosts, with a dozen or so MSCS clusters, each with 3 or 4 RDMs each, it becomes cubersome to make sure you have created a reservation for each one. And if they don’t have reservations, then the maitre d’ can make you wait an hour for a table, and you end up waiting in the lounge drinking Manhattans on an empty stomach and getting into a debate about how long the Euro will last with a hapless bystander.

    So you see why I need to have a script like that.

    Thanks for any help you might be able to provide.

  3. Stu says

    love the script, thank you!! However how could I make the script run on just one host vs. the entire cluster?

    • says

      Yep – should be able to change the lines:

      Get-Cluster $TargetCluster | Get-VMHost | % { Connect-VIServer $_ -Credential $RootCredentials | Out-Null }
      $RDMDisks = Get-VM -Location $TargetCluster | Get-HardDisk -DiskType “RawPhysical”,”RawVirtual” | Select ScsiCanonicalName

      to

      Get-VMHost $TargetCluster | % { Connect-VIServer $_ -Credential $RootCredentials | Out-Null }
      $RDMDisks = Get-VMHost $TargetCluster | Get-VM | Get-HardDisk -DiskType “RawPhysical”,”RawVirtual” | Select ScsiCanonicalName

      and then supply the VMHost name as the $TargetCluster parameter when you run the script.

  4. nnddb says

    Are you supposed to get some sort of output in the powercli window when you run the script? It prompts for credentials, gives me the certificate errors, then it just bumps me back to the command prompt. But if I use esxcli I can see the luns reserved.

    • says

      There’s no output by default, if you set $DebugPreference = “Continue” before running the script you will get a more verbose output.

  5. nnddb says

    Forgot to mention, was getting this too: Unable to find type [PSCredential}: make sure that the assembly containing this type is loaded. I got past it by removing PSCredentials from the script and just making it $RootCredentials = (get-credential).

    • says

      Hi – thanks for this, I’ve seen this when I am running the script, I think it might be the difference between Powershell 2 and 3, it used to work fine for me. I’ll remove it from the script :)

  6. Gab says

    Hi Sam, thanks for this script.. immensely usefull! I’m stuck with an error :Could not find VIContainer with name ‘Clusters VFarm’. I tried to pass the cluster name right after the script name in the powercli console. I also tried to pass the argument in the script $TargetCluster = “Clusters VFarm” and then leave the rest of script with $TargetCluster. I also tried this:

    param(
    #[string] $TargetCluster = “Clusters VFarm”,
    $RootCredentials = (Get-Credential)
    )

    # Create a connection object to all hosts in the Target Cluster
    Get-Cluster “Clusters VFarm” | Get-VMHost | % { Connect-VIServer $_ -Credential $RootCredentials | Out-Null }

    # Find the ScsiCanonicalName for all RDM Disks attached to VMs in the Target Cluster
    $RDMDisks = Get-VM -Location “Clusters VFarm” | Get-HardDisk -DiskType “RawPhysical”,”RawVirtual” | Select ScsiCanonicalName

    # Retrieve and EsxCli instance for each connection
    foreach($esxcli in Get-EsxCli) {
    # And for each RDM Disk
    foreach($RDMDisk in $RDMDisks) {
    # Set the configuration to “PereniallyReserved”.
    # setconfig method: void setconfig(boolean detached, string device, boolean perenniallyreserved)
    $esxcli.storage.core.device.setconfig($false, ($RDMDisk.ScsiCanonicalName), $true)
    }
    }

    but nothing. We am I wrong?

      • Gab says

        thanks for replying!

        it’s correct.. it says:

        PowerCLI C:\Scripts> Get-Cluster “Clusters VFarm”

        Name HAEnabled HAFailover DrsEnabled DrsAutomationLevel
        Level
        —- ——— ———- ———- ——————
        Clusters VFarm False 1 True FullyAutomated

        I connect to the vcenter before executing the script.. I’m prompted for pw and then I launch the script..

        • Gab says

          we have several datacenters in our vsphere client.. but I don’t think it should make a difference.. else the Get-Cluster command on its own should fail..

          • Gab says

            Hi Sam
            I’ve changed the script to do a single ESXi host of that vCluster and it works. It failed with the same error at first, then instead of just the host name I entered the fqdn and it worked.. I’ll run the script four times.. not too bad anyway!! Still better than entering 600 naa identifiers in the Host profile!!

          • says

            OK – glad it can save you some time – not sure why it’s throwing that error though! I have multiple datacenters and it seems to work fine? Does it make any difference if you use the -Name parameter in the script when doing a Get-Cluster?

  7. Gab says

    well.. actually thinking about it twice.. I’ve just reserved those naa that belongs to those VMs on that particular ESXi host. Unfortunately each ESXi host when booted see all the other datastores and RDMs inside that belong to the other VMs on the other ESXi hosts. Therefore…if I’m not mistaking I need to get the original script working. I’ll use the name paramater and let you know.. thanks Sam!

  8. Gab says

    well.. the Get-Cluster -Name $TargetCluster didn’t work and it disconnected the ESXi nodes from vcenter. It stayed hours without succeeding or throwing errors.. I had to stop it.. I’ll see to find a script that passes all those naa.identifiers to each host.. thanks anyway!

  9. Mike says

    Here is a simple effective way to get it done also. Simply create a text file with the naa.xxx numbers. Point this script to it run and your done. The output file is great as it confirms your change worked. You MUST run it on all hosts in the cluster!

    $HostName = “YourHostName”
    Connect-VIServer -Server “$HostName.YourDomain.com” -User YourUserHere -Pass YourPWHere
    $myesxcli= get-esxcli -VMHost “$HostName.YourDomain.com”
    Get-Content c:\PowerCLI\naaList.txt | Foreach-Object {
    $myesxcli.storage.core.device.setconfig($false,$_, $true)
    $myesxcli.storage.core.device.list($_) > C:\PowerCLI\$HostName\$_.txt
    }

  10. Tom says

    The VMware guide mentions to set this on the esxi hosts that are serving the passive MSCS nodes. Did you run this on all hosts in your cluster including the hosts with active MSCS nodes? If yes can you safely run the command on the host serving the active node? I’ve got 4 of 5 hosts upgraded from 5.0 to 5.5 and I’m trying to resolve this prior to upgrading the last host with the active node.

    http://pubs.vmware.com/vsphere-55/index.jsp?topic=%2Fcom.vmware.vsphere.mscs.doc%2FGUID-E794B860-E9AB-43B9-A6D0-F7DE222695A1.html

    • says

      Hi Tom,

      I’ve had no problems running this on the host with the active node, I’ve done it several times. I set the reservation on any host that the LUN is presented to regardless of active/passive node – if you fail over you don’t want to have to then set the reservation on the new passive node.

      Hope that helps,

      Sam

  11. Gordon says

    Hi SAM
    Thanks for the great article.
    Does this apply to MSCS RDMs only, I will shortly have a non clustered VM with RDMs.
    Should I use the script in this case ?

  12. Jim Owens says

    Great script and big time saver. I am having the same problem on esxi 5.5. HP blades taking a long time to boot. I am not very handy with powershell – getting an error – maybe you can help. I am starting script using .\Set-RDMreservations.ps1 “my cluster” and getting an error like another comment: PSCredential: make sure that the assembly contianing this type is loaded…. Thanks in advance

Leave a Reply