DefinIT

Using the VMware SSL Certificate Automation Tool with a Microsoft Certificate Authority

| 05/04/2013 | Tags: ,

vmware logoUpdating vCenter Server certificates has always been a pain – it has only got worse with the sheer number of services that are running under vSphere 5.1 – each service requiring a unique certificate and to be installed in many complex steps.

Fortunately , with the release of the SSL Certificate Automation Tool, VMware have gone some way to reducing the headache.

Gather all the components you need

OpenSSL installer: http://slproweb.com/products/Win32OpenSSL.html (I downloaded “Win32 OpenSSL v0.9.8y Light”, even though it’s a 64-bit server)

Microsoft Visual C++ 2008 Redistributable Package (x86): http://www.microsoft.com/en-us/download/details.aspx?id=29

VMware Certificate Automation Tool: https://my.vmware.com/group/vmware/get-download?downloadGroup=SSL-TOOL-10 (My VMware login required)

Install MS Visual C++ 2008 (pre-requisite for OpenSSL) Install OpenSSL – keep the defaults except for installing to c:\OpenSSL-Win32\, and install the binaries in the OpenSSL folder, not the System32.

Download the CA certificate chain from your CA

Navigate to the home page of the certificate server and click on Download a CA certificate, certificate chain or CRL.

Click the Base 64 option.

Click the Download CA Certificate chain link.

Save the certificate chain as cachain.p7b. in the c:\certs folder.

Double-click on the cachain.p7b file and navigate to C:\certs\cachain.p7b > Certificates.

Right-click on the each certificate listed and select All Actions > Export.

Click Next.

Select Base-64 encoded X.509 (.CER), and then click Next.

Save the export at C:\cert\root64.cer and click Next

For Subordinate Certificate Authorities, export as root64-1.cer, root64-2.cer etc. You need the whole chain.

My c:\cert folder now looks like this (with one subordinate CA):

05-04-2013 11-30-41

Generating the SSL Certificates

This is a slightly modified version of my original script to generate SSL certs, the main difference being that we don’t want to generate pfx files and we do want to create a chain.pem file. You need to create the certificate template just the same so I’ll assume you can do that and come back!

Update the variables at the head of this script to reflect your environment and save it as Generate-VMCerts.ps1 in the c:\certs folder

Download it here: Generate-VMCerts.ps1

# The "short" name of the Virtual Center Server
$vC_NETBIOS = "DefinIT-VC01"
# The DNS name of the Virtual Center Server
$vC_FQDN = "DefinIT-VC01.definit.local"
# The IP address of the Virtual Center Server
$vC_IP = "192.168.1.150"
# The Certificate Authority Name
$CA_Name = "DefinIT-CA\DefinIT"
# The SHORT name of the template you defined earlier
$CA_Template = "CertificateTemplate:VirtualCenterWebServer"
# The CA certificate(s) you exported earlier, start with the issuing CA back up the chain to the root CA.
$CA_Certificates = @("c:\certs\root64-1.cer","c:\certs\root64.cer")
# Administrative email to use in the certificate
$AdminEmail = <a href="mailto:admin@definit.co.uk">admin@definit.co.uk</a>
# A working directory under which the certificates and folders will be created
$WorkingDir = "c:\certs\"
# An array of the services we will generate the certificates for
$Services = @("vCenterServer","vCenterInventoryService","vCenterSSO","VMwareUpdateManager","vCenterWebClient","vCenterLogBrowser","VMwareOrchestrator")
# The path to the openssl executable
$OpenSSLExe =  "c:\OpenSSL-Win32\bin\openssl.exe"

if (!(Test-Path $OpenSSLExe)) {throw "$OpenSSLExe required"}
New-Alias -Name OpenSSL $OpenSSLExe

$RequestTemplate = "[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = DNS:ShortName,DNS:FQDN, DNS:IPADDRESS

[ req_distinguished_name ]
countryName = UK
stateOrProvinceName = West Sussex
localityName = Horsham
0.organizationName = DefinIT
organizationalUnitName = ORGREPLACE
commonName = FQDN
emailAddress = ADMINEMAIL

[ v3_ca ]
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer
basicConstraints = CA:true

[ req_attributes ]"

foreach ($CA_Certificate in $CA_Certificates) {
	$CertificateChain += Get-Content $CA_Certificate
}

if(!(Test-Path $WorkingDir)) {
	New-Item $WorkingDir -Type Directory
}
Set-Location $WorkingDir
ForEach ($Service in $Services) {
	if(!(Test-Path $Service)) {
		New-Item $Service -Type Directory
	}
	Set-Location $Service
	write-debug "$Service: Writing Template"
	$Out = ((((($RequestTemplate -replace "FQDN", $vC_FQDN) -replace "ShortName", $vC_NETBIOS) -replace "ORGREPLACE", $Service) -replace "ADMINEMAIL", $AdminEmail) -replace "IPADDRESS", $vC_IP) | Out-File "$WorkingDir\$Service\$Service.cfg" -Encoding Default -Force
	write-debug "$Service: Generating CSR"
	OpenSSL req -new -nodes -out "$WorkingDir\$Service\$Service.csr" -keyout "$WorkingDir\$Service\rui-orig.key" -config "$WorkingDir\$Service\$Service.cfg"
	write-debug "$Service: Converting Private Key"
	OpenSSL rsa -in "$WorkingDir\$Service\rui-orig.key" -out "$WorkingDir\$Service\rui.key"
	write-debug "$Service: Submitting to $CA_Name"
	certreq -submit -attrib $CA_Template -config "$CA_Name" "$Service.csr" "$WorkingDir\$Service\rui.crt"
	write-debug "$Service: Generating chain.pem"
	Get-Content rui.crt | Add-Content chain.pem
	Add-Content chain.pem $CertificateChain
	Set-Location $WorkingDir
}

Now we can run the script to generate the required certificates, open a PowerShell console, navigate to c:\certs and run Generate-VMCerts.ps1. This should run through without any errors – if you have errors make sure you fix them before moving on!

My folder now looks like this:

05-04-2013 11-54-45

And under each service folder, there are the following:

05-04-2013 12-07-22

Running the SSL Certificate Automation Tool

Now we’re ready to follow the steps on Deploying and using the SSL Certificate Automation Tool (2041600). At this point, I took a snapshot of my virtual vCenter server! I’ve messed this up before – trust me, you want a fall back point! Since this vCenter is a simple install – all services on one box – it should be fairly simple.

I unzipped the Certificate Automation Tool under the c:\certs folder.

The first step is to edit the ssl-environment.bat file and carefully add the paths to the generated keys (the chain.pem and rui.key files are the ones you need). There are also a few other options like what type of SSO you are using, and some advanced optional parameters for load balancers etc. It’s important to add the user ID for the SSO admin and the Virtual Center admin.

Run the ssl-environment.bat file from the command prompt to set up the variables the tool needs – there is no output from running this file.

I then ran ssl-updater.bat, selected option 1 to plan the steps. I am updating everything except update manager on this server, so I enter 1,2,3,4,5,6 and get a detailed plan of how to update the certificates. Copy and paste this into a new text file and follow it closely! Enter 9 to go back to the main menu.

05-04-2013 12-52-12

Now go through each step very carefully, there’s a lot of text with very similar names, check for the successful completion of each task before moving on to the next. Make sure you have handy credentials for a vCenter admin, the SSO admin and the Virtual Center database. There were 16 very similar steps on my plan, I won’t bore you with a detailed explanation of all of them! The basic principle is to take the each step and run the correct process – eg for:

1. Go to the machine with Single Sign-On installed and – Update the Single Sign-On SSL certificate.

I select “2. Update Single Sign-on” and then “1. Update the Single Sign-On SSL Certificate” – you then get prompted for certificate locations, user IDs and passwords needed to perform that task.

I received an error updating the vCenter Server SSL Certificate “Cannot continue with the operation due to errors”. I looked in the vc-update-ssl.log file and found the following:

Could not reload vCenter SSL Certificates [05/04/2013 – 14:00:02.31]: “”Cannot reload the vCenter Server SSL certificates. The certificate might not be unique.”” [05/04/2013 – 14:00:02.32]: Deleting the new certificates and keys… [05/04/2013 – 14:00:02.32]: Restoring the original certificates and keys…         1 file(s) copied.         1 file(s) copied.         1 file(s) copied. [05/04/2013 – 14:00:02.34]: Attempting rollback… Could not reload vCenter SSL Certificates [05/04/2013 – 14:00:03.13]: “”Cannot reload the vCenter Server SSL certificates. The certificate might not be unique.””

This is a known issue and is mentioned on the KB for using the tool – “This may be caused by vpxd having multiple service IDs for the Lookup service in the vpxd.cfg file“. I looked in the config file and had 3 service IDs sitting there. I found the correct service ID by using:

C:\Program Files\VMware\Infrastructure\SSOServer\ssolscli>ssolscli.cmd listServices https://:7444/lookupservice/sdk

The output contains the correct ID and after deleting the 2 spurious entries and restarting, the tool was able to install the vCenter Server SSL Certificate.

Final word…

This tool takes much of the complexity out of this job and hopefully goes a step towards helping people secure their vCenter Server installations. If there’s one piece of advice for anyone doing this, it’s take your time and follow every step precisely! Oh, and don’t forget to delete that snapshot once you’ve verified everything is working OK!

 Update 14/05/2013

A great procedure for updating vShield Manager Appliance SSL is here: http://longwhiteclouds.com/2012/03/31/updating-ssl-certificate-in-vshield-manager-made-easy/