VCAC 6.0 build-out to distributed model – Part 1: Certificates
This is the first article in a series about how to build-out a simple vCAC 6 installation to a distributed model.
In a simple installation you have the Identity Appliance, the vCAC appliance (which includes a vPostgres DB and vCenter Orchestrator instance) and an IaaS server. The distributed model still has a single Identity Appliance but clusters 2 or more vCAC appliances behind a load balancer, backed by a separate vPostgres database appliance. The IaaS components are installed on 2 or more IaaS Windows servers and are load balanced, backed by an external MSSQL database. Additionally, the vCenter Orchestrator appliance is used in a failover cluster, backed by the external vPostgres database appliance.
The distributed model can improve availability, redundancy, disaster recovery and performance, however it is more complex to install and manage, and there are still single points of failure – e.g. the vPostgres database is not highly available and although protected by vSphere HA could be the cause of an outage. Clustering the database would provide an improved level of availability but may not be supported by VMware. Similarly the Identity Appliance is currently a single point of failure, although there are also options for high availability there too.
An overview of the steps required is below:
- Issue and install certificates
- Deploy an external vPostgres appliance and migrate the vCAC database
- Configure load balancing
- Deploy a second vCAC appliance and configure clustering
- Install and configure additional IaaS server
- Deploy vCenter Orchestrator Appliance cluster
First of all, it’s important to understand that the certificates used for vCAC (whether distributed or not) should not be left as the self-signed certificates generated by default. These are OK for a proof of concept, or perhaps a lab installation, but really should be issued by a trusted CA – whether that’s internal or external.
When you move to a distributed model, it becomes more important that all the moving parts trust each other’s certificates so that communication is secured between the different components. With a load balancer in play, CA signed certificates become crucial.
You can use either a wildcard certificate, or a certificate with Subject Alternative Names, to replace all the certificates in the vCAC installation. To future proof the certificate, I opted for a wildcard certificate protecting *.definit.local, however if wildcards are not permitted in your organisation then generate a single certificate with all the required SAN. In reality, there aren’t many to protect (ID Appliance, vCAC Appliance load balanced address, IaaS web server load balanced address and IaaS manager service load balanced address – if different from the web server).
Requesting and issuing a wildcard certificate using a Microsoft Certificate Services CA
This process uses the certificate template described in VMware KB 2062108.
Create a new file “VCAC.cfg” and paste in the following, adjusting for your environment:
[ req ] default_md = sha512 default_bits = 2048 default_keyfile = vcac.key distinguished_name = req_distinguished_name encrypt_key = no prompt = no string_mask = nombstr req_extensions = v3_req [ v3_req ] basicConstraints = CA:FALSE keyUsage = digitalSignature, keyEncipherment, dataEncipherment extendedKeyUsage = serverAuth, clientAuth [ req_distinguished_name ] countryName = UK stateOrProvinceName = West Sussex localityName = Horsham 0.organizationName = DefinIT organizationalUnitName = VCAC commonName = *.definit.local
Create a new request and private key:
openssl req –new –nodes –out “c:\certs\vcac.csr” –keyout “c:\certs\vcac.key” –config “c:\certs\vcac.cfg”
Submit the request to your CA:
certreq –submit –attrib “CertificateTemplate:VMware-SSL” –config “DC-01.definit.local\DefinIT-Root” –f c:\certs\vcac.csr c:\certs\vcac.cer
Now log onto your Identity Appliance’s management website (mine is https://vcac-sso.definit.local:5480) and navigate to the certificates page (SSO > SSL).
Select “Import PEM encoded certificate” from the menu. Open the vcac.key file and copy and paste it into the RSA Private Key field.
The Certificate Chain key needs to be in the format of the image below: first paste in the contents of vcac.cer, then the contents of any Base64 encoded intermediate and finally the Root CA, again in Base64. Make sure there’s no whitespace in between. Since you’ll be using this later on, create a new file “vcac-chain.pem” and paste the certificates in the correct order:
Now copy and paste the vcac-chain.pem contents into the Certificate Chain field:
Hit the “Replace Certificate” button and hold your breath until you see:
Update the vCAC Appliance with the Identity Appliance Certificate
SSH to the VCAC Appliance and log on using “root” and the root password.
Update the connection to the SSO server using:
/usr/sbin/vcac-config import-certificate --alias websso --url https://vcac-sso.definit.local:7444
Reboot the appliance and wait until all the services are started – this can be up to 15m.
Update the IaaS components with the Identity Appliance Certificate
At this point you would typically update the IaaS component certificates – if you’re following this series and building out a simple install to a distributed install, you will need to re-install the IaaS components to re-point them at the load balanced URLs – this means that there’s no point in updating the certificates in the current installation! However, for completeness, here’s the process:
On each server running an IaaS component import the CA certificates into the Computer > Trusted Root Certification Authorities store. To do this, open mmc.exe, add the snap in for Certificates, select Computer Account and Local computer. Expand the “Trusted Root Certification Authorities\Certificates” folder and check if the CA is trusted. If not, right click and import the CA, and any intermediaries, into the store.
Now you can import the Identity Appliance certificate on the IaaS server running the Model Data Manager component using the following commands – open a command prompt and enter:
cd C:\Program Files (x86)\VMware\vCAC\Server\Model Manager Data\Cafe vcac-config.exe DownloadRootCertificates --RootCertPath "C:\Program Files (x86)\VMware\vCAC\Server\Website\SSO root.cer" --SignCertPath "C:\Program Files (x86)\VMware\vCAC\Server\Website\SSO signing.cer" –v vcac-config.exe DownloadRootCertificates --RootCertPath "C:\Program Files (x86)\VMware\vCAC\Web API\SSO root.cer" --SignCertPath "C:\Program Files (x86)\VMware\vCAC\Web API\SSO signing.cer" –v
This will update the database with the certificate from the Identity Appliance – all other components will import the certificate from the database.
Reboot the IaaS servers.
Install the certificate on the vCAC Appliance
Installing the certificate on the vCAC Appliance is identical to the Identity Appliance, so log into the management console on port 5480 (e.g. https://vcac-app.definit.local:5480) using the root user and password. Navigate to “vCAC Settings” and then select SSL.
Once again, the certificate format is important – paste the contents of the earlier generated “vcac.key” file into the “RSA Private Key” field. Paste the contents of the earlier generated “vcac-chain.pem” file into the “Certificate Chain” field. Click the “Replace Certificate” button – you should see “SSL Certificate is replaced successfully”:
Reboot the vCAC appliance.
Update the Identity Appliance with the vCAC Appliance’s certificate
Once the vCAC appliance is back up and running (remember, up to 15 minutes!) log back onto the management console and click on the SSO tab.
You can safely ignore the warning about the hostname and certificate common name not matching – this is expected with both a wildcard and SAN certificates.
Enter the credentials for the SSO admin user (hint, likely to be email@example.com) and click “Save settings” – the SSO can take quite a few minutes to update, just be patient and wait it out! You should see “SSO Configuration is updated successfully”.
Update IaaS components with the vCAC Appliance’s Certificate
As before, there’s no point doing this step if you’re planning to build out to a distributed model – you will need to re-install the IaaS components and as such will not need to update the certificate. However, if you’re just updating the vCAC certificates then log onto your IaaS Model Data Manager server and run the following:
vcac-Config.exe UpdateServerCertificates -d vcac_database -s sql_database_server –v
vcac-Config.exe UpdateServerCertificates -d vCACDB -s sql-01.definit.local -v
Reboot your IaaS server(s).
Update IaaS servers with the new certificate
This needs to be done whether you’re updating an existing IaaS install, or building out the distributed model.
On each of the IaaS server(s), install the wildcard or SAN certificate (including the Private Key) so that it’s available for IIS to use. To do this we need to create a PFX file from the component .cer and .key files we created earlier using OpenSSL:
openssl pkcs12 -export -out c:\certs\vcac.pfx -inkey c:\certs\vcac.key -in c:\certs\vcac.cer
To import the certificate, open mmc.exe, add the Certificates snap-in for the local machine and right click Personal > Certificates and import the PFX file. Double click the certificate once it’s installed and ensure the Private Key is installed, and that the certificate chain is trusted:
This next step only needs doing if you’re updating an existing IaaS install and will not be re-installing IaaS as part of a distributed setup.
Open IIS manager and update the bindings for HTTPS to use the newly installed SSL certificate. If you don’t see the SSL certificate in the drop down then verify that the private key is installed, and that the certificate chain is trusted.
Restart IIS using the command “iisreset”.
It’s important NOT to create the binding to port 443 before you do the IaaS components install – if you do the installer will fail because the port isn’t available.
Update vCAC appliance with the IaaS certificate
The final step, and once again, only necessary if you are not doing a distributed install which requires the re-install of IaaS components, is to update the vCAC appliance database with the IaaS certificate. This is done using the vcac-config.exe on each of your IaaS server(s). Open a command prompt and enter the following commands. If you are using a load balancer already, then make sure you use the load balanced URL for each component.
Ensure each command completes successfully before moving on.
cd C:\Program Files (x86)\VMware\vCAC\Server\Model Manager Data\Cafe Vcac-Config.exe RegisterEndpoint --EndpointAddress https://<IaaS Web Server>/vCAC/ --Endpoint ui -v Vcac-Config.exe RegisterEndpoint --EndpointAddress https://<IaaS Web Server>/vCAC/SslCallback.aspx --Endpoint ssl -v Vcac-Config.exe RegisterEndpoint --EndpointAddress https://<IaaS Model Manager Web Server>/Repository --Endpoint repo -v Vcac-Config.exe RegisterEndpoint --EndpointAddress https://<IaaS Web Server>/WAPI --Endpoint wapi -v Vcac-Config.exe RegisterEndpoint --EndpointAddress https://<IaaS Web Server>/WAPI/api/status --Endpoint status -v
That concludes this first part in the series – stay tuned for the rest coming up over the next few weeks!