Install VMware vCloud Director 8.10 for Service Providers
Updated: 28/02/2017 – Added Code Snippets.
Updated: 02/03/2017 – Added CentOS Routing For Two NICS When Using Same Subnet. (Thanks to Matt F)
Before proceeding with the VMware vCloud Director installation I would advise you prepare your environment by installing VMware NSX If this is not already installed. This will be required for this version of vCloud Director as it no longer supports the vShield Manager that was used in previous versions. You can find my installation guide for VMware NSX 6.2.2 for vCloud Director here
vCloud Director 8.10 for Service Providers Release Notes
VMware vCloud Director 8.10 for Service Providers Documentation
vCloud Director 8.10 Installation and Upgrade Guide
vCloud Director 8.10 Administrator’s Guide
Initial requirements –
I will be assuming you already have a supported vCloud Director machine configured before proceeding with this installation, I may create a guide for how to configure a CentOS 7 machine in the future so stay tuned! The supported operating systems for a vCloud Director Cell are as below –
CentOS 6
CentOS 7
Red Hat Enterprise Linux 5, update 4-10
Red Hat Enterprise Linux 6, updates 1-7
Red Hat Enterprise Linux 7
I will be using RHEL 6.6 as I have an existing template for this version of Linux.
Required packages as per the VMware Documentation –
alsa-lib bash chkconfig coreutils indutils glibc grep initscripts krb5-libs libgcc libICE libSM libstdc libX11 libXau libXdmcp libXext libXi libXt libXtst module-init-tools net-tools pciutils procps redhat-lsb sed tar which
I also needed the following packages as they were not installed on my RHEL 6.6 template, they were required for the setting up the NFS configuration –
nfs-utils nfs-utils-lib cifs-utils
You should be able to use the below command to install these packages depending on the version of Linux you are running & or If you have a RHEL subscription.
yum install alsa-lib bash chkconfig coreutils indutils glibc grep initscripts krb5-libs libgcc libICE libSM libstdc libX11 libXau libXdmcp libXext libXi libXt libXtst module-init-tools net-tools pciutils procps redhat-lsb sed tar which nfs-utils nfs-utils-lib cifs-utils
You will be requested to download the packages, select Y for the installation to continue.
We will also need a NFS Share configured that will be used as the transfer storage between the vCloud Director cells, I will cover this later on in the article, If you plan to only install one cell in a lab environment you can ignore this part of the installation.
You will need to configure the following resources on your vCloud Director Cell VM, You can adjust these to your environment –
vCloud Director Cell 1
2x VMXNET3 Network Adapters
2vCPU
4GB RAM
You will need to configure two network adapters for vCloud Director, one for the http service and one for the console proxy service. In vCloud Director 8.10 you can now use only one IP If required but you will need to change the default ports. You will also need to make sure the DNS Is pre-configured before starting the vCloud Director installation. For example my configuration below for my cell.
JACOB-vCD01
JACOB-vCD01.JACOB.local – 172.16.0.231
JACOB-vCD01-RC.JACOB.local – 172.16.0.232 (RC = Remote Console)
Configuring Firewall Requirements on vCloud Director Cell
Port requirements can be found in the vCloud Director Installation Guide Here – http://pubs.vmware.com/vcd-810/topic/com.vmware.ICbase/PDF/vcd_810_install.pdf – As this is being installed in a Lab Environment I will be turning off the firewall completely on the machine to avoid any issues when installing the software. On RHEL/CentOS this can be completed using the below commands –
Note – I do not advise you use these commands in a production environment.
service iptables stop
chkconfig iptables off
Usually port 443 and port 80 will be sufficient for the incoming firewall on the vCloud Director Cells, this can be changed by using the below commands –
Save these rules using the below command –
service iptables save
Then restart the service using the below command –
service iptables restart
Configuring vCloud Director SQL Database on SQL 2014 as per the vCloud Director Installation Guide.
You will need a configured SQL Server as per the vCloud SQL Server requirements, they can be found here on the VMware Product Interoperability Matrixes http://partnerweb.vmware.com/comp_guide2/sim/interop_matrix.php
You can change the file names as required.
USE [master]
GO
CREATE DATABASE [vcloud] ON PRIMARY
(NAME = N’vcloud’, FILENAME = N’C:\vcloud.mdf’, SIZE = 100MB, FILEGROWTH = 10% )
LOG ON
(NAME = N’vcdb_log’, FILENAME = N’C:\vcloud.ldf’, SIZE = 1MB, FILEGROWTH = 10%)
COLLATE Latin1_General_CS_AS
GO
Set the transaction isolation level.
The following script sets the database isolation level to READ_COMMITTED_SNAPSHOT.
USE [vcloud]
GO
ALTER DATABASE [vcloud] SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
ALTER DATABASE [vcloud] SET ALLOW_SNAPSHOT_ISOLATION ON;
ALTER DATABASE [vcloud] SET READ_COMMITTED_SNAPSHOT ON WITH NO_WAIT;
ALTER DATABASE [vcloud] SET MULTI_USER;
GO
Create the vCloud Director database user account.
The following script creates database user name vcloud with password vcloudpass.
USE [vcloud]
GO
CREATE LOGIN [vcloud] WITH PASSWORD = ‘vcloudpass’, DEFAULT_DATABASE =[vcloud],
DEFAULT_LANGUAGE =[us_english], CHECK_POLICY=OFF
GO
CREATE USER [vcloud] for LOGIN [vcloud]
GO
Assign permissions to the vCloud Director database user account.
The following script assigns the db_owner role to the database user created in the previous step.
USE [vcloud]
GO
sp_addrolemember [db_owner], [vcloud]
GO
If you have an AlwaysOn Avalibility Group you will need to create a Full Backup and then add the database to the Availability Group on the primary replica. You also need to make sure that SQL Server and Windows Mode Authentication is turned on in the SQL Server Settings as vCloud Director uses SQL (Local) Accounts.
Configuring Certificates for vCloud Director Cell
We will need to create a directory for the VMware private keys to be stored in, I have created the following directory /opt/keystore to store these files. This can be created by typing the below commands –
cd /opt
mkdir keystore
Now you can cd to /opt/keystore before proceeding to the next steps.
Download VMware Public Keys
wget https://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-DSA-KEY.pub
wget https://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-RSA-KEY.pub
At this point we will need to upload the vCloud Director Installer as we need to run prerequisites checker as this also installs the relevant tools to create the certificates. You need to download the ISO via your VMware Partner Portal and upload this to your vCloud Director cell. The easiest way I find to do this is to use WinSCP to transfer the file. I have uploaded mine to the /media directory as you can see in the screenshot below –
You will now need to make sure that the file is executable by running the below command as per the installation guide –
chmod u+x installation file name – In my example it would be
chmod u+x vmware-vcloud-director-distribution-8.10.0-3880025.bin
Now run the installation by running the below command –
./vmware-vcloud-director-distribution-8.10.0-3880025.bin
You need to make sure you select No here (N) as we need to continue with setting up the vCloud Director cell before we run this script.
Untrusted Certificates –
I will not be covering this section as I will be using Trusted Certificates requested from my CA in my Lab Environment.
HTTP Service –
keytool -keystore certificates.ks -alias http -storepass passwd -keypass passwd -storetype JCEKS -genkeypair -keyalg RSA -keysize 2048 -validity 365 -dname “CN=vcd1.example.com, OU=Engineering, O=Example Corp, L=Palo Alto S=California C=US” -ext “san=dns:vcd1.example.com,dns:vcd1,ip:10.100.101.9”
Console Proxy Service –
keytool -keystore certificates.ks -alias consoleproxy -storepass passwd -keypass passwd -storetype JCEKS -genkeypair -keyalg RSA -keysize 2048 -validity 365 -dname “CN=vcd2.example.com, OU=Engineering, O=Example Corp, L=Palo Alto S=California C=US” -ext “san=dns:vcd2.example.com,dns:vcd2,ip:10.100.101.10”
Confirm the certificates are now in the certificates.ks keystore
keytool -storetype JCEKS -storepass passwd -keystore certificates.ks -list
Trusted Certificates –
You will need to change directory to the following directory as this is where the keytool that we will be using to create the certificates is installed.
cd /opt/vmware/vcloud-director/jre/bin
HTTP Service –
Configure these variables for your environment –
CN=FQDN of your vCD Cell
OU – Organisational Unit
O – Organization
L – Locality
S – State
C – Country
SAN=FQDN of your vCD Cell
DNS – DNS Shortname of your vCD Cell
IP = HTTP IP of your cell
./keytool -keystore certificates.ks -storetype JCEKS -storepass passwd -genkey -keyalg RSA -keysize 2048 -alias http -dname “CN=JACOB-vCD01.JACOB.local, OU=Information Technology, O=JACOB, L=Northampton S=Northamptonshire C=GB” -ext “san=dns:JACOB-vCD01.JACOB.local,dns:JACOB-vCD01,ip:172.16.0.231”
Fill in the relevant certificate information.
Console Proxy Service –
./keytool -keystore certificates.ks -storetype JCEKS -storepass passwd -genkey -keyalg RSA -keysize 2048 -alias consoleproxy -dname “CN=JACOB-vCD01-RC.JACOB.local, OU=Information Technology, O=JACOB, L=Northampton S=Northamptonshire C=GB” -ext “san=dns:JACOB-vCD01-RC.JACOB.local,dns:JACOB-vCD01-RC,ip:172.16.0.232”
HTTP Service Certificate Signing Request –
keytool -keystore certificates.ks -storetype JCEKS -storepass passwd -certreq -alias http -file http.csr
Console Proxy Service Certificate Signing Request
keytool -keystore certificates.ks -storetype JCEKS -storepass passwd -certreq -alias consoleproxy -file consoleproxy.csr
You will now need to submit these .CSR files to your certificate authority and save the responses as .CER files. These will be in the following directory /opt/vmware/vcloud-director/jre/bin I recommend you use WinSCP as this is an easy method.
Navigate to your certificate authority web browser, usually http://CAFQDN/certsrv or https://CAFQDN/certsrv – This is references a Windows CA.
You will initially need to download the CA Certificate, Click Download a CA certificate, certificate chain or CRL.
You should name this certificate root.cer
Now you should navigate back to the previous page and Select Request a Certificate
Submit an Advanced Certificate Request
Open the http certificate file and copy the contents
Paste this in to the Advanced Certificate Request Box and select the Web Server template.
Select DER Encoded and Download the certificate. Rename this to http.cer as this makes it easier for us to determine the correct certificate for each service.
Repeat these steps for the Console Proxy certificate.
You should now see these two certificates and the root certificate in the directory.
You should now upload these .CER files to the vCloud Director cell. Again this can be completed easily using WinSCP.
I recommend uploading this to the same directory as this is the directory that keytool is installed.
We initially need to import the root certificate, this can be completed using the following command –
keytool -storetype JCEKS -storepass passwd -keystore certificates.ks -import -alias root -file root.cer
Import these to the certificates directory we created earlier then run the below commands –
Import HTTP Service –
keytool -storetype JCEKS -storepass passwd -keystore certificates.ks -import -alias http -file http.cer
Import Console Proxy Service –
keytool -storetype JCEKS -storepass passwd -keystore certificates.ks -import -alias consoleproxy -file consoleproxy.cer
Confirm the certificates are now in the certificates.ks keystore
keytool -storetype JCEKS -storepass passwd -keystore certificates.ks -list
Remember you will need to repeat these steps on any other vCloud Director cell you will be adding to vCloud Director.
Configuring NFS Transfer Storage for vCloud Director Cells
We now need to setup the transfer storage on the vCloud Director Cell. I have a NFS server that I will be using for this, but you can setup a NFS share on a Synology box If you have one or another NAS device such as FreeNAS. You can find my guide for setting up a NFS server on Ubuntu 14.03 here
We need to navigate to the following directory as this is where we will be mounting the NFS share.
cd /opt/vmware/vcloud-director/data/transfer
Mount the NFS share to the /opt/vmware/vcloud-director/data/transfer directory using the below command –
mount -t nfs IPofNFS:/Share /opt/vmware/vcloud-director/data/transfer
We now need to edit the fstab file so we can start this NFS mount when the vCloud Director Cell starts.
vi /etc/fstab
Press I to change to Insert mode inside the text editor.
Enter the following to the file so that this starts the NFS mount on boot. Change the relevant share to your NFS share.
172.16.0.13:/media/transfer /opt/vmware/vcloud-director/data/transfer nfs rw 0 0
Press Esc and then press :wq! To save the file.
We can confirm this is mounted by running the below command –
df -h
vCloud Director Installer
We can finally start the vCloud Director Installation! We will need to navigate to the correct directory using the below command –
cd /opt/vmware/vcloud-director/bin
./configure
Select the IP for the HTTP Service and select the IP for the Remote Console Proxy.
Enter the path to the keystore we created earlier called certificates.ks – Enter the directory below and type your keystore password (Default was passwd).
/opt/vmware/vcloud-director/jre/bin/certificates.ks
At this point you can configure a syslog host name or IP address If you have one on your network.
We now need to configure the SQL Database. Specify the following information –
Hostname of SQL Database/SQL Cluster Listener DNS Name/IP Address.
Default Port – 1433
vCloud Database Name, if you used the scripts above it will be called – vcloud
Database Instance – This will be default unless you have multiple instances on your SQL Server.
Enter the Database Username – vcloud
Enter the Database Password – Default as per the scripts above was – vcloudpass
The vCloud Director installer will configure the database as you can see below and then ask you if you want to start the vmware-vcd service.
When the service is started you can navigate to the web site as specified below –
You will now be able to navigate to the vCloud Director website as specified above.
Enter your vCloud Director License Key after agreeing to the License Agreement.
Specify an account name and password and also specify the contact Info.
Select Next and Finish
You will now be presented with the vCloud Director Login page!
Adding Additional vCloud Director Cells
To add additional cells to the deployment we will need to copy the responses.properties file, this is stored in the following location –
cd /opt/vmware/vcloud-director/etc
When you are in this location you can run the below command to copy this file to the vCloud Director Transfer storage.
cp responses.properties /opt/vmware/vcloud-director/data/transfer
You will now need to repeat the steps for the vCloud Director cell and when you reach the vCloud Director installation stage you need to run this command instead of the usual ./configure command that we used for this cell.
./configure -r /opt/vmware/vcloud-director/transfer/responses.properties
This will add an additional cell to vCloud Director!
Configuring CentOS Routing For Two NICS When Using Same Subnet (Thanks to Matt F)
Shutting Down vCloud Director Service
To start and stop the vmware-vcd service you need to use the below methods, If you shut down the cell without using these commands you may experience issues with vCloud Director starting correctly.
To check the status of the service –
service vmware-vcd status
You should use the cell-management-tool to shut down the cell. You will need to navigate to the below directory –
/opt/vmware/vcloud-director/bin/
Check the status of the vCloud Director cell by using this command –
./cell-management-tool -u administrator cell –status
Suspend the scheduler by using this command –
./cell-management-tool -u administrator cell –quiesce true
Check the existing tasks that are running on the vCloud Director cell –
./cell-management-tool -u username -p password cell -t
Wait until the Job count shows 0 then you can run the below command to shut down the cell –
./cell-management-tool -u administrator cell –shutdown
If you re-run the service vmware-vcd status command you should see the services have now stopped.
Thanks, this was really helpful!
Only thing to keep in mind is that some formatting changed double dashes (–) and quotes to a different one that doesn’t work when pasted in to CLIs 🙂
No problems! Thanks for pointing that out, I’ll edit this now to make sure they are fixed!
Nice write up! One question, when using multiple nics how do you configure the linux routing to use both nics when they are in the same subnet?
Hello Matt,
Not sure exactly what version of Linux you are using, but I know when using RHEL6 It uses ‘Reverse Path Filtering’ that effectively points the traffic to the NIC it believes should be receiving the packet. The following command should resolve this issue on RHEL6 –
echo ‘net.ipv4.conf.eth0.rp_filter = 0’ >> /etc/sysctl.conf
echo ‘net.ipv4.conf.eth1.rp_filter = 0’ >> /etc/sysctl.conf
echo ‘net.ipv4.conf.lo.rp_filter = 0’ >> /etc/sysctl.conf
sysctl -p
Disclaimer: I Do not claim to be a Linux expert!
Thanks for commenting and I hope this helps!
I’m using centos 7, I’ll give that a try and see what happens! Thanks!
You pointed me in the right direction! This worked for CentOS7
Tested with
sysctl -a | grep rp_filter | grep -v arp_filter
echo “1” > /proc/sys/net/ipv4/conf/all/rp_filter
made permanent with
vi /etc/sysctl.conf
net.ipv4.conf.all.rp_filter = 1
Hello Matt,
Thanks for the feedback! Glad it’s now working as expected. I’ll update the article with this information!