Building Cloud Foundry on vSphere using BOSH Part 2

Install BOSH CLI

Important Prerequisites:

Throughout the installation process of BOSH and Cloud Foundry, direct internet connection is required. This is very important because some of the software bits are downloaded directly from internet sources, such as Ruby gems and some open source software. Setting up a web proxy server between your VMs and the internet won’t work.

The other prerequisite is a stable internet connection. If your network is slow or unreliable when downloading files from internet, your installation may fail with timeout or connection error.

We have seen many unsuccessful setups due to internet connection issues. It is highly recommended you check with your network administrator before starting the BOSH installation.

Create a Cluster in vCenter

Assume all nodes are virtual machines, we first install vSphere (we use V5.x in this article) on all bare metal servers. The vSphere servers are connected by the Management Network VLAN. Once it is done, we create a VM on one of the hypervisors to install Windows 2008 R2 64 bit. We then install vCenter on this Windows 2008 VM. The next step is to connect to vCenter using vSphere client so that we can manage the servers.

We can install a vSphere client on any Windows machine (even a virtual machine). After that, we can connect to vCenter by the vSphere client remotely. Firstly, let’s create a datacenter in vCenter. Right click on the vCenter node in the left pane and choose “New Datacenter” to add a new data center.Create a Data Center in vCenter

Next is to right click the newly created data center node and select “New Cluster…”.

During the “New Cluster Wizard”,  if you turned on vSphere DRS feature, you will be asked to configure VMware DRS. Make sure the “automation level” is set to “partially automated” or “Fully automated” as shown below. If you choose “Manual”, you will be as prompted by a popup to enter your choice. This behavior may block your BOSH automated installation.

Then, select the “Enable Host Monitoring” checkbox and select “Disable: Power on VMs that violate availability constraints”:

Then, go to the sub-part of “VM Monitoring”, choose “Disabled”:

Click Next, choose as follows:

Adding vSphere Hosts

The next step is to place hypervisors into the cluster we just created. Right click on the cluster node and choose “Add Host…”. For each vSphere server, enter its IP address, administrator’s username and password and confirm your configuration:

After adding all hosts, they will be listed in the Datacenter->Hosts tab:

Attach Datastore to Hosts

All hosts in the cluster should share the same NFS storage. For each host, we add in the storage as datastore. In vCenter, click on the vSphere host and choose the “Configuration” tab. Select the “Hardware”->”Storage”. Click the upper right “Add Storage…Create a Data Center in vCenter

In the “Select Storage Type” dialogue, choose “Network File System” .

Enter the ip address of your NFS storage, the folder name, and the datastore name. It is very important to use exactly the same datastore name for all hosts in the cluster. In this example, we use the name “NFSdatastore”.

Create Folders for VMs and Templates

From vCenter’s navigation bar, choose the view Home->Inventory->VMs and Templates, create folders as below:

These folders are used later for grouping BOSH and Cloud Foundry VMs.  In the above example, “template_folder_bosh” is used to place BOSH stemcells. “vm_folder_bosh” is used for BOSH nodes. “template_folder” keeps Cloud Foundry stemcells. “vm_folder” stores Cloud Foundry nodes. These names will be used in the deployment manifest file later.

Network Configuration

The VMs of Cloud Foundry will be deployed onto one or multiple networks. Before a deployment, we need to create networks in vSphere. The below diagram illustrates the network connections required by Cloud Foundry.

On each vSphere host, we create two networks:

1)      CF Network: mapped to CF VLAN

2)      Service Network: mapped to Service VLAN.

Most of the VMs reside on the CF Network. Only the Router VMs are dual-homed with the Service Network and CF network.

NOTE: In an experimental environment, you can put all VMs on the same network to simplify the installation process. Hence a single network on a hypervisor may suffice.

To create a network, choose the “Hosts and Clusters” view. Select a host and switch to the “configuration” tab. Then select “Networking” and click “Add Networking”:

The connection type should be “Virtual Machine”:

Use an existing vSwitch:

In next step, rename the network label to “CF Network”. If your network administrator has assigned a VLAN ID, enter the CF VLAN ID accordingly.

Then Click “finish” and complete the network creation. Repeat the steps to create “Service Network”, just name the network label as “Service Network”.

It is important to keep the network name exactly the same on all hosts in the same cluster. The below figure shows two networks had been created for a host. We name them as “CF Network” and “Service Network”. These names will be used in the bosh and cloud foundry yml files later.

Also, if you check from the Datacenter->Inventory->Networking view, it shows the following:

Create a VM for BOSH CLI

In vCenter, we select one of the hosts in the cluster to create a VM. Click “Create a new virtual machine”. We will install on this VM a 64bit Ubuntu 10.04 OS. Assign the VM with 2 vCPUs, 2GB memory, 20GB disk space (or more). During the installing, make sure to set network manually.

We now start to install BOSH CLI on the VM. Log on to Ubuntu and follow the below steps. (Note, these steps are mostly from the official BOSH document. They are listed here for your convenience.)

1)      Install ruby via rbenv:
1. Bosh is written in Ruby. Let’s install Ruby’s dependencies:

$ sudo apt-get install git-core build-essential libsqlite3-dev curl \
libmysqlclient-dev libxml2-dev libxslt-dev libpq-dev genisoimage

2. Get the latest version of rbenv

$ git clone git://github.com/sstephenson/rbenv.git .rbenv

3. Add ~/.rbenv/bin to your $PATH for access to the rbenv command-line utility

$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile

4. Add rbenv init to your shell to enable shims and autocompletion

$ echo 'eval "$(rbenv init -)"' >> ~/.bash_profile

5. Download Ruby 1.9.2
Note: You can also build ruby using ruby-build plugin for rbenv. See https://github.com/sstephenson/ruby-build

$ wget http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.2-p290.tar.gz

6. Unpack and install Ruby

$ tar xvfz ruby-1.9.2-p290.tar.gz
$ cd ruby-1.9.2-p290
$ ./configure --prefix=$HOME/.rbenv/versions/1.9.2-p290
$ make
$ make install

7. Restart your shell so the path changes take effect

$ source ~/.bash_profile

8. Set your default Ruby to be version 1.9.2

$ rbenv global 1.9.2-p290

Note: The rake 0.8.7 gem may need to be reinstalled when using this method

$ gem pristine rake

9. Update rubygems and install bundler.
Note: After installing gems (gem install or bundle install) run rbenv rehash to add new shims

$ rbenv rehash
$ gem update –system
$ gem install bundler
$ rbenv rehash

2) Install BOSH CLI:
1. Sign up for the cloud foundry gerrit server at http://reviews.cloudfoundry.org
2. Set up your ssh public key (accept all default):

$ ssh-keygen –t rsa

3. Copy your key from ~/.ssh/id_rsa.pub into your Gerrit account.
4. Create and upload your public SSH key in your Gerrit account profile.
5. Set your name and email:

$ git config --global user.name "Firstname Lastname"
$ git config --global user.email your_email@youremail.com

6. Install out gerrit-cli gem:

$ gem install gerrit-cli

7. Run some rake tasks to install the BOSH CLI:

$ gem install bosh_cli
$ rbenv rehash
$ bosh –version

If everything works well, the last command will show the bosh version you just installed. This indicates that BOSH CLI has been successfully installed.

How to deploy Cloud Foundry using BOSH: