Saturday, November 3, 2012

CloudStack 2.14 + KVM + Openwitch + openflow (Part 3)

Part 3: Install and configure CloudStack Agent with NFS storage and Openvswitch bridge.


In the part 1 and part 2, I have introduced the basic install of CentOS, Openvswitch, CloudStack manager. In this part, I will introduce the CloudStack Agent installation with OVS bridge and NFS storage

After the end of part 2, Host1 and Host2 configuration as following:

Host1

  • Manager interface: cloudbr0 (eth0) - IP: 10.0.0.2/24
  • Guest and Public interface: cloudbr1 (eth1)
  • Sorfware installed: CentOS 6.0, OVS, CloudStack manager and database
Host2

  • Manager interface: cloudbr0 (eth0) - IP: 10.0.0.3/24
  • Guest and Public interface: cloudbr1 (eth1)
  • Sorfware installed: CentOS 6.0, OVS
You have built a NFS server with IP: 10.0.0.4/24 with URL: 10.0.0.4/kvm-ovs. This link will use for Primary storage on KVM cluster.

Now, You must install cloudstack on both hosts.
  • Check for a fully qualified hostname.
    #hostname --fqdn
  • This should return a fully qualified hostname such as "hostX.cloudkvm.com". If it does not edit /etc/hosts so that it does.
  • Remove qemu-kvm. The CloudStack provides a patched version.
    #yum erase qemu-kvm
  •     Disable SELinux
    Set SELinux up to be permissive by default. First, set the SELINUX variable in /etc/selinux/config to “permissive”. This ensures that the CloudStack Agent can run properly on system reboot. Then set SELinux to permissive until the system is rebooted:

    # setenforce permissive

    Install the CloudStack packages. You should have a file in the form of CloudStack-NNNN.tar.gz. Untar the file and and then run the install.sh script inside it.

    # tar xzf CloudStack-2.2.0-1-centos.tar.gz
    # cd CloudStack-2.2.0-1-centos
    # ./install.sh
    Setting up the temporary repository...
    Cleaning Yum cache...
    Loaded plugins: fastestmirror
    11 metadata files removed
    Welcome to the Cloud.com CloudStack Installer.  What would you like to do?

        M) Install the Management Server
        A) Install the Agent
        S) Install the Usage Monitor
        D) Install the database server
        Q) Quit

        > A

    Choose A to install the Agent software 

The CloudStack Agent is now installed. Later in the installation you will add this host to the CloudStack via the Management Server. This step will configure the Agent on the Host. 

The next, you must edit configuration file of CloudStack Agent as following:

#Fri Nov 02 16:38:04 ICT 2012
guest.network.device=cloudbr1
workers=5
private.network.device=cloudbr0
port=8250
resource=com.cloud.agent.resource.computing.LibvirtComputingResource
pod=1
zone=1
guid=c3434f39-0252-3c45-9a42-a713d5bcf456
public.network.device=cloudbr1
cluster=1
local.storage.uuid=80c6bd25-449c-492b-b2c1-43b6abd5ca81
LibvirtComputingResource.id=1
host=10.0.0.X

Note:
To CloudStack Agent can run with OVS bridge you must check brctl command:

#brctl addbr testbridge  => successful with no error
#brctl delbr testbridge   => delete testbridge

Now, you must download CloudStack-NNNN from source code and change some files and rebuild it from source.
/CloudStack-NNN/scripts/vm/network/vnet/modifyvlan.sh, 
/CloudStack-NNN/python/lib/cloud_utils.py
/CloudStack-NNN/target/scripts/util/qemu-ifup
/CloudStack-NNN/core/src/com/cloud/agent/resource/virtualnetwork/VirtualRoutingResource.java
=> You find the command brctl and replace by the ovs-vsclt command with the command structure of the ovs-vsclt
Example: "brctl show |grep bridge_name"  to "ovs-vsctl show |grep  bridge_name"

Now, you can add Host1 and Host2 and create VMs.

Part 4: Create VM with OVS bridge and configure QoS and network throttling on VMs


CloudStack 2.14 + KVM + Openwitch + openflow (Part 2)


Part 2: Create bridge interface (use openvswitch bridge) for Cloud agent and configure agent.


Step 1: Create bridge interface for CloudStack.

Reference links: http://cloudstack.org/, http://openvswitch.org/

In part 1, I have introduced the basic installs for Host 1 and Host 2, in the second part I will introduce the network configuration on hosts to prepare for install Cloudstack and KVM.

Host 1 and Host 2 have 2 interface eth0 and eth1.

You must configure eth0 for manager interface (cloudbr0) => 10.0.0.2 for Host1 and 10.0.0.3 for Host2
                              eth1 for guest and public interface (cloudbr1)

You must perform the following commands on both Hosts:

Create parent bridge on openvswitch:
ovs-vsctl add-br br0

cloudbr0:
ovs-vsctl add-br cloudbr0 br0 => Create cloudbr0 bridge
ovs-vsctl add-port cloudbr0 eth0 => Add eth0 interface to cloudbr0 bridge
ifconfig cloudbr0 10.0.0.2/24 up => on Host1
ifconfig cloudbr0 10.0.0.3/24 up => on Host2

cloudbr1
ovs-vsctl add-br cloudbr1 br0 => Create cloudbr1 bridge
ovs-vsctl add-port cloudbr1 eth1 => Add eth1 interface to cloudbr1 bridge

You can check the bridge of openvswitch which ran with CloudStack by adding a bridge interface (br-ovs example):
brctl addbr br-ovs  => Ok with no error. If you see any error, please check compat module on OVS.

You have finished setup bridge interface on OVS for CloudStack Agent.

Step 2: Install CloudStack manager

Install Manager
Install the CloudStack packages. You should have a file in the form of CloudStack-NNNN.tar.gz. Untar the file and the run the install.

Choose "M" to install the Management server software.

Install Database
Run install.sh again and choose "D" to install MySQL
# ./install.sh
Setting up the temporary repository...
Cleaning Yum cache...
Loaded plugins: fastestmirror
11 metadata files removed
Welcome to the Cloud.com CloudStack Installer.  What would you like to do?

    A) Install the Agent
    S) Install the Usage Monitor
    D) Install the database server
    U) Upgrade the CloudStack packages installed on this computer
    R) Stop any running CloudStack services and remove the CloudStack packages from this computer
    Q) Quit

    > D

Edit the MySQL configuration (/etc/my.cnf) and insert the following lines in the [mysqld] section. You can put these lines below the datadir line.

innodb_rollback_on_timeout=1
innodb_lock_wait_timeout=600
max_connections=350

Restart the MySQL service and configure it to start on boot

# service mysqld restart
# chkconfig mysqld on

Use the following script to create the cloud user on the database with a password of your choice. This script deploys the database using the credentials in the deploy-as parameters. By default on RHEL/CentOS/Fedora MySQL is installed with no root password. If you have set a password you will need to postpend that to the deploy-as parameter as in "--deploy-as=root:password

# cloud-setup-databases cloud:dbpassword@localhost --deploy-as=root
Final, you must run #cloud-setup-management


Part 3: Install and configure Agent with NFS storage and openvswitch bridge.