Welcome to CloudStack Documentation!¶

Multiple Hypervisor Support¶

CloudStack works with a variety of hypervisors and hypervisor-like technologies. A single cloud can contain multiple hypervisor implementations. As of the current release CloudStack supports:

• BareMetal (via IPMI)
• Hyper-V
• KVM
• LXC
• vSphere (via vCenter)
• Xenserver
• Xen Project

Massively Scalable Infrastructure Management¶

CloudStack can manage tens of thousands of physical servers installed in geographically distributed datacenters. The management server scales near-linearly eliminating the need for cluster-level management servers. Maintenance or other outages of the management server can occur without affecting the virtual machines running in the cloud.

Automatic Cloud Configuration Management¶

CloudStack automatically configures the network and storage settings for each virtual machine deployment. Internally, a pool of virtual appliances support the operation of configuration of the cloud itself. These appliances offer services such as firewalling, routing, DHCP, VPN, console proxy, storage access, and storage replication. The extensive use of horizontally scalable virtual machines simplifies the installation and ongoing operation of a cloud.

Graphical User Interface¶

CloudStack offers an administrators web interface used for provisioning and managing the cloud, as well as an end-user’s Web interface, used for running VMs and managing VM templates. The UI can be customized to reflect the desired service provider or enterprise look and feel.

API¶

CloudStack provides a REST-like API for the operation, management and use of the cloud.

AWS EC2 API Support¶

CloudStack provides an EC2 API translation layer to permit the common EC2 tools to be used in the use of a CloudStack cloud.

High Availability¶

CloudStack has a number of features to increase the availability of the system. The Management Server itself may be deployed in a multi-node installation where the servers are load balanced. MySQL may be configured to use replication to provide for failover in the event of database loss. For the hosts, CloudStack supports NIC bonding and the use of separate networks for storage as well as iSCSI Multipath.

Management Server Overview¶

The management server orchestrates and allocates the resources in your cloud deployment.

The management server typically runs on a dedicated machine or as a virtual machine. It controls allocation of virtual machines to hosts and assigns storage and IP addresses to the virtual machine instances. The Management Server runs in an Apache Tomcat container and requires a MySQL database for persistence.

The management server:

• Provides the web interface for both the adminstrator and end user.
• Provides the API interfaces for both the CloudStack API as well as the EC2 interface.
• Manages the assignment of guest VMs to a specific compute resource
• Manages the assignment of public and private IP addresses.
• Allocates storage during the VM instantiation process.
• Manages snapshots, disk images (templates), and ISO images.
• Provides a single point of configuration for your cloud.

Cloud Infrastructure Overview¶

Resources within the cloud are managed as follows:

• Regions: A collection of one or more geographically proximate zones managed by one or more management servers.
• Zones: Typically, a zone is equivalent to a single datacenter. A zone consists of one or more pods and secondary storage.
• Pods: A pod is usually a rack, or row of racks that includes a layer-2 switch and one or more clusters.
• Clusters: A cluster consists of one or more homogenous hosts and primary storage.
• Host: A single compute node within a cluster; often a hypervisor.
• Primary Storage: A storage resource typically provided to a single cluster for the actual running of instance disk images. (Zone-wide primary storage is an option, though not typically used.)
• Secondary Storage: A zone-wide resource which stores disk templates, ISO images, and snapshots.

Networking Overview¶

CloudStack offers many types of networking, but they typically fall into one of two scenarios:

• Basic: Most analogous to AWS-classic style networking. Provides a single flat layer-2 network where guest isolation is provided at layer-3 by the hypervisors bridge device.
• Advanced: This typically uses layer-2 isolation such as VLANs, though this category also includes SDN technologies such as Nicira NVP.

About Regions¶

To increase reliability of the cloud, you can optionally group resources into multiple geographic regions. A region is the largest available organizational unit within a CloudStack deployment. A region is made up of several availability zones, where each zone is roughly equivalent to a datacenter. Each region is controlled by its own cluster of Management Servers, running in one of the zones. The zones in a region are typically located in close geographical proximity. Regions are a useful technique for providing fault tolerance and disaster recovery.

By grouping zones into regions, the cloud can achieve higher availability and scalability. User accounts can span regions, so that users can deploy VMs in multiple, widely-dispersed regions. Even if one of the regions becomes unavailable, the services are still available to the end-user through VMs deployed in another region. And by grouping communities of zones under their own nearby Management Servers, the latency of communications within the cloud is reduced compared to managing widely-dispersed zones from a single central Management Server.

Usage records can also be consolidated and tracked at the region level, creating reports or invoices for each geographic region.

Regions are visible to the end user. When a user starts a guest VM on a particular CloudStack Management Server, the user is implicitly selecting that region for their guest. Users might also be required to copy their private templates to additional regions to enable creation of guest VMs using their templates in those regions.

About Zones¶

A zone is the second largest organizational unit within a CloudStack deployment. A zone typically corresponds to a single datacenter, although it is permissible to have multiple zones in a datacenter. The benefit of organizing infrastructure into zones is to provide physical isolation and redundancy. For example, each zone can have its own power supply and network uplink, and the zones can be widely separated geographically (though this is not required).

A zone consists of:

• One or more pods. Each pod contains one or more clusters of hosts and one or more primary storage servers.
• A zone may contain one or more primary storage servers, which are shared by all the pods in the zone.
• Secondary storage, which is shared by all the pods in the zone.

Zones are visible to the end user. When a user starts a guest VM, the user must select a zone for their guest. Users might also be required to copy their private templates to additional zones to enable creation of guest VMs using their templates in those zones.

Zones can be public or private. Public zones are visible to all users. This means that any user may create a guest in that zone. Private zones are reserved for a specific domain. Only users in that domain or its subdomains may create guests in that zone.

Hosts in the same zone are directly accessible to each other without having to go through a firewall. Hosts in different zones can access each other through statically configured VPN tunnels.

For each zone, the administrator must decide the following.

• How many pods to place in each zone.
• How many clusters to place in each pod.
• How many hosts to place in each cluster.
• (Optional) How many primary storage servers to place in each zone and total capacity for these storage servers.
• How many primary storage servers to place in each cluster and total capacity for these storage servers.
• How much secondary storage to deploy in a zone.

When you add a new zone using the CloudStack UI, you will be prompted to configure the zone’s physical network and add the first pod, cluster, host, primary storage, and secondary storage.

In order to support zone-wide functions for VMware, CloudStack is aware of VMware Datacenters and can map each Datacenter to a CloudStack zone. To enable features like storage live migration and zone-wide primary storage for VMware hosts, CloudStack has to make sure that a zone contains only a single VMware Datacenter. Therefore, when you are creating a new CloudStack zone, you can select a VMware Datacenter for the zone. If you are provisioning multiple VMware Datacenters, each one will be set up as a single zone in CloudStack.

About Pods¶

A pod often represents a single rack. Hosts in the same pod are in the same subnet. A pod is the third-largest organizational unit within a CloudStack deployment. Pods are contained within zones. Each zone can contain one or more pods. A pod consists of one or more clusters of hosts and one or more primary storage servers. Pods are not visible to the end user.

About Clusters¶

A cluster provides a way to group hosts. To be precise, a cluster is a XenServer server pool, a set of KVM servers, , or a VMware cluster preconfigured in vCenter. The hosts in a cluster all have identical hardware, run the same hypervisor, are on the same subnet, and access the same shared primary storage. Virtual machine instances (VMs) can be live-migrated from one host to another within the same cluster, without interrupting service to the user.

A cluster is the fourth-largest organizational unit within a CloudStack deployment. Clusters are contained within pods, and pods are contained within zones. Size of the cluster is limited by the underlying hypervisor, although the CloudStack recommends less in most cases; see Best Practices.

A cluster consists of one or more hosts and one or more primary storage servers.

CloudStack allows multiple clusters in a cloud deployment.

Even when local storage is used exclusively, clusters are still required organizationally, even if there is just one host per cluster.

When VMware is used, every VMware cluster is managed by a vCenter server. An Administrator must register the vCenter server with CloudStack. There may be multiple vCenter servers per zone. Each vCenter server may manage multiple VMware clusters.

About Hosts¶

A host is a single computer. Hosts provide the computing resources that run guest virtual machines. Each host has hypervisor software installed on it to manage the guest VMs. For example, a host can be a Citrix XenServer server, a Linux KVM-enabled server, an ESXi server, or a Windows Hyper-V server.

The host is the smallest organizational unit within a CloudStack deployment. Hosts are contained within clusters, clusters are contained within pods, pods are contained within zones, and zones can be contained within regions.

Hosts in a CloudStack deployment:

• Provide the CPU, memory, storage, and networking resources needed to host the virtual machines
• Interconnect using a high bandwidth TCP/IP network and connect to the Internet
• May reside in multiple data centers across different geographic locations
• May have different capacities (different CPU speeds, different amounts of RAM, etc.), although the hosts within a cluster must all be homogeneous

Additional hosts can be added at any time to provide more capacity for guest VMs.

CloudStack automatically detects the amount of CPU and memory resources provided by the hosts.

Hosts are not visible to the end user. An end user cannot determine which host their guest has been assigned to.

For a host to function in CloudStack, you must do the following:

• Install hypervisor software on the host
• Assign an IP address to the host
• Ensure the host is connected to the CloudStack Management Server.

About Primary Storage¶

Primary storage is associated with a cluster, and it stores virtual disks for all the VMs running on hosts in that cluster. On KVM and VMware, you can provision primary storage on a per-zone basis.

You can add multiple primary storage servers to a cluster or zone. At least one is required. It is typically located close to the hosts for increased performance. CloudStack manages the allocation of guest virtual disks to particular primary storage devices.

It is useful to set up zone-wide primary storage when you want to avoid extra data copy operations. With cluster-based primary storage, data in the primary storage is directly available only to VMs within that cluster. If a VM in a different cluster needs some of the data, it must be copied from one cluster to another, using the zone’s secondary storage as an intermediate step. This operation can be unnecessarily time-consuming.

For Hyper-V, SMB/CIFS storage is supported. Note that Zone-wide Primary Storage is not supported in Hyper-V.

Ceph/RBD storage is only supported by the KVM hypervisor. It can be used as Zone-wide Primary Storage.

CloudStack is designed to work with all standards-compliant iSCSI and NFS servers that are supported by the underlying hypervisor, including, for example:

• SolidFire for iSCSI
• Dell EqualLogic™ for iSCSI
• Network Appliances filers for NFS and iSCSI
• Scale Computing for NFS

If you intend to use only local disk for your installation, you can skip adding separate primary storage.

About Secondary Storage¶

Secondary storage stores the following:

• Templates — OS images that can be used to boot VMs and can include additional configuration information, such as installed applications
• ISO images — disc images containing data or bootable media for operating systems
• Disk volume snapshots — saved copies of VM data which can be used for data recovery or to create new templates

The items in secondary storage are available to all hosts in the scope of the secondary storage, which may be defined as per zone or per region.

To make items in secondary storage available to all hosts throughout the cloud, you can add object storage in addition to the zone-based NFS Secondary Staging Store. It is not necessary to copy templates and snapshots from one zone to another, as would be required when using zone NFS alone. Everything is available everywhere.

For Hyper-V hosts, SMB/CIFS storage is supported.

CloudStack provides plugins that enable both OpenStack Object Storage (Swift, swift.openstack.org) and Amazon Simple Storage Service (S3) object storage. When using one of these storage plugins, you configure Swift or S3 storage for the entire CloudStack, then set up the NFS Secondary Staging Store for each zone. The NFS storage in each zone acts as a staging area through which all templates and other secondary storage data pass before being forwarded to Swift or S3. The backing object storage acts as a cloud-wide resource, making templates and other data available to any zone in the cloud.

About Physical Networks¶

Part of adding a zone is setting up the physical network. One or (in an advanced zone) more physical networks can be associated with each zone. The network corresponds to a NIC on the hypervisor host. Each physical network can carry one or more types of network traffic. The choices of traffic type for each network vary depending on whether you are creating a zone with basic networking or advanced networking.

A physical network is the actual network hardware and wiring in a zone. A zone can have multiple physical networks. An administrator can:

• Add/Remove/Update physical networks in a zone
• Configure VLANs on the physical network
• Configure a name so the network can be recognized by hypervisors
• Configure the service providers (firewalls, load balancers, etc.) available on a physical network
• Configure the IP addresses trunked to a physical network
• Specify what type of traffic is carried on the physical network, as well as other properties like network speed

Compatibilty¶

The following table shows the compatiblity of Cloudian Connector with CloudStack.

Table: Support Matrix

Single-Sign-On Integration¶

The connector plugin adds a Cloudian Storage button to the CloudStack UI. This button is available for all users on the bottom left of the menu.

When a user clicks this button, a new window or tab (depending on the web browser preferences) is opened for the HyperStore CMC GUI. The CloudStack user is automatically logged in to CMC as the correctly mapped HyperStore user using Single-Sign-On (SSO).

With the connector enabled, when the user clicks ‘Log Out’ in the CloudStack UI this first logs the user out of CloudStack and then redirects the page to log out any logged-in Cloudian user out of CMC GUI and finally redirects the page to the CloudStack login page.

Single-Sign-On is a technique where CloudStack and HyperStore are configured to trust each other. This is achieved by configuring both HyperStore and the CloudStack connector with the same SSO Shared Key. The CloudStack connector creates a special login URL for CMC which it signs using this shared key. Upon receiving the special signed login URL, CMC validates the request by comparing the signature to its own copy of the shared key and the user is automatically logged in.

User Mapping and Provisioning/De-provisioning¶

CloudStack domains are mapped to Cloudian Groups. CloudStack accounts within those domains are mapped to Cloudian users. The Cloudian user and group are created on demand if it doesn’t already exist when the CloudStack user accesses CMC through the Cloudian Storage button. When accounts and domains are created or removed in CloudStack, they automatically create or remove users or groups in CMC.

Table: Mapping Between Cloudian and CloudStack

Special Admin User Mapping¶

The special CloudStack admin is are mapped to a special HyperStore Admin user account which defaults to the user id admin. As the admin user on HyperStore is configurable, there is a configuration option to control this mapping. This mapping dictates which HyperStore user is automatically logged in using SSO when the CloudStack admin user clicks « Cloudian Storage ».

DNS Resolution Requirements¶

The CloudStack Management Server will need to be access the Cloudian admin service. The Cloudian admin service is commonly run on the same nodes as your Cloudian S3 servers. The admin service is used to provision and deprovision Cloudian users and groups automatically by CloudStack.

Additionally, your CloudStack users will need to be able to resolve your CMC server hostname on their desktops so that they can access CMC.

Example domain names that should resolve:

Table: DNS Name Resolution Example

Prerequisites¶

Cloudian ships with SSO disabled by default. You will need to enable it on each CMC server. Additionally, you will need to choose a unique SSO shared key that you will also configure in the CloudStack connector further below.

Edit Puppet config to enable SSO on all CMC servers:

# vi /etc/cloudian-[version]-puppet/modules/cmc/templates/mts-ui.properties.erb
  sso.enabled=true
  sso.shared.key=YourSecretKeyHere

Connector Configuration¶

The main way to configure, enable and disable the connector is using the CloudStack global setting. The global settings provide an easy way to configure the connector and synchronize setting across multiple management server(s). The following global setting can be accessed and changed using the CloudStack UI:

Table: Cloudian Connector Global Settings

Enabling the Cloudian Connector¶

The Cloudian Connector comes disabled by default, enabling the connector is the last step. You should have already configured the Cloudian Connector global settings. To enable the connector, ensure that the global setting « cloudian.connector.enabled » is set to true. Finally, restart each of the management server(s) to reload and enable the connector.

For example, here is how you can restart the CloudStack management server installed on CentOS7:

# systemctl restart cloudstack-management

Troubleshooting¶

Most of the trouble you may run into will be configuration related.

There are a few things which can go wrong for SSO. Here are the most common problems and things to check:

• Does the global settings cloudian.cmc.admin.user point to the correct Cloudian (admin) user?
• Is SSO configured and enabled on Cloudian HyperStore CMC?
• Check for errors in the CMC log file.
• Are both CloudStack and HyperStore CMC configured with the same cloudian.sso.key?
• Check the /var/log/cloudstack/management/management-server.log file and search for errors relating to SSO.
• Try access the CMC host directly from the problem users host using the configured cloudian.cmc.host, cloudian.cmc.port and cloudian.cmc.protocol configured in the CloudStack global settings.
• If you log out of the management server and log in again, does the Cloudian Storage button work?

Adding/Deleting Domains or Accounts fails: These operations use the Cloudian Admin Server. It’s likely that something has changed with the connection or the admin server is down. Check list:

• Is the admin server alive and listening?
• Try access the admin server host directly from the problem users host using the configured cloudian.admin.host, cloudian.admin.port and cloudian.admin.protocol configured in the CloudStack global settings. Check the configured auth settings cloudian.admin.user and cloudian.admin.password.
• If you’re experiencing timeout issues, trying changing the API timeout value defined in cloudian.api.request.timeout global setting.
• Look for errors in the admin log file /var/log/cloudian/cloudian-admin.log.

NFS Secondary Storage Staging Server Requirement¶

The use of S3 as Secondary Storage for CloudStack also requires an NFS server. The NFS server is required because the various hypervisors cannot yet talk directly to S3 but instead talk through the standard file system API. As such, CloudStack requires an NFS staging server which the Hypervisors use to read and write data from/to. The NFS storage requirements for the staging server are small however as space is only required while objects are staged (moving) between the S3 server and the VMs.

DNS Name Resolution Requirement¶

All CloudStack Management Servers, system VMs and customer VMs (if required) must be able to resolve your S3 bucket names. Usually, if you already have Cloudian installed and running in your environment, this is already working. At a minimum the following names should resolve to the correct IP addresses using the DNS server that your Management Server and System VMs are using.

Table: Example Domain Names that should Resolve on CloudStack Servers

Setup a Cloudian User and Bucket for Secondary Storage¶

S3 Secondary Storage stores the CloudStack templates, snapshots etc in a dedicated S3 Bucket. To properly configure CloudStack you will need to know the S3 Bucket name and how to access your S3 Server (the S3 endpoint, access key and secret key).

Below, we will create a dedicated Cloudian user and a dedicated bucket which we will assign for use as Secondary Storage.

Create a dedicated user/group:

• Login to the Cloudian Management Console (CMC) as the Cloudian admin user.
• Create a new group called cloudstack. Any group name is ok.
• Create a new user called cloudstack in the cloudstack group. Any user name is ok.

Create a dedicated bucket:

• Login to CMC as the cloudstack user created above.
• Create a bucket called secondary. Any bucket name will do.
• On the top menu bar on the right hand side, use the drop down menu under your user name to select Security Credentials and copy and paste your Access and Secret Keys to a note for later use. CloudStack will need these when you attach Cloudian as Secondary Storage in a later step below.

Open Up Access to your S3 Network from Secondary Storage¶

If your S3 server is on a different network to your Secondary Storage VM, you will need to open up access to the S3 network. This also allows users to download templates from their S3 object store areas.

Add an NFS Secondary Storage Staging Server¶

As mentioned previously, S3 Secondary Storage currently requires the use of an NFS Secondary Staging Server. Add NFS Secondary Storage Staging Server:

• Login to CloudStack Management Server as the admin user.
• Navigate to Infrastructure → Secondary Storage.
• Click Select View and select Secondary Staging Store.
• Click Add Secondary Staging Store.
• Configure the zone, server and path for your desired secondary staging store. For example nfs.mycloud.com and /export/staging.

Attach Cloudian as Secondary Storage¶

CloudStack supports using either S3 or NFS as Secondary Storage but not both. The below instructions assume you are not using Secondary Storage on NFS and that you can delete it to add the S3 storage.

Adding S3 Secondary Storage:

• Login to CloudStack Management Server as the admin user.
• Navigate to Infrastructure → Secondary Storage.
• If it exists, select and delete any existing NFS Secondary Storage server setting. NOTE: Do not do this if you want to migrate existing NFS secondary storage to S3. Instead, see warning above.
• Click the Add Secondary Storage button. This will open up a pop-up form which you can fill out similarly to below.

When you have finished adding Cloudian as Secondary Storage in the previous steps, CloudStack will populate the new secondary storage with the system and default templates. This can take some time do download as the templates are quite big.

CloudStack should now ready to use Cloudian HyperStore for S3 Secondary Storage.

Features of the Nicira NVP Plugin¶

The following table lists the CloudStack network services provided by the Nicira NVP Plugin.

Table: Supported Services

The following hypervisors are supported by the Nicira NVP Plugin.

Table: Supported Hypervisors

Prerequisites¶

Before enabling the Nicira NVP plugin the NVP Controller needs to be configured. Please review the NVP User Guide on how to do that.

Make sure you have the following information ready:

• The IP address of the NVP Controller
• The username to access the API
• The password to access the API
• The UUID of the Transport Zone that contains the hypervisors in this Zone
• The UUID of the Gateway Service used to provide router and NAT services.

Zone Configuration¶

CloudStack needs to have at least one physical network with the isolation method set to « STT ». This network should be enabled for the Guest traffic type.

Enabling the service provider¶

The Nicira NVP provider is disabled by default. Navigate to the « Network Service Providers » configuration of the physical network with the STT isolation type. Navigate to the Nicira NVP provider and press the « Enable Provider » button.

Device Management¶

In CloudStack a Nicira NVP setup is considered a « device » that can be added and removed from a physical network. To complete the configuration of the Nicira NVP plugin a device needs to be added to the physical network. Press the « Add NVP Controller » button on the provider panel and enter the configuration details.

Network Offerings¶

Using the Nicira NVP plugin requires a network offering with Virtual Networking enabled and configured to use the NiciraNvp element. Typical use cases combine services from the Virtual Router appliance and the Nicira NVP plugin.

Table: Isolated network offering with regular services from the Virtual Router.

Isolated network with network services. The virtual router is still required to provide network services like dns and dhcp.

Table: Isolated network offering with network services

Supported VPC features¶

The Nicira NVP plugin supports CloudStack VPC to a certain extent. Starting with CloudStack version 4.1 VPCs can be deployed using NVP isolated networks.

It is not possible to use a Nicira NVP Logical Router for as a VPC Router

It is not possible to connect a private gateway using a Nicira NVP Logical Switch

VPC Offering with Nicira NVP¶

To allow a VPC to use the Nicira NVP plugin to provision networks, a new VPC offering needs to be created which allows the Virtual Networking service to be implemented by NiciraNVP.

This is not currently possible with the UI. The API does provide the proper calls to create a VPC offering with Virtual Networking enabled. However due to a limitation in the 4.1 API it is not possible to select the provider for this network service. To configure the VPC offering with the NiciraNVP provider edit the database table “vpc_offering_service_map” and change the provider to NiciraNvp for the service “Connectivity”

It is also possible to update the default VPC offering by adding a row to the “vpc_offering_service_map” with service “Connectivity” and provider “NiciraNvp”

VPC Network Offerings¶

The VPC needs specific network offerings with the VPC flag enabled. Otherwise these network offerings are identical to regular network offerings. To allow VPC networks with a Nicira NVP isolated network the offerings need to support the Virtual Networking service with the NiciraNVP provider.

In a typical configuration two network offerings need to be created. One with the loadbalancing service enabled and one without loadbalancing.

Table: VPC Network Offering with Loadbalancing

UUID References¶

The plugin maintains several references in the CloudStack database to items created on the NVP Controller.

Every guest network that is created will have its broadcast type set to Lswitch and if the network is in state « Implemented », the broadcast URI will have the UUID of the Logical Switch that was created for this network on the NVP Controller.

The Nics that are connected to one of the Logical Switches will have their Logical Switch Port UUID listed in the nicira_nvp_nic_map table

Database tables¶

The following tables are added to the cloud database for the Nicira NVP Plugin

Table: nicira_nvp_nic_map

Table: external_nicira_nvp_devices

Table: nicira_nvp_router_map

Supported Features¶

The following table lists the supported Network services in a CloudStack deployment with NuageVsp being the Connectivity/Virtual Networking provider, with their providers and supported CloudStack versions.

Table: Supported Network Services

Supported Hypervisors¶

The following hypervisors are supported by the Nuage VSP Plugin, with their supported CloudStack versions.

Table: Supported Hypervisors

Supported Nuage VSP SDN Platform Versions¶

The following Nuage VSP SDN Platform versions are supported by the Nuage VSP Plugin, with their supported CloudStack versions.

Table: Supported Nuage VSP SDN Platform Versions

Prerequisites¶

Before enabling and using the Nuage VSP Plugin with CloudStack.

1. Verify that the CloudStack deployment (hypervisors) and Nuage VSP SDN Platform version you intend to use is being supported.

2. Prepare and configure the hypervisors for CloudStack integration with Nuage VSP SDN Platform.

Required Nuage VSD Configuration¶

When configuring Nuage VSP as the network service provider in a CloudStack Zone, a CSP user must be added in Nuage VSD, and this user must be added to the CMS group. See Enable Nuage VSP Network Service Provider.

Zone Configuration¶

Select VSP Isolation Method¶

The Nuage VSP solution is NOT supported in Basic zone provisioning mode.

1. When adding a zone, the CloudStack administrator should select Advanced mode in the zone wizard.
2. When laying out the physical network configuration during zone provisioning, the Guest network traffic should be put in a separate physical network of its own.
3. This physical network carrying the Guest traffic should have VSP as the Isolation Method.

Setting Isolation Method to VSP

Update Traffic Labels¶

Guest Traffic Type

Select Edit on the Guest traffic type panel and update the Traffic Label:

• For KVM, use alubr0 as the KVM Traffic Label.

Specifying the Traffic Type in KVM

• For VMware ESXi, use the switch name used by dVRS for guest networking as the vSwitch Name, leave the VLAN ID field blank, and select VMware vNetwork Distributed Switch in the vSwitch Type drop down field.

Specifying the Traffic Type in VMware ESXi

Enable Nuage VSP Network Service Provider¶

Nuage VSP must be added and enabled as a Network Service Provider in the CloudStack Zone before it can be used.

Adding Nuage VSD as the Network Service Provider

Enabling Nuage VSP Network Service Provider

Viewing Network Service Providers Status

Network Offerings¶

There are three types of Network Offerings that can be created:

• If Isolated Networks are required, then create a Isolated guest type network offering for use with Isolated Networks.
• If VPC deployments are required, then create a new Isolated guest type network offering for such deployments.
• If Shared Networks are required, then create a new Shared guest type network offering for use with Shared Networks.

Create and Enable Isolated Network Offering¶

1. Select Service Offerings > Select Offering: Network Offerings > Add network offering, which brings up the Add network offering.
2. In the Add network offering panel, add a Name and a Description to the network offering. Select Isolated as the Guest Type. In the Supported Services field select services and providers that are supported by the Nuage VSP Plugin for Isolated Networks, see Supported Features at the beginning of this document.

Creating Isolated Network Offering

3. Click the OK button to create the network offering.
4. After the network offering has been successfully created, enable it from the Service Offerings - Network Offerings list.

Create and Enable VPC Network Offering¶

1. Select Service Offerings > Select Offering: Network Offerings > Add network offering, which brings up the Add network offering.
2. In the Add network offering panel, add a Name and a Description to the network offering. Select Isolated as the Guest Type. Select the VPC field. In the Supported Services field select services and providers that are supported by the Nuage VSP Plugin for VPCs, see Supported Features at the beginning of this document.

Creating VPC Network Offering

3. Click the OK button to create the network offering.
4. After the network offering has been successfully created, enable it from the Service Offerings - Network Offerings list.

Create and Enable Shared Network Offering¶

1. Select Service Offerings > Select Offering: Network Offerings > Add network offering, which brings up the Add network offering.
2. In the Add network offering panel, add a Name and a Description to the network offering. Select Shared as the Guest Type. In the Supported Services field select services and providers that are supported by the Nuage VSP Plugin for Shared Networks, see Supported Features at the beginning of this document.

Creating Shared Network Offering

3. Click the OK button to create the network offering.
4. After the network offering has been successfully created, enable it from the Service Offerings - Network Offerings list.

VPC Offerings¶

Pre-created and Enabled Nuage VSP VPC Offering¶

A VPC offering by the name Nuage VSP VPC Offering is pre-created and enabled in the list of Service Offerings - VPC Offerings (Select Service Offerings > Select Offering: VPC Offerings) which contains all the services and providers that are supported by the Nuage VSP Plugin for VPCs.

Pre-created and Enabled Nuage VSP VPC Offering

(Optional) Create and Enable VPC Offering¶

1. Select Service Offerings > Select Offering: VPC Offerings > Add VPC Offering, which brings up the Add VPC Offering.
2. In the Add VPC Offering panel, add a Name and a Description to the network offering. In the Supported Services field select services and providers that are supported by the Nuage VSP Plugin for VPCs, see Supported Features at the beginning of this document.

Creating VPC Offering

3. Click the OK button to create the VPC Offering.
4. After the VPC Offering has been successfully created, enable it from the Service Offerings - VPC Offerings list.

Nuage VSP Domain Template Feature Support for CloudStack¶

All the constructs (parameters and abstractions) defined in a Nuage VSD domain template can be made available to domain instances (i.e. networks) created in CloudStack. To do this, configure the Nuage VSP Plugin to use a pre-created Nuage VSD domain template when instantiating domains (i.e. creating networks). Networks created in CloudStack will then use domain instances created from the domain template.

Typical use-cases are:

• The basic ACLs on the top and bottom that bracket or “contain” the end-user’s ACLs.
• Leakable domains/GRT Leaking (Nuage VSP feature).

To configure a Nuage VSP domain template for use by CloudStack, use the Nuage VSD Architect (VSP’s GUI) to create a domain template and configure it in the following CloudStack global settings.

Table: CloudStack Global Settings For Configuring Nuage VSP Domain Template Feature

Nuage VSP Source NAT via the Underlay Feature Support For CloudStack¶

Supported CloudStack versions: >= 4.10

CloudStack provides Source NAT service to enable guest VMs to send traffic out to the Internet without requiring a Static NAT IP (public IP) assigned to the VM. The Source NAT service must be enabled as part of the network offering used for creating the guest network. When a network is created using this network offering, the first public IP from the assigned public IP range is automatically acquired as the Source NAT IP for the network. All VMs attached to this network then use that Source NAT IP to send traffic to the Internet.

The Nuage VSP Plugin for CloudStack supports CloudStack’s native Source NAT service and enhances it by restricting to a minimum the number of public IP addresses assigned to any given tenant. This is achieved by not allocating a Source NAT IP for every network that is created.

The Source NAT service that Nuage VSP calls the Port Address Translation (PAT) feature uses the hypervisor IP as the Source NAT IP address for all VMs in the hypervisor that need to send traffic out to the Internet. Configure this during Nuage VSP installation using the instructions given in the Nuage VSP Install Guide.

This feature is supported for both VPCs and Isolated Networks. In the case of VPCs, Source NAT is applied at the Nuage VSP domain level, therefore there is no customization on the individual VPC network (tier) level.

All VPCs and Isolated networks that are created from a Nuage VSP Source NAT-enabled network offering have this feature enabled automatically. An example Nuage VSP Source NAT-enabled network offering is shown in the figure « Nuage VSP Source NAT-enabled Network Offering » below.

Nuage VSP Source NAT-enabled Network Offering

Nuage VSP Static NAT via the Underlay Feature Support For CloudStack¶

Supported CloudStack versions: >= 4.10

Static NAT is supported in Nuage VSP as FIP (Floating IP). Prior to Nuage VSP v3.2, FIP in Nuage VSP required a VXLAN GW/PE to be present in the data center. In Nuage VSP v3.2 and above FIP is supported via the underlay, which removes the requirement for a GW/PE in the DC.

For the Static NAT without GW/PE feature to be operational in the CloudStack plugin, FIP in Nuage VSP must be configured to use the underlay. This operation takes place during Nuage VSP installation; instructions can be found in the Nuage VSP Install Guide.

A new API called nuageunderlayvlaniprange has been introduced to enable/disable Static NAT via the Underlay feature support for CloudStack public IP ranges being used for Static NAT service. This API specifies whether the FIP to underlay support is required for the corresponding FIP subnet in Nuage VSD since there is no GW/PE in the data center. When the nuageunderlayvlaniprange API has been enabled/disabled for a public IP range and Static NAT is enabled on at-least one of its Public IPs, the plugin creates the corresponding shared FIP subnet in Nuage VSD using the sharednetworkresources API with the underlay flag set accordingly. The nuageunderlayvlaniprange API usage is shown in the figure « nuageunderlayvlaniprange API Usage » below.

nuageunderlayvlaniprange API Usage

By default, the Nuage VSP Plugin creates the corresponding shared FIP subnet in Nuage VSD with the underlay flag set to false (disabled). There is no support for the nuageunderlayvlaniprange API from the CloudStack UI.

Configure Nuage VSP API¶

To configure Nuage VSP as a Network Service Provider in the CloudStack Zone.

1. Add Nuage VSP as a Network Service Provider in the Physical Network 2:

cloudmonkey add networkserviceprovider name=NuageVsp physicalnetworkid=<physicalNetwork2Id>

2. Add the Nuage VSD as a Nuage VSP Device in the Physical Network 2:

cloudmonkey add nuagevspdevice physicalnetworkid=<physicalNetwork2Id> hostname=<hostnameOfNuageVsp> username=<usernameOfNuageVspUser> password=<passwordOfNuageVspUser> port=<portUsedByNuageVsp> apiversion=<apiVersionOfNuageVsp> retrycount=<nrOfRetriesOnFailure> retryinterval=<intervalBetweenRetries>

Nuage VSP Marvin Config File Format¶

Format for the Marvin config file required to run The Nuage VSP Plugin specific Marvin tests.

{
 "zones": [
     {
         "name": "ZONE1NAME",
         "physical_networks": [
             {
                 "name": "Physical Network 1",
                 "isolationmethods": [
                     "VLAN"
                 ]
             },
             {
                 "name": "Physical Network 2",
                 "isolationmethods": [
                     "VSP"
                 ],
                 "providers": [
                     {
                         "name": "NuageVsp",
                         "devices": [
                             {
                                 "username": "VSDUSERNAME",
                                 "retryinterval": "60",
                                 "hostname": "VSDSERVER",
                                 "apiversion": "VSDVERSION",
                                 "retrycount": "4",
                                 "password": "VSDUSERPASSWORD",
                                 "port": VSDPORT
                             }
                         ]
                     }
                 ]
             }
         ],
         "dcInternetConnectivityInfo" : {
             "available": "INTERNETAVAILABLE",
             "httpProxy": "HTTPPROXY",
             "httpsProxy": "HTTPSPROXY"
         }
     },
     {
         "name": "ZONE2NAME",
         "physical_networks": [
             {
                 "name": "Physical Network 1",
                 "isolationmethods": [
                     "VLAN"
                 ]
             },
             {
                 "name": "Physical Network 2",
                 "isolationmethods": [
                     "VSP"
                 ],
                 "providers": [
                     {
                         "name": "NuageVsp",
                         "devices": [
                             {
                                 "username": "VSDUSERNAME",
                                 "retryinterval": "60",
                                 "hostname": "VSDSERVER",
                                 "apiversion": "VSDVERSION",
                                 "retrycount": "4",
                                 "password": "VSDUSERPASSWORD",
                                 "port": VSDPORT
                             }
                         ]
                     }
                 ]
             }
         ],
         "dcInternetConnectivityInfo" : {
             "available": "INTERNETAVAILABLE",
             "httpProxy": "HTTPPROXY",
             "httpsProxy": "HTTPSPROXY"
         }
     }
 ],
 "dbSvr": {
     "dbSvr": "DBSERVER",
     "passwd": "DBPASSWORD",
     "db": "cloud",
     "port": 3306,
     "user": "DBUSERNAME"
 },
 "logger":
     {
         "LogFolderPath": "/tmp/LOGFOLDERNAME"
     },
 "mgtSvr": [
     {
         "mgtSvrIp": "MGNTSERVERIP",
         "port": 8096,
         "user": "MGNTUSERNAME",
         "passwd": "MGNTPASSWORD"
     }
 ]
 }

Check the capability of your system¶

To check the capability of your system, execute the following commands.

$ sudo modprobe vxlan && echo $?
# Confirm the output is "0".
# If it's non-0 value or error message, your kernel doesn't have VXLAN kernel module.

$ ip link add type vxlan help
# Confirm the output is usage of the command and that it's for VXLAN.
# If it's not, your iproute2 utility doesn't support VXLAN.

Important note on MTU size¶

When new vxlan interfaces are created, kernel will obtain current MTU size of the physical interface (ethX or the bridge) and then create vxlan interface/bridge that are exactly 50 bytes smaller than the MTU on physical interface/bridge. This means that in order to support default MTU size of 1500 bytes inside VM, your vxlan interface/bridge must also have MTU of 1500 bytes, meaning that your physical interface/bridge must have MTU of at least 1550 bytes. In order to configure « jumbo frames » you can i.e. make physical interface/bridge with 9000 bytes MTU, then all the vxlan interfaces will be created with MTU of 8950 bytes, and then MTU size inside VM can be set to 8950 bytes.

Important note on max number of multicast groups (and thus VXLAN intefaces)¶

Default value of « net.ipv4.igmp_max_memberships » (cat /proc/sys/net/ipv4/igmp_max_memberships) is « 20 », which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host). Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it’s own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host. On Linux kernel 3.x you actually can provision more than 20, but ARP request will silently fail and cause client’s networking problems On Linux kernel 4.x you can NOT provision (start) more than 20 VXLAN interfaces and error message « No buffer space available » can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces. Increase needed parameter to sane value (i.e. 100 or 200) as required. If you need to operate more than 20 VMs from different client’s network, this change above is required.

Advanced: Build kernel and iproute2¶

Even if your system doesn’t support VXLAN, you can compile the kernel and iproute2 by yourself. The following procedure is an example for CentOS 6.4.

$ sudo yum groupinstall "Development Tools"
$ sudo yum install ncurses-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel bc

$ KERNEL_VERSION=3.10.4
# Declare the kernel version you want to build.

$ wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-${KERNEL_VERSION}.tar.xz
$ tar xvf linux-${KERNEL_VERSION}.tar.xz
$ cd linux-${KERNEL_VERSION}
$ cp /boot/config-`uname -r` .config
$ make oldconfig
# You may keep hitting enter and choose the default.

$ make menuconfig
# Dig into "Device Drivers" -> "Network device support",
# then select "Virtual eXtensible Local Area Network (VXLAN)" and hit space.
# Make sure it indicates "<M>" (build as module), then Save and Exit.

# You may also want to check "IPv4 NAT" and its child nodes in "IP: Netfilter Configuration"
# and "IPv6 NAT" and its child nodes in "IPv6: Netfilter Configuration".
# In 3.10.4, you can find the options in
# "Networking support" -> "Networking options"
#   -> "Network packet filtering framework (Netfilter)".

$ make # -j N
# You may use -j N option to make the build process parallel and faster,
# generally N = 1 + (cores your machine have).

$ sudo make modules_install
$ sudo make install
# You would get an error like "ERROR: modinfo: could not find module XXXX" here.
# This happens mainly due to config structure changes between kernel versions.
# You can ignore this error, until you find you need the kernel module.
# If you feel uneasy, you can go back to make menuconfig,
# find module XXXX by using '/' key, enable the module, build and install the kernel again.

$ sudo vi /etc/grub.conf
# Make sure the new kernel isn't set as the default and the timeout is long enough,
# so you can select the new kernel during boot process.
# It's not a good idea to set the new kernel as the default until you confirm the kernel works fine.

$ sudo reboot
# Select the new kernel during the boot process.

$ sudo yum install db4-devel

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
$ cd iproute2
$ git tag
# Find the version that matches the kernel.
# If you built kernel 3.10.4 as above, it would be v3.10.0.

$ git checkout v3.10.0
$ ./configure
$ make # -j N
$ sudo make install

Configure hypervisor¶

$ sudo vi /etc/sysconfig/network-scripts/ifcfg-cloudbr1

DEVICE=cloudbr1
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=none
IPV6INIT=no
IPV6_AUTOCONF=no
DELAY=5
STP=yes

DEVICE=cloudbr1
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.0.2.X
NETMASK=255.255.255.0
IPV6INIT=no
IPV6_AUTOCONF=no
DELAY=5
STP=yes

$ sudo vi /etc/network/interfaces

auto lo
iface lo inet loopback

# The primary network interface
auto eth0.100
iface eth0.100 inet static
    address 192.168.42.11
    netmask 255.255.255.240
    gateway 192.168.42.1
    dns-nameservers 8.8.8.8 8.8.4.4
    dns-domain lab.example.org

# Public network
auto cloudbr0
iface cloudbr0 inet manual
    bridge_ports eth0.200
    bridge_fd 5
    bridge_stp off
    bridge_maxwait 1

# Private network
auto cloudbr1
iface cloudbr1 inet manual
    bridge_ports eth0.300
    bridge_fd 5
    bridge_stp off
    bridge_maxwait 1

auto lo
iface lo inet loopback

# The primary network interface
auto eth0.100
iface eth0.100 inet static
    address 192.168.42.11
    netmask 255.255.255.240
    gateway 192.168.42.1
    dns-nameservers 8.8.8.8 8.8.4.4
    dns-domain lab.example.org

# Public network
auto cloudbr0
iface cloudbr0 inet manual
    bridge_ports eth0.200
    bridge_fd 5
    bridge_stp off
    bridge_maxwait 1

# Private network
auto cloudbr1
iface cloudbr1 inet static
    addres 192.0.2.X
    netmask 255.255.255.0
    bridge_ports eth0.300
    bridge_fd 5
    bridge_stp off
    bridge_maxwait 1

$ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT

$ sudo iptables-save > /etc/sysconfig/iptables

$ sudo service network restart
$ sudo reboot

$ sudo ufw allow proto udp from any to any port 8472

$ sudo service networking restart
$ sudo reboot

Setup zone using VXLAN¶

In almost all parts of zone setup, you can just follow the advanced zone setup istruction in « PRODUCT Installation Guide » to use this plugin. It is not required to add a network element nor to reconfigure the network offering. The only thing you have to do is configure the physical network to use VXLAN as the isolation method for Guest Network.

Configure the physical network¶

CloudStack needs to have one physical network for Guest Traffic with the isolation method set to « VXLAN ».

Guest Network traffic label should be the name of the physical interface or the name of the bridge interface and the bridge interface and they should have an IPv4 address. See ? for details.

Configure the guest traffic¶

Specify a range of VNIs you would like to use for carrying guest network traffic.

Avertissement

VNI must be unique per zone and no duplicate VNIs can exist in the zone. Exercise care when designing your VNI allocation policy.

Features of the OVS Plugin¶

The following table lists the CloudStack network services provided by the OVS Plugin.

Table: Supported Services

The following hypervisors are supported by the OVS Plugin.

Table: Supported Hypervisors

Prerequisites¶

Before enabling the OVS plugin the hypervisor needs to be install OpenvSwitch. Default, XenServer has already installed OpenvSwitch. However, you must install OpenvSwitch manually on KVM. CentOS 6.4 and OpenvSwitch 1.10 are recommended.

KVM hypervisor:

• CentOS 6.4 is recommended.
• To make sure that the native bridge module will not interfere with openvSwitch the bridge module should be added to the blacklist. See the modprobe documentation for your distribution on where to find the blacklist. Make sure the module is not loaded either by rebooting or executing rmmod bridge before executing next steps.

Zone Configuration¶

CloudStack needs to have at least one physical network with the isolation method set to “GRE”. This network should be enabled for the Guest traffic type.

Agent Configuration¶

• Configure network interfaces:

  /etc/sysconfig/network-scripts/ifcfg-eth0
  DEVICE=eth0
  BOOTPROTO=none
  IPV6INIT=no
  NM_CONTROLLED=no
  ONBOOT=yes
  TYPE=OVSPort
  DEVICETYPE=ovs
  OVS_BRIDGE=cloudbr0
  
  /etc/sysconfig/network-scripts/ifcfg-eth1
  DEVICE=eth1
  BOOTPROTO=none
  IPV6INIT=no
  NM_CONTROLLED=no
  ONBOOT=yes
  TYPE=OVSPort
  DEVICETYPE=ovs
  OVS_BRIDGE=cloudbr1
  
  /etc/sysconfig/network-scripts/ifcfg-cloudbr0
  DEVICE=cloudbr0
  ONBOOT=yes
  DEVICETYPE=ovs
  TYPE=OVSBridge
  BOOTPROTO=static
  IPADDR=172.16.10.10
  GATEWAY=172.16.10.1
  NETMASK=255.255.255.0
  HOTPLUG=no
  
  /etc/sysconfig/network-scripts/ifcfg-cloudbr1
  DEVICE=cloudbr1
  ONBOOT=yes
  DEVICETYPE=ovs
  TYPE=OVSBridge
  BOOTPROTO=none
  HOTPLUG=no
  
  /etc/sysconfig/network
  NETWORKING=yes
  HOSTNAME=testkvm1
  GATEWAY=172.10.10.1
  

• Edit /etc/cloudstack/agent/agent.properties

  network.bridge.type=openvswitch
  libvirt.vif.driver=com.cloud.hypervisor.kvm.resource.OvsVifDriver
  

Enabling the service provider¶

The OVS provider is disabled by default. Navigate to the « Network Service Providers » configuration of the physical network with the GRE isolation type. Navigate to the OVS provider and press the « Enable Provider » button.

Network Offerings¶

Using the OVS plugin requires a network offering with Virtual Networking enabled and configured to use the OVS element. Typical use cases combine services from the Virtual Router appliance and the OVS plugin.

Table: Isolated network offering with regular services from the Virtual Router.

Isolated network with network services. The virtual router is still required to provide network services like dns and dhcp.

Table: Isolated network offering with network services

On Ubuntu 12.04¶

First update and upgrade your system:

apt-get update
apt-get upgrade

NTP might already be installed, check it with service ntp status. If it’s not then install NTP to synchronize the clocks:

apt-get install openntpd

Install openjdk. As we’re using Linux, OpenJDK is our first choice.

apt-get install openjdk-7-jdk

Install tomcat6, note that the new version of tomcat on Ubuntu is the 6.0.35 version.

apt-get install tomcat6

Next, we’ll install MySQL if it’s not already present on the system.

apt-get install mysql-server

Remember to set the correct mysql password in the CloudStack properties file. Mysql should be running but you can check it’s status with:

service mysql status

Developers wanting to build CloudStack from source will want to install the following additional packages. If you dont” want to build from source just jump to the next section.

Install git to later clone the CloudStack source code:

apt-get install git

Install Maven to later build CloudStack

apt-get install maven

This should have installed Maven 3.0, check the version number with mvn --version

A little bit of Python can be used (e.g simulator), install the Python package management tools:

apt-get install python-pip python-setuptools

Finally install mkisofs with:

apt-get install genisoimage

On CentOS 6.4¶

First update and upgrade your system:

yum -y update
yum -y upgrade

If not already installed, install NTP for clock synchornization

yum -y install ntp

Install openjdk. As we’re using Linux, OpenJDK is our first choice.

yum -y install java-1.7.0-openjdk-devel

Install tomcat6, note that the version of tomcat6 in the default CentOS 6.4 repo is 6.0.24, so we will grab the 6.0.35 version. The 6.0.24 version will be installed anyway as a dependency to cloudstack.

wget https://archive.apache.org/dist/tomcat/tomcat-6/v6.0.35/bin/apache-tomcat-6.0.35.tar.gz
tar xzvf apache-tomcat-6.0.35.tar.gz -C /usr/local

Setup tomcat6 system wide by creating a file /etc/profile.d/tomcat.sh with the following content:

export CATALINA_BASE=/usr/local/apache-tomcat-6.0.35
export CATALINA_HOME=/usr/local/apache-tomcat-6.0.35

Next, we’ll install MySQL if it’s not already present on the system.

yum -y install mysql mysql-server

Remember to set the correct mysql password in the CloudStack properties file. Mysql should be running but you can check it’s status with:

service mysqld status

Install git to later clone the CloudStack source code:

yum -y install git

Install Maven to later build CloudStack. Grab the 3.0.5 release from the Maven website

wget http://mirror.cc.columbia.edu/pub/software/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
tar xzf apache-maven-3.0.5-bin.tar.gz -C /usr/local
cd /usr/local
ln -s apache-maven-3.0.5 maven

Setup Maven system wide by creating a /etc/profile.d/maven.sh file with the following content:

export M2_HOME=/usr/local/maven
export PATH=${M2_HOME}/bin:${PATH}

Log out and log in again and you will have maven in your PATH:

mvn --version

This should have installed Maven 3.0, check the version number with mvn --version

A little bit of Python can be used (e.g simulator), install the Python package management tools:

yum -y install python-setuptools

To install python-pip you might want to setup the Extra Packages for Enterprise Linux (EPEL) repo

cd /tmp
wget http://mirror-fpt-telecom.fpt.net/fedora/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm

Then update you repository cache yum update and install pip yum -y install python-pip

Finally install mkisofs with:

yum -y install genisoimage

Adding DevCloud as an Hypervisor¶

Picking up from a clean build:

mvn -Pdeveloper,systemvm clean install
mvn -P developer -pl developer,tools/devcloud -Ddeploydb

At this stage install marvin similarly than with the simulator:

pip install tools/marvin/dist/Marvin-0.1.0.tar.gz

Start the management server

mvn -pl client jetty:run

Then you are going to configure CloudStack to use the running DevCloud instance:

cd tools/devcloud
python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg

If you are curious, check the devcloud.cfg file and see how the data center is defined: 1 Zone, 1 Pod, 1 Cluster, 1 Host, 1 primary Storage, 1 Seondary Storage, all provided by Devcloud.

You can now log in the management server at http://localhost:8080/client and start experimenting with the UI or the API.

Do note that the management server is running in your local machine and that DevCloud is used only as a n Hypervisor. You could potentially run the management server within DevCloud as well, or memory granted, run multiple DevClouds.

Getting Started¶

To get started using the CloudStack API, you should have the following:

• URL of the CloudStack server you wish to integrate with.
• Both the API Key and Secret Key for an account. This should have been generated by the administrator of the cloud instance and given to you.
• Familiarity with HTTP GET/POST and query strings.
• Knowledge of either XML or JSON.
• Knowledge of a programming language that can generate HTTP requests; for example, Java or PHP.

Roles¶

The CloudStack API supports three access roles:

1. Root Admin. Access to all features of the cloud, including both virtual and physical resource management.
2. Domain Admin. Access to only the virtual resources of the clouds that belong to the administrator’s domain.
3. User. Access to only the features that allow management of the user’s virtual instances, storage, and network.

API Reference Documentation¶

You can find all the API reference documentation at the below site:

http://cloudstack.apache.org/docs/api/

Making API Requests¶

All CloudStack API requests are submitted in the form of a HTTP GET/POST with an associated command and any parameters. A request is composed of the following whether in HTTP or HTTPS:

• CloudStack API URL: This is the web services API entry point(for example, http://www.example.com:8080/client/api)
• Command: The web services command you wish to execute, such as start a virtual machine or create a disk volume
• Parameters: Any additional required or optional parameters for the command

A sample API GET request looks like the following:

http://localhost:8080/client/api?command=deployVirtualMachine&serviceOfferingId=1&diskOfferingId=1&templateId=2&zoneId=4&apiKey=miVr6X7u6bN_sdahOBpjNejPgEsT35eXq-jB8CG20YI3yaxXcgpyuaIRmFI_EJTVwZ0nUkkJbPmY3y2bciKwFQ&signature=Lxx1DM40AjcXU%2FcaiK8RAP0O1hU%3D

Or in a more readable format:

1. http://localhost:8080/client/api
2. ?command=deployVirtualMachine
3. &serviceOfferingId=1
4. &diskOfferingId=1
5. &templateId=2
6. &zoneId=4
7. &apiKey=miVr6X7u6bN_sdahOBpjNejPgEsT35eXqjB8CG20YI3yaxXcgpyuaIRmFI_EJTVwZ0nUkkJbPmY3y2bciKwFQ
8. &signature=Lxx1DM40AjcXU%2FcaiK8RAP0O1hU%3D

The first line is the CloudStack API URL. This is the Cloud instance you wish to interact with.

The second line refers to the command you wish to execute. In our example, we are attempting to deploy a fresh new virtual machine. It is preceded by a (?) to separate itself from the CloudStack API URL.

Lines 3-6 are the parameters for this given command. To see the command and its request parameters, please refer to the appropriate section in the CloudStack API documentation. Each parameter field-value pair (field=value) is preceded by an ampersand character (&).

Line 7 is the user API Key that uniquely identifies the account. See Signing API Requests on page 7.

Line 8 is the signature hash created to authenticate the user account executing the API command.

Signing API Requests¶

Whether you access the CloudStack API with HTTP or HTTPS, it must still be signed so that CloudStack can verify the caller has been authenticated and authorized to execute the command. Make sure that you have both the API Key and Secret Key provided by the CloudStack administrator for your account before proceeding with the signing process.

To show how to sign a request, we will re-use the previous example.

http://http://localhost:8080/client/api?command=deployVirtualMachine&serviceOfferingId=1&diskOfferingId=1&templateId=2&zoneId=4&apiKey=miVr6X7u6bN_sdahOBpjNejPgEsT35eXq-jB8CG20YI3yaxXcgpyuaIRmFI_EJTVwZ0nUkkJbPmY3y2bciKwFQ&signature=Lxx1DM40AjcXU%2FcaiK8RAP0O1hU%3D

Breaking this down, we have several distinct parts to this URL.

• Base URL: This is the base URL to the CloudStack Management Server.

  http://localhost:8080
  

• API Path: This is the path to the API Servlet that processes the incoming requests.

  /client/api?
  

• Command String: This part of the query string comprises of the command, its parameters, and the API Key that identifies the account.

  Note

  As with all query string parameters of field-value pairs, the « field » component is case insensitive while all « value » values are case sensitive.

• Signature: This is the signature of the command string that is generated using a combination of the user’s Secret Key and the HMAC SHA-1 hashing algorithm.

  &signature=Lxx1DM40AjcXU%2FcaiK8RAP0O1hU%3D
  

Every API request has the format Base URL+API Path+Command String+Signature.

To generate the signature.

1. For each field-value pair (as separated by a “&”) in the Command String, URL encode each value so that it can be safely sent via HTTP GET.

   Note

   Make sure all spaces are encoded as « %20 » rather than « + ».

2. Lower case the entire Command String and sort it alphabetically via the field for each field-value pair. The result of this step would look like the following.

   apikey=mivr6x7u6bn_sdahobpjnejpgest35exq-jb8cg20yi3yaxxcgpyuairmfi_ejtvwz0nukkjbpmy3y2bcikwfq&command=deployvirtualmachine&diskofferingid=1&serviceofferingid=1&templateid=2&zoneid=4
   

3. Take the sorted Command String and run it through the HMAC SHA-1 hashing algorithm (most programming languages offer a utility method to do this) with the user’s Secret Key. Base64 encode the resulting byte array in UTF-8 so that it can be safely transmitted via HTTP. The final string produced after Base64 encoding should be « Lxx1DM40AjcXU%2FcaiK8RAP0O1hU%3D ».

   By reconstructing the final URL in the format Base URL+API Path+Command String+Signature, the final URL should look like:

   http://localhost:8080/client/api?command=deployVirtualMachine&serviceOfferingId=1&diskOfferingId=1&templateId=2&zoneId=4&apiKey=miVr6X7u6bN_sdahOBpjNejPgEsT35eXq-jB8CG20YI3yaxXcgpyuaIRmFI_EJTVwZ0nUkkJbPmY3y2bciKwFQ&signature=Lxx1DM40AjcXU%2FcaiK8RAP0O1hU%3D
   

How to sign an API call with Python¶

To illustrate the procedure used to sign API calls we present a step by step interactive session using Python.

First import the required modules:

$python
Python 2.7.3 (default, Nov 17 2012, 19:54:34)
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> import urllib
>>> import hashlib
>>> import hmac
>>> import base64

Define the endpoint of the Cloud, the command that you want to execute and the keys of the user.

>>> baseurl='http://localhost:8080/client/api?'
>>> request={}
>>> request['command']='listUsers'
>>> request['response']='json'
>>> request['apikey']='plgWJfZK4gyS3mOMTVmjUVg-X-jlWlnfaUJ9GAbBbf9EdM-kAYMmAiLqzzq1ElZLYq_u38zCm0bewzGUdP66mg'
>>> secretkey='VDaACYb0LV9eNjTetIOElcVQkvJck_J_QljX_FcHRj87ZKiy0z0ty0ZsYBkoXkY9b7eq1EhwJaw7FF3akA3KBQ'

Build the request string:

>>> request_str='&'.join(['='.join([k,urllib.quote_plus(request[k])]) for k in request.keys()])
>>> request_str
'apikey=plgWJfZK4gyS3mOMTVmjUVg-X-jlWlnfaUJ9GAbBbf9EdM-kAYMmAiLqzzq1ElZLYq_u38zCm0bewzGUdP66mg&command=listUsers&response=json'

Compute the signature with hmac, do a 64 bit encoding and a url encoding:

>>> sig_str='&'.join(['='.join([k.lower(),urllib.quote_plus(request[k].lower().replace('+','%20'))])for k in sorted(request.iterkeys())])
>>> sig_str 'apikey=plgwjfzk4gys3momtvmjuvg-x-jlwlnfauj9gabbbf9edm-kaymmailqzzq1elzlyq_u38zcm0bewzgudp66mg&command=listusers&response=json'
>>> sig=hmac.new(secretkey,sig_str,hashlib.sha1)
>>> sig
<hmac.HMAC instance at 0x10d91d680>
>>> sig=hmac.new(secretkey,sig_str,hashlib.sha1).digest()
>>> sig
'M:]\x0e\xaf\xfb\x8f\xf2y\xf1p\x91\x1e\x89\x8a\xa1\x05\xc4A\xdb'
>>> sig=base64.encodestring(hmac.new(secretkey,sig_str,hashlib.sha1).digest())
>>> sig
'TTpdDq/7j/J58XCRHomKoQXEQds=\n'
>>> sig=base64.encodestring(hmac.new(secretkey,sig_str,hashlib.sha1).digest()).strip()
>>> sig
'TTpdDq/7j/J58XCRHomKoQXEQds='
>>> sig=urllib.quote_plus(base64.encodestring(hmac.new(secretkey,sig_str,hashlib.sha1).digest()).strip())

Finally, build the entire string and do an http GET:

>>> req=baseurl+request_str+'&signature='+sig
>>> req
'http://localhost:8080/client/api?apikey=plgWJfZK4gyS3mOMTVmjUVg-X-jlWlnfaUJ9GAbBbf9EdM-kAYMmAiLqzzq1ElZLYq_u38zCm0bewzGUdP66mg&command=listUsers&response=json&signature=TTpdDq%2F7j%2FJ58XCRHomKoQXEQds%3D'
>>> res=urllib2.urlopen(req)
>>> res.read()
'{ "listusersresponse" : { "count":3 ,"user" : [  {"id":"7ed6d5da-93b2-4545-a502-23d20b48ef2a","username":"admin","firstname":"admin","lastname":"cloud","created":"2012-07-05T12:18:27-0700","state":"enabled","account":"admin","accounttype":1,"domainid":"8a111e58-e155-4482-93ce-84efff3c7c77","domain":"ROOT","apikey":"plgWJfZK4gyS3mOMTVmjUVg-X-jlWlnfaUJ9GAbBbf9EdM-kAYMmAiLqzzq1ElZLYq_u38zCm0bewzGUdP66mg","secretkey":"VDaACYb0LV9eNjTetIOElcVQkvJck_J_QljX_FcHRj87ZKiy0z0ty0ZsYBkoXkY9b7eq1EhwJaw7FF3akA3KBQ","accountid":"7548ac03-af1d-4c1c-9064-2f3e2c0eda0d"}, {"id":"1fea6418-5576-4989-a21e-4790787bbee3","username":"runseb","firstname":"foobar","lastname":"goa","email":"joe@smith.com","created":"2013-04-10T16:52:06-0700","state":"enabled","account":"admin","accounttype":1,"domainid":"8a111e58-e155-4482-93ce-84efff3c7c77","domain":"ROOT","apikey":"Xhsb3MewjJQaXXMszRcLvQI9_NPy_UcbDj1QXikkVbDC9MDSPwWdtZ1bUY1H7JBEYTtDDLY3yuchCeW778GkBA","secretkey":"gIsgmi8C5YwxMHjX5o51pSe0kqs6JnKriw0jJBLceY5bgnfzKjL4aM6ctJX-i1ddQIHJLbLJDK9MRzsKk6xZ_w","accountid":"7548ac03-af1d-4c1c-9064-2f3e2c0eda0d"}, {"id":"52f65396-183c-4473-883f-a37e7bb93967","username":"toto","firstname":"john","lastname":"smith","email":"john@smith.com","created":"2013-04-23T04:27:22-0700","state":"enabled","account":"admin","accounttype":1,"domainid":"8a111e58-e155-4482-93ce-84efff3c7c77","domain":"ROOT","apikey":"THaA6fFWS_OmvU8od201omxFC8yKNL_Hc5ZCS77LFCJsRzSx48JyZucbUul6XYbEg-ZyXMl_wuEpECzK-wKnow","secretkey":"O5ywpqJorAsEBKR_5jEvrtGHfWL1Y_j1E4Z_iCr8OKCYcsPIOdVcfzjJQ8YqK0a5EzSpoRrjOFiLsG0hQrYnDA","accountid":"7548ac03-af1d-4c1c-9064-2f3e2c0eda0d"} ] } }'

Enabling API Call Expiration¶

You can set an expiry timestamp on API calls to prevent replay attacks over non-secure channels, such as HTTP. The server tracks the expiry timestamp you have specified and rejects all the subsequent API requests that come in after this validity period.

To enable this feature, add the following parameters to the API request:

• signatureVersion=3: If the signatureVersion parameter is missing or is not equal to 3, the expires parameter is ignored in the API request.
• expires=YYYY-MM-DDThh:mm:ssZ: Specifies the date and time at which the signature included in the request is expired. The timestamp is expressed in the YYYY-MM-DDThh:mm:ssZ format, as specified in the ISO 8601 standard.

For example:

expires=2011-10-10T12:00:00+0530

A sample API request with expiration is given below:

http://<IPAddress>:8080/client/api?command=listZones&signatureVersion=3&expires=2011-10-10T12:00:00+0530&apiKey=miVr6X7u6bN_sdahOBpjNejPgEsT35eXq-jB8CG20YI3yaxXcgpyuaIRmFI_EJTVwZ0nUkkJbPmY3y2bciKwFQ&signature=Lxx1DM40AjcXU%2FcaiK8RAP0O1hU%3D

Limiting the Rate of API Requests¶

You can limit the rate at which API requests can be placed for each account. This is useful to avoid malicious attacks on the Management Server, prevent performance degradation, and provide fairness to all accounts.

If the number of API calls exceeds the threshold, an error message is returned for any additional API calls. The caller will have to retry these API calls at another time.

Configuring the API Request Rate¶

To control the API request rate, use the following global configuration settings:

• api.throttling.enabled - Enable/Disable API throttling. By default, this setting is false, so API throttling is not enabled.
• api.throttling.interval (in seconds) - Time interval during which the number of API requests is to be counted. When the interval has passed, the API count is reset to 0.
• api.throttling.max - Maximum number of APIs that can be placed within the api.throttling.interval period.
• api.throttling.cachesize - Cache size for storing API counters. Use a value higher than the total number of accounts managed by the cloud. One cache entry is needed for each account, to store the running API total for that account.

Limitations on API Throttling¶

The following limitations exist in the current implementation of this feature.

• In a deployment with multiple Management Servers, the cache is not synchronized across them. In this case, CloudStack might not be able to ensure that only the exact desired number of API requests are allowed. In the worst case, the number of API calls that might be allowed is (number of Management Servers) * (api.throttling.max).
• The API commands resetApiLimit and getApiLimit are limited to the Management Server where the API is invoked.

API Responses¶

<listipaddressesresponse>
   <allocatedipaddress>
   <ipaddress>192.168.10.141</ipaddress>
   <allocated>2009-09-18T13:16:10-0700</allocated>
   <zoneid>4</zoneid>
      <zonename>WC</zonename>
      <issourcenat>true</issourcenat>
   </allocatedipaddress>
</listipaddressesresponse>

{ "listipaddressesresponse" :
  { "allocatedipaddress" :
    [
      {
        "ipaddress" : "192.168.10.141",
        "allocated" : "2009-09-18T13:16:10-0700",
        "zoneid" : "4",
        "zonename" : "WC",
        "issourcenat" : "true"
      }
    ]
  }
}

Maximum Result Pages Returned¶

For each cloud, there is a default upper limit on the number of results that any API command will return in a single page. This is to help prevent overloading the cloud servers and prevent DOS attacks. For example, if the page size limit is 500 and a command returns 10,000 results, the command will return 20 pages.

The default page size limit can be different for each cloud. It is set in the global configuration parameter default.page.size. If your cloud has many users with lots of VMs, you might need to increase the value of this parameter. At the same time, be careful not to set it so high that your site can be taken down by an enormous return from an API call. For more information about how to set global configuration parameters, see « Describe Your Deployment » in the Installation Guide.

To decrease the page size limit for an individual API command, override the global setting with the page and pagesize parameters, which are available in any list* command (listCapabilities, listDiskOfferings, etc.).

• Both parameters must be specified together.
• The value of the pagesize parameter must be smaller than the value of default.page.size. That is, you can not increase the number of possible items in a result page, only decrease it.

For syntax information on the list* commands, see the API Reference.

Error Handling¶

If an error occurs while processing an API request, the appropriate response in the format specified is returned. Each error response consists of an error code and an error text describing what possibly can go wrong. Below is a list of possible error codes:

You can now find the CloudStack-specific error code in the exception response for each type of exception. The following list of error codes is added to the new class named CSExceptionErrorCode.

4250 : « com.cloud.utils.exception.CloudRuntimeException »

4255 : « com.cloud.utils.exception.ExceptionUtil »

4260 : « com.cloud.utils.exception.ExecutionException »

4265 : « com.cloud.utils.exception.HypervisorVersionChangedException »

4270 : « com.cloud.utils.exception.RuntimeCloudException »

4275 : « com.cloud.exception.CloudException »

4280 : « com.cloud.exception.AccountLimitException »

4285 : « com.cloud.exception.AgentUnavailableException »

4290 : « com.cloud.exception.CloudAuthenticationException »

4295 : « com.cloud.exception.CloudExecutionException »

4300 : « com.cloud.exception.ConcurrentOperationException »

4305 : « com.cloud.exception.ConflictingNetworkSettingsException »

4310 : « com.cloud.exception.DiscoveredWithErrorException »

4315 : « com.cloud.exception.HAStateException »

4320 : « com.cloud.exception.InsufficientAddressCapacityException »

4325 : « com.cloud.exception.InsufficientCapacityException »

4330 : « com.cloud.exception.InsufficientNetworkCapacityException »

4335 : « com.cloud.exception.InsufficientServerCapacityException »

4340 : « com.cloud.exception.InsufficientStorageCapacityException »

4345 : « com.cloud.exception.InternalErrorException »

4350 : « com.cloud.exception.InvalidParameterValueException »

4355 : « com.cloud.exception.ManagementServerException »

4360 : « com.cloud.exception.NetworkRuleConflictException »

4365 : « com.cloud.exception.PermissionDeniedException »

4370 : « com.cloud.exception.ResourceAllocationException »

4375 : « com.cloud.exception.ResourceInUseException »

4380 : « com.cloud.exception.ResourceUnavailableException »

4385 : « com.cloud.exception.StorageUnavailableException »

4390 : « com.cloud.exception.UnsupportedServiceException »

4395 : « com.cloud.exception.VirtualMachineMigrationException »

4400 : « com.cloud.exception.AccountLimitException »

4405 : « com.cloud.exception.AgentUnavailableException »

4410 : « com.cloud.exception.CloudAuthenticationException »

4415 : « com.cloud.exception.CloudException »

4420 : « com.cloud.exception.CloudExecutionException »

4425 : « com.cloud.exception.ConcurrentOperationException »

4430 : « com.cloud.exception.ConflictingNetworkSettingsException »

4435 : « com.cloud.exception.ConnectionException »

4440 : « com.cloud.exception.DiscoveredWithErrorException »

4445 : « com.cloud.exception.DiscoveryException »

4450 : « com.cloud.exception.HAStateException »

4455 : « com.cloud.exception.InsufficientAddressCapacityException »

4460 : « com.cloud.exception.InsufficientCapacityException »

4465 : « com.cloud.exception.InsufficientNetworkCapacityException »

4470 : « com.cloud.exception.InsufficientServerCapacityException »

4475 : « com.cloud.exception.InsufficientStorageCapacityException »

4480 : « com.cloud.exception.InsufficientVirtualNetworkCapcityException »

4485 : « com.cloud.exception.InternalErrorException »

4490 : « com.cloud.exception.InvalidParameterValueException »

4495 : « com.cloud.exception.ManagementServerException »

4500 : « com.cloud.exception.NetworkRuleConflictException »

4505 : « com.cloud.exception.PermissionDeniedException »

4510 : « com.cloud.exception.ResourceAllocationException »

4515 : « com.cloud.exception.ResourceInUseException »

4520 : « com.cloud.exception.ResourceUnavailableException »

4525 : « com.cloud.exception.StorageUnavailableException »

4530 : « com.cloud.exception.UnsupportedServiceException »

4535 : « com.cloud.exception.VirtualMachineMigrationException »

9999 : « org.apache.cloudstack.api.ServerApiException »

An HTTP error code of 401 is always returned if API request was rejected due to bad signatures, missing API Keys, or the user simply did not have the permissions to execute the command.

Asynchronous Commands¶

Asynchronous commands were introduced in CloudStack 2.x. Commands are designated as asynchronous when they can potentially take a long period of time to complete such as creating a snapshot or disk volume. They differ from synchronous commands by the following:

• They are identified in the API Reference by an (A).

• They will immediately return a job ID to refer to the job that will be responsible in processing the command.

• If executed as a « create » resource command, it will return the resource ID as well as the job ID.

  You can periodically check the status of the job by making a simple API call to the command, queryAsyncJobResult and passing in the job ID.

Job Status¶

The key to using an asynchronous command is the job ID that is returned immediately once the command has been executed. With the job ID, you can periodically check the job status by making calls to queryAsyncJobResult command. The command will return three possible job status integer values:

• 0 - Job is still in progress. Continue to periodically poll for any status changes.
• 1 - Job has successfully completed. The job will return any successful response values associated with command that was originally executed.
• 2 - Job has failed to complete. Please check the « jobresultcode » tag for failure reason code and « jobresult » for the failure reason.

Example¶

The following shows an example of using an asynchronous command. Assume the API command:

command=deployVirtualMachine&zoneId=1&serviceOfferingId=1&diskOfferingId=1&templateId=1

CloudStack will immediately return a job ID and any other additional data.

<deployvirtualmachineresponse>
   <jobid>1</jobid>
   <id>100</id>
</deployvirtualmachineresponse>

Using the job ID, you can periodically poll for the results by using the queryAsyncJobResult command.

command=queryAsyncJobResult&jobId=1

Three possible results could come from this query.

Job is still pending:

<queryasyncjobresult>
   <jobid>1</jobid>
   <jobstatus>0</jobstatus>
   <jobprocstatus>1</jobprocstatus>
</queryasyncjobresult>

Job has succeeded:

<queryasyncjobresultresponse cloud-stack-version="3.0.1.6">
   <jobid>1</jobid>
   <jobstatus>1</jobstatus>
   <jobprocstatus>0</jobprocstatus>
   <jobresultcode>0</jobresultcode>
   <jobresulttype>object</jobresulttype>
   <jobresult>
      <virtualmachine>
         <id>450</id>
         <name>i-2-450-VM</name>
         <displayname>i-2-450-VM</displayname>
         <account>admin</account>
         <domainid>1</domainid>
         <domain>ROOT</domain>
         <created>2011-03-10T18:20:25-0800</created>
         <state>Running</state>
         <haenable>false</haenable>
         <zoneid>1</zoneid>
         <zonename>San Jose 1</zonename>
         <hostid>2</hostid>
         <hostname>905-13.sjc.lab.vmops.com</hostname>
         <templateid>1</templateid>
         <templatename>CentOS 5.3 64bit LAMP</templatename>
         <templatedisplaytext>CentOS 5.3 64bit LAMP</templatedisplaytext>
         <passwordenabled>false</passwordenabled>
         <serviceofferingid>1</serviceofferingid>
         <serviceofferingname>Small Instance</serviceofferingname>
         <cpunumber>1</cpunumber>
         <cpuspeed>500</cpuspeed>
         <memory>512</memory>
         <guestosid>12</guestosid>
         <rootdeviceid>0</rootdeviceid>
         <rootdevicetype>NetworkFilesystem</rootdevicetype>
         <nic>
            <id>561</id>
            <networkid>205</networkid>
            <netmask>255.255.255.0</netmask>
            <gateway>10.1.1.1</gateway>
            <ipaddress>10.1.1.225</ipaddress>
            <isolationuri>vlan://295</isolationuri>
            <broadcasturi>vlan://295</broadcasturi>
            <traffictype>Guest</traffictype>
            <type>Virtual</type>
            <isdefault>true</isdefault>
         </nic>
         <hypervisor>XenServer</hypervisor>
      </virtualmachine>
   </jobresult>
</queryasyncjobresultresponse>

Job has failed:

<queryasyncjobresult>
   <jobid>1</jobid>
   <jobstatus>2</jobstatus>
   <jobprocstatus>0</jobprocstatus>
   <jobresultcode>551</jobresultcode>
   <jobresulttype>text</jobresulttype>
   <jobresult>Unable to deploy virtual machine id = 100 due to not enough capacity</jobresult>
</queryasyncjobresult>

Overview of How to Write a Storage Plugin¶

To add a third-party storage option to CloudStack, follow these general steps (explained in more detail later in this section):

1. Implement the following interfaces in Java:
   • DataStoreDriver
   • DataStoreLifecycle
   • DataStoreProvider
   • VMSnapshotStrategy (if you want to customize the VM snapshot functionality)
2. Hardcode your plugin’s required additional input fields into the code for the Add Secondary Storage or Add Primary Storage dialog box.
3. Place your .jar file in plugins/storage/volume/ or plugins/storage/image/.
4. Edit /client/tomcatconf/componentContext.xml.in.
5. Edit client/pom.xml.

Implementing DataStoreDriver¶

DataStoreDriver contains the code that CloudStack will use to provision the object store, when needed.

You must implement the following methods:

• getTO()
• getStoreTO()
• createAsync()
• deleteAsync()

The following methods are optional:

• resize()
• canCopy() is optional. If you set it to true, then you must implement copyAsync().

Implementing DataStoreLifecycle¶

DataStoreLifecycle contains the code to manage the storage operations for ongoing use of the storage. Several operations are needed, like create, maintenance mode, delete, etc.

You must implement the following methods:

• initialize()
• maintain()
• cancelMaintain()
• deleteDataStore()
• Implement one of the attach*() methods depending on what scope you want the storage to have: attachHost(), attachCluster(), or attachZone().

Implementing DataStoreProvider¶

DataStoreProvider contains the main code of the data store.

You must implement the following methods:

• getDatastoreLifeCycle()
• getDataStoreDriver()
• getTypes(). Returns one or more types of storage for which this data store provider can be used. For secondary object storage, return IMAGE, and for a Secondary Staging Store, return ImageCache.
• configure(). First initialize the lifecycle implementation and the driver implementation, then call registerDriver() to register the new object store provider instance with CloudStack.
• getName(). Returns the unique name of your provider; for example, this can be used to get the name to display in the UI.

The following methods are optional:

• getHostListener() is optional; it’s for monitoring the status of the host.

Implementing VMSnapshotStrategy¶

VMSnapshotStrategy has the following methods:

• takeVMSnapshot()
• deleteVMSnapshot()
• revertVMSnapshot()
• canHandle(). For a given VM snapshot, tells whether this implementation of VMSnapshotStrategy can handle it.

Place the .jar File in the Right Directory¶

For a secondary storage plugin, place your .jar file here:

plugins/storage/image/

For a primary storage plugin, place your .jar file here:

plugins/storage/volume/

Edit Configuration Files¶

First, edit the following file tell CloudStack to include your .jar file. Add a line to this file to tell the CloudStack Management Server that it now has a dependency on your code:

client/pom.xml

Place some facts about your code in the following file so CloudStack can run it:

/client/tomcatconf/componentContext.xml.in

In the section “Deployment configurations of various adapters,” add this:

<bean>id=”some unique ID” class=”package name of your implementation of DataStoreProvider”</bean>

In the section “Storage Providers,” add this:

<property name=”providers”>
   <ref local=”same ID from the bean tag's id attribute”>
</property>

Minimum Required Interfaces¶

The classes, interfaces, and methods used by CloudStack from the Amazon Web Services (AWS) Java SDK are listed in this section. An object storage that supports the S3 interface is minimally required to support the below in order to be compatible with CloudStack.

How to Write a Plugin: Overview¶

The basic procedure for writing a plugin is:

1. Write the code and create the other files needed. You will need the plugin code itself (in Javascript), a thumbnail image, the plugin listing, and a CSS file.

   

   All UI plugins have the following set of files:

   +-- cloudstack/
     +-- ui/
       +-- plugins/
         +-- csMyFirstPlugin/
           +-- config.js            --> Plugin metadata (title, author, vendor URL, etc.)
           +-- icon.png             --> Icon, shown on side nav bar and plugin listing
                                        (should be square, and ~50x50px)
           +-- csMyFirstPlugin.css  --> CSS file, loaded automatically when plugin loads
           +-- csMyFirstPlugin.js   --> Main JS file, containing plugin code
   

   The same files must also be present at /tomcat/webapps/client/plugins.

2. The CloudStack administrator adds the folder containing your plugin code under the CloudStack PLUGINS folder.

   

3. The administrator also adds the name of your plugin to the plugin.js file in the PLUGINS folder.

   

4. The next time the user refreshes the UI in the browser, your plugin will appear in the left navigation bar.

   

How to Write a Plugin: Implementation Details¶

This section requires an understanding of JavaScript and the CloudStack API. You don’t need knowledge of specific frameworks for this tutorial (jQuery, etc.), since the CloudStack UI handles the front-end rendering for you.

There is much more to the CloudStack UI framework than can be described here. The UI is very flexible to handle many use cases, so there are countless options and variations. The best reference right now is to read the existing code for the main UI, which is in the /ui folder. Plugins are written in a very similar way to the main UI.

1. Create the directory to hold your plugin.

   All plugins are composed of set of required files in the directory /ui/plugins/pluginID, where pluginID is a short name for your plugin. It’s recommended that you prefix your folder name (for example, bfMyPlugin) to avoid naming conflicts with other people’s plugins.

   In this example, the plugin is named csMyFirstPlugin.

   $ cd cloudstack/ui/plugins
   $ mkdir csMyFirstPlugin
   $ ls -l
   
   total 8
   drwxr-xr-x  2 bgregory  staff   68 Feb 11 14:44 csMyFirstPlugin
   -rw-r--r--  1 bgregory  staff  101 Feb 11 14:26 plugins.js
   

2. Change to your new plugin directory.

   $ cd csMyFirstPlugin
   

3. Set up the listing.

   Add the file config.js, using your favorite editor.

   $ vi config.js
   

   Add the following content to config.js. This information will be displayed on the plugin listing page in the UI:

   (function (cloudStack) {
     cloudStack.plugins.csMyFirstPlugin.config = {
       title: 'My first plugin',
       desc: 'Tutorial plugin',
       externalLink: 'http://www.cloudstack.org/',
       authorName: 'Test Plugin Developer',
       authorEmail: 'plugin.developer@example.com'
     };
   }(cloudStack));
   

4. Add a new main section.

   Add the file csMyFirstPlugin.js, using your favorite editor.

   $ vi csMyFirstPlugin.js
   

   Add the following content to csMyFirstPlugin.js:

   (function (cloudStack) {
     cloudStack.plugins.csMyFirstPlugin = function(plugin) {
       plugin.ui.addSection({
         id: 'csMyFirstPlugin',
         title: 'My Plugin',
         preFilter: function(args) {
           return isAdmin();
         },
         show: function() {
           return $('<div>').html('Content will go here');
         }
       });
     };
   }(cloudStack));
   

5. Register the plugin.

   You now have the minimal content needed to run the plugin, so you can activate the plugin in the UI by adding it to plugins.js. First, edit the file:

   $ cd cloudstack/ui/plugins
   $ vi plugins.js
   

   Now add the following to plugins.js:

   (function($, cloudStack) {
     cloudStack.plugins = [
       'csMyFirstPlugin'
     ];
   }(jQuery, cloudStack));
   

6. Check the plugin in the UI.

   First, copy all the plugin code that you have created so far to /tomcat/webapps/client/plugins. Then refresh the browser and click Plugins in the side navigation bar. You should see your new plugin.

7. Make the plugin do something.

   Right now, you just have placeholder content in the new plugin. It’s time to add real code. In this example, you will write a basic list view, which renders data from an API call. You will list all virtual machines owned by the logged-in user. To do this, replace the “show” function in the plugin code with a “listView” block, containing the required syntax for a list view. To get the data, use the listVirtualMachines API call. Without any parameters, it will return VMs only for your active user. Use the provided “apiCall” helper method to handle the server call. Of course, you are free to use any other method for making the AJAX call (for example, jQuery’s $.ajax method).

   First, open your plugin’s JavaScript source file in your favorite editor:

   $ cd csMyFirstPlugin
   $ vi csMyFirstPlugin.js
   

   Add the following code in csMyFirstPlugin.js:

   (function (cloudStack) {
     cloudStack.plugins.csMyFirstPlugin = function(plugin) {
       plugin.ui.addSection({
         id: 'csMyFirstPlugin',
         title: 'My Plugin',
         preFilter: function(args) {
           return isAdmin();
         },
   
         // Render page as a list view
         listView: {
           id: 'testPluginInstances',
           fields: {
             name: { label: 'label.name' },
             instancename: { label: 'label.internal.name' },
             displayname: { label: 'label.display.name' },
             zonename: { label: 'label.zone.name' }
           },
           dataProvider: function(args) {
             // API calls go here, to retrive the data asynchronously
             //
             // On successful retrieval, call
             // args.response.success({ data: [data array] });
             plugin.ui.apiCall('listVirtualMachines', {
               success: function(json) {
                 var vms = json.listvirtualmachinesresponse.virtualmachine;
   
                 args.response.success({ data: vms });
               },
               error: function(errorMessage) {
                 args.response.error(errorMessage)
               }
             });
           }
         }
       });
     };
   }(cloudStack));
   

8. Test the plugin.

   First, copy all the plugin code that you have created so far to /tomcat/webapps/client/plugins. Then refresh the browser. You can see that your placeholder content was replaced with a list table, containing 4 columns of virtual machine data.

9. Add an action button.

   Let’s add an action button to the list view, which will reboot the VM. To do this, add an actions block under listView. After specifying the correct format, the actions will appear automatically to the right of each row of data.

   $ vi csMyFirstPlugin.js
   

   Now add the following new code in csMyFirstPlugin.js. (The dots … show where we have omitted some existing code for the sake of space. Don’t actually cut and paste that part):

   ...
      listView: {
        id: 'testPluginInstances',
        ...
   
        actions: {
          // The key/ID you specify here will determine what icon is
          // shown in the UI for this action,
          // and will be added as a CSS class to the action's element
          // (i.e., '.action.restart')
          //
          // -- here, 'restart' is a predefined name in CloudStack that will
          // automatically show a 'reboot' arrow as an icon;
          // this can be changed in csMyFirstPlugin.css
          restart: {
            label: 'Restart VM',
            messages: {
              confirm: function() { return 'Are you sure you want to restart this VM?' },
              notification: function() { return 'Rebooted VM' }
            },
            action: function(args) {
              // Get the instance object of the selected row from context
              //
              // -- all currently loaded state is stored in 'context' as objects,
              //    such as the selected list view row,
              //    the selected section, and active user
              //
              // -- for list view actions, the object's key will be the same as
              //    listView.id, specified above;
              //    always make sure you specify an 'id' for the listView,
              //     or else it will be 'undefined!'
              var instance = args.context.testPluginInstances[0];
   
              plugin.ui.apiCall('rebootVirtualMachine', {
                // These will be appended to the API request
                //
                // i.e., rebootVirtualMachine&id=...
                data: {
                  id: instance.id
                },
                success: function(json) {
                  args.response.success({
                    // This is an async job, so success here only indicates
                    // that the job was initiated.
                    //
                    // To pass the job ID to the notification UI
                    // (for checking to see when action is completed),
                    // '_custom: { jobID: ... }' needs to always be passed on success,
                    // in the same format as below
                    _custom: { jobId: json.rebootvirtualmachineresponse.jobid }
                  });
                },
   
   
                error: function(errorMessage) {
                  args.response.error(errorMessage); // Cancel action, show error message returned
                }
              });
            },
   
            // Because rebootVirtualMachine is an async job, we need to add
            // a poll function, which will perodically check
            // the management server to see if the job is ready
            // (via pollAsyncJobResult API call)
            //
            // The plugin API provides a helper function, 'plugin.ui.pollAsyncJob',
            /  which will work for most jobs
            // in CloudStack
            notification: {
              poll: plugin.ui.pollAsyncJob
            }
          }
        },
   
        dataProvider: function(args) {
          ...
   ...
   

10. Add the thumbnail icon.

    Create an icon file; it should be square, about 50x50 pixels, and named icon.png. Copy it into the same directory with your plugin code: cloudstack/ui/plugins/csMyFirstPlugin/icon.png.

11. Add the stylesheet.

    Create a CSS file, with the same name as your .js file. Copy it into the same directory with your plugin code: cloudstack/ui/plugins/csMyFirstPlugin/csMyFirstPlugin.css.

HostAllocator Interface¶

The interface defines the following two methods.

/**
  * Checks if the VM can be upgraded to the specified ServiceOffering
  * @param UserVm vm
  * @param ServiceOffering offering
  * @return boolean true if the VM can be upgraded
**/

publicboolean isVirtualMachineUpgradable(final UserVm vm, final ServiceOffering offering);

/**
  * Determines which physical hosts are suitable to allocate the guest virtual machines on
  *
  * @paramVirtualMachineProfile vmProfile
  * @paramDeploymentPlan plan
  * @paramType type
  * @paramExcludeList avoid
  * @paramint returnUpTo
  * @returnList<Host>List of hosts that are suitable for VM allocation
**/

publicList<Host> allocateTo( VirtualMachineProfile<?extendsVirtualMachine> vmProfile,  DeploymentPlan plan, Type type, ExcludeList avoid, intreturnUpTo);

A custom HostAllocator can be written by implementing the ‘allocateTo’ method

StoragePoolAllocator Interface¶

A custom StoragePoolAllocator can be written by implementing the ‘allocateTo’ method.

/**
  * Determines which storage pools are suitable for the guest virtual machine
  * @param DiskProfile dskCh
  * @param VirtualMachineProfile vmProfile
  * @param DeploymentPlan plan
  * @param ExcludeList avoid
  * @param int returnUpTo
  * @return List<StoragePool> List of storage pools that are suitable for the VM
**/

public List<StoragePool> allocateToPool(DiskProfile dskCh, VirtualMachineProfile<? extends VirtualMachine> vm, DeploymentPlan plan, ExcludeList avoid, int returnUpTo);

This interface also contains some other methods to support some legacy code. However your custom allocator can extend the existing com.cloud.storage.allocator. AbstractStoragePoolAllocator. This class provides default implementation for all the other interface methods.