Eucalyptus Beginner’s Guide – UEC Edition

Table of Contents

1. Introduction to UEC and its components
2. Installation & Configuration
3. Web Interface
4. Image Management
5. Instance Management
6. Storage Management
7. Network Management
8. Security
9. Troubleshooting
10. Euca Commands
11. Hacks

This is a live book and will be updated on an ongoing basis based on your feedback and comments.

UEC: Bundling Windows instances on Lucid Lynx

Bundling Windows instances on UEC setup on Lucid Lynx (Ubuntu 10.04) was not straight forward as it was with Karmic Koala (Ubuntu 9.10).  The steps for bundling, uploading and registering were the same as I mentioned here. The instance comes up nicely from pending to running state, acquires the public and private ip addresses.

Lo!!! there ends the similarity….

I wasn’t able to access the instance. No network connectivity….. neither PING nor RDP works. And this was frustrating!!!

As mentioned in my earlier post about UEC/Eucalyptus – Debugging instances, I tweaked the /usr/share/gen_kvm_libvirt_xml file on the Node Controller and added the VNC display option.

The outcome of this was surprising!!!!

UEC: Windows instance on Lucid Lynx [HACK]

UEC/Eucalyptus – Debugging instances

I have had an issue with my MS-Windows 2003 server instance raised on one of the Node Controllers (NC) on my UEC setup on Lucid Lynx( Ubuntu 10.04 ). The instance came up moving from pending state to running state as expected and my hybrid fox showed public and private IPs. However, I was not able to ping or RDP into the instance. The Windows image was custom bundled and tried and tested on my Karmic UEC setup, so I was quite puzzled as to why I was unable to connect to the instance. I wished I had the luxury of connecting to the instance for troubleshooting using VNC, like I normally do when I start a VM on KVM or Xen. A little bit of probing of the Eucalyptus files on NC proved useful. Here is the hack that helped me attach a VNC display to the instance and ultimately I could identify and fix the actual problem.

It is not safe to leave this option enabled on a production setup.

Bringing up eucalyptus instance with VNC attached

UEC: Storage Management

Storage Controller

The Storage Controller is a block storage in Eucalyptus which is compatible with Amazon’s Elastic Block Store. euca2ools or EC2 commands can be used to control these volumes. Below we specify on using euca2ools commands for volume creation, attaching a volume to a running instance and how-to use the attached volume in the instance.

Interacting with Storage Controller

Linux Instances

First create a volume of the required size using euca-create-volume

$ euca-create-volume -s 10 -z mycloud
VOLUME  vol-333C04B8    10       creating        2010-03-26T05:20:56.383Z

Check the available volumes and their status with euca-describe-volumes

$ euca-describe-volumes
VOLUME  vol-333C04B8    10   mycloud   <strong>available</strong> 2010-03-26T05:20:56.383Z

Now attach the volume to a running instance using euca-attach-volume

$ euca-attach-volume -i i-41620887 -d /dev/sdb vol-333C04B8

The volumes status changes from “available” to “in-use” if it is attached properly.

$ euca-describe-volumes
VOLUME  vol-333C04B8    10  mycloud <strong>in-use</strong>  2010-03-26T05:41:28.019Z
ATTACHMENT      vol-331F04B2    i-41620887      unknown,requested:/dev/sdb  2010-03-26T05:43:51.343Z

UEC installation on Ubuntu 10.04 (Lucid Lynx)

Tested on Ubuntu 10.04 beta2 Tested and deployed on Ubuntu 10.04 final release

Installation

The following sections describe how to install and configure a basic UEC setup spanning 3 servers. 2 Servers(Server1 and Server2) will run Lucid 64-bit server version and server will run Lucid Desktop 64-bit version(Client1). We have decided on installing the Desktop version on Client1 so that we can use Firefox or other browsers to access the web interface of UEC and also to use RDP/VNC clients during the image creation process.

Server1 – To run Cloud Controller, Cluster Controller, Walrus Storage service and Storage Controller

Hardware       Minimum            Suggested
CPU            1GHz               2 x 2GHz
Memory         1GB                2GB
Disk           5400rpm IDE        7200rpm SATA
Disk Space     40GB               200GB
Networking     100Mbps            1000Mbps

Server2 – To run node-controller for running Virtual Machine (VM) instances

Hardware       Minimum            Suggested
CPU            VT extensions      VT, 64-bit, Multicore
Memory         1GB                4GB
Disk           5400rpm IDE        7200rpm SATA or SCSI
Disk Space     40GB               100GB
Networking     100Mbps            1000Mbps

Client1 – To function as the client for the cloud and also to be used for bundling images etc.

Hardware       Minimum            Suggested
CPU            VT extensions      VT, 64-bit, Multicore
Memory         1GB                4GB
Disk           5400rpm IDE        7200rpm SATA or SCSI
Disk Space     40GB               100GB
Networking     100Mbps            1000Mbps

Introduction to UEC and its components

Cloud

Cloud Computing is a computing model, where resources such as computing power, storage, network and software are abstracted and provided as services on the Internet in a remotely accessible fashion. Billing models for these services are generally similar to the ones adopted for public utilities. On-demand availability, ease of provisioning, dynamic and virtually infinite scalability are some of the key attributes of Cloud Computing.

An infrastructure setup using the cloud computing model is generally referred to as ‘Cloud’. The following are the broad categories of services available on the ‘Cloud’:

1. Infrastructure As A Services (IAAS)
2. Platform As A Service (PAAS)
3. Software As A Service (SAAS)

This ‘Cloud’ is generally available to anyone on the Internet willing to pay. However, a variant called ‘Private Cloud’ is increasingly becoming popular for private infrastructure that has some of the attributes of the ‘Cloud’ as mentioned above.

Eucalyptus

Eucalyptus is a software available under GPL that helps in creating and managing a private or even a publicly accessible cloud. It provides an EC2 compatible cloud computing platform and S3 compatible cloud storage platform. Eucalyptus has become very popular and is seen as one of the key open source cloud platforms. Since Eucalyptus makes its services available through EC2/S3 compatible APIs, the client tools written for AWS can be used with Eucalyptus as well.

UEC: Log Files

The following log files would be of interest for debugging errors encountered while working with UEC.

On CC

axis2c.log – This file logs messages specific to Axis2c web services.

cc.log – This file logs messages specific to Cluster controller.

cc-registration.log – This file logs the success or failure of Cluster Controller registration. For example,

SUCCESS: new cluster 'mycloud' on host '192.168.1.1' successfully registered.

cloud-debug.log, cloud-error.log, cloud-output.log – These files log messages specific to Cloud Controller.

UEC: Bundling Windows Image

Creating new disk image

This will be the main HDD in the virtual image, so make sure to give it as much space as you’ll need. Since we’re building a kvm image, we can use a qcow2 format for disk images. Qcow2 is an expandable image format, so it’ll only take as much storage space as it’s actually used within the image.

$ kvm-img create -f qcow2 win-2k3.img 20G

OS Installation

Get the iso file of the Windows distribution to be installed in the image. And start the installation process.

$sudo kvm -m 1024 -cdrom Win2003_cd1.iso -drive file=win-2k3.img,if=scsi,boot=on -nographic -vnc :0

After finishing the installation and in case the installation needs 2nd CD as well, reboot the VM and launch the VM by the following command.

$ sudo kvm -m 1024 -boot c -cdrom Win2003_cd2.iso -drive file=win-2k3.img,if=scsi,boot=on -nographic -vnc :0

Once installation is complete, create the boot disk needed for Windows 2003 server.

UEC: Attach EBS volume to a running instance

Linux Instances

First create a volume of the required size using euca-create-volume

$ euca-create-volume -s 10 -z mycloud
VOLUME  vol-333C04B8    10       creating        2010-03-26T05:20:56.383Z

Check the available volumes and their status with euca-describe-volumes

$ euca-describe-volumes
VOLUME  vol-333C04B8    10   mycloud   available     2010-03-26T05:20:56.383Z

Now attach the volume to a running instance using euca-attach-volume

$ euca-attach-volume -i i-41620887 -d /dev/sdb vol-333C04B8

The volumes status changes from “available” to “in-use” if it is attached properly.

$ euca-describe-volumes
VOLUME  vol-333C04B8    10  mycloud in-use  2010-03-26T05:41:28.019Z
ATTACHMENT      vol-331F04B2    i-41620887      unknown,requested:/dev/sdb  2010-03-26T05:43:51.343Z

UEC: Bundling Linux Image

Creating a new disk Image

This will be the main HDD in the virtual image, so make sure to give it as much space as youll need. Since were building a kvm image, we can use a qcow2 format for disk images. Qcow2 is an expandable image format, so itll only take as much storage space as its actually used within the image.

$ kvm-img create -f qcow2 image.img 5G

OS Installation

Download the iso file of the Linux distribution you want installed in the image.

$ wget http://releases.ubuntu.com/karmic/ubuntu-9.04-server-amd64.iso

And start the installation process:

$ sudo kvm -m 256 -cdrom ubuntu-9.04-server-amd64.iso -drive file=image.img,if=scsi,index=0 \
-boot d -net nic,vlan=0,model=e1000,macaddr= 00:16:3e:de:ad:01 -net tap -nographic -vnc :0

If your installation process requires more than 256MB of RAM change the -m option, and if you need more processors available, you can use the -c option.