Xen
From DuncanWiki
This page is completely geared towards CentOS as I'm running CentOS as the host operating system & CentOS for all of my virtual hosts.
The point of this page is to have someone who's never used Xen before to be able to have virtual machines up and running in under an hour by following the Setting Up A Xen Virtual Host section.
Please keep in mind that this isn't intended as a complete guide to using or configuring Xen and will by no means make you an expert on the subject. This will get you where you need to be quickly and enable you to learn more about Xen virtualization.
Who knows, you might even become somewhat proficient using Xen after going through this guide.
Any messages with errors, inconsistencies, omissions or improvements are more than welcome. See the Duncan page for contact information.
What Is Xen?
The Xen® hypervisor, the powerful open source industry standard for virtualization, offers a powerful, efficient, and secure feature set for virtualization of x86, x86_64, IA64, PowerPC, and other CPU architectures. It supports a wide range of guest operating systems including Windows®, Linux®, Solaris®, and various versions of the BSD operating systems.
Terminology
- Hypervisor : Xen itself, it controls all interaction between the domains and the hardware, including booting Domain0.
- Domain0 (Dom0) : The privileged GNU/Linux domain started by the hypervisor which has access to the physical machine's hardware and also has the ability to start, stop & otherwise manipulate the other domains.
- DomainU (DomU) : An unprivileged domain without access to the physical hardware. This is your standard virtualized machine and what you'll eventually install after you've got Xen running on your system.
What I'm Doing With It
I have in my possession a computer that's far more than anything I realistically need as a single machine. It's a quad core Intel Xeon X3220 @ 2.4GHz & sports a beefy 8GB of RAM.
So, what does that mean? It means that I'm going to have this machine host several virtual machines that will be everything from web servers, database servers & more.
Setting Up A Xen Virtual Host
Installing Xen
You'll first need to install the hypervisor as well as a kernel that will work underneath it.
yum install kernel-xen xen
Booting Xen Kernel
After installing your Xen kernel you'll need to set it as the default kernel to boot into, this is specified in /etc/grub.conf. You'll need to update the default= line to read as the Xen kernel, in this case it'll be 0 since it's the first one. Update to point at a Xen kernel if necessary & reboot.
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/sda2
# initrd /initrd-version.img
#boot=/dev/sda
default=0
timeout=5
serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1
terminal --timeout=5 serial console
title CentOS (2.6.18-128.2.1.el5xen)
root (hd0,0)
kernel /xen.gz-2.6.18-128.2.1.el5
module /vmlinuz-2.6.18-128.2.1.el5xen ro root=LABEL=/ rhgb quiet console=ttyS0,9600n8 console=tty1
module /initrd-2.6.18-128.2.1.el5xen.img
title CentOS (2.6.18-92.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-92.el5 ro root=LABEL=/ rhgb quiet console=ttyS0,9600n8 console=tty1
initrd /initrd-2.6.18-92.el5.img
Grabbing Xen Install Files
For a 64-bit machine :
wget -O /tmp/centos_5.img http://mirror.centos.org/centos/5/os/x86_64/images/xen/initrd.img wget -O /tmp/centos_5 http://mirror.centos.org/centos/5/os/x86_64/images/xen/vmlinuz
For a 32-bit machine :
wget -O /tmp/centos_5.img http://mirror.centos.org/centos/5/os/i386/images/xen/initrd.img wget -O /tmp/centos_5 http://mirror.centos.org/centos/5/os/i386/images/xen/vmlinuz
Where DomU Filesystems Live
On my Dom0 system I've created a dedicated filesystem that lives under /vm/ where I keep my virtual hosts, one to a directory. Let's make one for our new host, Stilgar.
mkdir /vm/stilgar
Creating A Disk Image
Modify the red text to specify the size of the drive you want to create in gigabytes.
NOTE: this command allocates all the space for the drive image at once, it will not grow. This is to prevent disk space contention issues on the Dom0 host if multiple DomU systems start to grow out their disk images.
disk_gigabytes=20 cd /vm/stilgar dd if=/dev/zero of=hda.img oflag=direct bs=1M count=$[ $disk_gigabytes * 1024 ]
Alternately we can create a drive image that will grow as our needs require up to the maximum we specify. Since we're not allocating the space we need beforehand we could come into disk space contention issues in the future if we need space that's not available.
disk_gigabytes=20 cd /vm/stilgar dd if=/dev/zero of=hda.img oflag=direct bs=1M seek=$[ $disk_gigabytes * 1024 ] count=1
Xen Configuration File
This is done in two stages. The first stage is an installation & the second is the every-day configuration.
Installation Configuration
I've saved the following file as /etc/xen/stilgar. Below you'll see a few sections marked in red.
- mac= : this determines the MAC address of your virtual machine. You'll want to make sure that it's unique on your network. 00:16:3e is unique to Xen systems and has been reserved as such, you'll want to ensure that the last three aren't repeated on your local network.
- ks= : this is the location of your kickstart file, I've pointed this at the one I host on my files site. You'll want to change this to i386 if you're running a 32-bit system.
- name = : this is the name that the hypervisor knows the system as
- disk = : this is where the disk image resides
kernel = "/tmp/centos_5" ramdisk = "/tmp/centos_5.img" extra = "text ks=http://files.duncanbrown.org/linux/xen/kickstart-x86_64.ks" name = "stilgar" memory = "512" disk = [ "tap:aio:/vm/stilgar/hda.img,xvda,w", ] vif = [ "mac=00:16:3e:01:01:01,bridge=xenbr0" ] vcpus=1 on_reboot = "destroy" on_crash = "destroy"
Kickstart Files
I keep a copy of my kickstart files available online in case anyone wants to refer to them for their own Xen installs. The CentOS mirror referenced is a very high-speed server that I've clocked > 1MB/s transfer rates with. Since it's just a minimal install this shouldn't take long at all.
Start Your Installation
The following command will create your DomU host as well as connect you to the console with -c. The installation should go smoothly, though any errors will appear in your terminal. If you've ever performed a network installation of RedHat, Fedora or CentOS there really aren't any surprises to be found. It's the text-mode installer.
xm create -c /etc/xen/stilgar
The -c flag tells Xen to connect to the console. CTRL-] will disconnect you.
Final Configuration
Now that you've finished your base installation you'll want to have your DomU start when the Dom0 host boots, this is achieved by saving a configuration file in /etc/xen/auto/. I saved the following as /etc/xen/auto/stilgar.
name = "stilgar" memory = "2048" disk = [ "tap:aio:/vm/stilgar/hda.img,xvda,w", ] vif = [ "mac=00:16:3e:01:01:01,bridge=xenbr0" ] bootloader="/usr/bin/pygrub" vcpus=1 on_reboot = "restart" on_crash = "restart"
Controlling Virtual Hosts
The xm command allows you to control your hosts as well as connect to their consoles.
While there's alot more to xm than this, you can read more about it on it's man page.
Start a DomU instance based on a config file
xm create [DomU_config_file]
Start a DomU instance based on a config file and immediately attach to the console
xm create -c [DomU_config_file]
Show DomU hosts running
xm list
Connect to the virtual serial console of a DomU, CTRL-] to disconnect.
xm console [DomU]
Pause a DomU
xm pause [DomU]
Un-pause a DomU
xm unpause [DomU]
Shut down a DomU safely
xm shutdown [DomU]
Reboot a DomU safely
xm reboot [DomU]
Save the current condition of a DomU to a statefile
xm save [DomU] save_statefile
Restore a DomU from a statefile
xm restore save_statefile
Pull the plug on a DomU
xm destroy [DomU]
Give an overview of current running DomU systems, example display can be seen at http://files.duncanbrown.org/linux/xen/xm_top.png
xm top
Prebuilt Xen Image
This is a 64-bit CentOS 5 Xen DomU machine that I've been using as a template. The default root password is changeme. It's only a 5GB drive in order to conserve disk space. You can add additional disks.
Additional Xen Usage
DO NOT USE ANYTHING IN THIS SECTION : I'm currently fleshing it out and things may be inaccurate or downright dangerous to use.
Installing Other Operating Systems
-- http://nixcraft.com/xen/13792-xen-ubuntu-9-10-a.html
kernel = "/usr/lib/xen/boot/hvmloader" builder = 'hvm' memory = 512 name = "winxp" vcpus = 1 vif = [ 'type=ioemu, bridge=xenbr0' ] disk = [ 'file:/var/xen/images/WinXP.img,ioemu:hda,w' ] device_model = '/usr/lib/xen/bin/qemu-dm' cdrom='/dev/hda' ne2000=0 boot='d'
DomU Display Output
Serial/TTY
VNC
VNC is a method used to access a GUI over a network. Xen has the ability to create a virtual framebuffer for remote VNC access of DomU systems.
Configuration For Dom0
If you want remote hosts to be able to access the DomU hosts over VNC you'll need to update /etc/xen/xend-config.sxp. The default in CentOS is to disable remote VNC connections, you'll need to update the vnc-listen directive.
# The interface for VNC servers to listen on. Defaults # to 127.0.0.1 To restore old 'listen everywhere' behaviour # set this to 0.0.0.0 #(vnc-listen '127.0.0.1')
I've changed mine to read as follows :
(vnc-listen '0.0.0.0')
You'll also see a vncpasswd directive, you can leave that blank or set it for a global VNC password. In the next section you'll see how you can set individual VNC passwords on individual DomU systems.
(vncpasswd 'mylamepassword')
Configuration For DomU
You'll want to set the virtual framebuffer to use VNC as well as give it a unique port and optionally a password. The example below assigns the host the VNC port of 5910 with the password mylamepassword.
If you don't supply anything for vncdisplay it will automatically assign the next available port starting at 5900.
You add this to the Xen DomU configuration on the Dom0.
vfb = [ 'type=vnc,vncdisplay=10,vncpasswd=mylamepassword' ]
Next you'll need to update /etc/grub.conf on the DomU host and remove console=xvc0.
title CentOS (2.6.18-164.el5xen)
root (hd0,0)
kernel /vmlinuz-2.6.18-164.el5xen ro root=/dev/VolGroup00/LogVol00 console=xvc0
initrd /initrd-2.6.18-164.el5xen.img
VNC & Avahi/mDNS
Avahi/mDNS is a method of having services broadcast their availability to the network. This is what iTunes does when you share your music on the network to inform other systems that it has a library to share.
You can similarly use this service to broadcast the availability of your Xen VNC server on a per-host basis.
I saved this file as /etc/avahi/services/stilgar_vnc.service :
<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name replace-wildcards="yes">StilgarVNC</name>
<service>
<type>_rfb._tcp</type>
<port>5910</port>
</service>
</service-group>
Once you have this configured you need to restart Avahi.
service avahi-daemon restart
If you're running Mac OS X you'll now see your VNC server listed in the Finder.
Storage
CD/DVD Drives
Add this to your disk= configuration :
‘phy:/dev/cdrom,xvdb:cdrom,r’
CD/DVD Images (.iso)
If you want to have an .iso file attached on boot you'll just need to add it to your Xen configuration file for that particular DomU. I've marked what you may want to change in case /dev/hdd is already in use.
disk = [ 'tap:aio:/vm/stilgar/hda.img,xvda,w', 'file:/path/to/your/dvd.iso,hdd:cdrom,r' ]
Alternatively you can add it to an already running DomU.
Additional Virtual Drives
Physical Drives
Add this to your disk= configuration :
‘phy:/dev/sda1,xvda,w’,
RAID
This probably isn't any different from using actual physical devices since the RAID aspect is handled by the Dom0.
Networking
Bridged
NAT
Mac OS X DomU
Something possibly promising, though the thread died.
Windows DomU
BSD DomU
What I Hope To Document
- Proper DomU use of USB drives connected to Dom0
- Datacenter In A Box : basically a way to have a self-contained environment from domain authentication, DNS & NAT to have a self-contained set of systems that are isolated from everything else (save for forwarded ports)
