Setting up the domains is pretty straight forward, RedHat has a nice guide using their virt-manager GUI tool. There are also excellent guides at the Centos wiki.

Two things were not obvious to me, and I'm posting my findings here to help others.

Issue 1 Multiple networks

After building the domain, the first thing that stopped me was that I needed multiple networks. The virtual hosts need to be on three networks, and so I needed three virtual devices. The RedHat guide has a nice section on how to create the necessary br0 (network bridge devices) devices, so modifying the scripts to go from one br0 to three (br0, br2, br3) each corresponding to a real NIC on the host was straightforward. However, how to get the virtual devices to recognize and use these new bridges? There is only one config file for your domain, and it looks like this:

# Automatically generated xen config file
name = "myVirtualPony"
memory = "1024"
disk = [ 'phy:/dev/virt_vg/some_virtdisk,xvda,w', ]
vif = [ 'mac=00:16:3e:3d:11:f7, bridge=xenbr0', ]
vfb = ["type=vnc,vncunused=1"]
uuid = "fff0f6c0-60e2-xxxx-xxxx-xxxxxxxxxx"
bootloader="/usr/bin/pygrub"
vcpus=1
on_reboot   = 'restart'
on_crash    = 'restart'

It should be obvious to any Python programmer, but it wasn't to me initially. Eventually I realized you had to add some new attributes to the vif list. The Redhat guide has a nice python snippet to generate new MAC addresses for you. I used that to generate two more MACs, then added them in:

# Automatically generated xen config file
name = "myVirtualPony"
memory = "1024"
disk = [ 'phy:/dev/virt_vg/some_virtdisk,xvda,w', ]
vif = [ 'mac=00:16:3e:3d:11:f7, bridge=xenbr0',
        'mac=00:16:3e:19:11:ed, bridge=xenbr2',
        'mac=00:16:3e:49:11:40, bridge=xenbr3', ]
vfb = ["type=vnc,vncunused=1"]
uuid = "fff0f6c0-60e2-xxxx-xxxx-xxxxxxxxxx"
bootloader="/usr/bin/pygrub"
vcpus=1
on_reboot   = 'restart'
on_crash    = 'restart'

Now just reboot the domain with a quick

$ xm reboot myVirtualPony

and loginto the domain. Copy the /etc/sysconfig/network-scripts/ifcfg-eth0 to ifcfg-eth1, ifcfg-eth2 and edit them, using the new MAC addresses you created and the proper TCP/IP info.

There is an excellent Page on Xen Networking at Xensource Wiki. I wish I had found this when I was banging my head on this problem.

Issue 2 - Virtual Console

After my domains were setup most Xen docs talk about using the virtual console to login to your domain from the host, similar to what you can do with Solaris Zones.

On the master host you should be able to login to the virtual console of the local domains. However, when you build your domains with RedHat's virt-manger they don't setup the virtual console correctly. They attach serial output to the virtual-framebuffer. So if you try and use the virtual tty, you only get the output of the kernel booting, then it stops and your terminal is stuck.

To fix this, you can follow the directions Xen Centos Tips and Tricks page:

You just need to add

co:2345:respawn:/sbin/agetty xvc0 9600 vt100-nav

in /etc/inittab

and add

xvc0

to the file /etc/securetty. Then do a @ telinit q to re-load the /etc/inittab and you should be able to use the virtual console from a tty.

Now with that out of the way I can do this to login into my domain:

xm console myVirtualPony

Xen is fun.

On sale worldwide in August from Asus. Is this going to be a hit or what? I'll pick one up for that. It's less than the Nokia 800, and it has a keyboard. For when I need to do more typing that I can stand on my iPhone.

Nice hands-on review and many pictures at Notebookreview.com