This write-up is incomplete, but it has some useful information about my first attempt at ganeti. I’ll be attempting this install on another set of machines in the near future, and i’ll post another blog entry.
I’m a system administrtor, I started out as a hacker, a tinkerer, and a curious sort. No, I’ve never stolen your credit card number from Bob’s pet mart’s web site, and i’ve never defaced whitehouse.gov. I’m using hacker in a more positive light. Something you’ll find I’m adamant about.
I have a wife, and a family now, so I don’t get to devote nearly as much time to tinkering as i have in the past, but Tonight we’ve decided not to get our usuall husband and wife WoW time in, and my wife’s over in the bedroom reading one of her romance novels, and I find myself with an unusually free evening. It’s already after 21:00, but that’s ok with me. I’ve grabbed a Guinness from the fridge, and I’m planning on giving a project that I’ve been looking at recently a shot. Ganeti.
Ganeti is virtualization cluster tool which uses commodity hardware, WITHOUT a SAN to provide high availablility storage, and virtualization infrastructure. You migh call it the basis of a Cloud. In fact, i’m told that this is what Google uses to run their cloud. If it’s good enough for Google…..
Note: I started this attempting to use the ganeti docs, which build from source. I then found that this method was dependant on Ubuntu’s system, and did not work on RHEL, or Fedora. I’m testing on Fedora, and will eventually build on RHEL6. I found the following regarding Ganeti on RHEL.
I wrote this blog AS i was working on the install, so just realize that I started out with the source distribution, and then later tried the rpm build, altering the steps listed in this blog entry as I did.
Per the Ganeti docs, AND the RHEL docs, I created lv’s on both machines. My desktop have 3 drives that used to be a RAID 5 array until i found that no software which both linux AND windows 7 used could see the array. So now theyre individual drives. Drives 2 and 3 mirror eachother via an rsync job. So, I used the 3rd drive in my desktop, and a usb attached sata drive on my laptop.
Ended up with:
[root@azure ~]# vgdisplay ganeti
— Volume group —
VG Name ganeti
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 596.17 GiB
PE Size 4.00 MiB
Total PE 152619
Alloc PE / Size 0 / 0
Free PE / Size 152619 / 596.17 GiB
VG UUID 2KjqSD-Y8d2-reQI-f1wj-Uu9i-Oz2d-BBRAsW[root@warchild ~]# vgdisplay ganeti
— Volume group —
VG Name ganeti
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 55.89 GiB
PE Size 4.00 MiB
Total PE 14307
Alloc PE / Size 0 / 0
Free PE / Size 14307 / 55.89 GiB
VG UUID PoRXz2-Td76-LkB7-cAMh-HZb6-BMPG-7dunyS
Per install recommendation, added the following filter to /etc/lvm/lvm.conf.
filter = [“r|/dev/cdrom|”, “r|/dev/drbd[0-9]+|” ]
Installed the following packages, and their dependencies. DRBD had a ton.
May 13 22:46:29 warchild yum[2856]: Installed: drbd-8.3.8.1-1.fc14.i686
May 13 23:05:50 warchild yum[3562]: Installed: socat-1.7.1.3-1.fc14.i686
May 13 23:06:43 warchild yum[3566]: Installed: python-paramiko-1.7.6-3.fc14.noarch
May 13 23:09:03 warchild yum[3583]: Installed: pyparsing-1.5.5-1.fc14.noarch
May 13 23:09:59 warchild yum[3591]: Installed: python-inotify-0.9.1-1.fc14.noarch
May 13 23:11:12 warchild yum[3599]: Installed: pynetsnmp-0.28.14-4.fc14.noarchMay 13 23:17:06 warchild yum[4525]: Installed: pandoc-1.6.0.1-2.fc14.i686
May 13 23:18:53 warchild yum[4531]: Installed: python-sphinx-1.0.7-1.fc14.noarch
May 13 23:20:49 warchild yum[4859]: Installed: graphviz-2.26.3-2.fc14.i686
May 13 23:20:50 warchild yum[4859]: Installed: pydot-1.0.2-7.fc14.noarch
At this point I went and grabbed the rpm’s listed on http://linuxnotes.us/archives/294 and installed ganeti, and ganeti-instance-bootstrap.
Fedora 14 runs Python 2.7, and the el6 rpm assumes Python 2.6, which is what’s in Rhel. So, the libraries installed are in the wrong place. I just sym-linked them.
I initialized the cluster with the following command:
[root@warchild rhel]# gnt-cluster init –nic-parameters mode=routed,link=100 –enabled-hypervisors=kvm –master-netdev eth0 –vg-name ganeti ganeti.undrground.org
A quick talk about the hosts file.
Ganeti depends on dns names, it needs to be able to find its members via dns lookups, so a “host” command should return the proper value. You also need a cluster IP, which also needs a dns name. I used ganeti.undrground.org for this. So my hosts file looks like this:
10.1.1.51 azure.undrground.org azure
10.1.1.42 ganeti.undrground.org ganeti
10.1.1.101 warchild.undrground.org warchild
Provosioning vm’s.
We intend to run RHEL vm’s on these boxes, so debootstrap wont do. I found (per the instructions on the RHEL ganeti link above) a package called ganeti-image. Which essentially lets you make tar.gz’s of installed systems, and use them as base images for your guests. I quickly kickstarted a RHEL6 vm, and got such a tar.gz of a freshly installed system.
This is as far as I’ve got on these systems. In a few days I should have systems to attempt a clean RHEL6 install on. I’ll write a fresh install guide, and link from there. If someone finishes where I left off, please feel free to comment.