Gallery

SAN on the cheap

Posted on 18 Jan 2006 at 14:37

Simon Brock and Ian Wrigley use off-the-peg open-source software to build a basic implementation of a SAN using iSCSI

Target iqn.2005-11.uk.co.widearea:storage2.diskc

This names a target according to the normal iSCSI convention, which is year-month followed by a reversed domain name and a machine identifier. The final line, which must be indented, is:

Lun 0 Path=/dev/sdb,Type=fileio

This says that we're going to offer the SCSI disk drive /dev/sdb as an iSCSI device. Having done this, we then did something similar on the other machine.

Now we needed an initiator. Rather than using Linux for everything, we wanted to prove that this really does work cross-platform, so we used the Microsoft iSCSI initiator. The boxout shows how to get this to work with iSCSI Enterprise Target; in our case, we mounted the iSCSI disk drive from each machine on a single Windows machine. At that point, we turned them into dynamic disks and put them into a mirror. After the usual formatting and synchronisation, we had a resilient mirror on a Windows machine implemented over an IP SAN onto Linux boxes, and everything worked. If one machine does disappear, the mirror will be broken and Windows will continue to update the other machine. When the other machine returns, the mirror will be fixed and Windows will synchronise the disks. However, the big problem is that Microsoft doesn't support the use of iSCSI for dynamic disks, so we need another solution.

The first thing we needed to do was to get DRBD running. Again, this is fairly simple if you refer to the pre-installed /etc/drbd.conf file. All we did was find the first resource section and within that there are two 'on' sections that define how the disk block device you're trying to implement is put together. You have to provide your own machine names, addresses and disk partitions. Copy the file to both machines, run the script /etc/init.d/drbd start and everything should come up. A quick check in the file /proc/drbd shows everything is sound and the disk is in secondary/secondary state on both machines. When it's working correctly, one machine will be in the state primary/secondary and the other in the state secondary/primary. However, initially it will show the state as inconsistent and not allow you to set one to primary. The first time a DRBD device is set up, you have to tell the pair of machines which one is primary, and therefore on one machine you must run the command 'drbdadm -- --do-what-I-say primary all'. After this, a quick check in /proc/drbd will show it synchronising away.

Now that we had the DRDB device working, we had to change the /etc/ietd.conf to include a new device called /dev/drbd0, where previously we had /dev/sdb. Having done that, it was instructed to start the iSCSI target and prove we could mount that on another machine.

Having shown that this all worked, the next stage was to set up the heartbeat software. For this, we needed to find the three files to change - examples are distributed with the package and the Readme file in /etc/ha.d will tell you where to find the examples. To set up everything, we needed to do the following: first set up an ha.cf file, which will be different on each machine. In that file, we needed to add common 'node' lines that give the names of the two nodes and a different 'ucast' line, which is the address of the other machine. Second, we needed to set up the authkeys file, which is documented in the comments in the file. This file should be the same on both machines. Finally, we needed to put an haresources file on both machines, which looked something like this: