I Like My Data And I Want To Keep It

Take me to the files
ilmdaiwtki (I Like My Data And I Want To Keep It) was something Chris often said (usually to people trying to persuade him into using resierfs). Well now you can like your data too and treat it with the respect it deserves.

ilmdaiwtki is a suite of two programs and a pile of ugly wrapper scripts.

The first program ilmdaiwtkid does most of the work. It:
- Scans all the devices /dev/hd? and attemps to execute the ATA IDENTIFY command on each one. If the device responds and isn't removable. ilmdaiwtkid makes a smartd.conf entry for it, and forks a process that reads bytes from the disk at a rate that will read the entire disk aproxximately once every week.
- Scans all the devices /dev/sd? it first tries an ATA IDENTIFY command, the various SCSI INQUITY commands. If the IDENTIFY command works a smartd.conf entry is made for it. If either the INQURIY or the IDENTIFY commands work and the device isn't a 3ware/AMCC 9500 series device then again a process is forked to read the disk or unit.
- Scans /proc/devices for twe or twa drivers (3ware/AMCC cards) and attempts to contact them (generating /dev/twe? and /dev/twa? device files). For each device found, it probes the ports and adds and disks it finds to smartd.conf. For each controller found it starts a copy of tw_check (see below).
- writes a smartd.conf file and invokes its copy of smartd on it
The second program tw_check checks on a 3ware/AMCC controller once per day. It checks that none of the units one the controller is degraded, and if the BBU has an `OK' status. It also remembers if the card ever had a BBU to catch cards which sometimes drop their BBUs and report that they don't have them. It also turns on autoverify for all units if the controller supports it. (This is why ilmdaiwtkid doesn't verify 9500 series units).
the remaining programs are perl wrappers around tw_cli, ilmdaiwtkid, smartd and smartctl and some tools to install the software. The wrappers look at the current architecture and select a precompiled static binary to run.

How to install

installation is very simple, download the latest ilmdaiwtki_bundle-x.y.z.tar.gz file, and untar it, and do the configure, make, make install dance. (if you're upgrading you need to uninstall the old version before make install)

[root@artemis bar]# wget http://www.madingley.org/james/resources/ilmdaiwtki/fil
es/ilmdaiwtki_bundle-1.0.14.tar.gz
--19:09:58--  http://www.madingley.org/james/resources/ilmdaiwtki/files/ilmdaiwt
ki_bundle-1.0.14.tar.gz
           => `ilmdaiwtki_bundle-1.0.14.tar.gz'
Resolving www.madingley.org... 131.111.64.221
Connecting to www.madingley.org[131.111.64.221]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3,412,088 [application/x-gzip]

100%[====================================>] 3,412,088     99.97K/s    ETA 00:00

19:10:33 (95.98 KB/s) - `ilmdaiwtki_bundle-1.0.14.tar.gz' saved [3,412,088/3,412
,088]

[root@artemis bar]# tar xzf ilmdaiwtki_bundle-1.0.14.tar.gz 
[root@artemis bar]# cd ilmdaiwtki_bundle-1.0.14

Set prefix to where you would like the software installed. The package contains precompiled versions of smartctl, smartd and tw_cli that you can run. They are placed in ${prefix}/bin.

[root@artemis ilmdaiwtki_bundle-1.0.14]# ./configure --prefix=/software
checking for gcc... gcc
[...]
config.status: creating scripts/tw_check
config.status: executing depfiles commands
config.status: executing default commands
[root@artemis ilmdaiwtki_bundle-1.0.14]# make
Making all in scripts
[...]
make[1]: Nothing to be done for `all-am'.
make[1]: Leaving directory `/root/bar/ilmdaiwtki_bundle-1.0.14'

Uninstall the old version if present where ${prefix} is the value of prefix above

[root@artemis ilmdaiwtki_bundle-1.0.14]# ${prefix}/bin/ilmdaiwtkid_install -u
killall  /software/lib/ilmdaiwtki_bundle/bin/ia32/ilmdaiwtkid -- OK
[...]
killall  /software/lib/ilmdaiwtki_bundle/bin/ia32/smartd -- OK

Now make install

[root@artemis ilmdaiwtki_bundle-1.0.14]# make install
Making install in scripts
[...]
make[1]: Leaving directory `/root/bar/ilmdaiwtki_bundle-1.0.14'
[root@artemis ilmdaiwtki_bundle-1.0.14]#

Finally run the installer - this will add an entry in /etc/inittab and start up the monitoring. Replace myemail@address with the address you want reports to be sent to. (replace ${prefix} with the value of prefix above)

[root@artemis ilmdaiwtki_bundle-1.0.14]# ${prefix}/bin/ilmdaiwtkid_install myemail@address

Advanced installation for deployment across a large number of machines

If you intend to deploy ilmdaiwtki across lots of computers you can skip the configure/make/make install steps (the compiled binaries are architecture and installation independant). Follow the instuctions above as far as make install but don't do the make install step instead make a temporary directory, and use DESTDIR.

[root@artemis ilmdaiwtki_bundle-1.0.14]# mkdir /tmp/deploy
[root@artemis ilmdaiwtki_bundle-1.0.14]# make DESTDIR=/tmp/deploy install
Making install in scripts
[...]
make[1]: Leaving directory `/root/bar/ilmdaiwtki_bundle-1.0.14'
[root@artemis ilmdaiwtki_bundle-1.0.14]#

Now tar up your installation image

[root@artemis ilmdaiwtki_bundle-1.0.14]# tar czfC ilmdaiwtki-image.tar.gz /tmp/deploy .

Now for each machine you want to install on copy the ilmdaiwtki-image.tar.gz file to /tmp and run. (The first step uninstalls an old version)

[root@artemis tmp]# ${prefix}/bin/ilmdaiwtkid_install -u
killall  /software/lib/ilmdaiwtki_bundle/bin/ia32/ilmdaiwtkid -- OK
[...]
killall  /software/lib/ilmdaiwtki_bundle/bin/ia32/smartd -- OK
[root@artemis tmp]# tar xzfC ilmdaiwtki-image.tar.gz  /
[root@artemis tmp]# ${prefix}/bin/ilmdaiwtkid_install myemail@address

That's it

Processes

Once it's sucessfully installed you should see a variety of processes running. There should be one sleeping ilmdaiwktid process, one verify process for each disk, one tw_check process for each controller, and finally one smartd process.

[root@artemis ~]# ps -ax
    1 ?        S      0:01 init [5]  
[...]
 5283 ?        Ss     0:00 ilmdaiwktid: sleeping                                
 5305 ?        Ss     0:03 ilmdaiwktid: /dev/sdb verifying                      
 5307 ?        Ss     0:03 ilmdaiwktid: /dev/sdc verifying                      
 5308 ?        Ss     0:03 ilmdaiwktid: /dev/sdd verifying                      
 5309 ?        Ss     0:03 ilmdaiwktid: /dev/sde verifying                      
 5311 ?        S      0:00 perl /software/bin/tw_check 0 smart@madingley.org
 5337 ?        S      0:00 /software/lib/ilmdaiwtki_bundle/bin/x86_64/smartd -c  /software/etc/smartd.conf
[...]
 5457 pts/1    Ss     0:00 -bash
 5490 pts/1    R+     0:00 ps -ax
[root@artemis ~]#

Take me to the files