With the release of Lustre 2.4 support for ZFS Lustre servers has arrived. Historically, Lustre has only supported ext4/ldiskfs servers and while those servers have performed well they do suffer from a number of well known limitations. Extending Lustre to use a next generation filesystem like ZFS allows it to achieve greater levels of scalability and introduces new functionality.

• Data Integrity - Data is always checksummed and self repairing to avoid silent corruption.
• Pooled Storage - Easy aggregation of multiple devices in to a single OST.
• Capacity - A 256 zettabytes (2^78 bytes) OST size limit enables larger servers.
• Snapshots - Snapshot the Lustre filesystem prior maintenance for worry free updates.
• Compression - Transparent compression increases your total usable capacity.
• Copy-on-Write - Improves write performance by transforming random I/O in to sequential I/O.

Support for ZFS has been merged in to the Lustre mainline source tree and will be officially available when Lustre 2.4 is released. A pre-release version of this code is running on Lawrence Livermore's Sequoia system which is currently ranked second on the top 500 list.

Lustre DKMS packages for RHEL/CentOS 6 are available from the ZFS on Linux EPEL repository. They track the upstream Lustre master branch and are refreshed as new pre-releases are tagged. Once the official Lustre 2.4 release is tagged the repository will switch to tracking the 2.4 maintenance branch. Packages are provided for:

• Releases: RHEL/CentOS 6.x
• Arches: x86_64

To add the repository to your system install the zfs-release package as shown below. This will add the /etc/yum.repo.d/zfs.repo and required signing keys to your system. You can then install Lustre like any other EPEL package using yum. As new updated packages are made available they will be detected and installed as part of the standard update process.

$ sudo yum localinstall --nogpgcheck http://archive.zfsonlinux.org/epel/zfs-release-1-2.el6.noarch.rpm
$ sudo yum install lustre

To familiarize yourself with the ZFS Lustre servers you can quickly setup a new test filesystem using the llmount.sh script. Make sure SELinux is disabled and the lustre-tests package is installed, then run:

$ sudo yum install lustre-tests
$ sudo FSTYPE=zfs /usr/lib64/lustre/tests/llmount.sh
  ...

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
lustre-mdt1/mdt1      154M  2.9M  149M   2% /mnt/mds1
lustre-ost1/ost1      154M  1.3M  151M   1% /mnt/ost1
lustre-ost2/ost2      154M  1.2M  151M   1% /mnt/ost2
centos-6-amd64@tcp:/lustre
                      308M  2.4M  302M   1% /mnt/lustre
 

That's it! You've create a single node Lustre filesystem using ZFS servers which is mounted at /mnt/lustre. When you're done testing it can be unmounted and the servers stopped by running, 'sudo FSTYPE=zfs /usr/lib64/lustre//tests/llmountcleanup.sh'. To deploy a more realistic configuration you should refer to the official Lustre documentation.