Lustre

Lustre ("Linux Clusters") is an open source, high performance distributed file system for large clusters of Linux computers. Developed by Cluster File Systems and released as open source under the GNU GPL, Lustre is a software only platform that is POSIX compliant and can be deployed on commodity hardware.

The Lustre architecture separates the metadata processing--such as the creation of new inodes for files and the setting or changing of file attributes--from actual file data I/O. Metadata operations are handled by a pair of servers (providing failover) dedicated to the task; while actual data I/O operations are handled by Object Storage Servers, which are deployed in failover pairs.

Multiple abstraction layers are built into the platform that enable it to operate with multiple existing forms of storage and networking topologies. The vendor notes that the disk storage utilized by the Object Storage Servers can be individually attached disks, SAN, or shared RAID storage arrays; and several networking communications protocols are supported between the Lustre file system and the client nodes, including Quadrics Elan 3/4, TCP/IP, InfiniBand and Myrinet. Client nodes themselves require a driver to access the file system. Currently, only native Linux drivers are available; however, the vendor notes the existence at this time of both a beta client for Mac OS X and a beta parallel-CIFS client for Windows.

In the Lustre architecture, files are manipulated as objects; with the Metadata servers tracking their locations across the Object Storage Servers and keeping a transactional record of metadata changes. Object Storage Servers (OSS) then actually hold the data for the files, which may be striped across several OSSs. Once the location of the file is known by the client (from the Metadata server) all subsequent I/O operations are handled directly between the client and the OSS. With parallel processing capabilities possible, the vendor notes that aggregate data throughputs have been measured at up to 15 GB/sec, with a single client achieving up to 2 GB/sec throughput.

For recovery purposes, individual clients can access LDAP servers to determine the appropriate replacement server should the client incur a timeout attempting to access data. The client then directs subsequent requests to that replacement server, and meanwhile new file creation operations automatically avoid the faulty servers.

The latest release of Lustre, which has been available directly to Cluster File Systems' customers since August, is now available to the general public (and the vendor additionally notes that future Lustre releases will be made available to both CFS customers and the public simultaneously). Key features of this release include:

- Production-quality clients and servers on Linux 2.6

- Support for multiple network interfaces in a single node

- Support for clusters of more than 5,000 nodes and 400 I/O servers

- SNMP integration

Lustre is available now, as a free download from the Cluster File Systems Web site. Visit the site for further information.