Clustering, Beowulf style

Even obsolete PCs can be brought back to life by Beowulf software.

You can achieve supercomputer performance from off-the-shelf PCs by running a Linux clustering system called Beowulf. New developments add the ability to run Beowulf clusters on 64-bit AMD Opteron processors, dramatically improving the performance of clustered computers.

Beowulf provides one way to group a set of computers to work on a single task. One PC acts as the master of the cluster, controlling the other computers. The other computers each act as stripped-down computation devices, performing operations in parallel. Each computer in the cluster gets one small piece of an overall task. All the computers in the cluster communicate over a high-speed internal network.

The power of Beowulf clustering lies in the usage of off-the-shelf hardware, dramatically reducing the cost for creating what can be supercomputer performance, at least for tasks that work well with clusters. Beowulf clusters work best for computational tasks that can be divided into relatively independent pieces. For example, a lot of weather prediction and graphics ray-tracing for movie special effects fit well into Beowulf-style clusters. One of the neat things about the clusters is that the software can work on older PCs, turning boxes relegated to boat anchors and door stops into computation engines.

Beowulf, though, isn’t just one software package. There are several packages you can install to make up parts of a Beowulf cluster such as Parallel Virtual Machine (PVM), Message Passing Interface (MPI), or, Local Area Multicomputer MPI (LAM/MPI). Different sites will require different packages and configurations. This complexity has given rise to a number of commercial solutions. On top of the open-source Beowulf software, Scyld adds simplified integration and installation, improved administration, commercial support, and manuals. If you are performing important scientific calculations, it is likely worthwhile to purchase support, since you want your staff to concentrate more on science and less on administration.

Scyld sells a version of the Beowulf clustering software for Linux, called the Scyld Beowulf Cluster Operating System. Scyld’s recent developments include support for the AMD Opteron processor. Part of the appeal of the AMD Opteron processor lies in its ability to run 32-bit or 64-bit code, which translates to existing applications and new applications.

Scyld isn’t the only firm providing Beowulf solutions. HP and Northrup Grumman both provide integrated hardware and software for clustering. Other attempts to simplify cluster installation include FAI, and Compaq’s Combined Utilities for Beowulf.

Related posts

Leave a Comment