counters

Improve your cluster daily administration by using a scalable Python API and through provided command- line tools (clush, nodeset, ...)

Stable version: 1.9.2

Why you need ClusterShell

ClusterShell is an event-driven open source Python library, designed to run local or distant commands in parallel on server farms or on large Linux clusters. No need to reinvent the wheel: you can use ClusterShell as a building block to create cluster aware administration scripts and system applications in Python. It will take care of common issues encountered on HPC clusters, such as operating on groups of nodes, running distributed commands using optimized execution algorithms, as well as gathering results and merging identical outputs, or retrieving return codes. ClusterShell takes advantage of existing remote shell facilities already installed on your systems, like SSH.

ClusterShell's primary goal is to improve the administration of high-performance clusters by providing a lightweight but scalable Python API for developers. It also provides clush, clubak and nodeset/cluset, convenient command-line tools that allow traditional shell scripts to benefit from some of the library features.

Scalability perspective

The ClusterShell library implements a scalable parallel execution model in a small footprint using asynchronous, non-blocking I/O (EDA). No massive multithreading is needed, making it easier for the developer to avoid race conditions, deadlocks or resource starvation. When multitasking cannot be avoided, the library provides task mailbox mechanisms.

ClusterShell is used daily at on several PetaFLOPS-class Linux supercomputers (with more than 5000 compute nodes) but also on server farms and storage clusters at CEA.

Requirements

The only requirement to use ClusterShell is Python (v2.7+ or v3.4+) on an Unix-like operating system (Linux, *BSD and Mac OS X are supported). Although not restricted to Secure Shell (SSH), it is the default remote shell used by ClusterShell to access remote nodes.

ClusterShell is distributed under the GNU Lesser General Public License, version 2.1 or later (LGPLv2+).