Introduction
StorPool is the fastest software-defined block storage software on the market, used by companies building public and private clouds to deliver an amazing end-user experience. It comes as software plus a fully-managed data storage service that transforms commodity hardware into a fast, highly available, and scalable shared-storage system for running demanding applications.
Overview
StorPool is distributed block storage software. It pools the attached storage (HDDs, SSDs or NVMe drives) of standard servers to create a single pool of shared storage. The StorPool software is installed on each server in the cluster. It combines the performance and capacity of all drives attached to the servers into one global namespace.
StorPool provides standard block devices. You can create one or more volumes through its sophisticated volume manager. StorPool is compatible with ext3 and XFS file systems and with any system designed to work with a block device, e.g. databases and cluster file systems (like OCFS and GFS). StorPool can also be used with no file system, for example when using volumes to store VM images directly or as LVM physical volumes.
Redundancy is provided by multiple copies (replicas) of the data written synchronously across the cluster. Users may set the number of replication copies or an erasure coding scheme. The replication level directly correlates with the number of servers that may be down without interruption in the service. For replication 3 and all N+2 erasure coding schemes the number of the servers (see Fault sets) that may be down simultaneously without losing access to the data is 2.
StorPool protects data and guarantees its integrity by a 64-bit checksum and version for each sector on a StorPool volume or snapshot. StorPool provides a very high degree of flexibility in volume management. Unlike other storage technologies, such as RAID or ZFS, StorPool does not rely on device mirroring (pairing drives for redundancy). So every disk that is added to a StorPool cluster adds capacity and improves the performance of the cluster, not just for new data but also for existing data. Provided that there are sufficient copies of the data, drives can be added or taken away with no impact to the storage service. Unlike rigid systems like RAID, StorPool does not impose any strict hierarchical storage structure dictated by the underlying disks. StorPool simply creates a single pool of storage that utilizes the full capacity and performance of a set of commodity drives.
Architecture
StorPool works on a cluster of servers in a distributed shared-nothing architecture. All functions are performed by all servers on an equal peer basis. It works on standard off-the-shelf servers running GNU/Linux.
Each storage node is responsible for data stored on its local drives. Storage nodes collaborate to provide the storage service. StorPool provides a shared storage pool combining all the available storage capacity. It uses synchronous replication across servers. The StorPool client communicates in parallel with all StorPool servers. The StorPool iSCSI target provides access to volumes exported through it to other initiators.
The software consists of two different types of services - storage server
services and a storage client services - that are installed on each physical
server (host, node). The storage client services are the native block on Linux
based systems, the iSCSI target, or the NVMeTCP target for other systems. Each
host can be a storage server, a storage client, iSCSI target, NVMeOF controller
or any combination. The StorPool volumes appear to storage clients as block
devices under the /dev/storpool/ directory and behave as normal disk
devices. The data on the volumes can be read and written by all clients
simultaneously; its consistency is guaranteed through a synchronous replication
protocol. Volumes may be used by clients as they would use a local hard drive or
disk array.
Feature highlights
Scale-out, not Scale-up
The StorPool solution is fundamentally about scaling out (by adding more drives or nodes) rather than scaling up (adding capacity by replacing a storage box with larger storage box). This means StorPool can scale independently by IOPS, storage space and bandwidth. There is no bottleneck or single point of failure. StorPool can grow without interruption and in small steps - a drive, a server and/or a network interface at a time. For more information, see Adding and removing nodes and Drive management.
High performance
StorPool combines the IOPS performance of all drives in the cluster and optimizes drive access patterns to provide low latency and handling of storage traffic bursts. The load is distributed equally between all servers through striping and sharding.
High availability and reliability
StorPool uses a replication mechanism that slices and stores copies of the data on different servers. For primary, high performance storage this solution has many advantages compared to RAID systems and provides considerably higher levels of reliability and availability. In case of a drive, server, or other component failure, StorPool uses some of the available copies of the data located on other nodes in the same or other racks significantly decreasing the risk of losing access to or losing data. For more information, see Redundancy.
Commodity hardware
StorPool supports drives and servers in a vendor-agnostic manner, allowing you to avoid vendor lock-in. This allows the use of commodity hardware, while preserving reliability and performance requirements. Moreover, unlike RAID, StorPool is drive agnostic - you can mix drives of various types, make, speed or size in a StorPool cluster. For more information, see Hardware requirements.
Co-existence with hypervisor software
StorPool can utilize repurposed existing servers and can co-exist with hypervisor software on the same server. This means that there is no dedicated hardware for storage, and growing an IaaS cloud solution is achieved by simply adding more servers to the cluster.
Compatibility
StorPool is compatible with 64-bit Intel and AMD based servers. We support all Linux-based hypervisors and hypervisor management software. Any Linux software designed to work with a shared storage solution such as an iSCSI or FC disk array will work with StorPool. StorPool guarantees the functionality and availability of the storage solution at the Linux block device interface.
CLI interface and API
StorPool provides an easy to use yet powerful command-line interface (CLI) tool for administration of the data storage solution. It is simple and user-friendly - making configuration changes, provisioning, and monitoring fast and efficient. For an introduction to CLI’s functionality, see CLI tutorial.
StorPool also provides a RESTful JSON API reference, and Python bindings exposing all the available functionality, so you can integrate it with any existing management system. This is the basis on which the StorPool integrations are built.
Reliable support
StorPool comes with reliable dedicated support:
Remote installation and initial configuration by StorPool’s specialists
24x7 support
Live software updates without interruption in the service
More information
The Installation and setup guide provides details on choosing hardware for your cluster, installing StorPool, and performing initial setup.
The Administration guide provides low-level details on how StorPool functions, and how to get the optimal performance from it.
The Operations guide provides details on activities related to ensuring StorPool is running well. This includes monitoring, maintenance, and best practices.
The Reference guide provides details on the command line interface (CLI) tools and the REST API provided by StorPool.
The StorPool integrations guide provides details on using StorPool with hypervisors and cloud management and orchestration tools.