StorPool 19.1 Change Log

Release 19.2, more details here.

More details for the changes are available at Release Notes

19.1 revision 19.01.1628.1b627d0

Released: 07 Dec 2020

  • Now collecting additional configuration data in storpool_stat for additional per-host monitoring checks, includes lldpd as dependency.

  • Adds Volumecare version 1.18 that includes support for multiple sub-clusters sending to a backup location.

  • Includes improvements and and fixes in internal tools, core and services.

  • Adds compression and proxy authorization configuration support options for the monitoring collection scripts.

  • Adds the SP_MIRROR_DIR option for containerized workloads.

  • Adds support for the Micron 9300 MTFDHAL3T2TDR NVMe drive.

  • Adds compression support for monitoring data collection scripts.

19.1 revision 19.01.1548.00e5a5633

Released: 27 Nov 2020

  • Adds fix for storpool_stat for handling statistics for unnamed volumes.

  • Adds support for Samsung PM173X NVMe drives.

  • Adds fix for iSCSI re-assign targets implementation to retry on transaction reset.

  • Improves storpool_stat send performance after longer periods of no connectivity

  • Adds storpool_initidks –no-notify option for easier StorPool disk attributes changes.

  • Adds reliability improvements in the in-server disk tester implementation

  • Adds an additional helper option to the iface-genconf tool (Auto-interface configuration)

  • Adds fix for disk not getting ejected by the server after a drive was re-balanced out.

  • Adds workaround for a bug in the i40e driver in CentOS6 based NICs (hardware acceleration)

19.1 revision 19.01.1511.0b533fb

Released: 28 Oct 2020

  • Adds fixes for idle service VF resets with ixgbe driver based NICs.

  • Adds fixes for volumecare tags handling with unnamed volumes.

  • Adds initial support for CentOS 8.2 installations.

  • Adds tools for setting up volume overrides for many volumes from the same parent use case.

  • Increases the listen backlog for the storpool_mgmt and storpool_bridge services.

19.1 revision 19.01.1468.90a9873

Released: 16 Oct 2020

  • Adds support for Micron 9300 NVMe.

  • Adds workaround for firmware issues with bnxt_en NICs (detect and reset stuck VF).

  • Improves handling of LACP mode i40e based NICs.

  • Fixes an issue with handling old-style iSCSI reservations used by VMWare 4.x and 5.x.

  • Fixes aborts in storpool_iscsi with bnxt_en driver when a large ICMP packet is received.

  • Improves StorPool API operation with large number of volumes and snapshots with multicluster environment.

  • Adds more granular flags for snapshot dematerialization status.

  • Adds fixes for iSCSI sessions list (known as empty target/initiator in CLI).

  • Adds fix for CLI not showing unnamed volumes name in attachments list.

  • Removes deprecated options from storpool.conf.example.

  • Includes updates and fixes in internal tools.

19.1 revision 19.01.1413.b807f92

Released: 26 Sep 2020

  • Updated dependencies when MLNX OFED is used.

  • Balancer fix for cases where overrides are enabled.

  • Adds bridge multicluster stability fixes.

  • Updates python bindings with missing functionality available in the present API.

  • Adds diagnostic features for the /usr/lib/storpool/monitor utility.

  • Adds handling for misbehaving i40e based NICs that could lead to aborted services.

  • Adds updates and fixes in internal tools.

19.1 revision 19.01.1376.ce07826

Released: 21 Aug 2020

  • Adds volumecare support for unnamed volumes.

  • The in-server disk tester now sends FUA writes during tests (except for –no-fua y disks).

  • Improved performance when relocating data from SSD or NVMe disks

  • Adds handling in api-ip for some obscure kernel 4.18 behavior leaving leftover routes.

  • Adds a fix for API to not show deleted snapshots addressed by global ID.

  • Adds a fix for volumeMoveRemote API call now properly handling IOPS and bandwidth limits.

  • Includes updates and fixes in internal tools.

19.1 revision 19.01.1357.39c014c

Released: 12 Aug 2020

  • Fixes a bug in handling old-style iSCSI reservations used by VMWare 4.x and 5.x

  • Adds storpool_volumecare as an installable module (More on storpool_volumecare here)

  • Adds support for Micron 7300 Pro NVMe

  • Updates and improvements in internal tools

19.1 revision 19.01.1346.dd68fa2c6

Released: 04 Aug 2020

  • Cluster volume/snapshot limit bumped to 60k volumes/snapshots.

  • StorPool API now automatically fails over in case its address vanishes for any reason.

  • The storpool_cg tool now properly handles disks with custom objects and/or entries counts.

  • Now automatically patching /etc/kdump.conf to blacklist non-important but memory intensive drivers.

  • New default TCP sysctl settings for better throughput performance for bridge enabled nodes.

  • Updated installation helper scripts handling better latest rpm/deb packages and other improvements.

19.1 revision 19.01.1293.cfcb869

Released: 11 Jul 2020

  • Improvements in storpool_initdisk tool - exit code 0 on --nvme-smart, added --json output

  • Adds spopenstack in the main source tree

  • Initial CentOS 8 build support

  • Major installation package system changes - new implementation of most core installation tools, all service files are now handled as rpm/deb packages

  • Critical fix for all systems running with hard disk drives, resiliency on power outages.

19.1 revision 19.01.1217.1635af7

Released: 02 Jul 2020

  • Improves handling of lost packets in iSCSI connections

  • Adds full cluster maintenance mode monitoring improvements

  • Server now automatically tests and returns disks on transient failures

  • Improvement in multicluster - volume create/update flags now mandatory for MultiCluster API endpoint, adds VolumeRevert operation

  • Improvement in multicluster - volume exports and remote volumes can now be listed

  • Performance improvements - more efficient bridge and API interoperability

  • Performance improvements in dematerialization

  • Adds CLI improvements - iSCSI list interfaces

  • Updates cgtool defaults - the system.slice is now with 4G limit by default

  • Adds minor fixes and improvements in balancer, build and automation tools

19.1 revision 19.01.1108.02703b8c5

Released: 04 Jun 2020

  • storpool_bridge performance improvements

  • Oracle Linux support

  • Kernel 5.4 support

  • Increase limit of total volumes and snapshots to 45000

  • Better support for LACP hashing handling

19.1 revision 19.01.1025.0baac06a6

Released: 04 Jun 2020

  • Maintenance mode

  • Support for volume creation with global IDs instead of volume names, see global ID volumes

  • Revert to snapshot functionality

  • NIC initialization and configuration is now non-blocking, see non-blocking NIC configuration

  • Availability of information for active iSCSI sessions

  • Acceleration support for virtio-net based adapters

  • Add support for a global active operations list

  • storpool_initdisk now has extra validation of flags and devices

  • reuseServer implicit on a down disk in small clusters, more on this here

19.1 revision 19.01.878.7b1f83e3d

Released: 04 Jun 2020

  • Infrastructure to allow customers to access StorPool packages

  • Tools to help with installing new modules and services

  • Add bridge buffer size options (location update command), see also Bridge throughput performance

  • Fix for sending monitoring data in blackholed IPv6 networks

19.1 revision 19.01.759.024d1bd

Released: 18 Mar 2020

  • iSCSI TCP fixes and optimizations

  • Improvements in handling of I/O errors for NVMe devices

  • Compatibility improvement in Python bindings: fix the type of the allowRemoteExported field so that it’s not always sent to the StorPool API

19.1 revision 19.01.742.47c6e9c

Released: 06 Mar 2020

  • Major improvement in aggregation performance with a significantly more efficient internal trigger

  • Improvements in dematerialization with mixed server and controller versions in the same cluster

  • Various fixes and improvements in the in-iSCSI TCP implementation

  • Performance improvements in the storpool_controller service

  • Fix for thread priorities in some services that were leading to timeouts and service restarts

19.1 revision 19.01.719.c40dd8c

Released: 26 Feb 2020

  • Various improvements/fixes in multicluster, more on mc here

  • Various iSCSI fixes and optimizations

  • A better handling of TRIMs over iSCSI

  • Improvements in balancer tools, handling some edge cases

  • Various other improvements in internal tools and services

19.1 revision 19.01.544.39e62dbad

Released: 13 Jan 2020

  • Adds setirqaff validation option

  • Adds a required default kernel configuration option for improved performance with known supported storage controllers

  • Improved storpool_bridge performance under constrained CPU and network bandwidth conditions

  • Adds colorless output for the storpool_tree utility with flags for volumes and snapshots

19.1 revision 19.01.496.6d2c5bf83

Released: 03 Dec 2019

  • Removes excessive debug print logging

  • Improved kdump setup tooling for Debian based systems

  • Support for larger set of switch hash functions for LACP and ECMP

  • Updates in fio testing framework scripts, closely following latest monitoring alerting

19.1 revision 19.01.385.fc63315ef

Released: 19 Nov 2019

  • Adds support for monitoring through IPv6 only hosts.

  • Adds hardware acceleration support for BCM57412 NetXtreme-E 10Gb Ethernet Controllers.

  • Adds improved stability of quorum keeping service in unstable network conditions.

  • Adds stability improvements in the quorum keeping service in unstable networking conditions.

  • Updates the detection for hardware accelerated NICs for the bnxt_en driver in the cgroup configuration helper tools.

  • Adds an API fix and improvements related to multicluster support with CloudStack deployments, more on mc here

  • Adds expansion support for Kubernetes integration.

19.1 revision 19.01.375.684a0da12

Released: 07 Nov 2019

  • Adds fix for the CLI when a volume is created without a template.

  • Improvements in logging, handles unexpected verbosity.

  • Improvements in statistics collection daemons, safer methods for data collection.

  • Improvements for stability and edge cases handling in API.

  • Improvements in storpool_cg for converged nodes, single-thread hosts migration fixes.

  • Improvements in monitoring collection scripts, now collecting and handling more data.

19.1 revision 19.01.355.896c5ebaf

Released: 23 Oct 2019

  • Improvements in API performance when balancer overrides are in place.

  • Improvements for iSCSI when persistent reservations (MicroSoft CSV) are being used with target failover.

  • Improvements for aggregation logic during recovery.

  • Adds change in CLI for template propagate, now stating propagate is mandatory on template changes.

  • Adds fix for removing the reuseServer flag for volumes/snapshots previosly applied with reuseServer

  • Adds an automatic disabling through vf-udevhelper for the internal i40e based NICs LLDP listener.

  • Adds an update storpool_daemon with systemd, now no longer forking and other smaller improvements.

  • Adds more variables reported by the server_stat tool, visible also in the collected statistics for the cluster.

  • Adds stability fixes in the storpool_iscsi service.

  • Adds performance fixes with improved read latency for HDD pools.

  • Adds default configuration changes for internal command output timeouts when collecting periodic reports.

  • Improvements in storpool_cg for server cache autodetection during cgroup config migration, various other improvements.

19.1 revision 19.01.318.daa3c5938

Released: 07 Oct 2019

  • Adds CLI support for multicluster mode, more on mc here

  • Improvements for iSCSI with LACP bonds, unrealiable network, etc.

  • Adds fixes in python bindings

  • Improvements in cgtool chain

  • Improvements in kubernetes integration code

  • Improvements in API responsiveness during dematerialization

19.1 revision 19.01.301.f1b25e7

Released: 24 Sep 2019

  • Enables setting the SMP affinity of MegaRAID controllers

  • Adds support for CentOS 7.7 (kernel 3.10.0-1062.1.1.el7.x86_64)

  • Adds support for mixed routed/switched networks for iSCSI

  • Adds various storpool_iscsi improvements and fixes

  • Various other improvements in internal tools and services

19.1 revision 19.01.271.de5921845

Released: 12 Sep 2019

  • Improvements in the irq pinning tools for better CPU congestion avoidance on storage service CPU cores

  • Improvements in internal tools for stress testing drives

  • Adds support for non-jumbo frames routed networks

  • Adds support for hardware or no sleep options for bridge service and cgroup configuration tools

  • Adds support for upgrading 18.02 clusters with iSCSI to accelerated storpool_iscsi

  • Adds various storpool_iscsi improvements and fixes

  • Adds initial support for storpool_iscsi persistent reservations

19.1 revision

Released: 10 Sep 2019

  • Adds support for 4.18 line of kernels

  • Improvements in python-bindings, monitoring services, cgroup tools

  • Various other improvements in internal tools and services

19.1 revision 19.01.199.b98e9d4

Released: 29 Aug 2019

  • Adds priority based recovery for better handling of remote bridge recovery operations

  • Adds tools for easier upgrade from 18.02 with multi site setup

  • Adds common configuration for all supported hardware accelerated NIC drivers

  • Improvements in the tooling for enabling required hardware acceleration settings

19.1 revision

  • Adds support for multiple clusters in the same location

  • Read operations now can be forwarded through the active bridge connection for not fully recovered snapshots

  • Adds support for 4.20 and 5.0 kernel line

  • Adds significant protocol improvements for small block size and low IO queue depth read/write operations latency

  • Adds significant performance improvements for storpool_iscsi, now running only on addapters with support for hardware acceleration (i40e, ixgbe, mlx4_ib, mlx5_ib drivers)

  • Adds support for iSCSI on layer 3 only networks (routed networks)

  • Adds support for hardware accelerated bnxt driver based Broadcom adapters

  • Adds backwards compatibility with pre 19.01 release versions

  • Fixes AMD EPYC v.1 hardware sleep latency issues


  • Adds fix for storpool_beacon service backup nets detection with single-vlan and misconfigured network

  • Adds support for Samsung PM981 NVMe drives

  • Adds support for up to 7 server instances on a single node

  • Improves snapshot dematerialization without impact processing of other API calls

  • Adds fix for snapshot space miscalcuated gains for snapshots with very small amount of data

  • The CLI now properly shows all units in KiB, MiB, GiB to prevent any confusion

  • Adds a new service storpool_abrtsync, which will automatically send reports from aborted services (repsync deprecated and removed on the first run of storpool_abrtsync)

  • Handles per-host alerts in storpool_stat for processes in the root cgroup and unreachable API (for nodes configured to access it)

  • Consolidates most cron tasks in a single cron job handled by the storpool_periodic tool

  • Renames the storpool_initdisk’s’ --force option to the unabmiguous --wipe-all-data

  • Includes various other improvements in the storpool_server, storpool_beacon services, python bindings, cgroup setup tools and in the disk stress testing tool


  • Fixes in automation for rebasing long volume-snapshot chains

  • Speed up TRIM operations for volumes exposed through iSCSI, now ~600GB/s instead of 2-12GB/s

  • Adds SP_PREFERRED_PORT option, enabling active-backup redundant networking

  • Improvements in relocator in aggregate heavy enviroments, such as HDD-only placement templates

  • Improvements in balancer tools when mixing drives in the same placement group


  • Add storpool_process tool for listing and reclassifying processes in their intended cgroups

  • Add the storpool_cg tool for creating cgroups based on predefined config or by autodetecting settings based on installed services and hardware

  • The monitor collector tool now detects the SSL CA list location

  • The multi-server-helper tool now supports configurations with selectable number of SSD only instances

  • Fix for the storpool_initdisk tool when the underlying devices are with a long PCI address

  • Improvements in storpool_stat collecting service, now gathering data for services network traffic and iostat for all drives, including system ones

  • Fixed inventory collection not updating the latest kernel booted


  • The storpool_initdisk tool now handles NVMe journals for hard disk drives

  • The storpool_initdisk tool now lists empty NVMe partitions

  • Fix storpool_controller service might end up in storpool.slice cgroup on systemd daemon-reload

  • Add support for 4.19+ & OFED in the storpool_rdma kernel module

  • Fix templates status output when the same placement group is used more than once

  • Drops support for Ubuntu 14.04 and Debian 8 distributions

  • Various other improvements in internal tools and services


  • Workaround for bnx2x based NICs getting in partial reset when large-receive-offload is disabled

  • Improvements in TRIM performance on top of a StorPool volume

  • Improvements in recovery times with remote backups

  • Python bindings python3 support

  • Added proxy support for sending monitoring and statistics data

  • Various other improvements in internal tools and services


  • Support for 4k native NVMe devices

  • Support for Intel Optane drives

  • Support for Micron 9200MAX NVMe

  • Support for AMD EPYC hardware sleep

  • Support for Intel X550 in hardware accelerated mode with ixgbe driver

  • Support for MTU 1500 with bnx2x and ixgbe based NICs

  • Updates in tools for automatic hardware acceleration configuration

  • Updated on-boot hugepages reservation mechanism

  • Performance and stability improvements in API service

  • Updated ixgbe driver based acceleration to evade issues with in-kernel driver

  • Updated ixgbe driver based acceleration with support for Debian 9 based nodes

  • Added Ubuntu 18.04 LTS support

  • Updates in statistics gathering - support for sending to secondary databases, cgroups memory statistics, various reliability and performance improvements

  • Updated dependencies

  • Various reliability and stability improvements in internal tools and services


  • Support for Kubernetes

  • Support for ARM64 architecture

  • Support for POWER architecture

  • Improvements in the installation procedure, VF helper included

  • hardware acceleration for NICs with bnx2x driver

  • Initial support for Alpine Linux

  • New API call to quickly return just the status of all volumes

  • Changes in the way data on the underlying drives is aggregated in a more efficient manner

  • New hardware support - Micron 9200 NVMe

  • Native 4k drives support

  • Various improvements in internal tools, monitoring and services


  • Add support for Samsung PM1725a NVMe

  • Add new tool, transfer_status to show the status of the currently running bridge transfers

  • Add new tool, sDump to print network stats in human-readable format

  • Fix for data transfer issues caused by the kernel remapping some of StorPool’s memory

  • Add preliminary support for Ubuntu 18.04

  • Work-around for kernel crashes caused by bugs in the ixgbe kernel driver when using acceleration

  • Add stats collection for per-CPU statistics, to be available in


  • Add additional logging for storpool_bridge service

  • Higher default parallelism for the storpool_bridge transfers

  • Lower CPU usage, higher efficiency for storpool_block during TRIM operations

  • Exit when the support for hardware sleep is not available and hardware sleep is requested

  • Added additional network statistics through the use of the statsDump tool and storpool_ping tools, added netlatency tool to quickly measure and report the network latency from one of the nodes to the other participants in the cluster

  • Added support for storpool_bd kernel module with 4.10+ kernels and storpool_rdma with 4.15+ kernels

  • storpool_server/storpool_initdisk/storpool_nvmed now support hotplug NVMe devices

  • Various other reliability fixes and improvements

  • No longer ignore snapshots being deleted when re-basing long snapshot-volume chains

  • Fix a bug in handling aborted iSCSI operations in storpool_iscsi service

  • Fix in storpool_server to not forget to delete moved snapshot objects

  • Add support for centos 7.5 in the storpool_rdma kernel module

  • Add support for 4.14+ & OFED in the storpool_rdma kernel module

  • Add support for 4.12+ in the storpool_pci kernel module

  • Improve some of the diagnostic and monitoring tools

  • Make the “recovery” state of a task clearly visible in the output of storpool task list

  • Disable check of BBU status for genreport

  • Fix service name and symlinks on CentOS 6 for the storpool_controller service

  • Fix installing dependency packages on Ubuntu 14.04

  • Fix the installation of the CentOS 6 cgmove init script

  • Remove “reload” for init scripts, as it’s not supported by the StorPool services and actually does “restart”

  • Add /usr/lib/storpool/get_tech_support - Collects useful information to be sent to support for initial troubleshooting in case of no access to the cluster.

  • Use port 443 (instead of 8086) in the storpool_stat service to get through firewalls more easily

  • Let the StorPool CLI prompt be customized using the SP_CLI_PROMPT variable

  • Add support for GCC 7

  • Strip the zero VLAN tag in the storpool_rdma module

  • Multiple fixes and improvements on the server and bridge services

  • Remove log entries for recovered objects (prevents syslog flooding)

  • Handle the new fio 3.1+ log format in the disk_tester tool

  • Improve the fio test scripts

  • Run the mgmt and bridge services in a separate memory cgroup - mgmt.slice

  • Add the SP_CPU_DMA_LATENCY variable to the storpool.conf.example file (more info at and

  • Add a check for the status of the BBU/Cache Vault of RAID controllers

  • Improve repsync service

  • Add a tool for filtering git logs

  • Add a tool for rotating snapshots on number of volumes

  • Add a tool for to quickly check the network latency to other nodes in the cluster (uses addresses in /etc/storpool.conf)