Drives

You can use the /etc/storpool.conf configuration file to define a specific driver to be used for NVMe SSD drives, a group owner for the StorPool devices, and other drive-related options.

Exclude disks globally or per server instance

A list of paths to drives to be excluded at instance boot time:

SP_EXCLUDE_DISKS=/dev/sda1:/dev/sdb1

Can also be specified for each server instance individually:

SP_EXCLUDE_DISKS=/dev/sdc1
SP_EXCLUDE_DISKS_1=/dev/sda1

Disable drive ejection

Locally disable ejecting disks or journals on exceeding average request latency thresholds. Useful if the cluster cannot get up due to a incorrect maximum latencies configuration. Default: 0 (drive ejection is enabled)

SP_DISABLE_LATENCY_EJECT=0

Group owner for the StorPool devices

The system group to use for the /dev/storpool and /dev/storpool-byid/ directories and the /dev/sp-* raw disk devices:

SP_DISK_GROUP=disk

Permissions for the StorPool devices

The access mode to set on the /dev/sp-* block devices:

SP_DISK_MODE=0660

Used on the client. When a volume is attached a block device is created automatically.

Mirror directory

If set, a directory containing device nodes with the same names as the volume/snapshot names exposed in /dev/storpool so that it may be (for example) bind-mounted in a container as a whole. Note that the directory must already exist! Default: unset

SP_MIRROR_DIR=

Mirror directory offset

A number to add to the owner ID and group ID of the devices in the mirror directory so that they may be accessed by processes running in an unprivileged container. Default: 0

SP_MIRROR_OWNER_OFFSET=0

NVMe SSD drives

The storpool_nvmed service automatically detects all initialized StorPool devices, and attaches them to the configured SP_NVME_PCI_DRIVER.

To configure a driver for storpool_nvmed that is different than the default storpool_pci, use the SP_NVME_PCI_DRIVER option:

SP_NVME_PCI_DRIVER=vfio-pci

The vfio-pci driver requires the iommu=pt option for both Intel/AMD CPUs and the intel_iommu=on option in addition for Intel CPUs on the kernel command line.

The ability to use the SP_NVME_PCI_DRIVER option is available starting with the 19.2 revision 19.01.1795.5b374e835 release.

Resetting stuck NVMes

Some NVMe devices occasionally stop responding. While resetting the PCI device sometimes resolves the issue, not all hardware handles this correctly, leading to server crashes and ever weirder behaviors.

You can use the SP_NVMED_RESET_INTEL_NVMES option to set whether the nvmed service should try resetting stuck Intel NVMe drives. Default: (empty)

SP_NVMED_RESET_INTEL_NVMES=