Maintenance mode

The storpool maintenance command is used to configure one or more nodes in a cluster in maintenance mode. The following checks with be performed prior to a node entering maintenance mode, to ensure there is no danger to the operations of the cluster:

  • If there will be any volume or snapshot with remaining replication 1 if the node goes down;

  • If the cluster’s quorum will go below a safety margin for available voting nodes.

Command parameters

You can perform maintenance-related operations on the command line using the storpool maintenance command with the following parameters:

  • set to start maintenance of a node.

  • list to see which nodes are in maintenance.

  • complete to complete the maintenance of a node.

Setting a node in maintenance mode

You should construct the command with which you start maintenance of a node in the following way:

  1. Start with storpool maintenance set.

  2. Add node, and after it add the ID of the node or here (for the current node).

  3. Add duration, and after it the planned duration of the maintenance; for example, 2h. After the specified time period expires the system would generate a corresponding alert, see Monitoring alerts.

  4. Add description, and after it a short text description of the reason to put the node in maintenance.

  5. Optionally, add one or more of the following parameters:

    maxNodesInMaintenance

    Maximum number of nodes simultaneously in maintenance.

    maxReplicationDecrease

    Maximum number not OK objects after the disk is ejected.

    minOkReplicas

    Minimum number of remaining OK copies after the disk is ejected.

    serverOnly

    When yes - skip mgmt/bridge service node availability checks. When no - check mgmt/bridge service node availability.

    votingSafetyMargin

    Minimum number of voting nodes not in maintenance above the minimum.

Here is an example:

# storpool maintenance set node 23 duration 10m description kernel_update
OK

The above will configure node ID 23 in maintenance state for 10 minutes and will configure the description to “kernel_update”.

Listing nodes in maintenance

To list the present nodes in maintenance:

# storpool maintenance list
------------------------------------------------------------
| nodeId | started             | remaining | description   |
------------------------------------------------------------
|     23 | 2020-09-30 12:55:20 | 00:09:50  | kernel_update |
------------------------------------------------------------

Completing maintenance

To complete a maintenance for a node:

# storpool maintenance complete node 23
OK

More information

All non-cluster threatening issues related to the node in maintenance mode will not be sent by the monitoring system to external entities. All alerts will still be received by StorPool support and will be classified as “under maintenance” internally, while the node is in maintenance mode.

Attention

Any alerts that are cluster threatening will still send super-critical alerts to both StorPool support and any other configured endpoint. For details, see Severity levels.

Consider that a full cluster maintenance mode is also available. For more information on how to do this with storpool mgmtConfig maintenanceState, see Cluster maintenance mode.