Updating already configured machines
Sometimes you might need to update machines that are already configured and run
the StorPool services. A possible solution for this scenario is to create a new
configuration and reboot the machine. However, often you won’t be able to
reboot, so storpool_cg offers a solution for ‘live’-migrating machines. It
is activated by the -M (--migrate) command line option.
When you have a configuration that you want to apply to a machine (you know with
what options you want to run storpool_cg) you have two options:
Run the
storpool_cgto createcgconfig.dfiles, and then reboot.Use
storpool_cg confwith the same options plus the-Moption, and let it try to apply it.
Attention
Note the following:
Before attempting a live migration,
storpool_cgwill run a series of checks to verify that it is safe to try the migration.Migrating machines with
storpool_cgis a pseudo-transactional procedure. If the migration process fails, a rollback procedure will be attempted to restore the initial machine state. The rollback operation is not guaranteed to succeed! Some ‘extreme’ conditions must have occurred for the rollback to fail, though.
You can use the migration in the following cases:
Changing slice limits
Enabling hardware acceleration
Adding and removing services
Adding and removing disks
Migrating to new-style configuration
Let’s look at the following machine:
$ storpool_cg print
slice: storpool.slice limit: 26G
slice: system.slice limit: 2G
slice: user.slice limit: 2G
socket:0
core:0 cpus:[ 0 1] -- nic | rdma
core:1 cpus:[ 2 3] -- server | block
core:2 cpus:[ 4 5] -- server_1 |
core:3 cpus:[ 6 7] -- server_2 | beacon
core:4 cpus:[ 8 9] --
core:5 cpus:[10 11] --
core:6 cpus:[12 13] --
core:7 cpus:[14 15] --
You can run a ‘dry run’ migration with -NM to see the desired configuration
and the steps the tool will take to achieve it. All other arguments of
storpool_cg conf can be used with -M, so (for example) if you need to
tweak the number of cores, you can still use cores=4.
$ storpool_cg conf -NM
W: NIC is expected to be on cpu 2
########## START SUMMARY ##########
slice: storpool limit: 26696M
subslice: storpool/common limit: 26696M
subslice: storpool/alloc limit: 0G
slice: system limit: 2G
slice: user limit: 2G
###################################
cpus for StorPool: [2, 3, 4, 5, 6, 7]
socket:0
core: 0 cpu: 0, 1
core: 1 cpu: 2, 3 <--- 2 - nic; 3 - rdma
core: 2 cpu: 4, 5 <--- 4 - server; 5 - server_1
core: 3 cpu: 6, 7 <--- 6 - server_2; 7 - block,beacon
core: 4 cpu: 8, 9
core: 5 cpu:10,11
core: 6 cpu:12,13
core: 7 cpu:14,15
########### END SUMMARY ###########
echo 2 > /sys/fs/cgroup/cpuset/storpool.slice/rdma/cpuset.cpus
echo 3 > /sys/fs/cgroup/cpuset/storpool.slice/server/cpuset.cpus
echo 4 > /sys/fs/cgroup/cpuset/storpool.slice/block/cpuset.cpus
echo 5 > /sys/fs/cgroup/cpuset/storpool.slice/server_1/cpuset.cpus
echo 2-7 > /sys/fs/cgroup/cpuset/storpool.slice/cpuset.cpus
echo 0-1,8-15 > /sys/fs/cgroup/cpuset/user.slice/cpuset.cpus
echo 0-1,8-15 > /sys/fs/cgroup/cpuset/system.slice/cpuset.cpus
echo 4 > /sys/fs/cgroup/cpuset/storpool.slice/server/cpuset.cpus
echo 3 > /sys/fs/cgroup/cpuset/storpool.slice/rdma/cpuset.cpus
echo 7 > /sys/fs/cgroup/cpuset/storpool.slice/block/cpuset.cpus
echo 26696M > /sys/fs/cgroup/memory/storpool.slice/memory.memsw.limit_in_bytes
echo 26696M > /sys/fs/cgroup/memory/storpool.slice/memory.limit_in_bytes
mkdir /sys/fs/cgroup/memory/storpool.slice/common
echo 1 > /sys/fs/cgroup/memory/storpool.slice/common/memory.use_hierarchy
echo 3 > /sys/fs/cgroup/memory/storpool.slice/common/memory.move_charge_at_immigrate
echo 26696M > /sys/fs/cgroup/memory/storpool.slice/common/memory.limit_in_bytes
mkdir /sys/fs/cgroup/memory/storpool.slice/alloc
echo 1 > /sys/fs/cgroup/memory/storpool.slice/alloc/memory.use_hierarchy
echo 3 > /sys/fs/cgroup/memory/storpool.slice/alloc/memory.move_charge_at_immigrate
echo 0G > /sys/fs/cgroup/memory/storpool.slice/alloc/memory.limit_in_bytes
echo 6143 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 6682 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 6692 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 6913 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 6926 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 6977 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 6987 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 7174 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 7185 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 7585 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
echo 7604 > /sys/fs/cgroup/memory/storpool.slice/common/cgroup.procs
When you are happy with the configuration and want to migrate to it, run the
migration without -N. Then check if everything is OK with storpool_cg
print -S and storpool_cg check.
$ storpool_cg conf --migrate
W: NIC is expected to be on cpu 2
$ storpool_cg print --slices
slice: storpool.slice limit: 26696M
subslice: storpool.slice/alloc limit: 0G
subslice: storpool.slice/common limit: 26696M
slice: system.slice limit: 2G
slice: user.slice limit: 2G
socket:0
core:0 cpus:[ 0 1] -- system user | system user
core:1 cpus:[ 2 3] -- storpool: nic | storpool: rdma
core:2 cpus:[ 4 5] -- storpool: server | storpool: server_1
core:3 cpus:[ 6 7] -- storpool: server_2 | storpool: beacon,block
core:4 cpus:[ 8 9] -- system user | system user
core:5 cpus:[10 11] -- system user | system user
core:6 cpus:[12 13] -- system user | system user
core:7 cpus:[14 15] -- system user | system user
$ storpool_cg check
M: ==== memory ====
W: memory:system.slice has more than 80% usage
Attention
One more thing you need to check after a migration is the output
of storpool_process reclassify -N -v. If it suggests moving
some processes to different cgroups than the ones they are
currently running in, you can do it by running just
storpool_process reclassify. Note that storpool_process
will make suggestions based on the current SP_{X}_CGROUPS
variables in the storpool.conf.