iSCSI
The StorPool iSCSI support is documented more extensively in the Setting iSCSI targets section; these are the commands used to configure it and view the configuration.
To set the cluster’s iSCSI base IQN iqn.2019-08.com.example:examplename:
# storpool iscsi config setBaseName iqn.2019-08.com.example:examplename
OK
Creating a portal group
To create a portal group examplepg used to group exported volumes for access by initiators using 192.168.42.247/24 (CIDR notation) as the portal IP address:
# storpool iscsi config portalGroup examplepg create addNet 192.168.42.247/24 vlan 42
OK
To create portal for the initiators to connect to (for example portal IP address 192.168.42.202 and StorPool’s SP_OURID 5):
# storpool iscsi config portal create portalGroup examplepg address 192.168.42.202 controller 5
OK
Note
This address will be handled by the storpool_iscsi process
directly and will not be visible in the node with normal instruments
like ip or ifconfig, check the Using iscsi_tool for these
purposes.
Registering an initiator
To define the iqn.2019-08.com.example:abcdefgh initiator that is allowed to connect from the 192.168.42.0/24 network (without authentication):
# storpool iscsi config initiator iqn.2019-08.com.example:abcdefgh create net 192.168.42.0/24
OK
To define the iqn.2019-08.com.example:client initiator that is allowed to connect from the 192.168.42.0/24 network and must authenticate using the standard iSCSI password-based challenge-response authentication method using the user username and the secret password:
# storpool iscsi config initiator iqn.2019-08.com.example:client create net 192.168.42.0/24 chap user secret
OK
Exporting a volume
To specify that the existing StorPool volume tinyvolume should be exported to one or more initiators:
# storpool iscsi config target create tinyvolume
OK
Note
Please note that changing the volume name after creating a target will not change the target name. Re-creating (unexport/re-export) the target will use the new volume name.
To actually export the StorPool volume tinyvolume to to the iqn.2019-08.com.example:abcdefgh initiator via the examplepg portal group (the StorPool iSCSI service will automatically pick a portal to export the volume through):
# storpool iscsi config export initiator iqn.2019-08.com.example:abcdefgh portalGroup examplepg volume tinyvolume
OK
Note
The volume will be visible to the initiator as IQN <BaseName>:<volume>
Using the same command without specifying an initiator will export the target to
all registered initiators and will be visible as the * initiator:
# storpool iscsi config export portalGroup examplepg volume tinyvolume
OK
# storpool iscsi initiator list exports
-----------------------------------------------------------------------------------------------------------------
| name | volume | currentControllerId | portalGroup | initiator |
-----------------------------------------------------------------------------------------------------------------
| iqn.2019-08.com.example:examplename:tinyvolume | test | 23 | examplepg | * |
-----------------------------------------------------------------------------------------------------------------
Getting iSCSI configuration
To view the iSCSI cluster base IQN:
# storpool iscsi basename
---------------------------------------
| basename |
---------------------------------------
| iqn.2019-08.com.example:examplename |
---------------------------------------
To view the portal groups:
# storpool iscsi portalGroup list
---------------------------------------------
| name | networksCount | portalsCount |
---------------------------------------------
| examplepg | 1 | 2 |
---------------------------------------------
To view the portals:
# storpool iscsi portalGroup list portals
--------------------------------------------------
| group | address | controller |
--------------------------------------------------
| examplepg | 192.168.42.246:3260 | 1 |
| examplepg | 192.168.42.202:3260 | 5 |
--------------------------------------------------
To view the defined initiators:
# storpool iscsi initiator list
---------------------------------------------------------------------------------------
| name | username | secret | networksCount | exportsCount |
---------------------------------------------------------------------------------------
| iqn.2019-08.com.example:abcdefgh | | | 1 | 1 |
| iqn.2019-08.com.example:client | user | secret | 1 | 0 |
---------------------------------------------------------------------------------------
To view the present state of the configured iSCSI interfaces:
iscsi interfaces list
--------------------------------------------------
| ctrlId | net 0 | net 1 |
--------------------------------------------------
| 23 | 2A:60:00:00:E0:17 | 2A:60:00:00:E0:17 |
| 24 | 2A:60:00:00:E0:18 | 2A:60:00:00:E0:18 |
| 25 | 2A:60:00:00:E0:19 | 2E:60:00:00:E0:19 |
| 26 | 2A:60:00:00:E0:1A | 2E:60:00:00:E0:1A |
--------------------------------------------------
Note
These are the same interfaces configured with SP_ISCSI_IFACE in
the order of appearance:
# storpool_showconf SP_ISCSI_IFACE
SP_ISCSI_IFACE=sp0,spbond1:sp1,spbond1:[lacp]
In the above output, the sp0 interface is net ID 0 and sp1 is
net ID 1.
To view the volumes that may be exported to initiators:
# storpool iscsi target list
-------------------------------------------------------------------------------------
| name | volume | currentControllerId |
-------------------------------------------------------------------------------------
| iqn.2019-08.com.example:examplename:tinyvolume | tinyvolume | 65535 |
-------------------------------------------------------------------------------------
To view the volumes currently exported to initiators:
# storpool iscsi initiator list exports
--------------------------------------------------------------------------------------------------------------------------------------
| name | volume | currentControllerId | portalGroup | initiator |
--------------------------------------------------------------------------------------------------------------------------------------
| iqn.2019-08.com.example:examplename:tinyvolume | tinyvolume | 1 | | iqn.2019-08.com.example:abcdefgh |
--------------------------------------------------------------------------------------------------------------------------------------
Getting active sessions
To list the presently active sessions in the cluster:
# storpool iscsi sessions list
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| id | target | initiator | portal addr | initiator addr | timeCreated | nopOut | scsi | task | dataOut | otherOut | nopIn | scsiRsp | taskRsp | dataIn | r2t | otherIn | t free | t dataOut | t queued | t processing | t dataResp | t aborted | ISID |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 23.0 | iqn.2020-04.com.storpool:autotest:s18-1-iscsi-test-hybrid-win-server2016 | iqn.1991-05.com.microsoft:s18 | 10.1.100.123:3260 | 10.1.100.18:49414 | 2020-07-07 09:25:16 / 00:03:54 | 209 | 89328 | 0 | 0 | 2 | 209 | | | 45736 | 0 | 2 | 129 | 0 | 0 | 0 | 0 | 0 | 1370000 |
| 23.1 | iqn.2020-04.com.storpool:autotest:s11-1-iscsi-test-hybrid-centos6 | iqn.2020-04.com.storpool:s11 | 10.1.100.123:3260 | 10.1.100.11:44392 | 2020-07-07 09:25:33 / 00:03:37 | 218 | 51227 | 0 | 0 | 1 | 218 | | | 25627 | 0 | 1 | 129 | 0 | 0 | 0 | 0 | 0 | 3d0002b8 |
| 24.0 | iqn.2020-04.com.storpool:autotest:s11-1-iscsi-test-hdd-centos6 | iqn.2020-04.com.storpool:s11 | 10.1.100.124:3260 | 10.1.100.11:51648 | 2020-07-07 09:27:27 / 00:01:43 | 107 | 424 | 0 | 0 | 1 | 107 | | | 224 | 0 | 1 | 129 | 0 | 0 | 0 | 0 | 0 | 3d0002b9 |
| 24.1 | iqn.2020-04.com.storpool:autotest:s18-1-iscsi-test-hdd-win-server2016 | iqn.1991-05.com.microsoft:s18 | 10.1.100.124:3260 | 10.1.100.18:49422 | 2020-07-07 09:28:22 / 00:00:48 | 43 | 39568 | 0 | 0 | 2 | 43 | | | 19805 | 0 | 2 | 128 | 0 | 0 | 1 | 0 | 0 | 1370000 |
| 25.0 | iqn.2020-04.com.storpool:autotest:s13-1-iscsi-test-hybrid-centos7 | iqn.2020-04.com.storpool:s13 | 10.1.100.125:3260 | 10.1.100.13:45120 | 2020-07-07 09:20:46 / 00:08:24 | 481 | 154086 | 0 | 0 | 1 | 481 | | | 78308 | 0 | 1 | 129 | 0 | 0 | 0 | 0 | 0 | 3d0000a8 |
| 26.0 | iqn.2020-04.com.storpool:autotest:s13-1-iscsi-test-hdd-centos7 | iqn.2020-04.com.storpool:s13 | 10.1.100.126:3260 | 10.1.100.13:43858 | 2020-07-07 09:22:52 / 00:06:18 | 369 | 147438 | 0 | 0 | 1 | 369 | | | 74883 | 0 | 1 | 129 | 0 | 0 | 0 | 0 | 0 | 3d0000a9 |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here, the fields are:
idIdentifier for the node and connection. The first part matches the
SP_OURIDof the node with thestorpool_iscsiservice running, and the second is the export number.targetThe target IQN.
initiatorThe initiator IQN.
portal addrThe portal group floating address and port.
initiator addrThe initiator address and port.
timeCreatedThe time when the session was created.
Initiator:
nopOutNumber of NOP-out requests from the initiator.
scsiNumber of SCSI commands from the initiator for this session.
taskNumber of SCSI Task Management Function Requests from the initiator.
dataOutNumber of SCSI Data-Out PDUs from the initiator.
otherOutNumber of non SCSI Data-Out PDUs sent to the target (Login/Logout/SNACK or Text).
ISIDThe initiator part of the session identifier, explicitly specified by the initiator during login.
Target:
nopInNumber of NOP-in PDUs from the target.
scsiRspNumber of SCSI response PDUs from the target.
taskRspNumber of SCSI Task Management Function Response PDUs from the target.
dataInNumber of SCSI Data-In PDUs from the target.
r2tNumber of Ready To Transfer (R2T) PDUs from the target.
otherInNumber of non SCSI Data-In PDUs from the target (Login/Logout/SNACK or Text).
Task queue:
t freeNumber of free task queue slots.
t dataOutWrite request waiting for data from TCP.
t queuedNumber of IO requests received ready to be processed.
t processingNumber of IO requests sent to the target to process.
t dataRespRead request queued for sending over TCP.
t abortedNumber of aborted requests.
Operations
To stop exporting the tinyvolume volume to the initiator with iqn iqn.2019-08.com.example:abcdefgh and the examplepg portal group:
# storpool iscsi config unexport initiator iqn.2019-08.com.example:abcdefgh portalGroup examplepg volume tinyvolume
OK
If a target was exported to all initiators (with *), not specifying an
initiator will unexport from all:
# storpool iscsi config unexport portalGroup examplepg volume tinyvolume
OK
To remove an iSCSI definition for the tinyvolume volume:
# storpool iscsi config target delete tinyvolume
OK
To remove access for the iqn.2019-08.com.example:client iSCSI initiator:
# storpool iscsi config initiator iqn.2019-08.com.example:client delete
OK
To remove the portal 192.168.42.202 IP address:
# storpool iscsi config portal delete address 192.168.42.202
OK
To remove portal group examplepg after all the portals have been removed:
# storpool iscsi config portalGroup examplepg delete
OK
Note
Only portal groups without portals may be deleted.
Using iscsi_tool
With the hardware accelerated iSCSI all traffic from and to the initiators is
handled by the storpool_iscsi service directly. For example, with the above
setup the addresses exposed on each of the nodes could be queried with
/usr/lib/storpool/iscsi_tool:
# /usr/lib/storpool/iscsi_tool
usage: /usr/lib/storpool/iscsi_tool change-port 0/1 ifaceName
usage: /usr/lib/storpool/iscsi_tool ip net list
usage: /usr/lib/storpool/iscsi_tool ip neigh list
usage: /usr/lib/storpool/iscsi_tool ip route list
To list the presently configured addresses:
# /usr/lib/storpool/iscsi_tool ip net list
10.1.100.0/24 vlan 1100 ports 1,2
10.18.1.0/24 vlan 1801 ports 1,2
10.18.2.0/24 vlan 1802 ports 1,2
To list the neighbours and their last state:
# /usr/lib/storpool/iscsi_tool ip neigh list
10.1.100.11 ok F4:52:14:76:9C:B0 lastSent 1785918292753 us, lastRcvd 918669 us
10.1.100.13 ok 0:25:90:C8:E5:AA lastSent 1785918292803 us, lastRcvd 178521 us
10.1.100.18 ok C:C4:7A:EA:85:4E lastSent 1785918292867 us, lastRcvd 178099 us
10.1.100.108 ok 1A:60:0:0:E0:8 lastSent 1785918293857 us, lastRcvd 857181794 us
10.1.100.112 ok 1A:60:0:0:E0:C lastSent 1785918293906 us, lastRcvd 1157179290 us
10.1.100.113 ok 1A:60:0:0:E0:D lastSent 1785918293922 us, lastRcvd 765392509 us
10.1.100.114 ok 1A:60:0:0:E0:E lastSent 1785918293938 us, lastRcvd 526084270 us
10.1.100.115 ok 1A:60:0:0:E0:F lastSent 1785918293954 us, lastRcvd 616948781 us
10.1.100.123 ours
[snip]
The above output includes also the portalGroup addresses residing on the node with the lowest ID in the cluster.
To list routing information:
# /usr/lib/storpool/iscsi_tool ip route list
10.1.100.0/24 local
10.18.1.0/24 local
10.18.2.0/24 local
Using iscsi_targets
The /usr/lib/storpool/iscsi_targets tool is a helper tool for Linux
based initiators, showing all logged in targets on the node:
# /usr/lib/storpool/iscsi_targets
/dev/sdn iqn.2020-04.com.storpool:autotest:s11-1-iscsi-test-hybrid-centos6
/dev/sdo iqn.2020-04.com.storpool:autotest:s11-1-iscsi-test-hdd-centos6
/dev/sdp iqn.2020-04.com.storpool:autotest:s11-2-iscsi-test-hybrid-centos6
/dev/sdq iqn.2020-04.com.storpool:autotest:s11-2-iscsi-test-hdd-centos6