iSCSI Target Failover using TGT on Linux

Available from RSF-1 version 1.11 onwards

RSF-1 has built in support for iSCSI target failover using the TGT iSCSI framework. Each pool/service can have its own TGT configuration which is handled seamlessly by RSF-1 on service start and failover.

Individual TGT configuration files are located within the pool so they migrate with the pool during failover.

Prerequisites

Install TGT on each cluster node:
debian/ubuntu
```
apt install tgt
```
Create a zvols directory in which to place individual zvols (not strictly necessary, but HAC recommended best practice):
```
zfs create pool1/zvols
```
For each target data source create a zvol block device to be used as backing store for the iSCSI targets:
```
zfs create -V 200M pool1/zvols/disk01
```

Note

It is possible to use a regular file as backing store. However this is not an approach we would recommend as, from a performance point of view, it introduces the unnecessary overhead of a filesystem layer for I/O operations and somewhat negates the performance advantage of using block level storage.

Target configuration

Once zvols have been created the next step is to configure iSCSI targets that utilize those zvols. A target configuration file is held in each pool. That configuration is specific to the zvols in that pool and should not include zvols from any other pool - each pool needs to have its own configuration.

Create a tgt config file named .tgt-ha.conf in a pools root directory, i.e. /<pool>/.tgt-ha.conf. For example:

# cat /pool1/.tgt-ha.conf
<target iqn.2023-09.com.hac.deb.target01:>
    <backing-store /dev/zvol/pool1/zvols/disk01>
    </backing-store>
</target>

Stop and start the service via the webapp to initially load the configuration into TGT.

Check the target is now configured by running the following command:

# tgtadm --mode target --op show
Target 1: iqn.2023-09.com.hac:deb.target01
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 210 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/zvol/pool1/zvols/disk01
            Backing store flags:
    Account information:
    ACL information:
        ALL

The target should now be discoverable by the client via the service VIP (here the VIP is configured as 10.6.19.21 and TGT is listening on the default port of 3260):

# iscsiadm --mode discovery --type sendtargets --portal 10.6.19.21:3260
10.6.19.21:3260,1 iqn.2023-09.com.hac:deb.target01

# iscsiadm -m node -T iqn.2023-09.com.hac:deb.target01 -l
Logging in to [iface: default, target: iqn.2023-09.com.hac:deb.target01, portal: 10.6.19.21,3260]
Login to [iface: default, target: iqn.2023-09.com.hac:deb.target01, portal: 10.6.19.21,3260] successful.

# lsscsi
[4:0:0:0]    storage IET      Controller       0001  -
[4:0:0:1]    disk    IET      VIRTUAL-DISK     0001  /dev/sdc

Additional Pools

When RSF-1 has additional pools configured into a service there are two approaches for the TGT configuration:

Create separate configuration files in each additional pool and include them in the main pools configuration. This is our recommended approach as it facilitates easier pool management. In this example pool1 is the main pool:

# cat /pool1/.tgt-ha.conf
<target iqn.2023-09.com.hac:deb.target01>
    <backing-store /dev/zvol/pool1/vol/disk01>
    </backing-store>
</target>
include /pool2/.tgt-ha.conf

# cat /pool2/.tgt-ha.conf
<target iqn.2023-09.com.hac:dlp.target03>
    <backing-store /dev/zvol/pool2/vol/v1>
    </backing-store>
</target>
<target iqn.2023-09.com.hac:dlp.target04>
    <backing-store /dev/zvol/pool2/vol/v2>
    </backing-store>
</target>

Declare all targets in the main pool configuration. For example:

# cat /pool1/.tgt-ha.conf
<target iqn.2023-09.com.hac:deb.target01>
    <backing-store /dev/zvol/pool1/vol/disk01>
    </backing-store>
</target>
<target iqn.2023-09.com.hac:dlp.target03>
    <backing-store /dev/zvol/pool2/vol/v1>
    </backing-store>
</target>
<target iqn.2023-09.com.hac:dlp.target04>
    <backing-store /dev/zvol/pool2/vol/v2>
    </backing-store>
</target>