Linux Fibre Channel SCSI Target using SCST

Fibre channel or Fiber Channel is also another way to present SCSI devices over a network medium using a complete different protocol suite then my previous article on iSCSI. With Fiber Channel transfer speeds and protocl delivery is much faster than iSCSI. The fundamental difference between the two is iSCSI uses TCP/IP protocol suite to deliver SCSI messages, and Fiber Channel uses Fiber channel to deliver SCSI message. This means that you will have to have network equipment that is Fiber Channel capable, such as Fiber Channel switches, Fiber Channel HBA (similar to TCP/IP NICs), etc. For the purposes of this article I will not go into how Fiber Channel works, or how it does it’s job of delivering SCSI messages two and from SCSI initiators and SCSI targets. This article will step through how to turn a Linux machine into a Fiber channle SCSI Target. There are a few things you will need to even attempt this:

A Linux machine running Kernel 3.2+
SCST and SCSTAdmin (see below steps)
A separate machine that will act as a Fiber channel initiator, this can be another Linux machine, or an ESX host, etc
At least two(2) Fiber channel HBA’s with one physical WWN port each installed in both machines
A OM2 or 3 Multi-mode Fiber cable with connectors capable of being used with the HBAs
Enough disk space to create a Virtual Disk so we may present it as a LUN
And lastly, some excitement!! you are about to enter the new world of Fiber Channel!!

Preparing the builds

Identify card Qlogic card

YOu may see these types of errors on boot or in your messages log.

dmesg | grep Fibre dmesg | Fiber

1
2

dmesg | grep Fibre
dmesg | Fiber

Download firmware and copy to firmware directory

wget http://ldriver.qlogic.com/firmware/ql2400_fw.bin
cp ql2400_fw.bin /lib/firmware/

1 2	wget http://ldriver.qlogic.com/firmware/ql2400_fw.bin cp ql2400_fw.bin /lib/firmware/

# preferred
wget http://ftp.us.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-qlogic_0.36+wheezy.1_all.deb
dpkg -i  firmware-qlogicf_0.36+wheezy.1_all.deb

# preferred

wget http://ftp.us.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-qlogic_0.36+wheezy.1_all.deb

dpkg -i firmware-qlogicf_0.36+wheezy.1_all.deb

Check if alreadyexists

ls /lib/firmware/ | grep ql*

1	ls /lib/firmware/ \| grep ql*

Remove the current qla2xxx module

rmmod qla2xxx echo blacklist qla2xxx >/etc/modprobe.d/blacklist-qla2xxx.conf update-initramfs -u reboot

1
2
3
4

rmmod qla2xxx
echo blacklist qla2xxx >/etc/modprobe.d/blacklist-qla2xxx.conf
update-initramfs -u
reboot

Check if module is running after reboot:

lsmod | grep ql*

1

lsmod | grep ql*

Download dependencies to build scst and Patch the kernel

Build Dependencies

apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev gcc libncurses5-dev linux-headers-$(uname -r) lsscsi patch subversion

1	apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev gcc libncurses5-dev linux-headers-$(uname -r) lsscsi patch subversion

Get Kernel and SCST Source:

Kernel Source:

mkdir /tmp/KernelSource cd /tmp/KernelSource apt-get source linux-source-3.2 #Make an easy symbolic link ln -s linux-3.2.51 ../Linux

1
2
3
4
5
6

mkdir /tmp/KernelSource
cd /tmp/KernelSource
apt-get source linux-source-3.2

#Make an easy symbolic link
ln -s linux-3.2.51 ../Linux

SCST Source:

cd /tmp/ svn co https://svn.code.sf.net/p/scst/svn/trunk scst

1
2

cd /tmp/
svn co https://svn.code.sf.net/p/scst/svn/trunk scst

Patching the Kernel Source pre-build:

Now we patch the kernelsource

cd Linux
patch -p1 < {your_scst_source_directory}/scst/kernel/scst_exec_req_fifo-3.2.patch
patch -p1 < {your_scst_source_directory}/scst/iscsi-scst/kernel/patches/put_page_callback-3.2.patch

cd Linux

patch -p1 < {your_scst_source_directory}/scst/kernel/scst_exec_req_fifo-3.2.patch

patch -p1 < {your_scst_source_directory}/scst/iscsi-scst/kernel/patches/put_page_callback-3.2.patch

Build the newly patched Kernel

Enabling the SCST Target Module Source

Before we run menuconfig to select the items to build for our new kernel, we have to replace the current qla2xxx module (pre-packaged with the linux source code we downloaded) with SCST qla2x00t target module. Reason being, the default qla2xxx is only an Initiator enabled. I could not figure out a way to manually switch the module to Target enabled it without rebuilding the module (which will will do below). So, we will rename the current qlax2xxx source directory and symbolically link the SCST qla2x00t module source directory respectively. This will allow in menuconfig to enable this module to be built during kernel compile.

mv {your_kernel_source}/drivers/scsi/qla2xxx {your_kernel_source}/drivers/scsi/qla2xxx_orig
ln -s {your_scst_source}/scst/qla2x00t {your_kernel_source}/drivers/scsi/qla2xxx

1 2	mv {your_kernel_source}/drivers/scsi/qla2xxx {your_kernel_source}/drivers/scsi/qla2xxx_orig ln -s {your_scst_source}/scst/qla2x00t {your_kernel_source}/drivers/scsi/qla2xxx

Enabling modules to be built with menuconfig

#Now we run menuconfig make menuconfig

1
2

#Now we run menuconfig
make menuconfig

Make sure using CFQ I/O scheduler
“Enable the block layer” –> “IO Schedulers” –> Default I/O scheduler

Change Preemption Model to Server Mode:
Processor type and features” –> Preemption Model –> No Forced Preemption (Server)

Last but not least make sure the SCST qla2xxx target mode is enabled and will be built.
“Device Drivers” –> “SCSI device support” –> “SCSI low level drivers” –> “Qlogic 2xxx target mode support”

NOTICE: You will not see this option if step 1-“Enabling the SCST Target Module Source” above was not performed.

Start the build (This may take some time, grab a beer!)

(Following Tom’s kernel build suggestions)

export CONCURRENCY_LEVEL=5  //for Quad Core Processors
make-kpkg clean
fakeroot make-kpkg --initrd --append-to-version=-scst-enabled kernel-image kernel-headers

export CONCURRENCY_LEVEL=5 //for Quad Core Processors

make-kpkg clean

fakeroot make-kpkg --initrd --append-to-version=-scst-enabled kernel-image kernel-headers

Two(2) .deb files will be created in the parent directory, image and headers. Install them.

dpkg -i linux-image-3.2.51-scst-enabled_3.2.51-scst-enabled-10.00.Custom_amd64.deb 
dpkg -i linux-headers-3.2.51-scst-enabled_3.2.51-scst-enabled-10.00.Custom_amd64.deb 
reboot #or coldreboot if debian

dpkg -i linux-image-3.2.51-scst-enabled_3.2.51-scst-enabled-10.00.Custom_amd64.deb

dpkg -i linux-headers-3.2.51-scst-enabled_3.2.51-scst-enabled-10.00.Custom_amd64.deb

reboot #or coldreboot if debian

Upon reboot you should see new GRUB options:

Build SCST.

cd {scst_source_directory} make 2release make all BUILD_2X_MODULE=y CONFIG_SCSI_QLA_FC=y CONFIG_SCSI_QLA2XXX_TARGET=y make all install

1
2
3
4

cd {scst_source_directory}
make 2release
make all
BUILD_2X_MODULE=y CONFIG_SCSI_QLA_FC=y CONFIG_SCSI_QLA2XXX_TARGET=y make all install
Enabling the new modules on boot

vi /etc/modules #ensure you have this scst qla2xxx_scst qla2x00tgt scst_vdisk scst_user scst_disk

1
2
3
4
5
6
7
8
9

vi /etc/modules
#ensure you have this
scst
qla2xxx_scst
qla2x00tgt
scst_vdisk
scst_user
scst_disk

reboot #or coldreboot if debian

1

reboot #or coldreboot if debian

NOTICE: A reboot is not necessary to get the modules loaded. You can manually load them 1-by-1 using modprobe {module name}. I like to be thorough and make sure they are loading on boot, just in case I ever have to reboot my Fibre Channel Targets will come back up.

Check to see if all modules loaded on startup:

lsmod | grep scst

1

lsmod | grep scst
Build SCSTadmin

cd {your_scst_source}/scstadmin make make install #check install scstadmin

1
2
3
4
5
6

cd {your_scst_source}/scstadmin
make
make install

#check install
scstadmin

Collecting current configuration: done. No valid operations specified.

1
2
3
4
5

Collecting current configuration: done.

No valid operations specified.

Create VDISKS and Configuring SCST and LUNs

Creating VDISKs

Use dd to create a image file filled of 0’s

cd /opt/RAIDMOUNT/LUNs dd if=/dev/zero of=LinuxLUN01.img bs=1M count=1000000 dd if=/dev/zero of=WindowsLUN01.img bs=1M count=1000000

1
2
3

cd /opt/RAIDMOUNT/LUNs
dd if=/dev/zero of=LinuxLUN01.img bs=1M count=1000000
dd if=/dev/zero of=WindowsLUN01.img bs=1M count=1000000

Make scst.conf

vi /etc/scst.conf

<pre><code>HANDLER vdisk_fileio {
	DEVICE LinuxDisk {
		filename /opt/RAIDMOUNT/LUNs/LinuxLUN04.img
	}
	DEVICE WindowsDisk {
		filename /opt/RAIDMOUNT/LUNs/WindowsLUN12.img
	}
}
TARGET_DRIVER qla2x00t {
	TARGET 21:00:00:1b:32:1b:d2:35 {
		HW_TARGET

		enabled 1
		rel_tgt_id 1

		GROUP host1 {

			LUN 4 LinuxDisk
			LUN 12 WindowsDisk

			INIITATOR 10:00:00:00:c9:63:94:9e	
			INITIATOR 10:00:00:00:c9:63:94:9f
		}
	}

	TARGET 21:01:00:1b:32:3b:d2:35 {
		HW_TARGET

                enabled 1
		rel_tgt_id 2
	
		GROUP host2 {
			LUN 4 LinuxDisk
			LUN 12 WindowsDisk

			INITIATOR 21:00:00:1b:32:13:e4:07
                        INITIATOR 21:01:00:1b:32:33:e4:07	
       		 }
	}
}

vi /etc/scst.conf

<pre><code>HANDLER vdisk_fileio {

DEVICE LinuxDisk {

filename /opt/RAIDMOUNT/LUNs/LinuxLUN04.img

}

DEVICE WindowsDisk {

filename /opt/RAIDMOUNT/LUNs/WindowsLUN12.img

}

TARGET_DRIVER qla2x00t {

TARGET 21:00:00:1b:32:1b:d2:35 {

HW_TARGET

enabled 1

rel_tgt_id 1

GROUP host1 {

LUN 4 LinuxDisk

LUN 12 WindowsDisk

INIITATOR 10:00:00:00:c9:63:94:9e

INITIATOR 10:00:00:00:c9:63:94:9f

}

TARGET 21:01:00:1b:32:3b:d2:35 {

HW_TARGET

enabled 1

rel_tgt_id 2

GROUP host2 {

LUN 4 LinuxDisk

LUN 12 WindowsDisk

INITIATOR 21:00:00:1b:32:13:e4:07

INITIATOR 21:01:00:1b:32:33:e4:07

}

TARGET — is the Port WWN of the HBA on our Linux SCSI Target
INITIATOR — is the Port WWN of the HBA on our SCSI Initiators
Group — is just a name to group together INITIATORS
LUN ID — Used to identify numerically a LUN uniquely, only up to 255 ( I believe), and these must be unique for each LUN per Target PWWN.
In summary of the config above, I am presenting both LinuxLUN04 and WindowsLUN12 to both host1 and host2

Issue a scstadmin reload

Issue a scstadmin -conf /etc/scst.conf
DO NOT do a service scst restart
Check Initiator (in this case a ESX host)

Sources:

Linux Fibre Channel SCSI Target using SCST

Preparing the builds

Identify card Qlogic card

Download firmware and copy to firmware directory

Remove the current qla2xxx module

Download dependencies to build scst and Patch the kernel

Get Kernel and SCST Source:

Patching the Kernel Source pre-build:

Build the newly patched Kernel

Enabling the SCST Target Module Source

Enabling modules to be built with menuconfig

Start the build (This may take some time, grab a beer!)

Build SCST.

Enabling the new modules on boot

Build SCSTadmin

Create VDISKS and Configuring SCST and LUNs

Creating VDISKs

Make scst.conf

Issue a scstadmin reload

Check Initiator (in this case a ESX host)

There are 2 comments left Go To Comment

Leave a Reply to thejimmahknows Cancle Reply

Recent Posts

Recent Comments