Fibre channel or Fiber Channel is also another way to present SCSI devices over a network medium using a complete different protocol suite then my previous article on iSCSI. With Fiber Channel transfer speeds and protocl delivery is much faster than iSCSI. The fundamental difference between the two is iSCSI uses TCP/IP protocol suite to deliver SCSI messages, and Fiber Channel uses Fiber channel to deliver SCSI message. This means that you will have to have network equipment that is Fiber Channel capable, such as Fiber Channel switches, Fiber Channel HBA (similar to TCP/IP NICs), etc. For the purposes of this article I will not go into how Fiber Channel works, or how it does it’s job of delivering SCSI messages two and from SCSI initiators and SCSI targets. This article will step through how to turn a Linux machine into a Fiber channle SCSI Target. There are a few things you will need to even attempt this:
- A Linux machine running Kernel 3.2+
- SCST and SCSTAdmin (see below steps)
- A separate machine that will act as a Fiber channel initiator, this can be another Linux machine, or an ESX host, etc
- At least two(2) Fiber channel HBA’s with one physical WWN port each installed in both machines
- A OM2 or 3 Multi-mode Fiber cable with connectors capable of being used with the HBAs
- Enough disk space to create a Virtual Disk so we may present it as a LUN
- And lastly, some excitement!! you are about to enter the new world of Fiber Channel!!
Preparing the builds
-
Identify card Qlogic card
YOu may see these types of errors on boot or in your messages log.
12dmesg | grep Fibredmesg | Fiber -
Download firmware and copy to firmware directory
12wget http://ldriver.qlogic.com/firmware/ql2400_fw.bincp ql2400_fw.bin /lib/firmware/OR
123# preferredwget http://ftp.us.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-qlogic_0.36+wheezy.1_all.debdpkg -i firmware-qlogicf_0.36+wheezy.1_all.debCheck if alreadyexists
1ls /lib/firmware/ | grep ql* -
Remove the current qla2xxx module
1234rmmod qla2xxxecho blacklist qla2xxx >/etc/modprobe.d/blacklist-qla2xxx.confupdate-initramfs -urebootCheck if module is running after reboot:
1lsmod | grep ql* -
Download dependencies to build scst and Patch the kernel
Build Dependencies
1apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev gcc libncurses5-dev linux-headers-$(uname -r) lsscsi patch subversion -
Get Kernel and SCST Source:
Kernel Source:
123456mkdir /tmp/KernelSourcecd /tmp/KernelSourceapt-get source linux-source-3.2#Make an easy symbolic linkln -s linux-3.2.51 ../LinuxSCST Source:
12cd /tmp/svn co https://svn.code.sf.net/p/scst/svn/trunk scst -
Patching the Kernel Source pre-build:
Now we patch the kernelsource
123cd Linuxpatch -p1 < {your_scst_source_directory}/scst/kernel/scst_exec_req_fifo-3.2.patchpatch -p1 < {your_scst_source_directory}/scst/iscsi-scst/kernel/patches/put_page_callback-3.2.patch
Build the newly patched Kernel
-
Enabling the SCST Target Module Source
Before we run menuconfig to select the items to build for our new kernel, we have to replace the current qla2xxx module (pre-packaged with the linux source code we downloaded) with SCST qla2x00t target module. Reason being, the default qla2xxx is only an Initiator enabled. I could not figure out a way to manually switch the module to Target enabled it without rebuilding the module (which will will do below). So, we will rename the current qlax2xxx source directory and symbolically link the SCST qla2x00t module source directory respectively. This will allow in menuconfig to enable this module to be built during kernel compile.
12mv {your_kernel_source}/drivers/scsi/qla2xxx {your_kernel_source}/drivers/scsi/qla2xxx_origln -s {your_scst_source}/scst/qla2x00t {your_kernel_source}/drivers/scsi/qla2xxx -
Enabling modules to be built with menuconfig
12#Now we run menuconfigmake menuconfigMake sure using CFQ I/O scheduler
“Enable the block layer” –> “IO Schedulers” –> Default I/O scheduler
Change Preemption Model to Server Mode:
Processor type and features” –> Preemption Model –> No Forced Preemption (Server)
Last but not least make sure the SCST qla2xxx target mode is enabled and will be built.
“Device Drivers” –> “SCSI device support” –> “SCSI low level drivers” –> “Qlogic 2xxx target mode support”
NOTICE: You will not see this option if step 1-“Enabling the SCST Target Module Source” above was not performed. -
Start the build (This may take some time, grab a beer!)
(Following Tom’s kernel build suggestions)
123export CONCURRENCY_LEVEL=5 //for Quad Core Processorsmake-kpkg cleanfakeroot make-kpkg --initrd --append-to-version=-scst-enabled kernel-image kernel-headersTwo(2) .deb files will be created in the parent directory, image and headers. Install them.
123dpkg -i linux-image-3.2.51-scst-enabled_3.2.51-scst-enabled-10.00.Custom_amd64.debdpkg -i linux-headers-3.2.51-scst-enabled_3.2.51-scst-enabled-10.00.Custom_amd64.debreboot #or coldreboot if debianUpon reboot you should see new GRUB options:
-
Build SCST.
1234cd {scst_source_directory}make 2releasemake allBUILD_2X_MODULE=y CONFIG_SCSI_QLA_FC=y CONFIG_SCSI_QLA2XXX_TARGET=y make all install -
Enabling the new modules on boot
123456789vi /etc/modules#ensure you have thisscstqla2xxx_scstqla2x00tgtscst_vdiskscst_userscst_disk1reboot #or coldreboot if debianNOTICE: A reboot is not necessary to get the modules loaded. You can manually load them 1-by-1 using modprobe {module name}. I like to be thorough and make sure they are loading on boot, just in case I ever have to reboot my Fibre Channel Targets will come back up.
Check to see if all modules loaded on startup:1lsmod | grep scst -
Build SCSTadmin
123456cd {your_scst_source}/scstadminmakemake install#check installscstadmin12345Collecting current configuration: done.No valid operations specified.
Create VDISKS and Configuring SCST and LUNs
-
Creating VDISKs
Use dd to create a image file filled of 0’s
123cd /opt/RAIDMOUNT/LUNsdd if=/dev/zero of=LinuxLUN01.img bs=1M count=1000000dd if=/dev/zero of=WindowsLUN01.img bs=1M count=1000000 -
Make scst.conf
12345678910111213141516171819202122232425262728293031323334353637383940414243vi /etc/scst.conf<pre><code>HANDLER vdisk_fileio {DEVICE LinuxDisk {filename /opt/RAIDMOUNT/LUNs/LinuxLUN04.img}DEVICE WindowsDisk {filename /opt/RAIDMOUNT/LUNs/WindowsLUN12.img}}TARGET_DRIVER qla2x00t {TARGET 21:00:00:1b:32:1b:d2:35 {HW_TARGETenabled 1rel_tgt_id 1GROUP host1 {LUN 4 LinuxDiskLUN 12 WindowsDiskINIITATOR 10:00:00:00:c9:63:94:9eINITIATOR 10:00:00:00:c9:63:94:9f}}TARGET 21:01:00:1b:32:3b:d2:35 {HW_TARGETenabled 1rel_tgt_id 2GROUP host2 {LUN 4 LinuxDiskLUN 12 WindowsDiskINITIATOR 21:00:00:1b:32:13:e4:07INITIATOR 21:01:00:1b:32:33:e4:07}}}TARGET — is the Port WWN of the HBA on our Linux SCSI Target
INITIATOR — is the Port WWN of the HBA on our SCSI Initiators
Group — is just a name to group together INITIATORS
LUN ID — Used to identify numerically a LUN uniquely, only up to 255 ( I believe), and these must be unique for each LUN per Target PWWN.
In summary of the config above, I am presenting both LinuxLUN04 and WindowsLUN12 to both host1 and host2 -
Issue a scstadmin reload
Issue a scstadmin -conf /etc/scst.conf
DO NOT do a service scst restart -
Check Initiator (in this case a ESX host)
Sources:
what version of linux were you running for this step through?
Debian 7, Linux Kernel 3.2.51 (recompiled with SCST) 64-bit