Linux Fibre Channel SCSI Target using SCST

Fibre channel or Fiber Channel is also another way to present SCSI devices over a network medium using a complete different protocol suite then my previous article on iSCSI. With Fiber Channel transfer speeds and protocl delivery is much faster than iSCSI. The fundamental difference between the two is iSCSI uses TCP/IP protocol suite to deliver SCSI messages, and Fiber Channel uses Fiber channel to deliver SCSI message. This means that you will have to have network equipment that is Fiber Channel capable, such as Fiber Channel switches, Fiber Channel HBA (similar to TCP/IP NICs), etc. For the purposes of this article I will not go into how Fiber Channel works, or how it does it’s job of delivering SCSI messages two and from SCSI initiators and SCSI targets. This article will step through how to turn a Linux machine into a Fiber channle SCSI Target. There are a few things you will need to even attempt this:

  • A Linux machine running Kernel 3.2+
  • SCST and SCSTAdmin (see below steps)
  • A separate machine that will act as a Fiber channel initiator, this can be another Linux machine, or an ESX host, etc
  • At least two(2) Fiber channel HBA’s with one physical WWN port each installed in both machines
  • A OM2 or 3 Multi-mode Fiber cable with connectors capable of being used with the HBAs
  • Enough disk space to create a Virtual Disk so we may present it as a LUN
  • And lastly, some excitement!! you are about to enter the new world of Fiber Channel!!

Preparing the builds

  1. Identify card Qlogic card

    YOu may see these types of errors on boot or in your messages log.

    dmesg | grep Fibre
    dmesg | Fiber

  2. Download firmware and copy to firmware directory

    wget http://ldriver.qlogic.com/firmware/ql2400_fw.bin
    cp ql2400_fw.bin /lib/firmware/

    OR

    # preferred
    wget http://ftp.us.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-qlogic_0.36+wheezy.1_all.deb
    dpkg -i  firmware-qlogicf_0.36+wheezy.1_all.deb

    Check if alreadyexists

    ls /lib/firmware/ | grep ql*
  3. Remove the current qla2xxx module

    rmmod qla2xxx
    echo blacklist qla2xxx >/etc/modprobe.d/blacklist-qla2xxx.conf
    update-initramfs -u
    reboot

    Check if module is running after reboot:

    lsmod | grep ql*
  4. Download dependencies to build scst and Patch the kernel

    Build Dependencies

    apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev gcc libncurses5-dev linux-headers-$(uname -r) lsscsi patch subversion
  5. Get Kernel and SCST Source:

    Kernel Source:

    mkdir /tmp/KernelSource
    cd /tmp/KernelSource
    apt-get source linux-source-3.2
    
    #Make an easy symbolic link
    ln -s linux-3.2.51 ../Linux

    SCST Source:

    cd /tmp/
    svn co https://svn.code.sf.net/p/scst/svn/trunk scst
  6. Patching the Kernel Source pre-build:

    Now we patch the kernelsource

    cd Linux
    patch -p1 < {your_scst_source_directory}/scst/kernel/scst_exec_req_fifo-3.2.patch
    patch -p1 < {your_scst_source_directory}/scst/iscsi-scst/kernel/patches/put_page_callback-3.2.patch

Build the newly patched Kernel

  1. Enabling the SCST Target Module Source

    Before we run menuconfig to select the items to build for our new kernel, we have to replace the current qla2xxx module (pre-packaged with the linux source code we downloaded) with SCST qla2x00t target module. Reason being, the default qla2xxx is only an Initiator enabled. I could not figure out a way to manually switch the module to Target enabled it without rebuilding the module (which will will do below). So, we will rename the current qlax2xxx source directory and symbolically link the SCST qla2x00t module source directory respectively. This will allow in menuconfig to enable this module to be built during kernel compile.

    mv {your_kernel_source}/drivers/scsi/qla2xxx {your_kernel_source}/drivers/scsi/qla2xxx_orig
    ln -s {your_scst_source}/scst/qla2x00t {your_kernel_source}/drivers/scsi/qla2xxx
  2. Enabling modules to be built with menuconfig

    #Now we run menuconfig
    make menuconfig

    Make sure using CFQ I/O scheduler
    "Enable the block layer" --> "IO Schedulers" --> Default I/O scheduler

    Change Preemption Model to Server Mode:
    Processor type and features" --> Preemption Model --> No Forced Preemption (Server)

    Last but not least make sure the SCST qla2xxx target mode is enabled and will be built.
    "Device Drivers" --> "SCSI device support" --> "SCSI low level drivers" --> "Qlogic 2xxx target mode support"

    NOTICE: You will not see this option if step 1-"Enabling the SCST Target Module Source" above was not performed.

  3. Start the build (This may take some time, grab a beer!)

    (Following Tom's kernel build suggestions)

    export CONCURRENCY_LEVEL=5  //for Quad Core Processors
    make-kpkg clean
    fakeroot make-kpkg --initrd --append-to-version=-scst-enabled kernel-image kernel-headers

    Two(2) .deb files will be created in the parent directory, image and headers. Install them.

    dpkg -i linux-image-3.2.51-scst-enabled_3.2.51-scst-enabled-10.00.Custom_amd64.deb 
    dpkg -i linux-headers-3.2.51-scst-enabled_3.2.51-scst-enabled-10.00.Custom_amd64.deb 
    reboot #or coldreboot if debian

    Upon reboot you should see new GRUB options:

  4. Build SCST.

    cd {scst_source_directory}
    make 2release
    make all
    BUILD_2X_MODULE=y CONFIG_SCSI_QLA_FC=y CONFIG_SCSI_QLA2XXX_TARGET=y make all install
  5. Enabling the new modules on boot

    vi /etc/modules
    #ensure you have this
    scst
    qla2xxx_scst
    qla2x00tgt
    scst_vdisk
    scst_user
    scst_disk
    
    reboot #or coldreboot if debian

    NOTICE: A reboot is not necessary to get the modules loaded. You can manually load them 1-by-1 using modprobe {module name}. I like to be thorough and make sure they are loading on boot, just in case I ever have to reboot my Fibre Channel Targets will come back up.

    Check to see if all modules loaded on startup:

    lsmod | grep scst
  6. Build SCSTadmin

    cd {your_scst_source}/scstadmin
    make
    make install
    
    #check install
    scstadmin
    
    Collecting current configuration: done.
    
    No valid operations specified.
    

Create VDISKS and Configuring SCST and LUNs

  1. Creating VDISKs

    Use dd to create a image file filled of 0's

    cd /opt/RAIDMOUNT/LUNs
    dd if=/dev/zero of=LinuxLUN01.img bs=1M count=1000000
    dd if=/dev/zero of=WindowsLUN01.img bs=1M count=1000000
  2. Make scst.conf

    vi /etc/scst.conf
    
    
    HANDLER vdisk_fileio {
    	DEVICE LinuxDisk {
    		filename /opt/RAIDMOUNT/LUNs/LinuxLUN04.img
    	}
    	DEVICE WindowsDisk {
    		filename /opt/RAIDMOUNT/LUNs/WindowsLUN12.img
    	}
    }
    TARGET_DRIVER qla2x00t {
    	TARGET 21:00:00:1b:32:1b:d2:35 {
    		HW_TARGET
    
    		enabled 1
    		rel_tgt_id 1
    
    		GROUP host1 {
    
    			LUN 4 LinuxDisk
    			LUN 12 WindowsDisk
    
    			INIITATOR 10:00:00:00:c9:63:94:9e	
    			INITIATOR 10:00:00:00:c9:63:94:9f
    		}
    	}
    
    	TARGET 21:01:00:1b:32:3b:d2:35 {
    		HW_TARGET
    
                    enabled 1
    		rel_tgt_id 2
    	
    		GROUP host2 {
    			LUN 4 LinuxDisk
    			LUN 12 WindowsDisk
    
    			INITIATOR 21:00:00:1b:32:13:e4:07
                            INITIATOR 21:01:00:1b:32:33:e4:07	
           		 }
    	}
    }
    

    TARGET -- is the Port WWN of the HBA on our Linux SCSI Target
    INITIATOR -- is the Port WWN of the HBA on our SCSI Initiators
    Group -- is just a name to group together INITIATORS
    LUN ID -- Used to identify numerically a LUN uniquely, only up to 255 ( I believe), and these must be unique for each LUN per Target PWWN.
    In summary of the config above, I am presenting both LinuxLUN04 and WindowsLUN12 to both host1 and host2

  3. Issue a scstadmin reload

    Issue a scstadmin -conf /etc/scst.conf
    DO NOT do a service scst restart

  4. Check Initiator (in this case a ESX host)

Sources:

There are 2 comments left Go To Comment

  1. Tony /

    what version of linux were you running for this step through?

    1. thejimmahknows / Post Author

      Debian 7, Linux Kernel 3.2.51 (recompiled with SCST) 64-bit

Leave a Reply to thejimmahknows Cancle Reply

↓