Configuring FCP Multipathing in RedHat Linux/CentOS

The task: Creating LUNs on a filer, discover them in RedHat Enterprise Linux or CentOS, and multipath them with the device-mapper-multipath (dm-multipath) mechanism.
Assumptions:
1. The FC HBA drivers are already installed and running.
2. This example is using two single-port QLogic HBAs.
3. The multipathing package is already installed - look for dm-multipath or device-mapper-multipath.

1. Connect the filer using the cabling rules documented in the guides.
2. Create a lun on the filer:

   lun create -s  -t linux /vol/volname/lunname
   

For example:

   lun create -s 100g -t linux /vol/vol1/db1disk1.lun
   

3. Detect the relevant FC HBA initiator WWNs on the filer:

   fcp show initiator
   

4. Configure the WWNs into an initiator group on the filer:

   igroup create -f  
   

5. Check if the Linux already recognizes the luns, it should:

   [root@hostname /]# cat /proc/scsi/scsi
   Attached devices:
   Host: scsi0 Channel: 00 Id: 00 Lun: 00
     Vendor: NETAPP   Model: LUN              Rev: 0.2
     Type:   Direct-Access                    ANSI SCSI revision: 04
   Host: scsi0 Channel: 00 Id: 01 Lun: 00
     Vendor: NETAPP   Model: LUN              Rev: 0.2
     Type:   Unknown                          ANSI SCSI revision: 04
   Host: scsi1 Channel: 00 Id: 00 Lun: 00
     Vendor: NETAPP   Model: LUN              Rev: 0.2
     Type:   Unknown                          ANSI SCSI revision: 04
   Host: scsi1 Channel: 00 Id: 01 Lun: 00
     Vendor: NETAPP   Model: LUN              Rev: 0.2
     Type:   Direct-Access                    ANSI SCSI revision: 04
   

6. Configure the multipath configuration file:

   root@hostname /]# cat /etc/multipath.conf
   
   ## Use user friendly names, instead of using WWIDs as names.
   defaults {
           user_friendly_names yes
   }
   
   # Blacklist all devices by default. Remove this to enable multipathing
   # on the default devices.
   devnode_blacklist {
           devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
           devnode "^hd[a-z]"
           devnode "^cciss!c[0-9]d[0-9]*"
   }
   
   multipaths {
   devices {
           device {
                   vendor                  "NETAPP"
                   product                 "LUN"
                   path_grouping_policy    multibus
                   features                "1 queue_if_no_path"
                   path_checker            readsector0
                   path_selector           "round-robin 0"
                   failback                immediate
                   no_path_retry           queue
           }
           device {
                   vendor                  "NETAPP"
                   product                 "LUN"
                   path_grouping_policy    multibus
                   features                "1 queue_if_no_path"
                   path_checker            readsector0
                   path_selector           "round-robin 0"
                   failback                immediate
                   no_path_retry           queue
           }
   }
   

7. The following commands can be used to rescan the SCSI bus. Verify the paths using ls /proc/scsi/scsi:

   echo "scsi-qlascan" > /proc/scsi/qla2xxx/0
   echo "- - -" > /sys/class/scsi_host/host0/scan
   echo "scsi-qlascan" > /proc/scsi/qla2xxx/1
   echo "- - -" > /sys/class/scsi_host/host1/scan
   echo "scsi add-single-device controller 0 0 1 ">/proc/scsi/scsi
   echo "scsi add-single-device controller 0 0 1 ">/proc/scsi/scsi
   

8. As a result of the configuration file, there should be multipathing devices created already:

   /dev/mapper/mpath0
   

9. Use the following commands to troubleshoot the multipathing setup:

   multipath
   multipath -d -l
   

10. Create a filesystem on top of the multipathing device. While it is possible to create partitions on the underlying luns, and then let the multipathing code discover the partitions (which seem to require a reboot, and will result in devices named /dev/mapper/mpath0p1 for example), it is not recommended and seems to be tricky at best. The steps to create the filesystem and mount it are simple:

   mkfs -t ext3 /dev/mapper/mpath0
   

11. Mount the filesystem and check df:

   [root@hostname ~]# df -k
   Filesystem           1K-blocks      Used Available Use% Mounted on
   /dev/mapper/VolGroup00-LogVol00
                       413267016   2766160 389508040   1% /
   /dev/cciss/c0d0p1       101086     23291     72576  25% /boot
   none                   8126884         0   8126884   0% /dev/shm
   /dev/mapper/mpath0   103212320     93852  97875588   1% /db
   /dev/mapper/mpath1   103212320     93852  97875588   1% /logs
   

Dr. Toaster Recommends:
There are issues that require further attention when creating partitions on NetApp luns. For more information on these issues and the way to avoid them check the following links: http://now.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=156121 and http://now.netapp.com/Knowledgebase/solutionarea.asp?id=kb8190.