The second assumption is that multipath has the basic setup for your particular storage frame. Now, lets ensure multipath is running and set to start on every boot:
# service multipathd status
multipathd is stopped
# chkconfig --list multipathd
multipathd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
# chkconfig multipathd on
# chkconfig --list multipathd
multipathd 0:off 1:off 2:off 3:on 4:off 5:on 6:off
# service multipathd start
As in the last post, I am dealing with RedHat 5.4 and an EMC CLARiiON array. Without any LUNs allocated multipath -ll should look something like this:
# multipath -ll
sdb: checker msg is "emc_clariion_checker: Logical Unit is umbound or LUNZ"
sdc: checker msg is "emc_clariion_checker: Logical Unit is umbound or LUNZ"
sdd: checker msg is "emc_clariion_checker: Logical Unit is umbound or LUNZ"
sde: checker msg is "emc_clariion_checker: Logical Unit is umbound or LUNZ"
These entries are the four paths available to CX controllers.
Adding Devices
No Existing Devices
I have so far been unable to use the simple scan method on an HBA without any devices at all, so this process will unload and reload the adapter driver. It's a disruptive process on the fibre channel bus but there aren't any devices anyway, so it shouldn't matter.
First find the driver you are using, it is likely either an Emulex (lpfc) or Qlogic (qla). In this example I am using an Emulex card.
# lsmod | grep lpfc
lpfc 352909 0
scsi_transport_fc 73801 1 lpfc
scsi_mod 196569 10 scsi_dh,sr_mod,sg,usb_storage,lpfc,scsi_transport_fc,mptsas,mptscsih,scsi_transport_sas,sd_mod
remove the module
# rmmod lpfc
insert the module
# modprobe lpfc
Instead of modprobe, you can also use insmod. The difference being insmod will only load the specified driver and modprobe will load the driver and any dependent drivers.
This will allow the device(s) to show under /dev/ora_rdsk but won't create any multipath entries. To do that we simply run multipath.
# multipath
reload: 36006016015a01900796464949a36df11 DGC,RAID 5
[size=50G][features=1 queue_if_no_path|features=1
queue_if_no_path][hwhandler=1 emc][n/a]
\_ round-robin 0 [prio=2][undef]
\_ 4:0:1:0 sdc 8:32 [active][ready]
\_ 5:0:1:0 sde 8:64 [undef][ready]
\_ round-robin 0 [prio=0][undef]
\_ 4:0:0:0 sdb 8:16 [undef][ready]
\_ 5:0:0:0 sdd 8:48 [undef][ready]
Existing Devices
If you have at least one Fibre device existing, you can simply rescan the bus. This will not take down the existing devices and is able to operate one path at a time ensuring I/O can continue to flow. You will need to know which host devices are your fibre HBAs. To find that, we can list the know fibre adapters as follows and then issue a scan for each.
# ls -l /sys/class/fc_host
drwxr-xr-x 3 root root 0 Apr 17 08:50 host4
drwxr-xr-x 3 root root 0 Apr 17 08:50 host5
# echo "- - -" > /sys/class/scsi_host/host4/scan
# multipath -ll
36006016015a01900e2acd0d4a549df11 dm-3 DGC,RAID 5
[size=8.0G][features=1 queue_if_no_path|features=1
queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=1][active]
\_ 1:0:0:1 sdh 8:112 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 1:0:1:1 sdi 8:128 [active][ready]
# echo "- - -" > /sys/class/scsi_host/host5/scan
# multipath -ll
36006016015a01900e2acd0d4a549df11 dm-3 DGC,RAID 5
[size=8.0G][features=1 queue_if_no_path|features=1
queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=2][enabled]
\_ 1:0:0:1 sdh 8:112 [active][ready]
\_ 2:0:0:1 sdj 8:144 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 1:0:1:1 sdi 8:128 [active][ready]
\_ 2:0:1:1 sdk 8:160 [active][ready]
# ls -lL /dev/ora_rdsk
brw-rw---- 1 root root 253, 3 Apr 17 15:57 36006016015a01900e2acd0d4a549df11
Renaming Devices
The newly scanned device has a WWID name which isn't terribly useful for something like Oracle as we want udev to apply appropriate permissions. To do this, cut and paste the ID into /etc/multipath.conf so it looks something like this:
multipath {
wwid 36006016015a01900796464949a36df11
alias ora_test
}
And then remove the old device name and re-import it into multipath
# multipath -f 36006016015a01900796464949a36df11
# multipath
create: ora_test (36006016015a01900796464949a36df11) DGC,RAID 5
[size=50G][features=1 queue_if_no_path|features=1
queue_if_no_path][hwhandler=1 emc][n/a]
\_ round-robin 0 [prio=2][undef]
\_ 1:0:1:0 sdc 8:32 [undef][ready]
\_ 2:0:1:0 sde 8:64 [undef][ready]
\_ round-robin 0 [prio=0][undef]
\_ 1:0:0:0 sdb 8:16 [undef][ready]
\_ 2:0:0:0 sdd 8:48 [undef][ready]
# ls \-lL /dev/ora_rdsk
brw-rw---- 1 oracle dba 253, 2 Apr 17 09:44 ora_test
If you get an error "must provide a map name to remove" when running multipath -f, make sure you don't have a shell inside /dev/ora_rdsk directory. Also, be careful not to use multipath -F as that will remove all devices, probably not what you want.
Removing Devices
Before doing the actual removal you will need to note several pieces of information; the multipath device name and all block devices assigned to it. All of which can be obtained from multipath -ll.
# multipath -ll
*ora_test2* (36006016015a01900e2acd0d4a549df11) dm-3 DGC,RAID 5
[size=8.0G][features=1 queue_if_no_path|features=1
queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=2][active]
\_ 1:0:0:1 *sdh* 8:112 [active][ready]
\_ 2:0:0:1 *sdj* 8:144 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 1:0:1:1 *sdi* 8:128 [active][ready]
\_ 2:0:1:1 *sdk* 8:160 [active][ready]
To remove the multipath device
# multipath -f ora_test2
Then remove the appropriate block devices from the system with 'echo 1 >
/sys/block/*dev*/device/delete'
# echo 1 > /sys/block/sdh/device/delete
# echo 1 > /sys/block/sdj/device/delete
# echo 1 > /sys/block/sdi/device/delete
# echo 1 > /sys/block/sdk/device/delete
No comments:
Post a Comment