DM Multipath

Michal Sedlak

based on Red Hat Documentation:http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/DM_Multipath/index.html

Outline

● Multipath● DM Multipath components● Discussion ...

How to attach drive

● DAS – direct attached storage - disk● PATA● SATA

● NAS - share● Windows share {SMB, CIFS}● NFS

● SAN – LUN - network behind servers● Fiber Channel● FcoE● iSCSI

Multipath

● 2x HBA - Host Bus Adapter

● 2x SAN - Storage Area Network (switch)

● 2x Controllers● Redundancy● Performance

Complex SAN Solution

Components

● dm_multipath kernel module -reroutes I/O

● mpathconf utility - configuration

● multipath command - list and configure devices

● multipathd daemon - monitors paths, interactive mode

● kpartx command - creates partition devices for mp

How to use it

● Install● multipath-tools.deb● device-mapper-multipath.rpm

● Generate configuration - mpathconf● Edit multipath.conf● Start the multipath daemon.

Test setup

How does it work

● Paths● /dev/sda● /dev/sdb

● DM devices● /dev/dm-0● /dev/mpath0● /dev/disk/by-uuid/uuid● /dev/uuid

root@initiator:/etc# iscsi_discovery 10.1.1.1 -liscsiadm: No active sessions.Set target iqn.2012-02.localdomain.target:luns to automatic login over tcp to portal 10.1.1.1:3260Set target iqn.2012-02.localdomain.target:luns to automatic login over tcp to portal 10.1.2.1:3260Set target iqn.2012-02.localdomain.target:luns to automatic login over tcp to portal 192.168.0.236:3260discovered 3 targets at 10.1.1.1

[ 7881.023877] scsi 8:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4[ 7881.091881] sd 8:0:0:0: Attached scsi generic sg2 type 0[ 7881.219889] sd 8:0:0:0: [sdb] 6291456 512-byte logical blocks: (3.22 GB/3.00 GiB)[ 7881.271893] sd 8:0:0:0: [sdb] Write Protect is off[ 7881.271893] sd 8:0:0:0: [sdb] Mode Sense: 77 00 00 08[ 7881.511908] sd 8:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA[ 7881.599913] scsi 8:0:0:1: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4[ 7881.687919] sd 8:0:0:1: Attached scsi generic sg3 type 0[ 7881.883931] sd 8:0:0:1: [sdc] 10485760 512-byte logical blocks: (5.36 GB/5.00 GiB)[ 7881.947935] sdb: unknown partition table[ 7881.971936] sd 8:0:0:1: [sdc] Write Protect is off[ 7881.971936] sd 8:0:0:1: [sdc] Mode Sense: 77 00 00 08[ 7882.151948] sd 8:0:0:1: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA[ 7882.423965] sd 8:0:0:0: [sdb] Attached SCSI disk[ 7882.563973] sdc: unknown partition table[ 7882.743985] sd 8:0:0:1: [sdc] Attached SCSI disk[ 7883.152010] scsi 9:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4[ 7883.400026] sd 9:0:0:0: Attached scsi generic sg4 type 0[ 7883.428027] sd 9:0:0:0: [sdd] 6291456 512-byte logical blocks: (3.22 GB/3.00 GiB)[ 7883.480031] sd 9:0:0:0: [sdd] Write Protect is off[ 7883.480031] sd 9:0:0:0: [sdd] Mode Sense: 77 00 00 08[ 7883.648041] sd 9:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA[ 7883.852054] scsi 9:0:0:1: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4[ 7883.964061] sd 9:0:0:1: Attached scsi generic sg5 type 0[ 7884.116070] sd 9:0:0:1: [sde] 10485760 512-byte logical blocks: (5.36 GB/5.00 GiB)[ 7884.144072] sdd: unknown partition table[ 7884.168074] sd 9:0:0:1: [sde] Write Protect is off[ 7884.168074] sd 9:0:0:1: [sde] Mode Sense: 77 00 00 08[ 7884.900119] sd 9:0:0:1: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA[ 7885.036128] sd 9:0:0:0: [sdd] Attached SCSI disk[ 7885.900182] sde: unknown partition table[ 7885.940184] scsi 10:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4[ 7885.996188] sd 10:0:0:0: Attached scsi generic sg6 type 0[ 7886.236203] sd 10:0:0:0: [sdf] 6291456 512-byte logical blocks: (3.22 GB/3.00 GiB)[ 7886.264205] sd 10:0:0:0: [sdf] Write Protect is off[ 7886.264205] sd 10:0:0:0: [sdf] Mode Sense: 77 00 00 08[ 7886.312208] sd 9:0:0:1: [sde] Attached SCSI disk[ 7886.356210] sd 10:0:0:0: [sdf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA[ 7886.672230] scsi 10:0:0:1: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4[ 7886.828240] sd 10:0:0:1: Attached scsi generic sg7 type 0[ 7886.852241] sdf: unknown partition table[ 7886.920246] sd 10:0:0:1: [sdg] 10485760 512-byte logical blocks: (5.36 GB/5.00 GiB)[ 7887.044253] sd 10:0:0:1: [sdg] Write Protect is off[ 7887.044253] sd 10:0:0:1: [sdg] Mode Sense: 77 00 00 08[ 7887.164261] sd 10:0:0:1: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA[ 7887.180262] sd 10:0:0:0: [sdf] Attached SCSI disk[ 7887.692294] sdg: unknown partition table[ 7888.064317] sd 10:0:0:1: [sdg] Attached SCSI disk

root@initiator:/etc# ls -la /dev/disk/by-path/total 0drwxr-xr-x 2 root root 260 Mar 28 23:16 .drwxr-xr-x 5 root root 100 Mar 28 20:29 ..lrwxrwxrwx 1 root root 9 Mar 28 23:16 ip-10.1.1.1:3260-iscsi-iqn.2012-02.localdomain.target:luns-lun-0 -> ../../sdblrwxrwxrwx 1 root root 9 Mar 28 23:16 ip-10.1.1.1:3260-iscsi-iqn.2012-02.localdomain.target:luns-lun-1 -> ../../sdclrwxrwxrwx 1 root root 9 Mar 28 23:16 ip-10.1.2.1:3260-iscsi-iqn.2012-02.localdomain.target:luns-lun-0 -> ../../sddlrwxrwxrwx 1 root root 9 Mar 28 23:16 ip-10.1.2.1:3260-iscsi-iqn.2012-02.localdomain.target:luns-lun-1 -> ../../sdelrwxrwxrwx 1 root root 9 Mar 28 23:16 ip-192.168.0.236:3260-iscsi-iqn.2012-02.localdomain.target:luns-lun-0 -> ../../sdflrwxrwxrwx 1 root root 9 Mar 28 23:16 ip-192.168.0.236:3260-iscsi-iqn.2012-02.localdomain.target:luns-lun-1 -> ../../sdg

root@initiator:/etc# ls -la /dev/mapper/total 0drwxr-xr-x 2 root root 140 Mar 28 23:16 .drwxr-xr-x 15 root root 4440 Mar 28 23:16 ..lrwxrwxrwx 1 root root 7 Mar 28 23:17 1494554000000000030000000000000000000000000000000 -> ../dm-2lrwxrwxrwx 1 root root 7 Mar 28 23:17 1494554000000000031000000000000000000000000000000 -> ../dm-3crw------- 1 root root 10, 236 Mar 28 20:30 controllrwxrwxrwx 1 root root 7 Mar 28 20:31 target-root -> ../dm-0lrwxrwxrwx 1 root root 7 Mar 28 20:30 target-swap_1 -> ../dm-1

Rename Multipath Deviceblacklist { wwid "*"}

blacklist_exceptions { wwid "1494554000000000030000000000000000000000000000000"}

multipaths {multipath {

wwid "1494554000000000030000000000000000000000000000000"alias "lun0"

}}

root@initiator:/etc# multipathd -kmultipathd> reconfigureokmultipathd> root@initiator:/etc# multipath -lllun0 (1494554000000000030000000000000000000000000000000) dm-2 IET,VIRTUAL-DISKsize=3.0G features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=1 status=active| `- 11:0:0:0 sdb 8:16 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 12:0:0:0 sdd 8:48 active ready running`-+- policy='round-robin 0' prio=1 status=enabled `- 13:0:0:0 sdf 8:80 active ready running

root@initiator:/etc# pvcreate /dev/mapper/lun0 Physical volume "/dev/mapper/lun0" successfully createdroot@initiator:/etc# vgcreate vg01 /dev/mapper/lun0 Volume group "vg01" successfully createdroot@initiator:/etc# lvcreate -n lun0 -L 1g vg01 Logical volume "lun0" createdroot@initiator:/etc# mkfs.ext4 /dev/vg01/lun0 mke2fs 1.41.14 (22-Dec-2010)Filesystem label=OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)

This filesystem will be automatically checked every 24 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.

root@initiator:/etc# mkdir /dataroot@initiator:/etc# mount /dev/vg01/lun0 /dataroot@initiator:/etc# df -hP |column -tFilesystem Size Used Avail Use% Mounted on/dev/mapper/target-root 19G 1015M 17G 6% /udev 242M 8.0K 242M 1% /devtmpfs 100M 496K 99M 1% /runnone 5.0M 0 5.0M 0% /run/locknone 249M 0 249M 0% /run/shm/dev/sda1 228M 44M 172M 21% /boot/dev/mapper/vg01-lun0 1008M 34M 924M 4% /data

root@initiator:/etc# multipath -ll1494554000000000031000000000000000000000000000000 dm-3 IET,VIRTUAL-DISKsize=5.0G features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=1 status=active| `- 8:0:0:1 sdc 8:32 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 9:0:0:1 sde 8:64 active ready running`-+- policy='round-robin 0' prio=1 status=enabled `- 10:0:0:1 sdg 8:96 active ready running1494554000000000030000000000000000000000000000000 dm-2 IET,VIRTUAL-DISKsize=3.0G features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=1 status=active| `- 8:0:0:0 sdb 8:16 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 9:0:0:0 sdd 8:48 active ready running`-+- policy='round-robin 0' prio=1 status=enabled `- 10:0:0:0 sdf 8:80 active ready running

Discussion

defaults { user_friendly_names yes}

Lun 0 Path=/dev/sdc,Type=fileio,ScsiId=xyz1,ScsiSN=xyz1Lun 0 Path=/dev/sdc,Type=fileio,ScsiId=xyz2,ScsiSN=xyz2

IncomingUser joe secretOutgoingUser jim 12charpasswd

DM MP & LVM

● lvm.conf - multiple devices with same uuid● filter = [ "r/block/", "r/disk/", "r/sd.*/", "a/.*/" ]

Thank you