multipath config

Sun Microsystems, Inc.www.sun.com

Submit comments about this document at: http://www.sun.com/hwdocs/feedback

System Administration Guide:Multipath Configuration

For Solaris™ 10 Operating System

Part No. 819-0139-10January 2005

PleaseRecycle

Copyright 2004 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.

Sun Microsystems, Inc. has intellectual property rights relating to technology that is described in this document. In particular, and withoutlimitation, these intellectual property rights may include one or more of the U.S. patents listed at http://www.sun.com/patents and one ormore additional patents or pending patent applications in the U.S. and in other countries.

This document and the product to which it pertains are distributed under licenses restricting their use, copying, distribution, anddecompilation. No part of the product or of this document may be reproduced in any form by any means without prior written authorization ofSun and its licensors, if any.

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.

Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark inthe U.S. and in other countries, exclusively licensed through X/Open Company, Ltd.

Sun, Sun Microsystems, the Sun logo, AnswerBook2, docs.sun.com, OpenBoot, Solstice DiskSuite, Sun StorEdge, and Solaris are trademarks orregistered trademarks of Sun Microsystems, Inc. in the U.S. and in other countries.

All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and in othercountries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.

The OPEN LOOK and Sun™ Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledgesthe pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sunholds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun’s licensees who implement OPENLOOK GUIs and otherwise comply with Sun’s written license agreements.

Netscape Navigator is a trademark or registered trademark of Netscape Communications Corporation in the United States and other countries.

U.S. Government Rights—Commercial use. Government users are subject to the Sun Microsystems, Inc. standard license agreement andapplicable provisions of the FAR and its supplements.

DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Copyright 2004 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, Californie 95054, Etats-Unis. Tous droits réservés.

Sun Microsystems, Inc. a les droits de propriété intellectuels relatants à la technologie qui est décrit dans ce document. En particulier, et sans lalimitation, ces droits de propriété intellectuels peuvent inclure un ou plus des brevets américains énumérés à http://www.sun.com/patents etun ou les brevets plus supplémentaires ou les applications de brevet en attente dans les Etats-Unis et dans les autres pays.

Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et ladécompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sansl’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en.

Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par desfournisseurs de Sun.

Des parties de ce produit pourront être dérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marquedéposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.

Sun, Sun Microsystems, le logo Sun, AnswerBook2, docs.sun.com, OpenBoot, Solstice DiskSuite, Sun StorEdge, et Solaris sont des marques defabrique ou des marques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays.

Toutes les marques SPARC sont utilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARC International, Inc.aux Etats-Unis et dans d’autres pays. Les produits protant les marques SPARC sont basés sur une architecture développée par SunMicrosystems, Inc.

L’interface d’utilisation graphique OPEN LOOK et Sun™ a été développée par Sun Microsystems, Inc. pour ses utilisateurs et licenciés. Sunreconnaît les efforts de pionniers de Xerox pour la recherche et le développement du concept des interfaces d’utilisation visuelle ou graphiquepour l’industrie de l’informatique. Sun détient une license non exclusive de Xerox sur l’interface d’utilisation graphique Xerox, cette licencecouvrant également les licenciées de Sun qui mettent en place l’interface d ’utilisation graphique OPEN LOOK et qui en outre se conformentaux licences écrites de Sun.

LA DOCUMENTATION EST FOURNIE "EN L’ÉTAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSESOU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENTTOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU AL’ABSENCE DE CONTREFAÇON.

Contents

Preface ix

1. Multipathing Overview 1

Fibre Channel and Multipathing Features 1

Multipath Driver Features 2

Supported Standards 4

2. Configuring Multipathing Software 5

Configuring Multipathing 5

Task Summary To Configure Multipathing 6

Reconfiguration Reboot Requirements 6

Enabling Multipathing Globally 6

▼ To Configure Multipathing Globally 7

Enabling Multipathing on a Per Port Basis 9

Considerations for Per-Port Configuration 10

▼ To Configure Multipathing by Port 10

▼ To Configure a Single PCI HBA 12

▼ To Configure a Dual PCI HBA 13

▼ To Configure an SBus HBA 14

Configuring Automatic Failback for Sun StorEdge Arrays 14

iii

▼ To Configure Automatic Failback Capability 15

▼ To Disable Automatic Failback Capability 15

Configuring Third-Party Symmetric Storage Devices for Multipathing 16

Considerations for Third-Party Device Configuration 16

▼ To Configure Third-Party Devices 16

Configuring Solstice DiskSuite or Solaris Volume Manager for Multipathing 18

Solaris DiskSuite or Solaris Volume Manager Configuration Overview 18

Example of Migrating Mirrored Devices 19

▼ To Migrate Two-Way Mirrored Metadevices 20

3. Configuring SAN Devices 29

SAN Device Considerations 29

Adding SAN Devices 30

▼ To Add a SAN Device 30

Fabric Device Node Configuration 31

Ensuring that LUN-Level Information Is Visible 31

▼ To Detect Fabric Devices Visible on a Host 32

Configuring Device Nodes Without Multipathing Enabled 33

▼ To Configure an Individual Device Without Multipathing 33

▼ To Configure Multiple Devices Without Multipathing 38

Configuring Device Nodes With Multipathing Enabled 40

▼ To Configure Individual Devices With Multipathing 41

▼ To Configure Multiple Devices With Multipathing 43

4. Configuring Multipathing Support for Storage Boot Devices 45

About the stmsboot Command 45

Considerations for stmsboot Operations 46

Displaying Potential /dev Device Name Changes 47

▼ To Display Potential /dev Device Name Changes 48

iv System Administration Guide: Multipath Configuration • January 2005

Enabling Multipathing on the Boot Controller Port 48

▼ To Enable Multipathing on the Boot Controller Port 49

Disabling Multipathing on the Boot Controller Port 51

▼ To Disable Multipathing on the Boot Controller Port 51

5. Configuring IPFC SAN Devices 53

Loading IPFC 53

IPFC Considerations 53

Determining Fibre Channel Adapter Port Instances 54

▼ To Determine Port Instances from the WWN 55

▼ To Determine Port instances From the Physical Device Path 55

▼ To Plumb an IPFC Instance 57

Invoking and Configuring IPFC 58

▼ To Start a Network Interface Manually 59

▼ To Configure the Host for Automatic Plumbing Upon Reboot 59

6. Unconfiguring Fabric Devices 61

Unconfiguring Fabric Devices 61

▼ To Unconfigure a Fabric Device 61

▼ To Unconfigure All Fabric Devices on a Fabric-Connected Host Port 62

▼ To Unconfigure a Fabric Device Associated With Multipathing EnabledDevices 64

▼ To Unconfigure One Path to a Multipathing Device 65

▼ To Unconfigure All Fabric-Connected Devices With MultipathingEnabled 67

A. Multipathing Configuration Samples 69

About Multipathing Configuration Samples 69

Configuration without Multipathing 70

Utility: format 70

Configuration with Multipathing 71

Contents v

Utility: format 71

Utility: luxadm probe 72

Utility: luxadm display 72

Utility: luxadm failover 75

B. Supported FC-HBA API 79

C. Zones and Ports 83

Zone Types 83

Port Types 84

D. Implementing Sun StorEdge SAN Software Dynamic Reconfiguration 87

Dynamic Reconfiguration 87

Dynamic Reconfiguration and Fabric Devices 87

▼ To Remove a Fabric Device Before Dynamic Reconfiguration 88

▼ To Maintain a Fabric Device Configuration With DynamicReconfiguration 88

▼ To Reconfigure Fabric Devices With Dynamic Reconfiguration 88

▼ To Reconfigure the Sun Enterprise 10000 Server With a FabricConnection 88

E. Multipathing Troubleshooting 91

System Crashes During Boot Operations 92

System Crashes During or After a Boot Enable Operation 92

System Crashes During or After a Multipathing Boot Disable Operation 94

Multipathing Is Not Running Properly 95

luxadm display and luxadm failover Commands Fail 96

Sun StorEdge T3, 6x20, or 3900 Arrays Do Not Show 96

System Failed During Boot With scsi_vhci Attachment 97

Connected Sun StorEdge A5200 Arrays Appear Under Physical Paths inFormat 97

vi System Administration Guide: Multipath Configuration • January 2005

System and Error Messages 98

Index 99

Contents vii

viii System Administration Guide: Multipath Configuration • January 2005

Preface

The System Administration Guide: Multipath Configuration provides an overview of theSolaris™ 10 Operating System (Solaris OS) software that is used to configuremultipathing, with an explanation of how to install and then configure the software.

This guide is intended for system, storage, and network administrators who createand maintain storage area networks (SANs) and have a high level of expertise in themanagement and maintenance of SANs.

Before You Read This BookBefore you read this book, review the late-breaking information described in theSolaris 10 Operating System Release Notes.

ix

How This Book is Organized■ Chapter 1 provides an overview of SAN and multipathing features and

guidelines.

■ Chapter 2 provides an overview of the configuration process.

■ Chapter 3 explains how to configure multipathing devices.

■ Chapter 4 describes how to configure multipathing for a storage boot device.

■ Chapter 5 explains how to configure IPFC SAN devices.

■ Chapter 6 describes the steps required to unconfigure the multipathing software.

■ Appendix A provides configuration samples.

■ Appendix B lists the HBA API library commands.

■ Appendix C provides Zone information.

■ Appendix D describes dynamic reconfiguration.

■ Appendix E provides a troubleshooting guide.

Using UNIX CommandsThis document may not contain information on basic UNIX® commands andprocedures such as shutting down the system, booting the system, and configuringdevices.

See one or more of the following for this information:■ Solaris Handbook for Sun Peripherals■ AnswerBook2™ online documentation for the Solaris™ Operating System

x System Administration Guide: Multipath Configuration • January 2005

Typographic Conventions

Shell Prompts

Typeface Meaning Examples

AaBbCc123 The names of commands, files,and directories; on-screencomputer output

Edit your .login file.Use ls -a to list all files.% You have mail.

AaBbCc123 What you type, whencontrasted with on-screencomputer output

% su

Password:

AaBbCc123 Book titles, new words or terms,words to be emphasized

Command-line variable; replacewith a real name or value

Read Chapter 6 in the User’s Guide.These are called class options.You must be superuser to do this.

To delete a file, type rm filename.

Shell Prompt

C shell machine_name%

C shell superuser machine_name#

Bourne shell and Korn shell $

Bourne shell and Korn shell superuser #

Preface xi

Related Documentation

Contacting Sun Technical SupportIf you need help installing or using this product, contact your service provider. Ifyou have a support contract with Sun, call 1-800-USA-4SUN, or go to:

http://www.sun.com/service/contacting/index.html

Accessing Sun DocumentationYou can view, print, or purchase a broad selection of Sun documentation, includinglocalized versions, at:

http://www.sun.com/documentation

A broad selection of Sun system documentation is located at:

http://www.sun.com/products-n-solutions/hardware/docs

Product Title Part Number

Solaris 10Operating System

Solaris 10 Operating System Release Notes 817-0552-nn

Systems Administration Guide: BasicAdministration

817-1985-nn

Systems Administration Guide: AdvancedAdministration

817-0403-nn

Systems Administration Guide: IP Services 816-4554-nn

Systems Administration Guide: Network Services 817-4555-nn

Solaris Administration Guide: Devices and FileSystems

817-5093-nn

xii System Administration Guide: Multipath Configuration • January 2005

A complete set of Solaris documentation and many other titles are located at:

http://docs.sun.com

Sun Welcomes Your CommentsSun is interested in improving its documentation and welcomes your comments andsuggestions. You can submit your comments by going to:

http://www.sun.com/hwdocs/feedback

Please include the title and part number of your document with your feedback:

System Administration Guide: Multipath Configuration, part number 819-0139-10

Preface xiii

xiv System Administration Guide: Multipath Configuration • January 2005

CHAPTER 1

Multipathing Overview

This chapter provides an overview of the multipathing capabilities of the Solaris™10 Operating System (Solaris OS). This information will prove helpful duringinstallation and configuration of the software.

This chapter contains the following sections:

■ “Fibre Channel and Multipathing Features” on page 1

■ “Supported Standards” on page 4

Fibre Channel and MultipathingFeaturesThe Solaris 10 OS enables Fibre Channel (FC) connectivity for Solaris hosts. Thesoftware resides on the server and identifies the storage and switch devices on yourSAN. It allows you to attach either loop or fabric SAN storage devices whileproviding a standard interface with which to manage them. This multipathingfunctionality were previously available through the Sun StorEdge™ SANFoundation Software and Sun StorEdge™ Traffic Manager products.

You can manage multiple redundant paths to storage for enhanced reliability andperformance as well as implement third-party multipathing solutions while usingSolaris 10 OS.

The Solaris 10 multipathing stack is compliant with the Sun Common SCSIArchitecture driver model, and allows either Sun or third party volume managers,file systems, and multipathing software to operate seamlessly.

Solaris 10 fibre channel and multipathing provide the following key features:

1

■ Dynamic storage recovery – The Solaris 10 OS automatically recognizes devicesand any modifications made to device configurations. This makes devicesavailable to the system without requiring you to reboot or manually changeinformation in configuration files.

■ Persistent device naming – Devices that are configured within the SAN maintaintheir device naming through reboots and/or reconfiguration. The only exceptionto this are tape devices, found in /dev/rmt, that are persistent and will notchange unless they are removed or later regenerated.

■ FCAL support – OpenBoot™ Prom (OBP) commands that are used on servers canaccess Fibre Channel-Arbitrated Loop (FCAL) attached storage for scanning theFibre Channel loop.

■ Fabric booting – Sun Supported 1 Gbit and 2 Gbit host bus adapters (HBAs)supported by Sun can boot from fabric devices as well as non-fabric devices.Fabric topologies with Fibre Channel switches provide higher speed, moreconnections, and port isolation.

■ Configuration management – Management can be performed with thecfgadm(1M) and luxadm(1M) commands. These commands control host-levelaccess to devices and allow you to configure hosts to see only necessary devices,providing an alternative to switch zoning.

■ T11 FC-HBA library – What was previously known as the Storage NetworkingIndustry Association (SNIA) Fibre Channel HBA(FC-HBA) library is now knownas the T11 FC-HBA library. The T11 FC-HBA library application programminginterface (API) enables management of Fibre Channel HBAs and provides astandards-based interface for other applications (such as Sun StorEdge EnterpriseStorage Manager Topology Reporter) that can be used to gather informationabout the SAN’s HBAs, switches, and storage.

Man pages for common FC-HBAs are included in the Solaris 10 OS. Foradditional information on fibre channel specifications (FC-MI) refer tohttp://www.t11.org. Appendix B lists functions supported by Sun’simplementation of the T11 FC-HBA.

Multipath Driver FeaturesThe Solaris 10 OS includes multipath client driver software. Once enabled, thisdriver software can be configured to implement a set of interfaces for host controllerinterface (HCI) drivers, known as the multipath driver interface (MDI). In additionto providing the multipath driver interface, the driver also provides the followingfeatures:

■ Path management – The Solaris 10 OS multipathing software dynamicallymanages the paths to any storage devices it supports. There are no configurationfiles to manage or databases to keep current for supported devices. The additionor removal of paths to a device is done automatically when a path is brought

2 System Administration Guide: Multipath Configuration • January 2005

online or removed from service. This allows hosts configured with multipathingto begin with a single path to a device and add more host controllers, increasingbandwidth and RAS, without changing device names or modifying applications.

■ Single device instances - Multipathing is fully integrated with the Solaris OS. Thisallows the software to display multipath devices as single device instancesinstead of as one device, or device link, per path. This reduces the cost ofmanaging complex storage architectures, since it enables utilities, such asformat(1M) or higher level applications such as Solaris Volume Manager, toaccess one representation of a storage device instead of a separate device for eachpath.

■ Failover support – Implementing higher levels of RAS requires redundant hostconnectivity to storage devices. Solaris 10 OS multipathing drivers manage boththe failure or hosts coming offline to storage paths while maintaining host I/Oconnectivity through available secondary paths. This enables applications tocontinue operating in the event of host disconnection or downstream path failure.Failed paths can be automatically re-enabled through dynamic path management.

■ Symmetrical/Asymmetrical Device Support – The Solaris 10 OS can manage allsymmetrical devices. Asymmetric device have proprietary commands that requireadditional individual device support. Solaris 10 supports all Sun asymmetricdevices, for example, the T3/6120. Using a switch in front of an asymmetricdevice will allow you to have a pool of preferred symmetric paths, as well as apool of inactive asymmetric paths.

■ I/O load balancing – In addition to providing simple failover support, thesoftware can use any active paths to a storage device to send and receive I/O.With I/O routed through multiple host connections, bandwidth can be increasedby the addition of host controllers. The Solaris 10 OS uses a round-robin load-balancing algorithm, by which individual I/O requests are routed to active hostcontrollers in a series, one after the other.

■ Queue depth – SCSI storage arrays present storage to a host in the form of a LUN.LUNs have a finite set of resources available, such as the amount of data that canbe stored, as well as the number of active commands that a device or LUN canprocess at one time. The number of active commands that can be issued before adevice blocks further I/O is know as queue depth. When multipathing software isenabled, a single queue is created for each LUN regardless of the number ofdistinct or separate paths it may have to the host. This prevents unnecessaryaction on storage array resources and allows upper layer drivers and programs toeffectively use the LUN without flooding it with I/O requests.

■ Dynamic reconfiguration – The Solaris 10 OS supports Solaris DynamicReconfiguration (DR).

Chapter 1 Multipathing Overview 3

Supported StandardsThe Solaris 10 OS is based on open standards for communicating with devices anddevice management, ensuring interoperability with other standards-based devicesand software. The following standards are supported by the Solaris 10 OS:

■ T10 standards, including SCSI-2, SAM, FCP, SPC, and SBC

■ T11.3 Fibre Channel standards, including FC-PH, FC-AL, FC-LE, and FC-GS

■ T11.5 storage management standards, including FC-HBA

■ IETF standards, including RFC 2625


CHAPTER 2

Configuring Multipathing Software

This chapter contains the following sections:

■ “Configuring Multipathing” on page 5

■ “Enabling Multipathing Globally” on page 6

■ “Enabling Multipathing on a Per Port Basis” on page 9

■ “Configuring Automatic Failback for Sun StorEdge Arrays” on page 14

■ “Configuring Third-Party Symmetric Storage Devices for Multipathing” onpage 16

■ “Configuring Solstice DiskSuite or Solaris Volume Manager for Multipathing” onpage 18

Configuring MultipathingMultipathing is provided by a driver that runs in the Solaris™ Operating System(Solaris OS) environment. Multipathing is disabled by default for the Solaris OS, butis enabled by default on Solaris OS on x86 based systems. If you are using anothermultipathing application, see the documentation for that application.

Note – These software features are not available for parallel SCSI devices but areavailable for Fibre Channel disk devices. Multipathing is not supported on tapedrives or libraries, or on FC over IP.

Configuration of the multipathing software depends on how you intend to use yoursystem. The software can be configured to control all Fibre Channel HBAs supportedby Sun as listed in the Sun Solaris 10 Operating System Release Notes. You can alsoconfigure multipathing for third-party symmetric storage devices to use the SolsticeDiskSuite™ software, Solaris Volume Manager software, or third-party volumemanagement and multipathing software.

5

Task Summary To Configure MultipathingTABLE 2-1 summarizes the tasks involved in configuring multipathing.

Reconfiguration Reboot RequirementsYou must perform a reconfiguration reboot whenever the following occurs:

■ You change the scsi_vhci.conf or fp.conf files.

■ In non-fabric environments, you change the mp_support field for Sun StorEdgeT3, 6x20, and 3900 arrays. For further information about Sun StorEdge T3, 6x20and 3900 arrays, refer to the documentation that came with your system.

■ Unless you explicitly enabled or disabled the software on a specific port, theglobal settings in multipathing will apply.

Enabling Multipathing GloballyMultipathing can be configured to control and provide load balancing for a subset ofthe HBAs supported by Sun. This section describes the files and parameters that areused to configure multipathing.

TABLE 2-1 Task Summary for Configuration of Multipathing

Task See

1. Configure• All ports globally• Individual ports

See “Reconfiguration Reboot Requirements” on page 6.See “Enabling Multipathing Globally” on page 6.See “Enabling Multipathing on a Per Port Basis” on page 9.

2. Configure other options. See “Configuring Third-Party Symmetric Storage Devicesfor Multipathing” on page 16.See “Configuring Solstice DiskSuite or Solaris VolumeManager for Multipathing” on page 18.

3. Configure boot controllerdevices.

See “Configuring Multipathing Support for Storage BootDevices” on page 45.

4. Perform a reconfigurationreboot (if necessary).

See “Reconfiguration Reboot Requirements” on page 6.

6 System Administration Guide: Multipath Configuration• January 2005

▼ To Configure Multipathing Globally1. Using any text editor, edit the /kernel/drv/fp.conf file as displayed in “Example

of fp.conf file” on page 8.

2. To enable multipathing globally, change the value of mpxio-disable to ”no”. Onthe Solaris 10 OS for Solaris, mpxio-disable is set to “yes” by default, which meansmultipathing is disabled. With the Solaris OS on x86 based systems, mpxio-disableis set to “no” by default, which means that multipathing is enabled.

3. (Optional) Enable multipathing support for third-party symmetric devices. Referto “Configuring Third-Party Symmetric Storage Devices for Multipathing” onpage 16.

4. Save the /kernel/drv/fp.conf file.

1. Using any text editor, edit the /kernel/drv/scsi_vhci.conf file as displayed in“Example of scsi_vhci.conf file” on page 9. Do not change the name and classdefinitions.

2. If you want the multipathing software to use all the available paths for loadbalancing, leave the load-balance field set to the default of “round-robin”.

Otherwise, change the definition to “none”.

3. Save the /kernel/drv/scsi_vhci.conf file.

4. Perform one of the following steps.

■ If you want to enable multipathing on the boot controller port, see “ConfiguringMultipathing Support for Storage Boot Devices” on page 45.

■ If you do not want to enable multipathing software on the boot controller port atthis time, perform a reconfiguration reboot now to shut down and restart yoursystem by using the following commands.

# touch /reconfigure# shutdown -g0 -y -i6

Chapter 2 Configuring Multipathing Software 7

CODE EXAMPLE 2-1 Example of fp.conf file

# Copyright 2004 Sun Microsystems, Inc. All rights reserved.# Use is subject to license terms.## Sun Fibre Channel Port driver configuration##ident "%Z%%M% %I% %E% SMI"#

name="fp" class="fibre-channel" port=0;name="fp" class="fibre-channel" port=1;

## To generate the binding-set specific ’compatible’ forms used to address# legacy issues the ’scsi-binding-set’ property must be defined. (do notremove)#scsi-binding-set="fcp";

## List of ULP modules for loading during port driver attach time#load-ulp-list="1","fcp";

## Force attach driver to support hotplug activity (do not remove theproperty)#ddi-forceattach=1;

## I/O multipathing feature (MPxIO) can be enabled or disabled using# mpxio-disable property. Setting mpxio-disable="no" will activate# I/O multipathing; setting mpxio-disable="yes" disables the feature.## To globally enable MPxIO on all fp ports set:# mpxio-disable="no";## To globally disable MPxIO on all fp ports set:# mpxio-disable="yes";## You can also enable or disable MPxIO per port basis. Per port settings# override the global setting for the specified ports.# To disable MPxIO on port 0 whose parent is /pci@8,600000/SUNW,qlc@4 set:# name="fp" parent="/pci@8,600000/SUNW,qlc@4" port=0 mpxio-disable="yes";## NOTE: If you just want to enable or disable MPxIO on all fp ports, it is# better to use stmsboot(1M) as it also updates /etc/vfstab.#mpxio-disable="yes";


CODE EXAMPLE 2-2 Example of scsi_vhci.conf file

Enabling Multipathing on a Per PortBasisAlthough multipathing may be globally enabled, individual, per-port multipathingsoftware settings have priority over global settings:

This section covers the following topics:

■ “Considerations for Per-Port Configuration” on page 10

# Copyright 2004 Sun Microsystems, Inc. All rights reserved.# Use is subject to license terms.##pragma ident"@(#)scsi_vhci.conf1.804/03/07 SMI"#name="scsi_vhci" class="root";## Load balancing global configuration: setting load-balance="none" willcause# all I/O to a given device (which supports multipath I/O) to occur via one# path. Setting load-balance="round-robin" will cause each path to thedevice# to be used in turn.#load-balance="round-robin";## Force load driver to support hotplug activity (do not remove thisproperty).#ddi-forceattach=1;## Automatic failback configuration# possible values are auto-failback="enable" or auto-failback="disable"auto-failback="disable";## For enabling MPxIO support for 3rd party symmetric device need an# entry similar to following in this file. Just replace the "SUN SENA"# part with the Vendor ID/Product ID for the device, exactly as reported by# Inquiry cmd.## device-type-scsi-options-list =# "SUN SENA", "symmetric-option";## symmetric-option = 0x1000000;


■ “To Configure Multipathing by Port” on page 10

Considerations for Per-Port ConfigurationBefore you start configuring the software by port, consider the following:

■ Load balancing is controlled by the global load-balance= variable and cannotbe applied on a per-port basis.

■ If a storage device is managed or controlled by a volume manager that is notsupported by Sun, the multipathing software must be disabled on that port.

■ Devices with multipathing software enabled are enumerated under/devices/scsi_vhci. Devices with multipathing software disabled areenumerated under physical path names.

■ With the multipathing software installed, all paths to storage devices must beconfigured with multipathing software enabled or disabled.

■ Configuring multipathing software by port enables the software to co-exist withother multipathing solutions like Alternate Pathing (AP), VERITAS DynamicMultipathing (DMP), or EMC PowerPath. However, storage devices and pathsshould not be shared between multipathing software and other multipathingsolutions.

▼ To Configure Multipathing by PortThis section provides an overview of the steps needed to configure the software on aper port basis. For specific steps, refer to:

■ “To Configure a Single PCI HBA” on page 12

■ “To Configure a Dual PCI HBA” on page 13

■ “To Configure an SBus HBA” on page 14

1. Log in as superuser.


Determine the HBAs that you want the multipathing software to control. Forexample, to select the desired device, perform an ls -l command on /dev/fc. Thefollowing is an example of the command output.

Note – The fp7 entry is a SBus HBA. The fp5 and fp6 include two /pci elements.This indicates a dual PCI HBA. The rest of the entries do not have additional PCIbridges and are single PCI HBAs.

lrwxrwxrwx 1 root root 49 Apr 17 18:14 fp0 ->../../devices/pci@6,2000/SUNW,qlc@2/fp@0,0:devctllrwxrwxrwx 1 root root 49 Apr 17 18:14 fp1 ->../../devices/pci@7,2000/SUNW,qlc@2/fp@0,0:devctllrwxrwxrwx 1 root root 49 Apr 17 18:14 fp0 ->../../devices/pci@6,2000/SUNW,qlc@2/fp@0,0:devctllrwxrwxrwx 1 root root 49 Apr 17 18:14 fp1 ->../../devices/pci@7,2000/SUNW,qlc@2/fp@0,0:devctllrwxrwxrwx 1 root root 49 Apr 17 18:14 fp2 ->../../devices/pci@a,2000/SUNW,qlc@2/fp@0,0:devctllrwxrwxrwx 1 root root 49 Apr 17 18:14 fp3 ->../../devices/pci@b,2000/SUNW,qlc@2/fp@0,0:devctllrwxrwxrwx 1 root root 50 Apr 17 18:14 fp4 ->../../devices/pci@12,2000/SUNW,qlc@2/fp@0,0:devctllrwxrwxrwx 1 root root 56 Apr 17 18:14 fp5 ->../../devices/pci@13,2000/pci@2/SUNW,qlc@4/fp@0,0:devctllrwxrwxrwx 1 root root 56 Apr 17 18:14 fp6 ->../../devices/pci@13,2000/pci@2/SUNW,qlc@5/fp@0,0:devctllrwxrwxrwx 1 root root 56 Apr 17 18:14 fp7 ->../../devices/sbus@7,0/SUNW,qlc@0,30400/fp@0,0:devctl


2. Open the /kernel/drv/fp.conf configuration file and explicitly enable or disablean HBA.

Add the property "mpxio-disable" to the HBA definition:

■ To enable multipathing on the port, set "mpxio-disable" to “no”.

■ To disable multipathing on the port, set "mpxio-disable" to “yes”.

3. Perform one of the following steps:

■ If you want to enable multipathing on the boot controller port, see “ConfiguringMultipathing Support for Storage Boot Devices” on page 45.

■ If you do not want to enable multipathing on the boot controller port at this time,perform a reconfiguration reboot now to shut down and restart your system byusing the following commands.

▼ To Configure a Single PCI HBAThe following steps display the configuration of a sample single PCI HBA.fp0 -> ../../devices/pci@6,2000/SUNW,qlc@2/fp@0,0:devctlThis sample HBA entry from the ls -l command output shown in “To ConfigureMultipathing by Port” on page 10 indicates a single PCI HBA.

1. The following lines are added to the fp.conf file:

■ To explicitly enable multipathing software on this port, add the following:

■ To explicitly disable multipathing software on this port, add the following:

2. Save and exit the file.


■ If you want to enable multipathing software on the boot controller port, see“Configuring Multipathing Support for Storage Boot Devices” on page 45.


name="fp" parent="/pci@6,2000/SUNW,qlc@2" port=0 mpxio-disable="no";

name="fp" parent="/pci@6,2000/SUNW,qlc@2" port=0 mpxio-disabl="yes";


■ If you do not want to enable multipathing software on the boot controller port atthis time, perform a reconfiguration reboot now to shut down and restart yoursystem by using the following commands:

▼ To Configure a Dual PCI HBAThe following steps display the configuration of a sample dual PCI HBA.fp6 -> ../../devices/pci@13,2000/pci@2/SUNW,qlc@5/fp@0,0:devctlThis sample HBA entry from the ls -l command output shown in “To ConfigureMultipathing by Port” on page 10 indicates a dual PCI HBA.

1. Add the following lines to the fp.conf file:








name="fp" parent="/pci@13,2000/pci@2/SUNW,qlc@5" port=0mpxio-disable="no";

name="fp" parent="/pci@13,2000/pci@2/SUNW,qlc@5" port=0mpxio-disable="yes";



▼ To Configure an SBus HBAThe following steps display the configuration of a sample SBus PCI HBA.fp7 -> ../../devices/sbus@7,0/SUNW,qlc@0,30400/fp@0,0:devctl ThisHBA entry from the ls -l command output shown in “To Configure Multipathingby Port” on page 10 indicates an SBus HBA.

1. Add the following lines to the fp.conf file:







Configuring Automatic Failback for SunStorEdge ArraysYou can configure the scsi_vhci.conf file so that primary paths fail backautomatically when they become available after being in an OFFLINE state. Thisfeature is applicable to Sun StorEdge T3+, 3900, and 6x20 arrays.

name="fp" parent="/sbus@7,0/SUNW,qlc@0,30400" port=0mpxio-disable="no";

name="fp" parent="/sbus@7,0/SUNW,qlc@0,30400" port=0mpxio-disable="yes";



For information on configuring your Sun StorEdge T3+, 3900 or 6x20 arrays, refer tothe documentation that came with your system.

▼ To Configure Automatic Failback Capability1. Open the scsi_vhci.conf file in your text editor.

2. Enable the automatic failback property.

3. Save and exit the scsi_vhci.conf file.

When automatic failback is enabled, the following message is printed in the/var/adm/messages file upon boot-up:

4. Perform a reconfiguration reboot so that the changes will take effect.

▼ To Disable Automatic Failback Capability1. Open the scsi_vhci.conf file in your text editor.

2. Disable the automatic failback property.


4. Perform a reconfiguration reboot so that the changes will take effect.

auto-failback = “enable”;

/scsi_vhci (scsi_vhci0): Auto-failback capability enabled throughscsi_vhci.conf file.


auto-failback = “disable”;



Configuring Third-Party SymmetricStorage Devices for MultipathingThe Solaris 10 Operating System software can provide multipathing and load-balancing to third-party symmetric storage devices, enabling greater deviceheterogeneity in your SAN or direct-connect storage environment.

Note – Before configuring any Third-Party Device, ensure that they are supported.Refer to your third-party user documentation, or third-party vendor for informationon proper product and vendor IDs, modes and various settings required by themultipathing software.

Considerations for Third-Party DeviceConfigurationBefore you configure third-party devices for multipathing, be aware of the following:

■ To use this functionality, you must edit parameters in the scsi_vhci.conf andfp.conf files.

■ You will need the storage devices vendor ID and product ID. You can obtain thevalues for the storage vendor_id and product_id variables in the scsi_vhci.conf

file by using the format command followed by the inquiry option on yoursystem. See the format(1M) man page.

■ Symmetric storage devices must be able to support the following functionality orcommands. Refer to your third party vendor for information.

■ Make all paths available for I/O, because paths will be accessed in a round-robin fashion.

■ Support the report_luns_scsi command.

■ Support scsi_inquiry command.

▼ To Configure Third-Party Devices1. Open the fp.conf file in a text editor such as vi(1M).


2. Enable mpxio.

3. Save and exit the fp.conf file.

4. Open the scsi_vhci.conf file in a text editor such as vi(1M).

5. Add the vendor_id and product_id properties.

The vendor ID (v_id) must be eight characters long. You must specify all eightcharacters. Trailing characters are spaces. Tabs and the tab character are not allowed.

The product ID (prod_id) can be up to 16 characters long. Trailing characters areblanks or spaces. Tabs and the tab character are not allowed.

Replace the variables with appropriate values for your system. For example:

6. Enable load balancing.

If you want the multipathing software to use all the available paths for loadbalancing, leave the load-balance field set to the default of “round-robin”.

Otherwise, change the definition to “none”.




■ If you do not want to enable multipathing software on the boot controller port atthis time, perform a reconfiguration reboot now to shut down and restart yoursystem by using the following commands:

mpxio-disable=”no”;

device-type-scsi-options-list="v_id prod_id ", "symmetric-option";symmetric-option=0x1000000;

device-type-scsi-options-list="ven-a pid_a_upto_here", "symmetric-option","ven-b pid_b_upto ", "symmetric-option","ven-c pid_c ", "symmetric-option";symmetric-option=0x1000000;



After you configure the scsi_vhci.conf file, confirm that the devices nodes arecreated under the /devices/scsi_vhci directory. If the devices were not created,confirm the vendor_id and product_id fields in the fp.conf file.

Configuring Solstice DiskSuite or SolarisVolume Manager for MultipathingSolstice DiskSuite or Solaris Volume Manager can be reconfigured to enable theSolaris 10 multipathing driver to control the HBAs that are supported by Sun. Usingthe stmsboot command, you can determine the mapping from the new multipathingdevice path name to the pre-multipathing device path name. This mapping will beused to reconfigure Solstice DiskSuite or Solaris Volume Manager to recognize thecorrect devices under the new multipathing device path names.

For more information concerning Solstice DiskSuite Solaris Volume Manager, checkthe Solstice DiskSuite or Solaris Volume Manager online manuals at:

http://docs.sun.com

Solaris DiskSuite or Solaris Volume ManagerConfiguration OverviewThe following is an overview of the steps to complete the configuration process. Adetailed example of this process follows.

Note – Backing up your data prior to performing any reconfiguration isrecommended.

1. Determine pre-multipathing Device Path Names by using the metadb command toget a list of devices used to store the volume manager configuration.

Use the metastat -p command to get the disk-to-metadevice mapping.


2. Unconfigure Solstice DiskSuite or Solaris Volume Manager for devices undermultipathing control.

Unmount the metadevices that will be under control of the multipathing software,and then clear the metadevices using the metaclear command.

Clear the metadevice database using the metadb -d -f command. This takes thelist of disks, output from the metadb command you performed in Step 1, as a list ofarguments.

3. Determine multipathing device name to pre-multipathing device name mapping.

Use the stmsboot -L command output to display the potential multipathing pathand device name changes and the luxadm display command to determine whichpre-multipathing device names are combined under each multipathing device name.

4. Enable multipathing.

5. Reconfigure the Solaris DiskSuite or Solaris Volume Manager

Create metadevices using multipathing path names with the metainit command.

Caution – For RAID-5 devices, be sure to used the “-k” option to preventinitialization of the disks.

Example of Migrating Mirrored DevicesThe following example shows how to migrate two-way mirrored metadevices. Inthis example:

■ A Sun StorEdge T3 partner pair is connected to the host, and mp_support on theSun StorEdge T3 array is set to rw.

■ multipathing software is initially disabled.

■ Four LUNs of equal size exist on the partner pair. Two metadb replicas exist onlun 0 (c2t1d0) and lun 1 (c2t1d1).

■ d10 and d11 are the sub-mirror metadevices created on the lun 2 (c2t1d2) andlun 3 (c2t1d3).

■ d14 is the mirror of d10 and d11.


▼ To Migrate Two-Way Mirrored Metadevices1. Check the metadevice paths.

# metastat -pd10 1 2 /dev/dsk/c2t1d2s1 /dev/dsk/c2t1d2s6d11 1 2 /dev/dsk/c2t1d3s1 /dev/dsk/c2t1d3s6d14 -m d10 d11


2. Save the pre-multipathing device information.

Collect and save the output of stmsboot -L, format, metadb, metastat, andmetastat -p commands.

See “To Display Potential /dev Device Name Changes” on page 48 for an exampleof stmsboot -L output.

# stmsboot -L

# formatSearching for disks...doneAVAILABLE DISK SELECTIONS:

0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>/pci@1f,4000/scsi@3/sd@0,0

1. c2t1d0 <T300 cyl 34145 alt 2 hd 24 sec 128>/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0ssd@w50020f2300000225,0

2. c2t1d1 <T300 cyl 34145 alt 2 hd 20 sec 128>/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w50020f2300000225,1

3. c2t1d2 <SUN-T300-0117 cyl 34145 alt 2 hd 32 sec 128>/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w50020f2300000225,2

Specify disk (enter its number):

# metadbflags first blk block count

a m pc luo 16 1034 /dev/dsk/c2t1d0s3a pc luo 1050 1034 /dev/dsk/c2t1d0s3a pc luo 16 1034 /dev/dsk/c2t1d1s3a pc luo 1050 1034 /dev/dsk/c2t1d1s3

# metastat -pd14 -m d10 d11 1d10 1 2 c2t1d2s1 c2t1d2s6 -i 32bd11 1 2 c2t1d3s1 c2t1d3s6 -i 32b


# metastatd14: Mirror Submirror 0: d10 State: Okay Submirror 1: d11 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 524288 blocksd10: Submirror of d14 State: Okay Size: 524288 blocks Stripe 0: (interlace: 32 blocks)

Device Start Block Dbase State Hot Spare c2t1d2s1 0 No Okay c2t1d2s6 0 No Okayd11: Submirror of d14 State: Okay Size: 524288 blocks Stripe 0: (interlace: 32 blocks)

Device Start Block Dbase State Hot Spare c2t1d3s1 0 No Okay c2t1d3s6 0 No Okay


3. Unconfigure the volume manager without losing the data.

a. Clear the submirrors and mirror devices.

b. Clear the meta database replicas.

c. Remove or comment out the /etc/lvm/md.tab entries.

4. Enable the multipathing software as described earlier in this chapter.

See “Enabling Multipathing Globally” on page 6 or “Enabling Multipathing on a PerPort Basis” on page 9.

# metaclear d14d14: Mirror is cleared# metaclear d11d11: Contact/Stripe is cleared# metaclear d10d10: Contact/Stripe is cleared

# metadb -d -f -c 2 /dev/dsk/c2t1d0s7 /dev/dsk/c2t1d1s7

# d10 1 2 /dev/dsk/c2t1d2s1 /dev/dsk/c2t1d2s6# d11 1 2 /dev/dsk/c2t1d3s1 /dev/dsk/c2t1d3s6# d14 -m d10 d11


5. Determine the multipathing software device name to pre-multipath device namemapping.

The output of stmsboot -L, format, luxadm probe, and luxadm display showsthe paths for each device. See “To Display Potential /dev Device Name Changes” onpage 48 for an example ofstmsboot -L output.

# formatAVAILABLE DISK SELECTIONS:


1. c4t60020F20000002253B220F99000D348Cd0 <SUN-T300-0117cyl 34145 alt 2 hd 32 sec 128>/scsi_vhci/ssd@g60020f20000002253b220f99000d348c

2. c4t60020F20000002253B220FC000086944d0 <SUN-T300-0117cyl 34145 alt 2 hd 32 sec 128>/scsi_vhci/ssd@g60020f20000002253b220fc000086944

3. c4t60020F20000002253B220FD400071CD8d0 <SUN-T300-0117cyl 34145 alt 2 hd 32 sec 128>/scsi_vhci/ssd@g60020f20000002253b220fd400071cd8

Specify disk (enter its number):

# luxadm probeNo Network Array enclosures found in /dev/es

Found Fibre Channel device(s): Node WWN:50020f20000001f6 Device Type:Disk device

Logical Path:/dev/rdsk/c4t60020F20000002253B22101D00029FC9d0s2 Node WWN:50020f2000000225 Device Type:Disk device

Logical Path:/dev/rdsk/c4t60020F20000002253B220FD400071CD8d0s2 Node WWN:50020f20000001f6 Device Type:Disk device

Logical Path:/dev/rdsk/c4t60020F20000002253B220FC000086944d0s2 Node WWN:50020f2000000225 Device Type:Disk device

Logical Path:/dev/rdsk/c4t60020F20000002253B220F99000D348Cd0s2

For each entry in luxadm probe, get the device path information.


(The following luxadm display output shows a single LUN only.)

# luxadm display /dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2DEVICE PROPERTIES for disk:/dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2 Vendor: SUN Product ID: T300 Revision: 0301 Serial Num: Unsupported Unformatted capacity: 35846.125 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0x0 Device Type: Disk device Path(s):

/dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2 /devices/scsi_vhci/ssd@g60020f20000007d44002e60f0002d8ca:c,raw Controller /devices/pci@6,2000/SUNW,jfca@2,1/fp@0,0 Device Address 50020f23000007d4,7 Host controller port WWN 200000017380a77e Class primary State ONLINE Controller /devices/pci@7,2000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000007a4,7 Host controller port WWN 210000e08b0a9675 Class secondary State STANDBY


6. Correlate the non-multipathing software device information collected in Step 1 bymatching the luxadm display paths output. Identify the appropriate LUN.

For example, find the c2t1d0 device. c2t1d0 is LUN 0 of the Sun StorEdge T3partner pair as seen from the format in Step 2.

From the luxadm display output of each device above, check the paths, controllerand device address fields.

Thus, in the above path:

■ c2t1d0 is now/dev/rdsk/c4t60020F20000002253B220F99000D348Cd0

Alternatively, the stmsboot -L command output also shows the device name changemapping.

7. Reconfigure the volume manager under the multipathing software by recreating/etc/lvm/md.tab with the new device names.

a. Re-create metadb database replicas with new device names.

b. Similarly, identify the metadevices with new device names by using themetatstat -p command.

1. c2t1d0 <T300 cyl 34145 alt 2 hd 24 sec 128>/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w50020f2300000225,0

/dev/rdsk/c4t60020F20000002253B220F99000D348Cd0s2/devices/scsi_vhci/ssd@g60020f20000002253b220f99000d348c:c,rawController /devices/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0Device Address 50020f2300000225,0

# metadb -a -f -c 2 /dev/rdsk/c4t60020F20000002253B220F99000D348Cd0s2/dev/rdsk/c4t60020F20000002253B220FC000086944d0s2


c. Recreate submirrors and mirror.

The best method for re-creating mirrors and submirrors is to create a one-waymirror and then, after verifying that the data is good, attach the second mirror.

d. Verify that all the data is now available in the one-way mirror device.

d14 contains the same data as earlier, before multipathing was enabled on thesame mirror.

e. Attach the second mirror.

# metainit d10# metainit d11# metainit d14 -m d10d14: Mirror is setup

# metattach d14 d11d14: Mirror is setup


CHAPTER 3

Configuring SAN Devices

This chapter provides information about configuring SAN devices and contains thefollowing sections:

■ “SAN Device Considerations” on page 29

■ “Adding SAN Devices” on page 30

■ “Adding SAN Devices” on page 30

■ “Fabric Device Node Configuration” on page 31

■ “Configuring Device Nodes Without Multipathing Enabled” on page 33

■ “Configuring Device Nodes With Multipathing Enabled” on page 40

SAN Device ConsiderationsBefore you configure the multipathing software, do the following.

■ In previous releases of Solaris, device paths had to be configured before use byusing the cfgadm -c configure command. If a path was not configured thestorage could not be accessed and was not visible to disk utilities such as format.In Solaris 10, paths are automatically configured, and the separate cfgadm step isnot necessary. All attached storage will be visible to the host. Likewise, storageneed not be unconfigured.

■ If you wish to use a Solaris Volume Manager (SVM) metadevice on a fabricattached disk, or a mirrored fabric boot device, you must use automatic pathconfiguration.

■ Manual configuration can be restored by adding the line enable_tapesry=on to/kernel/drv/fcp.conf.

■ Configure ports and zones according to the vendor-specific documentation forstorage and switches.

29

■ LUN masking enables specific LUNs to be seen by specific hosts. See yourvendor-specific storage documentation that describes masking.

■ Turn off power management on servers connected to the SAN to preventunexpected results as one server attempts to power down a device while anotherattempts to gain access. See power.conf(1M) for details about powermanagement.

■ Connect arrays and other storage devices to the SAN with or withoutmultipathing capability.

Adding SAN DevicesAdding and removing SAN devices requires knowledge of the following commands:

■ luxadm(1M)

■ format(1M)

■ fsck(1M)

■ newfs(1M)

■ cfgadm(1M) and cfgadm_fp(1M)

Note – If you use the format command when multipathing is enabled, you will seeonly one instance of a device identifier for each LUN. Without multipathing, youwill see one identifier for each path.

The cfgadm and cfgadm_fp commands are used most frequently to configure storagedevices on a SAN. Refer to the appropriate man page for detailed instructions abouthow to use each command. Appendix A contains information about thesecommands.

▼ To Add a SAN Device1. Create the LUN or LUNs desired.

2. If necessary, apply LUN masking for HBA control on the SAN device.

3. Connect the storage device to the system.

4. If necessary, create port-based or WWN zones on the switch on the SAN device.


5. If necessary, configure all paths to the storage device using the cfgadm -c

configure command on all the host bus adapters (HBAs) that have a path to thestorage device.

The cfgadm -c configure command creates device nodes. This step is necessary ifthe storage device is accessed by a host port connected to a fabric port and cfgadm isenabled.

6. Run the fsck or newfs commands on the device, if used for file systems.

7. Mount any existing file systems available on the storage device’s LUNs or diskgroups.

You might need to run the fsck command to repair any errors in the LUNs listed inthe /etc/vfstab file.

Fabric Device Node ConfigurationAfter you configure the hardware in your direct-attach system or SAN, you mustensure that the hosts recognize the switches and devices. This section explains hostrecognition of fabric devices, also known as 24-bit Fibre Channel addressing deviceson the SAN. After configuring the devices, ports, and zones in your SAN, you needto make sure that the host is aware of the devices and their switch connections. Youcan have up to 16 million fabric devices connected together on a SAN with FibreChannel support.

This section is limited to the operations required from the perspective of the SolarisOperating System. It does not cover other aspects, such as device availability anddevice-specific management. If devices are managed by other software, such as avolume manager, refer to the volume manager product documentation for additionalinstructions.

Ensuring that LUN-Level Information Is VisibleUse the cfgadm -al -o show_scsi_lun <controller_id> command to identify LUNlevel information.

If you issue the cfgadm -al -o show_scsi_lun <controller_id> commandimmediately after a system boots up, the output might not show the Fibre ChannelProtocol (FCP) SCSI LUN level information. The information does not appearbecause the storage device drivers, such as the ssd and st driver, are not loaded onthe running system. Use the modinfo command to check whether the drivers areloaded. After the drivers are loaded, the LUN level information is visible in thecfgadm output.

Chapter 3 Configuring SAN Devices 31

▼ To Detect Fabric Devices Visible on a HostThis section provides an example of the procedure for detecting fabric devices usingFibre Channel host ports c0 and c1. This procedure also shows the deviceconfiguration information that is displayed with the cfgadm(1M) command.

Note – In the following examples, only failover path attachment point IDs (Ap_Ids)are listed. The Ap_Ids displayed on your system depend on your systemconfiguration.

1. Become superuser.

2. Display the information about the attachment points on the system.

In this example, c0 represents a fabric-connected host port, and c1 represents aprivate, loop-connected host port. Use the cfgadm(1M) command to manage thedevice configuration on fabric-connected host ports.

By default, the device configuration on private, loop-connected host ports ismanaged by a host using the Solaris Operating System.

3. Display information about the host ports and their attached devices.

# cfgadm -lAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected unconfigured unknownc1 fc-private connected configured unknown

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected unconfigured unknownc0::50020f2300006077 disk connected unconfigured unknownc0::50020f23000063a9 disk connected unconfigured unknownc0::50020f2300005f24 disk connected unconfigured unknownc0::50020f2300006107 disk connected unconfigured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc1::220203708b8d45f2 disk connected configured unknownc1::220203708b9b20b2 disk connected configured unknown


Note – The cfgadm -l command displays information about Fibre Channel hostports. Also use the cfgadm -al command to display information about FibreChannel devices. The lines that include a port world wide name (WWN) in theAp_Id field associated with c0 represent a fabric device. Use the cfgadm configure

and unconfigure commands to manage those devices and make them available tohosts using the Solaris Operating System. The Ap_Id devices with port WWNsunder c1 represent private-loop devices that are configured through the c1 host port.

Configuring Device Nodes WithoutMultipathing EnabledThis section describes fabric device configuration tasks on a host that does not havemultipathing enabled.

The procedures in this section use specific devices as examples to illustrate how touse the cfgadm(1M) command to detect and configure fabric devices.

The devices attached to the fabric-connected host port are not configured by defaultand so are not available to the host using the Solaris Operating System. Use thecfgadm(1M) configure and unconfigure commands to manage device nodecreation for fabric devices. See the cfgadm_fp(1M) man page for additionalinformation.

The procedures in this section show how to detect fabric devices that are visible on ahost and to configure and make them available to a host using the Solaris OperatingSystem.

The device information that you supply and that is displayed by the cfgadm(1M)

command depends on your system configuration.

▼ To Configure an Individual Device WithoutMultipathingThis sample procedure describes how to configure a fabric device that is attached tothe fabric-connected host port c0.



2. Identify the device to be configured.

3. Configure the fabric device.

4. Verify that the selected fabric device is configured.

Notice that the Occupant column for both c0 and c0::50020f2300006077 displays asconfigured, indicating that the c0 port has a configured occupant and that thec0::50020f2300006077 device is configured.

Use the show_scsi_lun option to display FCP SCSI LUN information for multi-LUN SCSI devices.


# cfgadm -c configure c0::50020f2300006077

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected unconfigured unknownc0::50020f2300005f24 disk connected unconfigured unknownc0::50020f2300006107 disk connected unconfigured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc1::220203708b8d45f2 disk connected configured unknownc1::220203708b9b20b2 disk connected configured unknown


CODE EXAMPLE 3-1 shows that the physical devices connected through Ap_Id

c0:50020f2300006077 have four LUNs configured. The device is now available on

the host using the Solaris Operating System. The paths represent each SCSI LUN inthe physical device represented by c0::50020f2300006077. CODE EXAMPLE 3-2 is anexample of the luxadm(1M) output.

CODE EXAMPLE 3-1 show_scsi_lun Output Showing Two LUNs

# cfgadm -al -o show_scsi_lun c0

Ap_Id Type Receptacle Occupant Condition

c2 fc-fabric connected configured unknown

c0::50020f2300006077,0 disk connected configured unknown





CODE EXAMPLE 3-2 luxadm Output for Four Devices and a Single Array

# luxadm display 50020f2300006077

# luxadm display /dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2DEVICE PROPERTIES for disk:/dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2 Vendor: SUN Product ID: T300 Revision: 0301 Serial Num: Unsupported Unformatted capacity: 35846.125 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0x0 Device Type: Disk device Path(s):

/dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2 /devices/scsi_vhci/ssd@g60020f20000007d44002e60f0002d8ca:c,raw Controller /devices/pci@6,2000/SUNW,jfca@2,1/fp@0,0 Device Address 50020f23000007d4,7 Host controller port WWN 200000017380a77e Class primary State ONLINE Controller /devices/pci@7,2000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000007a4,7 Host controller port WWN 210000e08b0a9675 Class secondary State STANDBYDEVICE PROPERTIES for disk: 50020f23000007a4 Vendor: SUN Product ID: T300 Revision: 0301 Serial Num: Unsupported Unformatted capacity: 9217.688 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0x0 Device Type: Disk device Path(s):


/dev/rdsk/c14t60020F20000007D43EE74C7B00047E6Cd0s2 /devices/scsi_vhci/ssd@g60020f20000007d43ee74c7b00047e6c:c,raw Controller /devices/pci@6,2000/SUNW,jfca@2,1/fp@0,0 Device Address 50020f23000007d4,2 Host controller port WWN 200000017380a66e Class primary State ONLINE Controller /devices/pci@7,2000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000007a4,2 Host controller port WWN 210000e08b0a9675 Class secondary State STANDBY

DEVICE PROPERTIES for disk: 50020f23000007a4 Vendor: SUN Product ID: T300 Revision: 0301 Serial Num: Unsupported Unformatted capacity: 9217.688 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0x0 Device Type: Disk device Path(s):

/dev/rdsk/c14t60020F20000007D43EE74C630001A25Fd0s2 /devices/scsi_vhci/ssd@g60020f20000007d43ee74c630001a25f:c,raw Controller /devices/pci@6,2000/SUNW,jfca@2,1/fp@0,0 Device Address 50020f23000007d4,1 Host controller port WWN 200000017380a66e Class primary State ONLINE Controller /devices/pci@7,2000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000007a4,1 Host controller port WWN 210000e08b0a9675 Class secondary State STANDBY

CODE EXAMPLE 3-2 luxadm Output for Four Devices and a Single Array (Continued)


▼ To Configure Multiple Devices WithoutMultipathingMake sure you first identify the devices visible to the host with the procedure “ToDetect Fabric Devices Visible on a Host” on page 32. This procedure describes howto configure all unconfigured fabric devices that are attached to a fabric-connectedhost port. The port used as an example is c0.


DEVICE PROPERTIES for disk: 50020f23000007a4 Vendor: SUN Product ID: T300 Revision: 0301 Serial Num: Unsupported Unformatted capacity: 9217.688 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0x0 Device Type: Disk device Path(s):

/dev/rdsk/c14t60020F20000007D43EE74C93000612EBd0s2 /devices/scsi_vhci/ssd@g60020f20000007d43ee74c93000612eb:c,raw Controller /devices/pci@6,2000/SUNW,jfca@2,1/fp@0,0 Device Address 50020f23000007d4,3 Host controller port WWN 200000017380a66e Class primary State ONLINE Controller /devices/pci@7,2000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000007a4,3 Host controller port WWN 210000e08b0a9675 Class secondary State STANDBY

CODE EXAMPLE 3-2 luxadm Output for Four Devices and a Single Array (Continued)


2. Identify the devices to be configured.

3. Configure all of the unconfigured devices on the selected port.

Note – This operation repeats the configure operation of an individual device forall the devices on c0. This can be time consuming if the number of devices on c0 islarge.

4. Verify that all devices on c0 are configured.

Use the show_scsi_lun command to display FCP SCSI LUN information for multi-LUN SCSI devices.


# cfgadm -c configure c0

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc0::50020f2300005f24 disk connected configured unknownc0::50020f2300006107 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc1::220203708b8d45f2 disk connected configured unknownc1::220203708b9b20b2 disk connected configured unknown


CODE EXAMPLE 3-2 shows that the physical devices connected through Ap_Id

c0:50020f2300006077 have four LUNs configured. The example inCODE EXAMPLE 3-3 shows that the physical devices represented byc0::50020f2300006077 and c0::50020f2300006107 each have four LUNsconfigured. The physical devices represented by c0::50020f23000063a9 andc0::50020f2300005f24 each have two LUNs configured.

Configuring Device Nodes WithMultipathing EnabledThis section describes how to perform fabric device configuration steps on a hostthat has the multipathing software enabled.

The devices that are attached to fabric-connected HBA ports are not configured bydefault. These devices are thus not available to the host using the Solaris OperatingSystem when a host port is initially connected to a fabric. The procedures in thissection illustrate steps to detect fabric devices that are visible on a host and toconfigure them as multipathing devices to make them available to the host using theSolaris Operating System.

The device information that you supply, and that is displayed by the cfgadm(1M)

command, depends on your system configuration. For more information on thecfgadm command, see the cfgadm_fp(1M) and cfgadm(1M) man pages.

CODE EXAMPLE 3-3 show_scsi_lun Output for Multiple LUNs and Two Devices

# cfgadm -al -o show_scsi_lun c0


c0 fc-fabric connected configured unknown





c0::50020f23000063a9,0 disk connected configured unknown

c0::50020f23000063a9,1 disk connected configured unknown

c0::50020f2300005f24,0 disk connected configured unknown

c0::50020f2300005f24,1 disk connected configured unknown






▼ To Configure Individual Devices WithMultipathingThis sample procedure uses fabric-connected host ports c0 and c2 to configure fabricdevices as multipath devices on a host that has the multipathing software enabled.

Note – The cfgadm -c configure command for fabric devices is the sameregardless of whether or not multipathing is enabled, but the result is different.When the multipathing is enabled, the host using the Solaris Operating Systemcreates device node and path information that includes multipathing information.


2. Identify the port WWN of the device to be configured as a multipath device.

Look for devices on a fabric-connected host port, marked as fc-fabric. These arethe devices you can configure with the cfgadm -c configure command.

CODE EXAMPLE 3-4 cfgadm Listing of Fabric and Private-Loop Devices

In CODE EXAMPLE 3-4, the c0::50020f2300006077 and c2::50020f2300006107

Ap_Ids represent the same storage device with different port WWNs for the storagedevice controllers. The c0 and c2 host ports are enabled for use by multipathing.

3. Configure the fabric device and make multipathing devices available to the host.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected unconfigured unknownc0::50020f2300006077 disk connected unconfigured unknownc0::50020f23000063a9 disk connected unconfigured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc1::220203708b8d45f2 disk connected configured unknownc1::220203708b9b20b2 disk connected configured unknownc2 fc-fabric connected unconfigured unknownc2::50020f2300005f24 disk connected unconfigured unknownc2::50020f2300006107 disk connected unconfigured unknown

# cfgadm -c configure c0::50020f2300006077 c2::50020f2300006107


4. Verify that the selected devices are configured.

Notice that the Occupant column of c0 and c0::50020f2300006077 specifiesconfigured, which indicates that the c0 port has at least one configured occupantand that the c0::50020f2300006077 device is configured. The same change has beenmade in c2 and c2::50020f2300006107.

When the configure operation has been completed without an error, multipathing-enabled devices are created on the host using the Solaris Operating System. If thephysical device represented by c0::50020f2300006077 and c2::50020f2300006107

has multiple SCSI LUNs configured, each LUN is configured as a multipathingdevice. CODE EXAMPLE 3-5 shows that two LUNs are configured throughc0::50020f2300006077 and c2::50020f2300006107. Each Ap_Id is associated with apath to those multipath devices.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected unconfigured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc1::220203708b8d45f2 disk connected configured unknownc1::220203708b9b20b2 disk connected configured unknownc2 fc-fabric connected configured unknownc2::50020f2300005f24 disk connected unconfigured unknownc2::50020f2300006107 disk connected configured unknown

CODE EXAMPLE 3-5 show_scsi_lun Output for Two LUNs on a Device

# cfgadm -al -o show_scsi_lun c0::50020f2300006077c2::50020f2300006107







▼ To Configure Multiple Devices WithMultipathingBefore you configure or remove device nodes, be sure to first identify the fabricdevices by using the procedure “To Detect Fabric Devices Visible on a Host” onpage 32.

In this example, an Ap_Id on a fabric-connected host port is a path to a multipathdevice. For example, all devices with a path through c2 are to be configured, butnone through c0 are to be configured. c2 is an attachment point from the host to thefabric, whereas c2::50020f2300006107 is an attachment point from the storage tothe fabric. A host detects all the storage devices in a fabric for which it is configured.

Configuring an Ap_Id on a multipathing device that has already been configuredthrough another Ap_Id results in an additional path to the previously configureddevice. Note that a new Solaris device is not created in this case. A Solaris device iscreated only the first time an Ap_Id to a corresponding multipathing device isconfigured.


2. Identify the fabric-connected host port to be configured.

Devices represented by Ap_Ids c0::50020f2300006077 and c2::50020f2300006107

are two paths to the same physical device, with c0::50020f2300006077 alreadyconfigured. Configure the unconfigured devices on the selected port.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc1::220203708b8d45f2 disk connected configured unknownc1::220203708b9b20b2 disk connected configured unknownc2 fc-fabric connected unconfigured unknownc2::50020f2300005f24 disk connected unconfigured unknownc2::50020f2300006107 disk connected unconfigured unknown

# cfgadm -c configure c2


Note – This operation repeats the configure command of an individual device forall the devices on c2. This can be time-consuming if the number of devices on c2 islarge.

3. Verify that all devices on c2 are configured.

Notice that the Occupant column of c2 and all of the devices under c2 is marked asconfigured.

The show_scsi_lun command displays FCP SCSI LUN information for multipleLUN SCSI devices.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc1::220203708b8d45f2 disk connected configured unknownc1::220203708b9b20b2 disk connected configured unknownc2 fc-fabric connected configured unknownc2::50020f2300005f24 disk connected configured unknownc2::50020f2300006107 disk connected configured unknown


CHAPTER 4

Configuring Multipathing Supportfor Storage Boot Devices

This chapter provides information about configuring storage boot devices using thestmsboot command. It contains the following sections:

■ “About the stmsboot Command” on page 45

■ “Disabling Multipathing on the Boot Controller Port” on page 51

About the stmsboot CommandThe stmsboot command allows you to enable devices under the control of thevirtual controller. To display the /dev links the stmsboot command would create,use the stmsboot -l or stmsboot -L commands. The stmsboot command isapplicable to boot devices connected to Fibre Channel HBAs supported by Sun andto storage supported by multipathing. You cannot use this feature if the boot deviceis a SCSI or an IDE device.

Note – Do not add or remove devices from your configuration while enabling ordisabling boot capability with the stmsboot command.

45

The syntax of the stmsboot command is as follows:

stmsboot [-e | -d | -L | -l controller-number]

For additional information about the stmsboot command, see the stmsboot(1M)man page. A brief description of the syntax is presented here.

Considerations for stmsbootOperationsBefore performing stmsboot operations, be aware of the following:

Caution – After an stmsboot enable operation, do not remove /dev links of anydevice names that are not multipathing enabled devices for the boot device. Thisincludes, but is not limited to, executing commands such as devfsadm -C. Eventhough these boot device links are stale, they are needed if you want to disable thestmsboot features later. Removing these links might cause your system to lose thenecessary links to the devices that are not multipathing enabled, in which case thesystem will not boot. You then need to manually create the links by using the mknod

command.

■ You must back up data on the root device prior to performing the stmsboot

command.

■ You must first enable the multipathing software to use any stmsboot command.

Option Description

-e Enable the enumeration of devices connected to the boot controllerport under scsi_vhci (the multipathing virtual controller). Deviceson the boot controller port will be enumerated under the scsi_vhcivirtual controller and will be controlled by multipathing.

-d Disable the enumeration of devices connected to the boot controllerport under scsi_vhci. Devices on the boot controller port will beenumerated directly under the physical controller and will not becontrolled by multipathing.

-L Display the device name changes from non-multipathing devicenames to multipathing device names that would occur ifmultipathing were configured on this controller.

-l controller-number Display the device name changes from non-multipathing devicenames to multipathing device names that would occur for the givencontroller if multipathing were configured on this controller.


■ Plan to perform an immediate reconfiguration reboot of the host after enabling ordisabling multipathing on a boot controller port. Do not perform other systemtasks or other activities until you perform the reconfiguration reboot.

■ Once enabled, the stmsboot feature has a dependency on the boot path. Uponsuccessful completion of the stmsboot -e command, a per-port mpxio-disableentry for the boot controller path is added automatically to the/kernel/drv/fp.conf file. Do not change the boot path manually. Use the stmsboot

-d command to disable a configured boot path. This command automaticallyupdates both the eeprom-boot device and the /kernel/drv/fp.conf file’s port-related mpxio-disable entry. This command then allows the system to boot byusing the selected path.

■ If your system fails during an enabling or disabling stmsboot operation, youroriginal /etc/path_to_inst, /etc/vfstab, /etc/system, and/etc/dumpadm.conf files are saved with .sav file name extensions in the/var/tmp directory. The saved files can be useful in recovering from anyunexpected system crash.

■ If any Sun StorEdge T3, 3900, or 6x20 arrays are connected to the boot controllerport, modify the settings when prompted to do so by the stmsboot command.

■ After the stmsboot enable or disable operation, device names will change.Disable any volume manager that is actively using devices connected to the bootcontroller port, and reconfigure these devices with new multipathing devicenames. Some applications that use device names that are not multipathingenabled will need to be modified.

Displaying Potential /dev Device Name ChangesYou can determine what the potential device name changes will be beforeperforming an stmsboot command. This allows you to modify applications that usethe boot controller port after the system is rebooted. To help ease the transition tonewly-named boot controller devices, save the display of the device name changesfor the boot controller port.

Chapter 4 Configuring Multipathing Support for Storage Boot Devices 47

▼ To Display Potential /dev Device Name Changes

● Display the potential /dev links for all controllers, using the stmsboot -L

command.

● Display potential /dev links for one controller by using the stmsboot -l

controller-number command.

For example, to display the device name change for the boot controller port, obtainthe device name (cxtydz) corresponding to the root (/) directory from the/etc/vfstab file, where x is the controller number. In this example, x is equal to 3.

Enabling Multipathing on the BootController PortIn order to enable multipathing on the boot controller port, perform the following:

# stmsboot -LVersion : 1.8non-STMS device name STMS device name

/dev/rdsk/c1t64d0 /dev/rdsk/c9t20000020371A1D48d0/dev/rdsk/c1t65d0 /dev/rdsk/c9t20000020371A13F3d0

/dev/rdsk/c4t64d0 /dev/rdsk/c9t20000020371A1D48d0/dev/rdsk/c4t65d0 /dev/rdsk/c9t20000020371A13F3d0

# stmsboot -l 3Version: 1.8non-STMS device name STMS device name

/dev/rdsk/c3t64d0 /dev/rdsk/c9t20000020371A1D48d0/dev/rdsk/c3t65d0 /dev/rdsk/c9t20000020371A13F3d0/dev/rdsk/c3t66d0 /dev/rdsk/c9t20000020371A15D0d0/dev/rdsk/c3t67d0 /dev/rdsk/c9t20000020371A1605d0


▼ To Enable Multipathing on the Boot ControllerPortBefore you enable multipathing, see “Considerations for stmsboot Operations” onpage 46.


2. If multipathing is not enabled, perform the procedure described in “EnablingMultipathing Globally” on page 6.

3. If any applications are running on any device connected to the boot controllerport, deactivate them.

4. Verify that no devlinks or disks commands are running. If these commands arerunning, wait until they are finished.

5. Enable the multipathing software on the boot controller port.

6. Type y to continue.

The software then prompts you to perform a reconfiguration reboot.

# ps -elf | grep devlinks

# ps - elf | grep disks

# stmsboot -eVersion: 1.8WARNING!!! Important system files will be modified.

The devfsadmd daemon will be killed.New STMS links for devices connected to the boot controller portwill be created.

Please make sure that there are no processes running devfsadm,devlinks, or disks from now until the system is rebooted.This includes, but is not limited to, initiating DR events.

Do you want to continue ? [y/n] (default: n)


Note – In case of an unexpected error, the stmsboot operation will fail, indicatingan error condition, and then exit. Some device links might have been created whichare useless but the system files remain unmodified. Correct the error and repeat theoperation.

If Sun StorEdge T3, 3900, or 6x20 arrays are connected to the boot controller port, thesoftware will prompt you to change the mp_support setting on the storage arrayappropriately. Refer to your array documentation for additional information.

Note – Do not complete other tasks before rebooting your system.

7. Type y to perform a reconfiguration reboot of your system.

8. If necessary, modify applications to use the new multipathing device names.

Boot controller is c1

Listed below are the non-STMS device names and their correspondingnew STMS device names.

/dev/rdsk/c1t4d0 /dev/rdsk/c9t60020F200000024A3CDAA4BC00051BE5d0/dev/rdsk/c1t4d1 /dev/rdsk/c9t60020F200000024A3DCBC93E00095B12d0/dev/rdsk/c1t4d2 /dev/rdsk/c9t60020F200000024A3DCBC95700049C40d0

WARNING!!!There is at least one Sun StorEdge T3/T3B, or 6x20,that is not in "mpxio" mode.For the system to function properly, please setmp_support to "mpxio" mode on the device(s) now andpress any key to continue.

A reconfiguration reboot is mandatory for system sanity.While you may choose to complete other tasks prior to initiatingthe reboot, doing so is not recommended.Reboot the system now ? [y/n] (default:y)


Disabling Multipathing on the BootController PortDisplaying multipathing on the boot controller port changes the device names frommultipathing recognized device names to device names that are not recognized bymultipathing. If any applications are running on any device connected to the bootcontroller port, deactivate them. You might need to modify some applications aftersystem is rebooted to use device names that are not recognized by multipathing.

This stmsboot disable operation will have no effect on your system if you did notpreviously perform an stmsboot enable operation.

▼ To Disable Multipathing on the Boot ControllerPort


2. Verify that no devlinks or disks commands are running. If these commands arerunning, wait until they have finished.

3. Disable device configuration.

# ps - elf | grep devlinks

# ps -elf | grep disks

# stmsboot -dVersion: 1.5

WARNING!!! Important system files will be modified. The devfsadmd daemon will be killed.

Please make sure that there are no processes running devfsadm,devlinks, or disks from now on until the system is rebooted.This includes but, is not limited to initiating DR events.

Do you want to continue ? [y/n] (default: n)


4. Type y to continue.

Note – In case of an unexpected error, the stmsboot operation will fail, indicatingan error condition, and then exit. Correct the error and you can repeat the operation.

5. Type the device path and press Return.

This example shows the results for choice 2. The software then prompts you toperform a reconfiguration reboot.

Note – If Sun StorEdge T3, 3900, or 6x20 arrays are connected to the boot controllerport, the software will prompt you to change the mp_support setting on the storagearray appropriately. To ensure that failover works, select the ONLINE path.

6. Type y to perform a reconfiguration reboot of your system.

Please choose a device path [1 - 2 ]WARNING: Devices connected to the selected device pathwill be de enumerated from scsi_vhci1) /devices/pci@1f,4000/pci@4/SUNW,qlc@5/fp@0,02) /devices/pci@1f,4000/pci@2/SUNW,qlc@4/fp@0,0Choice [default: 1]:

Choice [default: 1]: 2

Updated eeprom boot-device to boot through the selected path

/pci@8,600000/SUNW,qlc@2/fp@0,0/disk@w21000004cf721119,0

WARNING!!!There is at least one Sun StorEdge T3/T3B or 6x20 in "mpxio" mode.For the sane operation of the system, please setmp_support to "rw" (or any appropriate mode otherthan "mpxio") on the device(s) now andpress any key to continue.

A reconfiguration reboot is mandatory for system sanity.While you may choose to complete other tasks prior to initiatingthe reboot, doing so is not recommended.Reboot the system now ? [y/n] (default:y)


CHAPTER 5

Configuring IPFC SAN Devices

To configure IP over Fibre Channel (IPFC), you should have already ensured that thehosts recognize the switch and all attached devices. This chapter describes hostrecognition of IPFC devices and implementation of IP over Fibre Channel in a SAN.The IPFC driver is based on RFC 2625 and it allows IP traffic to run over FibreChannel. This chapter contains the following topics:

■ “Loading IPFC” on page 53

■ “Invoking and Configuring IPFC” on page 58

Loading IPFCConfiguration of IPFC depends on the instance of FP, or host bus adapter ports. Ifmultiple host bus adapters (HBAs) are present, plumb manually after identifying theFP instance on which IP should be plumbed. The following sections describe thetasks required for identifying FP instances:

■ “IPFC Considerations” on page 53

■ “To Plumb an IPFC Instance” on page 57

IPFC ConsiderationsIPFC devices are supported for use with Network File System (NFS) software,Network Attached Storage (NAS) devices, and Sun StorEdge Network DataReplicator (Sun SNDR) software, or Sun StorEdge Availability Suite 3.1 remotemirror software

53

TABLE 5-1 shows the supported features available for IPFC.

The following restrictions apply:

■ IPFC is not supported on 1 Gbit Sun switches.

■ Promiscuous mode is not supported. The snoop(1M) utility cannot be used.

■ Multicasting is supported through broadcasting only.

■ You must assign the IP address of the IPFC port to a subnet different from that ofthe Ethernets on the same system.

■ Network cards using IPFC cannot be used as routers. The /etc/notrouter filemust be present on the host.

■ With IPFC, storage devices and hosts should be in separate zones. The storagedevice should have one path to one zone and another path to another zone forfailover and redundancy. The host can have more than one path to a specifiedzone, and it should have at least one path to each zone so that it can see therespective storage.

■ Any standard network commands can be used after IPFC is attached. There arenot any usage differences when these commands (telnet, ping, or ftp) areused in an Ethernet setup.

■ Turn off power management on servers connected to the SAN to preventunexpected results as one server attempts to power down a device while anotherattempts to gain access. See power.conf(1M) for details about powermanagement.

Determining Fibre Channel Adapter PortInstancesThere are two basic ways to determine Fibre Channel adapter port instances towhich IP can be plumbed. The first way requires that you know the world widename (WWN) of the card. The second way requires you to know the physicallocation of the card. The procedures include:

■ “To Determine Port Instances from the WWN” on page 55

TABLE 5-1 IPFC (NFS/NAS and SNDR)

Feature Supported

Cascading Yes, with fabric zones only

Zone type Fabric zone (with the HBA configured as anF-port point-to-point connection)

Maximum number of device ports per zone 253


■ “Invoking and Configuring IPFC” on page 58

▼ To Determine Port Instances from the WWN


2. Determine the fp driver instances in your system.

In the example below, there are four instances (0 through 3) of FP present in thesystem.

3. Manually load IPFC to each desired FP instance.

Use the ifconfig fcip0 plumb command, where fcip0 is a variable for the desiredfp instance number. For example:

When the command is successful, a message appears on both the console and in themessages file. For example:

If no other error message is displayed, manual plumbing has succeeded.

▼ To Determine Port instances From the Physical Device Path

1. Determine the HBA PCI adapter slot and the I/O board PCI slot.

You need this information to perform the calculation in Step 2.

For example, assume you have an array with an HBA card located in PCI adapterslot 5, and the PCI adapter is in slot 1 of the I/O board.

2. Determine the instance number.

# prtconf -v | grep fp

fp (driver not attached)fp, instance #0fp (driver not attached)fp, instance #1fp (driver not attached)fp, instance #2fp (driver not attached)fp, instance #3

# ifconfig fcip0 plumb

Sep 13 15:52:30 bytownite ip: ip: joining multicasts failed (7) onfcip0 - will use link layer broadcasts for multicast

Chapter 5 Configuring IPFC SAN Devices 55

a. Use an editor to search for the fp driver binding name in the/etc/path_to_inst file.

Entries have fp on the line.

Note – Determine the correct entry by finding the hardware path described in yourserver hardware manual or Sun System Handbook. The Sun System Handbook isavailable at http://sunsolve.sun.com/handbook_pub/.

b. Narrow the search by using the I/O board and slot information from Step 1.

Note – The following method of deriving Solaris device path of an HBA from itsphysical location in server may not work for all Sun server hardware.

i. Multiply the PCI adapter slot number by the number of adapter ports.

For example, if the HBA has two ports, multiply by 2. Using the array with anHBA in the PCI adapter slot 5, multiply 5 by 2 to get 10.

ii. Add the PCI adapter I/O board slot number to the number derived in Step i.

Using an HBA in PCI adapter slot 5 and PCI slot 1 of the I/O board, add 1 to10 for a sum of 11.

iii. Convert the number derived in Step ii to hexadecimal.

The number 11 converts to b in hexadecimal.

iv. Search for the fp entry with pci@hex where hex is the number you derivedin Step iii.

CODE EXAMPLE 5-1 shows a single Fibre Channel network adapter device path.TABLE 5-2 describes the elements of the device path.

CODE EXAMPLE 5-1 PCI Single Fibre Channel Network Adapter Device Path

"/pci@b,2000/SUNW,qlc@2/fp@0,0" 7 "fp"

TABLE 5-2 PCI Single Fibre Channel Network Adapter /etc/path_to_inst DevicePath Entry

Entry Item Entry Value

Physical Name /pci@b,2000/sunw,qlc@2/fp@0,0

Instance Number 7

Driver BindingName

fp


3. Manually plumb each FP instance.

Use the ifconfig <interface_number> plumb command. In this example, the value of<interface_number> is fcip7.

When the command is successful, a message appears on both the console and in themessages file. For example:

▼ To Plumb an IPFC InstanceEach FP instance on the system has an entry in /dev/fc. If HBAs have beenremoved, some stale links might exist. Use this procedure to load and plumb IPFC.

1. For each entry in /dev/fc, issue a luxadm -e dump_map command to view all thedevices that are visible through that HBA port:

2. Based on the list of devices, determine which destination HBAs are visible to theremote host with which you want to establish IPFC communications.

In the example for this procedure, the destination HBAs have port IDs 610100 and620d02. The originating HBA’s port ID is 620e00.


Sep 13 15:52:30 bytownite ip: ip: joining multicasts failed (7) onfcip0 - will use link layer broadcasts for multicast

# luxadm -e dump_map /dev/fc/fp0Pos Port_ID Hard_Addr Port WWN Node WWN Type0 610100 0 210000e08b049f53 200000e08b049f53 0x1f(Unknown Type)1 620d02 0 210000e08b02c32a 200000e08b02c32a 0x1f(Unknown Type)2 620f00 0 210000e08b03eb4b 200000e08b03eb4b 0x1f(Unknown Type)3 620e00 0 210100e08b220713 200100e08b220713 0x1f(Unknown Type,Host Bus Adapter)# luxadm -e dump_map /dev/fc/fp1 No FC devices found. - /dev/fc/fp1


3. List the physical path of the originating HBA port from which you can see thedestination HBA port, where originating-hba-link is a variable for the linkdetermined in Step .

For example, here 0 is the number for the originating-hba-link:

4. Search the physical path identified in Step 3.

You must remove the leading ../../devices from the path name output. Forexample

5. Determine the fp instance for the originating HBA port from the output of thecommand in Step 4.

The instance number precedes “fp” in the output. In the following example output,the instance number is 0.

6. Use the instance number from Step 5 to load IPFC and plumb the IPFC interface.

In this example, the instance is 0.

Invoking and Configuring IPFCImmediately upon installation, start IPFC manually with the ifconfig command.You can configure the host so that on subsequent reboot, the IPFC network interfacestarts automatically. This section describes both procedures:

# ls -l /dev/fc/fp originating-hba-link

# ls -l /dev/fc/fp0lrwxrwxrwx 1 root root 51 Sep 4 08:23 /dev/fc/fp0 ->../../devices/pci@8,600000/SUNW,qlc@1/fp@0,0:devctl

# grep pci@8,600000/SUNW,qlc@1/fp@0,0 /etc/path_to_inst"/pci@8,600000/SUNW,qlc@1/fp@0,0" 0 "fp"

"/pci@8,600000/SUNW,qlc@1/fp@0,0" 0 "fp"



■ “To Start a Network Interface Manually” on page 59■ “To Configure the Host for Automatic Plumbing Upon Reboot” on page 59

▼ To Start a Network Interface ManuallyUse this procedure when you want to plumb IPFC with specific netmask values andget the IPFC interface up and running.

1. Use the ifconfig command with the appropriate interface.

Ask your network administrator for an appropriate IP address and netmaskinformation. For example, to enable an IPFC interface associated with fp instance 0and an IP address of 192.9.201.10, type:

The ifconfig command is described in more detail in the ifconfig(1M)manpage.

2. Use the command ifconfig -a to verify the network is functioning.

The output of ifconfig -a should look like this:

▼ To Configure the Host for Automatic PlumbingUpon RebootEach network interface must have an /etc/hostname.interface file defining thename of the IP address associated with it. For example, IPFC network interfacefcip0 has a file named /etc/hostname.fcip0.

# touch /etc/notrouter# ifconfig fcip0 inet 192.9.201.10 netmask 255.255.255.0 up

lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4>mtu 8232 index 1inet 127.0.0.1 netmask ff000000fcip0: flags=1001843<UP,BROADCAST,RUNNING,MULTICAST,MULTI_BCAST,IPv4>mtu 1500 index 2inet 192.9.201.10 netmask ffffff00 broadcast 192.9.201.255 ether0:e0:8b:1:3c:f7hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4>mtu 1500 index 3inet 192.9.200.70 netmask ffffff00 broadcast 192.9.200.255ether 8:0:20:fc:e9:49


1. Manually create a /etc/hostname.interface file with a text editor so it contains asingle line that identifies the host name or interface IP address.

2. Use a text editor to make any additional entries to the /etc/inet/hosts file.

The Solaris installation program creates the /etc/inet/hosts file with minimumentries. You must manually make additional entries with a text editor. (See thehosts(4) man page for additional information.)

The /etc/inet/hosts file contains the hosts database. This file contains the hostnames and the primary network interface IP addresses, as well as the IP addresses ofother network interfaces attached to the system and of any other network interfacesthat the machine must know about.

CODE EXAMPLE 5-2 shows an example of an etc/inet/host file.

CODE EXAMPLE 5-2 sun1 machine /etc/inet/hosts

3. Edit the /etc/nsswitch.conf file so that all un-commented entries have theword files before any other nameservice.

The /etc/nsswitch.conf specifies which name service to use for a particularmachine. CODE EXAMPLE 5-3 shows an example of an /etc/nsswitch.conf file.

CODE EXAMPLE 5-3 sun1 machine /etc/nsswitch.conf File

127.0.0.1 localhost loghost192.9.200.70 sun1 #This is the local host name192.9.201.10 fcip0 #Interface to network 192.9.201.10

hosts: files nis


CHAPTER 6

Unconfiguring Fabric Devices

This chapter provides information about unconfiguring the multipathing software.

This section includes:

■ “Unconfiguring Fabric Devices” on page 61

Unconfiguring Fabric DevicesBefore you unconfigure a fabric device, stop all activity to the device and unmountany file systems on the fabric device. See the administration documentation for theSolaris Operating System for unmounting instructions. If the device is under anyvolume manager’s control, see the documentation for your volume manager beforeunconfiguring the device.

▼ To Unconfigure a Fabric DeviceThis procedure describes how to unconfigure a fabric device that is attached to thefabric-connected host port c0.


61

2. Identify the device to be unconfigured.

Only devices on a fabric-connected host port can be unconfigured.

3. Unconfigure the fabric device.

4. Verify that the selected fabric device is unconfigured.

▼ To Unconfigure All Fabric Devices on aFabric-Connected Host PortThis procedure describes how to unconfigure all configured fabric devices that areattached to a fabric-connected host port.


# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknown

# cfgadm -c unconfigure c0::50020f2300006077

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected unconfigured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknown


2. Identify the fabric devices to be unconfigured.

Only devices on a fabric-connected host port can be unconfigured.

3. Stop all activity to each fabric device on the selected port and unmount any filesystems on each fabric device. If the device is under any volume manager’scontrol, see the documentation for your volume manager before unconfiguring thedevice.

4. Unconfigure all of the configured fabric devices on a selected port.

Note – This operation repeats the unconfigure operation of an individual devicefor all the devices on c0. This can be time-consuming if the number of devices on c0

is large.

5. Verify that all the devices on c0 are unconfigured.

Notice that the Occupant column of c0 and all the fabric devices attached to it aredisplayed as unconfigured.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknown

# cfgadm -c unconfigure c0

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected unconfigured unknownc0::50020f2300006077 disk connected unconfigured unknownc0::50020f23000063a9 disk connected unconfigured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknown

Chapter 6 Unconfiguring Fabric Devices 63

▼ To Unconfigure a Fabric Device Associated WithMultipathing Enabled DevicesThis procedure shows fabric-connected host ports c0 and c2 to illustrate how tounconfigure fabric devices associated with multipathing devices.


2. Identify the port WWN of the fabric device to be unconfigured.

In this example, the c0::50020f2300006077 and c2::50020f2300006107 Ap_Idsrepresent different port WWNs for the same device associated with a multipathingdevice. The c0 and c2 host ports are enabled for use by the multipathing software.

3. Stop all device activity to each fabric device on the selected port and unmount anyfile systems on each fabric device. If the device is under any volume manager’scontrol, see the documentation for your volume manager for maintaining thefabric device.

4. Unconfigure fabric devices associated with the multipathing device.

Only devices on a fabric-connected host port can be unconfigured through thecfgadm -c unconfigure command.

Note – You can remove a device from up to eight paths individually, as in theexample command cfgadm -c unconfigure c0::1111, c1::2222, c3::3333, etc.As an alternative, you can remove an entire set of paths from the host, as in theexample cfgadm -c unconfigure c0.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc2 fc-fabric connected configured unknownc2::50020f2300005f24 disk connected configured unknownc2::50020f2300006107 disk connected configured unknown

# cfgadm -c unconfigure c0::50020f2300006077 c2::50020f2300006107


5. Verify that the selected devices are unconfigured.

Notice that the Ap_Ids c0::50020f2300006077 and c2::50020f2300006107 areunconfigured. The Occupant column of c0 and c2 still displays those ports asconfigured because they have other configured occupants.

The multipath devices associated with the Ap_Ids c0::50020f2300006077 andc2::50020f2300006107 are no longer available to the host using the SolarisOperating System. The following two multipath devices are removed from the host:

/dev/rdsk/c6t60020F20000061073AC8B52D000B74A3d0s2

/dev/rdsk/c6t60020F20000061073AC8B4C50004ED3Ad0s2

▼ To Unconfigure One Path to a MultipathingDeviceIn contrast to the procedure in the preceding section, this procedure shows how tounconfigure one device associated with c2::50020f2300006107 and leave the otherdevice, 50020f2300006077, configured. Only devices on a fabric-connected host portcan be unconfigured through the cfgadm unconfigure command.


# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected unconfigured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc2 fc-fabric connected configured unknownc2::50020f2300005f24 disk connected configured unknownc2::50020f2300006107 disk connected unconfigured unknown


2. Identify the Ap_Id of the multipathing device to be unconfigured.

In this example, c0::50020f2300006077 and c2::50020f2300006107 Ap_Idsrepresent different port WWNs for the same device.

3. Unconfigure the Ap_Id associated with multipathing device.

Note – If the Ap_Id represents the last configured path to the device, stop allactivity to the path and unmount any file systems on it. If the multipathing device isunder any volume manager’s control, see the documentation for your volumemanager for maintaining the fabric device.

In the example that follows, the path represented as c2::50020f2300006107 isunconfigured, and c0::50020f2300006077 remains configured to show how you canunconfigure just one of multiple paths for a multipathing device.

4. Verify that the selected path c2::50020f2300006107 is unconfigured.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc2 fc-fabric connected configured unknownc2::50020f2300005f24 disk connected configured unknownc2::50020f2300006107 disk connected configured unknown

# cfgadm -c unconfigure c2::50020f2300006107

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc2 fc-fabric connected configured unknownc2::50020f2300005f24 disk connected configured unknownc2::50020f2300006107 disk connected unconfigured unknown


The multipathing devices associated with that Ap_Id are still available to a hostusing the Solaris Operating System through the other path, represented byc0::50020f2300006077. A device can be connected to multiple Ap_Ids and anAp_Id can be connected to multiple devices.

▼ To Unconfigure All Fabric-Connected DevicesWith Multipathing EnabledAn Ap_Id on a fabric-connected host port is a path to a multipathing device.

When a multipathing device has multiple Ap_Ids connected to it, the device is stillavailable to the host using the Solaris Operating System after you unconfigure anAp_Id. After you unconfigure the last Ap_Id, no additional paths remain and themultipathing device is unavailable to the host using the Solaris Operating System.Only devices on a fabric-connected host port can be unconfigured


2. Identify the devices to be unconfigured.

3. Unconfigure all of the configured devices on the selected port.

Note – This operation repeats the unconfigure command of an individual devicefor all devices on c2. This can be time-consuming if the number of devices on c2 islarge.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc0::50020f23000063a9 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708b9b20b2 disk connected configured unknownc2 fc-fabric connected configured unknownc2::50020f2300005f24 disk connected configured unknown

# cfgadm -c unconfigure c2


4. Verify that all devices on c2 are unconfigured.

Notice that the Occupant column lists c2 and all the devices attached to c2 asunconfigured.

# cfgadm -alAp_Id Type Receptacle Occupant Conditionc0 fc-fabric connected configured unknownc0::50020f2300006077 disk connected configured unknownc1 fc-private connected configured unknownc1::220203708b69c32b disk connected configured unknownc1::220203708ba7d832 disk connected configured unknownc2 fc-fabric connected unconfigured unknownc2::50020f2300005f24 disk connected unconfigured unknownc2::50020f2300006107 disk connected unconfigured unknown


APPENDIX A

Multipathing ConfigurationSamples

This appendix describes two different types of configurations:

■ A system that does not use multipathing driver software

■ A system that uses multipathing driver software

Topics in this section include:

■ “About Multipathing Configuration Samples” on page 69

■ “Configuration without Multipathing” on page 70

■ “Configuration with Multipathing” on page 71

About Multipathing ConfigurationSamplesMultipathing device enumeration is different from legacy devices in that only onedevice path is shown per device regardless of the number of paths.

Storage devices such as the Sun StorEdge A3x00FC have their own multipathingsolution. multipathing can co-exist with such storage devices. However, thesedevices will not be enumerated under multipathing and will work the same as ifmultipathing were not installed.

Case Study: A host has the following storage attached.

■ Sun StorEdge T3 partner pair with 4 LUNs.

■ Sun StorEdge A5200 with both A and B loop connected.

69

Configuration without MultipathingBefore multipathing was installed and configured, the Sun StorEdge T3mp_support was set to “rw” format. The output is different from the configurationthat included multipathing.

Utility: formatThe output for a configuration that does not use multipathing is as follows:


0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>/pci@1f,4000/scsi@3/sd@0,

1. c2t3d0 <SUN-T300-0117 cyl 34145 alt 2 hd 24 sec 128>/pci@1f,4000/SUNW,qlc@2/fp@0,0/ssd@w50020f23000042d4,0




5. c3t4d0 <SUN-T300-0117 cyl 34145 alt 2 hd 24 sec 128>/pci@1f,4000/SUNW,qlc@4/fp@0,0/ssd@w50020f2300003fad,0




9. c4t68d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>/pci@1f,2000/pci@1/SUNW,qlc@4/fp@0,0ssd@w22000020371a1862,0

.

.

.>Specify disk (enter its number): ^D


Configuration with MultipathingAfter multipathing is installed and configured, Sun StorEdge T3 mp_support is setto “mpxio”.

Utility: formatThe output of format looks as follows:

Consider the following notes:

■ Devices enumerated under multipathing will have a /scsi_vhci/ssd entry.

■ The first four scsi_vhci entries are the four T3 LUNs which now have a longname which is Global Unique Identifier (GUID). Also, only one entry per LUN isvisible instead of two paths.

■ The next scsi_vhci entry is a disk in the Sun StorEdge A5200 array.

■ There is only one controller number assigned for all the devices encapsulatedunder multipathing.



1. c6t60020F20000042D43ADCBC4E000C41E2d0 <SUN-T300-0117cyl 34145 alt 2 hd 24 sec 128>/scsi_vhci/ssd@g60020f20000042d43adcbc4e000c41e2

2. c6t60020F20000042D43B0E926A000AA3FCd0 <SUN-T300-0116cyl 34145 alt 2 hd 24 sec 128>/scsi_vhci/ssd@g60020f20000042d43b0e926a000aa3fc

3. c6t60020F20000042D43B2753510008C9DFd0 <SUN-T300-0117cyl 34145 alt 2 h d 24 sec 128>/scsi_vhci/ssd@g60020f20000042d43b2753510008c9df

4. c6t60020F20000042D43B275377000877DDd0 <SUN-T300-0117cyl 34145 alt 2 h d 24 sec 128>/scsi_vhci/ssd@g60020f20000042d43b275377000877dd

5. c6t20000020371A1862d0 <SUN36G cyl 24620 alt 2 hd 27 sec107>

/scsi_vhci/ssd@g20000020371a1862Specify disk (enter its number): ^D

Appendix A Multipathing Configuration Samples 71

■ Use luxadm display to identify the mapping between device entries withoutmultipathing and the device entries with multipathing.

Utility: luxadm probe

The luxadm probe now shows the WWN for Sun StorEdge A5200 and GUID for T3.

Utility: luxadm display

The luxadm display has been enhanced for multipathing. For each entry in theformat or luxadm probe, luxadm display output indicates the following:

■ The number of paths to the storage device.

■ The mapping of the paths prior to multipathing and after multipathing (underPath(s) controller and device address).

■ The state and type of each path.

■ ONLINE indicates the active path(s) through which IO is going to the device.For more than one ONLINE path, multipathing will use load balancing (likeround robin scheme) or single IO to the device, depending on the setting ofload-balance variable in /kernel/drv/scsi_vhci.conf file.

■ STANDBY indicates the path is available if an ONLINE path fails or is switchedto another state. There can be many STANDBY paths. If a STANDBY path ischosen to be an active path for routing IO, its status will be changed toONLINE.

■ OFFLINE indicates the path(s) was previously existing but is not available now.

■ Class type for each path.

■ PRIMARY: This path is the preferred path for routing IO.

# luxadm probeFound Enclosure:SENA Name:f Node WWN:50800200000777d0 Logical Path:/dev/es/ses0Found Fibre Channel device(s): Node WWN:50020f20000042d4 Device Type:Disk device

Logical Path:/dev/rdsk/c6t60020F20000042D43B275377000877DDd0s2 Node WWN:50020f20000042d4 Device Type:Disk device

Logical Path:/dev/rdsk/c6t60020F20000042D43B2753510008C9DFd0s2 Node WWN:50020f20000042d4 Device Type:Disk device

Logical Path:/dev/rdsk/c6t60020F20000042D43B0E926A000AA3FCd0s2 Node WWN:50020f20000042d4 Device Type:Disk device

Logical Path:/dev/rdsk/c6t60020F20000042D43ADCBC4E000C41E2d0s2


■ SECONDARY: This path is the next priority path after PRIMARY.

If a Sun StorEdge T3 partner pair configuration is used, two paths for each lun exist.One path is ONLINE and the other is STANDBY. In this configuration, IO to the lun isactive on its ONLINE path. If this path fails, the STANDBY path becomes the ONLINEpath. However, if the first path later becomes available, it will go into STANDBYmode instead of ONLINE, thus saving an expensive failover operation, if auto-failback was disabled throughout the configuration. Use luxadm failover commandto bring the restored path to ONLINE (see the luxadm man pages).

There can be more than one primary and secondary path to a Sun StorEdge T3, 39x0or 6x20 device in a SAN environment.

As can be seen from the following two luxadm display output of some T3 LUNs,PRIMARY can be STANDBY and SECONDARY can be ONLINE for one LUN on the samephysical paths, whereas SECONDARY is STANDBY and PRIMARY is ONLINE foranother LUN.

# luxadm display 50020f23000007a4DEVICE PROPERTIES for disk: 50020f23000007a4 Vendor: SUN Product ID: T300 Revision: 0301 Serial Num: Unsupported Unformatted capacity: 9217.688 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0x0 Device Type: Disk device Path(s):

/dev/rdsk/c14t60020F20000007D43EE74C4B0000C910d0s2 /devices/scsi_vhci/ssd@g60020f20000007d43ee74c4b0000c910:c,raw Controller /devices/pci@6,2000/SUNW,jfca@2,1/fp@0,0 Device Address 50020f23000007d4,0 Host controller port WWN 200000017380a66e Class primary State ONLINE Controller /devices/pci@7,2000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000007a4,0 Host controller port WWN 210000e08b0a9675 Class secondary State STANDBY


# luxadm display \/dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2DEVICE PROPERTIES for disk:/dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2 Vendor: SUN Product ID: T300 Revision: 0301 Serial Num: Unsupported Unformatted capacity: 35846.125 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0x0 Device Type: Disk device Path(s):

/dev/rdsk/c14t60020F20000007D44002E60F0002D8CAd0s2 /devices/scsi_vhci/ssd@g60020f20000007d44002e60f0002d8ca:c,raw Controller /devices/pci@6,2000/SUNW,jfca@2,1/fp@0,0 Device Address 50020f23000007d4,7 Host controller port WWN 200000017380a66e Class primary State ONLINE Controller /devices/pci@7,2000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000007a4,7 Host controller port WWN 210000e08b0a9675 Class secondary State STANDBY


In the case of Sun StorEdge A5200 in the current configuration, there are two paths.Both are ONLINE and IO is load balanced on both the paths. If a path fails, thesecond path continues the IO. If the failed path comes back, it will be in ONLINEstate and starts participating in IO transfer if load-balancing is enabled.

Utility: luxadm failover

The luxadm failover command is used to failover LUNs from primary tosecondary paths and vice versa.

In case of Sun StorEdge T3 partner pair, 6x20 HA, or 39x0 configuration, when afailover happens (for example, a cable is removed from one storage controller), theLUNs owned by that controller are failed over to the alternate controller.

# luxadm display /dev/rdsk/c6t20000020371A1862d0s2DEVICE PROPERTIES for disk: /dev/rdsk/c6t20000020371A1862d0s2 Status(Port A): O.K. Status(Port B): O.K. Vendor: SEAGATE Product ID: ST136403FSUN36G WWN(Node): 20000020371a1862 WWN(Port A): 21000020371a1862 WWN(Port B): 22000020371a1862 Revision: 114A Serial Num: LT0187150000 Unformatted capacity: 34732.891 MBytes Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0xffff Location: In the enclosure named: f Device Type: Disk device Path(s): /dev/rdsk/c6t20000020371A1862d0s2 /devices/scsi_vhci/ssd@g20000020371a1862:c,raw

Controller /devices/pci@1f,2000/pci@1/SUNW,qlc@4/fp@0,0 Device Address 22000020371a1862,0 Class primary

State ONLINEController /devices/pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0

Device Address 21000020371a1862,0 Class primary State ONLINE


After a failure of one path is corrected, the LUN will not fail back to the originalconfiguration automatically if auto-failback is set to "disable" in/kernel/drv/scsi_vhci.conf. A luxadm failover subcommand must be issuedto perform failover to the original configuration.

The failover process occurs as follows:

1. The original state of a Sun StorEdge T3 LUN is obtained by luxadm displaycommand.

Note – The primary path is ONLINE and the secondary path is STANDBY.

# luxadm display /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41Ed0s2DEVICE PROPERTIES for disk:/dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 Status(Port A): O.K. Status(Port B): O.K. Vendor: SUN Product ID: T300 WWN(Node): 50020f20000042d4 WWN(Port A): 50020f23000042d4 WWN(Port B): 50020f2300003fad Revision: 0117 Serial Num: Unsupported Unformatted capacity: 51220.500 MBytes Write Cache: Enabled Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0x0 Device Type: Disk device Path(s): /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 /devices/scsi_vhci/ssd@g60020f20000042d43adcbc4e000c41e2:c,raw Controller /devices/pci@1f,4000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000042d4,0 Class primary State ONLINE Controller /devices/pci@1f,4000/SUNW,qlc@4/fp@0,0 Device Address 50020f2300003fad,0 Class secondary State STANDBY


2. Cable is pulled from the T3 controller 50020f23000042d4. Failover is triggeredand the primary path is OFFLINE and secondary path is ONLINE. LUN status nowis degraded.

3. Cable is reinserted to T3 controller 50020f23000042d4. The device state becomeoptimal but, the failover is not triggered. The Primary path comes up as STANDBYand still the secondary path is ONLINE.

# luxadm display /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41Ed0s2DEVICE PROPERTIES for disk:/dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 .. Path(s): /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 /devices/scsi_vhci/ssd@g60020f20000042d43adcbc4e000c41e2:c,raw Controller /devices/pci@1f,4000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000042d4,0 Class primary State OFFLINE Controller /devices/pci@1f,4000/SUNW,qlc@4/fp@0,0 Device Address 50020f2300003fad,0 Class secondary State ONLINE

# luxadm display /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41Ed0s2DEVICE PROPERTIES for disk:/dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2.. Path(s): /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 /devices/scsi_vhci/ssd@g60020f20000042d43adcbc4e000c41e2:c,raw Controller /devices/pci@1f,4000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000042d4,0 Class primary State STANDBY Controller /devices/pci@1f,4000/SUNW,qlc@4/fp@0,0 Device Address 50020f2300003fad,0 Class secondary State ONLINE


4. Type the luxadm failover command to failover to the primary path.

This now triggers the failover. The primary path becomes ONLINE and the secondarypath becomes STANDBY which is equivalent to the original state in Step 1.

1. To verify the failover operation, display the properties using the luxadm displaycommand.

For more details on the luxadm failover command, see the man pages.

# luxadm failover primary/dev/rdsk/c5t60020F20000042D43ADCBCE000C41E2d0s2## luxadm display /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41Ed0s2DEVICE PROPERTIES for disk:/dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 .. Path(s): /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 /devices/scsi_vhci/ssd@g60020f20000042d43adcbc4e000c41e2:c,raw Controller /devices/pci@1f,4000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000042d4,0 Class primary State ONLINE Controller /devices/pci@1f,4000/SUNW,qlc@4/fp@0,0 Device Address 50020f2300003fad,0 Class secondary State STANDBY

# luxadm display /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41Ed0s2DEVICE PROPERTIES for disk:/dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 .. Path(s): /dev/rdsk/c5t60020F20000042D43ADCBC4E000C41E2d0s2 /devices/scsi_vhci/ssd@g60020f20000042d43adcbc4e000c41e2:c,raw Controller /devices/pci@1f,4000/SUNW,qlc@2/fp@0,0 Device Address 50020f23000042d4,0 Class primary State ONLINE Controller /devices/pci@1f,4000/SUNW,qlc@4/fp@0,0 Device Address 50020f2300003fad,0 Class secondary State STANDBY


APPENDIX B

Supported FC-HBA API

This Appendix contains the Supported FC-HBC API. For further informationregarding the API, refer to “Fibre Channel and Multipathing Features” on page 1under ”FC-HBA and FCSM Packages”.

TABLE 0-1 Supported and Unsupported FC-HBA Interfaces

SNIA API Software Support

HBA_GetVersion Yes

HBA_LoadLibrary Yes

HBA_FreeLibrary Yes

HBA_GetNumberofAdapters Yes

HBA_GetAdapterName Yes

HBA_OpenAdapter Yes

HBA_CloseAdapter Yes

HBA_GetAdapterAttributes Yes

HBA_GetAdapterPortAttributes Yes

HBA_GetDiscoveredPortAttributes Yes

HBA_GetPortAttributesbyWWN Yes

HBA_SendCTPassThru Yes

HBA_SendCTPassThruV2 Yes

HBA_RefreshInformation Yes

HBA_GetFcpTargetMapping Yes

HBA_SendScsiInquiry Yes

HBA_SendReportLuns Yes

HBA_SendReadCapacity Yes

79

HBA_GetPortStatistics Yes

HBA_ResetStatistics No

HBA_GetFcpPersistentBinding No

HBA_GetEventBuffer No

HBA_SetRNIDMgmtInfo Yes

HBA_GetRNIDMgmtInfo Yes

HBA_SendRNID Yes

HBA_SendRNIDV2 Yes

HBA_ScsiInquiryV2 Yes

HBA_ScsiReportLUNsV2 Yes

HBA_ScsiReadCapacityV2 Yes

HBA_OpenAdapterByWWN Yes

HBA_RefreshAdapterConfiguration Yes

HBA_GetVendorLibraryAttributes Yes

HBA_GetFcpTargetMappingV2 Yes

HBA_SendRPL No

HBA_SendRPS No

HBA_SendSRL No

HBA_SendLIRR No

HBA_SendRLS Yes

HBA_RemoveCallback Yes

HBA_RegisterForAdapterEvents Yes

HBA_RegisterForAdapterAddEvents Yes

HBA_RegisterForAdapterPortEvents Yes

HBA_RegisterForAdapterPortStatEvents No

HBA_RegisterForTargetEvents Yes

HBA_RegisterForAdapterLinkEvents No

HBA_RegisterForAdapterTargetEvents Yes

HBA_GetFC4Statistics No

HBA_GetFCPStatistics No

TABLE 0-1 Supported and Unsupported FC-HBA Interfaces (Continued)



HBA_GetBindingCapability No

HBA_GetBindingSupport No

HBA_SetBindingSupport No

HBA_SetPersistentBindingV2 No

HBA_GetPersistentBindingV2 No

HBA_RemovePersistentBinding No

HBA_RemoveAllPersistentBindings No

TABLE 0-1 Supported and Unsupported FC-HBA Interfaces (Continued)


Appendix B Supported FC-HBA API 81

APPENDIX C

Zones and Ports

Understanding zoning and port usage is fundamental to understanding the use ofconfiguration rules with the supported hardware. This section explains the use ofzones and ports in preparation of the next section, which covers the configurationrules. Topics covered include:

■ “Zone Types” on page 83■ “Port Types” on page 84

Zone TypesZoning is a function of the switch that allows segregation of devices by ports orworld wide names (WWNs).You can create zones for a variety of reasons, such assecurity, simplicity, performance, or dedication of resources. The Solaris 10 OSsoftware supports both industry-standard port-based and WWN-based zones. Seeyour third-party vendor documentation for more information.

There are two main types of zones:

■ Name Server Zones - NS zones use fabric protocols to communicate with FibreChannel devices. Each NS zone defines which ports or devices receive NSinformation. The Sun StorEdge T3 arrays with firmware level 1.18.02 or higherand Sun StorEdge T3+ arrays with firmware level 2.1.04 or higher support loop(TL) port connections.

The Sun StorEdge T3+ array with firmware level 2.1.04 or higher supports fabricconnections. FL ports are supported only for Sun StorEdge L180/L700 tapelibraries. Please refer to your switch documentation for more information.

■ Segmented Loop Zones - The Solaris 10 OS software does not support SegmentedLoop (SL) zones or ports.

83

Port TypesTABLE C-1 lists port types for Sun switches.

When an array is configured, the host port is connected to an F port and the array isconnected to an F or TL port on the switch. The TL (translation loop) port representseight-bit addressing devices as 24-bit addressing devices and vice versa.

TABLE C-1 Sun Switch Port Types

Port Type Description Supported Devices

TL Ports Translation loop Storage devices connected to the Sunswitch only.

FL Ports Public loop Sun StorEdge L180/L700 tape libraries.

F Ports Point-to-point fabric Host bus adapters, storage devices.

E Ports Inter-switch port Cascaded switches acting as ISLs, whichare configured initially in fabric portmode.

G Ports General ports Automatically configured to E or F portsto support switches or fabric devices. Allswitch ports should be set to G-port,except for tape libraries that do notsupport F-port; see GL ports below.

Gx Ports Public loop or generalports

Automatically configured to FL or Gports to support hosts or switches (SunStorEdge Network 2 Gbit McDATASphereon 4300 and 4500 switches).

GL Ports General loop ports Automatically configured to FL, E, or Fports to support public loop,point-to-point, or switch devices. Thisport type is used only for settingL180/L700 tape libraries to FL.

TABLE 0-2 Port Types for Storage Devices

Port Type Storage Device

loop (TL port - SunSwitches only)

Sun StorEdge T3 array

loop or fabric (Fport)

Sun StorEdge T3+ array

Sun StorEdge 39x0 array


Although you can connect a Sun StorEdge T3 array with a TL port, the host busadapter recognizes it as a fabric device. Sun StorEdge T3+ arrays and the SunStorEdge 3510FC, 39x0, 69x0, and 99x0 storage arrays should be connected with Fports as a 24-bit addressing device for fabric connectivity. The STK 9840B tape driverequire F ports when connected to 2-Gbit switches. Sun StorEdge L5500/L6000libraries are not connected to the SAN.

fabric Sun StorEdge 69x0 arraySun StorEdge 6x20 array

loop or fabric Sun StorEdge 99x0 array

loop STK 9840b tape drive

loop or fabric STK 9940b tape drive

loop STK 9840 tape drive

loop, public loop (FLport)

Sun StorEdge L180/L700 tape libraries

TABLE 0-2 Port Types for Storage Devices

Port Type Storage Device

Appendix C Zones and Ports 85

APPENDIX D

Implementing Sun StorEdge SANSoftware Dynamic Reconfiguration

When you want to modify your system configuration with Dynamic Reconfiguration(DR), you must change the device configuration for SAN-based devices prior toimplementing DR.

Dynamic ReconfigurationDynamic reconfiguration (DR) works differently for non-fabric and fabric devices.With previously configured FC-AL devices, DR happens automatically uponaddition or removal of devices to a host I/O port. With the multipathing enabled,the Solaris Operating System host configures the devices as multipathing devices.

Dynamic Reconfiguration and Fabric DevicesWhen configuring a fabric device, you will need to modify the devices beforeimplementing DR.

Previously configured fabric devices are not automatically reconfigured when DR isimplemented. When you remove a system component on which switch-connectedhost ports reside, and then add that system component back to a host through DRoperations, the fabric device configurations are not persistent. The discussion of on-demand node creation in these sections applies to fabric devices, such as a host portconnected to an F port on a switch and an array connected to an F port or TL port ona switch.

87

▼ To Remove a Fabric Device Before DynamicReconfiguration

● Unconfigure the fabric devices that were configured through host ports on thesystem component with on-demand node creation.

■ If multipathing is not enabled, see “Unconfiguring Fabric Devices” on page 61”.■ If the multipathing is enabled, see “To Unconfigure a Fabric Device Associated

With Multipathing Enabled Devices” on page 64.”

▼ To Maintain a Fabric Device Configuration WithDynamic Reconfiguration

1. Reconfigure the device through on-demand node creation.

2. Perform DR operations according to the instructions in the documentation for thehost.

▼ To Reconfigure Fabric Devices With DynamicReconfiguration

1. Add the system component and make it available to the host.

2. Reconfigure the device(s) through on-demand node creation.

■ If the multipathing is not enabled, see “Configuring Device Nodes WithoutMultipathing Enabled” on page 33.

■ If the multipathing is enabled, see “Configuring Device Nodes With MultipathingEnabled” on page 40.

▼ To Reconfigure the Sun Enterprise 10000 ServerWith a Fabric ConnectionThe following procedure gives the sequence of operations for a Sun Enterprise 10000server board with a fabric connection.

1. Unconfigure the fabric devices on fabric-connected host ports on the board to bedetached.


2. Start the DR detach operations for the board.

See the Sun Enterprise 10000 Dynamic Reconfiguration Configuration Guide.

3. Start the DR attach operations when the board is ready.

See the Sun Enterprise 10000 Dynamic Reconfiguration Configuration Guide.

4. Configure any fabric devices on the attached boards.

See the sections in Chapter 3 that explain how to recognize the storage devices onthe host. On the newly attached board, the devices could be the same or completelynew devices.

Appendix D Implementing Sun StorEdge SAN Software Dynamic Reconfiguration 89

APPENDIX E

Multipathing Troubleshooting

This appendix provides solutions to potential problems that may occur whilerunning multipathing.

This appendix contains the following sections:

■ “System Crashes During Boot Operations” on page 92

■ “System Crashes During or After a Boot Enable Operation” on page 92

■ “System Crashes During or After a Multipathing Boot Disable Operation” onpage 94

■ “Sun StorEdge T3, 6x20, or 3900 Arrays Do Not Show” on page 96

■ “System Failed During Boot With scsi_vhci Attachment” on page 97

■ “Connected Sun StorEdge A5200 Arrays Appear Under Physical Paths inFormat” on page 97

■ “System and Error Messages” on page 98

91

System Crashes During Boot OperationsIf your system fails during the boot enable(stmsboot -e) or disable (stmsboot -d) operation, your original/etc/path_to_inst, /etc/vfstab, /etc/system, and /etc/dumpadm.conf filesare saved with .sav file name extensions in the /var/tmp directory.

Other important debug information is also saved in this directory.

System Crashes During or After a BootEnable OperationIf your system crashes or panics during or after a boot enable (stmsboot -e)operation with messages similar to the following:

Then check the following:

1. If the boot device is a Sun StorEdge T3 Raid Array LUN, did you change themp_support setting to “mpxio” mode? If the change was not performed beforesystem reboot, make this change and shut down and restart the system.

2. Are you specifying a different boot path manually? stmsboot has a dependencyon the path through which the system boots. See Chapter 4 “ConfiguringMultipathing Support for Storage Boot Devices”.

3. Did you have a per-port mpxio-disable entry corresponding to the bootcontroller port in the qlc.conf file? If so, remove the corresponding qlc.confentry.

4. Boot the system from another disk or over the network, and mount the bootdevice on /mnt.

Cannot assemble drivers for root/scsi_vhci/ssd@g20000004cf721119:aCannot mount root on /scsi_vhci/ssd@g20000004cf721119:a fstype ufs

or

cannot stat /dev/rdsk/...


■ Verify that /mnt/etc/vfstab is the same as /var/tmp/vfstab.sav. Are allthe boot controller device names shown as multipathing device names,compared with /var/tmp/vfstab.sav?

■ Verify that /mnt/etc/path_to_inst is the same as/var/tmp/path_to_inst.sav. This file should contain a unique ssd instancefor each scsi_vhci/ssd@gGUID device.

■ The /mnt/etc/system file should have a rootdev entry to the boot device ofscsi_vhci/ssd@gGUID. For example:

■ The /mnt/kernel/drv/fp.conf file should have the boot controller portentry set to mpxio-disable="no".

5. To restore the system to the "stmsboot disabled" state, use the followingcommands:

Note that path_to_inst.sav is not restored because the system remembers thedevice names. Remove the per-port mpxio-disable entry corresponding to theboot controller port in the /mnt/kernel/drv/fp.conf file, and reboot yoursystem.

If these steps do not resolve the problem, contact Sun Services.

rootdev:/scsi_vhci/ssd@g20000004cf721119:a

# cp /mnt/var/tmp/vfstab.sav /mnt/etc/vfstab

# cp /mnt/var/tmp/system.sav /mnt/etc/system

# cp /mnt/var/tmp/dumpadm /mnt/etc/dumpadm.conf


Appendix E Multipathing Troubleshooting 93

System Crashes During or After aMultipathing Boot Disable OperationIf your system crashes or panics during or after multipathing boot disable(stmsboot -d) operation, check the following:

If after multipathing boot disable (stmsboot -d) operation, your system panics withmessages similar to the following:

Then check the following:

1. If the boot device is a Sun StorEdge T3 Raid Array LUN, did you change themp_support setting to “rw” or “none”? If the change was not performed beforesystem reboot, make this change and shut down and restart the system.

2. Are you specifying a different boot path manually? stmsboot has a dependencyon the path through which the system boots. See Chapter 4 “ConfiguringMultipathing Support for Storage Boot Devices”.

3. Did you have a per-port mpxio-disable entry corresponding to the bootcontroller port in the qlc.conf file? If so, remove the corresponding qlc.confentry.

4. Boot the system from another disk or over the network and mount the bootdevice on /mnt.

■ The /mnt/etc/vfstab file should indicate the device name that is notmultipathing enabled instead of multipathing device names.

■ The /mnt/etc/system file should not have a rootdev entry.

■ The /mnt/kernel/drv/scsi_vhci.conf file should not have the bootcontroller port entry.

Cannot assemble drivers for root /dev/rdsk/....Cannot mount root on fstype /dev/rdsk/.... ufsor

cannot stat /dev/rdsk/...


5. To restore the system to the “stmsboot enabled” state, use the followingcommands:

Note that path_to_inst.sav is not restored because system remembers thedevice names. Add the per-port mpxio-disable=”no” entry corresponding to theboot controller port in the /mnt/kernel/drv/fp.conf file and reboot yoursystem:

If these steps do not resolve the problem, contact Sun Services.

Multipathing Is Not Running ProperlyThe first item to check is if multipathing has been installed correctly.

Verify that the multipathing drivers (that is, scsi_vhci and mpxio), are loaded withthe help of the modinfo command. (See the modinfo man pages.)

If modinfo output does not show that the drivers are loaded, verify that thefollowing binaries have been loaded in the proper directories:

■ /kernel/drv/scsi_vhci, /kernel/drv/v9/scsi_vhci (64 bit)

■ /kernel/misc/mpxio, /kernel/misc/sparcv9/mpxio (64-bit)

■ /kernel/drv/scsi_vhci.conf

■ /kernel/drv/fp.conf

# cp /mnt/var/tmp/vfstab.sav /mnt/etc/vfstab

# cp /mnt/var/tmp/system.sav /mnt/etc/system

# cp /mnt/var/tmp/dumpadm /mnt/etc/dumpadm.conf


# modinfo | grep mpxio23 102193e5 84cb - 1 mpxio (MDI Library)# modinfo | grep scsi_vhci121 781ea000 6a20 225 1 scsi_vhci (Sun Multiplexed SCSI vHCI)


If these binaries are not present in the specified directories, then multipathingsoftware was not installed properly. Repeat the installation process, making sure youare logged in as superuser.

luxadm display and luxadmfailover Commands FailIf the multipathing software is running properly, but the luxadm display andluxadm failover commands aren’t working, use the luxadm fcode -p command toensure that the system sees all your pHCIs. If you don’t see one or more of thepHCIs, their Fcode patch is out of date.

An alternative way to confirm this is by examining the device path. Notice the“scsi@4” in the example below. Make sure that none of the device paths looks likethe following:

/devices/pci@9d,600000/pci@1/scsi@4/fp@0,0:devctl

To correct this problem, download the latest Fcode software.

Sun StorEdge T3, 6x20, or 3900 ArraysDo Not ShowA Sun StorEdge T3, 6x20, or 3900 array can be correctly configured for multipathingwith the latest firmware revision. Be sure to set the master settings as follows:

While checking the array setup, confirm that the LPC version is also current. Afterthe device is configured, it is advisable to perform a reconfiguration reboot orequivalent.

hostname:/:<1> sys mp_support mpxio


System Failed During Boot Withscsi_vhci AttachmentIt is most likely due to an incomplete installation. The installation failed to providean entry for the scsi_vhci in the name_to_major database or duplicate entries for thesame number with different driver name references. If you do not see anyinstallation log errors, call Sun Support.

Connected Sun StorEdge A5200 ArraysAppear Under Physical Paths in FormatConsider the following:

■ The Sun StorEdge A5200 Array is not fabric supported, but can be connectedusing FCAL. Solaris 10 OS software supports the Sun StorEdge A5200, A3500FCarrays and FC tape devices. SL zones contain SL ports only. SL ports are notsupported in the Solaris 10 OS 4.x release but were in earlier releases.

■ Check whether the Sun StorEdge A5200 class devices are not connected to amultipathing supported HBA.

■ Check whether multipathing is disabled under /kernel/drv/fp.conf.

■ Check whether the system is booting from the Sun StorEdge A5200 disks. Whenyou boot from a multipathing device, all devices under the pHCI with the bootdevice are enumerated under scsi_vhci.

■ Make sure the pHCI has mpxio-disable=”no” set in the multipathing file/kernel/drv/qlc.conf or /kernel/drv/fp.conf.


System and Error MessagesThe following messages might appear in the course of operation.

■ When the automatic failback feature is enabled through the configuration file, youshould receive the following message:

■ If automatic failback succeeds, the following message is logged:

■ If automatic failback fails, the following message is logged:

■ Externally initiated failover of a Sun StorEdge T3 or T3+ array has been observed.

■ After an stmsboot enable (stmsboot -e) operation, the following messagecorresponding to the boot controller port (here, fp2) should not appear once thesystem is booted, that is, booted through the stmsboot-enabled device throughthe fp2 path):

Auto-failback capability enabled through scsi_vhci.conf file.

Auto failback operation succeeded for devices.

Auto failback operation failed for device.

Waiting for externally initiated failover to complete

mpxio: [ID 284422 kern.info] /pci@8,600000/SUNW,qlc@1/fp@0,0 (fp2)multipath capabilities disabled: controller for root filesystemcannot be multipathed.


Index

Numerics24-bit addressing devices, 8424-bit Fibre Channel addressing devices, 31

AAp_Id, 66array port, 84

Bbroadcasting, 54

Ccfgadm

-al, 32-c configure, 31-fp, 40-l, 32

cfgadm -al, 39, 66cfgadm -al -o show_FCP_dev, 40cfgadm -c configure, 39cfgadm(1M), 33configuration

examples, 70examples, luxadm command, 72, 75, 76fabric devices, 33

configuringHBAs, 13, 14third-party devices, 16

Ddevice

activity, 64configuration, 33node creation, 31node discovery, 38removal, 64storage, 31

device node creation, 40device node removal, 40disabling ports, 10drivers, 31

Eeight-bit addressing devices, 84enabling ports, 10error messages, 98

FF_port, 84fabric connectivity, 85fabric-connected host ports, 32fc-fabric, 41Fibre Channel

Protocol, 31

Hhost port, 84

IIPFC

guidelines, 53SCSI LUN level information, 31

99

Lloading drivers, 31LUN

level information, 31recognition, 31

luxadm, 35

Mmodinfo, 31multicasting, 54

NNAS, 53NFS, 53

Pphysical device representation, 42port, 84promiscuous mode, 54

Sshow_scsi_lun, 40SNDR, 53snoop, 54ssd driver, 31st driver, 31Sun StorEdge 39x0 series, 85Sun StorEdge 69x0 series, 85Sun StorEdge 99x0 series, 85support contact, xii

TTL

port, 84translation loop

port, 84

Uunconfiguring

a single path, 66devices, 61multipathed devices, 65multiple devices, 67

unconfiguring devices, 64

Zzoning, 83


multipath config

Documents