ldom oracle vm for sparc

LDOM Oracle VM for SPARC – Tutorial 1 February 18, 2014 in LDOM

SUN LDOMS aka oracle VM for SPARC is a complete virtual machines that runs an independent operating systems and contains its own virtual CPU’s,Storage,Memory, Cryptographic and console.Unlike Solaris zones,LDOMS can run its own operating system with different kernel patch levels. You can also install Solaris 8,Solaris 9 ,Solaris 10 and Solaris 11 on LDOMS. LDOM also provides the OBP for each logical domains but we won’t get that in zones. LDOM also allow to move the resources across the logical domains or virtual hosts exclusively. LDOM also allows you to configure zones under that. Tutorials Topics: Installation and Configuring the Primary domain (You are here) Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services

LDOM Quick Reference Prerequisite: Oracle/Sun T-series Server (T5140 Firmware:7.3.3 )

Solaris 10 or Solaris 11 (To be directly installed on T-series server)

LDOM Packages (Ex: 2.1)

Server Hardware Console Access

Contents: Basic Concepts of LDOM

Installation of LDOM Software

Configuring the Service Domains

Configuring the Control Domains

Saving the SP-Profile

Basic Concepts of LDOM or Oracle VM for SPARC: LDOM is a hardware based virtualization which works on top the hypervisor. To understand better, you should know the type of terms used in LDOM or oracle VM for SPARC. Control Domain is the place where you are going to install the LDOM software and managing the complete logical

domain environment.It used to configure the resources and guest domains

Service domain provides the various virtual services to guest domains.Virtaul services can be virtual disks,network switches or virtual consoles.Mostly we will be configuring the control domain as service domain too.

I/O Domains have a direct ownership of PCI Bus.You need to directly map the I/O devices to guest domain to improve the application performance in some cases. There is an another domain called “root domain” and this domain has direct ownership of PCI devcies. So it also called I/O domain.

Guest domains doesn’t performing any of the above mentioned roles.It jsut runs the operating system instances.

Control domain and service domain can be run together.But the user application should not be installed on this domain in a order to protect domain stability and performance.

http://www.unixarena.com/category/ldom

http://www.unixarena.com/2014/02/ldom-oracle-vm-sparc-tutorial-1.html

http://www.unixarena.com/2014/02/configure-guest-domains-ldom.html

http://www.unixarena.com/2014/02/configure-io-domain-oracle-vm-sparc.html

http://www.unixarena.com/2014/02/dynamically-add-resources-oracle-vm-sparc.html

http://www.unixarena.com/2014/02/migrate-guest-domain-ldom-oracle-vm.html

http://www.unixarena.com/2014/02/add-iso-image-multiple-ldoms.html

http://www.unixarena.com/2014/02/oracle-vm-sparc-ldom-configuration-backup-restore.html

http://www.unixarena.com/2014/02/remove-devices-services-ldom-oracle-vm-sparc.html

https://docs.google.com/file/d/0B6K_eEQKbx5ueWdRS3RFbGZDM0U/edit

LDOM Basic concept

Installation of LDOM software: Download the LDOM packages from the oracle support website. For your information ,you can’t download without having the oracle support login credentials. Once you have downloaded the packages ,just copy the packages to the Solaris host which you have already installed on T- series server. 1.The below mentioned packages needs to be installed .

-rwxr-xr-x 1 root root 3564851 Jan 16 23:26 OracleVM-Server-for-SPARC-2.0.0_(64-bit).zip

drwxrwxrwx 5 root root 512 Jan 16 23:26 SUNWldomr.v

drwxrwxrwx 5 root root 512 Jan 16 23:26 SUNWldomu.v

2.Install the SUNWldomr and SUNWldomu packages first.

-bash-3.2# pkgadd -d . SUNWldom*

Processing package instance from </>

Solaris Logical Domains (Root)(sparc.sun4v) 11.10.0,REV=2006.10.04.00.26

Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.

This appears to be an attempt to install the same architecture and

version of a package which is already installed. This installation

http://www.unixarena.com/wp-content/uploads/2014/02/LDOM-Basic-concept.jpg

will attempt to overwrite this package.

Using </> as the package base directory.

## Processing package information.

## Processing system information.

42 package pathnames are already properly installed.

## Verifying package dependencies.

## Verifying disk space requirements.

## Checking for conflicts with packages already installed.

## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-user

permission during the process of installing this package.

Do you want to continue with the installation of [y,n,?] y

Installing Solaris Logical Domains (Root) as

## Executing preinstall script.

## Installing part 1 of 1.

[ verifying class ]

## Executing postinstall script.

Installation of was successful.

Processing package instance from </>

Solaris Logical Domains (Usr)(sparc.sun4v) 11.10.0,REV=2006.08.08.12.13


This appears to be an attempt to install the same architecture and

version of a package which is already installed. This installation

will attempt to overwrite this package.

Using </> as the package base directory.


## Processing system information.

11 package pathnames are already properly installed.

## Verifying package dependencies.

## Verifying disk space requirements.

## Checking for conflicts with packages already installed.

## Checking for setuid/setgid programs.


permission during the process of installing this package.

Do you want to continue with the installation of [y,n,?] y

Installing Solaris Logical Domains (Usr) as

## Installing part 1 of 1.


-bash-3.2#

3.Install the LDOM software . Unzip the package and continue the installation like below

-bash-3.2# cd OVM_Server_SPARC-2_0

-bash-3.2# ls -lrt

total 12

-r--r--r-- 1 root root 3368 Sep 17 2010 README

drwxr-xr-x 2 root root 512 Sep 18 2010 Install

drwxr-xr-x 7 root root 512 Dec 16 2010 Product

-bash-3.2# cd Install/

-bash-3.2# ls -lrt

total 56

-rwxr-xr-x 1 root root 28471 Sep 18 2010 install-ldm

-bash-3.2# ./install-ldm

Welcome to the Oracle VM Server for SPARC installer.

You are about to install the LDoms Manager package, SUNWldm, that will enable

you to create, destroy and control other domains on your system. You will

also be given the option of running the Oracle VM Server for SPARC Configuration

Assistant (ldmconfig) to setup the control domain and create guest domains.

If the Solaris Security Toolkit (SST) is installed, you will be prompted to

optionally harden your control domain.

Installing LDoms packages.

pkgadd -n -d "/OVM_Server_SPARC-2_0/Product" -a pkg_admin SUNWldm.v


Feb 17 10:37:54 sol10-10 sendmail[2631]: [ID 702911 mail.alert] unable to qualify my own domain name (sol10-10)

-- using short name


pkgadd -n -d "/OVM_Server_SPARC-2_0/Product" -a pkg_admin SUNWldmp2v


Feb 17 10:38:54 sol10-10 sendmail[2705]: [ID 702911 mail.alert] unable to qualify my own domain name (sol10-10)

-- using short name


Oracle VM Server for SPARC Configuration

Once installed, you may configure your system for a basic LDoms

deployment. If you select "y" for the following question, the Oracle

VM Server for SPARC Configuration Assistant (tty) will be launched

following a successful installation of the packages.

(You may launch the Configuration Assistant at a later time with the

command: /usr/sbin/ldmconfig, or use the GUI Configuration Assistant

which is bundled in the Oracle VM Server for SPARC zip file - see

README.GUI for more details)

Select an option for configuration:

y) Yes, launch the Configuration Assistant after install

n) No thanks, I will configure the system manually later

Enter y or n [y]: n

Verifying that all packages are fully installed. OK.

You have new mail in /var/mail//root

-bash-3.2#

Configuring the Service Domains: Login to the Solaris host where you have installed the LDOM or oracle VM software. 1.List the configured services.By default it will be nothing.

-bash-3.2# ldm list-services

2.Create a new virtual disk service.This service is responsible to provide the virtual disks to guest domains.

-bash-3.2# ldm add-vdiskserver primary-vds0 primary


VDS

NAME LDOM VOLUME OPTIONS MPGROUP DEVICE

primary-vds0 primary

-bash-3.2#

3.Create a new virtual switch. By creating this , we can create N-number of VNIC and share to the guest domains.

-bash-3.2# dladm show-dev

nxge0 link: up speed: 1000 Mbps duplex: full




-bash-3.2# ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

inet 127.0.0.1 netmask ff000000

nxge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

inet 192.168.2.11 netmask ff000000 broadcast 192.168.2.255

ether 0:21:98:57:cd:16

-bash-3.2# ldm add-vswitch net-dev=nxge1 primary-vsw0 primary

-bash-3.2#


VSW

NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID

PVID VID MTU MODE

primary-vsw0 primary 00:14:4f:f9:05:b1 nxge1 0 switch@0 1 1

1500

VDS



-bash-3.2#

Here i have configured virtual switch using “nxge1″ . You can also use “nxge0″ configure the virtual swtich but you need to remove the IP from nxge0 and need to configure the IP over virtual switch.

-bash-3.2# ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1


vsw0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

inet 198.168.2.11 netmask ff000000 broadcast 192.168.2.255

ether 0:14:4f:98:83:f3

-bash-3.2#

4.Create a virtual service for terminal consoles.

-bash-3.2# ldm add-vconscon port-range=5000-5100 primary-vc0 primary


VCC

NAME LDOM PORT-RANGE

primary-vc0 primary 5000-5100

VDS



-bash-3.2#

VSW


PVID VID MTU MODE

primary-vsw0 primary 00:14:4f:f9:05:b1 nxge1 0 switch@0 1 1

1500

Now you have successfully configured the default services or service domain. (Creating the spconfig is must after making any changes on the service.Otherwise you will loose the changes after the system power cycle.) Configuring the control Domains: You need to login to Solaris host where you have installed the ldom software to configure the control domain. 1.List the logical domain devices using ldm list-device -a command.You can see all the resource has been allocated to the primary domain.

bash-3.2# ldm list-devices -a

CORE

ID %FREE CPUSET

0 0 (0, 1, 2, 3, 4, 5, 6, 7)

1 0 (8, 9, 10, 11, 12, 13, 14, 15)

2 0 (16, 17, 18, 19, 20, 21, 22, 23)

3 0 (24, 25, 26, 27, 28, 29, 30, 31)

4 0 (64, 65, 66, 67, 68, 69, 70, 71)

5 0 (72, 73, 74, 75, 76, 77, 78, 79)

6 0 (80, 81, 82, 83, 84, 85, 86, 87)

7 0 (88, 89, 90, 91, 92, 93, 94, 95)

VCPU

PID %FREE PM

0 0 no

1 0 no

2 0 no

3 0 no

4 0 no

5 0 no

6 0 no

7 0 no

8 0 no

9 0 no

10 0 no

11 0 no

12 0 no

13 0 no

14 0 no

15 0 no

16 0 no

17 0 no

18 0 no

19 0 no

20 0 no

21 0 no

22 0 no

23 0 no

24 0 no

25 0 no

26 0 no

27 0 no

28 0 no

29 0 no

30 0 no

31 0 no

64 0 no

65 0 no

66 0 no

67 0 no

68 0 no

69 0 no

70 0 no

71 0 no

72 0 no

73 0 no

74 0 no

75 0 no

76 0 no

77 0 no

78 0 no

79 0 no

80 0 no

81 0 no

82 0 no

83 0 no

84 0 no

85 0 no

86 0 no

87 0 no

88 0 no

89 0 no

90 0 no

91 0 no

92 0 no

93 0 no

94 0 no

95 0 no

MAU

ID CPUSET BOUND

0 (0, 1, 2, 3, 4, 5, 6, 7) primary

1 (8, 9, 10, 11, 12, 13, 14, 15) primary

2 (16, 17, 18, 19, 20, 21, 22, 23) primary

3 (24, 25, 26, 27, 28, 29, 30, 31) primary

4 (64, 65, 66, 67, 68, 69, 70, 71) primary

5 (72, 73, 74, 75, 76, 77, 78, 79) primary

6 (80, 81, 82, 83, 84, 85, 86, 87) primary

7 (88, 89, 90, 91, 92, 93, 94, 95) primary

MEMORY

PA SIZE BOUND

0x0 512K _sys_

0x80000 1536K _sys_

0x200000 94M _sys_

0x6000000 32M _sys_

0x8000000 96M _sys_

0xe000000 32544M primary

IO

DEVICE PSEUDONYM BOUND OPTIONS

pci@400 pci_0 yes

pci@500 pci_1 yes

bash-3.2#

2.List what are the resources assigned to the primary domain or control domains .

bash-3.2# ldm list-bindings primary

NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME

primary active -n-c-- SP 64 32544M 0.1% 9m

UUID

f063b0a8-8fb0-4e61-cd5b-91826d2c1550

MAC

00:21:28:57:cd:16

HOSTID

0x8557cd16

CONTROL

failure-policy=ignore

DEPENDENCY

master=

CORE

CID CPUSET

0 (0, 1, 2, 3, 4, 5, 6, 7)

1 (8, 9, 10, 11, 12, 13, 14, 15)

2 (16, 17, 18, 19, 20, 21, 22, 23)

3 (24, 25, 26, 27, 28, 29, 30, 31)

4 (64, 65, 66, 67, 68, 69, 70, 71)

5 (72, 73, 74, 75, 76, 77, 78, 79)

6 (80, 81, 82, 83, 84, 85, 86, 87)

7 (88, 89, 90, 91, 92, 93, 94, 95)

VCPU

VID PID CID UTIL STRAND

0 0 0 1.5% 100%

1 1 0 0.0% 100%

2 2 0 0.0% 100%

3 3 0 0.0% 100%

4 4 0 0.0% 100%

5 5 0 0.0% 100%

6 6 0 0.0% 100%

7 7 0 0.0% 100%

8 8 1 0.0% 100%

9 9 1 0.1% 100%

10 10 1 0.0% 100%

11 11 1 0.1% 100%

12 12 1 0.1% 100%

13 13 1 3.0% 100%

14 14 1 0.0% 100%

15 15 1 0.1% 100%

16 16 2 0.1% 100%

17 17 2 0.2% 100%

18 18 2 0.1% 100%

19 19 2 0.0% 100%

20 20 2 0.1% 100%

21 21 2 0.0% 100%

22 22 2 0.1% 100%

23 23 2 0.0% 100%

24 24 3 0.0% 100%

25 25 3 0.0% 100%

26 26 3 0.0% 100%

27 27 3 0.0% 100%

28 28 3 0.2% 100%

29 29 3 0.1% 100%

30 30 3 0.1% 100%

31 31 3 0.0% 100%

64 64 4 0.0% 100%

65 65 4 0.0% 100%

66 66 4 0.0% 100%

67 67 4 0.0% 100%

68 68 4 0.0% 100%

69 69 4 0.0% 100%

70 70 4 0.0% 100%

71 71 4 0.0% 100%

72 72 5 0.0% 100%

73 73 5 0.0% 100%

74 74 5 0.0% 100%

75 75 5 0.0% 100%

76 76 5 0.0% 100%

77 77 5 0.0% 100%

78 78 5 0.0% 100%

79 79 5 0.0% 100%

80 80 6 0.0% 100%

81 81 6 0.0% 100%

82 82 6 0.0% 100%

83 83 6 0.0% 100%

84 84 6 0.0% 100%

85 85 6 0.0% 100%

86 86 6 0.0% 100%

87 87 6 0.0% 100%

88 88 7 0.0% 100%

89 89 7 0.0% 100%

90 90 7 0.0% 100%

91 91 7 0.0% 100%

92 92 7 0.0% 100%

93 93 7 0.0% 100%

94 94 7 0.0% 100%

95 95 7 0.0% 100%

MAU

ID CPUSET

0 (0, 1, 2, 3, 4, 5, 6, 7)

1 (8, 9, 10, 11, 12, 13, 14, 15)

2 (16, 17, 18, 19, 20, 21, 22, 23)

3 (24, 25, 26, 27, 28, 29, 30, 31)

4 (64, 65, 66, 67, 68, 69, 70, 71)

5 (72, 73, 74, 75, 76, 77, 78, 79)

6 (80, 81, 82, 83, 84, 85, 86, 87)

7 (88, 89, 90, 91, 92, 93, 94, 95)

MEMORY

RA PA SIZE

0xe000000 0xe000000 32544M

IO

DEVICE PSEUDONYM OPTIONS

pci@400 pci_0

pci@500 pci_1

VCONS

NAME SERVICE PORT

SP

bash-3.2#

3.Set the resource control for the control domain aka primary domain.So the resource will be freed up for the logical domains / Guest domains.

bash-3.2# ldm set-mau 1 primary

bash-3.2# ldm set-vcpu 8 primary

bash-3.2# ldm set-memory 4G primary

A ldm set-mem 4G command would remove 28448MB, which is not a

multiple of 256MB. Instead, run ldm set-mem 4128M primary

to ensure a 256MB alignment.

bash-3.2# ldm set-memory 3872M primary

mau – Security Encryption (LDM commands will be sent in encrypted manner.)

vpcu – Virtual CPU cores

memory – Physical memory

4.Now list the devices again to verify the new settings.

bash-3.2# ldm ls-devices -a

CORE

ID %FREE CPUSET

0 0 (0, 1, 2, 3, 4, 5, 6, 7)

1 100 (8, 9, 10, 11, 12, 13, 14, 15) ----------->(1-7) Free CPU cores

2 100 (16, 17, 18, 19, 20, 21, 22, 23)

3 100 (24, 25, 26, 27, 28, 29, 30, 31)

4 100 (64, 65, 66, 67, 68, 69, 70, 71)

5 100 (72, 73, 74, 75, 76, 77, 78, 79)

6 100 (80, 81, 82, 83, 84, 85, 86, 87)

7 100 (88, 89, 90, 91, 92, 93, 94, 95)

VCPU

PID %FREE PM

0 0 no ----------These (0-7) CPU is not available since its assigned to primary domain

1 0 no

2 0 no

3 0 no

4 0 no

5 0 no

6 0 no

7 0 no

8 100 ---

9 100 ---

10 100 ---

11 100 ---

12 100 ---

13 100 ---

14 100 ---

15 100 ---

16 100 ---

17 100 ---

18 100 ---

19 100 ---

20 100 ---

21 100 ---

22 100 ---

23 100 ---

24 100 ---

25 100 ---

26 100 ---

27 100 ---

28 100 ---

29 100 ---

30 100 ---

31 100 ---

64 100 ---

65 100 ---

66 100 ---

67 100 ---

68 100 ---

69 100 ---

70 100 ---

71 100 ---

72 100 ---

73 100 ---

74 100 ---

75 100 ---

76 100 ---

77 100 ---

78 100 ---

79 100 ---

80 100 ---

81 100 ---

82 100 ---

83 100 ---

84 100 ---

85 100 ---

86 100 ---

87 100 ---

88 100 ---

89 100 ---

90 100 ---

91 100 ---

92 100 ---

93 100 ---

94 100 ---

95 100 ---

MAU

ID CPUSET BOUND

0 (0, 1, 2, 3, 4, 5, 6, 7) primary

1 (8, 9, 10, 11, 12, 13, 14, 15)

2 (16, 17, 18, 19, 20, 21, 22, 23)

3 (24, 25, 26, 27, 28, 29, 30, 31)

4 (64, 65, 66, 67, 68, 69, 70, 71)

5 (72, 73, 74, 75, 76, 77, 78, 79)

6 (80, 81, 82, 83, 84, 85, 86, 87)

7 (88, 89, 90, 91, 92, 93, 94, 95)

MEMORY

PA SIZE BOUND

0x0 512K _sys_

0x80000 1536K _sys_

0x200000 94M _sys_

0x6000000 32M _sys_

0x8000000 96M _sys_


0x10000000 28G -----------------------------Free Memory

0x710000000 3840M primary -----------Allocated to Primary or control Domain

IO


pci@400 pci_0 yes

pci@500 pci_1 yes

bash-3.2#

You have successfully configured the control domain. Note:Here you haven’t saved the configuration.These are all in-memory configuration.After the power cycle, you will loose the configuration if its not saved in sp config profile. Saving the sp-config file:

You have to save the configuration after every ldom settings that you made. For an example, if you didn’t save, ldom configuration will be flushed after server power cycle. 1.Create a new profile the current configuration.

bash-3.2# ldm add-spconfig unixarena

bash-3.2# ldm ls-spconfig

factory-default

unixarena [current]

bash-3.2#

In the above command output, you can see that currents settings are saved in unixarena and this settings will come up after the system reboot and power cycle. For an example , if i do any small changes on ldom ,it will not update in the spconfig file.

bash-3.2# ldm set-vcpu 12 primary


factory-default

unixarena [next poweron]

bash-3.2#

Here we just increased the number vpcus to 12 . The previous number of vpcu was 8. Here sp-config file “unixarena” will be activated on next power cycle. Which mean you will loose the cpu setting you just done. In a order to save the configuration, just re-create the sp-config like below.


factory-default

unixarena [next poweron]

bash-3.2# ldm remove-spconfig unixarena


factory-default [next poweron]

bash-3.2# ldm add-spconfig unixarena_new


factory-default

unixarena_new [current]

bash-3.2#

Checking the OBP,Hypervisor and LDOM version:

-bash-3.2# ldm -V

Logical Domain Manager (v 2.0)

Hypervisor control protocol v 1.4

Using Hypervisor MD v 1.1

System PROM:

Hypervisor v. 1.7.4. @(#)Hypervisor 1.7.4.a 2009/09/21 08:25\015

OpenBoot v. 4.30.4 @(#)OBP 4.30.4 2009/08/19 07:25

-bash-3.2#

Hope this article is first step to learn LDOM for Solaris beginners.In this article we have seen the basic concepts of oracle VM for SPARC (LDOM), packages installation,configuration the service domain,configuring the control domain or primary domain and saving the sp-configuration. I will try to complete the tutorial by adding another 6 to 8 articles. Hope you will like it . Please share it in social media to reach maximum number of people. Thank you for visiting UnixArena LDOM Tutorial Part 2 – Configuring the guest domain

How to configure the Guest domains on LDOM ? February 18, 2014 in LDOM

Guest domain is a top level virtual machine which we can create using the control domain. Guest domain will not perform any administrative operation on ldom but runs its own operating system on it.Once you have completed the Oracle VM for SPARC tutorial 1 ,then you can start following this article.We will see how we can create a new guest domain and allocate the resources to it. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains (You Are Here) Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services













Creating the new Guest Domain

1. Login to the control domain which we have already configured . (Refer Oracle VM for SPARC tutorial 1 ) 2. You need to find out how much resources are available in the system.Use ldm list-devices -a command to get those details.

PD-bash# ldm list-devices -a

CORE

ID %FREE CPUSET

0 0 (0, 1, 2, 3, 4, 5, 6, 7)

1 100 (8, 9, 10, 11, 12, 13, 14, 15)

2 100 (16, 17, 18, 19, 20, 21, 22, 23)

3 100 (24, 25, 26, 27, 28, 29, 30, 31)

4 100 (64, 65, 66, 67, 68, 69, 70, 71)

5 100 (72, 73, 74, 75, 76, 77, 78, 79)

6 100 (80, 81, 82, 83, 84, 85, 86, 87)

7 100 (88, 89, 90, 91, 92, 93, 94, 95)

VCPU


http://www.unixarena.com/wp-content/uploads/2014/02/Creating-the-new-guest-domains1.jpg

PID %FREE PM

0 0 no

1 0 no

2 0 no

3 0 no

4 0 no

5 0 no

6 0 no

7 0 no

8 100 --- -------------->Free CPU's cores starts here.

9 100 ---

10 100 ---

11 100 ---

12 100 ---

13 100 ---

14 100 ---

15 100 ---

16 100 ---

17 100 ---

18 100 ---

19 100 ---

20 100 ---

21 100 ---

22 100 ---

23 100 ---

24 100 ---

25 100 ---

26 100 ---

27 100 ---

28 100 ---

29 100 ---

30 100 ---

31 100 ---

64 100 ---

65 100 ---

66 100 ---

67 100 ---

68 100 ---

69 100 ---

70 100 ---

71 100 ---

72 100 ---

73 100 ---

74 100 ---

75 100 ---

76 100 ---

77 100 ---

78 100 ---

79 100 ---

80 100 ---

81 100 ---

82 100 ---

83 100 ---

84 100 ---

85 100 ---

86 100 ---

87 100 ---

88 100 ---

89 100 ---

90 100 ---

91 100 ---

92 100 ---

93 100 ---

94 100 ---

95 100 --- -------------->Free CPU's cores ends here.

MAU

ID CPUSET BOUND

0 (0, 1, 2, 3, 4, 5, 6, 7) primary

1 (8, 9, 10, 11, 12, 13, 14, 15)

2 (16, 17, 18, 19, 20, 21, 22, 23)

3 (24, 25, 26, 27, 28, 29, 30, 31)

4 (64, 65, 66, 67, 68, 69, 70, 71)

5 (72, 73, 74, 75, 76, 77, 78, 79)

6 (80, 81, 82, 83, 84, 85, 86, 87)

7 (88, 89, 90, 91, 92, 93, 94, 95)

MEMORY

PA SIZE BOUND

0x0 512K _sys_

0x80000 1536K _sys_

0x200000 94M _sys_

0x6000000 32M _sys_

0x8000000 96M _sys_


0x10000000 28416M ------------------------------> Free Physical memory

0x700000000 4G primary

IO


pci@400 pci_0 yes

pci@500 pci_1 yes

PD-bash#

#

3.Create a new Guest domain using the below command.

PD-bash# ldm add-domain ua1

PD-bash# ldm list-domain


primary active -n-cv- SP 8 4128M 0.3% 17m

ua1 inactive ------

PD-bash#

4.Add 8 vcpus to guest domain ua1.

PD-bash# ldm add-vcpu 8 ua1




ua1 inactive ------ 8

PD-bash#

5.Add 4G memory to logical domain ua1.

PD-bash# ldm add-memory 4G ua1




ua1 inactive ------ 8 4G

PD-bash#

6.Create a new virtual NIC on top the virtual switch and assigned to ua1. List the logical domain bindings to verify the configuration.

PD-bash# ldm add-vnet vnet0 primary-vsw0 ua1

PD-bash# ldm list-bindings ua1



UUID

1e8baa72-a123-6184-f724-feebad1585eb

CONTROL


DEPENDENCY

master=

NETWORK

NAME SERVICE ID DEVICE MAC MODE PVID VID

MTU LINKPROP

vnet0 primary-vsw0 0 00:14:4f:fb:54:92 1

PD-bash#

7.Now its time to add the new disks to logical domain “ua1″ . Here I am using “/dev/dsk/c4t0456RTE000005400002d0s2″ LUN for ua1 domain . 8.First i am allocating the LUN to virtual disk server.

PD-bash# ldm add-vdiskserverdevice /dev/dsk/c4t600144F0B8E706DA00005300F4350005d0s2 vol0@primary-vds0

PD-bash# ldm list-services vds0

LDom "vds0" was not found

PD-bash# ldm list-services

VCC



VSW


PVID VID MTU MODE

primary-vsw0 primary 00:14:4f:fa:12:0e nxge1 0 switch@0 1 1

1500

VDS


primary-vds0 primary vol0

/dev/dsk/c4t0456RTE000005400002d0s2

PD-bash#

9.Now bind the virtual disk to logical domain “ua1″ which you have created in step 8 .

PD-bash# ldm add-vdisk vdisk0 vol0@primary-vds0 ua1




UUID

1e8baa72-a123-6184-f724-feebad1585eb

CONTROL


DEPENDENCY

master=

NETWORK


MTU LINKPROP


DISK

NAME VOLUME TOUT ID DEVICE SERVER MPGROUP

vdisk0 vol0@primary-vds0 0

PD-bash#

10.The same way you can map the ISO image file to guest domain using the below set of commands.

PD-bash# ldm add-vdiskserverdevice /export/home/sol-10-u10-ga2-sparc-dvd.iso

iso0@primary-vds0

PD-bash# ldm add-vdisk iso0 iso0@primary-vds0 ua1




UUID

1e8baa72-a123-6184-f724-feebad1585eb

CONTROL


DEPENDENCY

master=

NETWORK


MTU LINKPROP


DISK


vdisk0 vol0@primary-vds0 0

iso0 iso0@primary-vds0 1

PD-bash#

11.Just bind the logical domain to the primary domain.





PD-bash#

PD-bash# ldm bind-domain ua1

LDom ua1 started




ua1 bind ------ 5000 8 4G

PD-bash#

12.Set the auto-boot is false to logical domain “ua1″ for Operating system installation.

PD-bash# ldm set-variable auto-boot\?=false ua1

13.Boot/start the logical domain “ua1″.

PD-bash# ldm start-domain ua1

LDom ua1 started




ua1 active -t---- 5000 8 4G 12% 1s

PD-bash#

14. Take the console of the logical domain “ua1″ .It uses the console port 5000 (see step:13)

ok boot iso0

PD-bash# telnet 192.168.2.11 5000

Trying 1192.168.2.11...

Connected to 192.168.2.11.

Escape character is '^]'.

Connecting to console "ua1" in group "ua1" ....

Press ~? for control options ..

{0} ok

15. List the devices to boot.

{0} ok devalias

iso0 /virtual-devices@100/channel-devices@200/disk@1

vdisk0 /virtual-devices@100/channel-devices@200/disk@0

vnet0 /virtual-devices@100/channel-devices@200/network@0

net /virtual-devices@100/channel-devices@200/network@0

disk /virtual-devices@100/channel-devices@200/disk@0

virtual-console /virtual-devices/console@1

name aliases

{0} ok

16.Boot the system from iso.

{0} ok boot iso0

Boot device: /virtual-devices@100/channel-devices@200/disk@1 File and args:

SunOS Release 5.10 Version Generic_147440-01 64-bit

Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.

Configuring devices.

Using RPC Bootparams for network configuration information.

Attempting to configure interface vnet0...

|

You have successfully created the guest domain and assigned the resources to it. You have also mapped the virtual disks to the virtualdisk server and mapped to the guest domain or logical domain. In the end of the article ,we have see how to access the console of the guest domains and mapped the ISO file for OS installation. Hope this article is informative to you .Please share it in social networks. Thank you for visiting UnixArena.

How to Configure the I/O Domain on Oracle VM for SPARC ? February 19, 2014 in LDOM

If you allow any of the guest domain or control domain to access the any I/O (PCI) cards directly from the hardware ,usually that domain is called I/O domain or root domain.By default, two virtual I/O services that you need to configure on primary domain to provide the I/O to the guest domains.One is virtual disk (vds) for storage and another one is virtual switch(vsw) for network communication.But in I/O domain,you can directly access the real physical LUN or physical network port.But the typical guest domain will be using the virtual I/O services.For an example, SAN LUN’s will be mapped to control domain and from there , you can map the physical LUNS to guest domains as virtual services. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain (You Are Here) Dynamically adding/removing Resources Live Migration for Guest Domains

Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services

IO Domain Example

In the above image, control domain aka primary domain and SOL10 Guest domain also called I/O domains since its directly accessing the PCI card(HBA). Here we will see how we can configure the guest domain as I/O domain. 1. Login to control domain and list the complete hardware details of the server .

-bash-3.2# ldm list-devices -a

CORE










http://www.unixarena.com/wp-content/uploads/2014/02/IO-Domain-Main1.jpg

ID %FREE CPUSET

0 0 (0, 1, 2, 3)

1 0 (4, 5, 6, 7)

2 0 (8, 9, 10, 11)

3 0 (12, 13, 14, 15)

4 0 (16, 17, 18, 19)

5 0 (20, 21, 22, 23)

6 0 (24, 25, 26, 27)

7 0 (28, 29, 30, 31)

VCPU

PID %FREE PM

0 0 no

1 0 no

2 0 no

3 0 no

4 0 no

5 0 no

6 0 no

7 0 no

8 0 no

9 0 no

10 0 no

11 0 no

12 0 no

13 0 no

14 0 no

15 0 no

16 0 no

17 0 no

18 0 no

19 0 no

20 0 no

21 0 no

22 0 no

23 0 no

24 0 no

25 0 no

26 0 no

27 0 no

28 0 no

29 0 no

30 0 no

31 0 no

MAU

ID CPUSET BOUND

0 (0, 1, 2, 3) primary

1 (4, 5, 6, 7)

2 (8, 9, 10, 11)

3 (12, 13, 14, 15)

4 (16, 17, 18, 19)

5 (20, 21, 22, 23)

6 (24, 25, 26, 27)

7 (28, 29, 30, 31)

MEMORY

PA SIZE BOUND

0x0 512K _sys_

0x80000 1536K _sys_

0x200000 62M _sys_

0x4000000 64M _sys_

0x8000000 2G primary

0x88000000 512M ldom1

0xa8000000 512M ldom3

0xc8000000 896M ldom2

IO


pci@780 bus_a yes

pci@7c0 bus_b yes

-bash-3.2#

In the bottom of the above command output shows that IO (pci@780 and pci7c0) have already bind to some of the domain. 2.To find where the PCI IO card has been bound , you need to use “ldm list-bindings” command.

-bash-3.2# ldm list-bindings


primary active -ndcv- SP 8 2G 0.6% 4h 58m

UUID

0066bb91-df76-c6ca-a92b-f18faf256978

MAC

00:14:4f:9d:ca:6a

HOSTID

0x849dca6a

CONTROL


DEPENDENCY

master=

CORE

CID CPUSET

0 (0, 1, 2, 3)

1 (4, 5, 6, 7)

VCPU


0 0 0 0.2% 100%

1 1 0 0.2% 100%

2 2 0 0.1% 100%

3 3 0 0.1% 100%

4 4 1 0.1% 100%

5 5 1 0.4% 100%

6 6 1 0.1% 100%

7 7 1 0.1% 100%

MAU

ID CPUSET

0 (0, 1, 2, 3)

MEMORY

RA PA SIZE

0x408000000 0x8000000 2G

VARIABLES

keyboard-layout=US-English

IO


pci@780 bus_a

pci@7c0 bus_b

The above outputs confirms both the PCI card has been assigned to primary domain.So our primary domain is also called I/O domain or root domain. 3.We will remove one the PCI card from primary and we will assign to one of the guest domain. Before removing,please make sure that card is not used in primary .If its in use, please un-configure it .Let me confirm which card can be removed .(pci@780 and pci@7c0) pci@780 is used for internal disks. We are good to move the pci@7c0 to one of the guest domains since there is no device is currently using it from primary domain.

AVAILABLE DISK SELECTIONS:

0. c0t0d0

/pci@780/pci@0/pci@9/scsi@0/sd@0,0

1. c0t1d0

/pci@780/pci@0/pci@9/scsi@0/sd@1,0


vsw0 link: up speed: 1000 Mbps duplex: full

e1000g0 link: up speed: 1000 Mbps duplex: full

e1000g1 link: unknown speed: 0 Mbps duplex: half



-bash-3.2# cat /etc/path_to_inst |grep e1000g

"/pci@780/pci@0/pci@1/network@0" 0 "e1000g"

"/pci@780/pci@0/pci@1/network@0,1" 1 "e1000g"

"/pci@7c0/pci@0/pci@2/network@0" 2 "e1000g"

"/pci@7c0/pci@0/pci@2/network@0,1" 3 "e1000g"

-bash-3.2#

4.List the configured guest domain.

-bash-3.2# ldm list-domain


primary active -n-cv- SP 8 2G 0.5% 4h 51m

ldom1 bound ------ 5000 8 512M

ldom2 bound ------ 5001 8 896M

ldom3 active -n---- 5002 8 512M 0.0% 3h 20m

-bash-3.2#

5. Remove or unbind the pci@7c0 to from primary domain.You can see couple of physical network adapters has been removed from primary.You may need to reboot the server to release the pci cards sometimes.

-bash-3.2# ldm remove-io pci@7c0





-bash-3.2#

-bash-3.2# ldm list-bindings



IO


pci@780 bus_a

6.Now add the pci card pci7c0 to ldom1 .sometimes you may need to reboot the system to see the new pci cards.

# ldm add-io pci@7c0 ldom1

Initiating a delayed reconfiguration operation on the primary domain.

All configuration changes for other domains are disabled until the primary

domain reboots, at which time the new configuration for the primary domain

will also take effect.

-bash-3.2#

7.You can list the binding using “ldm list-bindings” command to confirm the changes.sometimes you need to reboot the system to see the devices in operating system level.

ldm list-bindings ldom1

NAME SERVICE PORT

SP

------------------------------------------------------------------------------


ldom1 bound ------ 5000 8 512M

UUID

613e0c0c-3554-610d-e36f-9441124ce387

MAC

00:14:4f:fb:11:83

HOSTID

0x84fb1183

CONTROL


DEPENDENCY

master=

CORE

CID CPUSET

2 (8, 9, 10, 11)

3 (12, 13, 14, 15)

VCPU


0 8 2 100%

1 9 2 100%

2 10 2 100%

3 11 2 100%

4 12 3 100%

5 13 3 100%

6 14 3 100%

7 15 3 100%

MEMORY

RA PA SIZE

0x8000000 0x88000000 512M

VARIABLES

auto-boot?=false

boot-device=vdisk1:a disk net


IO


pci@7c0 bus_b

8. Login to ldom1 and see the pci card devices .





-bash-3.2#

Now ldom1 is also called as I/O domain since its accessing the network port directly without hypervisor. The same way if your system have multiple HBA-cards , you can assign one of the cards to guest domain and SAN team can directly allocate tothe luns to guest domain.(ohhh sorry now its called I/O domain) You have successfully removed the pci card from primary domain and assigned to the guest domain. So the guest domain also become I/O domain now. These I/O domain will be used on special application requirements. Otherwise we will configure only the guest domains.

How to Dynamically add the resources in Oracle VM for SPARC ? February 20, 2014 in LDOM

The beauty of LDOM or Oracle VM for SPARC is that resources can be dynamically added or removed from the domains(Primary/Guest). This will help you to perform the load balancing between the guest domains on the physical server. Dynamic resource allocation is only possible if the guest domain operating system is booted.If you add the resource during the guest domain OS start-up, allocation will fail. Note:Solaris operating system support dynamic resources allocation. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources (You Are Here) Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services

Here we will see how to add/remove resources dynamically when the guest domain is up and running.Before performing any resource management, please make sure you have enough resources on the system .(use “# ldm list-devices -a” to find free resources) 1.List the running domains from the control domain aka primary domain.

bash-3.2# ldm ls-domain


primary active -n-cv- SP 14 4128M 0.2% 7h 48m

ldom2 active -n---- 5000 4 4G 0.1% 9h 28m

ldom2clone active -n---- 5001 8 8G 0.1% 7h 7m










http://www.unixarena.com/wp-content/uploads/2014/02/LDOM-Dynamically-moving-the-resources.jpg

As per the above output,all the domains are in active state.Which means domains are in powered.(Login to the ldoms and verify OS is up or not .) 2.Add four more virtual CPU’s to ldom2 guest domain.

-bash-3.2# ldm add-vcpu 4 ldom2

-bash-3.2# ldm ls-domain



ldom2 active -n---- 5000 8 4G 0.3% 9h 28m


You can see number of vcpu has been increased from 4 to 8 for ldom2. 3.Number CPU’s can added by directly setting the new value.For an example, Currently ldom2 has just 8 vcpu .If you want to add four more, you just give the below command.

-bash-3.2# ldm set-vcpu 12 ldom2




ldom2 active -n---- 5000 12 4G 0.1% 9h 29m


4.CPU also can be removed using set command. Currently ldom2 has 12 vcpu’s. If you want to remove 8 from that, just give the below command.

-bash-3.2# ldm set-vcpu 4 ldom2




ldom2 active -n---- 5000 4 4G 3.4% 9h 29m


-bash-3.2#

5.Removing vcpu also can be done using remove-vpcu command.

-bash-3.2# ldm remove-vcpu 2 ldom2




ldom2 active -n---- 5000 2 6G 0.2% 9h 32m


-bash-3.2#

6. Dynamic memory management also same like the cpu management. You need to use ”memory” keyword instead of “vcpu”. To set the new memory size to the running domain,use the below commands.If the new value size is higher than the current one ,memory will be added .If the new value is lesser than the existing one, memory size will be reduced to the guest domains.




ldom2 active -n---- 5000 4 4G 0.1% 9h 30m


-bash-3.2# ldm set-memory 8G ldom2




ldom2 active -n---- 5000 4 8G 0.2% 9h 31m


7.We can also use the below method to increase the physical memory of guest domains.

-bash-3.2# ldm add-memory 2G ldom2




ldom2 active -n---- 5000 4 10G 0.1% 9h 31m


8. Like remove option in vcpu, remove-memory option is used to reduce the memory size to the guest domain.

-bash-3.2# ldm remove-memory 4G ldom2




ldom2 active -n---- 5000 4 6G 0.5% 9h 32m


-bash-3.2#

Other resources like virtual disk also can be added on fly to the virtual machine.Here I am creating a ZFS level LUN for test purpose.

-bash-3.2# zfs create -V 25G pool1/uavol1

-bash-3.2# zfs list

NAME USED AVAIL REFER MOUNTPOINT

pool1 25.4G 248G 31K /pool1

pool1/clonevol1 96.3M 248G 4.75G -

pool1/uavol1 25.3G 248G 4.70G -

pool1/vol1 25.3G 269G 4.70G -

Adding the disk to virtual disk service. (vds0)

-bash-3.2# ldm add-vdiskserverdevice /dev/zvol/dsk/pool1/uavol1 uavol1@primary-vds0

Assigning the LUN to guest domain from the virtual disk service.

-bash-3.2# ldm add-vdisk uavdisk uavol1@primary-vds0 ldom2

Connect to the server and scan the newly assiged LUN.



primary active -n-cv- SP 14 4128M 0.2% 8h

ldom2 active -n---- 5000 2 6G 0.2% 9h 40m


-bash-3.2# telnet 192.168.2.28 5000

Trying 192.168.2.28...



Connecting to console "ldom2" in group "ldom2" ....


ldom2-208 console login: root

Password:

# devfsadm

# echo |format

To terminate the ldom console, use “ctrl” + “]” key sequence to back to the control domain. You can also see the lun from the guest domain “OK” prompt. Let me halt the domain and start it.

-bash-3.2# ldm stop-domain ldom2

LDom ldom2 stopped

-bash-3.2# ldm start-domain ldom2

LDom ldom2 started

Connect to the guest domain console again.

-bash-3.2# telnet 192.168.2.28 5000

Trying 192.168.2.28...



Connecting to console "ldom2" in group "ldom2" ....


{0} ok devalias

uavdisk /virtual-devices@100/channel-devices@200/disk@0

iso_ldom2 /virtual-devices@100/channel-devices@200/disk@2

vdisk1_dom2 /virtual-devices@100/channel-devices@200/disk@1

vnet0 /virtual-devices@100/channel-devices@200/network@0

net /virtual-devices@100/channel-devices@200/network@0

disk /virtual-devices@100/channel-devices@200/disk@1

virtual-console /virtual-devices/console@1

name aliases

You can see the newly assigned disk in the OBP level as well.So the bottom line is that resource management is very simple and effective in LDOM aka oracle VM for SPARC. Hope you too liked the Oracle VM for SPARC resource management. Thank you for visiting UnixArena.

How to perform Live domain migration on LDOM ? Oracle VM SPARC February 20, 2014 in LDOM

In Oracle VM for SPARC,we can migrate the guest domains to another system without any downtime. This is similar to vmotion on VMware ESXi. This feature provides the operational flexibility to the customers who don’t want the system to be down for system maintenance.Prior to the system maintenance ,we can easily move the guest domains to another system without any downtime using this live migration.Once you have completed the system maintenance , you can just migrate the guest domains back to the original systems. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains (You Are Here) Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services










Normally we would allocate the same set of LUNS to source and target machines. The live-migration feature is just only for the Guest domain migration not for the storage.So when the control domains verify the remote system, it will check whether the guest machine storage is accessible or not. Prerequisites: Network access between Control Domains

Shared Storage

Compatible versions of Oracle VM Server for SPARC and Firmware.

Similar Hardware configuration on both the systems

Identical Virtual services between the nodes.

How the migration is happening without any downtime ? Migration has been initiated by one of the control domain root user

Control domain performs the nessasary check prior to the migration.It checks server compatibitliy,network connnectivity and available resources

Virtual machine configuration is copied from source to target

Virtaul machine’s in-memory contents will be copied to target system

Guest domain will be suspended on source system

Finally guest will be started on the target system

Here we will see how we can perform the live migration between two T5140 systems. 1. Login to the source system and list the guest domains.



primary active -n-cv- SP 12 4128M 0.5% 1d 15h 45m

ua1guest active -n---- 5000 12 6G 0.1% 1h 35m

-bash-3.2#

2.List the guest domain “ua1guest” bindings.Make sure the remote system should has enough enough resource to handle this guest domains. (Use # ldm list-devices -a)

-bash-3.2# ldm ls-bindings ua1guest


http://www.unixarena.com/wp-content/uploads/2014/02/Live-Migration-LDOM.jpg


UUID

7b74fd6b-88ea-c578-d0df-efe202fb8c12

MAC

00:14:4f:f9:87:e7

HOSTID

0x84f987e7

CONTROL


DEPENDENCY

master=

CORE

CID CPUSET

1 (8, 9, 10, 11, 12, 13, 14, 15)

3 (24, 25, 26, 27, 28, 29, 30, 31)

VCPU


0 8 1 0.2% 100%

1 9 1 0.1% 100%

2 10 1 0.1% 100%

3 11 1 0.0% 100%

4 12 1 0.0% 100%

5 13 1 0.3% 100%

6 14 1 0.0% 100%

7 15 1 0.0% 100%

8 24 3 0.0% 100%

9 25 3 0.0% 100%

10 26 3 0.0% 100%

11 27 3 0.1% 100%

MEMORY

RA PA SIZE

0x10000000 0x10000000 4G

0x400000000 0x110000000 2G

VARIABLES

auto-boot?=false


NETWORK


MTU LINKPROP

vnet0 primary-vsw0@primary 0 network@0 00:14:4f:fa:b2:d6 1

1500

PEER MAC MODE PVID VID MTU LINKPROP

primary-vsw0@primary 00:14:4f:fa:12:0e 1 1500

DISK


ua1guestdsk ua1guest@primary-vds0 0 disk@0 primary

VCONS

NAME SERVICE PORT

ua1guest primary-vc0@primary 5000

-bash-3.2#

3.List the source system’s services. You should also have the same services name configured on the target system.


VCC



VSW


PVID VID MTU MODE

primary-vsw0 primary 00:14:4f:fa:12:0e nxge1 0 switch@0 1 1

1500

VDS


primary-vds0 primary ua1guest

/dev/rdsk/c4t600144F0C1135B490000530383E50007d0s2

GSTISO ro /export/home/sol-10-u10-

ga2-sparc-dvd.iso

-bash-3.2#

4.Now login to target system and check whether the live migration can be done or not.


VCC



VSW


PVID VID MTU MODE

primary-vsw0 primary 00:14:4f:fb:a5:6d nxge1 0 switch@0 1 1

1500

VDS


primary-vds0 primary ua1guest

/dev/rdsk/c4t600144F0C1135B490000530383E50007d0s2



primary active -n-cv- SP 8 8G 0.2% 1d 6h 23m

subuldom inactive ------ 4 4G

-bash-3.2#

As per the above command output, all the services names are matching with source system. We also have the enough resource to handle the guest domain. 5. Login to the source system’s control domain and execute the migration command.It will prompt for target system root password. Just enter continue.



primary active -n-cv- SP 12 4128M 0.3% 1d 15h 53m


-bash-3.2# ldm migrate ua1guest 192.168.2.29

Target Password:

6.Login to the target system’s control domain and list the domains.







primary active -n-cv- SP 8 8G 14% 1d 6h 26m

ua1guest bound -----t 5000 1 6G


-bash-3.2#

In the above output, you can see , “ua1guest domain is copying from the source system. 7.In the source system , you may get a warning like the below one after successfully completion of live-migration.

Domain ua1guest is not running an operating system that is

compatible with the latest migration functionality.

Upgrading to the latest software will remove restrictions on

a migrated domain that are in effect until it is rebooted.

Consult the product documentation for a full description of

these restrictions.

-bash-3.2#

8.In target system, the guest domain flag will be changed from “t” to “n” .



primary active -n-cv- SP 8 8G 44% 1d 6h 28m

ua1guest bound -----t 5000 1 6G







-bash-3.2#

9.You didn’t believe the live migration ? Let me double check.

-bash-3.2# telnet localhost 5000

Trying ::1...

telnet: connect to address ::1: Network is unreachable

Trying 127.0.0.1...

Connected to localhost.


Connecting to console "ua1guest" in group "ua1guest" ....


UAMIGD console login: root

Password:

Last login: Thu Feb 20 00:50:39 on console

Oracle Corporation SunOS 5.10 Generic Patch January 2005

# uptime

2:02am up 1:43, 1 user, load average: 0.00, 0.02, 0.01

#

As per the above command output, nothing has happened to the guest domain. The system is up from last 2hr. The migration has been initiated few minutes back.So the guest domain was up during the migration time too. Hope you will also give a try on LDOM guest live migration . Thank you for visiting UnixArena.

How to Add the Same ISO image file in Multiple LDOMS ? February 20, 2014 in LDOM

In oracle VM for SPARC (LDOM), you need to attach the ISO file to each logical domain, for the first time operating system installation. If you have pre-configured jumpstart , You can use it for the guest domain operating system installation. If you don’t have jumpstart,then you need to install it using the ISO image. By default LDOM will not allow to map the same device file to multiple guest domains .Here we will see how we can map the same ISO image to multiple LDOMS. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests (You Are Here) Configuration Backup & Restore Removing the LDOM and Services

1. Login to the control domain and list the ISO image which you need to map it to ldoms.










http://www.unixarena.com/wp-content/uploads/2014/02/LDOM-ISO-file1.jpg

# ls /export/home/sol-10-u10-ga2-sparc-dvd.iso

sol-10-u10-ga2-sparc-dvd.iso

2.List the available services. Create a new virtual device for the virtual disk server using the below command.


VCC


primary-vcc0 primary 5000-5100

VSW


PVID VID MTU MODE

primary-vsw0 primary 00:14:4f:fa:fe:49 e1000g0 0 switch@0 1 1

1500

VDS


primary-vds0 primary vol1 /dev/zvol/dsk/pool1/vol1

clonevol1

/dev/zvol/dsk/pool1/clonevol1

vol0

/dev/dsk/c2t600144F0B8E706DA00005300F47B0007d0s2

vol10

/dev/dsk/c2t600144F0C1135B4900005303839A0002d0s2

# ldm add-vdsdev options=ro /export/home/sol-10-u10-ga2-sparc-dvd.iso newiso@primary-vds0


VCC



VSW


PVID VID MTU MODE

primary-vsw0 primary 00:14:4f:fa:fe:49 e1000g0 0 switch@0 1 1

1500

VDS



clonevol1 /dev/zvol/dsk/pool1/clonevol1

vol0

/dev/dsk/c2t600144F0B8E706DA00005300F47B0007d0s2

vol10

/dev/dsk/c2t600144F0C1135B4900005303839A0002d0s2

iso0 ro /export/home/sol-10-u10-ga2-sparc-dvd.iso

3.Add the newly created virtual read-only iso image to guest domain.

-bash-3.2# ldm list-domain



ua1 active -n---- 5000 8 4G 0.1% 58m

ua2 active -n---- 5001 8 2G 0.0% 58m

ua3 active -n---- 5002 4 1G 0.1% 30m

-bash-3.2# ldm add-vdisk viso1 newiso@primary-vds0 ua1

-bash-3.2#

4.You can see that ISO file has been attached to the ua1 ldom.

-bash-3.2# ldm ls-bindings ldom1


ldom1 bound ------ 5000 8 512M

DISK


vdisk1 vol1@primary-vds0 2 disk@2 primary

viso1 newiso@primary-vds0 0 disk@0 primary

5.When try to map the same iso virtual image to another domain, you will get below error.

-bash-3.2# ldm add-vdisk viso2 newiso@primary-vds0 ua2

The volume "newiso" is already in use.

6.To avoid the above error, you need to create a new virtual read-only ISO image again.

-bash-3.2# ldm add-vdsdev options=ro /export/home/sol-10-u10-ga2-sparc-dvd.iso newiso2@primary-vds0

-bash-3.2#

7.Add the second virtual iso image to the another LDOM using the below command.

-bash-3.2# ldm add-vdisk viso2 newiso2@primary-vds0 ua2

-bash-3.2#

-bash-3.2# ldm list-bindings ua2


ua2 active -n---- 5001 8 2G 0.1% 1h 3m

DISK


vdisk1 ua1ldom2@primary-vds0 0 disk@0 primary

viso2 newiso2@primary-vds0 1 disk@1 primary

The same way you can map the same ISO file to N-number of guest domains.(6 and 7 ). Hope this article is informative to you. Thank you for visiting UnixArena.

Oracle VM SPARC – LDOM configuration Backup and Restore February 21, 2014 in LDOM

Oracle VM for SPARC configuration can be backed up in XML file and it can be restored it whenever required.This configuration backup will be very useful for future rebuilding. It also provides the option to take the backup of particular


domain and that can be restored using ldm add-domain command.In this article we will how take the individual domain backup,how to take the complete ldom configuration backup and restore methods. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore (You Are Here) Removing the LDOM and Services 1. Login to control Domain and list the current configuration file.

-bash-3.2# ldm list-spconfig

factory-default

config_new [current]

-bash-3.2#

2.To take the complete LDOM configuration backup,use below command.

# ldm list-constraints -x >config.xml

3.You can also take the configuration backup for specific LDOM,

# ldm list-constraints -x ldom_name > ldom_name.xml

How to restore the system to factory default ? 1. Login to the control domain console and remove the spconfig profile.


factory-default

config_new [current]

bash-3.2# ldm remove-spconfig config_new


factory-default [next poweron]

bash-3.2#

2.Power cycle the server to take the changes effect.

bash-3.2# init 5

bash-3.2# svc.startd: The system is coming down. Please wait.

svc.startd: 101 system services are now being stopped.

Feb 18 10:00:49 sol10-11 syslogd: going down on signal 15

svc.startd: The system is down.









syncing file systems... done

Serial console stopped.

-> show /SYS

Properties:

type = Host System

ipmi_name = /SYS

keyswitch_state = Normal

product_name = T5140

product_part_number = 4613802-42

product_serial_number = FML1015013

product_manufacturer = SUN MICROSYSTEMS

fault_state = Faulted

power_state = Off

-> start /SYS

Are you sure you want to start /SYS (y/n)? y

Starting /SYS

->

-> start /SP/console

Are you sure you want to start /SP/console (y/n)? y

Serial console started. To stop, type #.

Done

0:0:0>Master CPU Tests Basic....Done

0:0:0>Init MMU.....

0:0:0>L2 Tests....Done

0:0:0>Extended CPU Tests....Done

0:0:0>Scrub Memory....Done

0:0:0>Functional CPU Tests....Done

sol10-11 console login:

3. Login to the system and see whether factory default settings has been restored or not .

sol10-11 console login: root

Password:

Feb 18 10:10:58 sol10-11 login: ROOT LOGIN /dev/console

Last login: Tue Feb 18 01:19:27 on console

Oracle Corporation SunOS 5.10 Generic Patch January 2005

You have new mail.

-bash-3.2#


factory-default [current]

-bash-3.2#

Restoring the sp-config : 1.Once the system has restored to the factory default,then you can restore the system backup using the XML files which we have created. Login to the control domain and restore it.

# ldm init-system -i config.xml

# ldm add-spconfig unixarencfg1

# ldm list-spconfig

factory-default

unixarenacfg1 (current)

2.To restore the specific ldom/primary domain configuration

# ldm init-system -i ldom_name.xml

# ldm add-spconfig unixarencfg2

# ldm list-spconfig

factory-default

unixarenacfg1


3.To restore only the virtual service,

# ldm init-system -s -i filename.xml


factory-default

unixarenacfg1

unixarenacfg2[active on Reboot]

-bash-3.2# ldm add-spconfig unixarencfg3


factory-default

unixarenacfg1

unixarenacfg2


The old configuration copies can be removed using “ldm remove-spconfig profile_name” . If you make any small changes on LDOM configuration, you must create a new profile to save it.Otherwise you may loose the new changes after the system power cycle. Thank you for visiting UnixArena.

How to Remove Logical Domains and Services ? Oracle VM for SPARC February 20, 2014 in LDOM


Destroying the infrastructure is a easy task in all the environments.In oracle VM for SPARC also ,you can easily remove the guest domains in few seconds. The same way you can also remove the virtual disk resources and virtual network resources. In some cases you may not able to remove the resources if its in use or already bound to any guest domains. In this article ,we will see how to remove the guest domains and how to remove the virtual services like vds and vswitch. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services (You Are Here) 1. Login to the control domain and list the available domains.




ldom2 active -t---- 5000 2 6G 50% 4m


2.Halt the domain by logging in to the guest domain operating system. (init 5) . You can also perform the force shutdown from the control domain using the below command.

-bash-3.2# ldm stop-domain ldom2clone

LDom ldom2clone stopped

3.Un-bind the domain from the hypervisor.

-bash-3.2# ldm unbind-domain ldom2clone




ldom2 active -t---- 5000 2 6G 50% 5m

ldom2clone inactive ------ 8 8G

-bash-3.2#

4.Now you can remove the inactive guest domain from the hypervisor.

-bash-3.2# ldm remove-domain ldom2clone












ldom2 active -t---- 5000 2 6G 50% 7m

-bash-3.2#

You can see ldom2clone has been removed from the hypervisor completely. If you want to remove the specific resources from the guest domain,follow the below steps. 1. List the bounded resources for one of the guest domain. (ldom2)

-bash-3.2# ldm list-bindings ldom2


ldom2 active -t---- 5000 2 6G 50% 8m

UUID

f2aa98a4-ddae-6d93-948f-9e2e9e81ff40

MAC

00:14:4f:fb:b6:94

HOSTID

0x84fbb694

CONTROL


DEPENDENCY

master=

CORE

CID CPUSET

1 (8, 9)

VCPU


0 8 1 100% 100%

1 9 1 0.0% 100%

MEMORY

RA PA SIZE

0x10000000 0x10000000 4G

0x480000000 0x390000000 2G

VARIABLES

auto-boot?=false

boot-device=disk:a disk net


NETWORK


MTU LINKPROP

vnet0 primary-vds0@primary 0 network@0 00:14:4f:f9:53:57 1

1500


primary-vds0@primary 00:14:4f:fb:e2:f3 1 1500

DISK


vdisk1_dom2 vol1@primary-vds0 1 disk@1 primary

iso_ldom2 iso0@primary-vds0 2 disk@2 primary

uavdisk uavol1@primary-vds0 0 disk@0 primary

VCONS

NAME SERVICE PORT

ldom2 primary-vcc0@primary 5000

2.Try to remove the one of the virtual disk resource from the guest domain.

-bash-3.2# ldm remove-vdisk uavdisk ldom2

Please perform the operation while the LDom is bound or inactive

-bash-3.2# ldm remove-vdisk -f uavdisk ldom2

Please perform the operation while the LDom is bound or inactive

You are not able to remove the virtual disk since its in use on ldom2. 3.You can stop the guest domain and then remove it .

-bash-3.2# ldm stop-domain ldom2

LDom ldom2 stopped

-bash-3.2# ldm remove-vdisk uavdisk ldom2

-bash-3.2#

-bash-3.2# ldm list-bindings ldom2


ldom2 bound ------ 5000 2 6G

UUID

f2aa98a4-ddae-6d93-948f-9e2e9e81ff40

MAC

00:14:4f:fb:b6:94

HOSTID

0x84fbb694

CONTROL


DEPENDENCY

master=

CORE

CID CPUSET

1 (8, 9)

VCPU


0 8 1 100%

1 9 1 100%

MEMORY

RA PA SIZE

0x10000000 0x10000000 4G

0x480000000 0x390000000 2G

VARIABLES

auto-boot?=false

boot-device=disk:a disk net


NETWORK


MTU LINKPROP

vnet0 primary-vds0@primary 0 network@0 00:14:4f:f9:53:57 1

1500


primary-vds0@primary 00:14:4f:fb:e2:f3 1 1500

DISK


vdisk1_dom2 vol1@primary-vds0 1 disk@1 primary

iso_ldom2 iso0@primary-vds0 2 disk@2 primary

VCONS

NAME SERVICE PORT

ldom2 primary-vcc0@primary 5000

-bash-3.2#

4. Removing the devices from the vds service


VCC



VSW


PVID VID MTU MODE

primary-vds0 primary 00:14:4f:fb:e2:f3 nxge0 0 switch@0 1 1

1500

primary-vsw0 primary 00:14:4f:fa:bf:27 nxge1 1 switch@1 1 1

1500

VDS



iso0 /export/home/sol-10-u10-ga2-

sparc-dvd.iso

clonevol1

/dev/zvol/dsk/pool1/clonevol1

vol0

/dev/dsk/c3t600144F0B8E706DA00005300F4270004d0s2

vol2

/dev/dsk/c3t600144F0C1135B490000530383D70006d0

uavol1 /dev/zvol/dsk/pool1/uavol1

-bash-3.2# ldm remove-vdsdev uavol1@primary-vds0


VCC



VSW


PVID VID MTU MODE


1500

primary-vsw0 primary 00:14:4f:fa:bf:27 nxge1 1 switch@1 1 1

1500

VDS



iso0 /export/home/sol-10-u10-ga2-sparc-

dvd.iso


vol0


vol2

/dev/dsk/c3t600144F0C1135B490000530383D70006d0

-bash-3.2#

We have seen how to remove the assigned virtual disk from the guest and removing the disk from the virtual disk server. Now we will see how to remove the complete service from the service domain aka primary domain. 1.Try to remove the virtual switch service from service domain

-bash-3.2# ldm remove-vsw primary-vds0

Service is active and being used by bound client(s)

Failed to remove virtual switch

2.The above command failed because,that virtual service is used for some of the guest domains. First remove the those assigned virtual resources from guest domain the n try to remove it.

-bash-3.2# ldm remove-vsw primary-vsw0


VCC



VSW


PVID VID MTU MODE


1500

VDS



iso0 /export/home/sol-10-u10-

ga2-sparc-dvd.iso


vol0


vol2 /dev/dsk/c3t600144F0C1135B490000530383D70006d0

Using the same setup commands (showed in this article), you can remove the other guest domains and virtual services. Once you have cleared the all services and domains, you can uninstall the oracle VM for SPARC software using below method.

bash-3.2# pkgrm SUNWldm

The following package is currently installed:

SUNWldm LDoms Manager software

(sparc.sun4v) 2.0,REV=2010.09.02.10.08

Do you want to remove this package? [y,n,?,q] y

## Removing installed package instance


permission during the process of removing this package.

Do you want to continue with the removal of this package [y,n,?,q] y

## Verifying package dependencies in global zone


## Removing pathnames in class

Waiting up to 60 seconds for svc:/ldoms/ldmd:default to stop...


/var/svc/manifest/platform/sun4v

/var/svc/manifest/platform

/var/svc/manifest

/var/svc

<<<<<<<<>>>>

/opt/SUNWldm/bin/schemas/ovf-strings.xsd

/opt/SUNWldm/bin/schemas/ovf-section.xsd

/opt/SUNWldm/bin/schemas/ovf-envelope.xsd

/opt/SUNWldm/bin/schemas/ovf-core.xsd

/opt/SUNWldm/bin/schemas/ldmd-property.xsd

/opt/SUNWldm/bin/schemas/ldmd-binding.xsd

/opt/SUNWldm/bin/schemas/event-v3.xsd

/opt/SUNWldm/bin/schemas/combined-v3.xsd

/opt/SUNWldm/bin/schemas/cli-list-constraint-v3.xsd

/opt/SUNWldm/bin/schemas/cim-vssd.xsd

/opt/SUNWldm/bin/schemas/cim-rasd.xsd

/opt/SUNWldm/bin/schemas/cim-common.xsd

/opt/SUNWldm/bin/schemas

/opt/SUNWldm/bin/ldmd_start

/opt/SUNWldm/bin/ldmd

/opt/SUNWldm/bin/ldmconfig

/opt/SUNWldm/bin/ldm

/opt/SUNWldm/bin

/opt/SUNWldm

/opt

## Updating system information.

Removal of was successful.

bash-3.2# pkgrm SUNWldomu


SUNWldomu Solaris Logical Domains (Usr)

(sparc.sun4v) 11.10.0,REV=2006.08.08.12.13






/usr/lib/sparcv9/libpri.so.1

/usr/lib/sparcv9/libds.so.1

/usr/lib/sparcv9

/usr/lib/libpri.so.1

/usr/lib/libds.so.1

/usr/lib/ldoms/vntsd

/usr/lib/ldoms/ldmad

/usr/lib/ldoms/drd

/usr/lib/ldoms

/usr/lib

/usr



bash-3.2# pkgrm SUNWldomr


SUNWldomr Solaris Logical Domains (Root)

(sparc.sun4v) 11.10.0,REV=2006.10.04.00.26








## Executing preremove script.


Waiting up to 60 seconds for svc:/ldoms/vntsd:default to stop...

Waiting up to 60 seconds for svc:/ldoms/agents:default to stop...

Waiting up to 60 seconds for svc:/platform/sun4v/drd:default to stop...


/var/svc/manifest/platform/sun4v

/var/svc/manifest/platform

/var/svc/manifest

/var/svc

/var

/platform/sun4v/kernel/misc/sparcv9/platsvc

/platform/sun4v/kernel/misc/sparcv9/ldc

/platform/sun4v/kernel/misc/sparcv9/fault_iso

/platform/sun4v/kernel/misc/sparcv9/ds

/platform/sun4v/kernel/misc/sparcv9/dr_mem

/platform/sun4v/kernel/misc/sparcv9/dr_io

/platform/sun4v/kernel/misc/sparcv9/dr_cpu

/platform/sun4v/kernel/misc/sparcv9

/platform/sun4v/kernel/misc

/platform/sun4v/kernel/drv/sparcv9/vsw

/platform/sun4v/kernel/drv/sparcv9/vnet

/platform/sun4v/kernel/drv/sparcv9/vlds

/platform/sun4v/kernel/drv/sparcv9/vldc

/platform/sun4v/kernel/drv/sparcv9/vds

/platform/sun4v/kernel/drv/sparcv9/vdc

/platform/sun4v/kernel/drv/sparcv9/vcc

/platform/sun4v/kernel/drv/sparcv9/ds_snmp

/platform/sun4v/kernel/drv/sparcv9/ds_pri

/platform/sun4v/kernel/drv/sparcv9/drctl

/platform/sun4v/kernel/drv/sparcv9/cnex

/platform/sun4v/kernel/drv/sparcv9

/platform/sun4v/kernel/drv/ds_snmp.conf

/platform/sun4v/kernel/drv/ds_pri.conf

/platform/sun4v/kernel/drv/drctl.conf

/platform/sun4v/kernel/drv

/platform/sun4v/kernel

/platform/sun4v

/platform

/lib/svc/method/svc-vntsd

/lib/svc/method/svc-drd

/lib/svc/method/ldoms-agents

/lib/svc/method

/lib/svc

/lib



bash-3.2# pkginfo | grep -i ldom

application SUNWldmp2v LDoms P2V tool

bash-3.2# pkgrm SUNWldmp2v


SUNWldmp2v LDoms P2V tool

(sparc) 2.0,REV=2010.09.02.10.08










/usr/share/man/man1m/ldmp2v.1m

/usr/share/man/man1m

/usr/share/man

/usr/share

/usr/sbin/ldmp2v

/usr/sbin

/usr

/opt/SUNWldmp2v/man/man1m/ldmp2v.1m

/opt/SUNWldmp2v/man/man1m

/opt/SUNWldmp2v/man

/opt/SUNWldmp2v/lib/ldmp2v_convert

/opt/SUNWldmp2v/lib

/opt/SUNWldmp2v/etc/ldmp2v.conf.sample

/opt/SUNWldmp2v/etc

/opt/SUNWldmp2v/bin/ldmp2v_prepare

/opt/SUNWldmp2v/bin

/opt/SUNWldmp2v

/opt



bash-3.2# pkginfo | grep -i ldom*

bash-3.2# pkginfo | grep -i ldm*

bash-3.2#

You have successfully uninstall the oracle VM for SPARC software. You can safely recycle the control domain to remove the complete ldom stuffs. Thank you for visiting UnixArena.

LDOM console write access issue – Solaris March 27, 2014 in LDOM

Now a days i have mostly dealing with oracle VM for SPARC on T-4 servers.(LDOM). As we all know that hardware console can be writable for only one user at a time and if you take the second session , it will automatically connect in read only mode unless you use the force option. The same way oracle VM for SPARC console also behaves even though its a virtual consoles on the hypervisor levels. Sometimes , the first console session may terminate unexpectedly due to network issues and if you try take the console for second time ,you may get error that “You do not have write access” on the console screen.

root@sun5#ldm list-domain



ua1 active -n---- 1 12 3G 0.1% 1h 37m

ua2 active -n---- 2 12 2560M 0.1% 1h 37m

root@sun5#

root@sun5#telnet 127.0.0.1 1


Trying 127.0.0.1...





You do not have write access









telnet> quit

Connection to 127.0.0.1 closed.

To fix this issue, login to the control domain and list the telnet session.

root@sun5#ps -ef |grep telnet

root 6259 6118 0 12:07:22 pts/4 0:00 telnet 127.0.0.1 1

root 6035 374 0 12:01:21 ? 0:00 /usr/sbin/in.telnetd



root 6004 5870 0 12:00:49 pts/1 0:00 telnet localhost 2

root 6262 6038 0 12:07:46 pts/2 0:00 grep telnet

Kill the process which is associated to the guest domain. In this case , i am having issue with “ua1″ guest domain which has the console port-1.Let me terminate the process of that and try to connect the console again.

root@sun5#kill -9 6259

root@sun5#telnet 127.0.0.1 1

Trying 127.0.0.1...





ua1 console login: root

Password:

Mar 24 13:44:13 ua1 login: ROOT LOGIN /dev/console

Last login: Mon Mar 24 13:16:41 from 192.168.0.117

Oracle Corporation SunOS 5.11 11.1 September 2012

root@ua1:~# logout

ua1 console login:

telnet> quit

Connection to 127.0.0.1 closed.

root@sun5#

Awesome .We have fixed the issue .This is small trick but you need to remember this when you face this kind of issues while connecting the console. Share it ! comment it !! Be sociable !!!

How to configure IPMP on Solaris 10 ?

June 26, 2013 in Solaris Networking

This article describes how to configure link based IPMP interfaces in Solaris 10. IPMP eliminates single network card failure and it ensures system will be always accessible via network.You can also configure the failure detection seconds in “/etc/default/mpathd” file and the default value is 10 seconds.In this file there is an option called ”FAILBACK” to specify IP behavior when primary interface recovered from the fault. “in.mpathd” is a daemon which handles IPMP (Internet Protocol Multi-Pathing) operations.There are two type of IPMP configuration available in Solaris 10.

1.Link Based IPMP

The link based IPMP detects network errors by checking the “IFF_RUNNING” flag.Normally it doesn’t require any test IP

address like probe based IPMP.

2.Probe Based IPMP The probe based IPMP detects network errors by sending ICMP “ECHO_REQUEST” messages.It requires test IP Addresses unlike link based IPMP.

1.Link Based IPMP Request:

Configure IP address “192.168.2.50″ on e1000g1 & e1000g2 using Link based IPMP.

Step:1

Find out the installed NIC’s on the systems and its status.Verify the ifconfig output as well.

Make sure the NIC status are up and not in use.

Arena-Node1#dladm show-dev




Arena-Node1#ifconfig -a

lo0: flags=2001000849 mtu 8232 index 1


e1000g0: flags=1000843 mtu 1500 index 2

inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255

ether 0:c:29:ec:b3:af

Arena-Node1#

http://www.unixarena.com/category/solaris-networking

http://2.bp.blogspot.com/-u4wqQh77Afg/Ucslv55X9UI/AAAAAAAABB0/rbLnHR6NN8A/s1600/NIC+IPMP+unixArena+solaris.jpg

Step:2

Add the IP address in /etc/hosts and specify the netmask value in /etc/netmasks like below one.

Arena-Node1#cat /etc/hosts |grep 192.168.2.50

192.168.2.50 arenagroupIP

Arena-Node1#cat /etc/netmasks |grep 192.168.2

192.168.2.0 255.255.255.0

Arena-Node1#eeprom "local-mac-address?=true"

Step:3 Plumb the interfaces which you are going to use for new IP address. check the status in “ifconfig” output.

Arena-Node1#ifconfig e1000g1 plumb

Arena-Node1#ifconfig e1000g2 plumb








inet 0.0.0.0 netmask 0

ether 0:c:29:ec:b3:b9



ether 0:c:29:ec:b3:c3

Step:4 Configure IP on Primary interface and add the interfaces to IPMP group with your own group name.

Arena-Node1#ifconfig e1000g1 192.168.2.50 netmask 255.255.255.0 broadcast + up

Arena-Node1#ifconfig e1000g1




Arena-Node1#ifconfig e1000g1 group arenagroup-1

Arena-Node1#ifconfig e1000g2 group arenagroup-1









groupname arenagroup-1






Step:5 Now we have to ensure IPMP is working fine.This can be done in two ways. i.Test:1 Remove the primary LAN cable and check it.Here i have removed the LAN cable from e1000g1 and let see what happens.















e1000g2:1: flags=1000843 mtu 1500 index 4


Again i have connected back the LAN cable to e1000g1.

Arena-Node1#dladm show-dev


















Here the configured IP is going back to original interface where it was running before. Here I had specified “FALLBACK=yes” . That’s why IP is moving back to original interface.The same way you can also specify failure detection time to mpathd using parameter “FAILURE_DETECTION_TIME” in ms.

Arena-Node1#cat /etc/default/mpathd |grep -v "#"

FAILURE_DETECTION_TIME=10000

FAILBACK=yes

TRACK_INTERFACES_ONLY_WITH_GROUPS=yes

Arena-Node1#

ii.Test:2 Normally most of the Unix admins will be sitting in remote site. So you will be not able to perform the above test.In this case ,you can use “if_mpadm” command to disable the interface in OS level. Fist i am going to disable e1000g1 and let see what happens.

Arena-Node1#if_mpadm -d e1000g1















e1000g2:1: flags=1000843 mtu 1500 index 4


Now i am going to enable it back.

Arena-Node1#if_mpadm -r e1000g1















The same way you can manually failover the IP to one interface to another interface. In the both tests,we can clearly see IP is moving from e1000g1 to e1000g2 automatically without any issues.So we have successfully configured Link based IPMP on Solaris. These failover logs will be logged in /var/adm/messages like below.

Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 215189 daemon.error] The link has gone down on e1000g1

Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 594170 daemon.error] NIC failure detected on e1000g1 of group

arenagroup-1

Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g1 to

NIC e1000g2

Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 820239 daemon.error] The link has come up on e1000g1

Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 299542 daemon.error] NIC repair detected on e1000g1 of group

arenagroup-1

Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g1

Jun 26 21:03:59 node1 in.mpathd[3800]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g1 to

NIC e1000g2

Jun 26 21:04:07 node1 in.mpathd[3800]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g1

To make the above work persistent across the reboot create the configuration files for both the network interfaces.

Arena-Node1#cat /etc/hostname.e1000g1

arenagroupIP netmask + broadcast + group arenagroup up

Arena-Node1#cat /etc/hostname.e1000g2

group arenagroup up

Thank you for reading this article. Please leave a comment if its useful for you.

ldom oracle vm for sparc

Documents