Storage Admin Notes

Tuesday, January 4, 2011

Solaris NIC speed and duplex settings

On solaris 10 dladm is the admin utility for Data-Link Interface which helps to display informarthe like Link Status (UP/DOWN), Speed, Duplex, MTU, VLAN Tagged and crucially statistics of network traffic.
# dladm show-dev
nxge0           link: down      speed: 0     Mbps       duplex: unknown
nxge1           link: down      speed: 0     Mbps       duplex: unknown
nxge2 link: up        speed: 1000 Mbps       duplex: full
nxge3 link: up        speed: 1000 Mbps       duplex: full

Setting NIC speed and duplex
Solaris is often unable to correctly auto-negotiate duplex settings with a link partner (e.g. switch), especially when the switch is set to 100Mbit full-duplex. You can force the NIC into 100Mbit full-duplex by disabling auto-negotiation and 100Mbit half-duplex capability.

Example with hme0:

1. Make the changes to the running system.
# ndd -set /dev/hme adv_100hdx_cap 0
# ndd -set /dev/hme adv_100fdx_cap 1
# ndd -set /dev/hme adv_autoneg_cap 0

2. Make kernel parameter changes to preserve the speed and duplex settings after a reboot.
# vi /etc/system
Add:
# set hme:hme_adv_autoneg_cap=0
# set hme:hme_adv_100hdx_cap=0
# set hme:hme_adv_100fdx_cap=1

Note: the /etc/system change affects all hme interfaces if multiple NICs are present (e.g. hme0, hme1).
The /etc/system settings listed above are not supported for configuring ce Ethernet adapters during system startup; you may either use ndd commands in an /etc/rc?.d script or create a/platform/sun4u/kernel/drv/ce.conf file with appropriate settings.

Example: /etc/init.d/nddconfig

#!/bin/sh

ndd -set /dev/ce instance 0
ndd -set /dev/ce adv_1000fdx_cap 0
ndd -set /dev/ce adv_1000hdx_cap 0
ndd -set /dev/ce adv_100fdx_cap 1
ndd -set /dev/ce adv_100hdx_cap 0
ndd -set /dev/ce adv_10fdx_cap 0
ndd -set /dev/ce adv_10hdx_cap 0
ndd -set /dev/ce adv_autoneg_cap 0

# ln -s /etc/init.d/nddconfig /etc/rc2.d/S31nddconfig

$ dmesg | grep ce0
Jan 20 11:05:01 crmmdb22 genunix: [ID 611667 kern.info] NOTICE: ce0: xcvr addr:0x01 - link up 100 Mbps half duplex
Jan 20 11:05:15 crmmdb22 genunix: [ID 408822 kern.info] NOTICE: ce0: no fault external to device; service available
Jan 20 11:05:15 crmmdb22 genunix: [ID 611667 kern.info] NOTICE: ce0: xcvr addr:0x01 - link up 100 Mbps full duplex

Manually determining NIC speed and duplex

If you have ce or bge interfaces, use kstat ce and kstat bge, respectively, to return NIC settings. All other interfaces may use ndd to determine NIC settings.

ndd example with hme0, assuming "instance" is 0:

# ndd -get /dev/hme link_mode
Interpretation:
0 -- half-duplex
1 -- full-duplex

# ndd -get /dev/hme link_speed
Interpretation:
0 -- 10 Mbit
1 -- 100 Mbit
1000 -- 1 Gbit

To query a different NIC, such as hme1, set the "instance" to 1, and then perform the link_mode and link_speed queries above.

# ndd -set /dev/hme instance 1

Note: the ndd commands above must be run as root. Otherwise, you will receive errors such as "couldn't push module 'hme0', No such device or address."

ce Ethernet adapters

Older versions of the Sun GigaSwift Ethernet 1.0 driver do not support the ndd link_mode and link_speed parameters. You may either install the latest Sun GigaSwift Ethernet adapter patch (111883) or you may use kstat ce ce_device to get speed and duplex information for ce Ethernet adapters.

For example click here:

$ netstat -k ce | egrep 'link_speed|link_status|link_duplex'

Interpretation:
link_up - 0 down, 1 up
link_speed - speed in Mbit/s
link_duplex - 1 half duplex, 2 full duplex, 0 down

Host/link partner mismatch example

A large number of output errors or collisions may indicate a host and link partner mismatch. The following is netstat -in output from a system configured for 100 half-duplex while the switch was configured for 100 full-duplex.

Name Mtu Net/Dest    Address      Ipkts      Ierrs Opkts       Oerrs     Collis   Queue
ce0 1500 192.168.1.0 192.168.1.1  2707133478 25    2895422910  142310052 182856975 0
In this example, the switch was configured for 100 half-duplex and the system was configured for 100 full-duplex. Note the percentage of Ierrs/Ipkts (~ 1.7%).
Name Mtu Net/Dest    Address      Ipkts      Ierrs Opkts       Oerrs   Collis   Queue
qfe1 1500 10.0.0.0     10.0.0.3     1430247    24663 1779341    0   0   0

CHANGELOG

2007/07/10 -- Added e1000g interfaces
2006/11/29 -- Added dmfe interfaces, fixed bge and ce interfaces, many other modifications and fixes.
2006/04/27 -- Solaris 7 and earlier include subinterface information with netstat -i. Patch by Royce Williams to only query physical NIC interfaces.2006/03/09 -- Fixed bge interfaces, added support for iprb interfaces, added "support" for le interfaces, modified script to only require root access if using /usr/sbin/ndd to determine NIC speed and duplex settings2005/05/25 -- Fixed ce interface handling.2004/09/15 -- Added support for ce and bge interfaces.

Veritas Dynamic Multipathing (DMP)

What is Dynamic Multipathing (DMP)?
DMP is a method of providing two or more hardware paths to a single drive. For example, the physical hardware can have at least two paths, such as c1t1d0 and c2t1d0, directing I/O to the same drive. VxVM arbitrarily selects one of the two names and creates a single device entry, then transfers data across both paths to spread the I/O, DMP is enabled by default. Volume Manager detects multipath systems with the Universal Worldwide Device Identifiers(WWD IDs) and manages multipath targets, such as disk arrays, which define policies for using more than one path.
Benefits of DMP

High availability - DMP provides greater reliability through a path failover mechanism. In the event of a loss of one connection to a disk, the system continues to access the critical data over the other sound connections until the failed path is replaced.
Improved performance - DMP provides greater I/O throughput by balancing the I/O load uniformly across multiple I/O paths to the disk device.

The disk paths can be in a active/active (use both paths) or active/passive state (use one path).

list controllers	vxdmpadm listctlr all
list enclosures	vxdmpadm listenclosure all vxdmpadm listenclosure <enclosure>
Display Paths	vxdmpadm getsubpaths ctlr=fscsi2 vxdmpadm getsubpaths dmpnodename=<enclosure>
Who controls the path	vxdmpadm getdmpnode nodename=c3t2d0 vxdmpadm getdmpnode enclosure=<enclosure>
Controller (enable/disable)	vxdmpadm disable ctlr=<controller> vxdmpadm enable ctlr=<controller>
Statistics	vxdmpadm iostat start vxdmpadm iostat reset vxdmpadm iostat show all
I/O balancing & load	vxdmpadm getattr enclosure <enclosure> iopolicy vxdmpadm setattr enclosure <enclosure> iopolicy=<policy> adaptive priority balanced (default) round-robin minimumq single-active
Path Type	vxdmpadm setattr path <path-name> pathtype=<path-type> active nomanual nopreffered primary secondary standby
Displays the restore/error daemon	vxdmpadm stat restored vxdmpadm stat errord
Starts the restore daemon	vxdmpadm start restored
Stops the restore daemon	vxdmpadm stop restored

Saturday, January 1, 2011

EMC Powerpath and MPIO in a same box

You can run into an issue with EMC storage on AIX systems using MPIO (No Powerpath) for your boot disks:

After installing the ODM_DEFINITONS of EMC Symmetrix on your client system, the system won’t boot any more and will hang with LED 554 (unable to find boot disk).

The boot hang (LED 554) is not caused by the EMC ODM package itself, but by the boot process not detecting a path to the boot disk if the first MPIO path does not corresponding to the fscsiX driver instance where all hdisks are configured. Let me explain that more in detail:

Let’s say we have an AIX system with four HBAs configured in the following order:

# lscfg -v | grep fcs

fcs2 (wwn 71ca) -> no devices configured behind this fscsi2 driver
instance (path only configured in CuPath ODM table)

fcs3 (wwn 71cb) -> no devices configured behind this fscsi3 driver
instance (path only configured in CuPath ODM table)

fcs0 (wwn 71e4) -> no devices configured behind this fscsi0 driver
instance (path only configured in CuPath ODM table)

fcs1 (wwn 71e5) -> ALL devices configured behind this fscsi1 driver
instance

Looking at the MPIO path configuration, here is what we have for the rootvg disk:

# lspath -l hdisk2 -H -F”name parent path_id connection status”

name parent path_id connection status

hdisk2 fscsi0 0 5006048452a83987,33000000000000 Enabled

hdisk2 fscsi1 1 5006048c52a83998,33000000000000 Enabled

hdisk2 fscsi2 2 5006048452a83986,33000000000000 Enabled

hdisk2 fscsi3 3 5006048c52a83999,33000000000000 Enabled

The fscsi1 driver instance is the second path (pathid 1), then remove the 3 paths keeping only the path corresponding to fscsi1 :

# rmpath -l hdisk2 -p fscsi0 -d

# rmpath -l hdisk2 -p fscsi2 -d

# rmpath -l hdisk2 -p fscsi3 -d
# lspath -l hdisk2 -H -F”name parent path_id connection status”

Afterwards, do a savebase to update the boot lv hd5. Set up the bootlist to hdisk2 and reboot the host.

It will come up successfully, no more hang LED 554.

When checking the status of the rootvg disk, a new hdisk10 has been configured with the correct ODM definitions as shown below:

# lspv

hdisk10 0003027f7f7ca7e2 rootvg active

# lsdev -Cc disk

hdisk2 Defined 00-09-01 MPIO Other FC SCSI Disk Drive

hdisk10 Available 00-08-01 EMC Symmetrix FCP MPIO Raid6

To summarize, it is recommended to setup ONLY ONE path when installing an AIX to a SAN disk, then install the EMC ODM package then reboot the host and only after that is complete, add the other paths. Dy doing that we ensure that the fscsiX driver instance used for the boot process has the hdisk configured behind.

SAN Switch Migration - How to plan and what need to consider?

The Zoning Migration within SAN Fabric can import complete zone set information and aliases without any effect on existing SAN fabrics, simplifying SAN migration between different vendors. Such as Cisco, Brocade and McData.

Import of zone set actually save up a lot time for the migration work but there is some preparation need to be done before the migration. Prepare of the script of zone set, check the interopmode for future fabric expansion (In case other brand of switch will be add in the fabric is required)

Before you begin, save the current Production Fabric from all SAN Switches.

Example of Import Zone set. Export the zoning info from the Old switches and prepare the script and import to the new SAN. One important reminder, make sure you have configured the Interoperation mode “Interopmode” before you import the zone set. As the change of Interopmode setting will reset the zoning config. Make sure all Switch that going to merge / ISL are in the same or Compatible Interop-Mode.

Before ISL / merge the switches, you need to make sure all Switch in the fabric have a unique DID. You need to determine the principal switch in the fabric. This is to ensure that you have a proper fabric management for future expansion.

Migration between Brocade Switches is very Simple and easy. Below is the script that I prepared before I import to the new Brocade switch.

Create Zone

zonecreate "USUNIXSAN_HBA0_CX1234_SPA0","10:00:00:00:C9:2D:10:12;50:06:01:60:39:01:2D:xx"

zonecreate "USUNIXSAN_HBA0_CX1234_SPB0","10:00:00:00:C9:23:11:13;50:06:01:68:39:01:2D:xx"

zonecreate "SANDUEL_HBA0_CX1234_SPA0","10:00:00:00:C9:2A:10:17;50:06:01:60:39:01:2D:xx"

zonecreate "SANDUEL_HBA0_CX1234_SPB0","10:00:00:00:C9:23:12:3D;50:06:01:68:39:01:2D:xx"

zonecreate "WINAPPS2008_HBA0_CX1234_SPA0","10:00:00:00:C9:2D:10:12;50:06:01:60:39:01:2D:xx"

zonecreate "WINAPPS2008_HBA0_CX1234_SPB0","10:00:00:00:C9:23:11:13;50:06:01:68:39:01:2D:xx"

Config Create

cfgcreate "SANDUEL_FabricA", "USUNIXSAN_HBA0_CX1234_SPA0"

cfgadd "SANDUEL_FabricA", "USUNIXSAN_HBA0_CX1234_SPB0"

cfgadd "SANDUEL_FabricA", "SANDUEL_HBA0_CX1234_SPA0"

cfgadd "SANDUEL_FabricA", "SANDUEL_HBA0_CX1234_SPB0"

cfgadd "SANDUEL_FabricA", "WINAPPS2008_HBA0_CX1234_SPA0"

cfgadd "SANDUEL_FabricA", "WINAPPS2008_HBA0_CX1234_SPB0"

--------------------------------------------------------------------------------

enable the configure.

cfgenable “SANDUEL_FabricA”

EMC PowerPath basic commands

PowerPath is a multipathing software for Unix operating systems from EMC. If you have ever worked or you are going to work in an environment that includes EMC storage systems it is more than sure that Powerpath will be installed in the Unix hosts.

Following are some notes and tips I’ve been creating since the very first time I found Powerpath, of course this isn’t a full user guide but a sort of personal quick reference. I decide to put it here in the hope that it will be helpful to anyone and for my personal use.

Show powermt command version

[root@totoro] / # powermt version
EMC powermt for PowerPath (c) Version 5.1.0 (build 160)

Display PowerPath configuration.

[root@totoro] / # powermt display
Symmetrix logical device count=898
CLARiiON logical device count=0
Hitachi logical device count=0
Invista logical device count=0
HP xp logical device count=0
Ess logical device count=0
HP HSx logical device count=0
==============================================================================
----- Host Bus Adapters ---------  ------ I/O Paths -----  ------ Stats ------
###  HW Path                       Summary   Total   Dead  IO/Sec Q-IOs Errors
==============================================================================
 5 0/2/1/0.101.16.19.0           optimal      61      0       -     0      0
 6 0/2/1/0.101.16.19.1           optimal     102      0       -     0      0
 7 0/2/1/0.101.16.19.2           optimal      97      0       -     0      0
 8 0/2/1/0.101.16.19.3           optimal     113      0       -     0      0
 9 0/2/1/0.101.16.19.4           optimal      82      0       -     0      0
 11 0/2/1/0.101.43.19.0           optimal     128      0       -     0      0
 12 0/2/1/0.101.43.19.1           optimal      49      0       -     0      0
 13 0/2/1/0.101.43.19.2           optimal      57      0       -     0      0
 14 0/2/1/0.101.43.19.3           optimal      83      0       -     0      0
 15 0/2/1/0.101.43.19.4           optimal      74      0       -     0      0
 16 0/2/1/0.101.43.19.5           optimal      33      0       -     0      0
 17 0/2/1/0.101.43.19.6           optimal      19      0       -     0      0
 19 0/5/1/0.102.16.19.0           optimal      61      0       -     0      0
 20 0/5/1/0.102.16.19.1           optimal     102      0       -     0      0
 21 0/5/1/0.102.16.19.2           optimal      97      0       -     0      0
 22 0/5/1/0.102.16.19.3           optimal     113      0       -     0      0
 23 0/5/1/0.102.16.19.4           optimal      82      0       -     0      0
 25 0/5/1/0.102.43.19.0           optimal     128      0       -     0      0
 26 0/5/1/0.102.43.19.1           optimal      49      0       -     0      0
 27 0/5/1/0.102.43.19.2           optimal      57      0       -     0      0
 28 0/5/1/0.102.43.19.3           optimal      83      0       -     0      0
 29 0/5/1/0.102.43.19.4           optimal      74      0       -     0      0
 30 0/5/1/0.102.43.19.5           optimal      33      0       -     0      0
 31 0/5/1/0.102.43.19.6           optimal      19      0       -     0      0

[root@totoro] / #

Check for death paths and remove them.

[root@sheldon] / # powermt display
Symmetrix logical device count=34
CLARiiON logical device count=0
Hitachi logical device count=0
Invista logical device count=0
HP xp logical device count=0
Ess logical device count=0
HP HSx logical device count=0
==============================================================================
----- Host Bus Adapters ---------  ------ I/O Paths -----  ------ Stats ------
###  HW Path                       Summary   Total   Dead  IO/Sec Q-IOs Errors
==============================================================================
 17 UNKNOWN                       failed        1      1       -     0      0
 31 UNKNOWN                       failed        1      1       -     0      0
 37 1/0/14/1/0.109.85.19.0        optimal      32      0       -     0      0
 39 0/0/14/1/0.110.85.19.0        optimal      32      0       -     0      0

[root@sheldon] / # powermt check
Warning: Symmetrix device path c17t9d6 is currently dead.
Do you want to remove it (y/n/a/q)? y
Warning: Symmetrix device path c31t9d6 is currently dead.
Do you want to remove it (y/n/a/q)? y
[root@sheldon] / #

List all devices.

[root@totoro] / # powermt display dev=all

Remove all devices.

[root@totoro] / # powermt remove dev=all

Add a new disk in HP-UX, configure it and save the config:

After a rescan of the disks with ioscan and the creation of the device files with insf run the following command to add the new disk to PowerPath

[root@totoro] / # powermt config

Now display all the devices and look the for the Logical device ID of the disk.

[root@totoro] / # powermt display dev=all | more
...
...
Symmetrix ID=000287750035
Logical device ID=0004
state=alive; policy=SymmOpt; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
20 0/0/10/1/0.11.15.0.0.1.3 c20t1d3 SP A0 active alive 0 1
23 0/0/10/1/0.11.47.0.0.1.3 c23t1d3 SP B0 active alive 0 1
26 1/0/8/1/0.21.15.0.0.1.3 c26t1d3 SP A1 active alive 0 1
29 1/0/8/1/0.21.47.0.0.1.3 c29t1d3 SP B1 active alive 0 1
...
...

If everything went fine save the config.

[root@totoro] / # powermt save

And these are the most common tasks

Physical memory Status on Solaris server

How to check physical memory on solaris?

# /usr/sbin/prtconf | grep Memory

check Total physical memory:

# prtdiag -v | grep Memory

# prtconf | grep Memory

check Free physical Memory:

# top (if available)

# sar -r 5 10
Free Memory=freemen*8 (pagesize=8k)

# vmstat 5 10
Free Memory = free

—

For swap:

# swap -s
# swap -l

SAMPLE OUTPUT OF #prtdiag -v
System Configuration: Sun Microsystems sun4u Sun Fire V440
System clock frequency: 183 MHZ
Memory size: 16GB

==================================== CPUs ==================================
E$ CPU CPU Temperature
CPU Freq Size Implementation Mask Die Amb. Status
Location
— ——– ———- ——————— —– —- —- ——

0 1281 MHz 1MB SUNW,UltraSPARC-IIIi 2.4 – – online
1 1281 MHz 1MB SUNW,UltraSPARC-IIIi 2.4 – – online
2 1281 MHz 1MB SUNW,UltraSPARC-IIIi 2.4 – – online
3 1281 MHz 1MB SUNW,UltraSPARC-IIIi 2.4 – – online

IPMP and aggregate

Firstly do aggregates.

jumpserver@root #dladm show-link

e1000g0 type: non-vlan mtu: 1500 device: e1000g0

e1000g1 type: non-vlan mtu: 1500 device: e1000g1

e1000g2 type: non-vlan mtu: 1500 device: e1000g2

e1000g3 type: non-vlan mtu: 1500 device: e1000g3

e1000g4 type: non-vlan mtu: 1500 device: e1000g4

jumpserver@root #dladm show-linkprop

LINK PROPERTY VALUE DEFAULT POSSIBLE

e1000g0 zone – – –

e1000g1 zone – – –

e1000g2 zone – – –

e1000g3 zone – – –

e1000g4 zone – – –

jumpserver@root #dladm create-aggr -l passive -d e1000g1 -d e1000g2 1

jumpserver@root #dladm show-aggr

key: 1 (0×0001) policy: L4 address: 0:c:29:27:ca:17 (auto)

device address speed duplex link state

e1000g1 0:c:29:27:ca:17 1000 Mbps full up standby

e1000g2 80:5c:0:0:80:fe 1000 Mbps full up standby

jumpserver@root #dladm create-aggr -l passive -d e1000g3 -d e1000g4 2

jumpserver@root #dladm show-aggr

key: 1 (0×0001) policy: L4 address: 0:c:29:27:ca:17 (auto)

device address speed duplex link state

e1000g1 0:c:29:27:ca:17 1000 Mbps full up standby

e1000g2 80:5c:0:0:80:fe 1000 Mbps full up standby

key: 2 (0×0002) policy: L4 address: 0:c:29:27:ca:2b (auto)

device address speed duplex link state

e1000g3 0:c:29:27:ca:2b 1000 Mbps full up standby

e1000g4 80:5c:0:0:80:fe 1000 Mbps full up standby

jumpserver@root #dladm show-link

e1000g0 type: non-vlan mtu: 1500 device: e1000g0

e1000g1 type: non-vlan mtu: 1500 device: e1000g1

e1000g2 type: non-vlan mtu: 1500 device: e1000g2

e1000g3 type: non-vlan mtu: 1500 device: e1000g3

e1000g4 type: non-vlan mtu: 1500 device: e1000g4

aggr1 type: non-vlan mtu: 1500 aggregation: key 1

aggr2 type: non-vlan mtu: 1500 aggregation: key 2

jumpserver@root #dladm show-linkprop

LINK PROPERTY VALUE DEFAULT POSSIBLE

e1000g0 zone – – –

e1000g1 zone – – –

e1000g2 zone – – –

e1000g3 zone – – –

e1000g4 zone – – –

aggr1 zone – – –

aggr2 zone – – –

jumpserver@root #ifconfig aggr1 plumb up

jumpserver@root #ifconfig aggr2 plumb up

jumpserver@root #ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

inet 127.0.0.1 netmask ff000000

lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

zone zone00

inet 127.0.0.1 netmask ff000000

e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

inet 192.168.1.10 netmask ffffff00 broadcast 192.168.1.255

ether 0:c:29:27:ca:d

e1000g0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

zone zone00

inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255

aggr1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

inet 0.0.0.0 netmask ff000000

ether 0:c:29:27:ca:17

aggr2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4

inet 0.0.0.0 netmask ff000000

ether 0:c:29:27:ca:2b

jumpserver@root #dladm show-link -s

ipackets rbytes ierrors opackets obytes oerrors

e1000g0 759 69814 0 677 86520 0

e1000g1 0 0 0 0 0 0

e1000g2 0 0 0 0 0 0

e1000g3 0 0 0 0 0 0

e1000g4 0 0 0 0 0 0

aggr1 2 128 0 0 0 0

aggr2 2 128 0 0 0 0

jumpserver@root #dladm show-aggr -s

key: 1 ipackets rbytes opackets obytes %ipkts %opkts

Total 2 128 0 0

e1000g1 1 64 0 0 50.0 -

e1000g2 1 64 0 0 50.0 -

key: 2 ipackets rbytes opackets obytes %ipkts %opkts

Total 2 128 0 0

e1000g3 1 64 0 0 50.0 -

e1000g4 1 64 0 0 50.0 -

jumpserver@root #netstat -i

Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue

lo0 8232 loopback localhost 381 0 381 0 0 0

e1000g0 1500 jumpserver jumpserver 785 0 704 0 0 0

aggr1 1500 0.0.0.0 0.0.0.0 2 0 0 0 0 0

aggr2 1500 0.0.0.0 0.0.0.0 2 0 0 0 0 0

Now do IPMP on aggr1 and aggr2

jumpserver@root #cd /etc

jumpserver@root #cat > hostname.aggr1

extra_IP netmask + broadcast + group IPMP_GROUP0 up

jumpserver@root #cat > hostname.aggr2

group IPMP_GROUP0 up

jumpserver@root # vi /etc/hosts and add entry for

extra_IP and ipaddress

jumpserver@root #reboot

updating /platform/i86pc/boot_archive

jumpserver@root #zonecfg -z zone01 info

zonename: zone01

zonepath: /zone01

brand: native

autoboot: false

bootargs:

pool:

limitpriv:

scheduling-class:

ip-type: shared

net:

address: 192.168.1.4

physical: aggr1

defrouter not specified

attr:

name: wholerootzone

type: string

value: “Amit Lab whole Root Zone”

bash-3.00# zoneadm -z zone01 boot

bash-3.00# zoneadm list -cv

ID NAME STATUS PATH BRAND IP

0 global running / native shared 1 zone00 running /zone00 native shared 2 zone01 running /zone01 native sharedbas

bash-3.00# ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

inet 127.0.0.1 netmask ff000000

lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 ind ex 1

zone zone00

inet 127.0.0.1 netmask ff000000

lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 ind ex 1

zone zone01

inet 127.0.0.1 netmask ff000000

aggr1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

inet 192.168.1.5 netmask ffffff00 broadcast 192.168.1.255

groupname IPMP_GROUP0

ether 0:c:29:27:ca:17

aggr1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

zone zone01

inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255

aggr2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255

groupname IPMP_GROUP0

ether 0:c:29:27:ca:2b

e1000g0: flags=1000803<UP,BROADCAST,MULTICAST,IPv4> mtu 1500 index 4

inet 192.168.1.10 netmask ffffff00 broadcast 192.168.1.255

ether 0:c:29:27:ca:d

e1000g0:1: flags=1000803<UP,BROADCAST,MULTICAST,IPv4> mtu 1500 index 4

zone zone00

inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255

bash-3.00# if_mpadm -d aggr1

bash-3.00# ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

inet 127.0.0.1 netmask ff000000

lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 ind ex 1

zone zone00

inet 127.0.0.1 netmask ff000000

lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 ind ex 1

zone zone01

inet 127.0.0.1 netmask ff000000

aggr1: flags=89000842<BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER,OFFLINE> mtu 0 index 2

inet 0.0.0.0 netmask 0

groupname IPMP_GROUP0

ether 0:c:29:27:ca:17

aggr2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255

groupname IPMP_GROUP0

ether 0:c:29:27:ca:2b

aggr2:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

inet 192.168.1.5 netmask ffffff00 broadcast 192.168.1.255

aggr2:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

zone zone01

inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255

e1000g0: flags=1000803<UP,BROADCAST,MULTICAST,IPv4> mtu 1500 index 4

inet 192.168.1.10 netmask ffffff00 broadcast 192.168.1.255

ether 0:c:29:27:ca:d

e1000g0:1: flags=1000803<UP,BROADCAST,MULTICAST,IPv4> mtu 1500 index 4

zone zone00

inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255

bash-3.00# if_mpadm -r aggr1

bash-3.00# ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

inet 127.0.0.1 netmask ff000000

lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 ind ex 1

zone zone00

inet 127.0.0.1 netmask ff000000

lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 ind ex 1

zone zone01

inet 127.0.0.1 netmask ff000000

aggr1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

inet 192.168.1.5 netmask ffffff00 broadcast 192.168.1.255

groupname IPMP_GROUP0

ether 0:c:29:27:ca:17

aggr1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

zone zone01

inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255

aggr2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255

groupname IPMP_GROUP0

ether 0:c:29:27:ca:2b

e1000g0: flags=1000803<UP,BROADCAST,MULTICAST,IPv4> mtu 1500 index 4

inet 192.168.1.10 netmask ffffff00 broadcast 192.168.1.255

ether 0:c:29:27:ca:d

e1000g0:1: flags=1000803<UP,BROADCAST,MULTICAST,IPv4> mtu 1500 index 4

zone zone00

inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255

bash-3.00# ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

inet 127.0.0.1 netmask ff000000

lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 ind ex 1

zone zone00

inet 127.0.0.1 netmask ff000000

lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 ind ex 1

zone zone01

inet 127.0.0.1 netmask ff000000

aggr1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

inet 192.168.1.5 netmask ffffff00 broadcast 192.168.1.255

groupname IPMP_GROUP0

ether 0:c:29:27:ca:17

aggr1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

zone zone01

inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255

aggr2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255

groupname IPMP_GROUP0

ether 0:c:29:27:ca:2b

e1000g0: flags=1000803<UP,BROADCAST,MULTICAST,IPv4> mtu 1500 index 4

inet 192.168.1.10 netmask ffffff00 broadcast 192.168.1.255

ether 0:c:29:27:ca:d

e1000g0:1: flags=1000803<UP,BROADCAST,MULTICAST,IPv4> mtu 1500 index 4

zone zone00

inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255

bash-3.00#

Note:Remember for Aggregare (trunking to work) we need to make changes in cisco router also.

something like mentioned below , But i am not sure about it that what are exact set of commands

We choosed 520 on the Sun and 10520 on the Foundry. Our host is connected to port 2/11 and 2/12

en
conf t

interface ethernet 2/11 to 2/12
link-aggregate configure key 10520
link-aggregate active

show run

You should have something like this :

interface ethernet 2/11
port-name DRACO_Net-1
enable
no fdp enable
no cdp enable
sflow forwarding
link-aggregate configure key 10520
link-aggregate active
!
interface ethernet 2/12
port-name DRACO_Net-2
enable
no fdp enable
no cdp enable
sflow forwarding
link-aggregate configure key 10520
link-aggregate active

You can check the status of the ports using show interface brief :

(config)#sh interf brief e 2/11 to 2/12

Port Link L2 State Dupl Speed Trunk Tag Priori MAC Name
2/11 Up Forward Full 1G 67 Yes level0 000c.dbe0.cf00 DRACO_Net-1
2/12 Up Forward Full 1G 67 Yes level0 000c.dbe0.cf00 DRACO_Net-