Storage Admin Notes: How to configure FCIP on Brocade 7500′s

Over Brocade Silkworm 7500’s crossing an OC-12. This assumes you have prior knowledge of the Brocade Fabric OS.

When using the Silkworm 7500, you have 32 ports available, 16 physical FC ports, 2 GE ports, and 16 Virtual Ports

Definitions

Backbone Fabric – Silkworm 7500’s connected together via VE-Ports or VEX-Ports
Edge Fabric – Normal switched fabric connected to the backbone via E-Ports or EX-Ports
VE-Port – FCIP interconnected E-Port/ISL, i.e. fabrics will merge
VEX-Port – FCIP interconnected EX-Port, routing needed via lsan zoning to connect initiator to a target.
E-Port – Normal expansion connection between one switch to another.
EX-Port – A connection that connects 2 fabrics together without merging services
LSAN – A special zone that allows 2 fabrics interconnected via EX-Ports to communicate.

Link Generation

On the Backbone/Edge ensure that interopmode is disabled

Check to ensure you aren’t running in Mcdata Compatibility Mode

>>interopmode

On the Backbone/Edge ensure that secmode is disabled

Check to ensure Secure-Fabrics isn’t enabled

>>secmodeshow

On the Backbone ensure that msplatform DB is disabled

Check to ensure there is no external management to Brocade Fabrics

>>msplatshow

Enable the FC Routing, by default its disabled on the backbone

Ensure its disabled first

>>fosConfig –disable fcr

Set the Backbone Fabric ID

>>fcrconfigure

Ensure you use the same backbone id for all the routers in that backbone.
Enable the FC routing

>>fosconfig –enable fcr

Perform these steps on all Routers in the Backbone Fabric

Creating FCIP Connections

Step to enabling FCIP is to turn on your EX-Port from the backbone fabric (Routers) into the edge fabric (e.g. Tape SAN). On both sides of your WAN link.
- Pick the same port for each backbone fabric i.e. port-4 on Source Router and Destination Router, to keep things easy to manage, on both sides
- If plugging into the same switch types plug the router FC port into the same port on the other side of the EX-Port/Edge Fabric
- By default all ports on the routers are persistently disabled, ensuring no-one can just plug in and go.
- Here are the steps to configure the EX-Ports
  
  >>portcfgexport 4 -a 1 -f 30 (fcrap14c001)
  
  >>portcfgexport 4 –a 1 –f 31 (fcrap14c101)
  - This will configure port-4 to be an ex-port once its been enabled, with (-a 1, enabling), (-f 30, defines the fabric id)
- On the other side of the backbone, you simply flip the EX-port fabric id’s, other wise the backbone fabric router will not merge properly
  
  >>portcfgexport 4 –a 1 –f 31 (fcrrp21001)
  
  >>portcfgexport 4 –a 1 –f 30 (fcrrp21101)

Next you need to enable the port
- >>portcfgpersistentenable 4 on all routers, you have configured EX-ports on.
- This will enable the port, and assuming its plugged into the edge fabric it will synch up, and you have a routable environment from the FC-Side
- On the backbone you will see:
  
  EX-Port 10:00:00:05:1e:35:2c:9d "fcsap14c501" (fabric id = 30 )(Trunk master)
- On the edge fabrics you will see:
  
  E-Port 50:00:51:e3:81:2e:4e:1e "fcr_fd_160" (downstream)(Trunk master) enables a tunnel for ge0
- One thing to remember the main reason for configuration of an EX-Port on the backbone, is so the fabrics will not merge themselves, thus ensuring WAN link outages won’t cause multi-fabric reconfigurations.

IPSEC is supported on these routers, however this configuration uses a PVC, not a internet based VPN.
- You can configure only 1 tunnel per Ge port if IPSec is enabled, this configuration takes advantage of multiple tunnels over the same ports.
- As well Tape Pipelining and Fast-Write aren’t supported over a secure-tunnel, and we plan to use both of these items.

Next you need to ensure you enable the virtual ports on both routers
- Ge0 – 16 – 23 correspond to Physical 0-8
- Ge1 – 24 – 31 correspond to Physical 9-15

Go ahead and enable the virtual ports you need based on the number of tunnels you are running across the ge-ports.
- >>portcfgpersistentenable 16
  
  >>portcfgpersistentenable 24 enables a tunnel for ge1
  - Ensure this is done on both sides of the WAN, and both set of routers

Create the IP Interfaces for the ge-ports on each router
- LC-Router >>portcfg ipif ge0 create 10.255.2.3 255.255.255.0 2250
  - 2250 is the MTU size (2148, is FC Frame size)
- FC-Router >>portcfg ipif ge0 create 10.255.2.4 255.255.255.0 2250
  - In a flat subnet there is no need to configure a ip route, once the tunnel is up the route will be generated for you
  - If you wish to enable tunnels on ge1 then you need to perform the same procedure on those ports on each router.

Next are is to create the tunnels on both sides
- LC-Router >>portcfg fciptunnel ge0 create 0 10.255.2.4 10.255.2.3 512000 -c
  - Tunnel ID = 0
  - Dest IP = 10.255.2.4
  - Src IP = 10.255.2.3
  - comm_rate = 512000 (500 mbps) (What Networks has given us)
  - -c (compression enable)
  - -t (enables tapepiplining)
  - -f (enables fastwrite)
  **Note** HP Currently does not support the Fast Write option as it pertains to Continuous Access Journaling, if this is enabled you will not be able to create the needed pairs
- FC-Router >>portcfg fciptunnel ge0 create 0 10.255.2.3 10.255.2.4 512000 -c
  - You will need to ensure that you create tunnels on the other pais of routers as well.

Once this is complete the tunnels should come online, provided you have done everything correctly
Online VE-Port 10:00:00:05:1e:37:f2:fc "fcrrp21001" (upstream)
Creating LSAN zones for FC Routing Communications
At this point you are ready to create some LSAN zones and start moving some data On the Edge Fabrics Create an alias in the LC Fabric for the Target-Port on the FC Storage Platform aliCreate "Storage_1B_Alias","50:06:00:00:01:02:03:04" Create a zone with the remote alias and the local one zoneCreate "lsan_CAXP_Jrnl_Zone"," Storage_1B_Alias; XPAP14C001_1B_Alias" Add the new zone into the Defined zone schema cfgAdd "CFG_3"," lsan_CAXP_Jrnl_Zone" Add the new zone into the Effective zone schema cfgEnable "CFG_3" FC- Edge Fabric aliCreate "XPAP14C001_1B_Alias"," 50:06:0e:80:04:7e:69:01" zoneCreate "lsan_CAXP_Jrnl_Zone"," XPAP14C001_1B_Alias; XPRP21001_1B_Alias" Add the new zone into the Defined zone schema cfgAdd "FC_CFG4"," lsan_CAXP_Jrnl_Zone" Add the new zone into the Effective zone schema cfgEnable "Fc_CFG4" You will need to do the same process above for the other sides of the XP XPAP14C001_2B_Alias – XPRP21001_2B_Alias (Initiator – RCU Target) Since we are going both ways you will need to configure another lsan zone coming back the other way for the XPAP14C001_1D_Alias – XPRP21001_1D_Alias (RCU Target – Initiator) Must use LSAN or lsan in the beginning of any zone that will be routed. Useful Commands to monitoring and status > portcmd --ping ge0 -s 10.255.2.3 -d 10.255.2.4 Pinging 10.255.2.4 from ip interface 10.255.2.3 on 0/ge0 with 64 bytes of data Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64 Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64 Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64 Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64 Ping Statistics for 10.255.2.4: Packets: Sent = 4, Received = 4, Loss = 0 ( 0 percent loss) Min RTT = 12ms, Max RTT = 12ms Average = 12ms > portshow ipif ge0 Port: ge0 Interface IP Address NetMask MTU 0 10.255.2.3 255.255.255.0 2250 > portshow ipif ge1 Port: ge1 Interface IP Address NetMask MTU 0 10.255.2.5 255.255.255.0 2250 > portshow iproute ge0 Port: ge0 IP Address Mask Gateway Metric Flags 10.255.2.0 255.255.255.0 10.255.2.3 0 Interface > portshow iproute ge1 Port: ge1 IP Address Mask Gateway Metric Flags 10.255.2.0 255.255.255.0 10.255.2.5 0 Interface > portshow fciptunnel ge0 all Port: ge0 Tunnel ID 0 Remote IP Addr 10.255.2.4 Local IP Addr 10.255.2.3 Remote WWN Not Configured Local WWN 10:00:00:05:1e:38:12:e4 Compression on Fastwrite on Tape Pipelining on Committed Rate 636928 Kbps (0.636928 Gbps) SACK on Min Retransmit Time 100 Keepalive Timeout 10 Max Retransmissions 8 Status : Active Uptime 19 hours, 11 minutes, 2 seconds > portshow fciptunnel ge1 all Port: ge1 Tunnel ID 0 Remote IP Addr 10.255.2.6 Local IP Addr 10.255.2.5 Remote WWN Not Configured Local WWN 10:00:00:05:1e:38:12:e4 Compression on Fastwrite on Tape Pipelining on Committed Rate 636928 Kbps (0.636928 Gbps) SACK on Min Retransmit Time 100 Keepalive Timeout 80 Max Retransmissions 9 Status : Active Uptime 19 hours, 11 minutes, 13 seconds > portshow arp ge0 Port: ge0 IP Address Mac Address Flags 10.255.2.4 00:05:1e:37:f2:e0 Resolved > portshow arp ge1 Port: ge1 IP Address Mac Address Flags 10.255.2.6 00:05:1e:37:f2:e1 Resolved > portshow ge0 Eth Mac Address: 00.05.1e.38.12.c8 Port State: 1 Online Port Phys: 6 In_Sync Port Flags: 0x3 PRESENT ACTIVE Port Speed: 1G > portshow ge1 Eth Mac Address: 00.05.1e.38.12.c9 Port State: 1 Online Port Phys: 6 In_Sync Port Flags: 0x3 PRESENT ACTIVE Port Speed: 1G > portshow mode ge0 Port: ge0 Mode: FCIP > portshow mode ge1 Port: ge1 Mode: FCIP > fcrfabricshow FC Router WWN: 10:00:00:05:1e:38:12:e4, Dom ID: 50, Info: 10.236.190.51, "fcrap14c001" EX_Port FID Neighbor Switch Info (WWN, enet IP, name) 4 30 10:00:00:05:1e:35:2c:9d 10.236.189.121 "fcsap14c301" FC Router WWN: 10:00:00:05:1e:37:f2:fc, Dom ID: 51, Info: 10.12.23.194, "fcrrp21001" EX_Port FID Neighbor Switch Info (WWN, enet IP, name) 4 31 10:00:00:05:1e:03:11:e1 10.12.23.188 "fcsrp21001" > fcrresourceshow Daemon Limits: Max Allowed Currently Used LSAN Zones: 3000 0 LSAN Devices: 10000 0 Proxy Device Slots: 10000 0 WWN Pool Size Allocated

Storage Admin Notes

Saturday, January 1, 2011

How to configure FCIP on Brocade 7500′s

Creating FCIP Connections

No comments:

Post a Comment