STP Timers

STP Timers:

Hello timer – Periodic BPDU after every 2 sec by default – tune-able between 1 – 10 seconds.

Forward Delay – Listening + Learning state, equals to 15 seconds by default – tune-able between 4 – 30 seconds

Max Age – Maximum period of time that passes before a bridge port saves a configuration BPDU info, 20 sec by default – tune-able between 6 – 40 seconds.

Message Age – not a fixed value, the length of time that has passed since the root bridge initially originated the BPDU. When Root bridge sends it, its value is 0 and increments after every bridge-hop. It signified “how far is that bridge from Root”.

When a new configuration BPDU is received which is equal to or better than the previous configuration on that port, all the BPDU info is stored.

The age time starts at the message age that is received in that configuration BPDU. IF this age timer reaches max age before another BPDU is received that refreshes the timer, the info is aged out for that port.

Diameter of the STP domain (dia) – Maximum number of bridges between any 2 points on topology (any two points of end hosts attachment). IEEE recommends 7 with default timers.

Bridge Transit Delay (transit delay) – time taken by the bridge between reception and transmission of the same BPDU. Local latency. IEEE says it should be maximum 1 seconds.

BPDU transmission delay (BPDU delay) – delay between the time that a BPDU is received on a port and the time the configuration BPDU is effectively transmitted to another port. IEEE recommended 1 second maximum.

Message age increment over-estimate (msg overestimate) – This is the increment that each bridge adds to the message age before forwarding the BPDU. Cisco switches add 1 second.

Lost message (lost_msg) – number of BPDU that can be lost as the frame traverses from one end to the other end of the switched network.

Transmit halt delay (tx_halt_delay) – maximum amount of time that is necessary for a bridge to effectively move a port into the blocking state after the determination that the port needs to be blocked. IEEE recommendation is 1 second.

Medium access delay – med_access_delay – time that is necessary for a device to to gain access to the media for initial transmission. It is also the time between CPU decision to send a frame and the moment when the frame effectively begins to leave the bridge. IEEE says 0.5 seconds maximum.

End to end BPDU propagation delay: ((lost_msg + 1)XHello) + (BPDU_delayX(dia – 1))

14 seconds.

Message age overestimate – age of BPDU since origination –

(Dia – 1)Xoverestimate per bridge = 6seconds

Maximum frame lifetime – maximum time a frame that was previously sent to the bridge network remains in the network before the frame reaches that destination. = (diaXtransmit_delay) + Med access delay = 7.5 ~ 8seconds.

Maximum transmission halt delay – time that is necessary in order to effectively block a port after the decision to block is made – 1 seconds (max transmission halt delay).

Max-Age – end to end BPDU propagation delay + message age overestimate = 14+6 = 20 s

Forward Delay – time from first bridge’s port enters listening state and stays there through the subsequent reconfiguration to when the last bridge in the topology hears of the change in the active topology. In addition, we need to count the same delay that we use to count max age

Time for last bridge to stop forwarding frame that are received on the previous topology (max trans halt delay), until the last frame that is forwarded on the previous topology disappears (max frame lifetime).

2Xforward delay = end to end bpdu prop delay + mssg age overestimate + max frame lifetime + max trans halt delay = 28.5/2 = 15 seconds.

The tune-able parameters are –

  • Hello
  • Max age
  • Forward delay
  • Diameter

The other parameters should be considered fixed and non-tune-able. The formula comes out to be be:

Max_age = end to end propagation delay + message age overestimate

= 4XHello + 2XDia – 2

Forward_delay = (4XHello + 3XDia – 0.5) / 2 = (4XHello + 3XDia)/2

With this calculation –

If hello is 2 seconds – max_age = 14sec and forward_delay = 10 seconds for a dia of 4

If hello is 1 second – max_age= 10 sec and forward delay=8 seconds, for a dia of 4

Decrease the Hello timer to 1 second –

Easiest and surest way to reduce the timers value.

But, doubles the repletion and increases load on switches. Several switches and trunks will definitely add more load to the CPU.

Calculate the Dia –

Maximum switched hops from one end to the other end of the LAN.

Change the STP timers –

The timers changed on the root bridge is only required.

For redundancy, we should also configure them on the secondary/backup root bridge.

Posted in Uncategorized | Leave a comment

STP Topology Change

STP Topology Changes:

  • TC is normal in STP but too many of them can have ill impact on:
    • Applications response time
    • CPU Spikes
    • LAN meltdown
    • Instabilities in LAN
  • Default ageing of MAC address table is 5 mins (300 seconds), if a host is silent for 5 mins, its MAC address table entry will be flushed after that.

What problem does the topology changes solves – here is an example –


The topology above has hosts H1 and H2 connected to the switched network of B1, B2, B3 and B4. After the STP convergence it was found that the link between B1 and B4 is not going to transfer any data as the port on B1 is ‘Blocking’ state. The switched path is B=> B2=> B3=> B4=> H2… Due to the MAC address learning the H1’s MAC address is being learnt by the respective ports on B1, B2, B3 and B4 – represented by blue dots. In the event of the link failure between B2 and B3, the previously ‘Blocking’ port on B1 will turn to ‘Listening and Learning’ in less than one minute but the MAC address table still has the old entry. Maximum after 5 mins the old entry will be flushed away and then the H1 and H2 can re-start communicating, even though the STP was already converged. The toplogy Change Notifications solve this problem.

How does it solve:

  • The topology change occurs when:
    • The port which was forwarding is going down (say blocking state)
    • The port transitions to forwarding and the bridge has designated port.
  • The bridge where the outage occurred notifies the root bridge of the STP.
  • The root bridge broadcasts the info to the rest of the bridge network.


When a bridge need to signal a topology change, it send the TCN BPDU to the root port and the same happens at the upstream bridge also, until the TCN reaches the root bridge.

This bridge which starts the TCN does not stop sending the BPDU every Hello-Time interval (locally configured, not the one which is specified in configuration BPDU) until the other bridge responds with a Topology Change Acknowledgement (TCA). The same happens for other upstream switches.

  • The TCN is a very simple BPDU with no information.
  • The TCA BPDU is a normal configuration BPDU with ‘Topology Change Notification’ bit set.
  • Once the TCN reaches Root Bridge, it sends out the normal configuration BPDU broadcast to all bridges with TC bit set.
  • All other bridges relay this BPDU and broadcasts, including blocking ports.
  • As a result of TCN broadcast, the bridges reduce their ageing time to ‘forward delay’.

The TC bit is set by the root for a period of max_age + forward_delay = 20+15 = 35 seconds.

The portfast command:

  • Cisco proprietary command has two effects:
  • Ports that come up are put directly into forwarding state, skipping listening and learning. The STP still runs on those ports.
  • The switch never generates a TCN when such ports are going UP and Down.

Track the source of TCN:

  • Starting from the root bridge we can check the ‘show spanning-tree statistics’ output and get the bridge-ID of the switch which sent the TCN. This way we can check which switch and port is causing the TCN’s.

VLAN0200 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 8192, sysid 200, address 64a0.e745.cec1
Configured hello time 2, max age 20, forward delay 15
Current root has priority 4296, address 002a.6a5f.58bc
Root port is 802 (Ethernet6/34), cost of root path is 5
Topology change flag not set, detected flag not set
Number of topology changes 342 last change occurred 114:36:27 ago
         from Ethernet6/34
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0

Posted in Uncategorized | Leave a comment

FabricPath – Part 1, Introduction

FabricPath – Part I, Introduction   You must have an F Series Module in Cisco Nexus7k to run FabricPath. Allows L2 multipathing in the fabricPath network. Provides built-in loop prevention and mitigation with no need to use STP. Provides a … Continue reading

Gallery | 3 Comments

FEX – Fabric Extenders


  • FEX Integrates with its parent switch.
  • It forwards all traffic to its parent switch through 10G fabric uplinks.


  • Fabric Interface – A 10Gig ethernet uplink port designated for connection from FEX to its parent switch, it cannot be used for any other use.
    • On the parent switch enter the command ‘switchport mode fex-fabric’.
    • Port Channel Fabric interface – Fabric interfaces bundled together into port-channel. Virtual port, port-channel uplink port.
    • Host Interface –                Ethernet host interface for connection to a server or host system. Do not connect the switch or bride to these ports.
    • Port Channel Host interface – A port-channel host interface for connection to a server or host

Fabric Interface Features:

  • FEX fabric ports support static port-channel.
  • During the initial discovery and association process, SFP+ validation and digital optical monitoring (DOM) are performed as follows:
    • FEX performs the local check on the uplink SFP+ transceiver. If it fails, the LED flashes but the link is still allowed to come up.
    • The FEX local check is bypassed if it is running its backup image.
    • The parent switch performs SFP check again when the fabric interface is brought up. It keeps the fabric interface down if SFP validation fails.
    • Once an interface on the parent switch is configured for ‘fex-fabric’ mode, all other features which are not relevant are deactivated. If the interface is reconfigured to remove fex-fabric mode, the previous configurations are activated.

Host Interfaces:

  • Layer-3
  • Post NXOS 5.2, all host ports in FEX, with parent switch Nexus7K , are by default L3.
  • If you have just upgraded to 5.2, the existing FEX keeps the old config i.e. L2.
  • The host interfaces also support sub-interfaces, upto 32 subinterfaces.
  • Layer-2
  • All Fabric host ports run as spanning-tree edge ports with BPDU guard enabled and cannot be configured as network port.
  • Servers utilizing active/standby teaming, 802.3ad port-channel or other hosts based link link redundancy mechanism can be connected to FEX.
  • Any edge switch that leverages a link redundancy mechanism  not dependent on spanning tree such as Cisco FlexLink or vPC (BPDU filter enabled) can be connected to FEX host ports.

Host Interface Port Channel:


  • 8 ports in standard mode and 16 ports in LACP can be aggregated together for port-channel.
  • Port-channel resources are allocated when the port channel has one or more members.
  • All members from the FEC host interfaces and all members should be from FEX only, combination/mix of parent switch and FEX is not allowed.
  • Also supports sub-interfaces, up to 1000 sub-interfaces on a FEX host interface port channel.


  • 8 in standard and 16 in LACP mode of L2 port-channel.
  • All members should be FEC host ports and all members should be from FEX only, combination/mix of parent switch and FEX is not allowed.
  • Can be an access or trunk port.
  • vPC is allowed and 2 different parent switches are allowed in that case.


  • FEX does not support PVLANs.

Protocol Offload:

  • To reduce load on control plane of Cisco nexus 7k (or, whatever the parent is), Cisco NEXOS allows you to offload link level protocol processing to FEX CPU.
    • Link Layer Discovery Protocol and Datacenter Bridging Exchange (DCBX).
    • Cisco Discovery Protocol
    • Link Aggregation Control Protocol.


  • Uses 802.1p Class of Service (CoS) values to associate traffic with appropriate class.
  • Per-port QoS config is also allowed.
  • Host int support pause frames, which are implemented using 802.3x link-level flow control LLC.
  • By default, flow control send is ON and flow-control receive is OFF on all host interfaces.
  • Auto-negotiation is enabled on all ports.
  • Per-class flow control is set according to the QoS classes.


  • Supports a full range of ingress ACLs that are available on its parent Cisco Nexus series device.

IGMP Snooping:

  • Supported on all hosts interfaces.
  • Supports IGMPv2 and v3 snooping based only on destination multicast MAC address
  • It does not support snooping based on the source MAC address or on proxy reports.


  • Host interfaces can be configured as SPAN source port, but not destination port.
  • Only 1 SPAN session is supported for all the host interfaces on the same FEX.
  • Ingree-source, egress-source and both are allowed as options.
  • All IP multicast traffic on the VLANs that a FEX host interface belongs to is captured in the SPAN session. You cannot separate the traffic by IP multicast group membership.
  • If ingree monitoring and egress monitoring are configured on host interfaces on same FEX – you might see a packet twice – once as packet ingress on an interface with Rx configured and once as packet egress out an interface with Tx configured.


  • It is the function of the available fabric interfaces to active host interfaces, provides cost-effective scalability flexibility for ethernet environment.
  • 2248TP FEX
    • Has 4 10G fabric interfaces and 48 100/1000-Base-T interfaces.
    • 40 host interfaces and 4 fabric interfaces – no oversubscription if using host ports as 1G.
    • 48 host ports and 4 fabric ports – 1.2 – 1 oversubscription if using host ports as 1G.
    • 48 host ports and 1 fabric port – 4.8 – 1 oversubscription if using hosts ports as 1G.
    • No oversubscription if the host ports use 100Mbps instead if 1G.
    • 2248TP-E
      • All same as above.
      • 2232PP
        • 8 10G fabric ports, 32 10G Ethernet hosts ports
        • All host interfaces use all of the available fabric interfaces, static pinning is not supported, port-channel mode is supported only on fabric interfaces.
        • Max oversubscription is 4:1
        • 2232TM
          • 8 10G fabric ports and 32 1/10G ports
          • Static pinning is not supported, port-channel only on fabric ports
          • Max oversubscription ratio is 4:1
          • 2224TP
            • 2 10G fabric ports, 24 100/1000M-Base-T hosts interfaces.
            • Max oversubscription 1.2:1

Management Model:

  • Managed by the parent switch on fabric ports.
  • After discovery if the FEX has been correctly associated with the parent switch, the following operation occur:
    • The switch checks the software image compatibility and upgrade the FEX if necessary.
    • The switch and the FEX establish an in-band IP connectivity with each other – the switch assigns an IP address in the range of loopback address to FEX to avoid any IP conflict with the existing ranges of IP’s.
    • The switch pushes the config data to FEX and it does not store any config locally.
    • The FEX updates the switch with its operational status – all FEX info is displayed using the switch commands for monitoring and troubleshooting.

Forwarding Model:

  • FEX does not perform any local switching or forwarding. All traffic is sent to the parent switch that provides central forwarding and policy enforcement, including host-to-host communication on same FEX, 2 host ports.
  • BPDU guard cannot be disabled on FEX host ports.
  • The FEX supports egress multicast replication from the network to the host. Packets sent from the parent switch for multicast addresses attached to the fabric extender are replicated by FEX ASICS and are then sent to corresponding hosts.

Port-Channel Fabric interface connection:

  • When you configure the FEX to use a port channel fabric interface connection to its parent switch. The switch load-balances the traffic from the hosts that are connected to the host interface using following LB criteria:
    • For L2 frame, source and destination MAC address.
    • For L3 packets, source and destination MAC addresses and IP addreses.
    • A fabric port that goes down in a port-channel fabric interface, will not cause any outage as such as the hosts’ traffic will be redistributed amongst rest of the members of the port-channel.

Port Numbering Convention:

  • Int ethernet chassis/slot/port
    • Chassis: Configured by administrator. Chassis ID is configured on a port-channel on the switch to identify FEX. Range – 101 – 199
    • Slot:  identifies the slot number of the FEX.
    • Port: Identifies the specific port number in the FEX.

FEX Image Management:

  • FEX Image is bundled in the system Image of the parent switch.
  • Image is automatically verified and upgraded/updated during the association process.
  • ‘install all’ is the command.

Guidelines and Limitations:

  • Fabric Extender feature set is to be enabled in default VDC, before anything else. Only after that the FEX may be associated in any other VDC.
  • All fabric and hosts ports should be in same VDC. They cannot split among VDC’s..
  • The FEX must be connected to the N7K parent switch on –
    • 32 port 10G M1 module (N7K-M132XP-12),
    • 32 port 10G M1 module (N7K-M132XP-12L),
    • an M2 or
    • F2 module.
    • FEX feature set may cause the standby SUP to reload if it is already unstable i.e. may be service failure or powering up etc. ‘sho module’ gives an idea about the standby SUP’s status.
    • Queuing capabilities on the FEX host interfaces is limited.
    • The router connected to an L2 or L3 port cannot participate in routing protocol adjacency.
    • The FEX can not be used because when the congestion occurs, the control plane traffic is not prioritized.
    • This limitation also applies to ASA, ACE etc.
    • Static routes are supported.
    • Only following FEX devices are supported by F2 modules:
      • 2248TP
      • 2248TPE
      • 2232TP
      • 2232PP
      • 2232TM
      • 2224TP
      • Each port in the ASIC has an index. Allow only ports with similar indices across ASICs to be added to a port-channel.
        • For example – if port-1 has an index of 1 and port-2 has an index of 2 –
          • Port-1 of ASIC-1 and port-1 of ASIC-2 – supported.
          • Port-1 of ASIC-1 and port-2 of ASIC-2 – not supported.


  • Before proceeding with any configurations, please check “address reserved” string is disabled in the command ‘sho hardware ip verify’. If it is enabled, please disable it by ‘no hardware ip verify address reserved’

In default VDC:

Conf t

Install feature-set fex


Enabling feature-set at any VDC:

Switchto VDC-2

Conf t

Feature-set fex


Disallowing FEX Feature set:


Conf t

Vdc VDC-2

No allow feature-set fex



Associating a FEX with fabric ports:


Conf t

Int port-channel 20


Switchport mode fex-fabric

Fex associate 101

Int range ethernet 1/1 – 3

Channel-group 20


No shut



  • If you try to associate a physical port with a FEX associate number before that port is joined to the port-channel, that physical port moves to err-disable state – this does not apply when the config is done prior to cablin. So, always configure the port-channel for the FEX associate number (not physical int).

Sho int port-channel 20 fex-intf




Disassociating a FEX from all Interfaces:


Conf t

Int port-channel 20

No fex associate


Associating a Fabric Extender FEX to an F2 module:


Vdc switch

Limit-resource module-type f2

Int ethernet 1/1

Allocate interface ethernet 1

Switchport mode fex-fabric

Fex associate 101

Channel-group 20

No shut

Int po20

No shut


Configuring FEX Global Features:


Conf t

Fex 101

Description FEX to Application Servers

Type N2248T

Serial abcdefgh


Configuring FEX with L3 host interface:

L3 port, physical port:


Conf t

Int ethernet 101/1/1

No switchport

Ip address

Mtu 9000

No shut


L3 port, subinterface:


Conf t

Int ethernet 101/1/1.21

Ip address

Encapsulation dot1q 21

Mtu 850

No shut


L3 port, host port-channel:


Conf t

Int ethernet 101/1/2-3

No shut

Channel-group 2

No switchport


L3 port, host port channel sub-interface:


Conf t

Int ethernet 101/1/2-3

No shut

No switchport

Channel-group 2

Int port-channel 2.12

Ip address

Encapsulation dot1q 12

Mtu 1700

No shut


Configuring a host interface in a vPC topology connected to 2 FEX:


Conf t

Feature lacp

Int ethernet 101/1/1-4

Channel-group 12 mode active

No shut


Int port-channel 12

No shut


Switchport mode trunk switchport trunk allowed vlan 10, 20

Vpc 10


Conf t

Feature lacp

Int ethernet 101/1/1-4

Channel-group 12 mode active

No shut


Int port-channel 12

No shut


Switchport mode trunk

Switch trunk allowed vlan 10, 20

Vpc 10


Dual Homing of a server to a FEX using FabricPath:

  • Enable FabricPath on each switches.
  • Configure the interfaces which you want to designate as FabricPath.
  • Set the STP priority of the device to 8192 on all FabricPath L2 gateway devices.
  • Optionally, set the STP domain ID for each of the separate STP domains that are connected to the FabricPath network.
  • Optionally, configure the FEX switch ID.

Conf t

Feature fabricpath

Int ethernet 2/1

Switchport mode fabricpath


Spanning-tree vlan 10-13 priority 8192

Spanning-tree mst 1-5 priority 8192

Spanning-tree domain 5

If you are setting up an initial FEX VPC+ configuration on an F series module follow these steps:

  • In VPC domain config mode enable partial DF mode with the ‘FabricPath multicast load-balance’ command.
  • If disabled, enable TRILL style MAC address learning with the ‘mac address-table core-port-learning’ command.
  • In VPC config domain mode, configure the emulated switch ID with the ‘FabricPath switchID switch-ID’ command.
  • On each VPC/VPC+ peer link interfaces, in interface config mode, enter the ‘switchport mode fabricpath’ command.
  • On each VPC/VPC+ peer link port channel, enter ‘VPC peer-link’ command.
  • Configure VPC ID with ‘vpc vpcid’ command.

If you are changing an existing VPC configuration to a FEX VPC+ config on an F series, follow these steps:

  • In VPC domain configuration mode, enable partial DF mode by command ‘fabricpath multicast lload-balance’.
  • If disabled, enable TRILL style MAC address learning by the command ‘mac address-table core-port-learning’.
  • In the VPC domain config mode, configure the emulated switch ID  with command – ‘fabricpath switched switchID’.
Posted in Uncategorized | Tagged | Leave a comment

STP Extensions


STP Port Types:

  • Edge port – hosts, analogous to portFast, should not receive the STP BPDU.
  • Network port – network switches, bridges
  • Normal port – default

Misconfigs –

  • A Port connected to L2 switch/bridge is configured as Edge port – Loops…
  • A port connected to a host and configured as network port – Turns to Blocking automatically.

Bridge Assurance:

  • Only supported in RPVST+ and MST
  • Enabled by default and can be disabled globally.
  • Can only be enabled on point-to-point links.
  • Both ends of the point-to-point links should have the BridgeAssurance enabled. The connecting port is blocked in case one end does not has the BridgeAssurance enabled.
  • BPDU’s are sent out to all operational network ports, including alternate and backup ports for each hello time period – default 2 sec.
  • If there is no BPDU for a specified time, the port is moved to blocking and is not used for root port calculations. Once that port gets BPDU, it resumes to be normal STP transitions.

BPDU Guard:

  • Enabling BPDU guard shuts down that interface if a BPDU is received – regardless of the port types
  • When enabled globally, it is effective only on operational Edge ports – but it still shuts down the ports in case of BPDU reception on them.

BPDU Filtering:

  • Filtering is to stop a port to send/receive the BPDU’s .
  • When enabled globally, it is effective only on operational spanning tree edge ports.
  • If an operational spanning tree edge port receives a BPDU, it immediately returns to normal spanning tree port type and moves through the regular transitions.
  • Individual port config overrides the global configuration.
  • Enabling the BPDU filtering on Trunk port leads to the loops.

Loop Guard:

  • Prevents the loops which may be caused by unidirectional link failure on a point to point link.
  • When enabled globally, it affects only the point to point links.
  • Loopguard can be enabled on shared links too, per interface basis.
  • Loopguard may be used to determine that the root port and/or alternate/backup root ports are getting the BPDU’s or not. If the port which was previously receiving the BPDU’s and now not, loopguard puts the port into inconsistent (blocking) state until it starts to receive the BPDU’s again.
  • Disabling the Loopguard moves all loop-inconsistent ports into listening state.
  • Enabling LoopGuard on a Root device does not make any difference as such but it is beneficial in case iti becomes non-root device.

Root Guard:

  • When oyu enable root guard on a port, it does not allow that port to become Root port.
  • If, somehow, that designated port becomes Root because of topology changes, the Root guard puts it in Root-inconsistent status (blocked). Once it stops sending the better BPDU’s, it is again recovered back to normal automatically.
  • Root guard cannot be configured globally.

Guidelines and Limitations of STP Extensions:

  • STP network ports are for switches only, not for hosts.
  • STP Edge ports are for hosts, not for switches.
  • Bridge Assurance to be enabled and used throughout the LAN is recommended.
  • BPDU guard to be used on all Edge ports – recommended.
  • Enabling Loop guards only affects the point to point links.
  • Root Guard forces a port to be designated port – always.
  • You cannot enable Loop gurard and Root Guard on a port at the same time.
  • Spanning tree always choses the first port which comes online in a port-channel to send the BPDU’s – If that link becomes unidirectional, loop guard blocks the whole channel, even if the other links are working fine.
  • If you group together a set of ports that are already blocked by Loop Guard to form a channel, spanning tree loses all the state info for those ports and the new channel port may obtain the forwarding state with a designated role.
  • If a channel is blocked by Loop Guard and channel members go back to an individual link status, spanning tree loses all the state info. The individual ports may obtain the forwarding state with the designated role, even if one or more links of that port-channel are unidirectional.
  • Loop Guard should be enabled globally.

Configuring Spanning Tree Port type globally:

Conf t
Spanning-tree port type edge default .. or,
Spanning-tree port type network default

Configuring Spanning Tree port types per interface:

Conf t
Int Ethernet 3/22
Spanning-tree port type edge
Int Ethernet 3/23
Spanning-tree port type edge trunk
Int Ethernet 3/24
Spanning-tree port type network

Bridge Assurance runs automatically on network port types.

Configuring BPDU Guard:


Conf t
Spanning-tree port type edge bpduguard default


Conf t
Int Ethernet 5/21
Spanning-tree bpduguard enable

Int Ethernet 5/22
Spanning-tree bpduguard disable

Configuring BPDU Filtering:


Conf t
Spanning-tree port type edge bpdufilter default


Conf t
Int ethernet 6/44
Spanning-tree bpdufilter enable
Int ethernet 6/45
Spanning-tree bpdufilter disable

Configuring Loop Guard:


Conf t
Spanning-tree loopguard default

Conf t
Int ethernet 5/22
Spanning-tree guard loop

Int ethernet 5/23

Spanning-tree guard root


Configuring PVST Simulation –

MST interoperates with RPVST+, however to prevent an accidental connection to a device that does not run MST as the default mode of STP, you may want to disable this feature. If you disable the PVST simulation, the MST enables ports move to the blocking state once it detects once it detects it is connected to a RPVST+ enabled port. This port remains in inconsistent state until the port stops receiving the BPDU’s and then the port resumes the normal STP transition process.

This feature may be blocked globally or per-interface.


Conf t
No spanning-tree mst simulate pvst global

Int ethernet 5/22
Spanning-tree mst simulate pvst disable

Posted in Uncategorized | Tagged | Leave a comment

Configuring MST – NX-OS Perspective

Configuring MST in NX-OS:

  • MST – IEEE 802.1s
  • 2 or more VLANs to a single STP instance
  • MST instances with the same name, revision number and VLAN-to-Instance mapping is combinely called MST region – it is a single bridge to STP configuration outside the region.
  • Each MST instances uses Rapid PVST + for quick convergence
  • MAC address reduction is always ON and cannot be disabled
  • Backward compatibility with RPVST+ and original STP 802.1D

MST Regions:

  • A collection of one or more interconnected devices that have the same MST configuration is an MST Region.
  • A linked group of MST bridges with the same MST config.
    • The name of the region
    • Revision number
    • VLAN-to-MST instance assignment mapping
    • There is no limit to the number of MST regions in a network
    • Each device can support up to 65 MST instances (any number 1 – 4094), including instance-0, in a single MST region.
    • The system reserves instance-0 for a special instance, which is IST.
    • You can assign a VLAN to only one instance at a time.


  • Each device has only one MST BPDU per interface, and that BPDU carries an M-record for each MSTI on the device.
  • Only IST sends the BPDU for the MST region, all M-records are encapsulated in that one BPDU that the IST sends.
  • As MST BPDU carries the information for all the instances, the number of BPDU’s that need to be processed to support MST is significantly reduced compared with RPVST+.

MST Configuration Information:

  • Name – 32 char string, null padded and null terminated, identifying the MST Region.
  • Revision Number – Unsigned 16 bit number that identifies the revision of current MST config.
  • VLAN-to-MST instance mapping – 4096-element table that associate each of the potential 4094 VLANs supported in a VDC to a given instance with the first (0) and last element (4095) set to 0. The value of element number X represents the instance to which VLAN X is mapped.
  • When you change the VLAN-Instance mapping, the system re-converges MST.
  •   If any one of the above mentioned attributes are different the bridges consider it to be a different region.

IST, CIST and CST Overview:

  • IST is the spanning tree that runs in an MST region.
  • MST establishes and maintains the additional spanning tree within each MST region, these spanning trees are called multiple spanning tree instances MSTI.
  • Instance-0 – special instance for a region, known as IST.
  • IST always exists on all ports, this cannot be deleted.
  • By default all VLANs are assigned to IST-0 and all other MST instances are numbered 1-4094.
  •  IST is the only instance that sends and receives the BPDUs. All of the other MSTI information is contained in MST records (M-records), which are encapsulated within MST BPDU.
  • All MSTI’s within the same region share the same protocol timers, but each MSTI has its own topology parameters such as root bridge ID, root path cost etc.
  • MSTI is local to the region, does not cross the region boundary.
  • The CST connects the different regions – inter-region. It is the single STP instance for the entire bridged network and encompasses all MST regions and 802.1w and .1D instance.
  • CIST – Collection of IST in each MST region. It is the same as IST inside the region and same as CST outside the region.

Spanning Tree Operation within the MST region:

  • IST connects all the MST devices in a region.
  • CIST Regional Root – After the IST converges, the root of IST is called so.
  • If there is only one region in the network – CIST regional root is CIST root.
  • If the CIST root is outside the region, the protocol selects one of the MST devices at the boundary of the region as CIST regional root.
  • When an MST device comes alive, it sends the BPDU’s that identify itself as the CIST root and CIST regional roots (with path cost to CIST root and regional root as ZERO).
  • It does the same for all MSTI’s – “I am the root”.
  •   If the device receives a better BPDU i.e. lower path cost, lower priority etc, it surrenders its claim as CIST regional root.
  • All devices in MST region must agree on the same CIST regional root. Any two devices in the region will only synchronize their port roles for an MSTI if they converge to a common CIST regional root.

Spanning Tree operation between MST regions:

  • If you have multiple regions and/or 802.1w/802.1D STP instances within a network, MST establishes and maintains the CST, which include all MST regions and the 802.1w/.1D devices in the network.
  • The MSTI combines with the IST at the boundary of the region to become the CST.
  • IST connects all the MST devices in a region and appears as a sub-tree in the CIST that encompasses the entire switches domain.
  • The root of the sub-tree is the CIST regional root.
  • The MST region appears as a virtual device to adjacent STP devices and MST regions.
  • Only CST instance sends and receives the BPDU’s.
  • MSTI’s add their spanning tree information into the BPDU’s (as M-Records) to interact with neighboring devices within the same MST region and compute the final Spanning tree topology.
  • The spanning tree parameters related to the BPDU transmission (hello time, max-age, forward delay etc) are always configured on CST instance but affect all MSTI.
  • Parameters related to the STP topology (port priority, path cost, bridge priority etc) can be configured on both CST and MSTI’s.

MST Terminology:

  • As CIST is the only instance that spans the whole network, only its parameters require the external qualifiers and not the internal or regional qualifiers.
  • CIST root is the root bridge for CIST instance.
  • CIST external root path cost is the cost to the CIST root. This cost remains unchanged in an MST region. MST region looks like a single device to CIST.
  • CIST external root path cost is the root path cost calculated between these virtual devices and the devices which do not belong to any region.
  • If the CIST root is in the region, the regional root is the CIST root. Otherwise, the CIST regional root is the closest device to the CIST root in the region.
  • CIST regional root acts as the root for IST.
  • CIST internal root path cost is the cost to the CIST regional root in the region. This cost is only relevant to IST, instance-0.

Hop Count:

  • MST does not use the message-age, maximum-age information in the configuration BPDU to compute the STP topology inside the MST region.
  • The protocol uses the path cost to root and hop count mechanism similar to TTL in IP.
  • By using ‘spanning-tree mst max-hops’ global config command, you can configure the maximum hops inside the region and apply it to the IST and all MST instances in that region.
  • Hop count achieves the same results as the message-age info (triggers a reconfiguration).
  • The root bridge of the instance always sends a BPDU (or M-record) with a cost of ‘0’ and hop count set to maximum value. When a device gets this BPDU, it decrements the received hop count value by one and propagates this new value as the remaining hop counts in the BPDU that it generates. When the count gets to ‘zero’, the device discards the BPDU and ages the information held for the port.
  • The message-age and max-age info for 802.1w portion of the BPDU remain the same throughout the region (only on IST), and same values are propagated by the regional designated ports at the boundary.
  • Max-age timer – the number of seconds that a device waits without receiving the spanning configuration messages before attempting a re-configuration.

Boundary Ports:

  • A port that connects to a LAN, the designated bridge of which is either a bridge with a different MST configuration (and so, a separate MST region) or a RPVST+ / 802.1D STP bridge.
  • A designated port knows that it is on the boundary if it detects an STP bridge or receives an agreement proposal from an MST bridge with a different config or RPVST bridge.
  • Two ports that are internal to a region are allowed to share a segment with a port that belongs to a different region, creating a possibility of receiving both internal and external messages on a port.
  • At the boundary, the roles of MST ports do not matter; the system forces their state to be the same as the IST port state.
  • If the boundary flag is set for a port, the MST port-role selection process assigns a port role to the boundary and assigns the same state as the state of the IST port.
  • The IST port at the boundary can take up any port role except the backup port role.

Detecting unidirectional Link Failures – MST:

  • Currently this feature is not in IEEE MST standard.
  • It is based on dispute mechanism.
  • The software checks the consistency of the port role and state in received BPDUs to detect the unidirectional link failures that could cause bridging loops.
  • When a designated port detects a conflict, it keeps its role, but reverts to a discarding state because disrupting connectivity in case of inconsistency is preferable to opening a bridging loop.


Port Costs and priorities:

  • 10Mbps—2,000,000
  • 100Mbps—200,000
  • 1GigabitEthernet—20,000
  • 10GigabitEthernet—2,000
  • MST always uses the long path-cost calculation method.
  • The system uses port priorities to break ties among ports with the same cost. A lower number indicates a higher priority. The default port priority is 128. You can configure the priority to values between 0 and 224, in increments of 32.

Interoperability with IEE 802.1D:

  • In-built protocol migration feature to support the IEEE 802.1D bridges.
  • If an MST device gets the .1D BPDU (BPDU version 0), it sends only .1D BPDU on that port.
  • MST device detects a port to be boundary when it:
    • ..gets BPDU version 0 – for IEE 802.1D
    • ..gets BPDU version 3 – for another MST region
    • ..gets BPDU version 2 – for IEEE 802.1w.
    • MST can not automatically detects whether the device which was previously IEEE 802.1D, is actually still so or it has been configured for MST or else. Remedy is ‘clear spanning-tree detected-protocols’ command.
    • All RPVST+ switches and the STP switches can process the MST BPDU’s just as if they were 802.1w BPDU’s.
    • MST devices can send 2 types of BPDU’s:
      • BPDU version 0 configuration and topology change notification
      • BPDU version 3 for other MST regions (on boundary ports).
      • You can configure a port to proactively send prestandard MSTP messages.

MST Limitations and other Guidelines:

  • You cannot map VLANs 3968 – 4047 and 4094 to an MST instance. These VLANs are reserved for internal use by the device.
  • Max 65 MST instances on a device.
  • Max number of VLANs and ports 75000.
  • When working with private-VLANs, ensure that all secondary VLANs are mapped to the same MSTI as the primary VLAN. Enter the command ‘private-vlan synchronize’ command for it.
  • You can load-balance only within the MST region.
  • Ensure that trunks carry all VLANs mapped to an MSTI, or exclude all VLANs mapped to an MSTI.
  • Port cost of a port-channel is the aggregation of all the participating ports’ costs.
  • MST configuration sub-mode supports ‘pending regional config’ i.e. when you apply ‘abort’ all the changes made to the MST will be discarded. If you wish to commit the changes, end, exit or CTRL+Z are the options.
  • All boundary ports must be forwarding for load-balancing between RPVST+ and MST cloud or between a PVST+ and MST cloud. The CIST regional root of the MST cloud must be the root of the CST. If the MST cloud consists of multiple MST regions, one of the MST regions must contain the CST root and all other regions must have a better path to the root contained within the MST cloud than a path through RPVST+ or PVST+  cloud.
  • Deafult Maximum hop counts is 20 hops.

Configuring MST:

Spanning-tree mode mst

Entering the MST configuration sub-mode:

Spanning-tree mst configuration
Revision 5
Instance 1 vlan 10-20, 22

‘Show pending’ shows the pending database before committing.
When you change the VLAN-to-Instance mapping, the system re-converges MST.
A MSTI can be disabled.

No instance 1 vlan 10-20, 22

Mapping the Secondary VLAN to the same MSTI as Primary VLAN:

Switchto VDC-3
Conf t
Spanning-tree mode mst
Spanning-tree mst configuration
Version 5
Instance 2 vlan 10, 20
Private-vlan synchronize


Configuring the root bridge: – Primary and Secondary:

Config t
Spanning-tree mst 0  root primary diamet 15 hello 10  è diameter keyword is available only for IST-0
Spanning-tree mst 5 secondary

Configuring MST Switch Priority:

Config t
Spanning-tree mst 5 priority 61440

Configuring MST Port priority:

Conf t
Int ethernet5/20
Spanning-tree mst 5,6 port-priority 32

Configuring MST port Cost:

Conf t
Int Ethernet 5/21
Spanning-tree mst 7,8 cost 29837123

Configuring MST Hello timer, Forward-Delay timer and Max-age timer:

Conf t
Spanning-tree mst hello-time 5
Spanning-tree mst forward-delay 18
Spanning-tree mst max-age 25

Configures the hello time for all the MST Instances. The max-age time only applies to IST. Max-age is the time in seconds that a device waits without receiving spanning-tree configuration messages before attempting a reconfiguration.

Configuring the MST Maximum-hop count:

Conf t
Spanning-tree mst max-hops 22    … default is 20

This applies to all MSTI and IST.

Configuring an interface to Proactively Send pre-standard MSTP messages:


Conf t
Int Ethernet 5/22
Spanning-tree mst pre-standard

Interface with this config will always send the pre-standard MSTP messages.

Specifying the Link Type:


Conf t
Int Ethernet 5/33
Spanning-tree link-type point-to-point
Int Ethernet 5/34
Spanning-tree link-type shared

The systems learns about the link-type based on duplex settings/modes of the interface, by default.

Full Dup – point to Point

Half Dup – Shared – Falls back to IEEE 802.1D

Default link-type is ‘auto’ – decides based on duplex made.

Reinitializing the protocol for MST:

Clear spanning-tree detect-protocol interface Ethernet 5/44

Posted in Uncategorized | Leave a comment

Private VLAN – NX-OS Perspective

Private VLAN: NX-OS Perspective

WHAT is it –

  • Layer-2 isolation within a particular VLAN – partitions the L2 broadcast domain of a VLAN into sub-domains (Primary and Secondary VLANs).
  • The NX-OS system supports private VLAN promiscuous trunk ports and isolated trunk ports. Private-VLAN community ports cannot be trunk ports.
  • Enable the private VLAN feature before configuring it.
  • PVLAN – Primary VLANs and Secondary VLANs.
    • Primary VLANs – defines a broadcast domain with which the secondary VLANs are associated.
    • Secondary VLANs –
      • Isolated VLANs: Can just talk to Primary VLAN (promiscuous port). No other communication allowed.
      • Community VLANs: Can talk with all hosts in community VLAN and Primary VLAN promiscuous port.
      • Isolated VLANs and Community VLANs do not talk with each other.
      • Two Community VLANs do not talk with each other too.
      • Both Community and isolated private VLAN ports are labeled as PVLAN host ports.
      • The PVLANs can not be disabled if the device has any operational ports in a private VLAN mode.
      • The VLANs need to be created before associating them as Primary or secondary Private VLAN.
      • A Private VLAN domain can have multiple private VLAN pairs, one pair for each sub-domain. All VLAN pairs in a PVLAN domain share the same primary VLAN (there can be just one Primary VLAN in a PVLAN domain).
      • The Secondary VLANs differentiate one sub-domain from another.

WHAT does it solve:

  • Each VDC supports up to 4096 VLANS, so the number of customers that can be added are limited. With PVLANs, they may be isolated in the same VLAN. Solves scalability issues.
  • To enable IP routing each VLAN is assigned with a subnet address space or a block of addresses which can result in wasting the unused IP addresses and creating IP address Mgmt issues. Solves IP address management issues.

Types of PVLAN ports:

Promiscuous Ports –

  • It belongs to the primary VLAN and it can communicate with all interfaces (community and/or isolated) that belong to those secondary VLAN associated to the promiscuous port and associated with Primary VLAN.
  • You can have several promiscuous ports in primary VLAN. Each promiscuous port can have several secondary VLANs, or no secondary VLANs, associated to that port.
  • You can associate the secondary VLAN to more than one promiscuous ports as long as the Promiscuous port and secondary VLAN are within the same primary VLAN.
  • The secondary VLANs that are not associated with any promiscuous port/s do not communicate to the L3 interface.
  • The Primary VLAN becomes inactive after you remove all the mapped secondary VLANs to that primary VLAN.

Promiscuous Trunks-

  • On Cisco Nexus 7k, a promiscuous trunk port may be created to carry traffic for multiple primary VLANs.
  • You can configure maximum 16 pairs of primary and secondary VLANS (thus Private VLANs) to use the same promiscuous trunk port.
  • PVLAN trunk promiscuous ports also carry traffic for normal VLANs.

 Isolated Ports-

  • A host port that belongs to Isolated secondary VLAN
  • It can only communicate with respective associated promiscuous port.
  • There can be more than one ‘Isolated ports’ in Isolated VLAN, completely isolated from each other.

 Isolated or Secondary Trunk-

  • Isolated trunk port to carry multiple Isolated VLANs traffic but…
  • Each secondary VLAN on an Isolated trunk should be associated to a different primary VLAN.
  • Two secondary VLANs associated to same primary VLAN cannot be made trunks.
  • A maximum of 16 such Private VLAN pairs may be created on Isolated trunk ports.
  • Private VLAN Isolated trunk ports may carry traffic for the normal VLANs too.


Community Ports-

  • Host port that belongs to the Community Secondary VLAN for a Private VLAN.
  • They can communicate with other ports in the same community and with the promiscuous port – NOT with any other ports of other communities oro other Isolated ports.



  • The Isolated and community ports traffic may enter or leave the device through a trunk port.
  • The Private VLAN carries traffic from the promiscuous port to the secondary VLAN ports (Isolated and Community) and to other promiscuous ports.
  • Isolated secondary VLAN carries the unidirectional traffic from the downstream hosts to the upstream L3 gateway (promiscuous port).
  • Community Secondary VLANs carry the traffic to the promiscuous port and to other community ports in same community.
  • Once the traffic is out from the promiscuous port, there is no distinction among the Isolated and Community VLANs. It is again a normal traffic.
  • Promiscuous ports must be connected with a L3 gateway – Access-points, routers, firewalls, servers, workstation etc.
  • Only one gateway per primary private VLAN (mostly the default-gateway for the end hosts in Isolated and Community VLANs).
  • When you delete either the primary or the secondary VLANs, the associated ports become ‘Inactive’ and they regain their status once the related primary and secondary VLANs are re-created.
  • When you enter ‘no private-vlan’ command, the associated primary and secindart VLANs become just like normal VLANs.
  • The ’no vlan’  command for primary VLAN – all private VLAN association with that VLAN would be gone/lost forever.
  • The ‘no vlan’ command for any secondary VLAN – the private VLAN association for that VLAN is suspended and will return when you create the specified VLAN and configure as secondary VLAN.
  • In order to change the association between primary and secondary VLANs, we must first delete the old association and then put in new desired association.

Behavior of Broadcast traffic in PVLANs:

  • The broadcast traffic flows from all promiscuous ports to all ports in primary VLAN, including those ports which were not configured with private VLAN config.
  • Broadcast traffic from all Isolated ports is distributed only to the promiscuous ports which have been associated with the isolated port.
  • Broadcast traffic from a community ports is distributed to all community ports (same community only) and all promiscuous ports which are associated to the community port.

Private-VLAN and VLAN interfaces – SVI’s-

  • L3 devices communicate with private VLAN only through primary VLAN, not through secondary VLANs.
  • … so, configure VLAN SVI only for primary VLAN, not for secondary VLANs.
  • VLAN SVI’s for secondary VLANs are inactive/disabled if the VLAN is already configured as secondary VLAN.
  • If there is an SVI already configured for a VLAN and you are trying to configure that VLAN as secondary VLAN – Not allowed, it will not allow you to configure until you delete/shutdown that SVI.
  • When the primary VLAN is associated and mapped with a secondary VLAN, any configuration on the primary is propagated to the secondary VLANs also. For Ex. STP attributes, VACL, DHCP Snooping, IGMP etc.

Trunking Private VLANs across multiple devices-

  • The Private VLANs may be spanned across multiple devices by trunking the primary, Isolated and community VLANs among the devices which support PVLANs.
  • Configure the PVLAN on the devices even if they do not have any ports associated with it – to avoid other uses of private VLANs out of confusion/err.

High Availability of Private VLANs-

  • NX-OS supports high-availability for both stateless and statefull restarts for private VLANs.
  • For statefull restarts, the NX-OS supports maximum of 3 retries – if you try more than 3 times within 10 seconds of a restart, the software reloads the SUP.
  • The software may be upgraded/downgraded seamlessly, with respect to Private-VLANs.
  • If you configure PVLAN promiscuous or Isolated Trunk ports, you must un-configure those ports in order to downgrade the software.
  • PVLANs can not cross VDC’s – so no issues until it is in same VDC – Supports virtualization.

Guidelines for PVLANs-

  • Default VLAN (vlan-1) and internally allocated VLANs could not be used for primary or secondary VLANs in a PVLAN config.
  • A Primary VLAN can have multiple isolated and/or community VLANs associated with it BUT, the other way around is not correct – Isolated and Community VLANs can have just one primary VLANs association.
  • PVLANs provide isolation at L2 level, but the hosts can communicate at Layer-3.
  • The STP parameters of the primary VLAN are propagated to the secondary associated VLANs. You should manually check the STP configurations to ensure that the STP topologies for primary and secondary VLANs are exactly same – so that the VLANs can properly share the same forwarding database.
  • The Primary and all associated secondary VLANs must be in same MST instance.
  • There is a separate instance of STP for each primary and secondary VLANs in PVLANs – for a normal trunk port.
  • For non-trunking ports, STP is aware of only the primary VLAN for any private VLAN host port – STP does not run on secondary VLANs on a host port.
  • BPDU guard to be enables on host ports but NOT on promiscuous ports.
  • On a promiscuous trunk port, the Native VLAN must either be any other normal VLAN or the primary VLAN, but NOT the secondary VLAN.
  • On Isolated trunk port, the Native VLAN must be either any other normal VLAN or the secondary VLAN, but NOT the primary VLAN.
  • Different QoS configurations may be applied to the primary and secondary PVLANs.
  • To apply VACL to all Private VLAN traffic, first map the secondary VLANs to the primary private VLAN SVI and then configure the ACL on the primary VLAN SVI.
  • Before you configure a VLAN as secondary VLAN for Private VLAN, you must shut down the SVI corresponding to that VLAN.
  • To prevent inter-host communication in isolated private VLAN with the promiscuous port configure a Roll-Based ACL that disallows the hosts in that subnet from communicating with each other.
  • IGMP runs only on primary VLAN and uses the same config for secondary VLANs too.
  • Any IGMP request received by secondary VLAN is treated as if it were received by the primary VLAN.
  • PVLAN ports may be used a SPAN source ports.
  • VLAN based SPAN is also possible for all VLANs in private VLAN.
  • After you configure the association between Primary and Secondary VLANs:
    • The dynamic MAC addresses learnt at secondary VLAN are flushed down.
    • The static MAC addresses are inserted into Primary VLAN. Once the association the deleted, they revert back to the secondary VLAN.
    • Static MAC addresses will not be allowed to be created on Secondary VLAN.
    • Port Security is not supported in PVLANs.


Configuring Primary VLAN and its association to Secondary VLANs:

Conf t
Feature private-vlan
Vlan 100-102
Vlan 100
Private-vlan primary
Private-vlan association 101, 102
Vlan 101
Name SEC-101
Vlan 102
Name SEC-102

Configuring the Primary VLAN SVI for Secondary VLANs Mapping:

Int vlan 100
Private-vlan mapping  101-102

Configuring the L2 port for Private VLAN host port:

Int Eth 5/2
Switchport mode private-vlan host
Switchport private-vlan host association 100 101

Configuring the L2 port for Private VLAN Isolated trunk port:

Int gig5/3
Switchport mode private-vlan trunk secondary
Switchport private-vlan trunk native vlan 99
Switchport private-vlan trunk allowed vlan add 90-97
Switchport private-vlan association trunk 100 101
Switchport private-vlan association trunk 10 11

Configuring L2 port as Private VLAN Promiscuous port:

Conf t
Int gig5/4
Switchport mode private-vlan promiscuous
Switchport private-vlan mapping 100 102

Configuring L2 port as Private VLAN Promiscuous Trunk port:

Conf t
Int eth 5/5
Switchport mode private-vlan trunk promiscuous
Switchport private-vlan trunk native-vlan 111
Switchport private-vlan trunk allowed vlan add 112
Switchport private-vlan mapping trunk 100 add 103

Useful Commands:

Sho vlan private-vlan

Sho int private-vlan mapping

Sho int vlan primary-vlan-id private-vlan mapping

Sho int switchport

Posted in Uncategorized | Tagged , , | 1 Comment