In this post, I will outline the configuration for a VXLAN fabric using BGP EVPN and OSPF as the underlay. This is similar to my previous Arista project and VXLAN project.
The topology of this post will be four switches and six clients.
- Nexus 9000v version 9.3(1)
- OSPF will be running as point to point between the routed ports Eth1/1 and Eth1/2
- BGP will be between loopback0
- Loopback1 on the Spine switches is the multicast rendezvous point
- Loopback1 on the Leaf switches is for the NVE (VTEP)
For the configuration, I will go section by section for each device to get the full working config.
Interfaces
All physical interfaces that are part of the underlay fabric will have the MTU set to 9216 that will support jumbo frames for VXLAN. The ports will also be routed.
In this section, I am also configuring PIM sparse mode on the interfaces.
|
0 1 2 3 4 5 6 |
#######ALL####### interface Eth1/1 - 2 no switchport mtu 9216 no shut |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
#######Spine1 INTERFACES####### feature bgp feature nv overlay feature ospf feature pim nv overlay evpn interface loopback0 ip address 10.10.10.11/32 ip pim sparse-mode interface loopback1 ip address 1.1.1.1/32 ip pim sparse-mode interface Eth1/1 ip address 10.1.1.0/31 ip pim sparse-mode interface Eth1/2 ip address 10.1.1.2/31 ip pim sparse-mode |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
#######Spine2 INTERFACES####### feature bgp feature nv overlay feature ospf feature pim nv overlay evpn interface loopback0 ip address 10.10.10.12/32 ip pim sparse-mode interface loopback1 ip address 1.1.1.1/32 ip pim sparse-mode interface Eth1/1 ip address 10.1.2.0/31 ip pim sparse-mode interface Eth1/2 ip address 10.1.2.2/31 ip pim sparse-mode |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
#######Leaf1 INTERFACES####### feature ospf feature bgp feature pim feature interface-vlan feature nv overlay feature vn-segment-vlan-based nv overlay evpn interface loopback0 ip address 10.10.10.21/32 ip pim sparse-mode interface loopback1 ip address 2.2.2.1/32 ip pim sparse-mode interface Eth1/1 ip address 10.1.1.1/31 ip pim sparse-mode interface Eth1/2 ip address 10.1.2.1/31 ip pim sparse-mode |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
#######Leaf2 INTERFACES####### feature ospf feature bgp feature pim feature interface-vlan feature nv overlay feature vn-segment-vlan-based nv overlay evpn interface loopback0 ip address 10.10.10.22/32 ip pim sparse-mode interface loopback1 ip address 2.2.2.2/32 ip pim sparse-mode interface Eth1/1 ip address 10.1.1.3/31 ip pim sparse-mode no shut interface Eth1/2 ip address 10.1.2.3/31 ip pim sparse-mode no shut |
OSPF
OSPF will be used as the underlay protocol. All the loopbacks and physical Ethernet ports needs to be taking part in OSPF for all switches to heave learnt the complete fabric that will be used later for the overlay. The Ethernet ports are configured as point-to-point which reduces the amount of LSAs required.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
#######Spine1 OSPF####### router ospf UNDERLAY-NET router-id 10.10.10.11 interface loopback0 ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 interface loopback1 ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/1 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/2 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/3 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/4 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
#######Spine2 OSPF####### router ospf UNDERLAY-NET router-id 10.10.10.12 interface loopback0 ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 interface loopback1 ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/1 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/2 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/3 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/4 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
#######Leaf1 OSPF####### router ospf UNDERLAY-NET router-id 10.10.10.21 interface loopback0 ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 interface loopback1 ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/1 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/2 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/3 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/4 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
#######Leaf2 OSPF####### router ospf UNDERLAY-NET router-id 10.10.10.22 interface loopback0 ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 interface loopback1 ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/1 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/2 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/3 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 interface Ethernet1/4 ip ospf network point-to-point no ip ospf passive-interface ip router ospf UNDERLAY-NET area 0.0.0.0 |
PIM
PIM as always is for the multicast. Each spine has the rendezvous point Anycast IP of 1.1.1.1 for redundancy.
|
0 1 2 3 4 |
#######Spine1 PIM####### ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4 ip pim ssm range 232.0.0.0/8 |
|
0 1 2 3 4 |
#######Spine2 PIM####### ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4 ip pim ssm range 232.0.0.0/8 |
|
0 1 2 3 4 |
#######Leaf1 PIM####### ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4 ip pim ssm range 232.0.0.0/8 |
|
0 1 2 3 4 |
#######Leaf2 PIM####### ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4 ip pim ssm range 232.0.0.0/8 |
VLANs
These are the VLANs that will be used in the single tenant. The VLANs will be able to have traffic routed between them via VLAN 10, VNI 10000. This is also mapping the VLANs to the VNIs.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
#######Leaf1 VLANs####### vlan 1,10-15 vlan 10 vn-segment 10000 vlan 11 vn-segment 10011 vlan 12 vn-segment 10012 vlan 13 vn-segment 10013 vlan 14 vn-segment 10014 vlan 15 vn-segment 10015 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
#######Leaf2 VLANs####### vlan 1,10-15 vlan 10 vn-segment 10000 vlan 11 vn-segment 10011 vlan 12 vn-segment 10012 vlan 13 vn-segment 10013 vlan 14 vn-segment 10014 vlan 15 vn-segment 10015 |
VLAN Interfaces
These are the VLAN interfaces that contain the Anycast gateways and are mapped to the VRF for Tenant-1. VLAN 10 is here as a way to forward intervals traffic.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
#######Leaf1 VLAN INTERFACES####### fabric forwarding anycast-gateway-mac 1234.5678.9000 interface Vlan10 no shutdown vrf member Tenant-1 ip forward interface Vlan11 no shutdown vrf member Tenant-1 ip address 10.0.11.1/24 fabric forwarding mode anycast-gateway interface Vlan12 no shutdown vrf member Tenant-1 ip address 10.0.12.1/24 fabric forwarding mode anycast-gateway interface Vlan13 no shutdown vrf member Tenant-1 ip address 10.0.13.1/24 fabric forwarding mode anycast-gateway interface Vlan14 no shutdown vrf member Tenant-1 ip address 10.0.14.1/24 fabric forwarding mode anycast-gateway interface Vlan15 no shutdown vrf member Tenant-1 ip address 10.0.15.1/24 fabric forwarding mode anycast-gateway |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
#######Leaf2 VLAN INTERFACES####### fabric forwarding anycast-gateway-mac 1234.5678.9000 interface Vlan10 no shutdown vrf member Tenant-1 ip forward interface Vlan11 no shutdown vrf member Tenant-1 ip address 10.0.11.1/24 fabric forwarding mode anycast-gateway interface Vlan12 no shutdown vrf member Tenant-1 ip address 10.0.12.1/24 fabric forwarding mode anycast-gateway interface Vlan13 no shutdown vrf member Tenant-1 ip address 10.0.13.1/24 fabric forwarding mode anycast-gateway interface Vlan14 no shutdown vrf member Tenant-1 ip address 10.0.14.1/24 fabric forwarding mode anycast-gateway interface Vlan15 no shutdown vrf member Tenant-1 ip address 10.0.15.1/24 fabric forwarding mode anycast-gateway |
Tenant
This is the single tenant on the fabric. Tenants are the customers or separate business unit. The tenants should not communicate with one another. There can be a shared services tenant for things like; DHCP, DNS, NTP, etc. But generally tenants do not communicate and can contain overlapping network addresses.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
#######Leaf1 TENANT-1####### interface Vlan10 vrf member Tenant-1 interface Vlan11 vrf member Tenant-1 interface Vlan12 vrf member Tenant-1 interface Vlan13 vrf member Tenant-1 interface Vlan14 vrf member Tenant-1 interface Vlan15 vrf member Tenant-1 vrf context Tenant-1 vni 10000 rd auto address-family ipv4 unicast route-target both auto evpn |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
#######Leaf2 TENANT-1####### interface Vlan10 vrf member Tenant-1 interface Vlan11 vrf member Tenant-1 interface Vlan12 vrf member Tenant-1 interface Vlan13 vrf member Tenant-1 interface Vlan14 vrf member Tenant-1 interface Vlan15 vrf member Tenant-1 vrf context Tenant-1 vni 10000 rd auto address-family ipv4 unicast route-target both auto evpn |
BGP
This is the MP-BGP EVPN overlay. EVPN is used as the control plane for the VXLAN data plane. EVPN will distribute the IP/MAC addresses of the endpoint hosts between the two leaf switches. By default, VXLAN operates on a flood and learn model. This is an enhancement to that behaviour, reducing the flood and learn.
The spines are acting as a route reflector as this is forming iBGP adjacencies.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
#######Spine1 BGP####### feature bgp router bgp 65001 router-id 10.10.10.11 neighbor 10.10.10.21 remote-as 65001 update-source loopback0 address-family l2vpn evpn send-community send-community extended route-reflector-client neighbor 10.10.10.22 remote-as 65001 update-source loopback0 address-family l2vpn evpn send-community send-community extended route-reflector-client |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
#######Spine2 BGP####### feature bgp router bgp 65001 router-id 10.10.10.12 neighbor 10.10.10.21 remote-as 65001 update-source loopback0 address-family l2vpn evpn send-community send-community extended route-reflector-client neighbor 10.10.10.22 remote-as 65001 update-source loopback0 address-family l2vpn evpn send-community send-community extended route-reflector-client |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
#######Leaf1 BGP####### feature bgp router bgp 65001 router-id 10.10.10.21 neighbor 10.10.10.11 remote-as 65001 update-source loopback0 address-family l2vpn evpn send-community send-community extended neighbor 10.10.10.12 remote-as 65001 update-source loopback0 address-family l2vpn evpn send-community send-community extended vrf Tenant-1 address-family ipv4 unicast advertise l2vpn evpn evpn vni 10011 l2 rd auto route-target import auto route-target export auto vni 10012 l2 rd auto route-target import auto route-target export auto vni 10013 l2 rd auto route-target import auto route-target export auto vni 10014 l2 rd auto route-target import auto route-target export auto vni 10015 l2 rd auto route-target import auto route-target export auto vrf context Tenant-1 rd auto address-family ipv4 unicast route-target both auto evpn |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
#######Leaf2 BGP####### feature bgp router bgp 65001 router-id 10.10.10.22 neighbor 10.10.10.11 remote-as 65001 update-source loopback0 address-family l2vpn evpn send-community send-community extended neighbor 10.10.10.12 remote-as 65001 update-source loopback0 address-family l2vpn evpn send-community send-community extended vrf Tenant-1 address-family ipv4 unicast advertise l2vpn evpn evpn vni 10011 l2 rd auto route-target import auto route-target export auto vni 10012 l2 rd auto route-target import auto route-target export auto vni 10013 l2 rd auto route-target import auto route-target export auto vni 10014 l2 rd auto route-target import auto route-target export auto vni 10015 l2 rd auto route-target import auto route-target export auto vrf context Tenant-1 rd auto address-family ipv4 unicast route-target both auto evpn |
NVE Interface
This is the Network Virtual Endpoint, also known as the VTEP (Virtual Tunnel Endpoint). It is loopback1 in this topology that will form the VXLAN tunnel. The NVE is responsible for VXLAN encapsulation/decapsulation.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 |
#######Leaf1 NVE INTERFACE####### interface nve1 no shutdown host-reachability protocol bgp source-interface loopback1 member vni 10000 associate-vrf member vni 10011 mcast-group 239.0.0.11 member vni 10012 mcast-group 239.0.0.12 member vni 10013 mcast-group 239.0.0.13 member vni 10014 mcast-group 239.0.0.14 member vni 10015 mcast-group 239.0.0.15 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 |
#######Leaf2 NVE INTERFACE####### interface nve1 no shutdown host-reachability protocol bgp source-interface loopback1 member vni 10000 associate-vrf member vni 10011 mcast-group 239.0.0.11 member vni 10012 mcast-group 239.0.0.12 member vni 10013 mcast-group 239.0.0.13 member vni 10014 mcast-group 239.0.0.14 member vni 10015 mcast-group 239.0.0.15 |
Show Commands
As there are multiple stages to this configuration, there are lots of show commands. Some of them overlap by showing the same information.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
show int status sh ip ospf neighbors show ip route ospf-UNDERLAY show ip pim interface brief show ip pim neighbor show ip pim rp show vrf Tenant-1 detail show run vrf tenant-1 show interface nve 1 show nve vrf show nve vni show nve peers show vxlan show l2route evpn mac all show l2route evpn mac-ip all show bgp l2vpn evpn show bgp l2vpn evpn summary show system internal l2fwder mac |
OSPF
OSPF is configured as point to point networks between each physical interface. All Ethernet and loopback interfaces are taking part in OSPF.
Spine1
|
0 1 2 3 4 5 6 7 |
sh ip ospf neighbors OSPF Process ID UNDERLAY VRF default Total number of neighbors: 2 Neighbor ID Pri State Up Time Address Interface 10.10.10.21 1 FULL/BDR 18:04:18 10.1.1.1 Eth1/1 10.10.10.22 1 FULL/BDR 18:04:17 10.1.1.3 Eth1/2 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
show ip route ospf-UNDERLAY IP Route Table for VRF "default" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%<string>' in via output denotes VRF <string> 2.2.2.1/32, ubest/mbest: 1/0 *via 10.1.1.1, Eth1/1, [110/41], 18:04:31, ospf-UNDERLAY, intra 2.2.2.2/32, ubest/mbest: 1/0 *via 10.1.1.3, Eth1/2, [110/41], 18:04:31, ospf-UNDERLAY, intra 10.1.2.0/31, ubest/mbest: 1/0 *via 10.1.1.1, Eth1/1, [110/80], 18:04:31, ospf-UNDERLAY, intra 10.1.2.2/31, ubest/mbest: 1/0 *via 10.1.1.3, Eth1/2, [110/80], 18:04:31, ospf-UNDERLAY, intra 10.10.10.12/32, ubest/mbest: 2/0 *via 10.1.1.1, Eth1/1, [110/81], 18:04:25, ospf-UNDERLAY, intra *via 10.1.1.3, Eth1/2, [110/81], 18:04:29, ospf-UNDERLAY, intra 10.10.10.21/32, ubest/mbest: 1/0 *via 10.1.1.1, Eth1/1, [110/41], 18:04:31, ospf-UNDERLAY, intra 10.10.10.22/32, ubest/mbest: 1/0 *via 10.1.1.3, Eth1/2, [110/41], 18:04:31, ospf-UNDERLAY, intra |
Leaf1
|
0 1 2 3 4 5 6 7 |
sh ip ospf neighbors OSPF Process ID UNDERLAY VRF default Total number of neighbors: 2 Neighbor ID Pri State Up Time Address Interface 10.10.10.11 1 FULL/DR 18:05:26 10.1.1.0 Eth1/1 10.10.10.12 1 FULL/DR 18:05:21 10.1.2.0 Eth1/2 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
show ip route ospf-UNDERLAY IP Route Table for VRF "default" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%<string>' in via output denotes VRF <string> 1.1.1.1/32, ubest/mbest: 2/0 *via 10.1.1.0, Eth1/1, [110/41], 18:05:46, ospf-UNDERLAY, intra *via 10.1.2.0, Eth1/2, [110/41], 18:05:43, ospf-UNDERLAY, intra 2.2.2.2/32, ubest/mbest: 2/0 *via 10.1.1.0, Eth1/1, [110/81], 18:05:46, ospf-UNDERLAY, intra *via 10.1.2.0, Eth1/2, [110/81], 18:05:43, ospf-UNDERLAY, intra 10.1.1.2/31, ubest/mbest: 1/0 *via 10.1.1.0, Eth1/1, [110/80], 18:05:46, ospf-UNDERLAY, intra 10.1.2.2/31, ubest/mbest: 1/0 *via 10.1.2.0, Eth1/2, [110/80], 18:05:43, ospf-UNDERLAY, intra 10.10.10.11/32, ubest/mbest: 1/0 *via 10.1.1.0, Eth1/1, [110/41], 18:05:46, ospf-UNDERLAY, intra 10.10.10.12/32, ubest/mbest: 1/0 *via 10.1.2.0, Eth1/2, [110/41], 18:05:43, ospf-UNDERLAY, intra 10.10.10.22/32, ubest/mbest: 2/0 *via 10.1.1.0, Eth1/1, [110/81], 18:05:46, ospf-UNDERLAY, intra *via 10.1.2.0, Eth1/2, [110/81], 18:05:43, ospf-UNDERLAY, intra |
PIM
PIM is active on all interfaces on the fabric, and each switch will have two PIM neighbours.
Spine1
|
0 1 2 3 4 5 6 7 8 9 |
sh ip pim interface brief PIM Interface Status for VRF "default" Interface IP Address PIM DR Address Neighbor Border Count Interface Ethernet1/1 10.1.1.0 10.1.1.1 1 no Ethernet1/2 10.1.1.2 10.1.1.3 1 no loopback0 10.10.10.11 10.10.10.11 0 no loopback1 1.1.1.1 1.1.1.1 0 no |
|
0 1 2 3 4 5 6 7 |
sh ip pim neighbor PIM Neighbor Status for VRF "default" Neighbor Interface Uptime Expires DR Bidir- BFD ECMP Redirect Priority Capable State Capable 10.1.1.1 Ethernet1/1 15:05:36 00:01:38 1 yes n/a no 10.1.1.3 Ethernet1/2 15:05:37 00:01:43 1 yes n/a no |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
show ip pim rp PIM RP Status Information for VRF "default" BSR disabled Auto-RP disabled BSR RP Candidate policy: None BSR RP policy: None Auto-RP Announce policy: None Auto-RP Discovery policy: None Anycast-RP 1.1.1.1 members: 10.10.10.11* 10.10.10.12 RP: 1.1.1.1*, (0), uptime: 15:09:18 priority: 255, RP-source: (local), group ranges: 224.0.0.0/4 |
Leaf1
|
0 1 2 3 4 5 6 7 8 9 |
show ip pim interface brief PIM Interface Status for VRF "default" Interface IP Address PIM DR Address Neighbor Border Count Interface Ethernet1/1 10.1.1.1 10.1.1.1 1 no Ethernet1/2 10.1.2.1 10.1.2.1 1 no loopback0 10.10.10.21 10.10.10.21 0 no loopback1 2.2.2.1 2.2.2.1 0 no |
|
0 1 2 3 4 5 6 7 |
show ip pim neighbor PIM Neighbor Status for VRF "default" Neighbor Interface Uptime Expires DR Bidir- BFD ECMP Redirect Priority Capable State Capable 10.1.1.0 Ethernet1/1 18:01:44 00:01:40 1 yes n/a no 10.1.2.0 Ethernet1/2 18:01:42 00:01:35 1 yes n/a no |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
show ip pim rp PIM RP Status Information for VRF "default" BSR disabled Auto-RP disabled BSR RP Candidate policy: None BSR RP policy: None Auto-RP Announce policy: None Auto-RP Discovery policy: None RP: 1.1.1.1, (0), uptime: 18:01:52 priority: 255, RP-source: (local), group ranges: 224.0.0.0/4 |
Tenant
The tenant is the single business unit, organisation or customer that will reside in this single overlay. There can be multiple overlay networks for multiple tenants. All of them can have overlapping networks, but no communication between them.
The tenants will only be on the leaf switches.
Leaf1
|
0 1 2 3 4 5 6 7 8 9 |
show vrf Tenant-1 detail VRF-Name: Tenant-1, VRF-ID: 3, State: Up VPNID: unknown RD: 10.10.10.21:3 VNI: 10000 Max Routes: 0 Mid-Threshold: 0 Table-ID: 0x80000003, AF: IPv6, Fwd-ID: 0x80000003, State: Up Table-ID: 0x00000003, AF: IPv4, Fwd-ID: 0x00000003, State: Up |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
show run vrf tenant-1 interface Vlan10 vrf member Tenant-1 interface Vlan11 vrf member Tenant-1 interface Vlan12 vrf member Tenant-1 interface Vlan13 vrf member Tenant-1 interface Vlan14 vrf member Tenant-1 interface Vlan15 vrf member Tenant-1 vrf context Tenant-1 vni 10000 rd auto address-family ipv4 unicast route-target both auto evpn router bgp 65001 vrf Tenant-1 address-family ipv4 unicast advertise l2vpn evpn |
NVE
The NVE will only be on the leaf switches. It is the virtual interface that is responsible for the VXLAN encaps/decaps. The idea of BGP EVPN is that when the leaf switch that is connected to the endpoints learns of the client, it will forward that over to the other leaf switches, Leaf2 in this case.
|
0 1 2 3 4 5 6 7 8 9 10 11 |
show interface nve 1 nve1 is up admin state is up, Hardware: NVE MTU 9216 bytes Encapsulation VXLAN Auto-mdix is turned off RX ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes TX ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes |
|
0 1 2 3 4 5 |
show nve vrf VRF-Name VNI Interface Gateway-MAC ------------ ---------- --------- ----------------- Tenant-1 10000 nve1 0c02.30d4.9d07 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
show nve vni Codes: CP - Control Plane DP - Data Plane UC - Unconfigured SA - Suppress ARP SU - Suppress Unknown Unicast Xconn - Crossconnect MS-IR - Multisite Ingress Replication Interface VNI Multicast-group State Mode Type [BD/VRF] Flags --------- -------- ----------------- ----- ---- ------------------ ----- nve1 10000 n/a Up CP L3 [Tenant-1] nve1 10011 239.0.0.11 Up CP L2 [11] nve1 10012 239.0.0.12 Up CP L2 [12] nve1 10013 239.0.0.13 Up CP L2 [13] nve1 10014 239.0.0.14 Up CP L2 [14] nve1 10015 239.0.1.2 Up CP L2 [15] |
|
0 1 2 3 4 5 |
show nve peers Interface Peer-IP State LearnType Uptime Router-Mac --------- -------------------------------------- ----- --------- -------- ----------------- nve1 2.2.2.2 Up CP 18:12:45 0c02.3039.6b07 |
The output below shows the MAC addresses for the endpoint that have been learnt on the network. This will only work if traffic has been received by the leaf switch the endpoints are connected to.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
show l2route evpn mac all Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link (Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete (D):Del Pending (S):Stale (C):Clear, (Ps):Peer Sync (O):Re-Originated (Nho):NH-Override (Pf):Permanently-Frozen, (Orp): Orphan Topology Mac Address Prod Flags Seq No Next-Hops ----------- -------------- ------ ------------- ---------- --------------------------------------- 10 0c02.3039.6b07 VXLAN Rmac 0 2.2.2.2 11 0050.7966.6800 Local L, 0 Eth1/7 11 0050.7966.6801 Local L, 0 Eth1/6 11 0050.7966.6802 BGP SplRcv 0 2.2.2.2 14 0c02.300c.4c00 BGP SplRcv 0 2.2.2.2 15 0c02.3069.4500 BGP SplRcv 0 2.2.2.2 15 0c02.30ad.7d00 Local L, 0 Eth1/5 |
The next command is similar to the previous, but contains the IP and MAC of the endpoints.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 |
show l2route evpn mac-ip all Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link (Dup):Duplicate (Spl):Split (Rcv):Recv(D):Del Pending (S):Stale (C):Clear (Ps):Peer Sync (Ro):Re-Originated (Orp):Orphan Topology Mac Address Host IP Prod Flags Seq No Next-Hops ----------- -------------- --------------------------------------- ------ ---------- ---------- --------------------------------------- 11 0050.7966.6801 10.0.11.11 HMM L, 0 Local 11 0050.7966.6800 10.0.11.12 HMM L, 0 Local 11 0050.7966.6802 10.0.11.13 BGP -- 0 2.2.2.2 14 0c02.300c.4c00 10.0.14.11 BGP -- 0 2.2.2.2 15 0c02.30ad.7d00 10.0.15.11 HMM L, 0 Local 15 0c02.3069.4500 10.0.15.12 BGP -- 0 2.2.2.2 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
show bgp l2vpn evpn BGP routing table information for VRF default, address family L2VPN EVPN BGP table version is 2198, Local Router ID is 10.10.10.21 Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2 Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 10.10.10.21:32778 (L2VNI 10011) *>l[2]:[0]:[0]:[48]:[0050.7966.6800]:[0]:[0.0.0.0]/216 2.2.2.1 100 32768 i *>l[2]:[0]:[0]:[48]:[0050.7966.6801]:[0]:[0.0.0.0]/216 2.2.2.1 100 32768 i *>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[0]:[0.0.0.0]/216 2.2.2.2 100 0 i *>l[2]:[0]:[0]:[48]:[0050.7966.6800]:[32]:[10.0.11.12]/272 2.2.2.1 100 32768 i *>l[2]:[0]:[0]:[48]:[0050.7966.6801]:[32]:[10.0.11.11]/272 2.2.2.1 100 32768 i *>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[32]:[10.0.11.13]/272 2.2.2.2 100 0 i Route Distinguisher: 10.10.10.21:32781 (L2VNI 10014) *>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[0]:[0.0.0.0]/216 2.2.2.2 100 0 i *>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[32]:[10.0.14.11]/272 2.2.2.2 100 0 i Route Distinguisher: 10.10.10.21:32782 (L2VNI 10015) *>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[0]:[0.0.0.0]/216 2.2.2.2 100 0 i *>l[2]:[0]:[0]:[48]:[0c02.30ad.7d00]:[0]:[0.0.0.0]/216 2.2.2.1 100 32768 i *>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[32]:[10.0.15.12]/272 2.2.2.2 100 0 i *>l[2]:[0]:[0]:[48]:[0c02.30ad.7d00]:[32]:[10.0.15.11]/272 2.2.2.1 100 32768 i Route Distinguisher: 10.10.10.22:32778 *>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[0]:[0.0.0.0]/216 2.2.2.2 100 0 i *>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[32]:[10.0.11.13]/272 2.2.2.2 100 0 i Route Distinguisher: 10.10.10.22:32781 *>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[0]:[0.0.0.0]/216 2.2.2.2 100 0 i *>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[32]:[10.0.14.11]/272 2.2.2.2 100 0 i Route Distinguisher: 10.10.10.22:32782 *>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[0]:[0.0.0.0]/216 2.2.2.2 100 0 i *>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[32]:[10.0.15.12]/272 2.2.2.2 100 0 i Route Distinguisher: 10.10.10.21:3 (L3VNI 10000) *>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[32]:[10.0.11.13]/272 2.2.2.2 100 0 i *>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[32]:[10.0.14.11]/272 2.2.2.2 100 0 i *>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[32]:[10.0.15.12]/272 2.2.2.2 100 0 i |
BGP
BGP or iBGP to be precise is the overlay network. It is responsible for the leaf switches to be able to form the VXLAN tunnels between one another, which allows the endpoints learnt on each leaf switch to be shared.
|
0 1 2 3 4 5 6 7 8 9 10 11 |
show bgp l2vpn evpn summary BGP summary information for VRF default, address family L2VPN EVPN BGP router identifier 10.10.10.21, local AS number 65001 BGP table version is 2198, L2VPN EVPN config peers 1, capable peers 1 21 network entries and 21 paths using 3960 bytes of memory BGP attribute entries [19/3116], BGP AS path entries [0/0] BGP community entries [0/0], BGP clusterlist entries [1/4] Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 10.10.10.11 4 65001 1991 1329 2198 0 0 18:32:14 6 |
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
show system internal l2fwder mac Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link, (T) - True, (F) - False, C - ControlPlane MAC VLAN MAC Address Type age Secure NTFY Ports ---------+-----------------+--------+---------+------+----+------------------ * 15 0c02.3069.4500 static - F F nve-peer1 2.2.2.2 * 11 0050.7966.6800 dynamic 00:07:39 F F Eth1/7 G 11 0c02.30d4.9d07 static - F F sup-eth1(R) G 10 0c02.30d4.9d07 static - F F sup-eth1(R) G 13 0c02.30d4.9d07 static - F F sup-eth1(R) G 12 0c02.30d4.9d07 static - F F sup-eth1(R) G 15 0c02.30d4.9d07 static - F F sup-eth1(R) G 14 0c02.30d4.9d07 static - F F sup-eth1(R) * 11 0050.7966.6801 dynamic 00:07:51 F F Eth1/6 * 11 0050.7966.6802 static - F F nve-peer1 2.2.2.2 * 15 0c02.30ad.7d00 dynamic 00:01:54 F F Eth1/5 * 14 0c02.300c.4c00 static - F F nve-peer1 2.2.2.2 1 1 -12:34:56:78:90:00 - 1 |