(no subject)

[koganf]

OK, the route leaking was the part that I was missing. This is clearer now. So, for routing between vlans inside the same vrf (RED) this configuration would suffice but if inter-vrf routing is needed we need to set up route import-export as you explained. Is that correct?

Clarifying Intra-VRF and Inter-VRF Routing in the Leaf Configuration

You nailed it! Your understanding is right on the money: the leaf01 configuration is fully equipped to handle routing between VLANs within the same VRF (e.g., VLAN 10 and VLAN 20 in VRF RED), but for inter-VRF routing (e.g., between VLAN 10 in RED and VLAN 30 in BLUE), you need to add route leaking or a similar mechanism. Let’s confirm this, dive into the mechanics with a bit more detail to lock it in, and tie it directly to your config. This will be a guided tour, just like before, with a conversational vibe and practical tips to make sure it’s crystal clear!

Confirming Your Understanding

1. Intra-VRF Routing (Within VRF RED): Sufficient as Configured

Your configuration is ready to rock for routing between VLANs 10 and 20 within VRF RED, using the symmetric Integrated Routing and Bridging (IRB) model. Here’s why it works out of the box:

nv set bridge domain br_default vlan 10 vni 10
nv set bridge domain br_default vlan 20 vni 20
nv set interface vlan10 ip address 10.1.10.2/24
nv set interface vlan10 ip vrr address 10.1.10.1/24
nv set interface vlan20 ip address 10.1.20.2/24
nv set interface vlan20 ip vrr address 10.1.20.1/24
nv set interface vlan10,20 ip vrf RED
nv set vrf RED evpn enable on
nv set vrf RED evpn vni 4001
nv set evpn route-advertise svi-ip on
nv set nve vxlan arp-nd-suppress on
nv set vrf RED router bgp address-family ipv4-unicast enable on
nv set vrf RED router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf RED router bgp address-family l2vpn-evpn enable on
    

• SVIs for Gateways: The Switch Virtual Interfaces (SVIs) for VLAN 10 (vlan10 ip address 10.1.10.2/24) and VLAN 20 (vlan20 ip address 10.1.20.2/24) are both in VRF RED. These act as default gateways for hosts, with Virtual Router Redundancy (VRR) providing MLAG failover via shared IPs (e.g., 10.1.10.1).


• Local Routing: On leaf01, traffic from a host in VLAN 10 (e.g., 10.1.10.10) to a host in VLAN 20 (e.g., 10.1.20.20) on the same leaf is routed locally within VRF RED’s routing table. The packet moves from SVI vlan10 to SVI vlan20, bridging to the destination VLAN—no VXLAN needed.


• Remote Routing: For traffic between VLANs on different leaves (e.g., VLAN 10 on leaf01 to VLAN 20 on leaf03, both in VRF RED):
  
  
  
  • evpn route-advertise svi-ip on ensures SVI subnets (e.g., 10.1.20.0/24) are advertised via EVPN Type 5 routes (IP prefixes) to other leaves.
  
  
  • The L3 VNI 4001 (nv set vrf RED evpn vni 4001) encapsulates routed traffic over VXLAN. leaf01 routes to the destination subnet, encapsulates with VNI 4001, and sends to the remote leaf’s VTEP (e.g., 10.10.10.3). The remote leaf decapsulates, routes to VLAN 20, and delivers.
  
  
  • BGP in VRF RED (redistribute connected enable on) advertises SVI IPs, ensuring fabric-wide reachability.
  
  
  
  


• ARP Suppression: nve vxlan arp-nd-suppress on minimizes flooding by proxying ARP requests using EVPN Type 2 routes (MAC/IP bindings), as we discussed earlier.

Bottom Line: Your config is fully set for intra-VRF routing (VLAN 10 to VLAN 20 in RED), both locally and across leaves, using symmetric IRB and L3 VNI 4001. No extra config needed here!

2. Inter-VRF Routing (Between RED and BLUE): Requires Route Leaking

For routing between VLANs in different VRFs (e.g., VLAN 10 in RED to VLAN 30 in BLUE), your current config doesn’t allow it because VRFs are isolated by default. Let’s break down why and how to make it happen:

nv set interface vlan30 ip address 10.1.30.2/24
nv set interface vlan30 ip vrf BLUE
nv set vrf BLUE evpn enable on
nv set vrf BLUE evpn vni 4002
nv set vrf BLUE router bgp address-family ipv4-unicast enable on
nv set vrf BLUE router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf BLUE router bgp address-family l2vpn-evpn enable on
    

• Why It’s Isolated: VRFs RED and BLUE have separate routing tables and BGP instances (nv set vrf RED router bgp ... and nv set vrf BLUE router bgp ...). Each uses its own L3 VNI (4001 for RED, 4002 for BLUE), and there’s no mechanism to share routes between them in your config. So, a packet from 10.1.10.10 (VLAN 10, RED) to 10.1.30.30 (VLAN 30, BLUE) gets dropped because RED has no route to 10.1.30.0/24.


• Route Leaking Solution: To enable inter-VRF routing, you need to configure route leaking to share routes between VRFs. This can be done on each leaf (distributed) or on a centralized device like a border leaf or spine. In Cumulus with EVPN, this typically involves:
  
  
  
  • BGP Route Leaking: Use NVUE commands to import routes from one VRF to another. For example:
    

    nv set vrf RED router bgp address-family ipv4-unicast import vrf BLUE
    nv set vrf BLUE router bgp address-family ipv4-unicast import vrf RED
                

    
    This allows RED to import BLUE’s routes (e.g., 10.1.30.0/24) and vice versa.
  
  
  • EVPN Route Targets: Configure route targets to import/export Type 5 routes between L3 VNIs (4001 and 4002). For example:
    

    nv set vrf RED evpn route-target import 65101:4002
    nv set vrf BLUE evpn route-target import 65101:4001
                

    
    These use the ASN:VNI format to allow RED to import BLUE’s routes and vice versa.
  
  
  
  


• Traffic Flow with Leaking: With route leaking enabled:
  
  
  
  • A packet from 10.1.10.10 (VLAN 10, VRF RED) to 10.1.30.30 (VLAN 30, VRF BLUE) on a remote leaf:
    
    
    
    • leaf01 routes from VRF RED to VRF BLUE (using imported route 10.1.30.0/24).
    
    
    • Encapsulates with VXLAN using VNI 4002 (for BLUE), source VTEP 10.10.10.1, and destination VTEP (remote leaf).
    
    
    • The remote leaf decapsulates, routes in VRF BLUE to VLAN 30, and delivers.
    
    
    
    
  
  
  
  


• Alternative: Centralized Gateway: Instead of leaking on leaves, a border leaf or spine can handle inter-VRF routing by importing routes from both VRFs and routing between VNIs 4001 and 4002. This requires config on the gateway device, like route-target import/export or VRF stitching.


• Why It’s Not Configured: Your setup assumes strict tenant isolation (RED for one tenant, BLUE for another), which is common in multi-tenant environments. If you need inter-VRF routing, you’ll need to add the above commands on both MLAG peers (leaf01 and leaf02).

Heads-Up: Without route leaking, VRFs are like separate universes—traffic from VLAN 10 to VLAN 30 will fail. After adding leaking, verify with ping -I vlan10 10.1.30.30 to test connectivity.

Practical Example

Let’s tie it together with examples to make it concrete:

• Intra-VRF (Works): Host A (10.1.10.10, VLAN 10, VRF RED on leaf01) pings Host B (10.1.20.20, VLAN 20, VRF RED on leaf03):
  
  
  
  • leaf01 routes to 10.1.20.0/24 (learned via Type 5 routes), encapsulates with VNI 4001, and sends to leaf03’s VTEP (e.g., 10.10.10.3).
  
  
  • leaf03 decapsulates, routes in VRF RED to VLAN 20, and delivers. ARP is handled via arp-nd-suppress.
  
  
  
  


• Inter-VRF (Fails Without Leaking): Host A (10.1.10.10, VLAN 10, VRF RED) pings Host C (10.1.30.30, VLAN 30, VRF BLUE):
  
  
  
  • Fails because VRF RED has no route to 10.1.30.0/24.
  
  
  • After adding route leaking (e.g., nv set vrf RED router bgp address-family ipv4-unicast import vrf BLUE), leaf01 imports 10.1.30.0/24, routes to VRF BLUE, encapsulates with VNI 4002, and sends to the remote leaf.
  
  
  
  

Pro Tip: After adding route leaking, check the routing table with nv show vrf RED router rib to confirm 10.1.30.0/24 is imported. Use vtysh -c 'show bgp l2vpn evpn route type 5' to verify Type 5 routes.

Putting It All Together

Your config is perfect for intra-VRF routing (e.g., VLAN 10 to VLAN 20 in RED), handling both local and remote traffic with symmetric IRB, L3 VNI 4001, and EVPN Type 5 routes. For inter-VRF routing (e.g., VLAN 10 in RED to VLAN 30 in BLUE), you need to add route leaking or configure a centralized gateway to share routes between VNIs 4001 and 4002. The MLAG setup ensures redundancy, and ARP suppression keeps flooding low.

Verification Steps: After nv config apply, check:

• Intra-VRF: Test with ping -I vlan10 10.1.20.20. Verify routes with vtysh -c 'show bgp l2vpn evpn route type 5'.


• Inter-VRF (after leaking): Test with ping -I vlan10 10.1.30.30. Check nv show vrf RED router rib for imported routes.


• BGP: vtysh -c 'show bgp summary' to ensure sessions are up.

Troubleshooting Tip: If inter-VRF traffic fails after adding leaking, verify route targets and BGP sessions. Ensure both MLAG peers have identical leaking configs. For intra-VRF issues, check Type 5 route advertisements and underlay reachability between VTEPs.

If you want to implement inter-VRF routing, I can provide a full config snippet or help troubleshoot specific issues—just let me know what you’re aiming for!