Build a BGP ISP Lab with FRRouting - Part 2

In Part 1, we established a functional multi-AS BGP topology with route reflection and basic routing policies. This continuation dives into advanced BGP features essential for production ISP networks: AIGP for IGP metric preservation, BGP communities for route tagging and policy automation, and sophisticated route maps for granular traffic engineering.

Understanding BGP Best Path Selection

Before implementing advanced features, it's crucial to understand how BGP selects the best path when multiple routes exist for the same prefix. BGP evaluates paths in a specific order, stopping at the first tie-breaker that produces a unique result.

BGP Path Selection Algorithm

Step	Attribute	Preference	Description
1	Weight	Highest	Cisco-specific, local to router
2	Local Preference	Highest	Propagated within iBGP, influences outbound traffic
3	Locally Originated	Preferred	Routes originated locally (network/redistribute)
4	AS Path Length	Shortest	Fewer AS hops preferred
5	Origin	IGP < EGP < Incomplete	IGP (i) preferred over EGP (e) or Incomplete (?)
6	MED	Lowest	Multi-Exit Discriminator, influences inbound traffic
7	eBGP over iBGP	eBGP	External paths preferred over internal
8	IGP Metric	Lowest	Lowest IGP cost to next-hop
9	AIGP	Lowest	Accumulated IGP Metric (RFC 7311)
10	Router ID	Lowest	Lowest BGP router ID

Accumulated IGP Metric (AIGP)

AIGP is a BGP path attribute defined in RFC 7311 that allows IGP metrics to be preserved and accumulated across AS boundaries. This is particularly useful in scenarios where you need to make routing decisions based on IGP cost, even when routes traverse multiple autonomous systems.

When to Use AIGP

AIGP is valuable in several scenarios:

• Multi-AS Networks: When you need consistent path selection across AS boundaries based on IGP metrics
• Traffic Engineering: To prefer paths with lower cumulative IGP cost
• Service Provider Networks: For maintaining IGP-based routing decisions in BGP-only environments
• Confederation Scenarios: To preserve IGP metrics across confederation sub-ASes

1 Configure OSPF for IGP

First, we need an IGP running within AS 65001 to generate metrics. We'll configure OSPF on VM1, VM3, and VM4:

Enable OSPF Daemon

# On VM1, VM3, and VM4
sudo vim /etc/frr/daemons

# Enable ospfd
sudo sed -i 's/^ospfd=.*/ospfd=yes/' /etc/frr/daemons
sudo systemctl restart frr

Configure OSPF on VM1

# On VM1
sudo vtysh
configure terminal

router ospf
 network 192.168.139.0/24 area 0
 network 10.1.1.1/32 area 0
 passive-interface default
 no passive-interface enp0s3

end
write

Configure OSPF on VM3

# On VM3
sudo vtysh
configure terminal

router ospf
 network 192.168.139.0/24 area 0
 network 10.3.3.3/32 area 0
 passive-interface default
 no passive-interface enp0s3

end
write

Configure OSPF on VM4

# On VM4
sudo vtysh
configure terminal

router ospf
 network 192.168.139.0/24 area 0
 network 10.4.4.4/32 area 0
 passive-interface default
 no passive-interface enp0s3

end
write

2 Enable AIGP on BGP Sessions

Now we'll configure AIGP to be sent and received on BGP sessions. AIGP must be enabled on both sides of a BGP session to be effective.

Configure AIGP on VM1 (Route Reflector)

# On VM1
sudo vtysh
configure terminal

router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.225 aigp send cost-community 100
  neighbor 192.168.139.147 aigp send cost-community 100
  neighbor 192.168.139.145 aigp send cost-community 100
  neighbor 192.168.139.17 aigp send cost-community 100
 exit-address-family

end
write

Configure AIGP on VM3

# On VM3
sudo vtysh
configure terminal

router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.115 aigp send cost-community 100
 exit-address-family

end
write

Verify AIGP Configuration

# Check if AIGP is being sent
sudo vtysh -c "show ip bgp neighbors 192.168.139.225"

# View routes with AIGP attribute
sudo vtysh -c "show ip bgp 10.3.3.3/32"

BGP Communities

BGP communities are 32-bit values attached to routes that allow network operators to tag routes and apply policies based on these tags. Communities provide a flexible way to mark routes for filtering, prepending, local preference adjustments, and other policy actions.

Standard Community Format

Communities are typically represented as two 16-bit values: AS:VALUE. For example, 65001:100 means AS 65001, value 100. Well-known communities include:

Community	Numeric Value	Meaning
no-export	0xFFFFFF01	Do not advertise to eBGP peers
no-advertise	0xFFFFFF02	Do not advertise to any peer
local-as	0xFFFFFF03	Do not advertise outside local AS
internet	0xFFFFFF04	Advertise to internet community

1 Define Community Structure

Let's establish a community structure for our lab:

• 65001:100 - Customer routes (high local preference)
• 65001:200 - Peer routes (medium local preference)
• 65001:300 - Provider routes (low local preference)
• 65001:666 - Blackhole/do not advertise
• 65001:777 - Prepend once to all peers
• 65001:888 - Prepend twice to all peers

2 Set Communities on Inbound Routes

Configure Communities on VM1

We'll set communities based on the source AS of routes:

# On VM1
sudo vtysh
configure terminal

# Route map to tag customer routes
route-map SET_COMMUNITY_CUSTOMER permit 10
 set community 65001:100 additive

# Route map to tag provider routes
route-map SET_COMMUNITY_PROVIDER permit 10
 set community 65001:300 additive

# Apply route maps to neighbors
router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.17 route-map SET_COMMUNITY_CUSTOMER in
  neighbor 192.168.139.145 route-map SET_COMMUNITY_PROVIDER in
 exit-address-family

end
write
clear ip bgp 192.168.139.17 soft in
clear ip bgp 192.168.139.145 soft in

3 Use Communities for Local Preference

Now we'll use the communities to set local preference values:

# On VM1
sudo vtysh
configure terminal

# Match customer community and set high local preference
route-map SET_LP_CUSTOMER permit 10
 match community CUSTOMER_COMMUNITY
 set local-preference 200

# Match provider community and set low local preference
route-map SET_LP_PROVIDER permit 10
 match community PROVIDER_COMMUNITY
 set local-preference 50

# Define community lists
ip community-list standard CUSTOMER_COMMUNITY permit 65001:100
ip community-list standard PROVIDER_COMMUNITY permit 65001:300

# Apply route maps
router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.225 route-map SET_LP_CUSTOMER in
  neighbor 192.168.139.147 route-map SET_LP_CUSTOMER in
 exit-address-family

end
write
clear ip bgp 192.168.139.225 soft in
clear ip bgp 192.168.139.147 soft in

4 Verify Community Propagation

# View routes with communities
sudo vtysh -c "show ip bgp community"

# View specific community
sudo vtysh -c "show ip bgp community 65001:100"

# View route details with communities
sudo vtysh -c "show ip bgp 10.5.5.5/32"

Advanced Route Maps

Route maps provide powerful conditional logic for route manipulation. They consist of sequence numbers, match conditions, and set actions. Route maps are processed top-down, and the first match wins.

Route Map Components

• Sequence Number: Determines processing order
• Match Conditions: Criteria that must be met (prefix-list, community, AS-path, etc.)
• Set Actions: Modifications to apply (local-preference, MED, AS-path prepend, etc.)
• Permit/Deny: Whether to allow or block the route

1 Complex Route Map Example

Let's create a sophisticated route map that implements multiple policies:

Multi-Condition Route Map

# On VM1
sudo vtysh
configure terminal

# Create prefix lists
ip prefix-list CUSTOMER_ROUTES seq 10 permit 10.5.5.5/32
ip prefix-list PROVIDER_ROUTES seq 10 permit 10.2.2.2/32

# Create AS path access list
ip as-path access-list 10 permit ^65003$
ip as-path access-list 20 permit ^65002$

# Create community lists
ip community-list expanded BLACKHOLE permit 65001:666
ip community-list expanded PREPEND_ONCE permit 65001:777
ip community-list expanded PREPEND_TWICE permit 65001:888

# Complex route map for outbound to provider
route-map TO_PROVIDER deny 10
 match community BLACKHOLE

route-map TO_PROVIDER permit 20
 match as-path 10
 match community PREPEND_ONCE
 set as-path prepend 65001
 set community 65001:300 additive

route-map TO_PROVIDER permit 30
 match as-path 10
 match community PREPEND_TWICE
 set as-path prepend 65001 65001
 set community 65001:300 additive

route-map TO_PROVIDER permit 40
 match as-path 10
 set community 65001:300 additive

route-map TO_PROVIDER permit 50
 match as-path 20
 set local-preference 50
 set community 65001:300 additive

route-map TO_PROVIDER permit 60
 set community 65001:300 additive

# Apply to provider neighbor
router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.145 route-map TO_PROVIDER out
 exit-address-family

end
write
clear ip bgp 192.168.139.145 soft out

2 Route Map for Traffic Engineering

Create a route map that uses MED for traffic engineering:

# On VM1
sudo vtysh
configure terminal

# Route map to set MED based on source
route-map SET_MED_CUSTOMER permit 10
 match ip address prefix-list CUSTOMER_ROUTES
 set metric 100

route-map SET_MED_PROVIDER permit 10
 match ip address prefix-list PROVIDER_ROUTES
 set metric 200

# Apply to outbound
router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.145 route-map SET_MED_CUSTOMER out
  neighbor 192.168.139.17 route-map SET_MED_PROVIDER out
 exit-address-family

end
write
clear ip bgp 192.168.139.145 soft out
clear ip bgp 192.168.139.17 soft out

3 Conditional Community Setting

Set communities conditionally based on multiple criteria:

# On VM1
sudo vtysh
configure terminal

# Route map that sets communities based on AS path length
route-map SET_COMMUNITY_BY_ASPATH permit 10
 match as-path 10
 set community 65001:100 65001:777 additive

route-map SET_COMMUNITY_BY_ASPATH permit 20
 match ip address prefix-list CUSTOMER_ROUTES
 set community 65001:100 additive

route-map SET_COMMUNITY_BY_ASPATH permit 30
 set community 65001:300 additive

# Apply inbound
router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.17 route-map SET_COMMUNITY_BY_ASPATH in
 exit-address-family

end
write
clear ip bgp 192.168.139.17 soft in

BGP Route Dampening

Route dampening is a mechanism to reduce route flapping by penalizing routes that change state frequently. When a route flaps (goes up and down), it accumulates a penalty. Once the penalty exceeds a threshold, the route is suppressed.

1 Configure Route Dampening

# On VM1
sudo vtysh
configure terminal

router bgp 65001
 bgp dampening 15 750 2000 60

end
write

Dampening Parameters Explained

Parameter	Value	Description
Half-life	15	Time in minutes for penalty to decay by half
Reuse Threshold	750	Penalty value below which route is unsuppressed
Suppress Threshold	2000	Penalty value above which route is suppressed
Max Suppress Time	60	Maximum time in minutes a route can be suppressed

2 Verify Dampening

# View dampening information
sudo vtysh -c "show ip bgp dampening"

# View dampened routes
sudo vtysh -c "show ip bgp dampening dampened"

# View dampening parameters
sudo vtysh -c "show ip bgp dampening parameters"

Advanced Filtering Techniques

1 AS Path Filtering

Filter routes based on AS path patterns using regular expressions:

# On VM1
sudo vtysh
configure terminal

# AS path access lists
ip as-path access-list 10 permit ^65003$        # Routes originated in AS 65003
ip as-path access-list 20 permit _65002_       # Routes that transit AS 65002
ip as-path access-list 30 permit ^65003_65001$ # Routes from 65003 via 65001
ip as-path access-list 40 deny _65002_         # Deny routes transiting 65002
ip as-path access-list 40 permit .*            # Permit all others

# Apply AS path filter
router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.145 filter-list 40 in
 exit-address-family

end
write
clear ip bgp 192.168.139.145 soft in

2 Community-Based Filtering

# On VM1
sudo vtysh
configure terminal

# Community lists
ip community-list standard NO_EXPORT permit 65001:666
ip community-list expanded PEER_ROUTES permit 65001:200
ip community-list expanded CUSTOMER_ROUTES permit 65001:100

# Route map to filter based on communities
route-map FILTER_BY_COMMUNITY deny 10
 match community NO_EXPORT

route-map FILTER_BY_COMMUNITY permit 20
 match community CUSTOMER_ROUTES
 set local-preference 200

route-map FILTER_BY_COMMUNITY permit 30

# Apply
router bgp 65001
 address-family ipv4 unicast
  neighbor 192.168.139.145 route-map FILTER_BY_COMMUNITY in
 exit-address-family

end
write
clear ip bgp 192.168.139.145 soft in

Verification and Testing

1 Verify AIGP

# Check AIGP in route details
sudo vtysh -c "show ip bgp 10.3.3.3/32"

# View AIGP attribute
sudo vtysh -c "show ip bgp 10.3.3.3/32 json" | grep -i aigp

2 Verify Communities

# List all communities
sudo vtysh -c "show ip bgp community"

# View routes with specific community
sudo vtysh -c "show ip bgp community 65001:100"

# View community lists
sudo vtysh -c "show ip community-list"

3 Verify Route Maps

# View route map configuration
sudo vtysh -c "show route-map"

# View specific route map
sudo vtysh -c "show route-map TO_PROVIDER"

# Test route map matching
sudo vtysh -c "show ip bgp neighbor 192.168.139.145 advertised-routes"

4 Test Policy Changes

# From VM2, check received routes
sudo vtysh -c "show ip bgp"

# Check AS path prepending
sudo vtysh -c "show ip bgp 10.5.5.5/32"

# Check local preference
sudo vtysh -c "show ip bgp 10.2.2.2/32"

# Test connectivity
ping -c 4 10.5.5.5
traceroute 10.5.5.5

Troubleshooting Advanced BGP Features

Common Issues with AIGP

Common Issues with Communities

Route Map Debugging

# View route map processing
sudo vtysh -c "show route-map"

# Check what routes match a route map
sudo vtysh -c "show ip bgp neighbor 192.168.139.145 advertised-routes"

# View prefix lists
sudo vtysh -c "show ip prefix-list"

# View AS path access lists
sudo vtysh -c "show ip as-path-access-list"

# View community lists
sudo vtysh -c "show ip community-list"

Production Best Practices

Community Design

• Document your community structure clearly
• Use consistent numbering schemes (e.g., 100s for customers, 200s for peers)
• Reserve well-known communities appropriately
• Consider using extended communities for complex scenarios

Route Map Organization

• Use descriptive names for route maps
• Leave gaps in sequence numbers for future additions
• Always include a final permit statement for unmatched routes
• Test route maps in a lab before production deployment

AIGP Considerations

• Use AIGP only when IGP metrics are meaningful across AS boundaries
• Coordinate AIGP cost-community values with peers
• Monitor AIGP impact on path selection
• Document AIGP usage in network documentation

Conclusion

Next Steps

In Part 3, we'll dive into expert-level BGP topics including:

1. BGP route refresh capabilities for dynamic policy updates
2. BGP aggregate routes with AS-set for loop prevention
3. Remove private AS for clean AS path advertisement
4. BGP allowas-in for special routing scenarios
5. Prevent transit AS to avoid unwanted traffic forwarding
6. Backdoor routes for IGP preference over BGP
7. Unsuppress-map for selective route dampening control
8. Advanced route reflector configurations with clusters and hierarchies
9. MPLS Layer 3 VPN implementation using BGP as the control plane

These advanced BGP features form the foundation of modern ISP and enterprise network routing policies. Mastery of AIGP, communities, and route maps enables sophisticated traffic engineering and policy implementation that scales to production networks.