Cisco IOS-fu #5 – BGP / Redundancy

Welcome back!  I missed you.

For the last year or so, we've been on a long-term plan to eliminate single points of failure (SPOF) in our network. I've blogged about several of them – I'll list a few posts below that you might enjoy reading.

One of the last pieces of the puzzle – Internet Redundancy – fell into place yesterday afternoon.

We now have a fully redundant BGP setup – with 2x identical burstable AT&T EaMIS 100meg Internet connections, in two locations 7 miles apart – and connected via Gigaman (Gigabit Metro Ethernet).

I thought it would be helpful to share our router configs and some of the internal HSRP setup as well.  These are scrubbed with only the relevant info – so don't bother looking for fun stuff 🙂

HSRP – Internal Layer 3 VLANs
So, since we have internal redundancy (and have for a while) I thought I'd share that setup.  The purpose and goal is to make sure that each affected VLAN has a "shared" gateway, so if one side dies, the other takes over.

Here's an example.  Our data vlan 3 is for some "typical" enduser clients.  The subnet is the standard 192.168.1.0/24 class C.  The "gateway" is 192.168.1.254.

Side A – Primary – "data vlan 3" – Cisco 6509

interface Vlan3
 ip address 192.168.1.252 255.255.255.0
 ip helper-address 10.5.1.71
 standby 3 ip 192.168.1.254
 standby 3 priority 200
 standby 3 preempt
end

Side B – Secondary – "data vlan 3" – Cisco 4507R

interface Vlan3
 ip address 192.168.1.253 255.255.255.0
 ip helper-address 10.5.1.71
 standby 3 ip 192.168.1.254
end

So, if you follow along – Side A has the .252 IP.  Side B has the .253 IP.  Both "share" the .254 IP.  Side A has the highest priority – and "preempts" side B – which means, that Side A should ALWAYS be IP .254 unless it fails, then Side B takes over.  The "preempt" means that as soon as Side A comes available again, it will take back over.  So, it's active/standby failover, and it will "fix itself" if it does happen to failover.

HSRP – Public Internet
Very similarly to our Internal Layer 3 VLANs, I also use HSRP to handle the "inside" part of our Public Internet routers.  We utilize two public /24 (Class C) spaces.  One is subnet 211 and one is subnet 212.  Both subnets are advertised and have public HSRP setup.

Side A – Primary – "subnet 211" – Secondary – "subnet 212" – Cisco 3825

interface GigabitEthernet0/0
 description Ethernet to LAN
 ip address x.x.212.253 255.255.255.0 secondary
 ip address x.x.211.252 255.255.255.0
 no ip redirects
 no ip proxy-arp
 duplex auto
 speed auto
 media-type rj45
 no keepalive
 no cdp enable
 standby 211 ip x.x.211.1
 standby 211 priority 200
 standby 211 preempt
 standby 212 ip x.x.212.1
end

Side B – Secondary – "subnet 211" – Primary – "subnet 212" – Cisco 2821

interface GigabitEthernet0/0
 description Ethernet to LAN
 ip address x.x.211.253 255.255.255.0 secondary
 ip address x.x.212.252 255.255.255.0
 no ip redirects
 no ip proxy-arp
 duplex auto
 speed auto
 no keepalive
 traffic-shape rate 1288000 128000 128000 4096
 no cdp enable
 no mop enabled
 standby 211 ip x.x.211.1
 standby 212 ip x.x.212.1
 standby 212 priority 200
 standby 212 preempt
end

Make sense?  Notice that both Interfaces have IPs, and I have 2x (two) HSRP groups – one handling each subnet.  Also, I've traffic shaped Side B – we have 100megs, but pay for a simple T1 CIR, so, I want to make sure we aren't surprised by any major traffic.  I can adjust that in an emergency, but, I keep it traffic shaped in a "normal" situation.

BGP – Public Internet
This is the piece that was a little more challenging.  I won't go into the details of BGP, but I will tell you that HSRP will give our "internal" network it's proper gateway – so – HSRP will get me to the Internet.  But, BGP is what gets the "internet" back to me.  Both were needed.

In my environment, I was both subnet 211 and 212 to be available at all times, so, both routers will "Advertise" both subnets.  Let's take a look at the setup.

Side A – Primary – "subnet 211" – Secondary – "subnet 212" – Cisco 3825

interface GigabitEthernet0/1
 description Ethernet to WAN / EaMIS
 ip address x.x.x.106 255.255.255.252
 duplex full
 speed 100
 media-type rj45
 no keepalive
!
router bgp 2386
 no synchronization
 bgp log-neighbor-changes
 network x.x.211.0 mask 255.255.255.0
 network x.x.212.0 mask 255.255.255.0
 neighbor x.x.x.105 remote-as 7018
 neighbor x.x.x.105 prefix-list defaultonly in
 neighbor x.x.x.105 route-map 211primary out
 no auto-summary
!        
ip bgp-community new-format
!
ip prefix-list defaultonly seq 5 permit 0.0.0.0/0
!
access-list 1 permit x.x.211.0
access-list 2 permit x.x.212.0
!
route-map 211primary permit 10
 match ip address 1
!        
route-map 211primary permit 20
 match ip address 2
 set as-path prepend 2386 2386 2386      

Side B – Secondary – "subnet 211" – Primary – "subnet 212" – Cisco 2821

interface GigabitEthernet0/1
 description Ethernet to WAN / EaMIS
 ip address x.x.x.94 255.255.255.252
 duplex full
 speed 100
 no keepalive
 traffic-shape rate 1288000 128000 128000 4096
!
router bgp 2386
 no synchronization
 bgp log-neighbor-changes
 network x.x.211.0 mask 255.255.255.0
 network x.x.212.0 mask 255.255.255.0
 neighbor x.x.x.93 remote-as 7018
 neighbor x.x.x.93 prefix-list defaultonly in
 neighbor x.x.x.93 route-map 212primary out
 no auto-summary
!
ip bgp-community new-format
!
ip prefix-list defaultonly seq 5 permit 0.0.0.0/0
!
access-list 1 permit x.x.212.0
access-list 2 permit x.x.211.0
!
route-map 212primary permit 10
 match ip address 1
!        
route-map 212primary permit 20
 match ip address 2
 set as-path prepend 2386 2386 2386

Did you follow along?  Gig0/1 is the "public" / WAN interface.  The BGP ASN (Autonomous System Number) we're using is 2386 – that was given to us from AT&T and is a "common" ASN used by customers that are multi-homed and doing BGP.  The BGP ASN 7018 is the AT&T ASN.  I had to tell my router who my "neighbor" was, and then had to "advertise" both my 211 and 212 networks.  That's what you see in the "router bgp 2386" command.

The ""prefix-list" shows that instead of holding all routing tables, I'm only interested in my "next hop" route.  I just want to know how to get from my router into the AT&T backbone.  That's it.

The two route-maps help me tell my routers which subnet (211 or 212) is the "primary" subnet out that router.  You'll notice a "set as-path prepend 2386 2386 2386" command.  Basically, when using the prepend command, I'm telling my router that THAT path is secondary and not as desirable.  That makes sense, right?  For example on my Cisco 3825, it's "primary" for subnet 211, so the route-map prepends for subnet 212.

Anyway, here's a small Visio PDF that shows what this looks like.

Hope it helps.  It's hard to find "exact" or "production" BGP setups.  Ours is not complicated for what we want to do – and it does work – this is not theory – this is real life production – right now.