Here is a step by step example of how to set up a very simple MPLS-VPN. Like last time I am doing this entirely in GNS3 using 2691s running 12.4(25d).
The basic premise being you have one customer which needs access across your MPLS backbone. The customer should not have access to your backbone, nor should they have access to any other customers networks. Unlike other forms of VPNs – such as IPSEC VPNs, MPLS-VPNs have a great ability to scale.
Firstly, every time you add a new customer site you do not have to do any configuration on any other sites. Except if the customer is off a new PE, then you have the iBGP mesh issues – which can easily be handled by route reflectors, but this is outside of the point of this post.
Secondly, no matter how many routes your customer router has or how many customers you have, your core routers (Provider or P-layer) do not have any increase in their memory utilisation.
Here is our basic service provider network:
Note the topology in the MPLE-TE example is a little different, so you have that set up already keep an eye out, one thing of note is there is no link between the PE-Layer routers, Njord and Thor. This is to force traffic through Odin (our P layer router).
For those who haven’t seen the terms P, PE and CE before.
- P-Router – “Provider Router” – in an MPLS network this router does not contain any customer routes and will generally not run BGP. It will run MPLS and the IGP (in this example, OSPF). The idea is that these are go-fast routers and you try and reduce their complexity and memory requirements, much like the Core layer in campus design.
- PE-Router – “Provider Edge Router”. Will run MPLS into the core and run a routing protocol into the customer site – generally BGP. These also need to run iBGP between each other. If you are a service provider this will be the last router you control, so also a good place to do are your prefix checks, your accounting, enforce bandwidth etc. In an enterprise network it just marks the demarcation point between the MPLS network and the vanilla IP network
- CE-Router – “Customer Edge Router”. Generally runs BGP to the PE router and whatever routing protocol the “customer” side of things run – OSFP, EIGRP, ISIS? RIP?? Static? From a customer point of view this is where you would run your security policies, IPSEC tunnel back to HQ etc.
Step 1 – Basic MPLS SP Network
So building up the basic configuration for the SP network, lets give the routers some loopback addresses, point-to-point addresses, turn on our IGP (OSPF, all area 0), and turn on MPLS.
On Njord
Njord#conf t Enter configuration commands, one per line. End with CNTL/Z. Njord(config)#mpls ip Njord(config)#int lo 0 Njord(config-if)#ip add 192.0.2.3 255.255.255.255 Njord(config-if)#int f0/0 Njord(config-if)#mpls ip Njord(config-if)#ip add 198.51.100.6 255.255.255.252 Njord(config-if)#router ospf 1 Njord(config-router)#net 192.0.2.3 0.0.0.0 area 0 Njord(config-router)#net 198.51.100.6 0.0.0.0 area 0
On Thor:
Thor#conf t Enter configuration commands, one per line. End with CNTL/Z. Thor(config)#mpls ip Thor(config)#int lo 0 Thor(config-if)#ip add 192.0.2.2 255.255.255.255 Thor(config-if)#int f0/0 Thor(config-if)#mpls ip Thor(config-if)#ip add 198.51.100.2 255.255.255.252 Thor(config-if)#router ospf 1 Thor(config-router)#net 192.0.2.2 0.0.0.0 area 0 Thor(config-router)#net 198.51.100.2 0.0.0.0 area 0 *Mar 1 00:03:41.723: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
And finally on Odin:
Odin#conf t Enter configuration commands, one per line. End with CNTL/Z. Odin(config)#mpls ip Odin(config)#int lo 0 Odin(config-if)#ip add 192.0.2.1 255.255.255.255 Odin(config-if)#int f0/0 Odin(config-if)#mpls ip Odin(config-if)#ip add 198.51.100.1 255.255.255.252 Odin(config-if)#int f0/1 Odin(config-if)#ip add 198.51.100.5 255.255.255.252 Odin(config-if)#mpls ip Odin(config-if)#router ospf 1 Odin(config-router)#net 192.0.2.1 0.0.0.0 area 0 Odin(config-router)#net 198.51.100.1 0.0.0.0 area 0 Odin(config-router)#net 198.51.100.5 0.0.0.0 area 0 *Mar 1 00:09:44.803: %SYS-5-CONFIG_I: Configured from console by console Odin(config-router)#net 198.51.100.5 0.0.0.0 area 0 Odin(config-router)# *Mar 1 00:09:51.087: %OSPF-5-ADJCHG: Process 1, Nbr 192.0.2.2 on FastEthernet0/0 from LOADING to FULL, Loading Done *Mar 1 00:09:51.159: %OSPF-5-ADJCHG: Process 1, Nbr 192.0.2.3 on FastEthernet0/1 from LOADING to FULL, Loading Done Odin(config-router)# *Mar 1 00:10:03.691: %LDP-5-NBRCHG: LDP Neighbor 192.0.2.2:0 (1) is UP *Mar 1 00:10:03.931: %LDP-5-NBRCHG: LDP Neighbor 192.0.2.3:0 (2) is UP
And at the end you can see OSPF becomes adjacent and LDP Neighbors go up. If you didn’t see this, you can verify with show ip ospf neigh and show mpls ldp neigh.
Step 2: iBGP, VPNv4 and VRFs
Next we have to configure an iBGP neighborship between our two PE routers and enable those neighbors to communicate their extended VPNv4 communities. Once this is done we will also configure a VRF.
A VRF stands for “Virtual Routing and Forwarding”. This is what enables us to use multiple routing tables to keep either customers or security levels separate on a network level. This is to layer 3 what a VLAN is to layer 2. Ports in different VRFs cannot by default communicate with each other. As an aside, you can take advantage of VRFs without using MPLS – this is referred to as VRF-lite. It keeps your routing tables separate but you have to run a routing protocol for each VRF so it doesn’t scale too well.
Alright, configuring iBGP with VPNv4 on Thor:
Thor(config)#router bgp 64496 Thor(config-router)# no sync Thor(config-router)# neighbor 192.0.2.3 remote-as 64496 Thor(config-router)# neighbor 192.0.2.3 update-source Loop0 Thor(config-router)# ! Thor(config-router)# address-family vpnv4 Thor(config-router-af)# neighbor 192.0.2.3 activate Thor(config-router-af)# neighbor 192.0.2.3 send-community extended Thor(config-router-af)# exit-address-family
And on Njord:
Njord(config)#router bgp 64496 Njord(config-router)# no sync Njord(config-router)# bgp log-neighbor-changes Njord(config-router)# neighbor 192.0.2.2 remote-as 64496 Njord(config-router)# neighbor 192.0.2.2 update-source Loopback0 Njord(config-router)# ! Njord(config-router)# address-family vpnv4 Njord(config-router-af)# neighbor 192.0.2.2 activate Njord(config-router-af)# neighbor 192.0.2.2 send-community extended
If you aren’t used to BGP, keep in mind if you are using default timers like I am – the neighborship will take quite a while to come up compared to an IGP.
Also note how there was no configuration required on our P router.
Now configuring a VRF:
Thor#conf t Enter configuration commands, one per line. End with CNTL/Z. Thor(config)#ip vrf Olympus Thor(config-vrf)# rd 64496:1 Thor(config-vrf)# route-target export 64496:1 Thor(config-vrf)# route-target import 64496:1
This configuration is exactly the same on both Thor and Njord.
There are two terms here that are worth understanding – they can be fairly hard to get your head around at first, and they aren’t particularly useful to play around with unless you want to do something a little trickier than this post – like communicating between VRFs, deploying a common services VRF or load balancing through your IGP. All of these things I hope to cover in later posts.
RD – Route Distinguisher – This is a little like a route-tag. It allows us to have overlapping IP addresses and can tell us a little something about the route.
RT – Route-target. Tells the router that this VRF should import/export it’s routing table from routes in this “collection”. These numbers need to match on the VRF configuration on all your PE routers for a VRF.
Note even though RT = RD (and this is pretty common practice) they don’t have to, they are used for completely different functions. A common numbering scheme for both RD and RT is to use the local AS number followed a number representing the VRF – this is what I have used.
Step 3: Adding the CEs
In this section we add 2 CE routers, connect them to the PE routers and bring an eBGP link up between them and the PEs. There are two things that are new compared to a standard eBGP configuration. The CE-facing interface on the PEs must be assigned to the correct VRF and the BGP configuration on the PE is done within a IPv4 VRF address family.
Here is our extended topology:
So, first we will configure the CE routers. I am only giving them a loopback and point-to-point address, configuring BGP and advertising their locally connected networks to BGP.
On Zeus:
Zeus(config)#int lo 0 Zeus(config-if)#ip add 203.0.113.254 255.255.255.255 Zeus(config-if)#int f0/0 Zeus(config-if)#ip add 203.0.113.2 255.255.255.252 Zeus(config-if)#router bgp 65511 Zeus(config-router)#neighbor 203.0.113.1 remote-as 64496 Zeus(config-router)#neighbor 203.0.113.1 update-source f0/0 Zeus(config-router)#net 203.0.113.254 mask 255.255.255.255 Zeus(config-router)#net 203.0.113.0 mask 255.255.255.252
And on Poseidon:
Poseidon#conf t Enter configuration commands, one per line. End with CNTL/Z. Poseidon(config)#int lo 0 Poseidon(config-if)#ip add 203.0.113.253 255.255.255.255 Poseidon(config-if)#int f0/0 Poseidon(config-if)#ip add 203.0.113.6 255.255.255.252 Poseidon(config-if)#router bgp 65510 Poseidon(config-router)#neighbor 203.0.113.5 remote-as 64496 Poseidon(config-router)#neighbor 203.0.113.5 update-source f0/0 Poseidon(config-router)#net 203.0.113.253 mask 255.255.255.255 Poseidon(config-router)#net 203.0.113.4 mask 255.255.255.252
Next configuring out PEs
Thor(config)#int f0/1 Thor(config-if)#ip vrf forwarding Olympus Thor(config-if)#ip add 203.0.113.1 255.255.255.252 Thor(config-if)#router bgp 64496 Thor(config-router)#address-family ipv4 vrf Olympus Thor(config-router-af)#neighbor 203.0.113.2 remote-as 65511 Thor(config-router-af)#neighbor 203.0.113.2 update-source f0/1 Thor(config-router-af)#neighbor 203.0.113.2 next-hop-self Thor(config-router-af)#
Njord(config)#int f0/1 Njord(config-if)#ip vrf forwarding Olympus Njord(config-if)#ip add 203.0.113.5 255.255.255.252 Njord(config-if)#router bgp 64496 Njord(config-router)#address-family ipv4 vrf Olympus Njord(config-router-af)#neighbor 203.0.113.6 remote-as 65510 Njord(config-router-af)#neighbor 203.0.113.6 update-source f0/1 Njord(config-router-af)#neighbor 203.0.113.6 next-hop-self
Lastly, a bit of verification, you can see the routes from the CEs to each other.
Zeus#show ip route [snip] 203.0.113.0/24 is variably subnetted, 4 subnets, 2 masks C 203.0.113.254/32 is directly connected, Loopback0 B 203.0.113.253/32 [20/0] via 203.0.113.1, 00:12:27 C 203.0.113.0/30 is directly connected, FastEthernet0/0 B 203.0.113.4/30 [20/0] via 203.0.113.1, 00:03:11
But notice you can’t see any of the service providers backbone IP addresses.
Similarly on the PE routers, notice the VRF specific commands.
Thor#show ip route vrf Olympus Routing Table: Olympus [sic] 203.0.113.0/24 is variably subnetted, 4 subnets, 2 masks B 203.0.113.254/32 [20/0] via 203.0.113.2, 00:16:44 B 203.0.113.253/32 [200/0] via 192.0.2.3, 00:14:05 C 203.0.113.0/30 is directly connected, FastEthernet0/1 B 203.0.113.4/30 [200/0] via 192.0.2.3, 00:04:50
Pinging from Poseidon:
Poseidon#ping 203.0.113.254 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 203.0.113.254, timeout is 2 seconds: !!!!!
None of these routes are visible from the P layer:
Odin#show ip route vrf Olympus % IP routing table Olympus does not exist
This post is already monstrously too long so I will not go through all the BGP related commands and verifying. But show ip bgp vpnv4 all and show ip bgp vpnv4 all sum are quite useful.
Hopefully I can go into a bit more detail on verification and fault finding in a later post.
Hi,
How can i add extra networks on the Zeus and Poseidon network,
Imagine i want network 10.0.0.0/8 connected to router f0/1.
I cannot route between routers.
Many thanks for your article.
Regards,
Vitor
Hi Vitor,
I assume you are trying to add another access network? If F0/1 is the interface connected to your users on Poseidon you would just need to add the IP address and advertise it in BGP:
Poseidon(config)#int f0/1
Poseidon(config-if)#ip address 10.0.0.1 255.0.0.0
Poseidon(config-if)#router bgp 65510
Poseidon(config-router)#network 10.0.0.0 mask 255.0.0.0
Hope this helps.