Setting up /31 interfaces and BGP on a CentOS machine

After reading the title above, you might be wondering why the heck you would want to do such a thing. The reason in my particular case is to connect an Azure Stack installation to an Azure ExpressRoute circuit, since Azure Stack needs some kind of connectivity to Azure. Azure Stack uses internally BGP and /31 interfaces, hence the need for those technologies. You could obviously use a virtual router of your favorite network vendor (such as Cisco CSR 1000v, which would be a great fit for this use case), but if you are on the low side of the budget, Linux is your friend. And since my favorite web search engine did not show me any hit explaining how to do it, hence this blog.

There are essentially 4 steps to this process:

  1. Configure /31 interfaces in your Linux machine
  2. Configure IP forwarding
  3. Install and configure BGP


1. Configure /31 Interfaces

This is the trickiest part of the process. I used a CentOS distro, if you are using something else like SUSE or Ubuntu your mileage might vary.

First, we should clarify what is a /31 address, and why they are used out there. /30 addresses are easy: only the 2 last bits are variable, and they render two usable IP addresses:

  • 0: subnet address
  • 1: first usable address
  • 2: second usable address
  • 3: broadcast address

Hence, if you for example use the subnet, your 2 usable addresses are .1 and .2. The subnet address will be .0, and the broadcast address .3. You could argue that these two IP addresses are not used in a point-to-point link (when there is only one other subnet member out there, broadcast is not that meaningful), so essentially you are wasting 50% of your IP address space.

IETF RFC 3201 has standardized the usage of /31 subnets for point-to-point links in order to minimize IP address waste, where by not having subnet or broadcast addresses, IP address space utilization is optimized. For example, you could use (addresses .0 and .1) in one link, and (addresses .2 and .3) in another one.

The main problem here is that for every Ethernet interface, NetworkManager in CentOS automatically calculates the broadcast IP address, since it assumes that the interface is point-to-multipoint (as most Ethernet interfaces are). For example, if you configure the IP address, Network Manager will configure the broadcast address to When you try to ping the other side, CentOS will think that you are actually trying to broadcast:

[root@quagga-01 ~]# ifconfig eth1
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
 inet netmask broadcast
[root@quagga-01 ~]# ping
Do you want to ping broadcast? Then -b. If not, check your local firewall rules.

You therefore need to disable NetworkManager in your ifcfg script with the directive NM_CONTROLLED=no. You can set the broadcast address to something else, such as (they actual value does not really matter, we just want to have it different than the IP address of the other side). Here is my ifcfg-eth1 for reference:


If you restart the network (systemctl restart network) check the IP parameters of the interface, you should verify that the broadcast IP address is set accordingly, and now you can ping the other side:

[root@quagga-01 ~]# ifconfig eth1
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
 inet netmask broadcast
[root@quagga-01 ~]# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=3.74 ms
64 bytes from icmp_seq=2 ttl=64 time=0.285 ms


2. Configure IP Forwarding

This one should be a piece of cake, but don’t forget it or your Linux machine will not behave as a router. You will find many pages in Internet showing how to do this. You need to configure a system parameter in /etc/sysctl.conf (or in a file inside of the /etc/sysctl.d directory):

net.ipv4.ip_forward = 1

Now reload changes you’ve made to the kernel parameters configuration:

systctl -p

You can verify whether IP forwarding is enabled (should return 1):

 cat /proc/sys/net/ipv4/ip_forward


3. Install and Configure BGP

We will use the package quagga for this. Installation is easy enough (the reason of why we are installing telnet as well coming next):

yum install -y quagga telnet

Configuration is pretty easy too, but there are a few things about quagga you need to understand. Quagga is made out of multiple daemons, one for each routing protocol. The glue putting all together is Zebra. However, you do not start or stop any zebra daemon, but the individual routing protocol daemons (bgpd in our example).

You will then connect to a Cisco-CLI-like shell via telnet (that is why we installed telnet). But first things first. If you check your /etc/quagga directory, you will find two files: vtysh.conf (vtysh is the process providing the CLI) and zebra.conf. You can add a new configuration file for bgpd, starting from the samples you can find in /usr/share/doc/quagga- For example, my bgpd.conf file looks like this:

hostname quagga-01
password mysupersecretpassword
enable password mysupersecretpassword
router bgp 65001
 bgp router-id
 neighbor remote-as 65002
log file bgpd.log
log stdout

As you can see, this looks like a standard Cisco BGP configuration, so it should be familiar. Now you only need to configure the bgpd daemon to start, and you can connect to the router:

systemctl enable bgpd
systemctl start bgpd

Now you can connect to your router and run your familiar BGP show commands to start playing around (if you are not sure about show commands, the question mark is your friend!):

[root@quagga-01 ~]# telnet localhost bgpd
quagga-01> sho ip bgp summ
BGP router identifier, local AS number 65001
RIB entries 1, using 112 bytes of memory
Peers 1, using 4560 bytes of memory

Neighbor     V    AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down  State/PfxRcd  4 65002       6       9      0   0    0 00:04:58        1

Total number of neighbors 1


3 thoughts on “Setting up /31 interfaces and BGP on a CentOS machine

  1. Michael McConnell

    Unfortunately, this no longer works in RHEL/CentOS 8, as networking is now done exclusively through NetworkManager (and setting the BROADCAST variable ins the ifcfg-interface file is ignored). Apart from a dirty hack of putting a short delay in /etc/rc.d/rc.local then scripting it there manually I’ve yet to find a better way to do it.


    1. Michael McConnell

      Replying to my own comment, it appears /31 networking can be done on RHEL/CentOS 8 by installing the network-scripts package, then the steps above will work.
      Another hiccup is quagga isn’t present (RHEL states the functionality has been removed), but an alternative package BIRD is available on EPEL. It’s something I’ll need to experiment with.


      1. Thanks for posting your findings Michael!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: