Many-regions networking with Azure Route Server

The AI era has brought many good things: from endless cat videos to applications vibe-coded during the lunch break, but there is also something else they have brought to the table: capacity issues in public clouds. If you are a user of public cloud, you have probably experienced this problem: the virtual machine size you wanted or the GPU that you need is only available in this or that region, which happen not to be one of the regions where you have a connectivity hub.

So if you are using the self-managed hub-and-spoke model (instead of Virtual WAN), you might be confronted with the situation where you have a hub presence in two regions, but you are suddenly constrained to deploying virtual machines in a third region, as the next figure shows. What to do now?

Solution 1: create a new hub in every new region

The most intuitive solution would be creating a new hub in the third region:

If your hubs are not too complex, that might be the optimal to the problem, and this is also the recommended approach with Virtual WAN. However, what about if you have expensive network virtual appliances for firewalling and SDWAN in your hub? You might not want to replicate those licenses to a third region, a fourth one, etc.

Solution 2: peer new regions to an existing hub

In that case, you just peer the new regions to one of your two hubs:

This is definitely the most cost-effective pattern, but in the design above you have introduced a dependency on region 2. If region 2 goes down, the workloads in both regions 2 and 3 will also go down. Of course, you could have some manual failover action to replace the VNet peerings between region 3 and hub2 with new peerings to hub1 in case of a regional failure in region 2, which would probably be acceptable given how unlikely regional failures are in Azure.

Alternatively, you could alleviate the region dependency by using a multi-region architecture spreading the new application across region 3 and another region attached to hub1, but this past week a colleague of mine asked a different question: could you peer the spokes in region 3 to both hub1 and hub2, and configure them so that they prefer hub1 under normal circumstances, but they are able to failover to hub2 if hub1 became available?

Welcome to this post, this is probably the longest introduction I have ever written to a blog post.

Solution 3: create a new lightweight hub in every new region

The essence of this solution is enabling NVAs in hub1 and hub2 to inject routes dynamically via BGP in the spokes in the new region:

For this we need an Azure Route Server, that we deploy in region 3 so that we don’t introduce any new dependencies. If you need a regions 4, 5, 6, etc. you would only deploy an ARS in each of those regions, without having to replicate the NVAs in hub1 and hub2. This is the concept of light-weight hubs.

The spokes in region 3 need to be peered with the hub3 VNet using the “Use Remote Gateways” and “Allow Gateway Transit” settings, so that ARS can inject routes to them. If you have worked with VNet peering, you will be thinking the logical conclusion: the VNet peerings between region 3 and hub1 and hub2 cannot use those settings (only one VNet peering can use that configuration), so per default an ExpressRoute or a BGP-enabled VPN gateway wouldn’t advertise those prefixes to on-premises. That is why an additional ARS instance is required in hub1 and hub2.

In the following sections we would look in detail at how this setup works.

How it works: route injection to spokes

Each NVA will advertise a default route to ARS3, which will be learnt by the spokes. However, NVA2 will prepend its routes first, so that the spokes in region 3 prefer the default route to NVA1. The following commands show the routes that ARS3 learns from NVA1 and NVA2, where you can see the prepending applied to the default route coming from NVA2:

❯ az network routeserver peering list-learned-routes --routeserver ars-poland -g $rg --query 'RouteServiceRole_IN_0' -o table -n peer-nva1
AsPath    LocalAddress    Network        NextHop    Origin    SourcePeer    Weight
--------  --------------  -------------  ---------  --------  ------------  --------
65001     10.30.0.5       0.0.0.0/0      10.10.1.4  EBgp      10.10.1.4     32768
65001     10.30.0.5       10.10.0.64/27  10.10.1.4  EBgp      10.10.1.4     32768
65001     10.30.0.5       10.10.1.0/27   10.10.1.4  EBgp      10.10.1.4     32768

❯ az network routeserver peering list-learned-routes --routeserver ars-poland -g $rg --query 'RouteServiceRole_IN_0' -o table -n peer-nva2
AsPath             LocalAddress    Network        NextHop    Origin    SourcePeer    Weight
-----------------  --------------  -------------  ---------  --------  ------------  --------
65002-65002-65002  10.30.0.5       0.0.0.0/0      10.20.1.4  EBgp      10.20.1.4     32768
65002              10.30.0.5       10.20.0.64/27  10.20.1.4  EBgp      10.20.1.4     32768
65002              10.30.0.5       10.20.1.0/27   10.20.1.4  EBgp      10.20.1.4     32768

Only one route appears in the effective routes for the virtual machine in spoke5:

❯ az network nic show-effective-route-table -n nic-vm-c1-ep -g $rg -o table
Source                 State    Address Prefix    Next Hop Type          Next Hop IP
---------------------  -------  ----------------  ---------------------  -------------
Default                Active   10.31.0.0/24      VnetLocal
Default                Active   10.30.0.0/24      VNetPeering
VirtualNetworkGateway  Active   0.0.0.0/0         VirtualNetworkGateway  10.10.1.4
Default                Active   10.10.0.0/16      VNetGlobalPeering
Default                Active   10.20.0.0/16      VNetGlobalPeering

How it works: prefixes advertised to onprem

Not only NVA1 and NVA2 advertise routes to ARS3, but they also learn the prefixes of the spokes in region 3, so that they can advertise them forward to on-premises. Additionally, since the spokes in hub3 are going to prefer NVA1 for egress traffic, we need to make sure that inbound traffic from on-premises also prefers NVA1 under normal conditions, so I have configured BIRD on NVA2 to prepend the prefixes learnt from ARS3 when advertising them to its own ARS, ARS2.

The on-premises VPN gateway, simulated in my lab environment with an Azure VPN gateway will learns these prefixes advertised by NVA1 and NVA2 via their respective ARS instances:

❯ az network vnet-gateway list-learned-routes -g $rg --query 'value[].{Network:network, LocalAddress:localAddress, Peer:sourcePeer, NextHop:nextHop, ASPath: asPath, Origin:origin, Weight:weight}' -o table -n $vng
Network       LocalAddress    Peer       ASPath                   Origin    Weight    NextHop
------------  --------------  ---------  -----------------------  --------  --------  ---------
10.31.0.0/24  10.40.0.5       10.10.0.4  65515-65001              EBgp      32768     10.10.0.4
10.31.0.0/24  10.40.0.5       10.10.0.5  65515-65001              EBgp      32768     10.10.0.5
10.31.0.0/24  10.40.0.5       10.20.0.5  65515-65002-65002-65002  EBgp      32768     10.20.0.5
10.31.0.0/24  10.40.0.5       10.20.0.4  65515-65002-65002-65002  EBgp      32768     10.20.0.4
10.32.0.0/24  10.40.0.5       10.10.0.4  65515-65001              EBgp      32768     10.10.0.4
10.32.0.0/24  10.40.0.5       10.10.0.5  65515-65001              EBgp      32768     10.10.0.5
10.32.0.0/24  10.40.0.5       10.20.0.5  65515-65002-65002-65002  EBgp      32768     10.20.0.5
10.32.0.0/24  10.40.0.5       10.20.0.4  65515-65002-65002-65002  EBgp      32768     10.20.0.4

For simplicity I have removed some routes from the previous display, and I am only showing the routes that the onprem VPN gateway is learning about the spokes in region 3. I also removed some noise due to the fact that there are 2 BGP gateway instances speaking to each other. You can see that the advertisements that come from the VPN gateways in region 2 are prepended, so that the onprem network will prefer sending traffic to region 3 over NVA1, achieving our goal of symmetric routing.

Something to watch out for is that ARS always has the ASN of 65515. If the NVAs advertise the routes they learn from ARS3 to ARS1/ARS2 without any modification, the routes would be dropped when ARS1/ARS2 sees its own ASN (65515) in the AS path. Consequently, NVA1/NVA2 need to remove the 65515 when advertising the learned routes to ARS1/ARS2. The required BIRD configuration is as follows (as example the configuration in NVA1), you can probably configure something similar in your own NVAs:

[...]
filter export_to_hub_ars {
  bgp_path.delete(65515);
  accept;
}
[...]
protocol bgp ars_hub1_0 {
  local 10.10.1.4 as 65001;
  neighbor 10.10.0.68 as 65515;
  multihop 2;
  ipv4 {
    import all;
    export filter export_to_hub_ars;
  };
}

protocol bgp ars_hub1_1 {
  local 10.10.1.4 as 65001;
  neighbor 10.10.0.69 as 65515;
  multihop 2;
  ipv4 {
    import all;
    export filter export_to_hub_ars;
  };
}

The failover test

Under normal circumstances, you can see that the traffic from a VM in region 3 goes through NVA1 (10.10.1.4):

                             My traceroute  [v0.95]
vm-c1-ep (10.31.0.4) -> 10.40.1.4 (10.40.1.4)          2026-08-04T14:54:52+0000
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                       Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 10.10.1.4                         0.0%    15   21.6  21.6  21.2  22.7   0.5
 2. 10.40.1.4                         0.0%    15   37.8  32.1  30.5  39.0   2.8

We will simulate a regional outage shutting down NVA1. In a few seconds, traffic has converged to NVA2 (10.20.1.4):

                             My traceroute  [v0.95]
vm-c1-ep (10.31.0.4) -> 10.40.1.4 (10.40.1.4)          2026-08-04T14:57:34+0000
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                       Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 10.20.1.4                         0.0%     8   25.9  24.8  24.5  25.9   0.5
 2. 10.40.1.4                         0.0%     8   52.7  53.0  51.1  58.0   2.4

The effective routes in the region 3 VMs have also changed to NVA2:

❯ az network nic show-effective-route-table -n nic-vm-c1-ep -g $rg -o table
Source                 State    Address Prefix    Next Hop Type          Next Hop IP
---------------------  -------  ----------------  ---------------------  -------------
Default                Active   10.31.0.0/24      VnetLocal
Default                Active   10.30.0.0/24      VNetPeering
VirtualNetworkGateway  Active   0.0.0.0/0         VirtualNetworkGateway  10.20.1.4
Default                Active   10.10.0.0/16      VNetGlobalPeering
Default                Active   10.20.0.0/16      VNetGlobalPeering

And the onprem gateway is now only getting the region 3 prefix from NVA2:

❯ az network vnet-gateway list-learned-routes -g $rg --query 'value[].{Network:network, LocalAddress:localAddress, Peer:sourcePeer, NextHop:nextHop, ASPath: asPath, Origin:origin, Weight:weight}' -o table -n $vng                [26/560]
Network       LocalAddress    Peer       ASPath                   Origin    Weight    NextHop
------------  --------------  ---------  -----------------------  --------  --------  ---------
10.32.0.0/24  10.40.0.4       10.20.0.4  65515-65002-65002-65002  EBgp      32768     10.20.0.4
10.32.0.0/24  10.40.0.4       10.20.0.5  65515-65002-65002-65002  EBgp      32768     10.20.0.5
10.31.0.0/24  10.40.0.4       10.20.0.4  65515-65002-65002-65002  EBgp      32768     10.20.0.4
10.31.0.0/24  10.40.0.4       10.20.0.5  65515-65002-65002-65002  EBgp      32768     10.20.0.5

Restarting NVA1 will restore the previous flow.

The convergence times I measured were around 30 seconds, but you can achieve faster convergence times by tuning the keepalive and holddown timers in the BGP configuration of NVA1 and NVA2.

Wrapping up

If you have many different Azure regions but you don’t want to have as many virtual hubs, depending on your architecture you might use different network designs to provide connectivity to the regions without a local hub, using Azure Route Server gives you an elegant design with support for dual hubs.

Is this too complex? Possibly it is for most customers, but this design might be useful for some. And if not, at least hopefully you will have learnt something about Azure networking with this post.