You probably know Azure Virtual WAN, an Azure technology that abstracts hybrid networking by providing Microsoft-managed Virtual Hubs that use the Microsoft backbone to talk to each other. And you might know as well that those hubs can become Secured Virtual Hubs, including firewalling functionality powered by Azure Firewall.
Virtual WAN secure hubs are great, because they offer an easy-to-use abstraction for a network design that would otherwise imply multiple steps to implement. My favorite benefit of this setup is that the Virtual Networks that you connect to a secured virtual hub do not need to have any User-Defined Routes, because Virtual WAN will inject any required routes for you. That is great from a management perspective, especially if the spoke vnet administrators are not network specialists and they do not want to be bothered with boring stuff such as configuring routes and route tables.
However life is always more difficult that it looks: today Virtual WAN services do not support securing traffic cross-region. Is there anything we can do about it? Enter this post!
One possible workaround is creating one Virtual WAN per region, and interconnecting them via VPN (see my post Connect two VWANs to each other via VPN). However there are some drawbacks to this, such as the performance limits of traffic encryption as compared to native interconnection over the Microsoft backbone, as well as the lack of support for BGP (see the post for further details).
There is another possibility though. The current limitation of Virtual WAN is that traffic traversing two virtual hubs would be routed asymmetrically, meaning that it would not traverse the same firewalls in each direction, and it would consequently be dropped. We need to force Virtual WAN to bypass the firewalls for that cross-hub traffic. Yes, that traffic is not going through any firewall, but at least it is not dropped, and you can eventually secure by other means (like Network Security Groups).
In my test I have this topology:
So essentially we want to achieve these flows:
- Intra-hub flows go all through the firewall in each hub. For example for hub1: spoke11-spoke12, spoke11-branch1, spoke12-branch1
- Internet traffic from all vnets and branches goes through the closest hub, where that vnet or branch is attached
- Interhub flows between vnets will be routed directly, without traversing any firewall
- Interhub flows branch-to-vnet or vnet-to-branch will be firewalled
Achieving the first two points is straight forward: you could use the standard configuration for Virtual Secured Hubs:
- All vnet/branch connections associated to the default route table of their hub
- All vnet/branch connections propagating to the none route table
- A 0.0.0.0/0 route in the default route table of each hub pointing to the local firewall in that hub.
However, this ends up in the asymmetric routing situation we mentioned earlier. Consider the flow from spoke11 to spoke21:
- Traffic from spoke11 to spoke21 is sent through the Azure Firewall in hub1 (let’s call it AzFW1). From there, it goes straight to spoke21
- Traffic back from spoke21 to spoke11 will traverse the Azure Firewall in hub2 (AzFW2), and then straight back to spoke11. Asymmetric routing is served.
How can we force hub1 to send spoke11-to-spoke21 traffic directly to hub2, without using the default route pointing to the Firewall? In which we inject the more specific routes for the vnets and branches in hub2. We can do this with this configuration:
- All vnet/branch connections still associated to the default route table of their hub
- All vnet/branch connections propagating to the default route table of the other hub. For example, we would inject the routes for vnets and branches of hub2 into the default route table of hub1, so that hub1 prefers those more specific routes to the 0.0.0.0/0 pointing to the firewall
- Our 0.0.0.0/0 route is still in the default route table of each hub pointing to the local firewall in that hub, but it is more generic than the routes injected via vnet/branch propagation
We are almost there. There is one more change we need to introduce. We will use an extra route table for the vnets, so that we can separate routes for vnets and branches. The goal here is to have a more granular control on which routes apply to vnets, and which ones apply to branches. So we end up with this configuration:
As you can see in the diagram above, there are some unusual configurations:
- Vnet-Hub-Hub-Vnet traffic is not going through any firewall in this design. Please notice that this fact might make this design inappropriate for your requirements.
- Vnet-Hub-Hub-Branch traffic is not firewalled either. A variation of this design would be the having branches connected to both hubs, thus eliminating those VHHB or BHHV flows.
- Vnets and branches are propagating to the route tables in the the other hubs, but not to the hubs where they are connected
- This configuration is easy to do in the portal for vnet connections, but for branches you will need Azure CLI or Powershell. You can find an example script for the whole configuration in my Github repo here: https://github.com/erjosito/azcli/blob/master/vwan_2xshub.azcli
- We are using labels for propagation, note that the route tables are labeled with the hub they are located, so it is easy enough configuring connections to propagate to any remote hub
As mentioned above, have a look at my CLI script to implement this if you are curious about the exact commands. Enjoy!
Cloudtrooper 
Hi Jose,
Many thanks for blogs like this one.
I’m looking at https://docs.microsoft.com/en-us/azure/virtual-wan/virtual-wan-global-transit-network-architecture#security and it makes me feel that you can have two Secured Hubs, although (h) traffic isn’t explicitly mentioned. What am I missing?
LikeLike
Hey Tomas! Yes, as you can see in that doc, h is not supported (it says that Azure Firewall in the vitual hub supports flows e, i and j). We are working hard to bring VHHV and VHHB support to secured hubs (stay tuned), but as per my knowledge it doesnt work today.
What this post explains is how to remove the AzFW from the equation for the h flows.
LikeLike
Thanks for the clarification Jose.
Shouldn’t then the diagram only display one AzFW since only one can exist anywhere in VWAN?
LikeLike
Actually both firewalls, one in each hub, can exist, some flows will not work though
LikeLike
Hey thanks for sharing I am facing the same issue with secure hub multiple regions, still not supported?
LikeLike
Not yet, but we are getting closer. We have a preview going on for multi-secured-hub, with the constraint that those hubs need to be in the same region during the preview. For GA that constraint should disappear. You can check https://docs.microsoft.com/azure/virtual-wan/how-to-routing-policies
LikeLike
Thanks a million Jose as always really helpful 😉
LikeLike
So happy it helps, Alessandro!
LikeLike
Hi Jose, the secured vhub to secured vhub limitation used to be part of vWAN FAQ if my memoty servers me well, but I can’t find it anymore. May you please help me how to track the issue is still valid, or not ? thanks !
LikeLike
Yes, the issue is unfortunately still there
LikeLike
Do you know where the issue is documented currently please ?
LikeLike
Here: https://docs.microsoft.com/en-us/azure/virtual-wan/virtual-wan-global-transit-network-architecture#branch-to-branch-secured-transit-cross-region-f
LikeLike
Hi. I’ve come across this post whilst troubleshooting multi-secured hubs in my current deployment, these are in separate regions.
Is this now available as when I select all my VNETs to point to their respective Azure Firewall I am unable to traverse between hubs? (secured VNETs)
LikeLike
Hey Sven, this post describes a workaround for a limitation in Virtual WAN, which today still exists.
LikeLike
Thank you for confirming Jose and providing your workaround.
As STÉPHANE asked above is there any way we can track this feature as I am sure many of us are waiting patiently for implementation to complete?
LikeLike
I thought you guys might appreciate the update I have had since posting my question a while back. Working with Microsoft I now have a working solution albeit with a minor workaround by placing a VNET in-between the other VNETs in the respective region.
I suggest giving the below video a watch:https://www.youtube.com/watch?v=YZ0EQDut6_8
LikeLike
Thanks for sharing Sven! This is actually the more generic design that I describe here as well: https://blog.cloudtrooper.net/2022/01/18/is-the-core-distribution-access-design-dead/. That post was addressing a number of other additional topics (like modular designs and multicloud architectures), so the indirect spoke design might have got lost in the noise…
LikeLike
Unlike any other, your articles really challenge and further everyone’s understanding of the Azure networking technologies. This is very creative solution.
There was one detail I was wondering about: “Traffic from spoke11 to spoke21 is sent through the Azure Firewall in hub1 (let’s call it AzFW1). From there, it goes straight to spoke21”. I had been assuming that the default route table of hub2 contains only 0.0.0.0/0 since all connections are propagating to none. So, I am wondering, how would the traffic get past hub2 to spoke 21?
LikeLike
Hey Kasim! So happy the blog is useful! An oversimplification on my side, since at the time I didn’t want to speak about the internal structure of virtual hubs. In the meantime it is not a secret any more (since Microsoft release the BGP endpoint feature, that router is visible), so here you go:
AzFW1 will give the traffic to a virtual router in hub1 called Route Service, RS1. RS1 knows the spoke21 route from the RS in hub2, RS2, so it will send it there. RS2 knows the spoke21 route as “directly connected”, so it gives it forward.
The actual packet flow is VM_in_Spoke11 -> AzFW1 -> RS1 -> RS2 -> VM_in_Spoke21, and for the return traffic VM_in_Spoke21 -> AzFW2 -> RS2 -> RS1 -> VM_in_Spoke11
Does this help?
LikeLike
Yes, that helps. Thank you very much. I sort of figured out that traffic is being handed over between services in the hub but the order was not clear to me.
It has been somewhat difficult to understand why the Hub-Hub scenario is so different from the OnPrem-Hub scenario. The later scenario works with traffic flowing through the firewall in the hub – in both directions. My tentative conclusion now is that it must be because static routes (that force the hub-inbound traffic through the firewall) are not propagated between RS2 and RS1 but they are propagated to onprem.
LikeLike
Well, neither VHHV nor VHHB work with secured hub, at least until Microsoft releases routing intent. The thing with branches is that you can connect them to every hub, effectively transforming VHHB in VHB. Does that make sense?
LikeLike
Yes, Thank you very much!
LikeLike