If you are reading this, you probably already know what Azure Private Link is: a representation of a service such as Azure Storage, Azure SQL Database, Azure Application Service, or even some application running in a different Virtual Network, in your own Virtual Network with a private IP address of your own.
This is a fantastic feature, many organizations have been waiting for this for a long time. Now that say your Azure Storage account is accessible via a private IP address, what if you want to restrict traffic to it, so that not everybody in your organization has IP connectivity to it?
Please realize that not everybody will want to do this, since for many organizations authentication is enough security to control who can and who cannot access a certain service. However, others will like to combine authentication security with networking security.
Your first option to filter network traffic inside of a Virtual Network would be Network Security Groups (NSGs). Unfortunately, private link endpoints do not support NSGs (see Private Endpoint Limitations). Mmmh.
Of course, if you cannot apply NSGs at the destination (the private endpoint), you can apply at the source to apply those network restrictions. Wait a second: that would be a perfectly valid measure for other Azure workloads, but what is the source is not in Azure, but on-premises? Right, no NSGs on-premises. Our next tactics will be using a traffic filtering device such as the Azure Firewall. We will see a couple of tricks that we will need for this to work.
Sending traffic through an Azure Firewall (or any Network Virtual Appliance) in Azure is a two-step process: for a flow between the private endpoint and on-premises we need to send packets from on-prem to the Azure Firewall, as well as the return traffic from the private endpoint.
The first part is relatively easy: traffic coming from on-premises will land in either a VPN or an ExpressRoute gateway. In both cases, in your GatewaySubnet (unless you are using a 3rd-party network appliance to terminate your VPN, but the concept is the same). This means that to redirect traffic from on-prem to the Azure Firewall you need a route table on the GatewaySubnet.
There is one thing you need to know: creating private link endpoints injects /32 routes into your Azure subnets. Hence you want to create User-Defined Routes for your endpoint using specific /32 prefixes too (otherwise, the most specific prefix will win). Unfortunately you cannot verify the effective route table of your gateways, but believe me, there are few things out there that can beat a static /32 route.
Now for the return traffic, it depends on which Azure service you are routing to. For example, for Azure Storage it is going to be easier than you think. Private Link is not your grandma’s NIC, but it is using flow-based networking: when answering packets it will send them to the same place they came from. In English: in private endpoints for Azure Storage we don’t need UDRs for the return traffic, and no route tables in the private link subnet.
However, other Azure Services might behave differently and cause asymmetric routing, so SNATting your traffic at the NVA is probably a good idea. This way you can be sure that you have no asymmetric routing, regardless which Azure service your private link refers to.
You don’t believe me? Test Azure SQL and Azure Storage private link with Azure Firewall with my Azure CLI script here!
Cloudtrooper 
Hi erjosito, the penultimate paragraph you wrote down is in complete disagreement with Azure Documentation: https://docs.microsoft.com/en-us/azure/private-link/private-endpoint-overview#limitations
How can it be possible? I think you’ve been just lucky, somehow. Could you please double-triple-quadrouple check this asymmetric routing thing? Someone has to notify MSFT if you’re right 😀
LikeLiked by 1 person
Hey Antani! Microsoft docs are right. They say “may be asymmetric”, because it depends on which service you are looking at: for Azure Storage, as far as I tested, traffic will be symmetric. However, the docs are conservative, and recommend doing SNAT so that you dont need to think about which service you are talking to.
In essence, I should probably update that paragraph in my blog, thanks for the remark!
LikeLike
Thank you!
Actually it’s quite impressive that an article about Cloud services is still so valid after two years. Good job!
LikeLiked by 1 person
Thanks a lot for this very helpfull article. I was searching for a while why it did’nt work before faling on you article and the precision “Hence you want to create User-Defined Routes for your endpoint using specific /32 prefixes too (otherwise, the most specific prefix will win).” It was my issue, using a too large mask for my UDR…
LikeLike
So happy it helped, Bruno!
LikeLike
Hi Jose
Thanks for all your articles.
I get the point that the best practice is to SNAT traffic going via a firewall in Azure to a some (most?) private endpoint fronting resources in Azure.
Yet I’m frustrated not to understand what is happening when one does not SNAT.
The setup is:
I can see SYN packet going through Az Firewall (AZFWNetworkRule table). But I have no clue where the SYN ACK packet is going. It is not going via Az firewall (AZFWFlowTrace table) nor to the jumpbox (wireshark). Maybe it does even exist?! I failed to find a way to trace it as VNet flow logs do not handle PE traffic.
Can you please help me understand what is going on here?
Thanks
LikeLike
Hey, I am happy that the posts are useful! There are a few important concepts to understand:
– Private Link connectivity under the covers consists of tunneled traffic from an Azure resource to the PaaS service.
– When you send traffic to a private endpoint via AzFW, this tunnel is built between an AzFW instance and the PaaS service at play (AzSQL in your example).
The thing is that the implementation in the server side of Private Link varies across PaaS services, and some of them (such as AzSQL) need that the inner and outer address match (oversimplifying here). That is why for some services you need to SNAT. Otherwise the answer from the PaaS service will not be correctly routed by the Azure SDN, and will get lost in its dark corners.
By the way, Microsoft is working to eliminate the need for SNAT for Private Link for all services, but I can’t share any ETA.
LikeLiked by 1 person
Thanks so much Jose. I had always imagined the tunnel would be between the NIC linked to the private endpoint and some other network element close to the PaaS service. I didn’t imagine the tunnel would be built from Az firewall instance directly.
LikeLike
Yeah, the NIC is just a representation of the endpoint, but it only serves as an object in the data model where to attach a subnet, and it doesn’t participate in the data plane at all.
For example, if a VM is in VNetA and the private endpoint (and it’s corresponding NIC) in a peered VNetB, traffic between the VM and the endpoint will not traverse the peering. The Private Link tunnel is created from the VM directly to the PaaS service.
LikeLike
Apparently the feature to not need SNAT is already out! https://journeyofthegeek.com/2024/08/29/azure-networking-inspecting-traffic-to-private-endpoints/
LikeLike
[…] we discussed in the first post of the series as well as in this old post from 2020, private endpoints create /32 routes in all virtual machines in the VNet where they are located and […]
LikeLike