private-endpoint

When deciding between Public, Service, or Private endpoints in Azure, it’s essential to make an informed choice rather than blindly following generic recommendations.

Tools like Azure Secure Score, various benchmarks, and CAF documentation may seem convincing, but uncritical adherence can result in less resilient solutions and increased maintenance costs.

In this post, I will explain the differences between Public, Service, and Private endpoints and share the pros and cons of their use in common scenarios.

— We have a hole in security!
— Thank God, at least something is in security…

Internet meme

Endpoints comparison

To get started, let’s compare how each type of endpoint works.

Public Endpoints Service Endpoints Private Endpoints
Availability On all PaaS services On limited number of PaaS services On on most of PaaS Services
How routing works

Public Endpoints are available on public Internet IP addresses.

From public Internet anyone can reach them.

From private VNet, traffic is routed according to your routing tables (via Azure NAT or a Firewall).

On certain resources, such as storage, there is an option to prefer Microsoft routing, where the actual traffic remains within the Microsoft backbone.

Direct routes for public IP ranges are advertised to your subnet and are visible in Effective Routes. These routes are specific to resource types and regions.

Client traffic flows directly to the endpoint, bypassing the routing table attached to the subnet, but it remains within the Microsoft backbone. It appears as if NAT is not used, as the resources on service endpoints can see the client’s original private IP from the VNet.

An IP interface is created in the VNet where you deploy your private endpoint.

All traffic to endpoints remains within the private VNet.

Network security measures

On destination: Resource ACL

On client: NSG, firewalls

On destination: Resource ACL which can use public and private IPs, and specific VNet subnets

On client: Service Endpoints policies (only available for selected services), NSG

On destination: NSG (when Private Endpoint Network Policies are enabled)

On client: NSG, firewalls

Management burden None

Need to disable Public Access on Resource ACL

Need to enable Service Endpoints on each subnet where they are used

Need to disable Public Access on Resource ACL

Need to deploy PE to VNets, for each service (blob, file) on each resource (storage, key vault)

Consuming private IP addresses, especially if you deploy PEs in dedicated subnets (+5 reserved IP addresses wasted)

Requires changes in name resolution (from changing HOSTS file on each client up to implementing sophisticated DNS integration)

High complexity for name resolution when need to deploy multiple PEs of the same resource

General recommendations

Below are scenarios where using each type of endpoint is recommended.

Public Endpoints

Use Public endpoints for:

  • Public resources, like web hosting [app service, storage]
  • Resources for cloud-born applications, like SaaS, PaaS, and apps that are not integrated with your private network [web apps, functions, storage]
  • Shared internal resources, which should be available across networks, clouds, and various client devices [storage, monitoring, backup, site recovery]
  • Resources used from other clouds

Service Endpoints

Use service endpoints for:

  • Services requiring the highest possible bandwidth and least possible latency [storage, databases]
  • Shared services consumed from multiple VNets and regions [central file share, Cloud Witness for clusters]
  • Services requiring less administrative effort than managing private endpoints

Private Endpoints

Use private endpoints for:

  • Resources that need to be accessed from on-premises over a private network: Requires establishing hybrid connectivity and DNS integration.
  • Resources that are only required to be accessed from one VNet and not exposed outside of their system (see private data store pattern, like for internal databases and storage).
  • Resources for environments requiring the highest protection from data exfiltration: This class of systems should be treated separately from the general class. To effectively protect from data exfiltration, all inbound and outbound traffic should be restricted to tenant-specific (owned by you) destinations only. This may come with the cost of losing a number of cloud platform capabilities, like Monitoring, Backup, Site Recovery, Patching automation, and others. Most of these services are multi-tenant and do not work without opening wide IP ranges and wildcard URLs to destinations shared between multiple tenants.
  • Complying with specific regulations: Private Endpoints indeed may enable some scenarios, regulatory controls, or industry certifications that were not easily achievable in the cloud before PE introduction. Important to note that many regulatory standards do not take cloud infrastructure into account, but thanks to the ambiguity of their language, may allow pretty wide interpretations. Always refer to the original standard wording to find out if it really mandates you to use private endpoints (long before they were ever created).

The benefits of Private Endpoints are doubtful in the below scenarios:

  • Blindly following generic recommendations: Implementing Private Endpoints based on generic recommendations, “best practices,” benchmarks, or secure scores without considering your specific usage scenario can lead to increased system complexity, fragility, and higher maintenance costs. Enforcing Private Endpoints without a valid reason might result in losing cloud capabilities without significant benefits.
  • Relying solely on network security: In the zero-trust security model, it’s assumed that breaches can occur at any point in the data flow. Trusting the network to provide security along the data path is not recommended. While selective traffic routing, private networks, and firewalls offer some protection, they may not be sufficient in hybrid environments spanning multiple data centers, clouds, and VPN devices. Instead, comprehensive security measures like end-to-end encryption, authentication, and endpoint protection should be employed.
  • Providing access to corporate resources: Private Endpoints may not be the most efficient choice for providing access to corporate resources over a VPN or a weak WAN link. VPNs and local WAN links can introduce bottlenecks when accessing the resources on private endpoints. Leveraging public endpoints can utilize the Microsoft backbone (traffic managers, front door gateways, CDNs) for a more efficient data path.
  • Access across multiple Azure regions in a Hub-and-Spoke model: Private endpoints for the same resource in multiple regions or VNets can introduce significant complexity, especially concerning name resolution. This complexity can outweigh the benefits in scenarios where resources need to be accessed across various Azure regions or during failover situations.
  • Cloud-native application design: In cloud-native application design, where the goal is to leverage PaaS services extensively, implementing private endpoints for “grounding” interactions between PaaS components can add unnecessary complexity and costs. VNet integration options for PaaS services often come at premium price points.

A note on data exfiltration

The most popular reason for enforcing private endpoints is “to prevent data exfiltration”. While I agree that this could be a valid requirement, I need to show how challenging it is to implement it right.

While reducing exfiltration vectors may help prevent exfiltration attempts from unprofessional users, it requires much more effort to combat real professionals.

Relying solely on private endpoints for storage and key vaults and blocking outbound access to public endpoints may create a false sense of security.

There are other exfiltration vectors you need to consider as well:

  • DNS: Very few systems can operate without using DNS. If you allow the resolution of public DNS domains, a knowledgeable attacker may gradually export your data. You can use custom DNS servers to try to prevent this. Still, an attacker may use the platform’s magic IP 168.63.129.16 if it is not blocked.
  • Platform services like Azure AD, AVD, Monitoring and Backup are multi-tenant in nature and require opening wide ranges of destinations shared between customers.
  • Productivity tools like Microsoft 365 and even using the Azure Portal require opening almost a half of the Internet.
  • System updates, KMS activation, CRL endpoints, and time services can also be potentially exploited.

Mitigating all of these vectors becomes a highly complex task, limiting the capabilities of your workload in the direction of a disconnected server stored in a safe.

I hope this convinces you that effectively protecting cloud systems from data exfiltration by a professional adversary is challenging, and systems that genuinely require this level of protection should be treated separately from general workloads. Trying to implement the highest security level for all servers as a general rule may not be worth the effort and may result in missed opportunities.

Recommendations for Endpoint Types by Azure Service

Considering that each company may have a unique environment and specific requirements, it is valuable to document the guidance regarding the use of endpoint types tailored to particular scenarios.

Service Recommendation
Storage Account

Public Endpoints for publicly accessible storage, such as diagnostic data, Terraform state files, or exchanging information with external parties.

Service Endpoints for storage shared across VNets and/or regions.

Private Endpoint for private data for apps in high-security zone; for file shares when migrating data from on-premise.

Azure KeyVault

Note: The security of Key Vault is not significantly improved by using Private Endpoints since Key Vault access is protected with Azure AD authentication. Key Vault is not a practical target for data exfiltration, and Key Vault clients still require outbound access to AAD.

Public + Service Endpoints for shared Key Vaults used by Automation / DevOps / Terraform / PaaS / Admins.

Private Endpoints for app-specific KVs deployed on-demand for storing private data, like encryption keys.

Azure SQL Database

Public Endpoints for cloud-native PaaS apps not integrated into VNet.

Private Endpoints for private databases when used from IaaS servers.

..and so on..

Discussion

You are very welcome to discuss this topic in Comments.