Azure Network Security Modes
In this post, I challenge the traditional concept of network segmentation and suggest a model that is better suited for Azure: Network Security Modes.
Started as an effort to debunk outdated concepts, I ended up giving away my principles for designing network architecture, which I have been refining for the last 10+ years.
I keep it abstracted to only one step away from a complete LLD that you can reuse to build a modern Hub-and-Spoke network topology in Azure.
Network segmentation rationale
Ancient times
In the old days, when servers lived in on-premise datacenters, there was a concept of network segmentation.
Hubs and Switches
Servers were connected by hubs and switches.
Every server connected to the same network hub was receiving the same datagrams, including ones not addressed to them. These hubs were also called repeaters.
Network switches are helping to reduce the number of situations where datagrams are repeated on multiple ports when the switch “remembers” who is on the port. It is still not perfect at remembering everyone, especially in big networks.
Subnets
And networks were big. You remember subnet classes A/B/C? C means 256 addresses. It was normal to have hundreds and thousands of servers connected via hubs and switches, and they formed subnets or broadcast domains.
To reiterate: any traffic in the subnet is broadcast and can be overheard by any interface on that subnet.
Routers are used to interconnect and remove broadcast traffic between subnets, allowing only traffic destined for these subnets.
Limiting broadcast traffic is good, but routers are expensive. That’s why subnets were big and had many servers.
Traditional routers were not as smart as today’s, so they could not filter ports, protocols, and other attributes that security guys like.
Firewalls are used to filter traffic and are even rarer and more expensive than routers.
It was so expensive that it was only placed on critical network boundaries, like dividing internal networks from DMZ and internets.
The cost and rarity of firewalls forced companies to group their servers into network segments, where everyone trusts each other.
Network segmentation
At that time, network traffic was rarely encrypted – everyone in the subnet could potentially hear all network conversations.
Operating systems like Windows 2000 did not have a personal firewall, so all ports were open by default – making them vulnerable to any bad guy or a virus that is on the network
Authentication protocols and passwords were weak (and no MFA!) – to the pleasure of hackers.
These aspects led to the necessity of making sure that all neighbours in the networks are trusted.
Here comes the concept of a network segment – a part of a network that groups servers with a similar level of trust together, protected by a single firewall.
Typical network segments were Prod / Dev / Workplace / DMZ / Internet, and all devices within the same segment were treated equally.
Modern times
Nowadays, when you hear the term network hub, it does not mean repeater, but a central place connecting multiple virtual networks where routing and traffic filtering happen.
Virtual Networks in Azure run in a shared, multitenant environment of the cloud provider, and
- network virtualisation protocol ensures that traffic is not visible to other customers – no bad neighbours in the subnet can overhear your conversations
- traffic filtering platform is processing every datagram – firewall capabilities are available on every connected interface via NSG
More than that,
- modern operating systems have firewalls enabled by default (and it’s a shame if your company disable them as a standard)
- modern network protocols keep traffic properly authenticated (MFA/SSO/SAML/OAuth – all these beauties) and encrypted (TLS/AES/oh-yes) from end-to-end
This makes the concept of network segmentation mostly irrelevant for Azure virtual networks since the issues that segmentation was initially addressing no longer exist.
Henceforth, I declare the traditional network segmentation obsolete.
Azure Network Security Modes
New times are coming, security guys stay the same.
They still want to control and see the traffic on a firewall and don’t want to rely on zero-trust principles.
To please them, I want to introduce you to a modern way of organising network filtering, which I call Azure Network Security Modes.
Mode 1, aka Cloud Native
this is when you implement Virtual WAN in default configuration
- Traffic between VNets is routed and filtered by the Hub Firewall
- Firewal is managing only core security rules, like filtering connectivity between spokes, WAN, and internets, of course
- Traffic between subnets is routed directly by the virtual network and filtered on the NSG
- Only incoming rules are used in the NSG.
- Outgoing traffic is not controlled in the NSG
- Traffic inside the subnet is routed directly by the virtual network and can be optionally filtered on the NSG
- FW and (optionally) NSG Flogs are streamed to Log Analytics
Mode 2, aka Traditional
this is when you mimic a traditional datacenter
- Traffic between VNets is routed and filtered by the Hub Firewall
- Firewall is still managing core security rules
- Traffic between subnets is routed and filtered by the Hub Firewall, too
- Firewall is additionally filtering traffic between subnets, becoming an additional point of failure, latency, and costs
- Traffic inside the subnet is routed directly by the virtual network and can be optionally filtered on the NSG
Mode 3, aka Micro-Segmented
this is where you go to please some insane regulations
- Traffic between VNets is routed and filtered by the Hub Firewall
- Traffic between subnets is routed and filtered by the Hub Firewall
- Traffic inside the subnet is routed and filtered by the Hub Firewall
- All work and no play makes Firewall a dull boy: a point of failure, latency, and costs for all traffic in the subnet
Comparison
Here is the comparison table:
| Criteria / Mode | Mode 1.
Firewall between VNets "Inter-VNet FW" (Cloud Native) |
Mode 2.
Firewall between Subnets "Inter-Subnet FW" (Traditional) |
Mode 3.
Firewall between NICs "Intra-Subnet FW" (Micro-Segmented) |
|---|---|---|---|
| Traffic Filtering | |||
| VNet to VNet | via FW | via FW | via FW |
| Subnet to Subnet | via NSG | via FW | via FW |
| Inside Subnet | via NSG | via NSG | via FW |
| Cost Impact | Low/Med – NSGs are free; however, Flow Logs costs should be considered. | Medium – Costs to FW traffic and scaling | High – More costs to FW traffic and scaling |
| Management Efforts | • High – if manual management of traffic filtering (both FW and NSG) is centralised to the Network team. • Med – if automated end-to-end via code • Low – if NSG management is delegated to applications team (but requires a level of trust) |
• Med/Low – centralisation of filtering management to FW reduces management efforts, provided inter-subnet filtering is managed by app teams | • Med/High – need to filter and monitor all traffic increases management efforts |
| Routing | Simple, standardised UDR for all subnets | Unique UDR for each subnet with a few Routes | Simple, standardised UDR for all subnets |
| Throughput and Latency | • Max throughput, minimum latency | • As platform-native for traffic inside Subnets • Limited by FW for traffic between subnets |
• Limited by FW performance and scale • May not be supported by some high demanding apps |
| Log Management Costs and Efforts | Med – Fragmented log collection on NSG and FW, however, may stream to the same Log Analytics workspace | Low – Centralised log collection on FW, low volumes if NSG logs are not collected | Low – Centralised log collection on FW, but high volumes | Reliability | High – FW is only a point of failure for traffic between VNets | Medium – FW becomes a point of failure between subnets | Low – FW is PoF for any traffic |
| Use cases | • VNets for shared platform components that can protect themselves with NSGs or personal firewalls • VNets dedicated to a single app or groups of mutually trusted apps (like SAP landscapes) • VNets with a decentralised management model (where apps are managing their own NSGs) |
• Shared VNets, hosting multiple apps with centralised security management (where a central team is managing communications between apps) | • Special zones requiring maximum security (should only be used where absolutely required by regulations) |
What model should you use?
The comparison table above provides the pros and cons; details are in the implementation sections below.
I could tell you to decide… consider… blah-blah… for yourself, but… I numbered the modes in the order of my personal preference:
- Use Mode 1 by default
- Use Mode 2 when you have to
- Try not to use Mode 3
- Mix the modes in the same VNet if you think that your life is too easy
Be creative in meeting regulatory requirements and responding to audit reports written by someone who does not care how you should live with this.
Implementation
Here is some example technical documentation for implementing the security modes described above.
Diagram
The diagram from the page cover was used to draw attention. It does not completely reflect the architecture described here (but it is still better than a generic AI-generated picture). Once I update it, I will post the diagram in a higher resolution here.
VNet Topology
Traditional “Regional Hub-and-Spokes”, wherein each region has a dedicated Hub VNet with a Firewall used for traffic routing and filtering, and several spoke VNets hosting workloads are attached via peerings.
| Region | Address Spaces | Address Range | Remarks |
|---|---|---|---|
| West Europe | 10.0.0.0/10 | 10.0.0.0 – 10.63.255.255 | It is always recommended to pre-allocate simpler, reasonable chunks of IP space to specific regions… |
| North Europe | 10.64.0.0/10 | 10.64.0.0 – 10.127.255.255 | … This makes routing tables simpler and helps to avoid unnecessary complexity and hitting limits for the number of routes… |
| New region | 10.128.0.0/10 | 10.128.0.0 – 10.191.255.255 | … As you need to provision new spokes, you cut IP addresses from these pre-allocated ranges… |
| Free | 10.192.0.0/10 | 10.192.0.0 – 10.255.255.255 | … But always keep free space for potential growth… |
I might write a separate article on efficient IP range allocation. However, special tools for IPAM exist, like Efficient IP and Azure IPAM.
VNet Structure
A spoke VNet should be used as a security boundary, and Security mode should be assigned to the VNet at its creation.
Because VNets cannot span across Azure subscriptions, the VNets structure should follow the subscription structure of the company.
This means separate VNets could exist for company business units, service lines, and environments.
| VNet Name | Address Space | Region | Subscription | Comments |
|---|---|---|---|---|
| Hub-vnet | 10.0.0.0/24 | West Europe | Hub-sub | Regional Hub |
| SAP-vnet | 10.1.0.0/16 | West Europe | SAP-sub | Mode 1: VNet is dedicated to SAP application landscapes that trust each other |
| Shared-vnet | 10.2.0.0/16 | West Europe | Shared-sub | Mode 2: VNet is hosting multiple apps managed by different teams |
| Secret-vnet | 10.3.0.0/16 | West Europe | Secret-sub | Mode 3: VNet is hosting apps that need to comply with insane regulations |
Once assigned, security mode sets the default level for the workload provisioned in the spoke. Since we are in the cloud with software-defined networks, the flexibility of changing this is possible for some subnets or even for the whole VNet, but it is not possible to change the VNet name. That’s why I don’t encourage “tattooing” the initial network security mode in the VNet name.
Subnets Structure
Separate subnets should be used only when necessary to simplify management or implement security requirements like different route tables and NSGs.
In Native and Traditional security modes,
- Separate subnets can be used to delegate NSG control to specific application owners
In MicroSeg security mode,
- Network security may not rely on subnets; big flat subnets can be used in conjunction with centrally managed NSGs and application-specific ASGs (details below)
| VNet Name | Subnet Name | Address Space | Comments |
|---|---|---|---|
| SAP-vnet | 10.1.0.0/16 | Apps in this VNet are managed by the same team and use NSG/ASG | |
| SPM | 10.1.0.0/24 | ||
| GRC | 10.1.2.0/25 | ||
| CRM | 10.1.3.0/25 | ||
| Shared-vnet | 10.2.0.0/16 | Apps in this VNet are managed by different teams who do not trust each other | |
| App1 | 10.2.0.0/24 | App 1 | |
| App2 | 10.2.1.0/24 | App 2 | |
| App3_FE | 10.2.2.0/24 | App 3 Front-end | |
| App3_BE | 10.2.3.0/24 | App 3 Back-end | |
| App4_FE | 10.2.4.0/24 | App 4 Front-end | |
| App4_BE | 10.2.5.0/24 | App 4 Back-end | |
| Secret-vnet | 10.3.0.0/16 | Apps in this VNet have to comply with strict industry regulations | |
| PCIDSS | 10.3.0.0/24 | ||
| HIPAA | 10.3.1.0/24 |
Traffic Routing
All traffic routing to and from any Spoke Vnet is traversing the regional Hub FW:
- Between Spokes in the same region connected via peerings
- Between Spoke in the region and Virtual Network Gateways (Express Route, VPN, Other Regions connected to the same ER circuits)
- Between Spoke and the Internet
- Between Spoke and Hubs in other regions connected via global peerings
This requires each spoke VNet to have an assignment of Route tables (UDR) containing:
- BGP route propagation = disabled
- Single route to 0.0.0.0/0 via Hub NVA
For filtering traffic on FW between subnets, the route table should also contain:
- Route to current VNet address space via Hub NVA
To enable direct traffic inside the subnet, the route table should also contain:
- Route to the current Subnet address space via Virtual Network
Here is the configuration example:
| UDR Name | Assigned to | Routes | Comments |
|---|---|---|---|
| SAP-vnet-udr | SAP-vnet / *Subnets | 0.0.0.0/0 → NVA IP (Firewall) | To override system route pointing to Internet |
| Shared-vnet-Subnet_App1-udr | App1 | 0.0.0.0/0 → NVA IP (Firewall) | To override system route pointing to Internet |
| App1 | 10.2.0.0/16 → NVA IP (Firewall) | To override system route to VNet Address Space pointing to Virtual Network | |
| App1 | 10.2.0.0/24 → Virtual Network | To override route above to enable direct communication for interfaces in App1 subnet | |
| Shared-vnet-Subnet_App2-udr | App2 | 0.0.0.0/0 → NVA IP (Firewall) | To override system route pointing to Internet |
| App2 | 10.2.0.0/16 → NVA IP (Firewall) | To override system route to VNet Address Space pointing to Virtual Network | |
| App2 | 10.2.1.0/24 → Virtual Network | To override route above to enable direct communication for interfaces in App2 subnet | |
| Secret-vnet-udr | Secret-vnet / *Subnets | 0.0.0.0/0 → NVA IP (Firewall) | To override system route pointing to Internet |
| Secret-vnet / *Subnets | 10.3.0.0/16 → NVA IP (Firewall) | To override system route to VNet Address Space pointing to Virtual Network |
As you see, to implement Mode 2, you need a separate UDR resource for each subnet in the VNet, which complicates management unless it is fully automated.
To implement modes 1 and 3, a single route table object per VNet is enough.
Route table content can be enforced or checked for compliance with Azure Policies.
Security Zones
In some complex situations where you need flexibility to bypass firewall filtering between endpoints in different subnets, but keep filtering for any other subnets, you might need to implement a concept of security zones.
A Security Zone consists of subnets with the same level of trust. A security zone can be represented by a single subnet, a set of subnets in the same VNet, or even a set of subnets from different VNets (in rare cases, where it is required to peer VNets directly).
- The traffic within the same security zone can travel directly between VMs, bypassing the firewall; however, it can still be controlled on NSGs
- The traffic between different security zones should pass through the firewall for inspection.
For this to work, route tables should be configured for a combination of the security modes described.
We might call it Mode 1.5, in which some subnets are routed directly, but other subnets are still routed through a firewall.
I don’t encourage using this mode, but I describe it here to demonstrate the beauty of SDWAN flexibility.
In a nutshell, a VNet with a single security zone (all subnets equal) becomes Mode 1, and a VNet with security zones equal to the number of subnets becomes Mode 2.
I recommend using either mode 1 or 2 and keeping the option for Security zones for special occasions that might never happen.
| UDR Name | Assigned to | Routes | Comments |
|---|---|---|---|
| Shared-vnet_Zone_App3-udr | App3_FE App3_BE |
0.0.0.0/0 → NVA IP (Firewall) | To override system route pointing to Internet |
| App3_FE App3_BE |
10.2.0.0/16 → NVA IP (Firewall) | To override system route to VNet Address Space pointing to Virtual Network | |
| App3_FE App3_BE |
10.2.2.0/16 → Virtual Network 10.2.3.0/24 → Virtual Network |
To override route above to enable direct communication between App3_FE and App3_BE | |
| Shared-vnet-Zone_App4-udr | App4_FE App4_BE |
0.0.0.0/0 → NVA IP (Firewall) | To override system route pointing to Internet |
| App4_FE App4_BE |
10.2.0.0/16 → NVA IP (Firewall) | To override system route to VNet Address Space pointing to Virtual Network | |
| App4_FE App4_BE |
10.2.4.0/16 → Virtual Network 10.2.5.0/24 → Virtual Network |
To override route above to enable direct communication between App1_FE and App1_BE |
Filtering principles
Core rules at Firewall approach
In all cases, the regional firewall should enforce core security rules, such as filtering traffic between spokes, WAN (other regions and on-prem), and internets.
Pay special attention to filtering inbound and outbound internet traffic.
Control at destination approach
Besides core / foundational rules, like internet outbound, I recommend an approach where
- apply general traffic filtering at the destination (on FW or NSG) as inbound rules (app owners should know better who they want to let in)
- avoid applying filtering at the source as outbound rules (unless you don’t trust the app owners)
Doing this will give you a single control point without opening flows twice: at sources and destinations.
Firewall configuration
For spokes in Native security mode:
- Any undefined inbound and outbound traffic is allowed for this spoke VNet (but still managed on NSG)
- The traffic between the subnets of this spoke VNet is not passing through the firewall
For spokes in Traditional security mode:
- Any undefined inbound traffic is denied for this spoke VNet
- Outbound traffic from this spoke VNet is allowed by default
- Explicitly defined Inbound traffic is allowed
- The traffic between the subnets of Spoke VNet is passing through the firewall and should be explicitly opened
- Any traffic within the same subnet of this spoke VNet is not passing through the firewall (but still can be managed on NSG)
For spokes in MicroSeg security mode:
- Any undefined inbound and outbound traffic is denied for this spoke VNet
- Any traffic in the subnets of this spoke VNet is passing through the firewall and should be explicitly opened (no point to manage on NSG, unless required)
Here is an example of a firewall policy configuration to enable this model:
| Rule Collection Group | From | To | Settings |
|---|---|---|---|
| Core outbound - Local | {All connected spokes} | {Local ranges} | Permitted core services, like AD, DSN, NTP, etc |
| Core outbound - Internet | {All connected spokes} | {Internet} | Safe Internet destinations, like Azure Portal, Windows Update, etc |
| Mode 1 - inbound | * | {All spokes in Mode 1} | Any traffic – not filtered on firewall, managed on NSG |
| Mode 1 - outbound | {All spokes in Mode 1} | * | Any traffic – not filtered on firewall, managed on NSG |
| Mode 2 - App1 - outbound | App1 Subnet | {Internet} | Rules for outbound traffic from App1 |
| Mode 2 - App1 - inbound | * | App1 Subnet | Rules for inbound traffic from App1 |
| Mode 2 - App1_to_App2 | App1 Subnet | App2 Subnet | Rules between App1 and App2 |
| Mode 3 - NIC1 - inbound | NIC1 IP | {Internet} | Rules for outbound traffic from NIC1 in Secret Vnet |
| Mode 3 - NIC1 - outbound | * | NIC1 IP | Rules publishing specific services on NIC1 in VNetVNet |
The rules above are for example only. You need to carefully consider the rules’ order to ensure that rules opening from Mode 1 do not affect traffic in Modes 2 and 3, since the same firewall is used for all networks.
This also shows the benefits of using NSG when you cannot fully trust the firewall configuration.
Network Security Groups
In Traditional and Native security modes,
- NSG management can be delegated to application owners so they can control their applications’ traffic independently
- NSGs are attached at the Subnet level. Attaching NSGs to individual NICs is not encouraged due to overkill management complexity and overlapping functionality with personal firewalls.
- If required, NSGs can be managed centrally via code or 3rd party solutions like AlgoSec
- Use of ASG is encouraged to simplify and organise NSGs, where possible (several services like load balancer IPs are not able to use ASG)
In MicroSeg Security Mode,
- Use of NSGs does not make a lot of sense, since the Firewall already filters all traffic, unless you need additional protection and logging on top of the Firewall
Here is an example of NSGs’ configuration to enable this model:
| NSG | Source | Destination | Settings | Remarks |
|---|---|---|---|---|
| SAP-vnet-nsg | {Local ranges} | * | Allow In TCP_3389 | Opening flow to all destination in the VNet |
| * | 10.1.2.0/24 | Allow In TCP_443 | Opening flow to GRC subnet | |
| {Some Trusted Source} | {SAP_DB-asg} | Allow In TCP_1433 | Opening flow to specific ASG | |
| * | * | Deny In * | Deny all not explicitly open inbound traffic | |
| Shared-vnet_Subnet_App1-nsg | {Local ranges} | 10.2.0.0/24 | Allow In TCP_3389 | Opening flow to App1 subnet |
| Shared-vnet_Subnet_App2-nsg | {Local ranges} | 10.2.1.0/24 | Allow In TCP_22 | Opening flow to App2 subnet |
| Secret-vnet-nsg | {empty} | No point managing NSG for Mode 3 as everything is controlled on FW |
Application Security Groups
In Traditional and Native security modes:
- Use of ASG is encouraged to simplify and organise NSGs, where possible (several services like load balancer IPs are not able to use ASG)
- ASG may also be used to delegate some freedom to application teams, while NSG management is centralised and not in their control
In MicroSeg security mode:
- ASG may enable some flexibility when NSGs are still used in addition to the firewall
Here is an example of ASGs’ configuration of this model:
| Network Interface Config | Member of ASG |
|---|---|
| SAP-GRC-DB01-nic SAP-GRC-DB02-nic SAP-CRM-DB01-nic SAP-CRM-DB02-nic |
SAP_DB-asg |
| SAP-GRC-FE01-nic SAP-GRC-FE02-nic SAP-CRM-FE01-nic SAP-CRM-FE02-nic |
SAP_FE-asg |
Discussion
This page ended up much bigger than I initially expected, and might need polishing.
As usual, I’m happy to discuss this topic with you in LinkedIn comments. If you want to support this post to get more exposure, please use this link to react, comment, or repost.