making hybrid cloud a little less cloudy
Teeling Cloud Courant is a running commentary on hybrid cloud reality — especially Microsoft Azure Local, Azure Arc, and the connective tissue between on‑premises and cloud. I focus on these topics because they sit at the intersection of architecture, operations, and real‑world constraints — the place where Microsoft's intentions meet what customers can actually deploy. While much of the content centers on Microsoft's hybrid ecosystem, TCC isn't limited to a single vendor. Hybrid cloud is bigger than any one platform, and the publication occasionally features broader perspectives and guest contributors. If you work with hybrid cloud, Azure Local, or modern datacenter design, this publication keeps you current without drowning you in noise.
A recovery path when your datacenter network overlaps with Microsoft's reserved ranges
If you're already mid-deployment or can't renumber your network, there's a supported workaround that most customers don't know about.
Why dense NVMe nodes change the networking conversation sooner than most designs expect
As NVMe becomes the default choice for modern hyperconverged infrastructure, one question comes up repeatedly in both Storage Spaces Direct (S2D) and Azure Local designs: When does 100 GbE networking actually make sense?
For many years, 10 GbE — and later 25 GbE — was sufficient for most storage workloads. Even today, dual‑port 25 GbE designs are often considered "more than enough." But once you start building dense all‑NVMe nodes, those assumptions begin to break down.
Critical deadlines for 22H2 and 23H2 environments
Still on 22H2? Running a stretched cluster? Or sitting on 23H2 hoping it buys you time?
This is your wake‑up call — with clear facts and supported directions, not "just wait." Azure Local's 22H2 and 23H2 releases are on borrowed time. 22H2 is already out of support, and 23H2 — including stretched clusters — stops receiving security and quality updates after April 2026.
Why dense NVMe nodes change the networking conversation sooner than most designs expect
As NVMe becomes the default choice for modern hyperconverged infrastructure, one question comes up repeatedly in both Storage Spaces Direct (S2D) and Azure Local designs: When does 100 GbE networking actually make sense? For many years, 10 GbE — and later 25 GbE — was sufficient for most storage workloads. Even today, dual‑port 25 GbE designs are often considered "more than enough." But once you start building dense all‑NVMe nodes, those assumptions begin to break down. This article explains why 100 GbE becomes justified earlier than expected in all‑NVMe S2D and Azure Local deployments, why roughly 12 NVMe drives per node is a practical, conservative inflection point, how the economics of 4 × 25 GbE versus 2 × 100 GbE work out in real designs, and why the same logic applies across vendors like Dell and Cisco.
Before going further, it's important to anchor the discussion in realistic design practice:
No serious production S2D or Azure Local design uses a single storage network port. All scenarios discussed here assume:
This reflects real‑world Azure Stack HCI and Azure Local designs and avoids straw‑man comparisons like "100 GbE vs a single 10 GbE link."
A single enterprise NVMe drive is capable of multiple gigabytes per second of throughput. When a node contains 12, 16, or 24 NVMe drives, the aggregate storage capability becomes enormous. In Storage Spaces Direct, all east‑west storage traffic — including mirrored writes — flows over the network using SMB 3 with RDMA. Azure Local uses the same underlying S2D mechanisms for its local storage.
As NVMe density increases, the limiting factor stops being the media and quickly becomes available network bandwidth. At that point:
VMFleet, Microsoft's official S2D performance and validation tool, is designed to generate highly parallel I/O, stress storage, CPU, and network simultaneously, and expose architectural bottlenecks rather than idealized peaks. In all‑NVMe S2D clusters, VMFleet runs consistently show that nodes can sustain well over 5 GB/s of storage throughput, network links saturate before NVMe does, and scaling flattens once aggregate NIC bandwidth is consumed. This behavior appears even when SMB Multichannel is active and multiple NICs are available. Azure Local, running on the same S2D foundations, exhibits the same pattern when you push it with dense NVMe and realistic I/O mixes.
With 2 × 25 GbE, SMB Multichannel allows traffic to flow across both links concurrently. This meaningfully increases available bandwidth compared to a single port, but it's not a perfect doubling.
It's crucial to understand what SMB Multichannel does — and does not — provide:
As a result, dual‑port 25 GbE does not behave like a single flat 50 GbE pipe for storage workloads.
That's a substantial improvement over a single link — but it's also a ceiling that dense NVMe nodes can hit surprisingly quickly.
Once we assume 2 × 25 GbE as the baseline, a clear pattern emerges:
At roughly 12 NVMe drives per node, the node is often capable of driving more throughput than dual‑port 25 GbE can sustain under continuous load. Beyond that point, performance gains from additional NVMe are increasingly constrained by the network. This is the point where 100 GbE stops being "nice to have" and becomes architecturally justified.
Moving to 100 GbE does more than increase peak bandwidth:
With dual‑port 100 GbE:
This is why many all‑NVMe S2D and Azure Local reference architectures move directly to 100 GbE rather than trying to "stretch" 25 GbE with additional ports.
"The hybrid cloud is no longer a compromise—it's the optimal architecture for modern business."
— Core principle in modern HCI design
Once you accept that 2 × 25 GbE will not be enough for roughly 12–16 NVMe per node under sustained load, the design conversation changes:
You are no longer choosing between 25 GbE vs 100 GbE. You are choosing between 4 × 25 GbE and 2 × 100 GbE.
Option A: 4 × 25 GbE per node
Option B: 2 × 100 GbE per node
Even if a 100G optic or port costs more per unit than a 25G equivalent, multiplying the 25G option by four ports per node (and four optics) erodes that apparent savings very quickly. In real designs, the total fabric cost for 4 × 25 GbE versus 2 × 100 GbE frequently lands in the same ballpark, especially once you include support and cabling. In some cases, the 100 GbE option is actually slightly cheaper, while delivering more per‑node bandwidth, fewer TOR ports consumed, fewer points of failure, and a cleaner topology.
If you want a guideline that's easy to communicate and hard to argue with, use this: If a Storage Spaces Direct or Azure Local node can sustain more than ~5 GB/s of storage throughput, 100 GbE networking is justified. Dense NVMe nodes reach that line quickly — often before they hit 16 drives per system. Equivalently: If you are planning for roughly 12 or more NVMe drives per node, design for 100 GbE from day one. Don't assume 2 × 25 GbE will scale with you. This is deliberately conservative. It doesn't require exotic workloads, perfect tuning, or lab‑only conditions. It's based on what real VMFleet tests and production systems actually do under load.
All‑NVMe Storage Spaces Direct and Azure Local fundamentally change where performance bottlenecks appear. Even with SMB Multichannel and dual‑port networking, dense NVMe nodes can saturate available bandwidth earlier than many designs anticipate. Recommending 100 GbE networking for systems with more than about 12 NVMe drives per node is conservative rather than aggressive, architecturally sound based on real tools and behavior, aligned with the economics of 4 × 25 GbE vs 2 × 100 GbE fabrics, and focused on delivering usable performance, not just impressive component specs. If you're investing in dense NVMe, the network needs to keep up — or it will define your limits. In that world, 100 GbE is not a luxury upgrade; it's the rational default once you pass a certain NVMe density.
What You Need to Do Before April 2026
Still on 22H2? Running a stretched cluster? Or sitting on 23H2 hoping it buys you time? This is your wake‑up call — with clear facts and supported directions, not "just wait."
Azure Local (formerly Azure Stack HCI) has moved away from the old annual 22H2 / 23H2 cadence to a monthly release train with versions like 2507, 2508, 2509, and so on. These monthly releases align to a specific OS build (for example, 24H2 = 26100.xxxx) and follow Microsoft's Modern Lifecycle model, which requires customers to stay within six months of the latest release to remain supported. That change is good for features and fixes — but it also means older H2 releases have a hard runway.
So there are three overlapping issues: 1. You're still on 22H2 (possibly with stretched clusters). 2. You're on 23H2, but haven't moved to 24H2 and risk falling out of support in April 2026. 3. You're using stretched clusters and need to understand what replaces them. This post covers all three — at the support and architecture boundary level.
To keep this grounded:
If you're still on Azure Local / Azure Stack HCI 22H2 (20349.xxxx) today:
Remaining on 22H2 means:
If you're on 22H2:
That restores:
"22H2 is already out of support. Don't wait for April 2026 — upgrade now."
— Microsoft Support guidance
If you're running 23H2 OS (25398.xxxx), you're in better shape — but only temporarily. 11.2510 (October 2025) was the final 23H2 solution release. All 23H2 configurations are supported only until April 2026 After April 2026:
If the solution is already installed:
Azure Local follows a Modern Lifecycle cadence: you must stay within six months of the most recent release to remain supported.
Some environments upgraded the OS but never finished the solution upgrade.
Microsoft explicitly treats this as a temporary state and expects you to:
Sitting in OS‑only 23H2 as April 2026 approaches is not a safe long‑term position.
Microsoft's generic OS‑upgrade guidance includes this statement: "There is an exception to the preceding recommendation if you are using stretch clusters. Stretch clusters should wait to directly move to version 26100.xxxx (24H2)." This sentence is easy to misread. Context for stretched clusters: You have effectively been on borrowed time. Microsoft accommodated stretched clusters on 23H2 for customers that already implemented them on 22H2, but that accommodation ends when 23H2 support ends in April 2026. In practical terms:
Your next supported state must be a non‑stretched 24H2 design, delivered through deployment and migration, not in‑place conversion.
Rack‑aware clustering introduces a strict architectural constraint:
≤ 1 millisecond round‑trip latency between racks or zones
Rack‑aware clustering is intended for:
Rule of thumb:
Rack‑aware clustering is supported only for new deployments. Converting an existing standard or stretched cluster in place to rack‑aware is not supported. Migration requires a side‑by‑side deployment and workload move. If your current "stretched" design is actually two racks or rooms in the same datacenter with sub‑millisecond latency, rack‑aware clustering is often the simpler and more future‑proof replacement. If your sites are truly separate, rack‑aware is not an option.
When changing cluster topology, VM type matters.
If you move or replicate the underlying VM to another cluster:
You can still deploy the Azure Connected Machine (Arc) agent in the guest OS for OS‑level management, but that does not reconstitute it as an Azure Local‑managed VM. This is a design decision, not a migration footnote.
Accept that you're already out of support. Use the direct 22H2 → 24H2 OS upgrade. Complete the Azure Local solution upgrade.
Treat April 2026 as a real deadline. Keep taking solution updates to land on 24H2 (26100.xxxx). If you're OS‑only, finish the solution upgrade now.
Recognize that stretched clusters are supported only through April 2026. Plan for a non‑stretched 24H2 architecture. Expect this to involve deployment and migration, not conversion.
Azure Local's 22H2 and 23H2 releases are on borrowed time. 22H2 is already out of support, and 23H2 — including stretched clusters — stops receiving security and quality updates after April 2026. Microsoft now operates Azure Local on a monthly release train based on the 24H2 OS (26100.x). To remain supported, environments must move to 24H2 and the current solution train. This transition is also the point at which stretched cluster designs must be replaced with supported non‑stretched architectures, such as rack‑aware clusters for low‑latency single‑site deployments or independent clusters with replication for multi‑site scenarios.
A recovery path when your datacenter network overlaps with Microsoft's reserved ranges
Microsoft's documentation for Azure Local includes a strict requirement:
Two IP ranges must not be used anywhere in your Azure Local deployment:
These ranges are reserved for internal Azure Local and AKS Arc components. If your existing datacenter network uses any part of these blocks — and many do — deployment can fail or behave unpredictably.
This is especially common with 10.244.0.0/16, which appears in many legacy Kubernetes clusters, lab environments, and inherited network designs.
Microsoft's guidance is simple: don't use these ranges.
But real customer networks are rarely simple.
Fortunately, there is a clean, repeatable workaround.
Azure Local enforces reserved‑range validation during deployment. But Arc Resource Bridge (ARB) — the component that hosts AKS on Azure Local — can be safely deleted and re‑created afterward.
This gives you a path forward:
This allows you to avoid the reserved ranges and avoid redesigning your entire network.
Start by removing the Arc Resource Bridge that was automatically created during Azure Local setup.
Using PowerShell or the Azure CLI:
Remove-AksHci -ArcResource -Force
Or via Azure CLI for Arc-managed resources. The operation will clean up the associated Kubernetes cluster and all its networking configurations.
Once removed, you can deploy a new AKS Arc instance with custom networking parameters:
New-AksHci -Name` -ServiceCidr ` -PodCidr ` -DnsServiceIP
Example with custom ranges:
New-AksHci -Name azlocal-aks ` -ServiceCidr 172.20.0.0/16 ` -PodCidr 172.21.0.0/16 ` -DnsServiceIP 172.20.0.10
Replace the ranges with whatever fits your environment, as long as they don't collide with your existing infrastructure.
The DNS service IP must always end in .10 — The specific octet is non‑negotiable for AKS Arc.
The service CIDR and pod CIDR can be anything that fits your environment, as long as they don't overlap with the rest of your network.
This process modifies only the ARB/AKS layer — not your Azure Local deployment itself. Your VM hosts, storage, and fabric remain unchanged.
Azure Local enforces reserved ranges during deployment because ARB and AKS rely on them internally. But once the system is deployed, you can override the Kubernetes networking configuration by re‑creating the AKS Arc instance.
This gives customers with constrained or inherited networks a viable path forward without renumbering DNS servers, proxies, gateways, or entire IP blocks.
It's not documented. It's not obvious. But it's fully supported because you're using standard lifecycle operations.
Use this approach when:
It's also a great recovery path when the initial deployment fails due to IP conflicts and you want to avoid tearing everything down.
Azure Local's reserved IP ranges are strict — but they don't have to be blockers.
By deploying first and re‑creating the Arc Resource Bridge with your own CIDRs, you can run Azure Local in networks that would otherwise be incompatible.
This is one of those real‑world hybrid‑cloud challenges that customers hit all the time. Now you've got a clean, repeatable way to handle it.