How to Design Azure Kubernetes Service (AKS) Cluster

Author

Govardhana M K
November 20, 2025

In partnership with

TechOps Examples

Hey — It's Govardhana MK 👋

Welcome to another technical edition.

Every Tuesday – You’ll receive a free edition with a byte-size use case, remote job opportunities, top news, tools, and articles.

Every Thursday and Saturday – You’ll receive a special edition with a deep dive use case, remote job opportunities, and articles.

  • Kubernetes scaling is unpredictable. DevOps Pros are under fire.

  • Kedify launched a practical Kubernetes Autoscaling guide for free.

  • Practical solutions. Rewritten for the AI era.

  • Test ideas faster. Find what works for free.

👀 Remote Jobs

📚️ Resources

TOGETHER WITH THE DEEP VIEW

Become An AI Expert In Just 5 Minutes

If you’re a decision maker at your company, you need to be on the bleeding edge of, well, everything. But before you go signing up for seminars, conferences, lunch ‘n learns, and all that jazz, just know there’s a far better (and simpler) way: Subscribing to The Deep View.

This daily newsletter condenses everything you need to know about the latest and greatest AI developments into a 5-minute read. Squeeze it into your morning coffee break and before you know it, you’ll be an expert too.

Subscribe right here. It’s totally free, wildly informative, and trusted by 600,000+ readers at Google, Meta, Microsoft, and beyond.

Looking to promote your company, product, service, or event to 57,000+ Cloud Native Professionals? Let's work together. Advertise With Us

🧠 DEEP DIVE USE CASE

How to Design Azure Kubernetes Service (AKS) Cluster

Before you design an Azure Kubernetes Service cluster, it helps to step back and understand how Kubernetes itself is structured.

At its core, Kubernetes separates responsibilities between the control plane and the worker nodes. The control plane hosts the API server, scheduler, controller manager, and etcd. These components maintain the desired state of the cluster and make decisions on scheduling, orchestration, and cluster health.

Worker nodes run kubelet, kube proxy, and the container runtime which together handle pod execution, service routing, and communication with the control plane.

In short, The control plane manages the cluster state while the worker nodes focus on running workloads.

With that foundation clear, we can now look at how to design an Azure Kubernetes Service cluster in a way that reflects real production needs.

Cluster Architecture Design

1. Pick how your workloads will run

Decide what actually runs inside your cluster. For example:

  • API services on general purpose nodes

  • Background workers on CPU heavy nodes

  • ML or video apps on GPU nodes

  • Low priority jobs on spot nodes

This directly defines how many node pools you need.

2. Split system and application capacity

  • Create a small system pool for kube system components.

  • Create user pools only for application pods.

This avoids situations where app pods starve DNS or CNI and break the cluster.

3. Choose node sizes with real numbers

Select sizing based on known workload patterns. Examples:

  • API workloads: D4/D8 series

  • Memory heavy: E series

  • High IOPS: L series

  • GPU: NC/ND series

Keep nodes consistent within each pool to avoid unpredictable scaling.

4. Design how the cluster scales

Enable autoscaler on every pool:

  • Use HPA for CPU or memory based workloads

  • Use KEDA for queue length, HTTP rate, or event driven scaling
    Keep at least one extra node of headroom per pool for safe upgrades.

5. Decide how you isolate workloads

Pick one of these:

  • Single AKS per environment (most common)

  • Shared cluster with strict namespace and policy boundaries

Add taints and tolerations to keep workloads on the right node pools.

6. Plan storage per workload, not per cluster

Choose storage based on use case:

  • Azure Disks for databases or single writer apps

  • Azure Files for shared read/write

Keep databases on managed services unless you have a strong reason not to.

7. Map identity and secrets to how apps run

  • Cluster uses managed identity to talk to Azure resources

  • Apps use workload identity for their own Azure access

  • All secrets come from Key Vault, not from raw Kubernetes secrets

Networking is the most crucial aspect which needs special attention and which we would look next.

🔴 Get my DevOps & Kubernetes ebooks! (free for Premium Club and Personal Tier newsletter subscribers)

Upgrade to Paid to read the rest.

Become a paying subscriber to get access to this post and other subscriber-only content.

Already a paying subscriber? Sign In.

Paid subscriptions get you:

  • • Access to archive of 250+ use cases
  • • Deep Dive use case editions (Thursdays and Saturdays)
  • • Access to Private Discord Community
  • • Invitations to monthly Zoom calls for use case discussions and industry leaders meetups
  • • Quarterly 1:1 'Ask Me Anything' power session