Skip to content
All articles

Kubernetes + Istio Service Mesh, Explained

Dean Jain

Dean Jain

Senior Staff Software Engineer · Enterprise AI, Data & Cloud Architect

· 5 min read

KubernetesIstioService MeshInfrastructure
---
config:
  theme: neutral
  fontSize: 16
---
flowchart TB
    CP["Control plane<br/>API server · scheduler · controllers · etcd"]:::gov
    subgraph N["Node (kubelet)"]
        direction LR
        S["Service<br/>(stable IP)"]:::gate --> P["Pod"]:::server --> C["Container(s)"]:::obs
    end
    CP -.->|"reconciles desired state"|N
    classDef gov fill:#FFFFFF,stroke:#333333,stroke-width:1px,color:#111111
    classDef gate fill:#F0F0F0,stroke:#333333,stroke-width:1px,color:#111111
    classDef server fill:#DAE8FC,stroke:#333333,stroke-width:1px,color:#111111
    classDef obs fill:#DCDCDC,stroke:#333333,stroke-width:1px,color:#111111

Figure 1: The Kubernetes model a control plane reconciling desired state across nodes, each running services, pods, and containers.

Kubernetes orchestrates containers; a service mesh orchestrates the traffic between them. Teams reach for K8s, get their pods running, and then hit the wall it deliberately leaves bare: once you have dozens of services calling each other, how do you secure, route, observe, and control that traffic? That’s the gap Istio fills. Understanding both what Kubernetes does, and what it pointedly doesn’t is the difference between fighting your platform and using it.

Summary

  • Kubernetes is a declarative orchestrator. You describe the desired state in YAML; the control plane works continuously to make reality match it.
  • The hierarchy: Node → Service → Pod → Container. Pods are the smallest unit; Services give the ephemeral pods a stable address.
  • Control plane vs nodes. The control plane (API server, scheduler, controllers, etcd) decides; nodes (via the kubelet) execute.
  • K8s leaves service-to-service networking thin. Routing rules, mutual TLS, retries, and per-call observability aren’t its job.
  • A service mesh (Istio) adds that layer with sidecar proxies (Envoy) as the data plane and Istio as the control plane without changing your app code.

1. What Kubernetes actually is

Kubernetes (K8s) is an open-source container orchestrator born at Google, donated to the CNCF for deploying, scaling, and managing containerized apps, stateless or stateful, on-prem or any cloud. Its core idea is declarative intent: you write YAML describing the state you want, and Kubernetes treats it as a record of intent it must continuously uphold. Delete a pod by hand and the controller notices the gap and recreates it the system is always reconciling reality toward your declaration.

A cluster has two kinds of instances:

  • The control plane (cluster master) runs the brains: the API server (every interaction goes through it), the scheduler (decides which node a pod runs on), the controller manager (runs the controllers that manage deployments, replica sets, etc.), and etcd (the cluster’s consistent key-value store of truth).
  • Nodes are the workers that actually run workloads, each talking to the control plane through an agent called the kubelet.

The mental model worth keeping: the control plane decides, the nodes execute. You never imperatively “start a container” you declare what should exist, and the control plane makes it so and keeps it so.

2. The objects you’ll actually use

Kubernetes objects are declarative YAML records. A handful do most of the work:

  • Pod the basic unit: one or more containers sharing an IP and storage. Pods are ephemeral they come and go.
  • ReplicaSet keeps N identical pods running. You rarely touch it directly.
  • Deployment the one you actually use: declarative create/update/rollback/scale of pods. A rolling upgrade spins up a new ReplicaSet while draining the old one.
  • Service because pods are ephemeral and their IPs change, a Service gives a stable IP/port and load-balances across the matching pods (round-robin). Flavors:
    • ClusterIP internal-only, for service-to-service traffic inside the cluster.
    • NodePort exposes the service on each node’s IP at a fixed port.
    • LoadBalancer provisions a cloud load balancer (extra cost).
    • Ingress routes external HTTP(S) by path rules to Services (creates an HTTP(S) LB on GCP).
  • StatefulSet like a Deployment but pods get stable, persistent identities and storage (databases, queues).
  • DaemonSet runs a pod on every node (logging/monitoring agents like fluentd).
  • Job / CronJob run-to-completion tasks, one-off or scheduled.
  • PersistentVolume networked storage mounted via a claim.
  • Namespace virtual clusters for multi-team isolation (default, kube-system, …).

That’s a complete toolkit for running services. But notice what’s missing.

3. What Kubernetes deliberately leaves bare

Kubernetes gets your pods scheduled and gives them stable addresses and then stops. A Service load-balances round-robin and that’s about it. The moment you have many microservices calling each other, you want things K8s doesn’t natively provide:

  • Secure service-to-service comms (mutual TLS between every pod).
  • Fine-grained traffic control weighted routing, canaries, retries, timeouts, circuit breaking.
  • Service-level observability per-call metrics, distributed tracing, golden signals across the mesh.
  • Policy and service discovery beyond a basic round-robin Service.

You can bolt these into each application, but then every team reimplements TLS, retries, and tracing in every language inconsistently. That duplication, across a sprawl of services, is exactly the problem a service mesh exists to solve.

4. The service mesh: Istio

A service mesh moves all that cross-cutting networking out of your app and into a dedicated infrastructure layer. The trick is the sidecar proxy: a small proxy injected into every pod, sitting beside your container. Your app talks to its local proxy; the proxies talk to each other. Now TLS, retries, routing, and metrics live in the proxies uniform across every service, in every language, with zero application code changes.

---
config:
  theme: neutral
  fontSize: 16
---
flowchart LR
    subgraph PA["Pod A"]
        A1["App A"]:::obs --- E1["Envoy<br/>sidecar"]:::gate
    end
    subgraph PB["Pod B"]
        E2["Envoy<br/>sidecar"]:::gate --- B1["App B"]:::obs
    end
    E1 -->|"mutual TLS · retries · routing"|E2
    classDef obs fill:#FFFFFF,stroke:#333333,stroke-width:1px,color:#111111
    classDef gate fill:#F0F0F0,stroke:#333333,stroke-width:1px,color:#111111

Figure 2: The sidecar pattern apps talk to their local Envoy proxy; the proxies handle secure, controlled traffic between services.

Istio splits into two planes the same control-plane/data-plane idea as Kubernetes itself:

---
config:
  theme: neutral
  fontSize: 16
---
flowchart TB
    ISTIO["Control plane Istio<br/>config · discovery · certificates"]:::gov
    ISTIO -.->|"programs the proxies"|DP
    subgraph DP["Data plane Envoy sidecars"]
        direction LR
        S1["Envoy"]:::gate --- S2["Envoy"]:::gate --- S3["Envoy"]:::gate
    end
    classDef gov fill:#FFFFFF,stroke:#333333,stroke-width:1px,color:#111111
    classDef gate fill:#F0F0F0,stroke:#333333,stroke-width:1px,color:#111111

Figure 3: Istio's control plane configures the data plane of Envoy proxies that actually carry the traffic.

  • Data plane the Envoy proxies in every pod, which actually move the bytes and enforce the rules.
  • Control plane Istio, which configures all those proxies: pushing routing rules, handling service discovery, and issuing the certificates for mutual TLS.

With that in place you get the mesh feature set for free across all services: load balancing, fine-grained traffic policies, service discovery, monitoring, distributed tracing, routing, and secure service-to-service communication. Istio isn’t the only option Linkerd/Conduit (Go + Rust) and Consul (control plane with Envoy) play the same role but the architecture is the constant: a control plane programming a data plane of sidecar proxies.

Why it matters: Kubernetes and a service mesh answer two different questions. K8s answers “how do I run and keep these containers alive?” declaratively, via the control-plane/node model. A mesh answers “how do these services talk securely, controllably, and observably?” via sidecars. Don’t reach for a mesh on day one (it’s real operational weight), but the moment service-to-service security, traffic shaping, and tracing become recurring pain, that pain is the signal that you’ve outgrown what Kubernetes gives you alone.

References