Service Meshes

Pradeep P
3 days ago
3 min read

Series: Modern System Design · Layer 5 — Modern infrastructure

Layer 5 · Post 4 of 12

← Previous: Service Discovery → Next: Distributed Tracing

Layer 5 — Modern infrastructure · Post 54 of 88

A service mesh moves networking concerns like retries, mTLS, and traffic splitting out of application code and into the infrastructure.

What you'll learn

What a sidecar (or ambient proxy) intercepts, and what it does not
Why mTLS, retries, and canaries often live in the mesh — and when they should not
The cost: extra hops, YAML, and a new failure domain

The idea in one minute

Every microservice needs timeouts, retries, encryption in transit, and a way to shift 5% of traffic to a new version. You can put that in each language's HTTP client. A service mesh puts a proxy next to every instance (the sidecar) and a control plane that programs those proxies.

Checkout app <-> [ sidecar ] <----mTLS----> [ sidecar ] <-> Payments app ^ ^ +-------- control plane -------+

The app talks to localhost. The proxy does the network policy.

Why it matters

Layer 3 taught retries, timeouts, and circuit breakers. In a polyglot fleet, those libraries drift. The mesh is how platform teams offer one policy: encrypt everything east-west, retry only idempotent GETs, send 10% to v2.

Interviews: mention a mesh when you have many services, zero-trust between them, or traffic splitting that should not require a code deploy. Do not mention it for a two-service CRUD app.

How it works

A data plane (Envoy is the usual engine) sits in the Pod or on the node. iptables or eBPF redirects outbound (and inbound) traffic through it.
A control plane (Istio, Linkerd, Consul) watches Services and policy CRDs, pushes config to proxies.
mTLS: proxies mint or rotate certs, encrypt hop-by-hop, optionally enforce identity (spiffe://.../payments) not just "the IP was in the cluster."
Traffic: retries, timeouts, outlier ejection, weighted routes for canaries, fault injection for tests.

Discovery still comes from Kubernetes (or Consul). The mesh consumes those endpoints; it does not replace the idea.

Sidecar vs ambient

Sidecar: one proxy container per Pod. Isolation is clear; cost is RAM × replica count.

Ambient / node proxy: shared proxy per node or per namespace. Cheaper, slightly different blast radius. Same control-plane story.

A simple example

You want a canary of payments v2. Without a mesh, checkout needs a feature flag and two URLs. With Istio (or similar), a VirtualService sends 5% of requests to the v2 Deployment, 95% to v1. Apps stay unaware. You watch error rate, then 25%, then 100%.

Same mesh can require mTLS so a random Pod cannot scrape payments in plaintext. Policy is YAML, not a ticket to every team to "please enable TLS."

Common mistakes

Mesh as a retry dump. Retries in the mesh and in the client and in the gateway multiply load (Layer 3). Pick one layer; make retries idempotent.

Installing Istio because it is on a blog. You now debug Envoy, sidecars that fail to inject, and upgrades. Start with Kubernetes Services + Ingress. Add a mesh when mTLS and traffic policy are fleet-wide requirements.

Timeouts that ignore the app. The proxy times out at 1s; the handler still runs for 30s. You paid for the work and still failed. Align budgets.

Treating mTLS as authorization. Encryption proves the peer has a cert. You still need authz (who may call checkout). Meshes can do both; do not confuse them.

How this shows up in real systems

Istio, Linkerd, Consul Connect, AWS App Mesh: the named products. Linkerd markets simplicity; Istio markets knobs.
API gateways vs mesh: gateway is north-south (users → cluster). Mesh is east-west (service → service). Many shops use both.
eBPF data planes (Cilium): same goals, fewer sidecar hops, newer operational story.

If you already have a mature client library and three services, a mesh can be overhead. If you have eighty services in five languages, it starts to look like a platform.

Recap

A mesh is proxies + a control plane that take retries, mTLS, and routing out of app code.
Sidecars intercept traffic; Kubernetes still owns which endpoints exist.
Use it for fleet-wide policy, not as a substitute for timeouts and idempotency in the design.