Productionizing MCP Part 2: Security, Observability & Governance

Dean Jain

Senior Staff Software Engineer · Enterprise AI, Data & Cloud Architect

July 5, 2026 · 14 min read

MCPAgentic AIArchitectureSecurityObservability

Part 2 of 2. Defending MCP against semantic attacks, identity & token handling, observability with token-cost + audit-origin, governance via the registry, and testing with evals.

📚 Two-part series. This is Part 2 Security, Observability & Governance. Start with ← Part 1 Architecture & Design for the mental model, DDD server topology, and tool design the foundation this part builds on.

TL;DR

Security diverges hardest from REST: semantic attacks have no API-era analog. Tool poisoning, rug pulls, indirect prompt injection, cross-server exfiltration. An authenticated agent can be steered by untrusted text into misusing its legitimate access so authN + rate limits are necessary but not sufficient. For high-impact actions, Human-in-the-Loop (HITL) is a mandatory control.
Stop blindly passing access tokens. Don’t let an agent wield one broad “God token.” Accept only tokens minted for your server, never forward the agent’s token to your databases or downstream services, and exchange it for a strict, scoped token (RFC 8693) before doing backend work on the user’s behalf.
Observability changes when the caller is an AI. Traditional API logs aren’t enough. Track token cost as a first-class SLO, and keep a “why” audit log did the human request it, did the model decide autonomously, or was it tricked by a malicious prompt?
Evals are your new integration tests. Because tool descriptions dictate model behavior, a single word change can break your integration. Gate changes on an eval suite, not just code review.
The registry is your API catalog. Curate which tools each agent can even see “expose everything” is how you get tool explosion, model confusion, and ballooning token cost.

Part 1 covered the shape of a production MCP system: the mental model (a tool call is an RPC where the caller is a non-deterministic LLM), DDD-aligned servers behind a gateway + registry, and intent-shaped tools. This part covers the cross-cutting concerns the ~30% that doesn’t transfer cleanly from REST and is where production MCP is won or lost.

4. Security: where the agent era diverges hardest

Traditional API security authenticates a caller and rate-limits it. That is necessary and not sufficient here, because an authenticated agent can be driven by a poisoned prompt to misuse its legitimate access. You’re defending against semantic attacks, not just unauthenticated ones.

4.1 The three trust boundaries

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart LR
    U["👤 User"]:::neutral ---|" ① "| C["🖥️ AI client / host"]:::agent
    C ---|" ② "| S["⚙️ MCP server"]:::server
    S ---|" ③ "| D["🗄️ Downstream API / DB"]:::down
    B2["🔴 Boundary ②<br/>tool poisoning · sampling injection ·<br/>cross-server exfiltration"]:::danger -.-> C
    B2 -.-> S
    B3["🔴 Boundary ③<br/>confused deputy · token passthrough"]:::danger -.-> S
    B3 -.-> D
    classDef neutral fill:#ECECEC,stroke:#8A8A8A,stroke-width:2px,color:#0F172A
    classDef agent fill:#FFE08A,stroke:#E8A33D,stroke-width:2px,color:#0F172A
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A
    classDef down fill:#FFC2B4,stroke:#E8694A,stroke-width:2px,color:#0F172A
    classDef danger fill:#FFB3B3,stroke:#D14545,stroke-width:2px,color:#0F172A

Figure 1: The three trust boundaries and where attacks live.

Boundary 1 (user ↔ client): where the human interacts with the agent. The client authenticates the user, establishing the initial identity.
Boundary 2 (client ↔ server): tool descriptions/outputs cross into the model’s context. Tool poisoning, sampling injection, cross-server exfiltration live here.
Boundary 3 (server ↔ downstream): confused deputy and token passthrough live here.
Multiple servers sharing one model context all sit inside 2 together which is why co-mingling untrusted servers is dangerous (one server’s tool description can manipulate behavior toward another’s tools).

4.2 Identity & authorization the spec model (get this exactly right)

Treat your MCP server as a standard OAuth 2.1 Resource Server (RS) it validates tokens, it never issues them.

Delegate authentication to an external IdP. Don’t implement authentication in the server. Use a dedicated Identity Provider (Entra ID, Okta, Auth0, Keycloak) as the Authorization Server to issue tokens; the RS only verifies signature, issuer, expiry, and audience on each request.
Require PKCE for public clients. Most agents are public clients (IDE plugins, desktop apps) that cannot keep a client secret. Mandate the Authorization Code flow with PKCE (Proof Key for Code Exchange) and reject implicit-grant flows.
Advertise authorization metadata (RFC 9728). Reject unauthenticated calls with 401 and a WWW-Authenticate header pointing to your Protected Resource Metadata, so clients can discover the Authorization Server and required scopes rather than relying on hardcoded endpoints.
Enforce audience validation (RFC 8707). Accept a token only if it was issued for this resource (aud / Resource Indicators). This prevents token redirection a token minted for one server being replayed against another.
Use dynamic discovery. Expose standard OIDC/OAuth discovery endpoints (.well-known) so clients resolve issuer, JWKS, and scopes automatically instead of relying on static configuration.

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
    actorBorder: "#E8A33D"
    noteBorderColor: "#3FA34D"
---
sequenceDiagram
    autonumber
    participant C as 🖥️ MCP Client
    participant S as ⚙️ MCP Server (RS)
    participant I as 🔑 IdP (Auth Server)
    C->>S: tools/call (no token)
    S-->>C: 401 + WWW-Authenticate (resource_metadata=...)
    C->>S: GET /.well-known/oauth-protected-resource
    S-->>C: PRM { authorization_servers, scopes_supported }
    C->>I: Auth Code + PKCE (resource = this server)
    I-->>C: Access token (audience = this server)
    C->>S: tools/call + Bearer token
    Note over S: Validate signature, issuer, expiry, AUDIENCE
    S-->>C: Result ✅

Figure 2: OAuth handshake: 401 to PRM discovery to audience-bound token.

Insufficient scope at runtime → 403 with WWW-Authenticate: Bearer error="insufficient_scope", scope="personal-assistant:get_item" so the client can step up. Least privilege is enforced per operation, not per session.

4.3 The sidecar

You want authN/authZ resolved before a request reaches tool code. Two dominant patterns, and it’s the same gateway-vs-mesh debate REST already had:

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart TB
    subgraph GWP["🛡️ Gateway in front (centralized)"]
        direction LR
        a1["🤖 Client"]:::agent --> gw["PEP: validate + policy"]:::gate --> s1["⚙️ Server (pure RS)"]:::server
        pdp1["PDP: OPA / OpenFGA / Cedar"]:::gov -.->|"decision"| gw
    end
    subgraph SCP["🚗 Sidecar per server (mesh)"]
        direction LR
        a2["🤖 Client"]:::agent --> sc["PEP sidecar:<br/>mTLS + policy"]:::gate --> s2["⚙️ Server (pure RS)"]:::server
        pdp2["PDP"]:::gov -.->|"decision"| sc
    end
    classDef agent fill:#FFE08A,stroke:#E8A33D,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A
    classDef gov fill:#E0D6F5,stroke:#9B7EDE,stroke-width:2px,color:#0F172A

Figure 3: Auth enforcement: gateway vs sidecar (PEP/PDP).

Pattern	What it is	Use when
Gateway / proxy in front	Centralized edge validates tokens, enforces policy, then forwards	One place for policy/audit across heterogeneous teams; simplest governance
Sidecar per server	Co-located proxy does token validation + policy at the pod	You already run a mesh (Istio/Linkerd), want per-pod mTLS, or scale enforcement with the workload

Either way: keep the server itself a pure Resource Server. Validate the token at the edge, attach a verified identity/claims context, and let a Policy Decision Point (OPA, OpenFGA, Cedar) answer “can this identity invoke this tool on this resource?” the same PEP/PDP split you’d use for REST.

Sidecar responsibilities

The sidecar pattern maps directly from REST same Envoy/Istio proxy-per-pod model, same PEP role. If you already use it for REST APIs, MCP adoption is low-friction. Concretely, the sidecar owns:

RFC 9728 rejection unauthenticated requests get 401 + WWW-Authenticate header pointing to your IdP’s Protected Resource Metadata URL. Agents use this to autodiscover where to get tokens. A generic 401 without PRM breaks agent auth flows.
JWT validation verify signature (via cached JWKS from IdP), issuer, expiry, and audience on every inbound request. Cache the JWKS locally; don’t round-trip the IdP on every call.
Identity forwarding strip the Authorization header, inject verified claims (X-User-Id, X-Scopes, etc.) as internal headers so the MCP server code never sees raw tokens.
mTLS to the MCP server the channel between sidecar and server is mutual-TLS; the server trusts only the sidecar’s cert, refusing direct connections.

One MCP-specific nuance: MCP uses Streamable HTTP / SSE connections are long-lived, not a request-per-action like REST. Define an explicit re-auth policy: either re-validate the Bearer token on each tools/call within the session, or validate once at connection establishment and trust the session for its lifetime. Per-call validation is stricter and aligns with least-privilege; session-lifetime trust is lower latency. Enforce whichever at the sidecar, consistently.

Responsibility	Sidecar	MCP server
Validate inbound Bearer token	✅	❌
Return RFC 9728 401 + PRM URL	✅	❌
Cache JWKS, refresh on rotation	✅	❌
mTLS to server	✅	❌ trust only sidecar cert
Tool-level authZ (OPA/Cedar)	via PDP call	or inline, depending on latency
Token exchange for downstream (RFC 8693)	❌	✅
Downstream API calls	❌	✅

4.4 The MCP-native threats (no REST analog) and their controls

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
mindmap
  root((🕷️ MCP threats))
    Boundary 2 client to server
      Tool poisoning
      Rug pull
      Tool shadowing
      Indirect prompt injection
      Cross-server exfiltration
    Boundary 3 server to downstream
      Confused deputy
      Token passthrough
      SSRF via tool URLs
      Command injection
    Supply chain
      Malicious server packages
      about 5.5 percent had poisoning
    Session
      Session hijack
      Session used as auth

Figure 4: MCP threat map.

Threat	What it is	Controls
Tool poisoning	Malicious instructions hidden in descriptions, schemas, or returned content	Treat descriptions/outputs as untrusted; review + scan defs; sign/pin definitions; sanitize outputs before they re-enter context
Rug pull	A server changes tool definitions after approval	Pin version/hash; re-consent on change; registry change-detection
Tool shadowing / cross-server exfiltration	One server’s description steers behavior toward another’s tools; shared context leaks	Namespace isolation; don’t co-mingle untrusted servers; gateway allowlists
Indirect prompt injection	Untrusted retrieved content carries instructions	Output filtering/guardrails; never auto-execute high-impact actions; Human-in-the-loop (HITL) for sensitive tools
Confused deputy	Server uses its own broad privilege on an attacker’s behalf	Per-user token exchange; per-client consent; CSRF on consent pages; avoid static-client-ID + DCR pitfalls
SSRF via LLM-chosen URLs	A tool fetches a model-supplied URL → internal services / cloud metadata	Strict egress allowlist; validate/sanitize URLs; block link-local/metadata ranges
Command injection	Tool runs model-influenced shell/SQL	Parameterized APIs only; schema validation; never string-concatenate commands
Supply chain	Malicious/compromised server packages	A 2025 study of ~1,900 servers found ~5.5% with tool-poisoning issues provenance, registry curation, pinning + scanning, `.mcpb` integrity
Session hijack / session-as-auth	Predictable session IDs used for authentication	Never authenticate on session ID; verify a valid per-user token on every request

Map these to the emerging OWASP MCP guidance there’s now an OWASP MCP Security Cheat Sheet and an OWASP MCP Top 10 taxonomy. Use them as your review checklist.

4.5 Human-in-the-loop is a control

For high-impact tools (mutations, money movement, external sends, file writes), require action-level approval the backstop when a semantic attack slips through. Centralize the policy at the gateway for consistency and audit.

Frameworks (security)

IdPs: Entra ID, Okta, Auth0, Keycloak, WorkOS.
MCP auth libs: mcp-auth and equivalents that mount RFC 9728 PRM + Bearer middleware.
Policy engines (PDP): OPA, OpenFGA, Cedar.
Gateways w/ auth: Kong (AI MCP OAuth2), Envoy, ContextForge, Tyk, WSO2.
Scanners / guardrails: MCP-specific scanners (tool-definition + behavior analysis), prompt-injection classifiers, Llama-Guard-class filters.
Credential vaults (server-side injection): Vault/Infisical-style secrets injected at execution time, never shipped to clients/agents.

5. Observability, tracing & audit

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart TB
    subgraph BEF["❌ Before: two disconnected traces"]
        direction LR
        ag1["Agent side<br/>Trace A"]:::obs -. "no propagation 😵" .- sv1["Server side<br/>Trace B"]:::server
    end
    subgraph AFT["✅ After: one trace (OTel MCP conventions, v1.39+)"]
        direction LR
        h["Host"]:::agent --> c["Client"]:::agent --> g["Gateway"]:::gate --> s["Server"]:::server --> d["Downstream"]:::down
        ctx["context via traceparent header /<br/>_meta on tools/call"]:::good -.-> g
    end
    classDef obs fill:#AED6F1,stroke:#2E86C1,stroke-width:2px,color:#0F172A
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A
    classDef agent fill:#FFE08A,stroke:#E8A33D,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef down fill:#FFC2B4,stroke:#E8694A,stroke-width:2px,color:#0F172A
    classDef good fill:#BFEFC8,stroke:#3FA34D,stroke-width:2px,color:#0F172A

Figure 5: Two disconnected traces become one unified trace.

OpenTelemetry + MCP semantic conventions

The OTel MCP semantic conventions (introduced in OTel v1.39, under the GenAI conventions) standardize span/metric attributes for MCP. The spec recommends MCP conventions over generic RPC semantics.
Context propagation: HTTP transports → W3C traceparent/tracestate propagate naturally; protocol-level → MCP _meta on tools/call; stdio → propagate explicitly via params/env.

The three pillars plus two MCP needs that REST didn’t

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart TB
    OBS["📡 MCP Observability"]:::obs
    OBS --> TR["Traces<br/>span per tool call"]:::server
    OBS --> ME["Metrics<br/>latency · errors · sessions"]:::gate
    OBS --> LO["Logs<br/>structured + trace id"]:::neutral
    OBS --> CO["💰 Token cost<br/>(NEW pillar)"]:::warn
    OBS --> AU["📝 Audit origin<br/>(NEW pillar)"]:::good
    classDef obs fill:#AED6F1,stroke:#2E86C1,stroke-width:2px,color:#0F172A
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef neutral fill:#ECECEC,stroke:#8A8A8A,stroke-width:2px,color:#0F172A
    classDef warn fill:#FFE6A8,stroke:#E0A106,stroke-width:2px,color:#0F172A
    classDef good fill:#BFEFC8,stroke:#3FA34D,stroke-width:2px,color:#0F172A

Figure 6: Observability: three pillars plus token cost and audit origin.

Traces: a span per tool call, nested under the agent’s reasoning span. Capture tool name, server, agent/user id, duration, status, downstream spans. Be deliberate about args they may contain PII; redact or hash.
Metrics: mcp.client.operation.duration / mcp.server.* histograms, error rate, active sessions (UpDown counter), volume per tool/agent/user, and token consumption + cost. A bimodal latency histogram usually means a downstream dependency problem (cache miss, pool exhaustion, cold start).
Logs: structured, correlated by trace ID.

Audit: the governance-grade record (with a new dimension)

Standard API audit answers who / what / when / where / outcome. For MCP, add the dimension that matters most for incident response:

Did this action originate from a direct user instruction, an autonomous model decision, or a suspected injection?

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
    lineColor: "#5B5170"
---
flowchart TD
    ACT["🔧 Tool call happened"]:::server --> Q{"Where did it<br/>originate?"}:::gate
    Q -->|"user typed it"| UI["✅ user_instruction"]:::good
    Q -->|"model chose it"| MD["🔵 model_decision"]:::obs
    Q -->|"came from<br/>untrusted content"| SI["🚨 suspected_injection"]:::danger
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef good fill:#BFEFC8,stroke:#3FA34D,stroke-width:2px,color:#0F172A
    classDef obs fill:#AED6F1,stroke:#2E86C1,stroke-width:2px,color:#0F172A
    classDef danger fill:#FFB3B3,stroke:#D14545,stroke-width:2px,color:#0F172A

Figure 7: Audit origin: user instruction vs model decision vs injection.

Cost governance / FinOps (no native MCP control)

MCP tracks no token consumption and enforces no budgets an agent can loop tools and rack up model/DB cost unbounded. Enforce token budgets and spend/rate limits at the gateway, per agent/tenant/tool.

6. Governance & lifecycle: the registry is your API catalog

REST analog: API gateway + developer portal + API catalog + design review board. MCP needs the same control plane.

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart TB
    CENT["🏛️ Central IT registry<br/>approved servers + standards + quality grading"]:::gov
    LOB1["🧩 Team A registry"]:::gate
    LOB2["🧩 Team B registry"]:::gate
    LOB3["🧩 Team C registry"]:::gate
    CENT <-->|"bi-directional sync<br/>(synced entries read-only)"| LOB1
    CENT <-->|"sync"| LOB2
    CENT <-->|"sync"| LOB3
    GW["🛡️ Gateway exposes only<br/>scoped, approved tools per agent"]:::server
    CENT -.->|"feeds"| GW
    classDef gov fill:#E0D6F5,stroke:#9B7EDE,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A

Figure 8: Federated registry: central IT and line-of-business.

Use a registry. The official MCP Registry launched in 2025; enterprise private options include Azure API Center, AWS MCP Gateway & Registry, and GCP Apigee API Hub (GCP’s API catalog for discovery + governance, pairs with GCP API Gateway for enforcement).
Federate it. Real orgs need central IT ⇄ line-of-business registries with bidirectional sync, approved-server allowlists, and read-only synced entries showing their source. (Expedia, for example, layered quality grading + skill sync on top of the AWS registry.) Sync modes: all / whitelist / tag-filter.
Solve tool explosion with visibility scoping. Don’t expose every tool to every agent. Curate per agent/persona which tools are even discoverable fewer tools = better selection + lower token cost.
Gate listing on quality + provenance. Server quality grading, contract/conformance pass, security scan, ownership metadata before a server is discoverable.
Ownership model (golden path): each team owns its server’s SLOs, security posture, and lifecycle. A platform team owns the gateway, registry, conformance harness, and standards.

7. Testing & release engineering

Layer your tests; the top layers are REST-like, the bottom ones are new.

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart TB
    L1["⚙️ Unit tool logic"]:::good
    L2["🤝 Contract schema conformance"]:::obs
    L3["📐 Conformance protocol correctness"]:::gate
    L4["🔗 Integration server ↔ downstream"]:::down
    L5["🛡️ Security injection · SSRF · authZ · audience"]:::danger
    L6["🎯 Evals does the AGENT select the tool right? (NEW)"]:::warn
    L7["🌩️ Load / chaos AI-driven unpredictable load"]:::neutral
    L1 --> L2 --> L3 --> L4 --> L5 --> L6 --> L7
    classDef good fill:#BFEFC8,stroke:#3FA34D,stroke-width:2px,color:#0F172A
    classDef obs fill:#AED6F1,stroke:#2E86C1,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef down fill:#FFC2B4,stroke:#E8694A,stroke-width:2px,color:#0F172A
    classDef danger fill:#FFB3B3,stroke:#D14545,stroke-width:2px,color:#0F172A
    classDef warn fill:#FFE6A8,stroke:#E0A106,stroke-width:2px,color:#0F172A
    classDef neutral fill:#ECECEC,stroke:#8A8A8A,stroke-width:2px,color:#0F172A

Figure 9: The MCP testing pyramid (evals are the new layer).

Layer	What it checks	REST analog	New for MCP?
Unit	Tool logic	Same	No
Contract	Tool/resource schema conformance	Pact-style	No
Conformance	Version negotiation, error codes, auth challenge, PRM	OpenAPI validation	Slightly
Integration	Server ↔ real downstream	Same	No
Security	Injection/poisoning red-team, SSRF, authZ matrix, token-audience tests, scanners in CI	DAST/SAST + authZ	Extended
Evals	Does the agent select + use the tool correctly? Regression when a description changes		Yes net new
Load / chaos	Behavior under unpredictable AI-driven load; autoscaling under stateless design	Same tooling	Behavior differs

Key point on evals: because the tool description is behavior, a one-word change can silently break (or fix) the model’s willingness to call it. Treat description/prompt changes like code changes gate them on an eval suite, not a code review alone.

MCP Inspector for local/dev exploration; conformance harnesses in CI.

8. The REST→MCP master mapping

Concern	REST/API equivalent	What transfers 1:1 to MCP	What’s genuinely new in MCP
Topology	Microservices + API gateway	DDD boundaries, gateway, independent deploy	Tool explosion / token-cost as a design force
Service shape	Endpoints/resources	Versioning, deprecation, contracts	Tools are model-selected; descriptions are prompt + injection surface
AuthN/Z	OAuth 2.1 / OIDC, RBAC, PKCE	Resource-server model, scopes, least privilege	Audience binding (8707) + no token passthrough is unforgiving
Edge enforcement	API gateway / mesh sidecar (PEP/PDP)	Same patterns	Co-mingling untrusted servers is a context-level risk
Threats	OWASP API Top 10	Injection, SSRF, broken authZ	Semantic attacks: tool poisoning, rug pulls, indirect prompt injection, exfiltration
Observability	OTel traces/metrics/logs	The whole stack	Agent↔server trace stitching (`_meta`), token-cost telemetry
Audit	who/what/when/where/outcome	Same fields	Origin dimension: user vs model vs injection
Catalog	API portal/catalog	Discovery, ownership, review	Visibility scoping; federated registries
Testing	Unit/contract/integration	All of it	Evals as a release gate
Cost	Infra rate limits	Rate limiting	Token budgets as a first-class SLO

9. The 12 things to take away

Protocol ≠ platform. Budget for the control plane.
Carve by bounded context, federate behind a gateway + registry. Not monolith, not N×M.
Build stateless now so you inherit horizontal scaling (and the 2026-07-28 model) for free.
Design task-shaped tools, not 1:1 REST wrappers. Fewer, coarser, intent-level.
Server = pure OAuth Resource Server. Identity lives in your IdP.
Terminate authN/Z at the edge (gateway and/or sidecar PEP) → PDP for tool-level authZ.
Validate audience. Never passthrough. Always token-exchange downstream. No God token.
Defend semantic attacks treat descriptions/outputs as untrusted; pin definitions; re-consent on change; HITL for high-impact tools.
One trace, end to end, via OTel + MCP semantic conventions + _meta propagation.
Audit the origin of every action; centralize, make it tamper-evident and queryable.
Govern via the registry curate visibility, gate on quality/provenance, federate across LOBs.
Add two release gates REST never had: evals and token-cost budgets.

The sequencing that actually works

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart LR
    P1["1️⃣ One DDD server,<br/>done right"]:::server --> P2["2️⃣ Gateway<br/>auth · audit · telemetry"]:::gate --> P3["3️⃣ Registry + governance<br/>(at 3+ servers)"]:::gov --> P4["4️⃣ Evals + cost gates<br/>before autonomy"]:::good
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef gov fill:#E0D6F5,stroke:#9B7EDE,stroke-width:2px,color:#0F172A
    classDef good fill:#BFEFC8,stroke:#3FA34D,stroke-width:2px,color:#0F172A

Figure 10: The sequencing that works: one server, gateway, registry, evals and cost gates.

If you ship REST at scale, you already own most of this. Spend your attention on the four things that don’t transfer semantic attacks, descriptions-as-behavior, token cost as an SLO, and evals as tests. Those are where production MCP is won or lost.

Resist building the control plane before you have servers to control and resist letting agents take actions before you can audit their origin.

Specs & references worth bookmarking

MCP Specification 2025-11-25 stable and the 2026-07-28 release candidate / transport roadmap
MCP Authorization spec (2025-06-18 resource-server model; 2025-11-25 discovery updates) · RFC 9728 (Protected Resource Metadata) · RFC 8707 (Resource Indicators) · RFC 8693 (Token Exchange) · OAuth 2.1
MCP Security Best Practices (official) · OWASP MCP Security Cheat Sheet · OWASP MCP Top 10 · MCP Inspector
OpenTelemetry GenAI / MCP semantic conventions (OTel v1.39+)
MCP Registry · Kong AI Gateway · Envoy AI Gateway · AWS MCP Gateway & Registry · GCP Apigee API Hub