Architecture-as-Code

Prerequisite: Distributed Systems, API Design

Your payment service accepts a fund-transfer request. It publishes a transfer.initiated event to Kafka. Nothing consumes it. The topic exists, the message serializes correctly, the producer receives an ack — but the compliance service that should screen the transfer was redeployed last week with a typo in its subscription config. It subscribes to transfers.initiated (plural). The transfer completes without sanctions screening. You discover the gap 11 days later during a reconciliation audit.

The architecture diagram on Confluence shows the correct topology. The code does not match it. No alarm fired because no system compared the two.

Architecture-as-code is the pattern of encoding your system’s topology, dependencies, and preconditions as machine-readable data, then building runtime, boot-time, and CI checks that enforce that data against the live system. The diagram becomes the test.

The Manifest

The foundation is a declarative file that describes what your system’s topology should look like. Not a diagram — a structured document that tooling can parse and validate.

# architecture.yaml
flows:
  transfer:
    steps:
      - service: api-gateway
        produces: [transfer.initiated]
      - service: compliance-screening
        consumes: [transfer.initiated]
        produces: [transfer.screened]
      - service: ledger-writer
        consumes: [transfer.screened]
        produces: [transfer.posted]
      - service: notification-service
        consumes: [transfer.posted]

topics:
  transfer.initiated:
    schema: schemas/transfer-initiated.avsc
    partitions: 12
    retention: 7d
  transfer.screened:
    schema: schemas/transfer-screened.avsc
    partitions: 12
    retention: 7d
  transfer.posted:
    schema: schemas/transfer-posted.avsc
    partitions: 12
    retention: 30d

Every topic has a declared producer and at least one declared consumer. Every service lists what it consumes and what it produces. The manifest is the single source of truth for the system’s event topology — not the Kafka cluster’s actual topic list, not the service’s application config, not the architecture diagram.

The file lives in the repository, version-controlled alongside the code. Changing the topology requires changing the manifest, which triggers the same CI pipeline as a code change.

Boot-Time Topology Verification

At startup, each service reads the manifest and compares its declared subscriptions against its actual Kafka consumer group configuration. If the service is supposed to consume transfer.initiated but its consumer config points to transfers.initiated, the service fails to start.

def verify_topology(manifest_path: str, service_name: str):
    manifest = load_manifest(manifest_path)
    declared = manifest.get_subscriptions(service_name)
    actual = get_kafka_consumer_subscriptions()

    missing = declared - actual
    undeclared = actual - declared

    if missing:
        raise TopologyError(
            f"{service_name} must consume {missing} per manifest"
        )
    if undeclared:
        raise TopologyError(
            f"{service_name} consumes {undeclared} not in manifest"
        )

The check runs before the service accepts traffic. A misconfigured service never enters the load balancer. The failure is loud — a crash at boot with an explicit error message — rather than silent. The typo that cost you 11 days of unscreened transfers becomes a deploy failure that blocks the release pipeline.

Boot-time verification catches drift that happens between deploys: a config change that removes a subscription, a refactor that renames a topic in the producer but not the consumer, a new service that publishes to a topic no one consumes yet.

Boot-time checks verify that the running service matches the manifest. They do not verify that the manifest is correct. A manifest that omits a required flow passes boot-time validation on every service — because no service is told to expect the missing flow. CI validation catches that class of error.

CI Validation

The CI pipeline runs three categories of checks against the manifest on every pull request.

Orphan detection. A topic that appears in the manifest with a producer but no consumer is an orphan. Events publish into it and vanish. A topic with a consumer but no producer is a dead subscription — the consumer waits for messages that never arrive.

ERROR: topic 'transfer.screened' has producer
       'compliance-screening' but no declared consumer.
ERROR: topic 'refund.requested' has consumer
       'refund-processor' but no declared producer.

DAG cycle detection. The flow from transfer.initiated to transfer.posted is a directed acyclic graph. A cycle — service A produces an event that service B consumes and transforms into an event that service A consumes — creates an infinite loop. Topological sort of the flow graph detects cycles at CI time.

def detect_cycles(manifest):
    graph = build_dependency_graph(manifest)
    visited, in_stack = set(), set()

    def dfs(node):
        visited.add(node)
        in_stack.add(node)
        for neighbor in graph[node]:
            if neighbor in in_stack:
                raise CycleError(f"Cycle: {node} -> {neighbor}")
            if neighbor not in visited:
                dfs(neighbor)
        in_stack.remove(node)

    for node in graph:
        if node not in visited:
            dfs(node)

Flow continuity. Every declared flow must have a complete path from its entry point to its terminal step. If the transfer flow declares four steps but the compliance service’s consumer topic doesn’t match the API gateway’s producer topic, the flow is broken. CI walks each flow and verifies that every produces connects to the next step’s consumes.

Check	What it catches	When it matters
Orphan topics	Events published to nowhere	Refactors that remove consumers
Dead subscriptions	Consumers waiting for nonexistent events	Typos, stale config
DAG cycles	Infinite event loops	Complex choreography additions
Flow continuity	Broken end-to-end paths	Topic renames, service decomposition
Schema compatibility	Breaking changes to event contracts	Payload evolution

A system with 40 topics and 15 services has enough edges that manual review of topology changes is unreliable. CI validation makes topology a first-class artifact with the same quality gates as code.

Route Prerequisite Checking

The manifest extends beyond event topology to encode API-level preconditions. Before a request executes, the system verifies that its prerequisites are satisfied.

A fund transfer requires: KYC verification complete, source account funded, compliance screening passed, and idempotency key not previously used. These preconditions are not business logic buried in application code — they are declared in the manifest and enforced by middleware.

routes:
  POST /v1/transfers:
    prerequisites:
      - check: kyc_status
        service: identity-service
        condition: "status == 'verified'"
        cache_ttl: 3600s
      - check: account_funded
        service: ledger-service
        condition: "available_balance >= amount"
        cache_ttl: 0
      - check: compliance_cleared
        service: compliance-service
        condition: "screening_result != 'blocked'"
        cache_ttl: 300s

The API gateway reads the route’s prerequisites and evaluates each one before forwarding the request to the handler. A failed prerequisite returns a structured error that names the failing check, not a generic 400.

{
  "error": "prerequisite_failed",
  "check": "kyc_status",
  "detail": "KYC verification incomplete for account acct_8x29f",
  "required": "verified",
  "actual": "pending_review"
}

Prerequisite checks add latency. Each check is a service call (or cache hit). Three prerequisites with a 15ms p95 each add 45ms in serial, or 15ms if evaluated in parallel with a 45ms worst-case tail. The cache_ttl field controls the tradeoff: KYC status changes rarely and caches well. Available balance changes on every transaction and must be read fresh.

Prerequisite	Cacheable	Typical latency	Staleness risk
KYC status	Yes (hours)	2ms (cached)	Low — status changes are rare
Account balance	No	12ms (live read)	High — every transaction changes it
Compliance screening	Yes (minutes)	3ms (cached)	Medium — list updates are batched
Idempotency check	No	5ms (key lookup)	None — must be real-time

The alternative is scattering these checks across handler code. The transfer handler calls the identity service, the compliance service, and the ledger service in its first 40 lines. The refund handler duplicates some of those calls but not others. A new route forgets to check compliance because the developer copied the wrong handler as a template. Declarative prerequisites make the checks visible, consistent, and auditable without reading every handler’s implementation.

Schema Evolution as Architecture

The manifest declares which Avro or Protobuf schema each topic uses. Schema changes go through the same CI pipeline as topology changes. A producer that adds a required field to transfer.initiated breaks every consumer that doesn’t expect it. Schema compatibility checks — backward, forward, or full — run at CI time against the schema registry.

topics:
  transfer.initiated:
    schema: schemas/transfer-initiated.avsc
    compatibility: backward

Backward compatibility means consumers running the old schema can read messages produced with the new schema. This is the safe default for financial events: you can add optional fields, but you cannot remove fields or change types. A producer that needs to make a breaking change must create a new topic version (transfer.initiated.v2) and run both versions during the migration window.

The manifest tracks which services use which schema version. CI can answer the question “if I deploy this schema change, which services will break?” before the change reaches production.

See API Design. The same versioning discipline that applies to REST APIs applies to event schemas. A breaking change to an event is a breaking change to every consumer — and unlike a REST endpoint, you cannot see the consumers in your access logs.

Consumer-Driven Contract Testing

The manifest declares the topology. Contract tests verify that each service honors its declared interface. The producer publishes a message that matches its declared schema. The consumer proves it can deserialize and process messages in that schema.

Pact and similar frameworks automate this: the consumer declares what it expects from the producer’s messages, and the producer’s CI pipeline verifies it can satisfy those expectations. In a system with 15 services and 40 topics, contract tests replace integration tests that require the full topology to be running. A contract test for the compliance service verifies it can process transfer.initiated messages without starting the API gateway, the ledger, or the notification service.

The cost is maintaining the contracts. Each consumer-producer pair requires a contract definition. With 40 topics and an average of 2 consumers per topic, that is 80 contracts. Automated contract generation from the manifest reduces the maintenance burden — the manifest already declares which service consumes which topic with which schema.

Drift Detection in Production

CI catches errors before deployment. Boot-time checks catch errors during deployment. Drift detection catches errors that accumulate after deployment — a manual Kafka topic creation, a consumer group that quietly unsubscribes due to a rebalance bug, a service that starts consuming a topic it is not declared to consume.

A periodic reconciliation job (every 5 minutes in most systems) compares the manifest against the live Kafka cluster state and each service’s actual consumer group memberships.

DRIFT DETECTED:
  Topic 'settlement.confirmed' exists in cluster
  but is not declared in architecture.yaml.

  Service 'reporting-service' consumes
  'transfer.posted' but is not declared as
  a consumer in architecture.yaml.

Drift alerts route to the platform team, not the application team. The platform team decides whether the drift is an undeclared change (update the manifest) or an unauthorized change (revert the cluster state). In a regulated environment, undeclared topology changes are audit findings. Continuous drift detection converts a quarterly audit surprise into a same-day alert.

The reconciliation job is the closing loop. The manifest defines the intended state. CI validates changes to the intended state. Boot-time checks enforce the intended state at deploy. Drift detection verifies that the running state matches the intended state continuously. Together, they make the architecture self-enforcing — not through human discipline, but through automation that treats topology with the same rigor as code.