Chain Agent Architecture

Prerequisite: Blockchain Fundamentals

Your payment platform supports Ethereum, Solana, and Stellar. A single backend service handles all three chains — it submits EVM transactions via eth_sendRawTransaction, Solana transactions via sendTransaction, and Stellar transactions via Horizon’s POST endpoint. Then Ethereum introduces EIP-4844 blob transactions, and the change touches every code path in the service. Solana upgrades its RPC to version 2.0 with breaking changes to transaction format, and now you are patching two chain-specific regressions in a single deployment. A bug in the Stellar path takes down the entire service, halting all three chains simultaneously. The blast radius of a single-chain issue is your entire payment infrastructure.

The chain agent pattern eliminates this coupling. Each blockchain gets its own dedicated backend service — an agent — that owns all interaction with that chain. The agents share a common interface but deploy, scale, and fail independently.

One Agent Per Chain

A chain agent is a backend service responsible for exactly one blockchain. The EVM agent knows how to construct EVM transactions, manage nonces, estimate gas, submit to mempools, and track confirmations. The Solana agent knows about blockhashes, compute units, transaction expiry, and program accounts. Each agent is a specialist — it contains the full complexity of its chain and exposes a simplified interface to the rest of the system.

Agent	Chain-specific concerns	RPC protocol
EVM agent	Nonce management, gas estimation (EIP-1559), reorg detection, receipt polling	JSON-RPC (`eth_*`)
Solana agent	Blockhash freshness, compute unit budgets, transaction expiry, priority fees	JSON-RPC (`send/getTransaction`)
Stellar agent	Sequence numbers, fee bumps, Horizon REST API, ledger close times	REST + SSE
Tron agent	Energy/bandwidth metering, TRC-20 approval flows, Tron-specific ABI encoding	JSON-RPC + HTTP

Each agent deploys independently. An upgrade to the Solana agent — to support a new transaction version or handle an RPC deprecation — does not require redeploying the EVM agent. A crash in the Stellar agent does not affect Ethereum transactions. Failure isolation is the primary architectural benefit.

See Blockchain Fundamentals. Each chain has a fundamentally different execution model, consensus mechanism, and finality guarantee. The agent pattern exists because these differences are too deep to abstract cleanly into a single service.

The Common Interface

Despite chain-specific internals, every agent exposes the same set of operations to the rest of the platform. The orchestration layer — the service that decides “send 100 USDC from wallet A to address B on Ethereum” — does not know or care how EVM transactions work. It sends a command to the EVM agent and receives chain-agnostic events in return.

The interface has four operations:

SubmitTransaction(request) → transaction_id
GetTransactionStatus(transaction_id) → status
GetBalance(address, asset) → amount
ListEvents(filter) → events

SubmitTransaction accepts a chain-agnostic payload: source address, destination address, asset, amount, and an idempotency key. The agent translates this into chain-specific transaction construction — ABI encoding for EVM, instruction building for Solana, operation building for Stellar.

GetTransactionStatus returns a normalized status. Every chain agent maps its chain’s native states to a common enum:

Common status	EVM mapping	Solana mapping	Stellar mapping
`PENDING`	In mempool, 0 confirmations	Sent, not yet in a slot	Submitted, not in a ledger
`CONFIRMING`	1–11 confirmations	In slot, not rooted	In ledger, awaiting close
`CONFIRMED`	12+ confirmations	Rooted	Ledger closed
`FAILED`	Reverted receipt	Transaction error	`tx_failed` result
`EXPIRED`	Dropped from mempool	Blockhash expired	Sequence number consumed

The confirmation thresholds encode each chain’s finality model. Ethereum’s 12-block threshold (~2.5 minutes post-merge) reflects the practical finality window. Solana’s “rooted” status means the slot is confirmed by a supermajority of validators — roughly 30 seconds. Stellar ledgers close every 5–7 seconds with immediate finality.

The Worker Architecture

Chain agents consume commands from Kafka topics and emit events back to Kafka. The orchestration layer publishes a command to chain-commands.ethereum, and the EVM agent consumes it. When the transaction confirms, the EVM agent publishes an event to chain-events.confirmed — a chain-agnostic topic that the orchestration layer consumes regardless of which chain produced it.

Orchestration Layer
    ↓ publishes
chain-commands.ethereum    → EVM Agent
chain-commands.solana      → Solana Agent
chain-commands.stellar     → Stellar Agent
    ↑ consumes
chain-events.confirmed     ← all agents publish here
chain-events.failed        ← all agents publish here

Each agent runs multiple workers partitioned by source address. All transactions from the same address must process sequentially — on EVM, nonce ordering requires it; on Solana, recent blockhash reuse across concurrent transactions can cause duplicates. Partitioning by address ensures ordering within an address while allowing parallelism across addresses.

A typical EVM agent runs 16–64 Kafka consumer threads, each handling a partition of addresses. At a throughput of 50–200 transactions per second per agent, a single agent instance handles the transaction volume of most payment platforms. Horizontal scaling adds agent instances with additional Kafka partitions.

EVM: Nonce Gaps and Gas Storms

The EVM agent’s primary operational challenge is nonce management. Every transaction from an address carries a sequential nonce. Transaction N must confirm before transaction N+1 can be included in a block. If transaction N is stuck — gas price too low, or dropped from the mempool — transactions N+1 through N+50 queue behind it indefinitely.

The nonce management state machine:

                ┌─────────────┐
                │ QUEUED       │
                │ (nonce = N)  │
                └──────┬──────┘
                       │ submit to mempool
                       ▼
                ┌─────────────┐
      ┌────────│ PENDING      │────────┐
      │        │ (in mempool) │        │
      │        └──────┬──────┘        │
      │               │               │
      │ dropped       │ mined         │ timeout (5 min)
      │               ▼               │
      │        ┌─────────────┐        │
      │        │ CONFIRMING  │        │
      │        └──────┬──────┘        │
      │               │               │
      │               │ 12 blocks     │
      │               ▼               │
      │        ┌─────────────┐        │
      │        │ CONFIRMED   │        ▼
      │        └─────────────┘ ┌─────────────┐
      │                        │ SPEED UP     │
      └───────────────────────→│ (replace txn)│
                               └─────────────┘

When a transaction is stuck, the agent replaces it by resubmitting the same nonce with a higher gas price — the speed-up pattern. The replacement transaction must offer at least 10% higher maxPriorityFeePerGas for most node implementations to accept it. If the original was submitted at 2 gwei priority fee, the replacement must be at least 2.2 gwei.

Cancellation follows the same mechanism: submit a zero-value self-transfer at the stuck nonce with a higher gas price. The self-transfer consumes the nonce slot without executing any meaningful state change.

The agent tracks nonce state in a local database — not by querying eth_getTransactionCount on every submission. RPC calls to eth_getTransactionCount with "pending" are unreliable across different node implementations, and querying on every submit adds 10–50ms of latency. The agent maintains its own nonce counter per address and reconciles against the chain periodically (every 30 seconds) to detect gaps.

Solana: Blockhash Expiry and Compute Budgets

Solana transactions include a recent blockhash that acts as a TTL. If the transaction is not included in a block within approximately 60–90 seconds (roughly 150 slots), the blockhash expires and the transaction is permanently invalid. There is no mempool in the traditional sense — validators process transactions as they arrive and discard expired ones.

This creates two failure modes the agent must handle:

Stale blockhash at submission. The agent fetches a blockhash, constructs the transaction, but takes too long to submit. By the time the transaction reaches the validator, the blockhash has expired. The agent must fetch a fresh blockhash and resubmit.
Transaction disappears. The transaction was submitted but never appeared in a slot. Unlike EVM, where a transaction sits in the mempool until mined or manually replaced, Solana transactions simply vanish when their blockhash expires. The agent must detect this (polling getSignatureStatuses returns null) and resubmit with a new blockhash.

Compute unit budgets add another dimension. Every Solana transaction must declare a compute unit limit. If the actual computation exceeds this limit, the transaction fails but the fee is still charged. The agent estimates compute units using simulateTransaction before submission and adds a 20–40% buffer.

Parameter	EVM equivalent	Typical value
Compute unit limit	Gas limit	200,000–400,000 CU for token transfers
Compute unit price	Priority fee (gwei)	1–100 microlamports/CU
Blockhash TTL	None (mempool persistence)	~60–90 seconds

Priority fees on Solana work differently from EVM. Instead of a global gas market, Solana uses localized fee markets — priority fees apply per-account, and congestion on one program does not affect fees for another. The agent adjusts priority fees based on recent fees for the specific program it interacts with, not network-wide gas prices.

Stellar: Sequence Numbers and Fee Bumps

Stellar’s transaction model is closer to EVM’s nonce system. Each account has a sequence number that increments with every transaction. Unlike EVM, Stellar enforces strict sequence ordering at the protocol level — a transaction with sequence N+1 is rejected if N has not been applied to the ledger.

The agent manages sequence numbers by querying the account’s current sequence from Horizon and incrementing locally. Concurrent submissions from the same account require careful sequencing — the agent serializes all transactions per account, submitting N+1 only after N is confirmed.

Fee bumps on Stellar are a first-class protocol feature. If a transaction is not included in a ledger within the expected window (5–7 seconds per ledger close), the agent constructs a fee bump transaction that wraps the original with a higher fee. The original transaction’s hash remains the same — the fee bump is a wrapper, not a replacement.

// Stellar fee bump — wraps original transaction
POST /transactions
{
  "tx": "fee_bump_envelope_xdr",
  "inner_tx_hash": "original_hash"
}

Stellar’s deterministic ledger close time (5–7 seconds) means the agent knows quickly whether a transaction was included. If two ledger closes pass without inclusion, the transaction is effectively stuck and should be fee-bumped or resubmitted.

Confirmation Tracking and Finality

Each agent runs a confirmation tracker — a background process that monitors submitted transactions and updates their status as confirmations accumulate. The tracker’s behavior differs significantly per chain.

EVM tracker: Polls eth_getTransactionReceipt every block (~12 seconds). Watches for reorgs by comparing the block hash of the transaction’s block against the canonical chain. A reorg that removes a confirmed transaction resets its status to PENDING and triggers a resubmission. Reorgs beyond 1–2 blocks are rare post-merge but must be handled — the agent maintains a 12-block buffer before reporting CONFIRMED.

Solana tracker: Polls getSignatureStatuses with searchTransactionHistory: true. Transitions from processed → confirmed → finalized (rooted). The finalized state means 31+ validators have confirmed the slot — practically irreversible. Polling interval: 2–5 seconds.

Stellar tracker: Subscribes to Horizon’s SSE (Server-Sent Events) stream for the account. Stellar transactions are final on inclusion in a closed ledger — there are no reorgs. Confirmation tracking is binary: either the transaction appears in a ledger, or it does not.

Chain	Finality type	Time to finality	Reorg risk
Ethereum	Probabilistic (economic)	~15 minutes (12 blocks)	Low post-merge, non-zero
Solana	Deterministic (supermajority)	~30 seconds (rooted)	Extremely rare
Stellar	Immediate (SCP consensus)	5–7 seconds	None

Event Listening and Ingestion

Chain agents also listen for inbound events — transfers into monitored addresses, contract events, balance changes. Each chain’s event model requires a different ingestion strategy.

EVM agents use eth_getLogs with block-range filters, polling every block. For high-volume contracts, WebSocket subscriptions (eth_subscribe) provide lower latency but require reconnection logic. The agent maintains a cursor (last processed block number) and replays from the cursor on restart to avoid missing events.

Solana agents subscribe to account changes via accountSubscribe WebSocket or poll getSignaturesForAddress. Program log parsing extracts structured events from Solana’s log-based event system — there are no indexed event logs like EVM’s eth_getLogs.

Stellar agents subscribe to Horizon’s payment and effects streams via SSE. Stellar’s event model is account-centric — you subscribe to an account and receive all operations affecting it.

All agents normalize inbound events into a common schema before publishing to chain-events.inbound:

{
  "chain": "ethereum",
  "event_type": "transfer",
  "tx_hash": "0xabc...",
  "from": "0x123...",
  "to": "0x456...",
  "asset": "USDC",
  "amount": "1000000",
  "block_number": 18500000,
  "confirmed": true,
  "timestamp": "2024-10-15T14:30:00Z"
}

The orchestration layer processes these events without any chain-specific logic. A USDC transfer on Ethereum and a USDC transfer on Solana arrive in the same format, trigger the same business logic, and post to the same ledger.

Event ingestion must be idempotent. Chain reorgs (EVM), RPC node lag, and agent restarts all produce duplicate events. The consumer must deduplicate on (chain, tx_hash, log_index) — processing the same event twice means crediting an account twice.

Operational Topology

A production deployment runs one agent service per chain, each with its own deployment pipeline, monitoring dashboard, and on-call rotation. The agents share nothing except Kafka topics and the common event schema.

Component	Per-chain	Shared
Agent service	Yes	—
RPC connections	Yes (chain-specific nodes)	—
Nonce/sequence state DB	Yes	—
Kafka command topics	Yes (`chain-commands.<chain>`)	—
Kafka event topics	—	Yes (`chain-events.*`)
Monitoring dashboards	Yes (chain-specific metrics)	Yes (cross-chain SLA)
Deployment pipeline	Yes	—

Adding a new chain means deploying a new agent that implements the common interface. The orchestration layer requires zero code changes — it publishes commands to the new chain’s topic and consumes events from the shared topics. The agent encapsulates all chain-specific complexity behind the same four operations every other agent exposes.