Credit Decisioning Under Latency Constraints

Note: No direct handbook prerequisite. This chapter covers production ML systems for credit — the handbook covers compliance and money fundamentals but not ML deployment.

Your customer is standing at a checkout counter. They selected “pay in 4 installments” and the merchant’s terminal is waiting for an approve/decline response. Your system has 250ms to pull credit bureau data, retrieve internal features, run a fraud model, run an underwriting model, apply policy rules, and return a decision. At 251ms the terminal times out and the customer sees “something went wrong.” You just lost a sale, and the merchant just lost confidence in your product.

The 250ms budget is not aspirational. Visa and Mastercard mandate authorization response times under 2 seconds for card-present transactions, but the network and issuer processing consume most of that window. By the time the request reaches your decisioning service, you have 200–300ms. Point-of-sale integrations like Affirm’s adaptive checkout operate under even tighter constraints — the decision must return before the customer notices a pause.

The Latency Budget

A credit decision is a pipeline. Each stage consumes a slice of the 250ms budget, and the stages are partially serializable — some can run in parallel, others depend on upstream outputs.

Stage	Typical latency	Parallelizable
Request parsing and routing	2–5ms	No
Feature retrieval (internal)	15–40ms	Yes
Credit bureau pull	50–150ms	Yes
Fraud model inference	5–15ms	After features
Underwriting model inference	10–30ms	After features
Policy rules evaluation	3–10ms	After models
Offer generation	5–15ms	After policy
Response serialization	2–5ms	No

The critical path runs through the slowest parallel branch plus the serial stages. If feature retrieval and bureau pull run concurrently, the parallel stage costs max(40, 150) = 150ms. Add the serial stages and you are at 175–230ms. That leaves 20–75ms of headroom — enough to absorb a p90 day, not enough to absorb a p99 spike on the bureau call.

          ┌─ Feature Retrieval (15–40ms) ─┐
Request ──┤                                ├── Fraud Model ── Underwriting ── Policy ── Offer ── Response
  (3ms)   └─ Credit Bureau Pull (50–150ms)─┘     (10ms)        (20ms)        (5ms)    (10ms)     (3ms)

The bureau pull dominates the budget. Everything else must be engineered to stay under its shadow.

Feature Retrieval: The 40ms Ceiling

Credit decisioning features fall into three categories by freshness requirement and retrieval cost.

Category	Examples	Source	Latency
Real-time	Cart value, merchant category, time of day, device fingerprint	Request payload	< 1ms
Near-real-time	30-day transaction count, current balance, days since last payment	Feature store	5–30ms
Batch	Income estimate, credit utilization trend, payment history score	Pre-computed, cached	1–5ms

The near-real-time features are the bottleneck. A feature store like Feast or Tecton serves pre-aggregated features from a low-latency store (Redis, DynamoDB) keyed by customer ID. The aggregation happens asynchronously — a streaming pipeline updates the feature values as events arrive — so the serving path is a key-value lookup, not a computation.

features = await feature_store.get_online_features(
    entity_keys={"customer_id": customer_id},
    feature_refs=[
        "customer:txn_count_30d",
        "customer:avg_order_value_90d",
        "customer:days_since_last_payment",
        "customer:current_balance",
        "customer:utilization_ratio",
    ],
)

The failure mode is feature staleness. If the streaming pipeline lags by 30 minutes, the txn_count_30d feature is 30 minutes stale. For most credit decisions, 30 minutes of staleness is acceptable — a customer’s creditworthiness does not change in half an hour. But if the pipeline is down for 8 hours, the feature store is serving yesterday’s data. Your decisioning service needs a staleness threshold: if the feature’s last-updated timestamp is older than T, either use a fallback value or route to a degraded decision path.

Feature staleness is a silent failure. The model still returns a score, and the score still looks plausible. The only signal is the timestamp on the feature values — if your serving path does not check it, you will make decisions on stale data without knowing.

The Bureau Call Problem

A credit bureau pull (Experian, TransUnion, Equifax) is the single largest latency contributor. A soft pull returns in 50–150ms under normal conditions. Under load — Black Friday, month-end billing cycles — bureau response times spike to 300–500ms.

You cannot control bureau latency. You can control how you handle it.

Pre-fetching. If the customer is browsing a merchant’s site and has authenticated, pull bureau data before they reach checkout. Cache the result for 15–30 minutes. When they select “pay in 4,” the bureau data is already warm. Affirm uses this pattern — the adaptive checkout widget pre-qualifies the customer before they see the installment option.

Timeout with fallback. Set a hard timeout on the bureau call — 120ms is typical. If the bureau does not respond in 120ms, proceed without bureau data using an internal-only model. The internal model is less accurate but still makes a decision. The alternative — waiting 400ms for the bureau and timing out the merchant — is worse than a slightly less precise decision.

Cached bureau data. If the customer has a recent bureau pull (within 24–48 hours), reuse it. Bureau data changes slowly — a new tradeline or a 30-point score shift between consecutive pulls is rare. The regulatory constraint: some lending regulations require a pull within a specific recency window. Know your regulatory requirements before caching.

Strategy	Latency impact	Accuracy tradeoff
Pre-fetch at browse time	Eliminates from critical path	None if data is fresh
120ms hard timeout + fallback	Caps bureau contribution at 120ms	Internal-only model is ~85% as accurate
Cached bureau (24h window)	Eliminates from critical path	Marginal — bureau data moves slowly

Model Inference: Staying Under 30ms

The underwriting model runs after features are assembled. The inference budget is 10–30ms. At this latency, model architecture choices matter.

Gradient-boosted trees (XGBoost, LightGBM) are the standard for credit decisioning. A 500-tree ensemble with 8-level depth runs inference in 1–5ms on a single CPU core. No GPU required. The model binary is 10–50MB, fits in L3 cache, and prediction is pure arithmetic — no matrix multiplication, no attention heads.

Logistic regression is even faster — sub-millisecond inference. Some systems use logistic regression as the primary model and reserve the ensemble for borderline cases.

Neural networks are expensive. A small feed-forward network (3 layers, 256 units) runs in 5–15ms on CPU. Transformer-based architectures are impractical at this latency — a single attention layer on a 512-dimensional input costs 20–40ms without GPU acceleration.

Model type	Inference latency (CPU)	Model size	Accuracy (typical AUC)
Logistic regression	< 1ms	< 1MB	0.72–0.78
XGBoost (500 trees)	1–5ms	10–50MB	0.80–0.85
LightGBM (1000 trees)	2–8ms	20–80MB	0.81–0.86
Feed-forward NN (3 layers)	5–15ms	5–20MB	0.79–0.84

The model runs behind a prediction service — a stateless HTTP or gRPC endpoint that loads the model binary at startup and serves predictions from memory. Affirm and Upstart both run model inference as stateless microservices behind a load balancer, scaling horizontally by adding instances.

The fraud model and the underwriting model can share features but must run as separate inference calls. The fraud model produces a fraud probability; the underwriting model produces a default probability. Policy rules downstream consume both scores.

Policy Rules: The Last 10ms

After model inference, policy rules apply business constraints that the model does not encode. Policy rules are deterministic — if/then logic evaluated against model scores, features, and external constraints.

def evaluate_policy(decision_context):
    if decision_context.fraud_score > 0.85:
        return Decline(reason="HIGH_FRAUD_RISK")

    if decision_context.state in RESTRICTED_STATES:
        if decision_context.loan_amount > state_limits[decision_context.state]:
            return Decline(reason="STATE_LIMIT_EXCEEDED")

    if decision_context.underwriting_score < 0.30:
        return Decline(reason="CREDIT_RISK")

    if decision_context.underwriting_score < 0.50:
        return Approve(
            max_amount=min(decision_context.requested_amount, 500),
            apr=calculate_apr(decision_context, tier="high_risk"),
        )

    return Approve(
        max_amount=decision_context.requested_amount,
        apr=calculate_apr(decision_context, tier="standard"),
    )

Policy rules change frequently — new state regulations, updated risk thresholds, marketing promotions. The rule engine must support hot-reloading without redeploying the service. A common pattern is storing rules in a configuration store (Consul, etcd, DynamoDB) and polling for updates every 30–60 seconds.

The order of rule evaluation matters. Check hard declines first (fraud, sanctions, regulatory blocks) to short-circuit the pipeline. A fraud score above 0.85 should decline before the system wastes cycles on underwriting, offer generation, or APR calculation.

Policy rules encode regulatory requirements that change per jurisdiction. A model trained on national data cannot learn that Connecticut caps buy-now-pay-later fees at $5 per month — that is a rule, not a pattern.

Fallback Strategies

The system must return a decision even when components fail. Every stage in the pipeline needs a degraded mode.

Component failure	Fallback	Impact
Feature store down	Use batch-computed features from cache (hours stale)	Slightly less accurate scores
Bureau timeout	Internal-only model (no bureau features)	~15% accuracy reduction
Fraud model timeout	Apply conservative fraud rules (rule-based)	Higher false positive rate
Underwriting model timeout	Use pre-computed risk tier from nightly batch	Coarser segmentation
Policy service down	Apply default policy (conservative limits)	Lower approval rate
All models down	Decline with retry token	Customer can retry in 30s

The fallback hierarchy follows a principle: degrade accuracy before degrading availability. A slightly wrong approval is better than no response. A definitive decline is better than a timeout. The one exception: if the fraud model is unavailable and you cannot assess fraud risk, declining is safer than approving blind.

Affirm’s system implements cascading fallbacks — if the primary decisioning path fails, the request falls through to progressively simpler models. The simplest fallback is a lookup table: customer risk tier (computed nightly) × merchant category × loan amount → approve/decline. The lookup table makes no API calls, runs no models, and returns in under 1ms.

Observability in the Hot Path

Every decision must be logged with enough context to reconstruct it later — for model monitoring, regulatory examination, and dispute resolution. But logging on the hot path competes with the latency budget.

The pattern is async logging. The decision service returns the response to the caller synchronously, then writes the decision record to a Kafka topic asynchronously. A downstream consumer persists the record to a data warehouse. The decision record includes every input feature, every model score, every policy rule evaluated, and the final decision with reason codes.

{
  "decision_id": "d_8f3a2b1c",
  "timestamp": "2026-03-15T14:23:07.842Z",
  "latency_ms": 187,
  "customer_id": "cust_9x2k",
  "features": {
    "bureau_score": 712,
    "txn_count_30d": 14,
    "utilization_ratio": 0.43,
    "bureau_source": "prefetch_cache",
    "feature_staleness_seconds": 847
  },
  "scores": {
    "fraud": 0.12,
    "underwriting": 0.67
  },
  "policy_result": "APPROVE",
  "offer": {
    "max_amount": 1200,
    "apr": 0.1499,
    "term_months": 4
  },
  "fallbacks_used": []
}

The fallbacks_used array is critical for model monitoring. If 30% of decisions last Tuesday used the internal-only model because the bureau was slow, your model performance metrics for that day are not comparable to a normal day. Without tracking which fallbacks fired, you cannot distinguish model degradation from infrastructure degradation.

Latency percentiles are the primary operational metric. Track p50, p90, p95, and p99 separately for each pipeline stage. A p50 of 120ms with a p99 of 380ms means 1% of your decisions are timing out. That 1% is not randomly distributed — it clusters around bureau slowdowns and feature store latency spikes. Alert on p95 crossing 200ms, not on average latency. Average latency hides the tail that kills your approval rate.