Skip to main content

Signal Pipeline

The signal pipeline is the path a market event takes from raw price tick to a tradeable intent. It spans three services — Signal Engine, Signal Modifier, and Feasibility — each adding progressively more context before a trade decision is made.

Stage 1 — Market data ingestion​

Market data arrives via WebSocket from multiple venues simultaneously. The ingestion service:

  • Normalizes all ticks into a canonical MarketTick format (symbol, price, volume, bid/ask, timestamp)
  • Deduplicates ticks across venues (uses the median price for cross-venue symbols)
  • Publishes to the market_data Redis Stream at sub-100ms latency

On-chain and macro data (funding rates, DeFi TVL, FRED macro indicators) are polled on slower cadences and merged into the market context used by relevant signal generators.

Stage 2 — Signal generation​

The signal engine maintains a registry of signal generators. On each market tick, all relevant generators are evaluated in parallel. Each generator returns:

SignalEvent {
  signal_type:  str         # e.g. "momentum", "mean_reversion", "carry"
  symbol:       str         # trading pair
  direction:    str         # "long", "short", or "flat"
  confidence:   float       # 0.0 → 1.0
  metadata:     dict        # signal-specific context
  timestamp:    datetime
}

Signal types currently in production include momentum, mean-reversion, RSI-volume divergence, order flow imbalance, funding rate carry, correlation breakdown, and on-chain flow signals.

Regime gating is applied at this stage. The HMM regime classifier continuously classifies the current market state. Signal generators that are incompatible with the current regime are suppressed — their output is dropped before it reaches the stream. This prevents the system from taking momentum trades in a choppy, mean-reverting environment.

Stage 3 — Signal modification​

The signal modifier applies a set of adjustments to the raw signal stream:

Win-rate streak weighting — generators with a recent streak of accurate predictions receive a confidence multiplier. Generators with a losing streak are downweighted. This is computed per signal type using a rolling window tracker.

Information coefficient (IC) scaling — the IC of each signal type (correlation between predicted direction and actual outcome) is tracked over a rolling window. High-IC signals receive a scoring bonus; low-IC signals are dampened.

Hero signal promotion — the top N signal types by combined IC and win-rate are designated "hero signals" and receive priority processing. The hero list is re-evaluated weekly by the autonomous agent.

Opposing signal suppression — if long and short signals exist simultaneously on the same instrument (from different generators), they are evaluated for conflict. Where confidence is similar, both are suppressed. Where one dominates, the weaker is dropped.

The output of the modifier is the signals.modified stream — a cleaner, higher-quality feed ready for feasibility evaluation.

Stage 4 — Feasibility validation​

The feasibility service is the last gate before an order can be placed. It evaluates:

  • Minimum confidence threshold — signals below the profile's minimum are rejected outright
  • Position sizing — Kelly criterion calculation with portfolio-level constraints
  • Risk limit compliance — checks against all active limits (see Risk Management)
  • Market impact assessment — Almgren-Chriss model estimates impact at proposed size; if impact-adjusted expected value is negative, the trade is rejected

If all checks pass, the intent is published to the intents stream with a final approved size. If any check fails, a rejection event is logged with the specific reason.

Pipeline latency​

End-to-end latency from market data arrival to order submission is typically under 200ms in normal conditions. The dominant cost is the feasibility calculation, which involves multiple Redis reads and a Kelly sizing computation.

Signal generation and modification add approximately 5-20ms depending on the number of active generators and the complexity of the current portfolio state.