Safety Model
TRADEOS.tech is designed to fail safely. Every component defaults to blocking trades when it cannot confirm that prerequisites are met. This design principle — fail-closed — is enforced at every stage of the trade path.
What fail-closed means
A system can fail in two ways: fail-open (continue operating in a degraded state) or fail-closed (halt until the problem is resolved). Fail-open systems can compound losses during outages — a stale signal, a missed risk check, or a corrupted position state can all produce outsized losses if trading continues. TRADEOS.tech chooses the alternative: when uncertain, stop.
This is not just a convention. It is enforced at the code level. The final gate on the trade path wraps its entire evaluation in a try/except block where the except clause returns False. No exception — hardware fault, software bug, data corruption — can produce a trade approval verdict.
The safety gate inventory
| Scenario | What happens |
|---|---|
| Feasibility service crashes | No approval verdicts are produced; execution has nothing to execute — trading stops naturally |
| Redis unavailable | Cannot read balance, drawdown state, or policy settings; feasibility rejects all signals until Redis recovers |
| Database unavailable | Cannot write position records; execution blocks all order submission until the DB reconnects |
| Secret store unavailable | Cannot retrieve exchange API keys; no orders can be authenticated |
| Market data stale | Ingestion lag exceeds threshold; the ingestion lag circuit breaker blocks all signals from entering the pipeline |
| Policy file tampered | Policy hash mismatch detected; feasibility enters quarantine mode and rejects all signals |
| Heartbeat missed | Any critical service (feasibility, execution) misses a configurable number of consecutive heartbeats; all services halt trade entry |
| Infrastructure instability | Database or Redis reconnection rate exceeds threshold; the infra circuit breaker blocks trading until stability is confirmed |
Circuit breakers
Circuit breakers are individual safety gates that monitor specific failure conditions and trip when thresholds are breached. Once tripped, a circuit breaker blocks the affected trade path until the condition clears and the breaker is reset.
TRADEOS.tech has several circuit breakers operating in parallel:
Ingestion lag circuit breaker — monitors the age of market data arriving from exchanges. If data exceeds a configurable staleness threshold, all signals derived from that data are suppressed. Stale data produces stale signals; stale signals produce bad trades.
Infra circuit breaker — monitors the rate of database and Redis reconnection events. A sudden spike in reconnections indicates infrastructure instability (overloaded host, network partition, OOM kill). The circuit breaker trips to prevent the system from trading on potentially inconsistent state during recovery.
Drawdown circuit breaker — monitors portfolio drawdown from the high-water mark. When drawdown exceeds the profile threshold, the circuit breaker reduces position sizing and eventually blocks new entries entirely until the portfolio recovers.
VPIN circuit breaker — monitors the Volume-synchronized Probability of Informed trading (VPIN) indicator. High VPIN values indicate potential flash crash or informed-trading conditions. The circuit breaker suppresses long entries when VPIN exceeds the configured threshold.
Dead man's switch
Every critical service publishes a heartbeat to a Redis key on a fixed interval. The dead man's switch monitors these heartbeats. If any critical service (feasibility, execution) misses a configurable number of consecutive heartbeats, the switch fires:
- A
HALTcommand is published to therisk.eventsstream - All services that consume
risk.eventshalt new trade entry - The dashboard displays a critical alert
- The condition persists until the affected service recovers and resumes heartbeats
This guarantees that a silently failed service — one that has stopped processing but not crashed — cannot leave the system in a state where trades enter without proper risk checks.
Position state conservatism
If TRADEOS.tech cannot confirm the current state of a position (open or closed), it assumes the position is open. This conservative assumption prevents double-exposure — entering a new trade on an instrument when a prior trade may still be active. The position state is reconciled against the exchange API on every reconnection.
What this means in practice
The practical consequence of fail-closed design is that TRADEOS.tech errs on the side of missed opportunities rather than uncontrolled losses. During a transient infrastructure problem — for example, Redis briefly unavailable — trading pauses. No trades are entered with incomplete information. When the infrastructure recovers, trading resumes automatically.
This behavior is intentional and non-negotiable. Operators who want to override safety gates to capture opportunities during infrastructure events must do so explicitly through the configuration system — it cannot happen accidentally.