TokSeq documentation/Architecture

System architecture

The service boundaries, the decision pipeline, the dependency graph between cores and the invariants that hold the whole thing together.

The platform is a monorepo of independent services, each owning its own data, its own documentation and its own deployment unit. Services communicate through files and read-only contracts, not through shared mutable state. There is exactly one writer per dataset.

Service layout

ServiceResponsibility
CollectorContinuous acquisition of market data into the single source of truth. The only writer of that tree.
FeedsIsolated experimental acquisition of candidate sources. Writes to a separate root, never into the source of truth, and computes no derived features.
CoreThe live decision engine, its dashboard backend and the paper desks. Reads the data tree read-only.
ResearchOffline hypothesis generation, execution and validation against collected history. Never touches live data feeds.
Edge and telemetryTLS termination, authentication, rate limiting, service health and alerting.

The separation between Collector and Feeds is deliberate. A candidate source with an unstable schema or an unverified provenance must not be able to corrupt the tree that research and the live engine both depend on. Feeds asserts its isolation on every write and refuses to start if its root is nested inside another service's root.

The decision pipeline

The live engine runs a fixed pipeline per cycle:

  1. Data feed – closed bars and aggregates are read from the silver layer, read-only. If the data lag exceeds its bound, every signal in the cycle is marked stale before anything else happens.
  2. Feature store – features are computed once per cycle and memoized by (name, asset, timeframe). An unknown feature name raises rather than returning a default.
  3. Cores – ten decision cores execute in topological order of a lightweight dependency graph.
  4. Resolve – a chain of one-way downgrade rules. Rules may suppress a signal, never promote one.
  5. Post-resolve aggregates – narrative, agreement and a single structured situation object.
  6. Snapshot – a versioned contract written to the dashboard store and served by the API.
  7. Decision journal – every signal of every cycle, including suppressed ones, written to a diagnostic-only branch.

Two cadences drive it: a fast cycle on the five-minute close carrying the cheap cores and the transferred heavy cache, and a heavy cycle on the hourly close that recomputes everything including the committee. A daily maintenance pass settles the journal, runs the degradation monitor, drives the kill engine and evaluates a shadow-null placebo.

The dependency graph

Cores declare what they require and what they produce. The runtime performs a stable topological sort over those declarations at startup. Two properties fall out of this and both are enforced loudly:

Risk requires volatility and regime. Volatility requires regime. Direction deliberately requires nothing, so that a failure elsewhere cannot manufacture a directional opinion.

The ten cores

CoreProduces
RegimeCurrent regime, probability of change within the hour, next-state distribution, multi-timeframe coherence
VolatilityTrailing realized volatility, an expansion probability, a quantile forecast band, and a binary risk veto
StructureLevels, swings, sweeps, compression and liquidity structure
DirectionCommittee aggregate, with abstention as the default outcome
RiskStop distance as a regime-dependent multiple of ATR, position size, volatility-targeted leverage with an expansion damper
ContextCross-asset and macro correlation, cross-asset regime coherence
KnowledgeRetrospective hit rates per regime with sample sizes and confidence intervals, plus live forecaster scoring
NewsMultilingual sentiment of the news feed with an age guard
DeltaState deltas over 15, 30 and 60 minutes, streaks and a 24-hour memory
MicroOrder-flow sensor over one-minute aggregates

The micro core carries an explicit invariant: it is a sensor describing what is happening now, not a directional predictor. The platform's own audit found no directional edge in order flow, and the core is not allowed to claim one.

Committee aggregation

Where a directional opinion exists at all, it is produced by a committee of trained stacking ensembles rather than by a single model, and the aggregation across committee members is deterministic. There is no learning in the live path. The aggregation applies, in order: a freshness filter on member signatures, a monotone probability calibration map per committee, reliability weights with a maturity guard so that an immature member cannot dominate, a reliability-weighted median across calibrated probabilities, a quorum requirement, and a disagreement threshold.

The median rather than the mean is the load-bearing choice: one broken member moves a mean and does not move a median.

Invariants

These hold across the whole platform and each of them was written after an incident, not before: