TokSeq documentation/Models

Features and models

The feature families, the model classes that were tried, how ensembles are built, how probabilities are calibrated and how uncertainty is reported.

Feature families

Features are organized into named blocks so that a hypothesis declares which blocks it consumes and the search space over feature sets is explicit rather than accidental. Roughly twenty-five blocks span about a hundred and fifty columns.

• Price and momentum – returns, multi-horizon momentum, trend strength, channel position.
• Volatility – realized volatility over several windows, average true range and its normalization, band width, compression ratios.
• Statistics and information – autocorrelation, skewness, kurtosis, variance ratio, Hurst exponent, permutation entropy, spectral features.
• Oscillators and trend extensions – the classical indicator set, kept as a controlled baseline rather than as a belief.
• Seasonality – cyclic encodings of hour, day and month, plus session flags.
• Market structure – levels, swings, compression, liquidity structure with equal-high and equal-low detection and sweep flags, and on demand shape matching by dynamic time warping and matrix-profile motifs. Thirty-eight structural columns exist; the expensive shape blocks are computed only when a configuration asks for them, which is the difference between a sub-second and a several-minute feature build.
• Derivatives context – funding level and change, open interest change and z-score, implied volatility level and change.
• Cross-asset and macro – equity and dollar index returns, a metals proxy, policy rate, curve slope, money supply, balance sheet, and rolling cross-correlations with lead-lag variants.
• On-chain – activity, throughput, security and fee metrics as z-scores, plus mempool state.
• Narrative – news counts and sentiment by category, event proximity, event impact and event-window flags.
• Order flow – depth level and trend, book skew, taker imbalance and cumulative delta slope, spread z-score, and a liquidation cascade measure against a causal daily quantile.

Two disciplines apply to all of them. Every feature is past-only: a bar at time t sees nothing after t, and only closed bars are used. Every threshold inside a feature is a causal rolling quantile of the feature's own distribution, not a constant tuned once on the whole history.

Model classes tried

The search was not restricted to one family. What was executed, in rough order of complexity:

• Linear – standardized logistic regression, kept throughout as the honest baseline.
• Rule banks – classical strategy logic across moving-average crossovers, oscillators, breakout channels, directional index and mean reversion, combined by majority vote, conjunction and disjunction. No fitting at all. These exist to answer whether the machine learning is earning its keep.
• Gradient boosting – two independent implementations.
• Tree ensembles – random forest and extremely randomized trees.
• Neural sequence models – multilayer perceptrons, long short-term memory and gated recurrent networks.
• Deep forecasting architectures – a transformer family including patch-based, frequency-decomposed, inverted and exogenous-aware variants, alongside decomposition-linear, basis-expansion, hierarchical-interpolation and temporal convolutional models.

An unknown architecture name is a hard error. It is never silently downgraded to the baseline, which is a specific defect the platform found in its own history: results once attributed to a tree ensemble had in fact been produced by logistic regression because the dispatch branch did not exist. Those results were withdrawn and the audit that found it is documented.

Ensembles

Five aggregation methods were implemented and compared: mean, performance-weighted mean, majority vote, regime routing, and a trained stacking combiner over out-of-sample member probabilities plus the regime state.

The measured result is unambiguous and it is the reason the live system looks the way it does: only the trained stacking combiner works. Averaging, voting and regime routing are dead. Every member of the live registry is a stack.

Two further ensemble results are worth stating because they are counterintuitive:

• A committee does not reliably beat its own best member. Across 23 898 comparisons, committees beat their best member in 22.5 per cent of cases, with a median difference of −0.0294.
• Committees built from rejected members pass their own screen at a rate close to the screen's chance level, which is exactly what a screen should do and exactly what an unvalidated pipeline would not show.

A published-looking claim that a stacked combiner beats voting collapsed under a fair stratification: the effect measured on a biased stratum was five times its unbiased size, and the unbiased comparison was not significant.

Live aggregation

The aggregation across committee members in the live engine is deterministic. There is no learning in the live path, and a trainable orchestrator is admissible only as a pre-registered research stage.

Freshness is checked first – member signatures older than their bound are dropped. Surviving probabilities are mapped through a per-committee calibration map; a member without a map is used with halved confidence rather than trusted. Weights come from live reliability tracking with a maturity guard, so an immature member carries neutral weight, and a floor prevents any member from being weighted out entirely. The aggregate is a reliability-weighted median. A quorum requirement and a dispersion threshold both produce explicit abstention rather than a low-confidence number.

Calibration

Probability calibration uses isotonic regression, exported as monotone piecewise-linear maps, trained offline and applied statically. There is no Platt scaling anywhere in the code, and there is no refitting in the live path. A map that fails a monotonicity check at load time raises rather than loading.

Maps can be conditioned on regime. The multi-class regime transition model uses per-class isotonic calibration with renormalization. Training scripts hold out the final third of the out-of-sample period to measure calibration error on data the map never saw, then build the shipped map on the full out-of-sample period.

Calibration drift is monitored as a one-way degradation signal over a rolling window of about thirty thousand rows. It can trigger a freeze. It never triggers a refit.

The decision to calibrate at all was itself tested by removing the map: expected calibration error rose from 0.0187 to 0.0306, Brier score from 0.2495 to 0.2515 and log loss from 0.6922 to 0.7261, against 0.6931 for the constant one-half forecast. A separate attempt to refit the committee calibration produced an area under the curve of 0.5263 and was refused by its gate.

Uncertainty

Uncertainty is reported, not hidden in a point estimate:

• The realized-volatility forecast is a quantile model producing a tenth, fiftieth and ninetieth percentile band rather than a symmetric interval.
• Every retrospective statement carries its count and a Wilson interval.
• Committee dispersion is published as an explicit disagreement metric with its own trend, and crossing its threshold produces abstention.
• Expected calibration error is tracked alongside discrimination, because a well-ranked but badly calibrated probability is useless for sizing.
• Effective sample size replaces nominal count wherever labels overlap.
• A shadow-null placebo runs daily in the live system: outcomes are shifted by one and the hit rate must sit at about one half. A placebo that starts winning is a leak, and it is treated as one.

Registered forecasters

Three models are registered in the live engine, all gradient boosting, each with a declared gate it had to clear:

The 0.744 figure carries a mandatory caveat wherever it is quoted, including here. It is walk-forward out-of-sample on the training epoch of the artifact, not a measurement of live behaviour, and the target compares future realized volatility to a trailing estimate, so a substantial part of the absolute number is the trivial mean reversion of a noisy estimator. On a synthetic series with constant volatility the trivial score reaches about 0.83. The honest margin of the benchmark model over the trivial one on real five-minute bars is 0.02 to 0.04. Reading 0.744 as "expansion is called correctly three times in four" is wrong.

The gate for the regime model is deliberately change-AUC rather than macro-AUC, because regime is persistent and macro-AUC is inflated by that persistence into a number that looks impressive and means nothing.