TokSeq documentation/Engineering

Engineering practice

The stack, the deployment discipline, the security posture and the working rules that were written after incidents rather than before them.

The research is only as good as the machinery that produces it, and most of the rules below exist because something broke first.

Stack

The services are Python, with a columnar store for market data and embedded databases for journals and dashboard state. Model training uses gradient boosting and, for the heavy search stage, GPU-backed sequence models. The live engine runs on a single node under process supervision with per-service resource quotas; the public edge runs in containers behind a reverse proxy that terminates TLS.

The deliberate absences matter as much as the choices. There is no message bus, no orchestration cluster and no general-purpose metrics stack. For a single-node deployment each of those adds a failure mode without removing one, and the platform's own incident history is dominated by coordination failures rather than by capacity.

Deployment discipline

Testing and documentation as a gate

Each service carries a test suite and lint gates. Beyond that, the repository enforces its own bookkeeping with a documentation linter, and the rule it enforces is unusual enough to state:

A task may only be marked complete if the completion carries evidence – a file and line, a commit hash, a production acceptance date, or a test run result. The assertion "done" is not evidence. A deliberate refusal is also a valid closure, but it must carry the measurement that justified it.

That rule was written after an audit of 896 tracked items found 156 marked open while already implemented, 328 items from three reports that had never reached any task list, and 137 documents that contradicted themselves. The cost of a bookkeeping defect in a research project is not tidiness; it is repeated work and lost decisions.

The same audit produced a decision register: every operator decision is recorded in the repository, with its date and its wording, in the same session in which it was made. Agent memory is not a substitute, because the register has to be readable by anyone who reads the repository.

Security posture

The platform holds no customer funds and no customer data, and its exposure is a private analytics surface. The posture is correspondingly plain:

Working rules

A set of process rules is enforced across the project because each of them has a measured cost attached:

What runs where

Research runs offline against a frozen store, on the operator's own hardware, and never pulls live data. The live engine and its surfaces run on a single server. Data collection runs continuously and is the only writer of the source of truth. Nothing in the research path can write into the live path, and nothing in the live path can write into the collected history.