Engineering
How the terminal is built
Here is the plumbing: from the feeds to the store to your screen, and the contract each joint keeps.
upstream feeds
A React frontend, a FastAPI backend, a binary wire between them, background workers and Postgres. None of that is unusual on its own.
This page is about the contract that ties them together: how a row’s freshness travels from the database to the pixel, and what we got wrong on the way.
A ticker hits the search bar, and a Price Chart, KPI rail, Convexity Score, and freshness badge come up on screen. Here is what happens in between.
A route loads via React.lazy and useComprehensive("AAPL") fires. TanStack Query shows a cached payload at once if it holds one, and refetches in the background.
The request leaves the browser over HTTP/2 with an Accept: application/msgpack header. Responses come back as msgpack, decoded into typed objects without a string-parse hop.
A thin handler calls a services/… function, which reads Postgres (SQLAlchemy, asyncpg) and then Redis for the live quote, without blocking. One token bucket in Redis paces FMP calls across prod’s four uvicorn workers.
A response is { data, meta }, and meta.data_as_of is computed from the rows (MAX(transaction_date)), never from datetime.utcnow(). Two requests five minutes apart agree unless a row landed.
The ticker page’s company payload and its quote are not wrapped.
Every surface that shows enveloped data is required to render the one shared badge. “Last updated…” anywhere else in the frontend is a review-blocking violation.
JSON is fine for people and wasteful between machines: every payload repeats braces, quotes and keys, and every parse walks a string into a tree that two services sharing a schema do not need.
The terminal asks for msgpack by default. Any client that does not ask, curl included, gets JSON.
# JSON, compact -- 198 bytes {"ticker":"AAPL","price":217.42,"change":1.84, "change_pct":0.85,"volume":42180000, "ts":"2026-05-24T19:58:31.412Z", "meta":{"data_as_of":"2026-05-24T19:58:30Z", "sync_status":"healthy","source":"fmp"}} # MsgPack -- 173 bytes (-13%) 87 a6 74 69 63 6b 65 72 a4 41 41 50 4c a5 70 72 69 63 65 cb 40 6b 2d 70 a3 d7 0a 3d a6 63 68 61 6e 67 65 cb 3f fd 70 a3 d7 0a 3d 71 aa 63 68 61 6e 67 65 5f 70 63 74 cb 3f eb 33 33 33 33 33 33 …
Three surfaces showing the same data must show the same value and the same freshness. Without a contract they drift, and a sharp user notices in seconds. The contract:
Every sync run is required to write a row to data_sync_log, failures included, so sync_status is computed from the log instead of inferred. A successful run looks like this:
source sec_edgar_form4 started_at 2026-09-29 16:30:01.047 UTC completed_at 2026-09-29 16:30:30.610 UTC status success rows_added 67 error null
No rows means data_as_of is null, not Date.now(), and the envelope still returns, so the frontend shows “No data” instead of a crash. The time on screen is when the data is from, not when the request was made.
A six-feature Ridge composite (momentum, insider net flow, smart-money positioning, low-vol beta, gamma, revenue-growth acceleration) forecasting the sector-neutral 180-day forward return.
It refits weekday nights at 03:00 UTC for large caps above a $25B floor. Every published vintage is immutable and scored against realized returns as its window matures.
The methodology is open at /methodology/convexity-score, and CONVEXITY_SYNC_ENABLED turns production off in one flip.
Every AI call goes into the usage ledger by surface, model and token count. The ledger backs the cost claims on pricing, and per-user monthly caps are enforced server-side. Routing is by intent, not by provider:
| Intent | Default model | Fallback | Notes |
|---|---|---|---|
| Chat | claude-sonnet-5-5 | gpt-6-luna | Its own lane since 2026-09-29. Circuit breaker shifts traffic instantly on provider degrade. |
| Synthesis | gpt-6-luna | claude-sonnet-5 | Luna took this lane 2026-08-10, gpt-6-luna since 2026-09-28; the fallback crosses vendors in both directions. |
| Embeddings | voyage-4-large | – | RAG over SEC filings, call transcripts and press releases; queries embed with voyage-4-lite in the same space. |
| Search-grounded | sonar-pro | – | Added 2026-05-04, used by the search_analyze tool only. |
| News sentiment | gpt-6-luna | claude-haiku-5-5 | One label per ingested headline, stored with the model that gave it. The fallback since 2026-10-07, measured level with luna on headlines. |
| Deep Research | claude-opus-5-5 | – | Pro Plus only, 20 runs/mo. Target ≤ 4.5 search calls per report. |
| Cheap lookups | gpt-6-luna | claude-sonnet-5 | Free tier default; prompt cache hits where possible. A failed chat turn falls back to claude-haiku-5-5 instead. |
routing.py and tracked through the marketing pricing data, so a model commitment on the site is also a default in the code.| Metric | Target | How |
|---|---|---|
| Initial JS bundle | per-route | React.lazy + Suspense on every page except sign-in, activation and 404. |
| Loading state | skeletons | Content-shaped placeholders by default; some surfaces still show a spinner or plain text. |
Every system has receipts on the things it learned the hard way. Ours, briefly:
The Wave 1 envelope migration wrapped a response key inside data without bumping the cache buster in queryClient.ts, so every returning user’s cache still held the old shape.
TerminalDashboard crashed on first render with TypeError: Cannot read properties of undefined (reading 'total_count'). Fixed in eeacde3, and now a shape change and its cache bust land in one commit.
Quality Score v1 came back negative in backtest: a top-minus-bottom spread of -0.7% at four quarters, -3.1% at eight and -1.9% at twelve. We closed the pull request unmerged and shipped the useful part, the point-in-time helpers, on its own.
A score that looks plausible and quietly underperforms costs far more than a killed PR.
Five retained releases at 1.2 GB each, plus three partials from failed deploys the script never pruned, took a 96 GB volume to 100% on a weekday morning. We cleared the partials first, then the old releases.
The deploy script now prunes its own partials and keeps three releases, not five. The real fix was treating the disk as a feature, not a constant.
We did not build a language, an HTTP framework or an encryption library. We built on the people who did, and these are the projects that matter most.
data_sync_log row, every canonical query.We are looking for engineers who care about real-time systems, financial data, and shipping fast.