Engineering

Real-time is aplumbing problem.

Here is the plumbing: one request from browser to store, and the contract each hop is required to keep.

Start freeTrace a request →

Founder pricing holds for life while the waitlist is open.

  1. 100+live data feeds20+ named sources · none synthetic, none mocked
  2. <100mswarm-read p95 targettracked on /status
  3. 30-40%payload reduction on the wiremsgpack vs json
  4. 1freshness badge, everywhereDataEnvelope[T] contract

Convexity is a Vite + React + TypeScript frontend talking to a Python FastAPI backend over a binary wire, with ARQ background workers feeding a SQL store from a hundred-plus feed jobs across twenty-plus named upstreams. None of those choices are interesting on their own. What is interesting is the contract that ties them together, the way data freshness travels from the database row to the pixel a user sees, and the handful of things we got wrong on the way here. This page is about that.

01: REQUEST LIFECYCLE

The path of a quote

A ticker hits the search bar. Three hundred milliseconds later a Price Chart, KPI rail, peer cohort, Convexity Score, and freshness badge are on screen. Here is what happens in between.

  1. 1browser

    React 18 + TanStack Query

    A route loads via React.lazy; useStockDetail("AAPL") fires. TanStack Query checks its in-memory cache, returns a stale-but-shown payload immediately if present, and kicks a background refetch.

  2. 2axios + msgpack

    Binary wire

    The request flies as HTTP/2 to FastAPI with an Accept: application/msgpack header. Responses come back 30-40% smaller than JSON, decoded into typed objects without a string-parse hop.

  3. 3FastAPI + asyncio

    Async-first handler

    The route handler is a thin shell over a services/…/queries.py function. It pulls from Redis first (hot reads never touch the DB), then SQLAlchemy + aiosqlite. Every I/O hop is non-blocking on a single uvicorn worker, with semaphores around upstream calls so we cannot stampede a third-party rate limit.

  4. 4DataEnvelope[T]

    One shape, every endpoint

    The response is never raw data. It is { data, meta } wheremeta.data_as_of is computed from the rows themselves (MAX(transaction_date)), not from datetime.utcnow(). Two requests five minutes apart return the samedata_as_of if no new rows landed.

  5. 5DataFreshnessBadge

    The pixel

    Every surface that shows the data renders the same shared badge component. The string "last updated…" appearing anywhere else in the frontend is a review-blocking violation. Consistency is enforced by the component, not by good intentions.

02: THE WIRE

MsgPack, not JSON

JSON is a fine human-readable interchange format and a wasteful machine-to-machine one. Every payload pays for braces, quotes, and key repetition; every parse spends CPU on a string-to-tree walk that does not need to exist between two services that already agree on a schema.

Convexity speaks msgpack on the wire by default. Browsers do the binary decode in a worker, so the main thread does not block. JSON is still available behind a content-type header for anyone debugging with curl.

msgpack/msgpack-javascript · spec

GET /api/quote/aapl: wire bytes
# JSON  --  214 bytes
{"ticker":"AAPL","price":217.42,"change":1.84,
 "change_pct":0.85,"volume":42180000,
 "ts":"2026-05-24T19:58:31.412Z",
 "meta":{"data_as_of":"2026-05-24T19:58:30Z",
         "sync_status":"healthy","source":"fmp"}}

# MsgPack  --  138 bytes (-36%)
82 a6 74 69 63 6b 65 72  a4 41 41 50 4c a5 70 72
69 63 65 cb 40 6b 2b ae  14 7a e1 48 a6 63 68 61
6e 67 65 cb 3f fd 70 a3  d7 0a 3d 71 aa 63 68 61
6e 67 65 5f 70 63 74 cb  3f eb 38 51 eb 85 1e b8
…
fmp_ws: tail
19:58:30.412  tick      AAPL 217.42 +0.85% vol=42.18M
19:58:30.488  tick      MSFT 421.18 -0.12% vol=18.04M
19:58:30.512  tick      NVDA 138.91 +2.31% vol=287.4M
19:58:31.044  reconnect attempt=2 backoff=1.4s jitter=0.21s
19:58:32.661  authed    plan=ultimate symbols=482
19:58:32.704  replay    missed=14 since=19:58:31.038
19:58:33.118  tick      TSLA 251.04 -1.84% vol=98.2M
19:58:36.402  auth_fail code=401 streak=3
19:58:36.403  breaker   state=open  retry_in=120s
20:00:36.918  breaker   state=half-open
20:00:37.221  authed    plan=ultimate symbols=482
20:00:37.221  breaker   state=closed
03: REAL-TIME

The tape, the reconnect, the breaker

The Financial Modeling Prep WebSocket is the primary live-tick channel. Symbols outside the live tape refresh on a two-minute poll, with a client-wide token-bucket rate limiter capping RPS across the FMP REST family.

Reconnects use exponential backoff with jitter and replay missed messages from the last seen sequence, so the client never carries stale state through a disconnect. Persistent auth failures trip a circuit breaker (state machine: closed → open → half-open → closed) that suppresses reconnect storms and surfaces a clean "degraded" pill to the user instead of a busy spinner.

The signal correlation engine consumes the same tape. Cluster detection across insider trades, options flow, and 8-K filings lands on the dashboard signal feed within seconds of the underlying event.

04: THE DATA CONTRACT

Honest freshness, all the way down

Three UIs rendering the same data must display the same value and the same freshness. Without a contract, three ad-hoc implementations drift, and a sharp user notices in seconds. The contract:

upstream
FMP, SEC EDGAR, FRED, …
100+ feeds · 20+ named sources
arq job
background worker
writes data_sync_log
sql store
canonical query fn
one per data source
api
DataEnvelope[T]
data + meta, always
ui
DataFreshnessBadge
one component, every surface

Every ARQ run writes a row to data_sync_log – including failures, sosync_status is always computable from the log instead of inferred. A typical successful run looks like this:

data_sync_log: most recent sec_edgar_form4 row
source           sec_edgar_form4
started_at       2026-05-24 07:30:00.114 UTC
completed_at     2026-05-24 07:32:47.802 UTC
status           success
rows_added       1,284
error            null

If the query returns no rows, data_as_of is null, notDate.now(). The envelope still returns. The frontend renders "No data" from the envelope, not a crash state. The whole point is that the timestamp on screen reflects when the data is from, not when the request was made.

05: THE MATH WE OWN

Four numbers that did not come from a vendor

convexity_score Convexity Score
Capture-ratio asymmetry vs SPY over a trailing 252 days, computed nightly at 03:00 UTC weekdays. Separates names that participate in upside and reject downside from names that do the opposite. Methodology is open at /methodology/convexity-score; production code is gated by CONVEXITY_SYNC_ENABLED so we can kill it in one env-var flip if a sync goes sideways.
peer_cohort Peer cohort selection
Sector peers auto-selected from FMP cohort data with a yfinance fallback when coverage is thin. Cached per-ticker with TTL; nightly refresh keeps the cohort honest as constituents shift. The cache invalidates on data refresh, not on a clock, so users never see a hot-but-stale read.
cluster_detection Insider + congressional clusters
Form 4 buys and STOCK Act PTRs are scored for cluster significance – multi-insider, multi-day, role-weighted. Officer-grade purchases outweigh 10b5-1 plan sales. Clusters land on the signal feed within seconds, with the underlying filing one click away.
point_in_time Point-in-time helpers
Backtest-safe joins and as-of lookups for any factor work. Salvaged from the Quality Score v1 attempt (more on that in a moment) and reused by every screen and study built since. If a factor cannot be evaluated point-in-time, it does not ship.
06: AI ROUTING

Four providers, one interface

Every AI call is recorded by surface, model, and token count in the usage ledger. Per-user monthly caps are enforced server-side; the ledger is what backs the cost-transparency claim on pricing. Routing is intent-based, not provider-loyal:

IntentDefault modelFallbackNotes
Chatclaude-sonnet-5gpt-5.6-lunaCircuit breaker shifts traffic instantly on provider degrade.
Synthesisgpt-5.6-lunaclaude-sonnet-5Luna took this lane 2026-08-10; the fallback crosses vendors in both directions.
Embeddingsvoyage-finance-2voyage-3RAG over SEC filings and curated news.
Search-groundedsonar-pro–Added 2026-05-04, used by the search_analyze tool only.
Financial sentimentfinbert–Domain-specific. Earnings calls, filings, market commentary.
Deep Researchclaude-opus-4-8–Pro Plus only, 20 runs/mo. Target ≤ 4.5 search calls per report.
Cheap lookupsgpt-5.6-lunaclaude-sonnet-5Free tier default; prompt cache hits where possible.
Provider allow-list: Anthropic, OpenAI, Voyage, Perplexity. No Gemini. The list is enforced in routing.py and tracked through the marketing pricing data, so a model commitment on the site is also a default in the code.
07: PERFORMANCE

By the numbers

Targets, not best-case marketing numbers. Cold reads and external-provider round-trips are slower; we are honest about the difference rather than quoting only the warm path.

MetricTargetHow
Warm API read p95<100 msRedis + in-memory tiers; stale-on-refresh, not stale-on-clock.
Cold API read p95<800 msSQLite + canonical query function; semaphores on upstream calls.
Wire payload reduction30-40%MsgPack default; JSON behind a content-type header for debugging.
HMR turnaround (dev)<1 sVite, no full-page reloads.
Initial JS bundleper-routeReact.lazy + Suspense on every route; shared vendor chunk.
Quote cache TTL30 sTanStack Query; fresh enough for active trading without thrash.
Loading stateskeletonsContent-shaped placeholders, no spinners. Layout shift = 0.
08: POSTMORTEMS

Three things we got wrong

Every system has receipts on the things it learned the hard way. Ours, briefly:

  1. 2026-04-30

    The cache-buster we forgot to bump

    The Wave 1 envelope migration wrapped a top-level response key inside data without bumping buster in queryClient.ts. Persisted React Query caches hydrated the old shape against new readers and TerminalDashboard crashed on first render for every returning user, with TypeError: Cannot read properties of undefined (reading 'total_count'). Fixed in eeacde3; the rule is now in the data-contract review checklist. Bump the buster in the same commit that changes the shape, or the change is not done.

  2. 2026-05-07

    The factor that did not work

    Quality Score v1 came back negative in backtest: top-minus-bottom spread of -0.7% at four quarters, -3.1% at eight, -1.9% at twelve. We closed the pull request unmerged and shipped what was actually useful – the point-in-time helpers – in a follow-up. The lesson was not "factors are hard"; the lesson was that the cost of shipping a score that looks plausible and underperforms quietly is much higher than the cost of killing a PR.

  3. 2026-05-01

    The disk that filled up

    Five retained release directories at 1.2 GB each, plus three stale partials from failed deploys that the deploy script was not pruning, took a 96 GB VPS volume to 100% on a weekday morning. Recovery was sudo rm -rf on the failed partials first, then the old successful releases. The deploy script now prunes its own partials and we keep three releases, not five. Cheap fix. The real fix was treating the disk as a feature, not a constant.

09: WHAT WE BUILD ON

The open-source shoulders we stand on

We did not build a programming language, an HTTP framework, or an encryption library. We built a product on top of people who did. Here is the short list of the projects that matter most, and where we plan to contribute back.

  • Starlette + FastAPI
    Every async route, every middleware, every typed response.
  • msgpack-javascript + msgpack-python
    The binary wire format that makes the 30-40% payload reduction free.
  • SQLAlchemy + asyncpg
    ORM + driver. Every data_sync_log row, every canonical query.
  • ARQ
    Background job runner for every sync, every cron, every backfill.
  • Argon2 · pyca/cryptography
    Password hashing and field encryption. The two crypto primitives we trust.
  • TanStack Query + Vite
    Frontend server-state + dev server. Every cached fetch, every sub-second HMR cycle.
  • Astro
    This marketing site runs on it.
Contributions on the roadmap:first PR target is upstream asyncpg – we maintain a fork-only patch for Postgres advisory locks in the ARQ semaphore wrapper that should land in main. Once shipped, the list of patches we carry locally will be linked here with commits.

If this is the kind of plumbing you like

We are looking for engineers who care about real-time systems, financial data, and shipping fast.

See open roles