Follow one data point through one 24-hour cycle — from a raw price bar or SEC filing to a graded probability on this site. Every stage below runs unattended, every market day.
News and sentiment, analyst estimates, earnings dates, dividends and splits, insider filings, and macro releases are pulled before the open. Scraping runs on a Celery task queue with an explicit task registry (no dynamic imports), per-API rate ledgers, and idempotent tasks — a failed call retries without double-writing. At 9:10, real-time pre-market quotes land so the day's predictions price entry realistically.
A compact morning workflow rebuilds the warehouse increment, refreshes the model-ready dataset, and writes today's probabilities from the latest trained predictors — reviewed by 9:20. Entry is the next session's open; the barrier is entry × target, rounded to the cent.
Raw scrapes land in source tables and flow through staging, fact, and dataset layers in a single dbt project. Hourly EDGAR polls catch new 10-K/10-Q filings; a daily sweep picks up Form 4s, 13Fs, and 8-Ks. The SEC chain is self-sequencing: filings → metrics → fiscal assignment → periods and cleaning, each stage queuing the next.
Split-adjusted daily bars for every covered ticker, cross-checked between two independent price vendors. 14,164 daily bars and counting — roughly 26 years of history per ticker.
Models are gradient-boosted classifiers (CatBoost) answering barrier-hit questions: P(high ≥ +5% of next open within 5 trading days). Each model keeps one trained predictor per month-end — 13 of them, rolling — and every retrain only ever sees data available at its train date. Training rows require the full outcome window to be closed, so a label can never leak the future.
Technical indicators recompute, dbt rebuilds, the dataset re-exports, missing predictions backfill, and then the honest part: every past prediction is scored against realized prices. Out-of-time AUC per retrain, calibration by prediction decile, and realized hit rates of top picks versus the whole scored universe — published, including the misses.
209+ pytest tests run offline against recorded API fixtures — no network, no flakiness. The target-labeling logic is characterization-tested against a hand-verified gold fixture that code output is never allowed to regenerate. The 46 dbt tests guard the warehouse invariants on every build. If it isn't tested, it doesn't ship.
The 2026 rebuild — scrapers, warehouse, models, this site — was executed with agentic AI workflows doing the heavy lifting and the test suite deciding what's true. Design invariants live in version-controlled docs the agents must follow; the result is months of platform work compressed into days without giving up rigor.
Publish-timestamp joins, vintage-aware macro, outcome-window-closed labels — enforced by tests, not discipline.
Barrier-hit metrics, out-of-time scoring, calibration by decile, base rates beside every probability.
One queue and dispatcher, explicit task registry, JSON-only messages, idempotent tasks, per-API rate ledgers.
209+ pytest tests offline by default; a gold fixture no code may overwrite; dbt builds green or nothing ships.
The whole system — data, models, grades, and this site — regenerates from source every market evening.