{
 "title": "How to backtest earnings-call signals \u2014 StockQL",
 "description": "Build a leakage-controlled earnings transcript NLP pipeline with audited sections, post-reaction returns, and issuer-aware uncertainty.",
 "url": "https://stockql.ai/blog/how-to-backtest-earnings-call-signals.html",
 "slug": "blog/how-to-backtest-earnings-call-signals",
 "published": "2026-10-22",
 "site": "StockQL",
 "contact": "hello@stockql.ai",
 "body_text": "Earnings transcript research\n\nHow to backtest earnings-call signals with StockQL\n\nStart with an audited transcript split, a fixed point-in-time panel, and post-reaction returns. Then let uncertainty\u2014not an attractive story\u2014decide whether a signal survives.\n\nOctober 22, 2026 \u00b7 11 minute read\n\nIn this guide\n\nFreeze the research panel\n\nAudit prepared remarks and Q&A\n\nMeasure post-reaction returns\n\nKeep honest nulls\n\nPlug in a new signal\n\nAnswer the common questions\n\nTo backtest earnings-call sentiment, first lock the universe and call identity; separate prepared remarks from questions and answers; compute each feature from information available at the call; skip the immediate market reaction; and estimate uncertainty across both time and issuers. The language model or dictionary is only one replaceable part.\n\nWe built that research rail around StockQL and tested four plausible features: quarter-over-quarter finance tone, explicit references back to prepared remarks, semantic reuse of prepared remarks, and answer evasion. None earned promotion. That outcome is useful. A reusable pipeline should make a clean \u201cstop\u201d decision as easy to publish as a promising result.\n\nThis is research infrastructure, not a trading recommendation. The tested results are bounded to one liquid-U.S. panel from 2023 through mid-2026. The first three features produced bounded nulls; the evasion proxy failed calibration before its market result could be interpreted. None cleared its promotion standard.\n\n1. Freeze the panel before reading the result\n\nStockQL held 173,795 earnings-call transcripts spanning 2005\u20132026 in the research snapshot. The archive is the starting material, not the backtest universe. A historical test still needs explicit eligibility, identity, timing, and outcome rules.\n\nThe fixed base tone/return panel contained 17,641 adjacent-call pairs from 1,844 tickers across 14 call quarters . It used normalized U.S. common shares, retained inactive names, and required a pre-call 60-session median dollar volume of at least $25 million, a last pre-call close of at least $5, complete forward prices, and immediately adjacent fiscal quarters 45\u2013150 days apart. Feature-specific exclusions then left 17,570 calls for ACR and 17,285 events for evasion; those sample sizes are not interchangeable.\n\nLayer Locked rule Failure it prevents\n\nCall identity Fiscal identity, verbatim call date, transcript ID, and content hash Ticker reuse or revised text silently changing a row\n\nUniverse Point-in-time price and liquidity guards; no current-active filter Survivorship and present-day eligibility leaking backward\n\nPairing Immediate fiscal-quarter neighbors with a bounded day gap Comparing unrelated calls because a quarter is missing\n\nAvailability Features use call-time text; forward prices enter only as outcomes The future influencing feature construction\n\nA ticker by itself is not stable transcript identity. Acquisitions, reused symbols, differing fiscal labels, and provider corrections all make that shortcut unsafe. The separate guide A ticker is not an earnings transcript\u2019s identity covers those lineage checks.\n\nStockQL\u2019s contribution here is composition: transcripts, security identity, exchange-normalized prices, and SPY sessions are queryable in one PostgreSQL database. The research layer still owns the hypothesis and its exclusions. Keeping those jobs separate makes a new signal cheaper to test without pretending the database chose the strategy.\n\nThe fixed research panels are local, ignored artifacts\u2014not a downloadable StockQL dataset or a hosted turnkey rail. An entitled self-hosted StockQL user can reproduce or adapt the method against the transcript corpus their data license permits.\n\n2. Audit prepared remarks and Q&A instead of guessing\n\nPrepared remarks and analyst Q&A answer different research questions. Tone in a scripted opening is not interchangeable with language produced under questioning, and a similarity feature needs a defensible boundary between the two.\n\nThe source transcripts did not carry a reliable section field, so the rail used a deterministic character-offset splitter and audited it on a fixed 943-call sample. It produced 886 usable splits (94.0%), identified three legitimate calls with no Q&A, and quarantined 54 unresolved calls. Ambiguity became an exclusion, not a model-generated guess.\n\nThat gate runs before feature scoring. The base panel admits only ok current and prior splits and records each marker tier and transcript hash; downstream feature stages verify the inherited panel SHA, so a parser or source change cannot silently rewrite the sample. A new signal can require both sections, Q&A only, or prepared remarks only without rebuilding the boundary logic.\n\nKeep licensed text inside its permitted boundary. The source corpus is licensed, and the fixed research panels remain local. The public evidence contains aggregates, identifiers, hashes, formulas, and environment records\u2014not transcript excerpts, participant text, raw panel rows, credentials, or connection settings.\n\n3. Measure returns after the call-day reaction\n\nThe outcome was a stock\u2019s cumulative close-to-close return divided by SPY\u2019s cumulative return, minus one:\n\nabnormal return = (stock_end / stock_start)\n/ (SPY_end / SPY_start) - 1\n\nThe start was the first SPY session strictly after the call date. Endpoints were sessions 20, 40, and 60. In the primary 60-session test, call day and the first post-call close did not enter the return. This is the common +2-to-+20/40/60 research shape: ask about drift after the immediate reaction rather than relabeling the announcement move as prediction.\n\nContinuous features were ranked into equal-weight quintiles inside each call calendar quarter; explicit ACR compared calls with any reference against calls without one. The report then showed three views of uncertainty:\n\nthe observation-level top-minus-bottom spread;\n\nthe mean of 14 quarterly spreads with Newey\u2013West lag 1; and\n\na 2,000-draw ticker-cluster bootstrap interval for repeated issuers.\n\nThose views answer different failure modes. Quarter-level estimates expose time instability. Ticker clustering stops repeated calls from one issuer from masquerading as independent evidence. Horizon and robustness cuts show whether a result survives ordinary design choices.\n\nThe two semantic features\u2014implicit ACR and evasion\u2014used one frozen local 256-dimensional static embedding model, minishlab/potion-base-8M , with a pinned snapshot and environment. The tone baseline used a pinned Loughran\u2013McDonald dictionary instead. Neither model nor dictionary received return labels. The three scored pipelines took about 18.3 minutes total on CPU, including roughly 1.50 million chunks embedded for ACR and evasion. \u201cWeekend-scale\u201d is literal here: a new idea can be measured without turning into a new platform.\n\n4. Four attractive signals produced four stop decisions\n\nA pipeline is not validated because it finds a trade. It is validated when identity checks, calibration, uncertainty, and stopping rules remain intact after an appealing number appears.\n\nSignal Primary evidence Decision\n\nQ&A tone delta 60-session Q5\u2212Q1: \u221229.98 bp; ticker-cluster 95% CI [\u2212107.02, +46.26] No-go. Signs changed across horizons and quintiles were not monotone.\n\nExplicit prepared-reference rate 60-session with-reference minus without: \u221211.70 bp; CI [\u221261.47, +41.61] No-go. Small and null across horizon, split-confidence, and question-toughness cuts.\n\nImplicit prepared/answer similarity 60-session Q5\u2212Q1: \u221267.26 bp; CI [\u2212143.05, +7.31] No-go. Suggestive bearish direction, but every interval crossed zero and the sign was ambiguous before observation.\n\nAnswer evasion Positive-control rank AUC: 0.450; expected direction > 0.500 Reject the proxy. Its return result is uninterpretable, not a valid null.\n\nThe implicit-similarity result is the useful temptation. Its length-adjusted 60-session interval ended at +1.69 bp\u2014close enough to zero to invite a \u201cnear miss\u201d story. But several related cuts had been run, quarterly evidence was weak, and the economic sign was not uniquely fixed in advance. Promotion would have converted exploration into an after-the-fact hypothesis.\n\nThe evasion test shows a different stop. A feature can generate a clean numeric column and still fail to measure its label. Literal refusal and follow-up phrases scored less evasive than unflagged answers in the corpus. The validity failure comes before any return interpretation.\n\nRead the decision report \u2192 Inspect the evidence appendix \u2192\n\n5. Plug a new transcript hypothesis into the existing rail\n\nWithin this research implementation, a new hypothesis should replace the feature extractor, not rebuild the corpus, return engine, or inference code. The practical contract is one derived row per eligible call, keyed to the rail\u2019s existing identity and content hash. An entitled self-hosted user would reproduce or adapt the guarded local panel; the public site does not ship the panel or a hosted executor.\n\nStep What to lock before looking at returns\n\n1. State the claim Name the text section, scalar feature, expected direction, economic mechanism, primary horizon, and unit of analysis.\n\n2. Define validity Choose an independent validation or calibration check appropriate to the feature, such as a labeled set or directional positive control.\n\n3. Freeze extraction Pin code, dictionary or model snapshot, environment, chunking, missing-value verdicts, and quality exclusions.\n\n4. Preserve lineage Emit transcript ID, fiscal identity, call date, content hash, split verdict, feature version, numeric value, and quality verdict\u2014never substitute a ticker-only join.\n\n5. Join outcomes Verify hashes, attach the existing +20/+40/+60 abnormal returns, and keep the primary horizon distinct from robustness checks.\n\n6. Apply promotion gates Require valid measurement, stable interpretation, issuer/time-aware uncertainty, and a locked future test before calling anything investable.\n\nFor example, a hypothesis that \u201ca sudden rise in supply-chain specificity predicts positive drift\u201d must define specificity before seeing returns. It might use Q&A only, count a fixed vocabulary or score a frozen labeled model, and require agreement on a human-reviewed calibration set. The output joins the existing panel by transcript identity and hash; StockQL supplies prices and benchmark sessions; the same quarterly and ticker-aware evaluation decides whether the idea advances.\n\nIf a feature has a calibration requirement and fails it, stop there. If measurement is valid but its return interval crosses zero, publish the bounded null. If an exploratory cut looks promising, write a new pre-registered specification and test it on a non-overlapping future period. Do not rename the exploratory cut \u201cprimary.\u201d\n\nThe StockQL backtesting guide covers the underlying SQL discipline for universe formation, signal timing, transaction costs, and point-in-time results. The transcript rail adds text lineage, section quality, local feature computation, and issuer-aware inference.\n\n6. Questions quant-curious readers ask\n\nHow do you backtest earnings-call sentiment?\nFreeze a point-in-time universe, identify each call with more than a ticker, derive sentiment from call-time text, delay the outcome past the immediate reaction, compare stocks within the same period, and report uncertainty that accounts for repeated issuers. Lock the dictionary or model before joining returns.\nDoes StockQL perform sentiment analysis?\nNo. StockQL provides the self-hosted transcript, identity, price, and benchmark plumbing. A dictionary, local model, hosted model, notebook, or agent can produce derived features under the researcher\u2019s own data and model boundaries.\nWhy keep a pipeline after four tests did not advance?\nBecause the expensive reusable work is the audited split, stable identities, guarded panel, return construction, and inference. Three bounded nulls and one rejected proxy stopped with evidence; the next hypothesis can reuse the method instead of rebuilding weeks of plumbing.\nIs StockQL open source?\nNo. StockQL is source-available and self-hosted. You can inspect and run the supplied source on your own infrastructure under its license; \u201copen source\u201d would make a different licensing claim.\nWhat would justify reopening these signals?\nImplicit prepared/answer similarity needs a separately motivated, pre-registered sign and interaction tested on non-overlapping returns. Evasion needs an independent human-labeled corpus that validates the scorer before either alpha or policy use. Those are entry conditions, not scheduled follow-up.\n\nWhat StockQL contributes\n\nA transcript hypothesis is easy to describe and surprisingly easy to test badly. StockQL keeps the durable market-data layer local and queryable: long-form calls, fiscal and security identity, exchange-normalized prices, benchmark sessions, and the structured facts needed to define a historical universe.\n\nThe research rail composes those pieces with an audited section splitter, frozen feature code, leakage guards, and explicit uncertainty. That composition is the value: a quant-curious researcher can spend a weekend testing the signal rather than a month reconstructing the database\u2014and can keep an honest null when the signal does not survive.\n\nSee how StockQL handles the data \u2192"
}