Repository navigation
spec(FIX-898): Recover critical financial evidence from IPO filings - #785
cursor[bot] wants to merge 1 commit into
Conversation
Implementation spec for bounded prospectus/registration-form recovery when EDGAR companyfacts and Yahoo miss valuation-critical fields (SPCX-shaped failure). Docs-only; no changeset (BP-022). Co-authored-by: Jake Hoffner <jhoffner@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
There was a problem hiding this comment.
/simplify review — spec-only PR
Verdict: COMMENT (approve-quality architecture; tighten the spec before implementation). The core problem diagnosis and the rejected alternatives (investigate promote, widening PERIODIC_FORMS, soft LLM-only confidence) are right. What risks over-building is less the ladder itself than how the spec tells implementers to wire it: three parallel statement handlers + a combined critical-miss trigger + record/replay + a rich recoveryAudit + optional disclosure surfacing + a new goal family — without a FIX-895-style v1 reconciliation that subtracts modules and names one coordinator.
What is the right approach (keep)
- Statement-tool live chain (not Phase 1 investigate): matches
writeSubjectSpine/financialsDataand BP-011. - Sparse companyfacts → miss before Yahoo/recovery: today
mapEdgarCompanyFactsreturnssource: "edgar"with null criticals and short-circuits the chain — without this, IPO recovery is a no-op. REGISTRATION_FORMSseparate fromPERIODIC_FORMS: aligns withedgar-filings.ts/latestPeriodicand disclosure prompts.- Deterministic SEC HTML first, one bounded LLM, hard validation, no partial promote: same discipline as PDF holdings (
extract-holdings-generator+reconcile), without calling a generator from handlers. - Run-local single-flight for recovery (not 120s
getOrFetchfor LLM outcomes): correct.
Where the spec goes too far (simplify before impl)
-
“Wire into three statement tools” understates the coordinator. Combined critical miss is defined across income + cashflow after companyfacts + Yahoo. With fundamentals fan-out in
.parallel, per-tool recovery can fire early (sibling Yahoo still in flight), duplicate SEC/LLM work even with single-flight, or writerecoveryAuditwhile the spine is partial. The spec already says the recovery runtime is the solerecoveryAuditwriter — make that explicit as the only entry point: sharedloadStatementLive(field)that (a) runs EDGAR→Yahoo, (b) awaits the same(ticker,date)inflight recovery Promise, (c) never implements “combined miss” independently in three files. BP-024 factory can stay deferred, but one shared loader module should be required in v1, not “deepening.” -
Module fan-out for v1.
prospectus-financials.ts+validate-financial-candidate.ts+critical-financials-recovery.ts+recover-financials-extract.ts+ prompt is a lot for a lab feature. Consider collapsing validation + USD-billions promote into the recovery runtime until a second consumer exists; keep deterministic HTML extract separate only if unit-tested in isolation. -
recoveryAuditsurface vs consumers. FIX-781 consumption is explicitly non-blocking. Until a reader lands, a smaller v1 (attempted,outcome,urls,rejectionReasons) may suffice;outcome: "skipped"vs absent audit is redundant. TrimrecoveredSource: "issuer-ir"if v1 is SEC-only (§11 #2). -
Disclosure deliverable is optional for AC. Recovery can call
fetchRegistrationCandidatesdirectly. Extendingget_sec_filings/EdgarFilingsPayloadis polish — mark non-blocking in §1 deliverables so spine work does not drag disclosure + prompt edits. -
Pre-existing perf: 3×
fetchCompanyFacts. Parallel statement tools (and quant composites) already hit companyfacts independently. FIX-898 should require single-flight orgetOrFetchonfetchCompanyFactsas part of the same change (or a named prerequisite step) — orthogonal win, large constant-factor cut on every US name. -
Record/replay ordering. Per-tool
recordIfRecordingafter eachload()can persist pre-recoveryunavailablewhile spine later showsedgar-prospectus. Spec should state: record after recovery settles for statement tools, or record the post-recovery payload only. -
Goal check scope.
goals/trading-desk-financials/…should prefer spine/unit tests or a thin probe (pattern:goals/trading-desk-portfolio/*/run.mts), not fullanalyze+runSummaryunless explicitly optional live EDGAR.fsd:verify-trading-deskis the wrong default AC ladder for statement fields. -
Spec shape vs FIX-895. A short §0 v1 reconciliation (authoritative: shared loader + coordinator, defer disclosure goal polish, minimal audit, no cheerio/jsdom — reuse
edgar-filingsedgarFetch/ strip patterns) would prevent implementers from treating every §4.1 row as mandatory day one.
Third-party / reuse (no new deps needed)
- No cheerio/jsdom in the desk today;
edgar-filings.tsextractSectionstrip pattern is the precedent for prospectus text. - Export/refactor shared submissions iteration from
edgar-filings.tsinstead of a parallelfetchSubmissionsinedgar-registration.ts. - Reuse billions normalization from
edgar-companyfacts.ts/yahoo-timeseries.tswhen promotingFinancialCandidate.
Intentional complexity (not asking to revert)
- LLM on
fastfor critical miss: product choice vsinvestigategating; spec should keep the rationale (already in decision #4) but note the cost/latency tradeoff on fundamentals critical path.
Performance
- Bounded ≤3 HTML fetches is fine; risk is multi-MB bodies + sync parse on the event loop and duplicate submissions with disclosure analyst unless submissions are cached (
getOrFetchfor read-only SEC JSON is appropriate).
Skipped (larger than spec PR / product call)
- Deterministic-only on
fast, LLM onfullonly. - Orchestration-level
.tapafter fundamentals tools instead of in-tool recovery (cleaner hot path, more invasive).
Happy to approve after a light spec pass: mandatory shared loader/coordinator, §0 v1 scope, record ordering, companyfacts dedupe, and demote disclosure/goal/docs templates to non-blocking where AC is spine-only.
Sent by Cursor Automation: Review For Best Approach
| - **Recovery:** Add single-flight critical recovery + pure validation leaf; | ||
| treat sparse companyfacts (`source: "edgar"` with null critical fields) as | ||
| a miss (today only throws fall through); wire into the three statement | ||
| tools; add `edgar-prospectus` source tag + `recoveryAudit` on |
There was a problem hiding this comment.
Simplify wiring: "wire into the three statement tools" fights the combined critical-miss definition (§4.2). Recommend making a shared loadStatementLive + single recovery coordinator a v1 requirement (sole recoveryAudit writer), with the three handlers only passing field/tool — same as you already do for EDGAR→Yahoo duplication today. Defer BP-024 factory, not the shared loader.
| | Deterministic extract | Create `labs/trading-desk/lib/providers/prospectus-financials.ts` — HTML → typed candidate (pure where possible) | | ||
| | Validation | Create `labs/trading-desk/flows/analysis/lib/validate-financial-candidate.ts` (pure) | | ||
| | Single-flight recovery | Create `labs/trading-desk/flows/analysis/tools/runtime/critical-financials-recovery.ts` | | ||
| | LLM extract (bounded) | Create a **non-block** leaf under `tools/runtime/` (e.g. `recover-financials-extract.ts` + prompt) — same *discipline* as `flows/portfolio/extract-holdings-generator.ts` (transcribe + validate), but **not** a `generator` block invoked from a handler (BP-011). Prefer direct model+schema call from the recovery runtime, or a `router` if block kinds must differ (`get_social_sentiment.ts` precedent). | |
There was a problem hiding this comment.
v1 module budget: Four new runtime/flow files + prompt is heavy. Consider spec-authoritative merge of validation + promote mapping into critical-financials-recovery.ts for v1; keep prospectus-financials.ts only if deterministic extract has its own test matrix. Avoid a third normalization dialect — point implementers at edgar-companyfacts.ts / yahoo-timeseries.ts billions helpers.
| after companyfacts + Yahoo attempts, the **combined** critical set across | ||
| income + cashflow still lacks valuation-critical fields. Use **real schema | ||
| field names**: | ||
|
|
There was a problem hiding this comment.
Race / wasted work: "combined critical miss" while income/cashflow/balance load in parallel needs an explicit rule: recovery must not evaluate until all three Yahoo legs have settled (or coordinator reads spine after shared inflight Promise). Otherwise single-flight dedupes duplicate LLM but not premature SEC work.
| * Never leave prospectus dollars on the spine. | ||
| */ | ||
|
|
||
| export type RecoveryAudit = { |
There was a problem hiding this comment.
Audit minimalism: Until FIX-781 consumes this, consider shrinking v1 recoveryAudit (drop redundant skipped vs absent; defer issuer-ir in recoveredSource). Session citationIntegrity is the right analogy, wrong shape — keep audit on financialsData only.
| with scale. | ||
|
|
||
| 5. **Single-flight recovery runtime + source tag + recoveryAudit** | ||
| - Wire into three statement tools; gate on subject/`toSpine`. |
There was a problem hiding this comment.
Step 5 should name shared loader extraction as part of this step (not §8 deepening). Parallel spine test is good — add assertion that record mode captures post-recovery payloads, not pre-recovery unavailable per tool.
| attempts). | ||
|
|
||
| **Goal check:** Add | ||
| `goals/trading-desk-financials/ipo-prospectus-recovery/` (name flexible) that |
There was a problem hiding this comment.
Goal path: prefer vitest/spine probe or tsx goal over full analyze + runSummary for CI-adjacent checks (see goals/trading-desk-portfolio/*/run.mts). Optional live EDGAR ticker is fine; don't make fsd:verify-trading-desk the default AC.
|
|
||
| **Deepening opportunities (follow-up via `fsd:improve-codebase-architecture`):** | ||
|
|
||
| - Factory for the three near-identical statement handlers (BP-024). |
There was a problem hiding this comment.
Promote companyfacts single-flight / getOrFetch in fetchCompanyFacts from follow-up to FIX-898 step 0 or 2 — fixes 3× HTTP today and amplifies every IPO-miss path. Statement-tool factory can stay here.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9d3b82f571
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| |-------|--------| | ||
| | CIK matches `resolveCik(ticker)` | reject wrong-company | | ||
| | Conformed name fuzzy-match vs profile/submissions name (non-empty) | reject | | ||
| | `periodEnd` present and ≤ run `date` + small grace; not decades stale vs listing | reject stale | |
There was a problem hiding this comment.
Reject filings published after the run date
For historical analyses, candidate discovery reads the current SEC submissions catalog, but this validation only bounds periodEnd; a 424B4 filed after the requested date can contain an older audited period and therefore pass, leaking future information into the financials spine. Require filingDate <= run date during discovery or validation and add a historical-run regression.
Useful? React with 👍 / 👎.
| **Single-flight:** Parallel tools share one **run-local** in-flight Promise per | ||
| `(ticker, date)` (do **not** use process-TTL `getOrFetch` for LLM recovery — | ||
| failed attempts must not stick across sessions for 120s). Recovery may fill |
There was a problem hiding this comment.
Use request-scoped runOnce for the recovery
A Promise map keyed only by (ticker, date) is not run-local: concurrent sessions for the same ticker/date can share an attempt, while deleting the entry on settlement lets a slower sibling tool launch a second paid extraction. BlockContext.runOnce already memoizes per (requestId, userKey), including concurrent calls and retries (packages/core/src/types/block.ts), so the spec should require that request-scoped primitive instead of a module map.
Useful? React with 👍 / 👎.
| handlers — do **not** `block.run` a `generator` from inside them. Use a | ||
| plain async model+schema helper owned by the recovery runtime, or a `router` | ||
| if a true generator block is required (`get_social_sentiment.ts`). The |
There was a problem hiding this comment.
Route extraction through the generator runtime
The preferred plain model helper bypasses the generator runtime behavior implemented in packages/core/src/blocks/generator.ts, including model resolution, ctx.signal cancellation, usage/cost reporting, and generator test instrumentation. On a recovery miss this can leave a paid call running after cancellation and undercount the run summary, so specify a router/sequencer-owned generator path like get_social_sentiment.ts rather than a direct SDK call from the handler.
Useful? React with 👍 / 👎.
| - **Modes:** | ||
| - `fixture` — load fixtures only; never call live recovery (replay recorded | ||
| `edgar-prospectus` / `unavailable` + audit). | ||
| - `live` / `record` — run the live chain **including** recovery, then | ||
| `recordIfRecording` on `record` (same as today's statement tools). |
There was a problem hiding this comment.
Persist and hydrate the recovery audit in fixture mode
The current recordIfRecording/loadFixture path records and returns only each statement tool's output, while recoveryAudit is a separate financialsData field; because fixture mode also skips the recovery runtime (its sole writer), recorded audits cannot be replayed and the fixture goal requiring outcome === "promoted" will fail. Define a dedicated recorded audit artifact and fixture hydration path, or include the audit in a schema that the recorder actually persists.
Useful? React with 👍 / 👎.
| export const REGISTRATION_FORMS = new Set([ | ||
| "S-1", "S-1/A", "424B4", "424B3", "F-1", "F-1/A", | ||
| ]); |
There was a problem hiding this comment.
Wildcard Promise Excludes Valid Forms
The design promises 424B* recovery, but this allowlist accepts only 424B3 and 424B4. When EDGAR reports an available prospectus as another 424B variant, candidate discovery drops it and records no-candidates, leaving the financials unavailable despite an eligible SEC filing.
| }; | ||
| ``` | ||
|
|
||
| Extend `sourceTag` with `"edgar-prospectus"`. Extend |
There was a problem hiding this comment.
Issuer-IR Source Cannot Be Stored
Validation permits promotion from a SEC-linked issuer-IR URL and the audit can report issuer-ir, but the statement source enum gains only edgar-prospectus. A valid issuer-IR recovery must therefore either fail statement-schema validation or be stored with false EDGAR provenance.
Artifacts
- Contains supporting evidence from the run (text/typescript; charset=utf-8).
Repro: Vitest configuration used to execute the focused schema harness
- Contains supporting evidence from the run (text/typescript; charset=utf-8).
Repro: verbose runtime output showing issuer-ir and edgar-prospectus sourceTag validation failures
- Keeps the command output available without making the summary code-heavy.
| | `scale` parsed from table header / explicit units — never inferred from magnitude | reject if ambiguous | | ||
| | At least revenue + operatingIncome + (FCF or operating+capex) present after scale | reject insufficient | | ||
| | If operating + capex + FCF all present: `|FCF − (operating − \|capex\|)|` within tolerance | reject unreconciled | |
There was a problem hiding this comment.
Reconciliation Boundary Is Undefined
The required FCF gate says only “within tolerance” without defining an absolute or relative threshold. Rounded prospectus tables can therefore be rejected by one implementation and accepted by another, making promotion of the same filing nondeterministic and potentially leaving valuation-critical fields unavailable.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


Summary
Implementation spec for FIX-898.
Adds a bounded critical-source recovery path for the trading desk when EDGAR Company Facts and Yahoo miss valuation-critical financials (the SPCX / newly listed IPO failure mode). Deterministic SEC registration/prospectus candidates (S-1, 424B*) first, then one capped LLM extract, hard validation before promoting into
financialsData, honestunavailable+recoveryAuditotherwise.Docs-only PR — no changeset (BP-022). This PR is for spec review only;
fsd:implement-issuecloses it unmerged when implementation starts.Necessity verdict
Build as scoped — correctness gap in the lab app's statement spine; existing
investigatecannot promote typed numbers intofinancialsData.Key decisions (top 5)
investigate. Investigate stays citation-only.REGISTRATION_FORMSseparate fromPERIODIC_FORMS. MD&A/8-K extractors unchanged.costPreset. Correctness path may spend one model call onfastafter deterministic miss.recoveryAudit+edgar-prospectussource tag; FIX-808 case prepared but not blocked.Validator fixes folded in
operating,cashAndEquivalents)toSpine: falsegatedLinear sync gap
Cloud automation has no Linear API (
LINEAR_API_KEYnot injected; no Linear MCP). Could not: move issue to In Spec Dev / In Spec Review, attach Linear document, comment, or reframe the issue description. Repo spec + this PR are the reviewable artifacts; Linear mirror needs a follow-up with credentials.Spec document
docs/specs/FIX-898.mdOpen questions (defaults chosen)
registrationFilings(default) vs mixing intorecentFilingsGreptile Summary
This PR specifies recovery of valuation-critical financials from IPO filings. The main changes are:
Confidence Score: 4/5
The implementation contract needs fixes for filing coverage and issuer-IR provenance before it is used to build the recovery path.
Some valid 424B prospectuses can be filtered out before recovery. Accepted issuer-IR evidence has no matching statement source value. The hard FCF gate lacks a deterministic tolerance.
docs/specs/FIX-898.md
What T-Rex did
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[Load statement] --> B{Company Facts usable?} B -->|Yes| S[Write subject spine] B -->|No or sparse| C{Yahoo usable?} C -->|Yes| S C -->|No| D[Single-flight recovery] D --> E[Discover registration filings] E --> F[Fetch up to three documents] F --> G[Deterministic extraction] G --> H{Candidate sufficient?} H -->|No| I[One bounded model extraction] H -->|Yes| J[Hard validation] I --> J J -->|Pass| K[Normalize and promote] J -->|Fail| L[Keep unavailable] K --> M[Write promoted audit] L --> N[Write failure audit]%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% flowchart TD A[Load statement] --> B{Company Facts usable?} B -->|Yes| S[Write subject spine] B -->|No or sparse| C{Yahoo usable?} C -->|Yes| S C -->|No| D[Single-flight recovery] D --> E[Discover registration filings] E --> F[Fetch up to three documents] F --> G[Deterministic extraction] G --> H{Candidate sufficient?} H -->|No| I[One bounded model extraction] H -->|Yes| J[Hard validation] I --> J J -->|Pass| K[Normalize and promote] J -->|Fail| L[Keep unavailable] K --> M[Write promoted audit] L --> N[Write failure audit]Reviews (1): Last reviewed commit: "spec(FIX-898): critical financial recove..." | Re-trigger Greptile