Add accredited-investor portal attribution for Form D filings#197
Merged
Conversation
sroussey
force-pushed
the
claude/sec-accredited-investor-portals-vtsw65
branch
from
July 16, 2026 02:47
8dc1b84 to
4d27a2b
Compare
…ribution Accredited-investor portals (AngelList, Forge, EquityZen, ...) never register with the SEC, so a curated tier bootstraps from an embedded copy of the embarc portals-accredited list: accredited_portal (slug-keyed), plus accredited_portal_signal fingerprints (normalized entity names, phone international numbers, address hash ids) and derived form_d_portal_attribution rows. processFormD harvests issuer / related-person / sales-compensation candidates and PortalAttributor matches them exactly against the signal table (address > phone > name); 'sec accredited-portal attribute' recomputes attributions from stored observations with clear-then-recompute semantics. New 'sec accredited-portal' CLI group: import, list, signal add/list/remove, attribute, filings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS
Correctness: unscoped attribution now clears the filing's prior rows before recomputing (re-ingest replays self-heal after signal changes); the backfill filters bad-data placeholder tokens out of person name parts exactly like the ingest path (shared pushAttributionCandidates builder ends the divergence); the matches audit trail keeps every corroborating filing role and the strongest-match tie-break is deterministic. Efficiency: signal lookups batch through getBulk; clearPortal/clearAccession use deleteSearch instead of row-by-row deletes; the backfill streams observations with keyset pagination instead of materializing both tables. Cleanup: dead signal-type const objects removed; seed input type reuses the embedded seed entry; CLI --type is case-insensitive; shared option block for signal add/remove; clearer seed-file JSON error with the path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS
The backfill clears its whole recompute scope up front (clearAll / clearPortal), so the attributor's per-accession clear — needed only for the live-ingest replay path — was dead work repeated once per filing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS
sroussey
force-pushed
the
claude/sec-accredited-investor-portals-vtsw65
branch
from
July 16, 2026 03:16
4d27a2b to
7f218b9
Compare
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS
- Look up signals before clearing a filing's attribution rows and serialize the clear+write per accession (KeyedMutex), so a transient failure or a concurrent replay can no longer leave a filing stripped of rows. - Centralize the signature-role exclusion in pushAttributionCandidates and reject placeholder tokens (None, N/A, ...) inside normalizeNameSignal, so ingest, backfill, and CLI harvest identically by construction (with an idempotency test pinning normalize-twice parity). - Backfill: preload the signal table once (all or per portal), stream observations via the shared streamMatchingRows helper, and keep only candidates that match a loaded signal — memory is bounded by match count and unmatched accessions skip the filing lookup entirely. - CLI: address signals no longer inherit --country (ingest never has one), and --value for addresses must be a normalized pipe-joined hash. - Import: reject string live values in external seeds and report distinct portal count; drop the unused deletePortal method. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS
5 tasks
The CLI's machine-readable convention is the global --json flag (SEC_JSON_OUTPUT / isJsonOutput), not a per-command --format option; list and filings now follow it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an accredited-investor portal attribution subsystem for Form D filings by introducing curated portal + fingerprint (signal) storage, deterministic attribution writing, and CLI tooling to seed/curate/backfill attributions. This fits into the SEC ingest + storage layer by producing recomputable derived attribution rows from existing observations.
Changes:
- Introduces curated accredited-portal storage (portals, signals, Form D attributions) with normalization helpers and import-from-seed support.
- Adds a deterministic attribution engine (
PortalAttributor) plus an observation-based backfill to recompute attributions without raw filings. - Integrates attribution harvesting into the Form D ingest path and adds CLI commands + documentation/tests.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/storage/accredited-portal/SignalNormalization.ts | Normalizers for name/phone/address signals aligned with ingest output. |
| src/storage/accredited-portal/importAccreditedPortals.ts | Seed import (embedded or external JSON) with idempotent upserts and seed-signal refresh. |
| src/storage/accredited-portal/importAccreditedPortals.test.ts | Tests for seed import behavior and idempotency. |
| src/storage/accredited-portal/FormDPortalAttributionSchema.ts | TypeBox schema + DI token for derived Form D portal attributions. |
| src/storage/accredited-portal/FormDPortalAttributionRepo.ts | Repository wrapper for attribution CRUD and clearing scopes. |
| src/storage/accredited-portal/AccreditedPortalSignalSchema.ts | Signal schema keyed by (type,value) with seed/manual source tracking. |
| src/storage/accredited-portal/AccreditedPortalSignalRepo.ts | Signal repo helpers including “seed doesn’t overwrite manual” semantics. |
| src/storage/accredited-portal/AccreditedPortalSignalRepo.test.ts | Tests for normalization and signal repo behavior. |
| src/storage/accredited-portal/AccreditedPortalSchema.ts | Accredited portal schema + portal_id slug derivation helper. |
| src/storage/accredited-portal/AccreditedPortalRepo.ts | Portal repo helpers including seed upsert preserving curated fields. |
| src/storage/accredited-portal/AccreditedPortalRepo.test.ts | Tests for portal repo operations and slug generation. |
| src/sec/forms/exempt-offerings/Form_D.storage.ts | Harvests attribution candidates during Form D ingest and safely invokes attribution. |
| src/sec/forms/exempt-offerings/Form_D.attribution.test.ts | Integration tests verifying ingest-time attribution writes. |
| src/resolver/PortalAttributor.ts | Deterministic matching engine with scoped/unscoped recompute semantics and locking. |
| src/resolver/PortalAttributor.test.ts | Tests for attribution behavior, tie-breaking, deduping, and recompute behavior. |
| src/resolver/backfillFormDAttribution.ts | Backfill sweep recomputing attributions from stored observations + signals. |
| src/resolver/backfillFormDAttribution.test.ts | Tests for backfill correctness, clearing semantics, and scoping. |
| src/data/accreditedPortalsSeed.ts | Embedded seed list of accredited-investor portals. |
| src/config/TestingDI.ts | Registers in-memory storages for new accredited-portal tables in tests. |
| src/config/setupAllDatabases.ts | Ensures new storages are set up in integration runs. |
| src/config/resetAllDatabases.ts | Ensures new storages are cleared on resets. |
| src/config/DefaultDI.ts | Registers persistent storages for the new accredited-portal tables. |
| src/commands/index.ts | Wires accredited-portal commands into the CLI. |
| src/commands/accreditedPortal.ts | CLI for importing portals, curating signals, backfilling attribution, listing filings. |
| README.md | Documents accredited-investor portals and associated CLI workflows. |
| CLAUDE.md | Adds architecture + operational notes for the attribution system. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+26
to
+29
| matched_signal_type: Type.String({ | ||
| maxLength: 16, | ||
| description: "Strongest matching signal kind (address > phone > name)", | ||
| }), |
| ); | ||
| return null; | ||
| } | ||
| return { signal_type: "address", signal_value: opts.value.toLowerCase() }; |
| } | ||
| for (const p of portals) { | ||
| console.log( | ||
| `${p.portal_id}\t${p.name}\t${p.live ? "live" : "closed"}\t${p.url ?? ""}\t${p.brand ?? ""}` |
Comment on lines
+253
to
+256
| if (!opts.all && !opts.portal) { | ||
| fail("pass --all to recompute everything, or --portal <id> for one portal"); | ||
| return; | ||
| } |
Comment on lines
+389
to
+392
| // A related person's address is a portal fingerprint even when the person is | ||
| // a genuine individual (fund managers list the portal's back office). Their | ||
| // human name can't collide with normalized portal name signals, so it is | ||
| // safe to offer as a candidate too. |
- Trim pasted address_hash_id values before storing - Constrain matched_signal_type to the three signal-kind literals - Make attribute's --all and --portal mutually exclusive - Render null portal live status as 'unknown', not 'closed' - Soften an overclaiming comment about person-name collisions Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS
…ration hardening - Register portal-attributor as a resolver component: rows stamp the active slot's semver, 'sec version coverage resolver portal-attributor' reports the share of attribution rows at a version, and drop-previous purges rows at the retired version - Add 'sec accredited-portal suggest': surfaces address/phone values recurring across many distinct Form D filings that are not yet curated signals - Add 'sec accredited-portal set' for curated portal fields (cik, notes) - Require explicit --country for phone signals (region-sensitive parsing) - Widen address-hash columns (64 -> 512) and phone columns (20 -> 32) in the observation and canonical-junction schemas so Postgres can store full address_hash_id values (schema-only; no deployed data) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS
sroussey
added a commit
that referenced
this pull request
Jul 17, 2026
### Features - expose library surface for superset CLIs (#203) - add accredited-investor portal attribution for Form D filings (#197) - add SPAC consolidated report with lifecycle tracking (#164) - add Form 8-K parsing and event storage infrastructure (#68) - add CFPORTAL form parsing and Reg A query support (#133) - track company facts fetch/store outcomes with retry support (#132) - add offering terms, underwriters, and use-of-proceeds extraction (#128) - extend ReadOnlyTabularStorage with new getOffsetPage method and update getBulk signature - enhance ReadOnlyTabularStorage with pagination and query capabilities - enhance bootstrap tasks and error handling #### cli - run query and db commands as task graphs via the workflow renderer - honor --json flag in status and error output (#193) - sponsor-family alias management + alias-aware 'spac by-family' query - extend sec version coverage to resolver kind - add sec canonical {person|company} alias commands - add sec resolve --kind --version --all #### models - download model weights before use, with on-screen progress (#198) #### eval - default the S-1 oracle reference to opus - golden-truth oracle + reconcile table/bio roles - score field-values by F1 so over-production is penalized - surface per-row/field disagreements after a run - add OpenAI, Gemini, and xAI providers to the model harness - oracle comparison over real S-1 sections - model comparison harness + local model wiring #### spac - AI SPAC classifier, sponsor promote, de-SPAC linkage (#149, #150, #151) - investorpres slot, portal featured, family descriptions - target_description from merger proxies onto the deal + row - SPAC business profile + leadership bio/birth_year extraction #### xbrl - ISO date transforms + CIK/dimension query coverage (#154) (#195) #### s1 - skip nonce echo for local providers - treat a board chair as implying director (drop redundant "Director") - model person titles as a list of distinct roles (titles[]) - canonicalize management titles post-model - extract SPAC sponsors into legal-sponsor + family tiers with links - add extractSpacSponsors structured extractor - write deterministic SPAC classification from header SIC - add shared SGML-header + primary-document submission parser - add real S-1 fixture sampler and document the extraction flow - add dead-letter retry sweep, extractor CLI, and promote eligibility announce - add S-1 prospectus extraction pipeline and dispatch - add observation-provenance, beneficial-ownership, related-party, and dead-letter storage tiers #### config - node-llama-cpp GGUF local provider + workglow mega imports - register SEC AI models in the global model repository #### extractors - register AI providers + retry model-error dead-letters without a version bump #### sec - SPAC de-SPAC lifecycle — 8-K milestones, merger-proxy & redemption extraction (#166) #### forms - route F-1 / F-1/A / F-1MEF (foreign issuer) through the S-1 extractor - dispatch DRS/DRS/A to S-1 extractor; fetch registration forms as .txt - split fetch/parse failure domains in ProcessAccessionDocFormTask - ingest Section 16 (3/4/5) and Form 144 ownership filings (#116) #### canonical - add SponsorFamilyMembership + SpacSponsorLink tables with DI - add CanonicalSponsorFamily + alias tables with DI - emit current_canonical_* view DDL - add alias repos with single-hop invariant - add four canonical-level address/phone junction repos - add CompanyIdentityLinkRepo - add PersonIdentityLinkRepo - add CanonicalCompanyRepo - add CanonicalPersonRepo - add alias schemas - add canonical-level address/phone junction schemas - add identity-link schemas - add CanonicalCompanySchema - add CanonicalPersonSchema #### resolver - add SponsorFamilyResolver + register sponsor-family resolver kind - add EntityObserver helper for form storage modules - add CompanyResolver v1 with CIK/CRD/name rules + alias pass - add PersonResolver v1 with CIK/name rules + alias pass - add resolverIds module #### storage - add S1ClassificationRepo (filing-level SPAC record) + DI #### html - add form-agnostic EDGAR HTML to Document-tree converter #### versioning - add dropPrevious tests for extractor and resolver kinds - extend drop-previous to resolver rows + orphan cleanup - add ceremony CLI (start-dev, promote, rollback, drop-next, coverage, history); remove seed-test - add VersionCoverage and VersionHistory queries - add ceremonies module (startDev / promote / rollback / dropNext) - add getActiveSlot helper (next-if-exists routing) - add VersionEventRepo for ceremony audit log - add VersionEvent schema and DI wiring - add semver helpers (parse, major.minor, bump-progression validation) - add target_count column to component_versions - add ExtractorRunRepo helper - bootstrap extractor versions on db setup - add idempotent bootstrapExtractorVersions seeder - add FORM_TO_EXTRACTOR_ID mapping and ExtractorId type - add 'sec version status' and 'sec version seed-test' CLI - add VersionStatus query for CLI rendering - add VersionRegistry helper with slot accessors - register ComponentVersion and ExtractorRun repos in DI - add ExtractorRun schema and DI token - add ComponentVersion schema and DI token #### observation - add CompanyObservationRepo - add PersonObservationRepo with natural-key upsert - add CompanyObservationSchema - add PersonObservationSchema ### Bug Fixes - address xhigh code-review findings - update warning messages and improve test descriptions - DRSLTR dispatch, SPAC sponsor span verification, resolver test + RFC-9112 Content-Length (#127) - address code review issues — dropPrevious orphan check, DbStatus new tables, drop-previous CLI verb, resolve test coverage, CompanyResolver secondary keys - init #### sec - CLI validation, portal-attribute reporting, gguf path check (#201) - make local GGUF models usable in eval and exit cleanly - isolate redemption model-resolution + bulk-skip backfill candidates - record extractor_runs for redemption + idempotent, fast backfill #### query - reject empty CIK filters #### cli - report expected user-errors through task output ports; dedupe tier wiring - preserve JSON workflow error handling - make streamed query totalApprox a meaningful lower bound (#112) #### editorial - isolate multi-file import failures #### s1 - one owner per name; quote key lists in eval diffs - drop ownership subtotal rows; add beneficial-ownership golden truth - close the GBNF []-shortcut on nested titles[] too - harden sponsor extraction + sponsor-family storage from code review #### eval - fail loudly when an extractor has no fixtures - normalize commas and initial/suffix periods in name alignment - restore stderr progress for eval tasks when piped #### spac - make de-SPAC linkage a write-once close-time snapshot - stop deriving phantom deals after a completed combination - gate SQLite withSpacCikLock through process-wide mutex (avoid concurrent BEGIN IMMEDIATE crash) - allow caller-supplied pool client to recomputeSpacDeals to avoid deadlock with outer lock - atomic per-CIK rollup writes + monotonic history chain - record extractor_runs for merger-proxy + transactional recompute #### storage - implement no-op updateWhere on ReadOnlyTabularStorage #### resolver - stabilize person name parts across punctuation/glyph variants - instance-scope per-key mutex so multi-process race tests test what they claim (#160) - FamilyResolver — alias inside mutex + UPPER case-normalization (#129) - add readonly to PersonClaim/CompanyClaim; single timestamp per method #### s1,eval - director nominees + typographic name alignment #### eval,s1 - render titles arrays clearly in diffs; sharpen split prompt #### sec/424 - record deterministic SPAC IPO event even when AI model fails to resolve #### facts - derive fy sentinel from end_date to preserve PK width across period-agnostic facts - accept EDGAR facts with null fy/fp #### sec/extractors - move verify nonce out of untrusted fence into trusted preamble - add NONCE_MISMATCH dead-letter reason code - gate reapStaleObservations on version change to prevent LLM-variance data loss on same-version re-runs #### sec/html - strip title/svg/math to close prompt-injection bypass in body-level and foreign-content elements #### config - declare Anthropic model capabilities so StructuredGenerationTask resolves - init SQLite binding in setupAllDatabases before view DDL #### extractors - degrade gracefully when the AI model is unregistered #### submissions - store every filing row (proxy slice yielded undefined) #### review - clarify details doc + single-pass investorpres derivation - address code-review findings + prettier - close wave-2 (#177/#178/#179) review findings - close 6 findings from max-effort review of the consolidation #### canonical - serialize junction observation_count with KeyedMutex - align junction columns with address/phone schemas #### util - parseDate rejects calendar-invalid dates #### xbrl - XbrlFactRepo.replaceForAccession no-ops on 0 rows unless intentionalClear #### forms - close stripDoctype bypass via leading XML comment / PI - restore predefined-entity decoding via bounded processEntities + DOCTYPE strip - apply prompt-injection seal to merger-proxy + redemption extractors - treat empty fetch body as no-text and swallow dead-letter write failures - explicit null address_id/international_number in Form_1_Z signature observation - remove dead resolveCountryCode from Form_1_K.storage #### forms/s1 - close residual Unicode-invisible defang bypass - widen defang TAG_SHAPED to admit whitespace mid-tag (closes bypass) - wire new extractMergerDeal / extractRedemption to nonce-fence API - per-call nonce fence + raw-span cap + multi-stage defang #### forms/8-K - auto-resolve redemption-partial-oversized dead-letter (informational only) - cap redemption AI input bytes; drop oversized exhibits - persist redemption extraction even when SPAC has no deal yet - trust the actual repo nature, not the lingering SEC_DB_TYPE token - tx writes, versioned PK, accession unification, XML entity hardening #### versioning - partial-success outcome on extractor_runs - bump test timeout on multi-spawn CLI tests to 15s - address PR #109 review feedback - address final review items - reject all start-dev variants when next slot exists; drop dead check; comment promote atomicity - patch-gate listFilingsWithoutSuccessfulRun on major.minor prefix - address independent code review - address PR #107 review feedback - use cik: number in ExtractorRunRepo query API (matches codebase convention) - address PR #106 review feedback - enforce semver and coverage_complete invariants in putSlot - drop redundant PK-prefix indexes; wire resetAllDatabases; sort imports #### resolver,forms/s1 - PG unique-violation recognition + family-tier UNIQUE convergence + S-1/424 prompt-injection hardening (3× HIGH) (#163) #### forms,storage - Form 144/Ownership whitespace→null, PG dedup, Form_C/1-A stale-replay guards (#159) - 5 HIGH review findings — stale replay, point-in-time, undated guard, deal sort, Schedule A (#155) - Crowdfunding history on stale replay + CFPORTAL/A inheritance + docs (#135) - address Copilot review on PR #124 #### storage/canonical - enforce UNIQUE constraints to close resolver race (#158) #### forms,portal - undated 1-K/1-Z guard + deterministic stale-replay tie-break (2 HIGH from code review) (#156) #### forms,storage,cli - person-collision issuer guard + alias chain block + CLI input validation + coverage perf (#122) #### resolver,storage,cli - resolver race + observation_id PK + CSV NBSP + download leak (sec) (#121) #### section16 - preserve null vs 0 for empty numeric leaves on Forms 3/4/5 (#116 follow-up) (#117) #### observation - align raw_phone_id maxLength with PhoneSchema #### ci - set 30s global test timeout for multi-spawn CLI tests ### Refactors - remove unused fn #### task - move src/fetch to src/task/fetch - relocate EnsureModelDownloadedTask to src/task/model and name the file after the task #### config - make ensureModelDownloaded a task that infers provider from the model id - import node-llama-cpp via the workglow mega package - drop the spac narrative-column migration #### eval - inline oracle sweep into EvalS1Task; own AI subtasks - rename `sec eval s1 --candidates` to `--models` #### cli - graph-ify every command through the workflow renderer #### normalize - shared typographic-punctuation fold for names #### portal - drop the featured column #### canonical - extract shared CanonicalJunctionRepo base #### resolver - extract shared normalizeSponsorFamilyName for consistent family keys #### queue - integrate wrapQueueStorage for SecJobQueue components #### storage - delete legacy PersonRepo, CompanyRepo, and their schemas #### forms - rewrite Form_1_Z.storage onto EntityObserver - rewrite Form_1_K.storage onto EntityObserver - rewrite Form_1_A.storage onto EntityObserver - rewrite Form_C.storage onto EntityObserver - rewrite Form_D.storage onto EntityObserver - rewrite Form_D.storage onto EntityObserver #### versioning - route form-processing tasks via getActiveSlot (next-if-exists) - UpdateAllFormsTask reads extractor_runs, drops --force - write extractor_runs from ProcessAccessionDocFormTask - derive TypeBox literal unions from const arrays ### Tests - use vitest so we can try node when needed - add unit tests for SecFetchJob functionality #### 424 - isolate the priced-prospectus fixture from S-1 discovery globs - pin priced-prospectus pipeline with a real 424B4 golden fixture #### versioning - update registered-component assertions for sponsor-family resolver kind #### s1 - add synthetic DRS .txt fixture exercising header parse + DRS dispatch - end-to-end SPAC classification + sponsor family linkage + DRS dispatch #### forms - assert FETCH_ERROR status pending; clarify Domain 3 throw comment - cover fetch-layer dead-letter paths (PRIMARY_DOC_UNRESOLVED, FETCH_ERROR, parse rethrow) #### fixtures - add fetch-fixtures script for pulling real EDGAR data (#108) ### Documentation - update CLAUDE.md and new-module JSDoc for PR4 - update paths for design specs and plans in various skills and documentation #### eval - drop LFM2.5 references; default the sweeps to cloud models - document evaluating Bonsai 27B via the local GGUF path (#187) #### 8-K - note metadata items/report_date are authoritative #### s1 - document sponsor-text extraction strategy in processFormS1 ### Chores - rename bunsrc scripts for clarity - bump workglow/cli 0.3.26, compromise 14.16.0, fast-xml-parser 5.10.0 - keep typebox 1.3.6 from main after rebase - update deps - keep extractor versions at 1.0.0 (no data to re-extract) - update deps - update deps - ignore .worktrees directory - update deps - update deps - update dependencies and ESLint configuration - update dependencies and enhance CIK query functionality #### config - drop local HuggingFace/ONNX provider wiring - register observation/canonical repos in DI; add view DDL #### extractors - default AI model to claude-sonnet-5 via shared SecModelDefault #### deps - update @workglow/cli and related packages to version 0.3.13; add new domNodes.ts file for cheerio DOM node types #### versioning - rename bootstrapExtractorVersions; seed resolver components - register resolver:person and resolver:company - retire processed_filings; DbStatus reports extractor_runs ### CI - limit rebuilds ### Updated Dependencies - `@modelcontextprotocol/sdk`: ^1.29.0 - `@workglow/cli`: 0.3.26 - `commander`: ^15.0.0 - `compromise`: ^14.16.0 - `fast-xml-parser`: ^5.10.0 - `pdf2json`: ^4.0.3 - `pg`: ^8.22.0 - `typebox`: 1.3.6 - `workglow`: 0.3.26 - `@types/bun`: 1.3.14 - `bunset`: 1.0.13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements a curated accredited-investor portal system that matches Form D filings to known platforms (AngelList, Forge, EquityZen, etc.) by fingerprinting their reused entity names, phone numbers, and addresses. Portals are bootstrapped from an embedded seed (derived from the embarc repo) and maintained via CLI commands; attributions are computed deterministically from stored observations and are fully recomputable when portal signals change.
Key Changes
Portal curation infrastructure: New
accredited_portal,accredited_portal_signal, andform_d_portal_attributiontables with repos and schemas. Portals are keyed by a name-derived slug (portal_id) rather than a CIK since they never register with the SEC.Signal normalization: Shared normalizers (
SignalNormalization.ts) that delegate to existing ingest-path helpers (company name, phone, address) so stored signals match filing values by exact string equality. Signals are keyed by(type, value)and re-point on re-add.Attribution engine (
PortalAttributor.ts): Matches filing candidates against the signal table with deterministic signal-strength ordering (address > phone > name) to break ties. Writes one row per matched portal per filing, recording all corroborating roles and the strongest matched signal type.Form D ingest integration: Modified
Form_D.storage.tsto harvest name/phone/address fingerprints from issuer and related-person roles viacollectAttributionSignals, feeding them to the attributor after processing. Signatures are deliberately skipped (signers are individuals and add noise).Observation backfill (
backfillFormDAttribution.ts): Recomputes attributions from stored observations (extractor_id "D") so re-attribution after signal changes never needs raw filings. Clears the affected scope (one portal or the whole table) first so removed signals drop stale attributions. Mirrors ingest normalization exactly (drops bad-data placeholders, skips signatures).CLI commands (
accreditedPortal.ts):sec accredited-portal import [file]— bootstrap/refresh from embedded seed or external JSONsec accredited-portal signal add/remove— curate portal fingerprintssec accredited-portal signal list— inspect stored signalssec accredited-portal backfill— recompute attributions after signal changesEmbedded seed data (
accreditedPortalsSeed.ts): 30+ curated portals (AngelList, Forge, EquityZen, Hiive, OurCrowd, Republic, etc.) with metadata (URL, live status, featured flag).Tests: Comprehensive coverage of signal normalization, portal/signal repos, attribution matching, Form D ingest integration, and observation backfill.
Documentation: Updated CLAUDE.md with accredited-portal architecture and CLI usage.
Implementation Details
pushAttributionCandidateshelper is shared between ingest and backfill so both paths harvest identically — a recompute cannot attribute differently than first ingest.angellist,forge-global) derived from display names, enabling CLI references by either slug or name.https://claude.ai/code/session_01BdQcWqsBKVGXh12C17rrXS