feat(sdk): v2 SDK vertical slice — generated DTOs, async transport, resources, agentic CLI#231
Open
JonnyTran wants to merge 31 commits into
Open
feat(sdk): v2 SDK vertical slice — generated DTOs, async transport, resources, agentic CLI#231JonnyTran wants to merge 31 commits into
JonnyTran wants to merge 31 commits into
Conversation
…borev job 157)
The isolated references were built as `${seed.reference}-<spec>`, i.e. superstrings
of the shared seed reference. Since other specs assert on the shared record via
non-exact getByText(seed.reference), the isolated rows substring-matched those
assertions into Playwright strict-mode violations. Use a distinct base
(10.2000/e2e-*) and document the no-superstring constraint on createIsolatedRecord.
- Pin datamodel-code-generator>=0.26,<0.69 so a minor bump doesn't silently change emitted formatting and flip the byte-identical drift gate; add a comment in the test noting a generator bump requires re-running codegen - Replace check=True+swallowed stderr with explicit returncode assertion that includes proc.stderr, so CI logs show generator diagnostics when the codegen step fails
Slice from the first '{' before json.loads in test_snapshot_matches_server
so any banner/warning/deprecation line emitted to stdout doesn't cause a
spurious JSONDecodeError unrelated to snapshot drift; also replace check=True
with an explicit returncode assertion that surfaces stderr.
…straint Comment previously said >=0.68.1,<0.69 but manifest is >=0.26,<0.69; update comment to match and note the installed/tested version (0.68.1). Lower bound stays at 0.26 because >=0.68 requires Python>=3.10 and the SDK supports >=3.9.2, which would break uv resolution on Python 3.9.
…uard
str.index raises ValueError when no '{' is present; str.find + explicit
assertion gives a clear diagnostic message instead of a confusing
exception when openapi-dump produces no JSON on stdout.
test_schemas_resource.py and test_questions_resource.py were committed to the repo-root tests/unit/v2/ (Tasks 5,6) instead of extralit/tests/unit/v2/, so the SDK's testpaths=["tests"] never ran the committed copies. Move them alongside the other v2 tests. Full v2 suite: 45 passed, 1 skipped.
…ndle_errors - test_cli_schemas.py: use click-version-conditional CliRunner so result.stderr is captured on click 8.1 (Python 3.9) where mix_stderr defaults to True - _output.py: handle_errors now also catches ValueError (malformed UUID / JSON CLI input) and routes it through fail() for structured stderr + exit code 1
…sport Add a lazily-created asyncio.Lock (_auth_lock) so concurrent requests on a fresh username/password client fire exactly one POST /token instead of N; likewise concurrent 401s coalesce into a single POST /token/refresh via _refresh_if_stale. _ensure_logged_in double-checks the token after acquiring the lock to prevent stampede.
…tests
- _output.py: handle_errors now catches (ValueError, OSError) so missing/
unreadable --file paths are routed through fail() (structured stderr, exit 1)
instead of surfacing as a raw traceback — covers FileNotFoundError on the
JSONL piping path
- test_output.py: add tests asserting ValueError (bad UUID) and OSError
(missing file) both produce exit code 1 with {"error": {...}} on stderr
…ed get_client
The previous test could pass via credentials-resolution ValueError rather
than the UUID-parsing path it targeted. Now monkeypatches get_client so
the fake client succeeds and the ValueError provably originates from
UUID('not-a-uuid') inside upsert_records, routed through handle_errors.
_refresh_if_stale now takes the stale token and skips the refresh if another coroutine already rotated it while waiting for the auth lock, so N concurrent 401s trigger exactly one /token/refresh (was: one refresh per waiter). The test models the server by token (stale AT1 -> 401, rotated AT2 -> 200) via a single reusable callback, so asyncio.gather interleaving can't consume the wrong canned response.
… click-safe test assert - _read_jsonl opens files with encoding='utf-8' (portable) - delete_records splits --ids once instead of twice - test_cli_records failure message uses result.output (result.stderr raises on click 8.1)
…e verb set add_v2_commands now registers all six top-level v2 groups. Annotation-verb tests invoke through the mounted root app (extralit suggestions upsert / projection get) so the group+subcommand names are both required — typer 0.20 promotes a single-command Typer when invoked standalone, which isn't the mounted UX.
- test_boundaries.py: v2->v1 wall (only extralit.client.login allowed), v1->v2 wall (only cli/app.py), and a delta gate proving importing extralit.v2(+cli) adds no heavy modules (pandas/pandera/datasets/huggingface_hub) beyond 'import extralit' baseline. - Startup: 'extralit schemas --help' ~1.95s, dominated by v1 eager imports (extralit/__init__.py drags in datasets+huggingface_hub). The v2 side is clean: test_v2_adds_no_heavy_imports passes (zero heavy delta). v1 startup weight is Phase 6. - Appended a v2 SDK section to extralit/CLAUDE.md. v2 suite: 66 passed/1 skipped; boundary gates 3/3. Pre-existing v1 test_resources failures (Missing api_key, unrelated to v2 — those files untouched by this branch).
_refresh_if_stale only short-circuited when a prior refresh succeeded and rotated the token. When the refresh FAILS (token unchanged), each concurrent waiter re-entered and re-called _refresh(), so N concurrent 401s against a dead session fired up to N /token/refresh POSTs before all raised AuthError. Record the stale token whose refresh failed and short-circuit later waiters, so a dead session triggers exactly one refresh. Adds test_concurrent_401s_failed_refresh_coalesces (RED→GREEN verified).
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Python SDK v2 vertical slice
Builds the parallel
extralit.v2SDK package (schema-centric, async-native) plus top-level agentic CLI verbs that replace the v1schemassubcommand. Executed task-by-task via subagent-driven development; every task passed a spec+quality review, and the branch passed a final whole-branch review (verdict: ready-to-merge-with-fixes; the one Important finding is fixed in this branch).Stacked on #230 (
feat/v2-frontend-slice). Base should be retargeted todeveloponce #230 merges. All work is confined to theextralit/SDK subdir.What's included
_api/openapi.json) → generated pydantic v2 DTOs (_api/_generated.py) via datamodel-codegen. 3.9-safe (--no-use-union-operator→Optional/Union, not PEP 604). Two CI drift gates (regenerate byte-identical; snapshot vs liveopenapi-dump).httpx.AsyncClient; api-key header or username/password→bearer with a single transparent refresh on 401; concurrent login/refresh coalescing under a lazily-createdasyncio.Lock(loop-safe for the sync portal), including the failed-refresh coalescing fix from final review.{name:{"value":…}}helpers;SearchPage.Schemas(immutable version cache),Questions(owns the name↔id join; suggestions key by id, cells/values by name),Records(chunked concurrent bulk-upsert ≤500, order-preserving; search/list; delete ≤100; raw-slash reference lookups),Suggestions/Responses/Projections(null→None; raw slashes).AsyncClient(env /~/.extralit/credentials.jsonresolution) and a mechanical syncClient(background-thread event-loop portal — works inside a running loop / Jupyter).schemas,records,questions,suggestions,projection,references); JSON on--jsonor non-TTY, errors as JSON on stderr, exit codes 0/1/2/3; JSONL stdin piping forrecords upsert. Replaces the v1schemassubcommand (composition-root import exception incli/app.py).extralit.client.login; onlycli/app.pyimports v2) and a zero-heavy-import delta gate (importingextralit.v2+CLI adds no pandas/pandera/datasets/huggingface_hub beyond baseline).Testing
schemastakeover.tests/unit/test_resources/*v1 failures (Missing api_key, environment) are unrelated — untouched by this branch.Deferred (follow-ups, non-blocking)
pytest-envenvoption is set in pyproject but the plugin isn't installed (soHF_HUB_DISABLE_TELEMETRYisn't applied) — add the dep or drop the stanza.?/#-safe (latent; DOI/filename refs only today).wrap_response_values/ResponseUpsertexported but unused (responses-write path deferred per spec).Sourceis a plainEnum→ comparecell.source == Source.suggestion, not the bare string (JSON output unaffected).Note
Several commits are roborev-hook-driven hardening (transport concurrency, error-contract broadening, CliRunner 8.1/8.2 compat) that landed mid-execution; they were reviewed in a consolidated pass and are sound. Open roborev reviews on the branch remain for the author to close.