diff --git a/docs/superpowers/plans/2026-07-13-sdk-v2-vertical-slice.md b/docs/superpowers/plans/2026-07-13-sdk-v2-vertical-slice.md new file mode 100644 index 000000000..686cb4201 --- /dev/null +++ b/docs/superpowers/plans/2026-07-13-sdk-v2-vertical-slice.md @@ -0,0 +1,3057 @@ +# Python SDK v2 Vertical Slice Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build the parallel `extralit.v2` SDK package (generated DTOs from the server's `openapi-dump` snapshot, async-native transport + sync facade, five resources for the extraction loop) and top-level agentic CLI verbs (JSON-first) that replace the v1 `schemas` subcommand. + +**Architecture:** A new `src/extralit/v2/` package behind a hard import wall (v2 imports nothing from v1 except the credentials helper; only `cli/app.py` — the composition root — imports v2). Wire types are generated from a committed OpenAPI snapshot; hand-written resources form the anti-corruption layer where all wire quirks live; a background-thread event-loop portal gives every async method a mechanical sync mirror. + +**Tech Stack:** httpx (AsyncClient), pydantic v2, datamodel-code-generator (dev), typer, pytest + pytest-asyncio (strict mode) + pytest-httpx. + +**Spec:** `docs/superpowers/specs/2026-07-13-sdk-v2-redesign-design.md` + +## Global Constraints + +- All work happens in `extralit/` (the SDK dir of the monorepo). Run commands from `/home/jonny/Projects/Extralit/extralit/extralit/` unless stated otherwise. +- Python floor is `>=3.9.2`: in all v2 source, use `Optional[X]` — never `X | None` — in annotations that pydantic evaluates. Builtin generics (`list[str]`, `dict[str, Any]`) are fine. Put `from __future__ import annotations` at the top of non-model modules only. +- Package management via `uv` only (`uv add`, `uv run pytest`, …). Ruff line length is 120. +- Import wall: files under `src/extralit/v2/` may import stdlib, `httpx`, `pydantic`, `typer`, `rich`, other `extralit.v2.*` modules, and exactly one v1 module: `extralit.client.login` (credentials). No other `extralit.*` import. Outside `v2/`, only `src/extralit/cli/app.py` may import `extralit.v2.*`. +- All wire payloads are typed from `src/extralit/v2/_api/_generated.py`. Never hand-write a wire shape. +- Server caps (mirror, don't re-invent): bulk-upsert ≤ 500 items/request, delete ≤ 100 ids/request, search/list `limit` 1–1000 (default 50). +- Auth endpoints: `POST /api/v2/token` (form `username`/`password`) and `POST /api/v2/token/refresh` (JSON `{"refresh_token": …}`) both return **201** with `{"access_token", "refresh_token"}`. API-key header is `X-Extralit-Api-Key`. +- v2 async tests: pytest-asyncio is in strict mode (no `asyncio_mode` configured) — every async test file needs `pytestmark = pytest.mark.asyncio`. +- Tests live under `tests/unit/v2/`. Commit after every task (pre-commit runs ruff). +- Search/list `total` is approximate (stale index ids skipped; FTS saturates ~10k) — document, never assert exactness in ergonomics. + +--- + +### Task 1: Contract layer — snapshot, codegen, drift gates + +**Files:** +- Create: `src/extralit/v2/__init__.py` (placeholder), `src/extralit/v2/_api/__init__.py`, `src/extralit/v2/_api/openapi.json` (generated), `src/extralit/v2/_api/_generated.py` (generated), `tests/unit/v2/__init__.py`, `tests/unit/v2/test_contract.py` +- Modify: `pyproject.toml` (dev dep + ruff exclude) + +**Interfaces:** +- Produces: `extralit.v2._api._generated` exporting pydantic v2 models named by the server's OpenAPI components: `SchemaRead`, `Schemas`, `SchemaCreate`, `SchemaUpdate`, `SchemaVersionCreate`, `SchemaVersionRead`, `RecordUpsert`, `RecordsBulkUpsert`, `RecordRead`, `Records`, `RecordFilter`, `RecordSearchQuery`, `ReferenceGroup`, `ReferenceView`, `QuestionCreate`, `QuestionUpdate`, `QuestionRead`, `Questions`, `SuggestionUpsert`, `SuggestionRead`, `Suggestions`, `ResponseUpsert`, `ResponseRead`, `ProjectionCell`, `ProjectionRecord`, `ProjectionView`, and enums `SchemaStatus`, `V2RecordStatus`, `QuestionType`, `SuggestionType`, `ResponseStatus`. + +- [ ] **Step 1: Add the codegen dev dependency** + +```bash +cd /home/jonny/Projects/Extralit/extralit/extralit +uv add --dev "datamodel-code-generator>=0.26" +``` + +- [ ] **Step 2: Dump the OpenAPI snapshot from the server tree** + +```bash +mkdir -p src/extralit/v2/_api +cd ../extralit-server +uv run python -m extralit_server openapi-dump --output ../extralit/src/extralit/v2/_api/openapi.json +cd ../extralit +python -c "import json; json.load(open('src/extralit/v2/_api/openapi.json'))" # sanity: valid JSON +``` + +- [ ] **Step 3: Generate the DTO module** + +```bash +uv run datamodel-codegen \ + --input src/extralit/v2/_api/openapi.json \ + --input-file-type openapi \ + --output src/extralit/v2/_api/_generated.py \ + --output-model-type pydantic_v2.BaseModel \ + --target-python-version 3.9 \ + --use-double-quotes \ + --disable-timestamp +``` + +`--disable-timestamp` is load-bearing: the no-drift gate diffs regenerated output byte-for-byte. + +- [ ] **Step 4: Exclude the generated file from ruff** + +In `pyproject.toml`, extend the `[tool.ruff]` section: + +```toml +[tool.ruff] +line-length = 120 +extend-exclude = ["src/extralit/v2/_api/_generated.py"] +``` + +Create the two package inits (empty for now): + +```bash +touch src/extralit/v2/__init__.py src/extralit/v2/_api/__init__.py tests/unit/v2/__init__.py +``` + +- [ ] **Step 5: Write the contract tests** + +`tests/unit/v2/test_contract.py`: + +```python +import json +import subprocess +import sys +from pathlib import Path + +import pytest + +SDK_ROOT = Path(__file__).parents[3] +API_DIR = SDK_ROOT / "src" / "extralit" / "v2" / "_api" +SNAPSHOT = API_DIR / "openapi.json" +GENERATED = API_DIR / "_generated.py" +SERVER_DIR = SDK_ROOT.parent / "extralit-server" + +EXPECTED_MODELS = [ + "SchemaRead", "Schemas", "SchemaCreate", "SchemaUpdate", "SchemaVersionCreate", "SchemaVersionRead", + "RecordUpsert", "RecordsBulkUpsert", "RecordRead", "Records", "RecordFilter", "RecordSearchQuery", + "ReferenceGroup", "ReferenceView", + "QuestionCreate", "QuestionUpdate", "QuestionRead", "Questions", + "SuggestionUpsert", "SuggestionRead", "Suggestions", "ResponseUpsert", "ResponseRead", + "ProjectionCell", "ProjectionRecord", "ProjectionView", + "SchemaStatus", "V2RecordStatus", "QuestionType", "SuggestionType", "ResponseStatus", +] + + +def test_generated_models_importable(): + import extralit.v2._api._generated as gen + + missing = [name for name in EXPECTED_MODELS if not hasattr(gen, name)] + assert not missing, f"generated module lacks: {missing}" + + +def test_generated_matches_snapshot(tmp_path): + """No-drift gate: regenerating from the committed snapshot must be byte-identical.""" + out = tmp_path / "regen.py" + subprocess.run( + [ + sys.executable, "-m", "datamodel_code_generator", + "--input", str(SNAPSHOT), "--input-file-type", "openapi", + "--output", str(out), "--output-model-type", "pydantic_v2.BaseModel", + "--target-python-version", "3.9", "--use-double-quotes", "--disable-timestamp", + ], + check=True, capture_output=True, + ) + assert out.read_text() == GENERATED.read_text(), ( + "src/extralit/v2/_api/_generated.py drifted from openapi.json — rerun datamodel-codegen (see plan Task 1 Step 3)" + ) + + +@pytest.mark.slow +@pytest.mark.skipif(not SERVER_DIR.exists(), reason="server tree not present") +def test_snapshot_matches_server(): + """Snapshot-vs-server gate: committed snapshot must equal a fresh openapi-dump.""" + proc = subprocess.run( + ["uv", "run", "python", "-m", "extralit_server", "openapi-dump"], + cwd=SERVER_DIR, check=True, capture_output=True, text=True, + ) + assert json.loads(proc.stdout) == json.loads(SNAPSHOT.read_text()), ( + "openapi.json snapshot drifted from the server — re-dump it (see plan Task 1 Step 2)" + ) +``` + +- [ ] **Step 6: Run the tests** + +```bash +uv run pytest tests/unit/v2/test_contract.py -v --disable-warnings +``` + +Expected: `test_generated_models_importable` and `test_generated_matches_snapshot` PASS; `test_snapshot_matches_server` deselected (slow) — run it once explicitly with `--runslow` and confirm PASS. If `test_generated_models_importable` fails on a name, inspect `_generated.py` for the actual component name and update `EXPECTED_MODELS` *and* note the real name for later tasks (later tasks import these names). + +- [ ] **Step 7: Commit** + +```bash +git add pyproject.toml uv.lock src/extralit/v2 tests/unit/v2 +git commit -m "feat(sdk-v2): committed OpenAPI snapshot + generated DTOs with drift gates" +``` + +--- + +### Task 2: Error hierarchy and 422 normalizer + +**Files:** +- Create: `src/extralit/v2/_api/_errors.py`, `tests/unit/v2/test_errors.py` + +**Interfaces:** +- Produces: + - `class V2APIError(Exception)` — attrs `.status_code: int`, `.detail: Any` + - `class AuthError(V2APIError)`, `class NotFoundError(V2APIError)` + - `class ValidationError(V2APIError)` — extra attr `.errors: list[dict]` (each `{"loc": list, "msg": str}`) + - `def error_from_response(status_code: int, body: Any) -> V2APIError` + - `def normalize_validation_detail(detail: Any) -> list[dict]` + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_errors.py`: + +```python +from extralit.v2._api._errors import ( + AuthError, + NotFoundError, + V2APIError, + ValidationError, + error_from_response, + normalize_validation_detail, +) + + +def test_normalizes_string_detail(): + assert normalize_validation_detail("boom") == [{"loc": [], "msg": "boom"}] + + +def test_normalizes_fastapi_list_detail(): + detail = [{"loc": ["body", "items", 0, "reference"], "msg": "field required", "type": "missing"}] + assert normalize_validation_detail(detail) == [ + {"loc": ["body", "items", 0, "reference"], "msg": "field required"} + ] + + +def test_normalizes_none_and_junk(): + assert normalize_validation_detail(None) == [] + assert normalize_validation_detail({"weird": 1}) == [{"loc": [], "msg": "{'weird': 1}"}] + + +def test_error_from_response_maps_statuses(): + assert isinstance(error_from_response(401, {"detail": "nope"}), AuthError) + assert isinstance(error_from_response(403, {"detail": "nope"}), AuthError) + assert isinstance(error_from_response(404, {"detail": "gone"}), NotFoundError) + err = error_from_response(422, {"detail": "bad value"}) + assert isinstance(err, ValidationError) + assert err.errors == [{"loc": [], "msg": "bad value"}] + other = error_from_response(500, {"detail": "kaboom"}) + assert type(other) is V2APIError + assert other.status_code == 500 and other.detail == "kaboom" + + +def test_error_from_response_non_dict_body(): + err = error_from_response(502, "bad gateway") + assert err.detail == "bad gateway" +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_errors.py -v --disable-warnings` +Expected: FAIL — `ModuleNotFoundError: No module named 'extralit.v2._api._errors'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/_api/_errors.py`: + +```python +from __future__ import annotations + +from typing import Any + + +class V2APIError(Exception): + """Base error for /api/v2 calls.""" + + def __init__(self, status_code: int, detail: Any = None): + self.status_code = status_code + self.detail = detail + super().__init__(f"HTTP {status_code}: {detail!r}") + + +class AuthError(V2APIError): + """401/403, raised after any transparent token refresh has already been attempted.""" + + +class NotFoundError(V2APIError): + """404.""" + + +class ValidationError(V2APIError): + """422. The server emits two body shapes (detail: str | list[{loc, msg}]); `.errors` is normalized.""" + + def __init__(self, status_code: int, detail: Any = None): + super().__init__(status_code, detail) + self.errors = normalize_validation_detail(detail) + + +def normalize_validation_detail(detail: Any) -> list[dict]: + if detail is None: + return [] + if isinstance(detail, str): + return [{"loc": [], "msg": detail}] + if isinstance(detail, list): + out = [] + for item in detail: + if isinstance(item, dict): + out.append({"loc": list(item.get("loc", [])), "msg": str(item.get("msg", item))}) + else: + out.append({"loc": [], "msg": str(item)}) + return out + return [{"loc": [], "msg": str(detail)}] + + +def error_from_response(status_code: int, body: Any) -> V2APIError: + detail = body.get("detail") if isinstance(body, dict) else body + if status_code in (401, 403): + return AuthError(status_code, detail) + if status_code == 404: + return NotFoundError(status_code, detail) + if status_code == 422: + return ValidationError(status_code, detail) + return V2APIError(status_code, detail) +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_errors.py -v --disable-warnings` +Expected: 5 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/_api/_errors.py tests/unit/v2/test_errors.py +git commit -m "feat(sdk-v2): error hierarchy with dual-shape 422 normalizer" +``` + +--- + +### Task 3: Async transport — auth modes, refresh-once, error mapping + +**Files:** +- Create: `src/extralit/v2/_api/_transport.py`, `tests/unit/v2/test_transport.py` + +**Interfaces:** +- Consumes: `error_from_response` from Task 2. +- Produces: + - `class AsyncTransport` — `def __init__(self, api_url: str, api_key: Optional[str] = None, username: Optional[str] = None, password: Optional[str] = None, timeout: float = 60.0, retries: int = 5, extra_headers: Optional[dict] = None)` + - `async def request(self, method: str, path: str, *, params: Optional[dict] = None, json: Optional[Any] = None) -> Any` — `path` is relative to `/api/v2` (e.g. `"/schemas"`); returns parsed JSON, or `None` for 204/empty bodies. Raises the Task 2 hierarchy on ≥400. + - `async def aclose(self) -> None` + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_transport.py`: + +```python +import pytest + +from extralit.v2._api._errors import AuthError, NotFoundError, ValidationError +from extralit.v2._api._transport import AsyncTransport + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" + + +async def test_api_key_header_sent(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", json={"items": []}) + t = AsyncTransport(API, api_key="secret.key") + body = await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + assert body == {"items": []} + assert httpx_mock.get_requests()[0].headers["X-Extralit-Api-Key"] == "secret.key" + await t.aclose() + + +async def test_password_login_then_bearer(httpx_mock): + httpx_mock.add_response( + method="POST", url=f"{API}/api/v2/token", status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", json={"items": []}) + t = AsyncTransport(API, username="u", password="p") + await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + token_req, api_req = httpx_mock.get_requests() + assert b"username=u" in token_req.content and b"password=p" in token_req.content + assert api_req.headers["Authorization"] == "Bearer AT1" + await t.aclose() + + +async def test_refresh_once_on_401_then_retry(httpx_mock): + httpx_mock.add_response( + method="POST", url=f"{API}/api/v2/token", status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + httpx_mock.add_response( + url=f"{API}/api/v2/schemas?workspace_id=w1", status_code=401, json={"detail": "expired"} + ) + httpx_mock.add_response( + method="POST", url=f"{API}/api/v2/token/refresh", status_code=201, + json={"access_token": "AT2", "refresh_token": "RT2"}, + ) + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", json={"items": []}) + t = AsyncTransport(API, username="u", password="p") + body = await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + assert body == {"items": []} + refresh_req = httpx_mock.get_requests()[2] + assert b"RT1" in refresh_req.content + assert httpx_mock.get_requests()[3].headers["Authorization"] == "Bearer AT2" + await t.aclose() + + +async def test_401_after_failed_refresh_raises_auth_error(httpx_mock): + httpx_mock.add_response( + method="POST", url=f"{API}/api/v2/token", status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + httpx_mock.add_response( + url=f"{API}/api/v2/schemas?workspace_id=w1", status_code=401, json={"detail": "expired"} + ) + httpx_mock.add_response(method="POST", url=f"{API}/api/v2/token/refresh", status_code=401, json={"detail": "no"}) + t = AsyncTransport(API, username="u", password="p") + with pytest.raises(AuthError): + await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + await t.aclose() + + +async def test_api_key_401_raises_without_refresh(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", status_code=401, json={"detail": "bad key"}) + t = AsyncTransport(API, api_key="bad") + with pytest.raises(AuthError): + await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + assert len(httpx_mock.get_requests()) == 1 # no refresh attempt in api-key mode + await t.aclose() + + +async def test_error_mapping_and_204(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas/x", status_code=404, json={"detail": "gone"}) + httpx_mock.add_response(method="POST", url=f"{API}/api/v2/schemas", status_code=422, json={"detail": "bad"}) + httpx_mock.add_response(method="DELETE", url=f"{API}/api/v2/schemas/y/records?ids=a", status_code=204) + t = AsyncTransport(API, api_key="k") + with pytest.raises(NotFoundError): + await t.request("GET", "/schemas/x") + with pytest.raises(ValidationError): + await t.request("POST", "/schemas", json={}) + assert await t.request("DELETE", "/schemas/y/records", params={"ids": "a"}) is None + await t.aclose() +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_transport.py -v --disable-warnings` +Expected: FAIL — `ModuleNotFoundError: No module named 'extralit.v2._api._transport'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/_api/_transport.py`: + +```python +from __future__ import annotations + +from typing import Any, Optional + +import httpx + +from extralit.v2._api._errors import AuthError, error_from_response + +_API_PREFIX = "/api/v2" + + +def _safe_json(response: httpx.Response) -> Any: + try: + return response.json() + except ValueError: + return response.text + + +class AsyncTransport: + """One httpx.AsyncClient per client instance. Auth modes: api_key header (default, + no token lifecycle) or username/password -> bearer JWT with a single transparent + refresh on 401 (then AuthError).""" + + def __init__( + self, + api_url: str, + api_key: Optional[str] = None, + username: Optional[str] = None, + password: Optional[str] = None, + timeout: float = 60.0, + retries: int = 5, + extra_headers: Optional[dict] = None, + ): + self.api_url = api_url.rstrip("/") + self._api_key = api_key + self._username = username + self._password = password + self._access_token: Optional[str] = None + self._refresh_token: Optional[str] = None + self._http = httpx.AsyncClient( + base_url=self.api_url, + timeout=timeout, + transport=httpx.AsyncHTTPTransport(retries=retries), + headers=extra_headers or {}, + ) + + def _auth_headers(self) -> dict: + if self._access_token: + return {"Authorization": f"Bearer {self._access_token}"} + if self._api_key: + return {"X-Extralit-Api-Key": self._api_key} + return {} + + async def _login(self) -> None: + response = await self._http.post( + f"{_API_PREFIX}/token", data={"username": self._username, "password": self._password} + ) + if response.status_code >= 400: + raise AuthError(response.status_code, _safe_json(response)) + payload = response.json() + self._access_token = payload["access_token"] + self._refresh_token = payload.get("refresh_token") + + async def _refresh(self) -> bool: + if not self._refresh_token: + return False + response = await self._http.post( + f"{_API_PREFIX}/token/refresh", json={"refresh_token": self._refresh_token} + ) + if response.status_code >= 400: + return False + payload = response.json() + self._access_token = payload["access_token"] + self._refresh_token = payload.get("refresh_token", self._refresh_token) + return True + + async def request( + self, + method: str, + path: str, + *, + params: Optional[dict] = None, + json: Optional[Any] = None, + ) -> Any: + if self._username and not self._access_token and not self._api_key: + await self._login() + response = await self._http.request( + method, f"{_API_PREFIX}{path}", params=params, json=json, headers=self._auth_headers() + ) + if response.status_code == 401 and self._access_token and await self._refresh(): + response = await self._http.request( + method, f"{_API_PREFIX}{path}", params=params, json=json, headers=self._auth_headers() + ) + if response.status_code >= 400: + raise error_from_response(response.status_code, _safe_json(response)) + if response.status_code == 204 or not response.content: + return None + return response.json() + + async def aclose(self) -> None: + await self._http.aclose() +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_transport.py -v --disable-warnings` +Expected: 6 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/_api/_transport.py tests/unit/v2/test_transport.py +git commit -m "feat(sdk-v2): async transport with api-key/bearer auth and refresh-once" +``` + +--- + +### Task 4: Domain models and response-value wrap/unwrap + +**Files:** +- Create: `src/extralit/v2/models.py`, `tests/unit/v2/test_models.py` + +**Interfaces:** +- Consumes: generated DTOs from Task 1. +- Produces (all constructed via `Model.model_validate(payload)`): + - `class Schema(SchemaRead)`, `class Record(RecordRead)`, `class Question(QuestionRead)`, `class Suggestion(SuggestionRead)` — plain subclasses + - `class SchemaVersion(SchemaVersionRead)` with `def find_column(self, name: str) -> Optional[dict]` + - `class Response(ResponseRead)` with property `unwrapped_values: dict` + - `class SearchPage(BaseModel)` — `items: list[Record]`, `total: int` (approximate) + - `def wrap_response_values(values: dict) -> dict`, `def unwrap_response_values(values) -> dict` + - Re-exports: `ProjectionCell`, `ProjectionRecord`, `ProjectionView`, `ReferenceGroup`, `ReferenceView` + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_models.py`: + +```python +import uuid +from datetime import datetime, timezone + +from extralit.v2.models import ( + Record, + Response, + SchemaVersion, + SearchPage, + unwrap_response_values, + wrap_response_values, +) + + +def _version_payload(**overrides): + payload = { + "id": str(uuid.uuid4()), + "schema_id": str(uuid.uuid4()), + "version": 1, + "object_key": "schemas/x/v1.json", + "object_version_id": None, + "etag": "e", + "checksum": "c", + "parent_version_id": None, + "columns_cache": [{"name": "size", "dtype": "str"}, {"name": "country", "dtype": "str"}], + "review_widgets": {}, + "inserted_at": datetime.now(timezone.utc).isoformat(), + } + payload.update(overrides) + return payload + + +def test_schema_version_find_column(): + version = SchemaVersion.model_validate(_version_payload()) + assert version.find_column("size") == {"name": "size", "dtype": "str"} + assert version.find_column("nope") is None + + +def test_wrap_unwrap_roundtrip(): + """Server double-wraps response values ({name: {"value": ...}}) on both PUT and GET.""" + values = {"size": "120", "country": ["KE", "UG"]} + wrapped = wrap_response_values(values) + assert wrapped == {"size": {"value": "120"}, "country": {"value": ["KE", "UG"]}} + assert unwrap_response_values(wrapped) == values + assert unwrap_response_values(None) == {} + + +def test_response_unwrapped_values(): + response = Response.model_validate( + { + "id": str(uuid.uuid4()), + "record_id": str(uuid.uuid4()), + "user_id": str(uuid.uuid4()), + "values": {"size": {"value": "135"}}, + "status": "submitted", + "inserted_at": datetime.now(timezone.utc).isoformat(), + "updated_at": datetime.now(timezone.utc).isoformat(), + } + ) + assert response.unwrapped_values == {"size": "135"} + + +def test_search_page_holds_records(): + record = { + "id": str(uuid.uuid4()), + "schema_id": str(uuid.uuid4()), + "schema_version_id": str(uuid.uuid4()), + "reference": "10.1000/xyz", + "external_id": None, + "fields": {"size": "120"}, + "metadata": None, + "status": "pending", + "inserted_at": datetime.now(timezone.utc).isoformat(), + "updated_at": datetime.now(timezone.utc).isoformat(), + } + page = SearchPage(items=[Record.model_validate(record)], total=1) + assert page.items[0].reference == "10.1000/xyz" + assert page.total == 1 +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_models.py -v --disable-warnings` +Expected: FAIL — `ModuleNotFoundError: No module named 'extralit.v2.models'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/models.py` (no `from __future__ import annotations` here — pydantic models on 3.9, so use `Optional`/builtin generics directly): + +```python +from typing import Any, Optional + +from pydantic import BaseModel + +from extralit.v2._api._generated import ( + ProjectionCell, + ProjectionRecord, + ProjectionView, + QuestionRead, + RecordRead, + ReferenceGroup, + ReferenceView, + ResponseRead, + SchemaRead, + SchemaVersionRead, + SuggestionRead, +) + +__all__ = [ + "Schema", "SchemaVersion", "Record", "Question", "Suggestion", "Response", "SearchPage", + "ProjectionCell", "ProjectionRecord", "ProjectionView", "ReferenceGroup", "ReferenceView", + "wrap_response_values", "unwrap_response_values", +] + + +class Schema(SchemaRead): + pass + + +class SchemaVersion(SchemaVersionRead): + def find_column(self, name: str) -> Optional[dict]: + for column in self.columns_cache: + if column.get("name") == name: + return column + return None + + +class Record(RecordRead): + pass + + +class Question(QuestionRead): + pass + + +class Suggestion(SuggestionRead): + pass + + +def wrap_response_values(values: dict) -> dict: + """Server stores response values double-wrapped: {question_name: {"value": ...}}.""" + return {name: {"value": value} for name, value in values.items()} + + +def unwrap_response_values(values: Optional[dict]) -> dict: + return { + name: cell.get("value") if isinstance(cell, dict) else cell + for name, cell in (values or {}).items() + } + + +class Response(ResponseRead): + @property + def unwrapped_values(self) -> dict: + return unwrap_response_values(self.values) + + +class SearchPage(BaseModel): + """One page of records. `total` is approximate: stale index ids are skipped and + FTS saturates (~10k) — never present it as an exact count.""" + + items: "list[Record]" + total: int +``` + +Note: if `_generated.py` field types make subclass validation fail (e.g. enum vs str), check the generated field names — the contract test in Task 1 pinned the class names, and these tests pin the field behavior. + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_models.py -v --disable-warnings` +Expected: 4 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/models.py tests/unit/v2/test_models.py +git commit -m "feat(sdk-v2): domain models with double-wrap value helpers" +``` + +--- + +### Task 5: Schemas resource (create/list/get/get_by_name/update/publish/versions/columns) + +**Files:** +- Create: `src/extralit/v2/resources/__init__.py`, `src/extralit/v2/resources/_base.py`, `src/extralit/v2/resources/_schemas.py`, `tests/unit/v2/test_schemas_resource.py` + +**Interfaces:** +- Consumes: `AsyncTransport.request` (Task 3), models (Task 4), `NotFoundError` (Task 2). +- Produces: + - `class ResourceBase` — `def __init__(self, transport: AsyncTransport)`, attr `self._transport` + - `class Schemas(ResourceBase)` with async methods: + - `create(workspace_id, name, settings=None) -> Schema` + - `list(workspace_id) -> list[Schema]` + - `get(schema_id) -> Schema` + - `get_by_name(workspace_id, name) -> Schema` (raises `NotFoundError(404, ...)` when absent) + - `update(schema_id, *, name=None, settings=None) -> Schema` + - `publish(schema_id, schema, review_widgets=None) -> SchemaVersion` — `schema` is a Pandera `DataFrameSchema` (duck-typed via `.to_json()`) or an already-serialized JSON string + - `versions(schema_id) -> list[SchemaVersion]` + - `get_version(schema_id, version: int) -> SchemaVersion` — cached by `(schema_id, version)` (immutable) + - `columns(schema_id) -> list[dict]` + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_schemas_resource.py`: + +```python +import uuid +from datetime import datetime, timezone + +import pytest + +from extralit.v2._api._errors import NotFoundError +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.resources import Schemas + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" +WS = str(uuid.uuid4()) +SCHEMA_ID = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _schema(name="trials"): + return { + "id": SCHEMA_ID, "name": name, "status": "draft", "current_version_id": None, + "settings": {}, "workspace_id": WS, "inserted_at": NOW, "updated_at": NOW, + } + + +def _version(version=1): + return { + "id": str(uuid.uuid4()), "schema_id": SCHEMA_ID, "version": version, + "object_key": f"schemas/{SCHEMA_ID}/v{version}.json", "object_version_id": None, + "etag": "e", "checksum": "c", "parent_version_id": None, + "columns_cache": [{"name": "size"}], "review_widgets": {}, "inserted_at": NOW, + } + + +@pytest.fixture +async def schemas(): + transport = AsyncTransport(API, api_key="k") + yield Schemas(transport) + await transport.aclose() + + +async def test_create_and_get(httpx_mock, schemas): + httpx_mock.add_response(method="POST", url=f"{API}/api/v2/schemas", status_code=201, json=_schema()) + created = await schemas.create(WS, "trials") + assert created.name == "trials" + body = httpx_mock.get_requests()[0].read() + assert b'"workspace_id"' in body and b'"trials"' in body + + +async def test_get_by_name_found_and_missing(httpx_mock, schemas): + listing = {"items": [_schema("other") | {"id": str(uuid.uuid4())}, _schema("trials")]} + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id={WS}", json=listing) + found = await schemas.get_by_name(WS, "trials") + assert str(found.id) == SCHEMA_ID + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id={WS}", json={"items": []}) + with pytest.raises(NotFoundError): + await schemas.get_by_name(WS, "trials") + + +async def test_publish_accepts_pandera_object_or_string(httpx_mock, schemas): + httpx_mock.add_response( + method="POST", url=f"{API}/api/v2/schemas/{SCHEMA_ID}/versions", status_code=201, json=_version() + ) + + class FakePandera: # duck-type: anything with .to_json() + def to_json(self): + return '{"columns": {"size": {}}}' + + version = await schemas.publish(SCHEMA_ID, FakePandera(), review_widgets={"size": {"widget": "text"}}) + assert version.version == 1 + sent = httpx_mock.get_requests()[0].read() + assert b'"columns"' in sent and b'"review_widgets"' in sent + + +async def test_get_version_cached(httpx_mock, schemas): + httpx_mock.add_response(url=f"{API}/api/v2/schemas/{SCHEMA_ID}/versions/1", json=_version()) + v1 = await schemas.get_version(SCHEMA_ID, 1) + v1_again = await schemas.get_version(SCHEMA_ID, 1) # served from cache: no second request + assert v1_again is v1 + assert len(httpx_mock.get_requests()) == 1 + + +async def test_versions_and_columns(httpx_mock, schemas): + httpx_mock.add_response(url=f"{API}/api/v2/schemas/{SCHEMA_ID}/versions", json=[_version(1), _version(2)]) + httpx_mock.add_response(url=f"{API}/api/v2/schemas/{SCHEMA_ID}/columns", json=[{"name": "size"}]) + assert [v.version for v in await schemas.versions(SCHEMA_ID)] == [1, 2] + assert await schemas.columns(SCHEMA_ID) == [{"name": "size"}] +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_schemas_resource.py -v --disable-warnings` +Expected: FAIL — `ImportError` (no `extralit.v2.resources`) + +- [ ] **Step 3: Implement** + +`src/extralit/v2/resources/_base.py`: + +```python +from extralit.v2._api._transport import AsyncTransport + + +class ResourceBase: + def __init__(self, transport: AsyncTransport): + self._transport = transport +``` + +`src/extralit/v2/resources/_schemas.py`: + +```python +from __future__ import annotations + +from typing import Any, Optional + +from extralit.v2._api._errors import NotFoundError +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.models import Schema, SchemaVersion +from extralit.v2.resources._base import ResourceBase + + +class Schemas(ResourceBase): + def __init__(self, transport: AsyncTransport): + super().__init__(transport) + self._version_cache: dict = {} # (schema_id, version) -> SchemaVersion; versions are immutable + + async def create(self, workspace_id, name: str, settings: Optional[dict] = None) -> Schema: + payload = await self._transport.request( + "POST", "/schemas", + json={"name": name, "workspace_id": str(workspace_id), "settings": settings or {}}, + ) + return Schema.model_validate(payload) + + async def list(self, workspace_id) -> list[Schema]: + payload = await self._transport.request("GET", "/schemas", params={"workspace_id": str(workspace_id)}) + return [Schema.model_validate(item) for item in payload["items"]] + + async def get(self, schema_id) -> Schema: + return Schema.model_validate(await self._transport.request("GET", f"/schemas/{schema_id}")) + + async def get_by_name(self, workspace_id, name: str) -> Schema: + for schema in await self.list(workspace_id): + if schema.name == name: + return schema + raise NotFoundError(404, f"schema named {name!r} not found in workspace {workspace_id}") + + async def update(self, schema_id, *, name: Optional[str] = None, settings: Optional[dict] = None) -> Schema: + body: dict = {} + if name is not None: + body["name"] = name + if settings is not None: + body["settings"] = settings + return Schema.model_validate(await self._transport.request("PUT", f"/schemas/{schema_id}", json=body)) + + async def publish(self, schema_id, schema: Any, review_widgets: Optional[dict] = None) -> SchemaVersion: + """Publish a new schema version. `schema` is a pandera DataFrameSchema (anything with + .to_json()) or the already-serialized JSON string. review_widgets ride out-of-band + because pandera's to_json() drops Column.metadata.""" + body = schema.to_json() if hasattr(schema, "to_json") else schema + payload = await self._transport.request( + "POST", f"/schemas/{schema_id}/versions", + json={"body": body, "review_widgets": review_widgets or {}}, + ) + version = SchemaVersion.model_validate(payload) + self._version_cache[(str(schema_id), version.version)] = version + return version + + async def versions(self, schema_id) -> list[SchemaVersion]: + payload = await self._transport.request("GET", f"/schemas/{schema_id}/versions") + return [SchemaVersion.model_validate(item) for item in payload] + + async def get_version(self, schema_id, version: int) -> SchemaVersion: + key = (str(schema_id), version) + if key not in self._version_cache: + payload = await self._transport.request("GET", f"/schemas/{schema_id}/versions/{version}") + self._version_cache[key] = SchemaVersion.model_validate(payload) + return self._version_cache[key] + + async def columns(self, schema_id) -> list[dict]: + return await self._transport.request("GET", f"/schemas/{schema_id}/columns") +``` + +`src/extralit/v2/resources/__init__.py`: + +```python +from extralit.v2.resources._schemas import Schemas + +__all__ = ["Schemas"] +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_schemas_resource.py -v --disable-warnings` +Expected: 5 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/resources tests/unit/v2/test_schemas_resource.py +git commit -m "feat(sdk-v2): Schemas resource with immutable version caching" +``` + +--- + +### Task 6: Questions resource with cached name→id map + +**Files:** +- Create: `src/extralit/v2/resources/_questions.py`, `tests/unit/v2/test_questions_resource.py` +- Modify: `src/extralit/v2/resources/__init__.py` + +**Interfaces:** +- Consumes: `ResourceBase`, `Question` model, `NotFoundError`. +- Produces: `class Questions(ResourceBase)`: + - `async def list(self, schema_id) -> list[Question]` + - `async def get(self, question_id) -> Question` + - `async def id_for(self, schema_id, name: str) -> UUID` — cached per schema; on a cache miss for a known schema it refetches once (a question may have been added) before raising `NotFoundError`. **This is the name↔id join used by Suggestions — suggestions key by question id, while cells/response values key by name.** + - `def invalidate(self, schema_id) -> None` — drops the cached map + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_questions_resource.py`: + +```python +import uuid +from datetime import datetime, timezone + +import pytest + +from extralit.v2._api._errors import NotFoundError +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.resources import Questions + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" +SCHEMA_ID = str(uuid.uuid4()) +Q_SIZE = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _question(qid, name): + return { + "id": qid, "schema_id": SCHEMA_ID, "name": name, "title": name.title(), "description": None, + "type": "text", "columns": [name], "settings": {}, "required": False, + "inserted_at": NOW, "updated_at": NOW, + } + + +@pytest.fixture +async def questions(): + transport = AsyncTransport(API, api_key="k") + yield Questions(transport) + await transport.aclose() + + +async def test_list_and_id_for_uses_one_fetch(httpx_mock, questions): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", json={"items": [_question(Q_SIZE, "size")]} + ) + assert [q.name for q in await questions.list(SCHEMA_ID)] == ["size"] + assert str(await questions.id_for(SCHEMA_ID, "size")) == Q_SIZE # served from cache + assert len(httpx_mock.get_requests()) == 1 + + +async def test_id_for_refetches_once_then_raises(httpx_mock, questions): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", json={"items": [_question(Q_SIZE, "size")]} + ) + await questions.list(SCHEMA_ID) + q_new = str(uuid.uuid4()) + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", + json={"items": [_question(Q_SIZE, "size"), _question(q_new, "dosage")]}, + ) + assert str(await questions.id_for(SCHEMA_ID, "dosage")) == q_new # miss -> refetch -> hit + httpx_mock.add_response(url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", json={"items": []}) + with pytest.raises(NotFoundError): + await questions.id_for(SCHEMA_ID, "ghost") +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_questions_resource.py -v --disable-warnings` +Expected: FAIL — `ImportError: cannot import name 'Questions'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/resources/_questions.py`: + +```python +from __future__ import annotations + +from uuid import UUID + +from extralit.v2._api._errors import NotFoundError +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.models import Question +from extralit.v2.resources._base import ResourceBase + + +class Questions(ResourceBase): + """Callers address questions by NAME; the server keys suggestions by question ID + (cells/response values key by name). This resource owns that join via a cached map.""" + + def __init__(self, transport: AsyncTransport): + super().__init__(transport) + self._maps: dict = {} # str(schema_id) -> {name: UUID} + + async def list(self, schema_id) -> list[Question]: + payload = await self._transport.request("GET", f"/schemas/{schema_id}/questions") + items = [Question.model_validate(item) for item in payload["items"]] + self._maps[str(schema_id)] = {q.name: q.id for q in items} + return items + + async def get(self, question_id) -> Question: + return Question.model_validate(await self._transport.request("GET", f"/questions/{question_id}")) + + async def id_for(self, schema_id, name: str) -> UUID: + key = str(schema_id) + if key not in self._maps or name not in self._maps[key]: + await self.list(schema_id) # refetch once: the question may be newly created + if name not in self._maps.get(key, {}): + raise NotFoundError(404, f"question named {name!r} not found in schema {schema_id}") + return self._maps[key][name] + + def invalidate(self, schema_id) -> None: + self._maps.pop(str(schema_id), None) +``` + +Update `src/extralit/v2/resources/__init__.py`: + +```python +from extralit.v2.resources._questions import Questions +from extralit.v2.resources._schemas import Schemas + +__all__ = ["Questions", "Schemas"] +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_questions_resource.py -v --disable-warnings` +Expected: 2 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/resources tests/unit/v2/test_questions_resource.py +git commit -m "feat(sdk-v2): Questions resource owning the name-to-id join" +``` + +--- + +### Task 7: Records resource — chunked concurrent bulk upsert, search, list, delete + +**Files:** +- Create: `src/extralit/v2/resources/_records.py`, `tests/unit/v2/test_records_resource.py` +- Modify: `src/extralit/v2/resources/__init__.py` + +**Interfaces:** +- Consumes: `ResourceBase`, `Record`/`SearchPage`/`ReferenceView` models. +- Produces: `class Records(ResourceBase)`: + - `async def bulk_upsert(self, schema_id, items, *, reference=None, max_concurrency=4, on_progress=None) -> list[Record]` — `items`: list of dicts (full `RecordUpsert` shape if a `"fields"` key is present, else treated as bare field dicts) or a pandas DataFrame (lazy import; rows become field dicts, a `reference` column is lifted out). Chunks at 500, dispatches chunks concurrently under a semaphore, preserves input order in the returned list. `on_progress(done_count, total_count)` called per completed chunk. + - `async def search(self, schema_id, *, text=None, filters=None, offset=0, limit=50) -> SearchPage` — `filters`: list of `(column, op, value)` tuples or `{"column","op","value"}` dicts, `op ∈ {eq,in,ge,le}`. + - `async def list(self, schema_id, *, offset=0, limit=50, status=None, reference=None) -> SearchPage` + - `async def delete(self, schema_id, ids) -> None` — chunks at 100 ids per request (comma-separated query param). + - `async def get_reference(self, workspace_id, reference: str) -> ReferenceView` — path is NOT url-encoded for slashes (server route is `{reference:path}`). + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_records_resource.py`: + +```python +import json +import uuid +from datetime import datetime, timezone + +import pytest + +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.resources import Records + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" +SCHEMA_ID = str(uuid.uuid4()) +WS = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _record(i): + return { + "id": str(uuid.uuid4()), "schema_id": SCHEMA_ID, "schema_version_id": str(uuid.uuid4()), + "reference": "10.1000/xyz", "external_id": str(i), "fields": {"size": str(i)}, + "metadata": None, "status": "pending", "inserted_at": NOW, "updated_at": NOW, + } + + +@pytest.fixture +async def records(): + transport = AsyncTransport(API, api_key="k") + yield Records(transport) + await transport.aclose() + + +async def test_bulk_upsert_chunks_at_500_and_preserves_order(httpx_mock, records): + url = f"{API}/api/v2/schemas/{SCHEMA_ID}/records:bulk-upsert" + + def responder(request): + import httpx + + sent = json.loads(request.read())["items"] + assert len(sent) <= 500 + return httpx.Response(200, json={"items": [_record(item["external_id"]) for item in sent], "total": len(sent)}) + + for _ in range(3): + httpx_mock.add_callback(responder, method="POST", url=url) + + items = [{"fields": {"size": str(i)}, "reference": "10.1000/xyz", "external_id": str(i)} for i in range(1200)] + progress = [] + result = await records.bulk_upsert(SCHEMA_ID, items, on_progress=lambda done, total: progress.append((done, total))) + assert len(result) == 1200 + assert [r.external_id for r in result] == [str(i) for i in range(1200)] # input order preserved + assert len(httpx_mock.get_requests()) == 3 # 500 + 500 + 200 + assert progress[-1] == (1200, 1200) + + +async def test_bulk_upsert_bare_fields_and_shared_reference(httpx_mock, records): + url = f"{API}/api/v2/schemas/{SCHEMA_ID}/records:bulk-upsert" + httpx_mock.add_response(method="POST", url=url, json={"items": [_record(0)], "total": 1}) + await records.bulk_upsert(SCHEMA_ID, [{"size": "120"}], reference="10.1000/xyz") + sent = json.loads(httpx_mock.get_requests()[0].read())["items"] + assert sent == [{"fields": {"size": "120"}, "reference": "10.1000/xyz"}] + + +async def test_bulk_upsert_without_reference_raises(records): + with pytest.raises(ValueError, match="reference"): + await records.bulk_upsert(SCHEMA_ID, [{"size": "120"}]) + + +async def test_bulk_upsert_accepts_dataframe(httpx_mock, records): + pandas = pytest.importorskip("pandas") + url = f"{API}/api/v2/schemas/{SCHEMA_ID}/records:bulk-upsert" + httpx_mock.add_response(method="POST", url=url, json={"items": [_record(0), _record(1)], "total": 2}) + frame = pandas.DataFrame([ + {"size": "120", "reference": "10.1000/a"}, + {"size": "135", "reference": "10.1000/b"}, + ]) + await records.bulk_upsert(SCHEMA_ID, frame) + sent = json.loads(httpx_mock.get_requests()[0].read())["items"] + assert sent == [ + {"fields": {"size": "120"}, "reference": "10.1000/a"}, + {"fields": {"size": "135"}, "reference": "10.1000/b"}, + ] + + +async def test_search_normalizes_tuple_filters(httpx_mock, records): + url = f"{API}/api/v2/schemas/{SCHEMA_ID}/records:search" + httpx_mock.add_response(method="POST", url=url, json={"items": [_record(1)], "total": 41}) + page = await records.search(SCHEMA_ID, text="tumor", filters=[("age", "ge", 18)], limit=10) + assert page.total == 41 + body = json.loads(httpx_mock.get_requests()[0].read()) + assert body == { + "text": "tumor", + "filters": [{"column": "age", "op": "ge", "value": 18}], + "offset": 0, + "limit": 10, + } + + +async def test_list_passes_status_and_reference(httpx_mock, records): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/records?offset=0&limit=50&status=pending&reference=10.1000%2Fxyz", + json={"items": [], "total": 0}, + ) + page = await records.list(SCHEMA_ID, status="pending", reference="10.1000/xyz") + assert page.items == [] and page.total == 0 + + +async def test_delete_chunks_at_100(httpx_mock, records): + for _ in range(2): + httpx_mock.add_response( + method="DELETE", + url=__import__("re").compile(rf"{API}/api/v2/schemas/{SCHEMA_ID}/records\?ids=.*"), + status_code=204, + ) + await records.delete(SCHEMA_ID, [f"id-{i}" for i in range(150)]) + reqs = httpx_mock.get_requests() + assert len(reqs) == 2 + assert len(reqs[0].url.params["ids"].split(",")) == 100 + assert len(reqs[1].url.params["ids"].split(",")) == 50 + + +async def test_get_reference_keeps_slashes(httpx_mock, records): + httpx_mock.add_response( + url=f"{API}/api/v2/references/10.1000/j.abc?workspace_id={WS}", + json={"reference": "10.1000/j.abc", "groups": [], "total_records": 0}, + ) + view = await records.get_reference(WS, "10.1000/j.abc") + assert view.reference == "10.1000/j.abc" +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_records_resource.py -v --disable-warnings` +Expected: FAIL — `ImportError: cannot import name 'Records'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/resources/_records.py`: + +```python +from __future__ import annotations + +import asyncio +from typing import Any, Callable, Optional + +from extralit.v2.models import Record, ReferenceView, SearchPage +from extralit.v2.resources._base import ResourceBase + +BULK_UPSERT_MAX_ITEMS = 500 # server cap (RECORDS_BULK_UPSERT_MAX_ITEMS) +DELETE_MAX_IDS = 100 # server cap (DELETE_RECORDS_LIMIT) + + +def _normalize_items(items: Any, reference: Optional[str]) -> list[dict]: + if hasattr(items, "to_dict") and hasattr(items, "columns"): # pandas.DataFrame, kept lazy + items = items.to_dict(orient="records") + normalized = [] + for item in items: + if "fields" in item: + entry = dict(item) + else: + fields = dict(item) + entry = {"fields": fields} + if "reference" in fields: + entry["reference"] = fields.pop("reference") + if reference is not None: + entry.setdefault("reference", reference) + if "reference" not in entry: + raise ValueError("every record needs a reference (per-item or via the reference= argument)") + normalized.append(entry) + return normalized + + +def _normalize_filters(filters: Optional[list]) -> list[dict]: + normalized = [] + for item in filters or []: + if isinstance(item, dict): + normalized.append({"column": item["column"], "op": item["op"], "value": item["value"]}) + else: + column, op, value = item + normalized.append({"column": column, "op": op, "value": value}) + return normalized + + +class Records(ResourceBase): + async def bulk_upsert( + self, + schema_id, + items: Any, + *, + reference: Optional[str] = None, + max_concurrency: int = 4, + on_progress: Optional[Callable] = None, + ) -> list[Record]: + """Idempotent on external_id; metadata is patch-like (omitted keys preserved). + Auto-chunks at the server's 500-item cap; chunks fly concurrently but the + returned list preserves input order.""" + normalized = _normalize_items(items, reference) + chunks = [normalized[i : i + BULK_UPSERT_MAX_ITEMS] for i in range(0, len(normalized), BULK_UPSERT_MAX_ITEMS)] + results: list = [None] * len(chunks) + total = len(normalized) + done = 0 + semaphore = asyncio.Semaphore(max_concurrency) + + async def _run(index: int, chunk: list[dict]) -> None: + nonlocal done + async with semaphore: + payload = await self._transport.request( + "POST", f"/schemas/{schema_id}/records:bulk-upsert", json={"items": chunk} + ) + results[index] = payload["items"] + done += len(chunk) + if on_progress: + on_progress(done, total) + + await asyncio.gather(*(_run(i, c) for i, c in enumerate(chunks))) + return [Record.model_validate(item) for chunk in results for item in chunk] + + async def search( + self, + schema_id, + *, + text: Optional[str] = None, + filters: Optional[list] = None, + offset: int = 0, + limit: int = 50, + ) -> SearchPage: + payload = await self._transport.request( + "POST", + f"/schemas/{schema_id}/records:search", + json={"text": text, "filters": _normalize_filters(filters), "offset": offset, "limit": limit}, + ) + return SearchPage(items=[Record.model_validate(i) for i in payload["items"]], total=payload["total"]) + + async def list( + self, + schema_id, + *, + offset: int = 0, + limit: int = 50, + status: Optional[str] = None, + reference: Optional[str] = None, + ) -> SearchPage: + params: dict = {"offset": offset, "limit": limit} + if status is not None: + params["status"] = status + if reference is not None: + params["reference"] = reference + payload = await self._transport.request("GET", f"/schemas/{schema_id}/records", params=params) + return SearchPage(items=[Record.model_validate(i) for i in payload["items"]], total=payload["total"]) + + async def delete(self, schema_id, ids: list) -> None: + id_strings = [str(record_id) for record_id in ids] + for start in range(0, len(id_strings), DELETE_MAX_IDS): + chunk = id_strings[start : start + DELETE_MAX_IDS] + await self._transport.request( + "DELETE", f"/schemas/{schema_id}/records", params={"ids": ",".join(chunk)} + ) + + async def get_reference(self, workspace_id, reference: str) -> ReferenceView: + # Slashes stay raw: the server route is /references/{reference:path}. + payload = await self._transport.request( + "GET", f"/references/{reference}", params={"workspace_id": str(workspace_id)} + ) + return ReferenceView.model_validate(payload) +``` + +Update `src/extralit/v2/resources/__init__.py`: + +```python +from extralit.v2.resources._questions import Questions +from extralit.v2.resources._records import Records +from extralit.v2.resources._schemas import Schemas + +__all__ = ["Questions", "Records", "Schemas"] +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_records_resource.py -v --disable-warnings` +Expected: 8 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/resources tests/unit/v2/test_records_resource.py +git commit -m "feat(sdk-v2): Records resource with chunked concurrent bulk upsert and search" +``` + +--- + +### Task 8: Suggestions, Projections, and Responses (read-only) resources + +**Files:** +- Create: `src/extralit/v2/resources/_annotation.py`, `src/extralit/v2/resources/_projections.py`, `tests/unit/v2/test_annotation_resources.py` +- Modify: `src/extralit/v2/resources/__init__.py` + +**Interfaces:** +- Consumes: `Questions.id_for` (Task 6), models (Task 4). +- Produces: + - `class Suggestions(ResourceBase)` — `def __init__(self, transport, questions: Questions)`; + - `async def upsert(self, record, question, value, *, score=None, agent=None, type=None, schema_id=None) -> Suggestion` — `record`: a `Record` domain object or a record id; `question`: a question id (UUID/uuid-string) or a question **name**. A name needs a schema to resolve against: taken from `record.schema_id` when a `Record` object was passed, else from `schema_id=`; otherwise `ValueError`. + - `async def list(self, record_id) -> list[Suggestion]` + - `class Responses(ResourceBase)` — `async def get(self, record_id) -> Optional[Response]` — the server returns literal `null` with 200 when no response exists: map to `None`, not an error. + - `class Projections(ResourceBase)` — `async def get(self, workspace_id, reference: str) -> ProjectionView` + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_annotation_resources.py`: + +```python +import json +import uuid +from datetime import datetime, timezone + +import pytest + +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.models import Record +from extralit.v2.resources import Projections, Questions, Responses, Suggestions + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" +SCHEMA_ID = str(uuid.uuid4()) +RECORD_ID = str(uuid.uuid4()) +Q_SIZE = str(uuid.uuid4()) +WS = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _suggestion(): + return { + "id": str(uuid.uuid4()), "record_id": RECORD_ID, "question_id": Q_SIZE, "value": "120", + "score": 0.9, "agent": "claude", "type": "model", "inserted_at": NOW, "updated_at": NOW, + } + + +def _record_obj(): + return Record.model_validate( + { + "id": RECORD_ID, "schema_id": SCHEMA_ID, "schema_version_id": str(uuid.uuid4()), + "reference": "10.1000/xyz", "external_id": None, "fields": {}, "metadata": None, + "status": "pending", "inserted_at": NOW, "updated_at": NOW, + } + ) + + +@pytest.fixture +async def transport(): + t = AsyncTransport(API, api_key="k") + yield t + await t.aclose() + + +async def test_upsert_resolves_question_name_via_record_object(httpx_mock, transport): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", + json={"items": [{ + "id": Q_SIZE, "schema_id": SCHEMA_ID, "name": "size", "title": "Size", "description": None, + "type": "text", "columns": ["size"], "settings": {}, "required": False, + "inserted_at": NOW, "updated_at": NOW, + }]}, + ) + httpx_mock.add_response( + method="PUT", url=f"{API}/api/v2/records/{RECORD_ID}/suggestions", json=_suggestion() + ) + questions = Questions(transport) + suggestions = Suggestions(transport, questions) + result = await suggestions.upsert(_record_obj(), "size", "120", score=0.9, agent="claude") + assert str(result.question_id) == Q_SIZE + body = json.loads(httpx_mock.get_requests()[-1].read()) + assert body["question_id"] == Q_SIZE and body["agent"] == "claude" and body["score"] == 0.9 + + +async def test_upsert_name_without_schema_raises(transport): + suggestions = Suggestions(transport, Questions(transport)) + with pytest.raises(ValueError, match="schema_id"): + await suggestions.upsert(RECORD_ID, "size", "120") + + +async def test_upsert_accepts_question_id_directly(httpx_mock, transport): + httpx_mock.add_response( + method="PUT", url=f"{API}/api/v2/records/{RECORD_ID}/suggestions", json=_suggestion() + ) + suggestions = Suggestions(transport, Questions(transport)) + await suggestions.upsert(RECORD_ID, Q_SIZE, "120") # no questions fetch needed + assert len(httpx_mock.get_requests()) == 1 + + +async def test_response_get_maps_null_to_none(httpx_mock, transport): + httpx_mock.add_response(url=f"{API}/api/v2/records/{RECORD_ID}/responses", json=None) + assert await Responses(transport).get(RECORD_ID) is None + + +async def test_response_get_unwraps_values(httpx_mock, transport): + httpx_mock.add_response( + url=f"{API}/api/v2/records/{RECORD_ID}/responses", + json={ + "id": str(uuid.uuid4()), "record_id": RECORD_ID, "user_id": str(uuid.uuid4()), + "values": {"size": {"value": "135"}}, "status": "submitted", + "inserted_at": NOW, "updated_at": NOW, + }, + ) + response = await Responses(transport).get(RECORD_ID) + assert response.unwrapped_values == {"size": "135"} + + +async def test_projection_get(httpx_mock, transport): + httpx_mock.add_response( + url=f"{API}/api/v2/projection/references/10.1000/j.abc?workspace_id={WS}", + json={ + "reference": "10.1000/j.abc", + "records": [{ + "record_id": RECORD_ID, "schema_id": SCHEMA_ID, "reference": "10.1000/j.abc", + "cells": [{"question_name": "size", "value": "120", "source": "suggestion"}], + }], + "total_records": 1, + }, + ) + view = await Projections(transport).get(WS, "10.1000/j.abc") + assert view.records[0].cells[0].source == "suggestion" +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_annotation_resources.py -v --disable-warnings` +Expected: FAIL — `ImportError` on `Projections`/`Responses`/`Suggestions` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/resources/_annotation.py`: + +```python +from __future__ import annotations + +import uuid +from typing import Any, Optional + +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.models import Response, Suggestion +from extralit.v2.resources._base import ResourceBase +from extralit.v2.resources._questions import Questions + + +def _as_question_id(question: Any) -> Optional[str]: + try: + return str(uuid.UUID(str(question))) + except (ValueError, AttributeError, TypeError): + return None + + +class Suggestions(ResourceBase): + def __init__(self, transport: AsyncTransport, questions: Questions): + super().__init__(transport) + self._questions = questions + + async def upsert( + self, + record: Any, + question: Any, + value: Any, + *, + score: Optional[Any] = None, + agent: Optional[str] = None, + type: Optional[str] = None, + schema_id: Optional[Any] = None, + ) -> Suggestion: + """Upsert one suggestion per (record, question). Suggestions key by question ID on + the wire, but callers may pass a question NAME — resolved via the record's schema.""" + record_id = getattr(record, "id", record) + question_id = _as_question_id(question) + if question_id is None: # a name: resolve against the record's schema + resolve_schema = getattr(record, "schema_id", None) or schema_id + if resolve_schema is None: + raise ValueError( + "resolving a question name requires a Record object or an explicit schema_id=" + ) + question_id = str(await self._questions.id_for(resolve_schema, question)) + body: dict = {"question_id": question_id, "value": value} + if score is not None: + body["score"] = score + if agent is not None: + body["agent"] = agent + if type is not None: + body["type"] = type + payload = await self._transport.request("PUT", f"/records/{record_id}/suggestions", json=body) + return Suggestion.model_validate(payload) + + async def list(self, record_id) -> list[Suggestion]: + payload = await self._transport.request("GET", f"/records/{record_id}/suggestions") + return [Suggestion.model_validate(item) for item in payload["items"]] + + +class Responses(ResourceBase): + async def get(self, record_id) -> Optional[Response]: + """GET returns literal `null` with 200 (not 404) when no response exists yet.""" + payload = await self._transport.request("GET", f"/records/{record_id}/responses") + return None if payload is None else Response.model_validate(payload) +``` + +`src/extralit/v2/resources/_projections.py`: + +```python +from extralit.v2.models import ProjectionView +from extralit.v2.resources._base import ResourceBase + + +class Projections(ResourceBase): + async def get(self, workspace_id, reference: str) -> ProjectionView: + """Response-or-suggestion per question for every record sharing this reference. + Slashes stay raw: the server route is /projection/references/{reference:path}.""" + payload = await self._transport.request( + "GET", f"/projection/references/{reference}", params={"workspace_id": str(workspace_id)} + ) + return ProjectionView.model_validate(payload) +``` + +Update `src/extralit/v2/resources/__init__.py`: + +```python +from extralit.v2.resources._annotation import Responses, Suggestions +from extralit.v2.resources._projections import Projections +from extralit.v2.resources._questions import Questions +from extralit.v2.resources._records import Records +from extralit.v2.resources._schemas import Schemas + +__all__ = ["Projections", "Questions", "Records", "Responses", "Schemas", "Suggestions"] +``` + +Note: `json=None` in the `test_response_get_maps_null_to_none` mock makes httpx return a body of `null` — the transport returns `None` because `response.json()` is `None`? No: the transport checks `not response.content` — a `null` body has content `b"null"`, so `response.json()` returns `None` naturally. Either path yields `None`; the test pins the behavior. + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_annotation_resources.py -v --disable-warnings` +Expected: 6 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/resources tests/unit/v2/test_annotation_resources.py +git commit -m "feat(sdk-v2): Suggestions/Responses/Projections resources with name-id join" +``` + +--- + +### Task 9: AsyncClient assembly with env/credentials fallback + +**Files:** +- Create: `src/extralit/v2/client.py`, `tests/unit/v2/test_client.py` +- Modify: `src/extralit/v2/__init__.py` + +**Interfaces:** +- Consumes: everything above; `extralit.client.login.ExtralitCredentials` (the one allowed v1 import). +- Produces: + - `class AsyncClient` — `def __init__(self, api_url=None, api_key=None, username=None, password=None, timeout=60.0, retries=5)`. Resolution order: explicit args → `EXTRALIT_API_URL`/`EXTRALIT_API_KEY` env → `~/.extralit/credentials.json`. Raises `ValueError` if unresolvable. + - Attributes: `.schemas: Schemas`, `.questions: Questions`, `.records: Records`, `.suggestions: Suggestions`, `.projections: Projections`, `.responses: Responses` (plus `.references` → `records.get_reference` lives on Records; expose `get_reference` through `client.records`). + - `async def aclose()`, `async def __aenter__/__aexit__` + - `extralit.v2.__init__` exports: `AsyncClient` + all names from `models` + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_client.py`: + +```python +import json + +import pytest + +from extralit.v2 import AsyncClient + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" + + +async def test_explicit_args_and_resource_wiring(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w", json={"items": []}) + async with AsyncClient(api_url=API, api_key="k") as client: + assert await client.schemas.list("w") == [] + for name in ("schemas", "questions", "records", "suggestions", "projections", "responses"): + assert hasattr(client, name) + + +async def test_env_fallback(monkeypatch, httpx_mock): + monkeypatch.setenv("EXTRALIT_API_URL", API) + monkeypatch.setenv("EXTRALIT_API_KEY", "env-key") + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w", json={"items": []}) + async with AsyncClient() as client: + await client.schemas.list("w") + assert httpx_mock.get_requests()[0].headers["X-Extralit-Api-Key"] == "env-key" + + +async def test_credentials_file_fallback(monkeypatch, tmp_path): + monkeypatch.delenv("EXTRALIT_API_URL", raising=False) + monkeypatch.delenv("EXTRALIT_API_KEY", raising=False) + creds = tmp_path / "credentials.json" + creds.write_text(json.dumps({"api_url": API, "api_key": "file-key"})) + import extralit.client.login as login_mod + + monkeypatch.setattr(login_mod, "EXTRALIT_CREDENTIALS_FILE", creds) + client = AsyncClient() + assert client._transport._api_key == "file-key" + await client.aclose() + + +async def test_unresolvable_raises(monkeypatch): + monkeypatch.delenv("EXTRALIT_API_URL", raising=False) + monkeypatch.delenv("EXTRALIT_API_KEY", raising=False) + import extralit.client.login as login_mod + + monkeypatch.setattr(login_mod.ExtralitCredentials, "exists", classmethod(lambda cls: False)) + with pytest.raises(ValueError, match="api_url"): + AsyncClient() +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_client.py -v --disable-warnings` +Expected: FAIL — `ImportError: cannot import name 'AsyncClient'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/client.py`: + +```python +from __future__ import annotations + +import os +from typing import Optional + +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.resources import Projections, Questions, Records, Responses, Schemas, Suggestions + + +def _credentials_fallback() -> tuple: + # The single, documented v1 import: the credentials file outlives v1 retirement. + from extralit.client.login import ExtralitCredentials + + if not ExtralitCredentials.exists(): + return None, None + try: + credentials = ExtralitCredentials.load() + return credentials.api_url, credentials.api_key + except (OSError, KeyError, ValueError): + return None, None + + +class AsyncClient: + """Async-native /api/v2 client. Resolution order for connection settings: + explicit args > EXTRALIT_API_URL / EXTRALIT_API_KEY env > ~/.extralit/credentials.json.""" + + def __init__( + self, + api_url: Optional[str] = None, + api_key: Optional[str] = None, + username: Optional[str] = None, + password: Optional[str] = None, + timeout: float = 60.0, + retries: int = 5, + ): + api_url = api_url or os.environ.get("EXTRALIT_API_URL") + api_key = api_key or os.environ.get("EXTRALIT_API_KEY") + if not api_url or (not api_key and not username): + file_url, file_key = _credentials_fallback() + api_url = api_url or file_url + if not api_key and not username: + api_key = file_key + if not api_url: + raise ValueError("api_url is required (argument, EXTRALIT_API_URL, or ~/.extralit/credentials.json)") + if not api_key and not username: + raise ValueError("credentials required: api_key or username/password") + self._transport = AsyncTransport( + api_url, api_key=api_key, username=username, password=password, timeout=timeout, retries=retries + ) + self.schemas = Schemas(self._transport) + self.questions = Questions(self._transport) + self.records = Records(self._transport) + self.suggestions = Suggestions(self._transport, self.questions) + self.projections = Projections(self._transport) + self.responses = Responses(self._transport) + + async def aclose(self) -> None: + await self._transport.aclose() + + async def __aenter__(self) -> "AsyncClient": + return self + + async def __aexit__(self, *exc_info) -> None: + await self.aclose() +``` + +`src/extralit/v2/__init__.py`: + +```python +from extralit.v2.client import AsyncClient +from extralit.v2.models import ( + ProjectionCell, + ProjectionRecord, + ProjectionView, + Question, + Record, + ReferenceGroup, + ReferenceView, + Response, + Schema, + SchemaVersion, + SearchPage, + Suggestion, +) + +__all__ = [ + "AsyncClient", + "ProjectionCell", "ProjectionRecord", "ProjectionView", "Question", "Record", + "ReferenceGroup", "ReferenceView", "Response", "Schema", "SchemaVersion", + "SearchPage", "Suggestion", +] +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_client.py -v --disable-warnings` +Expected: 4 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/client.py src/extralit/v2/__init__.py tests/unit/v2/test_client.py +git commit -m "feat(sdk-v2): AsyncClient assembly with env and credentials-file fallback" +``` + +--- + +### Task 10: Sync facade — portal + mechanical mirrors + +**Files:** +- Create: `src/extralit/v2/_sync.py`, `tests/unit/v2/test_sync_client.py` +- Modify: `src/extralit/v2/__init__.py` + +**Interfaces:** +- Consumes: `AsyncClient` (Task 9), resources. +- Produces: + - `class Client` — same constructor signature as `AsyncClient`; exposes the same resource attributes where every async method is a sync mirror (dispatched to a background-thread event loop, so it works inside Jupyter where a loop is already running). `def close()`, context-manager support. + - Export `Client` from `extralit.v2`. + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_sync_client.py`: + +```python +import asyncio + +from extralit.v2 import Client + +API = "http://test:6900" + + +def test_sync_mirror_calls_through(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w", json={"items": []}) + with Client(api_url=API, api_key="k") as client: + assert client.schemas.list("w") == [] + + +def test_sync_client_works_inside_running_loop(httpx_mock): + """Jupyter simulation: a loop is already running in the calling thread. + asyncio.run-based facades explode here; the portal must not.""" + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w", json={"items": []}) + + async def main(): + with Client(api_url=API, api_key="k") as client: + return client.schemas.list("w") + + assert asyncio.run(main()) == [] + + +def test_non_coroutine_attrs_pass_through(httpx_mock): + with Client(api_url=API, api_key="k") as client: + client.questions.invalidate("some-schema") # sync method on a resource: plain passthrough +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/test_sync_client.py -v --disable-warnings` +Expected: FAIL — `ImportError: cannot import name 'Client'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/_sync.py`: + +```python +from __future__ import annotations + +import asyncio +import functools +import inspect +import threading + +from extralit.v2.client import AsyncClient +from extralit.v2.resources._base import ResourceBase + +_RESOURCE_NAMES = ("schemas", "questions", "records", "suggestions", "projections", "responses") + + +class _Portal: + """A background-thread event loop. Sync mirrors submit coroutines here, so they + work even when the calling thread already runs a loop (Jupyter).""" + + def __init__(self): + self._loop = asyncio.new_event_loop() + self._thread = threading.Thread(target=self._run, name="extralit-v2-portal", daemon=True) + self._thread.start() + + def _run(self) -> None: + asyncio.set_event_loop(self._loop) + self._loop.run_forever() + + def run(self, coroutine): + return asyncio.run_coroutine_threadsafe(coroutine, self._loop).result() + + def stop(self) -> None: + self._loop.call_soon_threadsafe(self._loop.stop) + self._thread.join() + + +class _SyncProxy: + """Wraps a resource: coroutine methods become sync calls through the portal; + everything else passes through unchanged. Mirrors are mechanical — never hand-written.""" + + def __init__(self, target: ResourceBase, portal: _Portal): + self._target = target + self._portal = portal + + def __getattr__(self, name: str): + attribute = getattr(self._target, name) + if inspect.iscoroutinefunction(attribute): + @functools.wraps(attribute) + def call(*args, **kwargs): + return self._portal.run(attribute(*args, **kwargs)) + + return call + return attribute + + +class Client: + """Sync facade over AsyncClient — same constructor, same resource surface.""" + + def __init__(self, *args, **kwargs): + self._portal = _Portal() + self._async = AsyncClient(*args, **kwargs) + for name in _RESOURCE_NAMES: + setattr(self, name, _SyncProxy(getattr(self._async, name), self._portal)) + + def close(self) -> None: + self._portal.run(self._async.aclose()) + self._portal.stop() + + def __enter__(self) -> "Client": + return self + + def __exit__(self, *exc_info) -> None: + self.close() +``` + +Add to `src/extralit/v2/__init__.py` imports/`__all__`: + +```python +from extralit.v2._sync import Client +``` + +(and `"Client"` in `__all__`.) + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/test_sync_client.py -v --disable-warnings` +Expected: 3 PASS. Also rerun the whole v2 suite: `uv run pytest tests/unit/v2 -v --disable-warnings` — all green. + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/_sync.py src/extralit/v2/__init__.py tests/unit/v2/test_sync_client.py +git commit -m "feat(sdk-v2): sync facade via background-thread portal" +``` + +--- + +### Task 11: CLI plumbing — JSON-first output, error contract, client context + +**Files:** +- Create: `src/extralit/v2/cli/__init__.py`, `src/extralit/v2/cli/_output.py`, `src/extralit/v2/cli/_context.py`, `tests/unit/v2/cli/__init__.py`, `tests/unit/v2/cli/test_output.py` + +**Interfaces:** +- Consumes: `Client` (Task 10), error hierarchy (Task 2). +- Produces: + - `_output.emit(data, json_flag: bool) -> None` — pydantic models/lists/dicts → JSON on stdout when `json_flag` **or stdout is not a TTY**; Rich table/pretty otherwise. + - `_output.fail(error: Exception) -> NoReturn` — `{"error": {"type", "status", "detail"}}` JSON on **stderr**; exits 3 for `ValidationError`, 1 otherwise. + - `_output.handle_errors(fn)` — decorator wrapping a command body: catches `V2APIError` → `fail`. + - `_context.get_client() -> Client` — env/credentials-file resolution (delegated to `Client()`); config errors exit 1 with the same stderr JSON shape. + - Exit-code contract: 0 success, 1 API/config error, 2 usage (typer's default), 3 validation. + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/cli/test_output.py`: + +```python +import json + +import pytest +import typer + +from extralit.v2._api._errors import V2APIError, ValidationError +from extralit.v2.cli._output import emit, fail, to_jsonable +from extralit.v2.models import SearchPage + + +def test_to_jsonable_handles_models_lists_dicts(): + page = SearchPage(items=[], total=3) + assert to_jsonable(page) == {"items": [], "total": 3} + assert to_jsonable([page]) == [{"items": [], "total": 3}] + assert to_jsonable({"a": 1}) == {"a": 1} + + +def test_emit_json_when_flag_set(capsys): + emit({"a": 1}, json_flag=True) + assert json.loads(capsys.readouterr().out) == {"a": 1} + + +def test_emit_json_when_not_a_tty(capsys): + emit({"a": 1}, json_flag=False) # pytest capture is not a tty -> auto-JSON + assert json.loads(capsys.readouterr().out) == {"a": 1} + + +def test_fail_validation_exits_3_with_stderr_json(capsys): + with pytest.raises(typer.Exit) as excinfo: + fail(ValidationError(422, "bad value")) + assert excinfo.value.exit_code == 3 + err = json.loads(capsys.readouterr().err) + assert err["error"]["status"] == 422 and err["error"]["type"] == "ValidationError" + + +def test_fail_api_error_exits_1(capsys): + with pytest.raises(typer.Exit) as excinfo: + fail(V2APIError(500, "kaboom")) + assert excinfo.value.exit_code == 1 + assert json.loads(capsys.readouterr().err)["error"]["detail"] == "kaboom" +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/cli/test_output.py -v --disable-warnings` +Expected: FAIL — `ModuleNotFoundError: No module named 'extralit.v2.cli'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/cli/_output.py`: + +```python +from __future__ import annotations + +import functools +import json +import sys +from typing import Any + +import typer + +from extralit.v2._api._errors import V2APIError, ValidationError + + +def to_jsonable(data: Any) -> Any: + if hasattr(data, "model_dump"): + return data.model_dump(mode="json") + if isinstance(data, list): + return [to_jsonable(item) for item in data] + if isinstance(data, dict): + return {key: to_jsonable(value) for key, value in data.items()} + return data + + +def emit(data: Any, json_flag: bool) -> None: + """JSON-first: --json forces JSON; a non-TTY stdout (pipes, agents, CI) defaults to it. + Humans at a terminal get Rich output.""" + if json_flag or not sys.stdout.isatty(): + typer.echo(json.dumps(to_jsonable(data), default=str)) + return + from rich.console import Console # lazy: JSON path must not pay for rich + + Console().print(to_jsonable(data)) + + +def fail(error: Exception) -> None: + status = getattr(error, "status_code", None) + payload = {"error": {"type": type(error).__name__, "status": status, "detail": str(getattr(error, "detail", error))}} + typer.echo(json.dumps(payload, default=str), err=True) + raise typer.Exit(code=3 if isinstance(error, ValidationError) else 1) + + +def handle_errors(fn): + @functools.wraps(fn) + def wrapper(*args, **kwargs): + try: + return fn(*args, **kwargs) + except V2APIError as error: + fail(error) + + return wrapper +``` + +`src/extralit/v2/cli/_context.py`: + +```python +from __future__ import annotations + +from extralit.v2._sync import Client +from extralit.v2.cli._output import fail + + +def get_client() -> Client: + """Non-interactive by construction: args come from env or the credentials file; + a missing configuration is a structured error, never a prompt.""" + try: + return Client() + except ValueError as error: + fail(error) + raise # unreachable; keeps type-checkers happy +``` + +`src/extralit/v2/cli/__init__.py` (placeholder for now; Task 12 fills in `add_v2_commands`): + +```python +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/cli/test_output.py -v --disable-warnings` +Expected: 5 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/cli tests/unit/v2/cli +git commit -m "feat(sdk-v2): CLI JSON-first output helpers and error contract" +``` + +--- + +### Task 12: CLI `schemas` verbs + top-level takeover in app.py + +**Files:** +- Create: `src/extralit/v2/cli/schemas.py`, `tests/unit/v2/cli/test_cli_schemas.py` +- Modify: `src/extralit/v2/cli/__init__.py`, `src/extralit/cli/app.py` + +**Interfaces:** +- Consumes: `get_client`, `emit`, `handle_errors` (Task 11); `Client.schemas` methods (Tasks 5/10). +- Produces: + - `extralit.v2.cli.schemas.app` — typer app with commands `list`, `get`, `create`, `publish`, `versions`. + - `extralit.v2.cli.add_v2_commands(app: typer.Typer) -> None` — registers all v2 verb groups at top level (this task: `schemas`; Task 13/14 extend it). + - `cli/app.py` no longer registers the v1 `schemas` subcommand; it calls `add_v2_commands(app)` instead. + +- [ ] **Step 1: Check v1 schemas CLI is safe to unregister** + +```bash +grep -rn "cli.schemas\|cli import schemas\|from extralit.cli.schemas" src/ tests/ --include="*.py" | grep -v "src/extralit/cli/schemas/" +``` + +Expected: hits only in `src/extralit/cli/app.py` and old tests (`tests/unit/cli/test_cli_schemas.py`). If another module (e.g. `cli/extraction`) imports from `extralit.cli.schemas`, keep the module files (we only unregister the subcommand) — that is the default posture anyway: **unregister, don't delete**. + +- [ ] **Step 2: Write the failing tests** + +`tests/unit/v2/cli/test_cli_schemas.py`: + +```python +import json +import uuid +from datetime import datetime, timezone +from types import SimpleNamespace + +import pytest +from typer.testing import CliRunner + +import extralit.v2.cli._context as context_mod +from extralit.v2._api._errors import ValidationError +from extralit.v2.cli.schemas import app +from extralit.v2.models import Schema + +runner = CliRunner() # click >= 8.2: stderr is separated by default (mix_stderr was removed) +WS = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _schema(name="trials"): + return Schema.model_validate( + { + "id": str(uuid.uuid4()), "name": name, "status": "draft", "current_version_id": None, + "settings": {}, "workspace_id": WS, "inserted_at": NOW, "updated_at": NOW, + } + ) + + +class FakeClient(SimpleNamespace): + def __enter__(self): + return self + + def __exit__(self, *exc_info): + pass + + +@pytest.fixture +def fake_client(monkeypatch): + client = FakeClient(schemas=SimpleNamespace(list=lambda workspace_id: [_schema()])) + monkeypatch.setattr(context_mod, "get_client", lambda: client) + # schemas.py imports get_client at call time via the module attribute: + import extralit.v2.cli.schemas as schemas_mod + + monkeypatch.setattr(schemas_mod, "get_client", lambda: client) + return client + + +def test_list_emits_json(fake_client): + result = runner.invoke(app, ["list", "--workspace-id", WS, "--json"]) + assert result.exit_code == 0 + items = json.loads(result.stdout) + assert items[0]["name"] == "trials" + + +def test_validation_error_exits_3(fake_client): + def boom(workspace_id): + raise ValidationError(422, "bad") + + fake_client.schemas.list = boom + result = runner.invoke(app, ["list", "--workspace-id", WS, "--json"]) + assert result.exit_code == 3 + assert json.loads(result.stderr)["error"]["type"] == "ValidationError" + + +def test_top_level_registration(): + from extralit.cli.app import app as root_app + + names = [t.name for t in root_app.registered_groups] + # Task 14 Step 4 extends this tuple to all six verbs once they exist: + # ("schemas", "records", "questions", "suggestions", "projection", "references") + for verb in ("schemas",): + assert verb in names, f"{verb} not registered at top level" + assert names.count("schemas") == 1, "v1 schemas subcommand must be unregistered" +``` + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/cli/test_cli_schemas.py -v --disable-warnings` +Expected: FAIL — `ModuleNotFoundError: No module named 'extralit.v2.cli.schemas'` + +- [ ] **Step 4: Implement** + +`src/extralit/v2/cli/schemas.py`: + +```python +from __future__ import annotations + +import json +from pathlib import Path +from typing import Optional +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Manage v2 schemas (Pandera, versioned)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("list") +@handle_errors +def list_schemas( + workspace_id: str = typer.Option(..., "--workspace-id", help="Workspace UUID"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit(client.schemas.list(UUID(workspace_id)), json_flag) + + +@app.command("get") +@handle_errors +def get_schema(schema_id: str = typer.Argument(...), json_flag: bool = JSON_FLAG): + with get_client() as client: + emit(client.schemas.get(UUID(schema_id)), json_flag) + + +@app.command("create") +@handle_errors +def create_schema( + name: str = typer.Argument(...), + workspace_id: str = typer.Option(..., "--workspace-id"), + settings: Optional[str] = typer.Option(None, "--settings", help="Settings as a JSON object"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit( + client.schemas.create(UUID(workspace_id), name, settings=json.loads(settings) if settings else None), + json_flag, + ) + + +@app.command("publish") +@handle_errors +def publish_version( + schema_id: str = typer.Argument(...), + file: Path = typer.Option(..., "--file", help="Pandera DataFrameSchema JSON (schema.to_json() output)"), + review_widgets: Optional[str] = typer.Option(None, "--review-widgets", help="JSON: {column: widget config}"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit( + client.schemas.publish( + UUID(schema_id), + file.read_text(), + review_widgets=json.loads(review_widgets) if review_widgets else None, + ), + json_flag, + ) + + +@app.command("versions") +@handle_errors +def list_versions(schema_id: str = typer.Argument(...), json_flag: bool = JSON_FLAG): + with get_client() as client: + emit(client.schemas.versions(UUID(schema_id)), json_flag) +``` + +`src/extralit/v2/cli/__init__.py`: + +```python +from __future__ import annotations + +import typer + + +def add_v2_commands(app: typer.Typer) -> None: + """Register v2 verbs at the TOP level of the extralit CLI (no `v2` prefix). + v2 owns these names; the v1 `schemas` subcommand is deliberately replaced.""" + from extralit.v2.cli import schemas + + app.add_typer(schemas.app, name="schemas") +``` + +Modify `src/extralit/cli/app.py` — remove `schemas` from the v1 import list and its registration, add the v2 call: + +```python +# In the import block at the top, DELETE the `schemas,` line: +from extralit.cli import ( + datasets, + documents, + extraction, + files, + info, + login, + logout, + training, + users, + whoami, + workflows, + workspaces, +) + + +# In register_subcommands(), DELETE `app.add_typer(schemas.app, name="schemas")` and append: +def register_subcommands(): + app.add_typer(datasets.app, name="datasets") + app.add_typer(documents.app, name="documents") + app.add_typer(extraction.app, name="extraction") + app.add_typer(files.app, name="files", hidden=True) + app.add_typer(info.app, name="info") + app.add_typer(login.app, name="login") + app.add_typer(logout.app, name="logout") + app.add_typer(training.app, name="training") + app.add_typer(users.app, name="users") + app.add_typer(whoami.app, name="whoami") + app.add_typer(workflows.app, name="workflows") + app.add_typer(workspaces.app, name="workspaces") + + # v2 owns the top-level verbs: schemas, records, questions, suggestions, projection, references. + from extralit.v2.cli import add_v2_commands # composition-root exception to the v1/v2 wall + + add_v2_commands(app) +``` + +Also delete or update the old v1 CLI test that asserts the v1 schemas commands exist (`tests/unit/cli/test_cli_schemas.py`): delete the file — the v1 subcommand is retired by design (spec §5). + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/cli/test_cli_schemas.py tests/unit/cli -v --disable-warnings` +Expected: new tests PASS; remaining v1 CLI tests PASS (no schemas registration asserted anywhere). + +- [ ] **Step 6: Commit** + +```bash +git add src/extralit/v2/cli src/extralit/cli/app.py tests/unit/v2/cli tests/unit/cli +git commit -m "feat(sdk-v2): top-level schemas CLI verbs replace v1 schemas subcommand" +``` + +--- + +### Task 13: CLI `records` and `questions` verbs (JSONL stdin piping) + +**Files:** +- Create: `src/extralit/v2/cli/records.py`, `src/extralit/v2/cli/questions.py`, `tests/unit/v2/cli/test_cli_records.py` +- Modify: `src/extralit/v2/cli/__init__.py` + +**Interfaces:** +- Consumes: Task 11 plumbing; `Client.records`/`Client.questions`. +- Produces: + - `records` verbs: `upsert SCHEMA_ID [--file PATH|-] [--reference R]` (reads JSONL: one item per line), `search SCHEMA_ID [--text T] [--filter col:op:value ...] [--offset N] [--limit N]`, `list SCHEMA_ID [--status S] [--reference R] [--offset N] [--limit N]`, `delete SCHEMA_ID --ids id1,id2,...` + - `questions` verbs: `list SCHEMA_ID` + - `--filter` value syntax: `column:op:value` where `value` is JSON-decoded when possible (so `age:ge:18` → int 18; `label:in:["a","b"]` → list) + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/cli/test_cli_records.py`: + +```python +import json +import uuid +from datetime import datetime, timezone +from types import SimpleNamespace + +import pytest +from typer.testing import CliRunner + +from extralit.v2.cli.records import _parse_filter, app +from extralit.v2.models import Record, SearchPage + +runner = CliRunner() # click >= 8.2: stderr is separated by default (mix_stderr was removed) +SCHEMA_ID = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _record(): + return Record.model_validate( + { + "id": str(uuid.uuid4()), "schema_id": SCHEMA_ID, "schema_version_id": str(uuid.uuid4()), + "reference": "10.1000/xyz", "external_id": None, "fields": {"size": "120"}, + "metadata": None, "status": "pending", "inserted_at": NOW, "updated_at": NOW, + } + ) + + +class FakeClient(SimpleNamespace): + def __enter__(self): + return self + + def __exit__(self, *exc_info): + pass + + +@pytest.fixture +def fake_client(monkeypatch): + calls = {} + + def upsert(schema_id, items, reference=None): + calls["upsert"] = (schema_id, items, reference) + return [_record()] + + def search(schema_id, text=None, filters=None, offset=0, limit=50): + calls["search"] = (schema_id, text, filters, offset, limit) + return SearchPage(items=[_record()], total=1) + + client = FakeClient(records=SimpleNamespace(bulk_upsert=upsert, search=search)) + import extralit.v2.cli.records as records_mod + + monkeypatch.setattr(records_mod, "get_client", lambda: client) + return calls + + +def test_parse_filter_json_decodes_value(): + assert _parse_filter("age:ge:18") == ("age", "ge", 18) + assert _parse_filter('label:in:["a","b"]') == ("label", "in", ["a", "b"]) + assert _parse_filter("country:eq:KE") == ("country", "eq", "KE") + + +def test_upsert_reads_jsonl_from_stdin(fake_client): + lines = '{"size": "120"}\n{"size": "135"}\n' + result = runner.invoke(app, ["upsert", SCHEMA_ID, "--reference", "10.1000/xyz"], input=lines) + assert result.exit_code == 0, result.stderr + schema_id, items, reference = fake_client["upsert"] + assert items == [{"size": "120"}, {"size": "135"}] + assert reference == "10.1000/xyz" + assert json.loads(result.stdout)[0]["fields"] == {"size": "120"} + + +def test_search_passes_filters(fake_client): + result = runner.invoke( + app, ["search", SCHEMA_ID, "--text", "tumor", "--filter", "age:ge:18", "--limit", "10"] + ) + assert result.exit_code == 0 + _, text, filters, _, limit = fake_client["search"] + assert text == "tumor" and filters == [("age", "ge", 18)] and limit == 10 + assert json.loads(result.stdout)["total"] == 1 +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/cli/test_cli_records.py -v --disable-warnings` +Expected: FAIL — `ModuleNotFoundError: No module named 'extralit.v2.cli.records'` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/cli/records.py`: + +```python +from __future__ import annotations + +import json +import sys +from typing import Optional +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Manage v2 records (schema-version-pinned, reference-keyed)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +def _parse_filter(raw: str) -> tuple: + """col:op:value — value is JSON-decoded when possible ('age:ge:18' -> int 18).""" + column, op, value = raw.split(":", 2) + try: + value = json.loads(value) + except ValueError: + pass # keep as string + return (column, op, value) + + +def _read_jsonl(file: Optional[str]) -> list[dict]: + stream = sys.stdin if file in (None, "-") else open(file) + try: + return [json.loads(line) for line in stream if line.strip()] + finally: + if stream is not sys.stdin: + stream.close() + + +@app.command("upsert") +@handle_errors +def upsert_records( + schema_id: str = typer.Argument(...), + file: Optional[str] = typer.Option(None, "--file", help="JSONL file of items; '-' or omitted reads stdin"), + reference: Optional[str] = typer.Option(None, "--reference", help="Reference applied to items lacking one"), + json_flag: bool = JSON_FLAG, +): + items = _read_jsonl(file) + with get_client() as client: + emit(client.records.bulk_upsert(UUID(schema_id), items, reference=reference), json_flag) + + +@app.command("search") +@handle_errors +def search_records( + schema_id: str = typer.Argument(...), + text: Optional[str] = typer.Option(None, "--text"), + filters: list[str] = typer.Option([], "--filter", help="col:op:value (op: eq|in|ge|le); repeatable"), + offset: int = typer.Option(0, "--offset"), + limit: int = typer.Option(50, "--limit"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit( + client.records.search( + UUID(schema_id), + text=text, + filters=[_parse_filter(raw) for raw in filters], + offset=offset, + limit=limit, + ), + json_flag, + ) + + +@app.command("list") +@handle_errors +def list_records( + schema_id: str = typer.Argument(...), + status: Optional[str] = typer.Option(None, "--status"), + reference: Optional[str] = typer.Option(None, "--reference"), + offset: int = typer.Option(0, "--offset"), + limit: int = typer.Option(50, "--limit"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit( + client.records.list(UUID(schema_id), status=status, reference=reference, offset=offset, limit=limit), + json_flag, + ) + + +@app.command("delete") +@handle_errors +def delete_records( + schema_id: str = typer.Argument(...), + ids: str = typer.Option(..., "--ids", help="Comma-separated record ids"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + client.records.delete(UUID(schema_id), ids.split(",")) + emit({"deleted": len(ids.split(","))}, json_flag) +``` + +`src/extralit/v2/cli/questions.py`: + +```python +from __future__ import annotations + +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Inspect v2 questions (column-bound)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("list") +@handle_errors +def list_questions(schema_id: str = typer.Argument(...), json_flag: bool = JSON_FLAG): + with get_client() as client: + emit(client.questions.list(UUID(schema_id)), json_flag) +``` + +Extend `src/extralit/v2/cli/__init__.py` `add_v2_commands`: + +```python + from extralit.v2.cli import questions, records, schemas + + app.add_typer(schemas.app, name="schemas") + app.add_typer(records.app, name="records") + app.add_typer(questions.app, name="questions") +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/cli/test_cli_records.py -v --disable-warnings` +Expected: 3 PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/extralit/v2/cli tests/unit/v2/cli/test_cli_records.py +git commit -m "feat(sdk-v2): records/questions CLI verbs with JSONL stdin piping" +``` + +--- + +### Task 14: CLI `suggestions`, `projection`, `references` verbs + +**Files:** +- Create: `src/extralit/v2/cli/suggestions.py`, `src/extralit/v2/cli/projection.py`, `src/extralit/v2/cli/references.py`, `tests/unit/v2/cli/test_cli_annotation.py` +- Modify: `src/extralit/v2/cli/__init__.py`, `tests/unit/v2/cli/test_cli_schemas.py` (extend registration assertion) + +**Interfaces:** +- Consumes: Task 11 plumbing; `Client.suggestions`/`Client.projections`/`Client.records.get_reference`. +- Produces: + - `suggestions upsert RECORD_ID (--question-id UUID | --question NAME --schema-id UUID) --value JSON [--score F] [--agent A]` + - `projection get REFERENCE --workspace-id UUID` + - `references get REFERENCE --workspace-id UUID` + - Final `add_v2_commands` registering all six verb groups. + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/cli/test_cli_annotation.py`: + +```python +import json +import uuid +from types import SimpleNamespace + +import pytest +from typer.testing import CliRunner + +from extralit.v2.cli.projection import app as projection_app +from extralit.v2.cli.suggestions import app as suggestions_app +from extralit.v2.models import ProjectionView + +runner = CliRunner() # click >= 8.2: stderr is separated by default (mix_stderr was removed) +RECORD_ID = str(uuid.uuid4()) +Q_ID = str(uuid.uuid4()) +WS = str(uuid.uuid4()) + + +class FakeClient(SimpleNamespace): + def __enter__(self): + return self + + def __exit__(self, *exc_info): + pass + + +def test_suggestions_upsert_decodes_json_value(monkeypatch): + calls = {} + + def upsert(record, question, value, score=None, agent=None, schema_id=None): + calls["args"] = (record, question, value, score, agent, schema_id) + return SimpleNamespace(model_dump=lambda mode: {"ok": True}) + + client = FakeClient(suggestions=SimpleNamespace(upsert=upsert)) + import extralit.v2.cli.suggestions as mod + + monkeypatch.setattr(mod, "get_client", lambda: client) + result = runner.invoke( + suggestions_app, + ["upsert", RECORD_ID, "--question-id", Q_ID, "--value", '"120"', "--score", "0.9", "--agent", "claude"], + ) + assert result.exit_code == 0, result.stderr + record, question, value, score, agent, schema_id = calls["args"] + assert (record, question, value, score, agent) == (RECORD_ID, Q_ID, "120", 0.9, "claude") + + +def test_suggestions_upsert_name_requires_schema_id(monkeypatch): + import extralit.v2.cli.suggestions as mod + + monkeypatch.setattr(mod, "get_client", lambda: FakeClient(suggestions=SimpleNamespace())) + result = runner.invoke(suggestions_app, ["upsert", RECORD_ID, "--question", "size", "--value", '"x"']) + assert result.exit_code == 2 # usage error: --question needs --schema-id + + +def test_projection_get(monkeypatch): + view = ProjectionView(reference="10.1000/j.abc", records=[], total_records=0) + client = FakeClient(projections=SimpleNamespace(get=lambda ws, ref: view)) + import extralit.v2.cli.projection as mod + + monkeypatch.setattr(mod, "get_client", lambda: client) + result = runner.invoke(projection_app, ["get", "10.1000/j.abc", "--workspace-id", WS]) + assert result.exit_code == 0 + assert json.loads(result.stdout)["reference"] == "10.1000/j.abc" +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `uv run pytest tests/unit/v2/cli/test_cli_annotation.py -v --disable-warnings` +Expected: FAIL — `ModuleNotFoundError` + +- [ ] **Step 3: Implement** + +`src/extralit/v2/cli/suggestions.py`: + +```python +from __future__ import annotations + +import json +from typing import Optional +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Write v2 suggestions (per record x question)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("upsert") +@handle_errors +def upsert_suggestion( + record_id: str = typer.Argument(...), + question_id: Optional[str] = typer.Option(None, "--question-id", help="Question UUID"), + question: Optional[str] = typer.Option(None, "--question", help="Question NAME (needs --schema-id)"), + schema_id: Optional[str] = typer.Option(None, "--schema-id", help="Schema UUID for name resolution"), + value: str = typer.Option(..., "--value", help="Suggested value as JSON (e.g. '\"120\"' or '[1,2]')"), + score: Optional[float] = typer.Option(None, "--score"), + agent: Optional[str] = typer.Option(None, "--agent"), + json_flag: bool = JSON_FLAG, +): + if question_id is None and question is None: + raise typer.BadParameter("pass --question-id or --question") + if question is not None and question_id is None and schema_id is None: + raise typer.BadParameter("--question (a name) requires --schema-id to resolve against") + with get_client() as client: + emit( + client.suggestions.upsert( + record_id, + question_id or question, + json.loads(value), + score=score, + agent=agent, + schema_id=UUID(schema_id) if schema_id else None, + ), + json_flag, + ) +``` + +`src/extralit/v2/cli/projection.py`: + +```python +from __future__ import annotations + +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Read v2 projections (response-or-suggestion per question)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("get") +@handle_errors +def get_projection( + reference: str = typer.Argument(..., help="Reference (DOI/URL/filename; slashes fine)"), + workspace_id: str = typer.Option(..., "--workspace-id"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit(client.projections.get(UUID(workspace_id), reference), json_flag) +``` + +`src/extralit/v2/cli/references.py`: + +```python +from __future__ import annotations + +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Cross-schema reference views", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("get") +@handle_errors +def get_reference( + reference: str = typer.Argument(..., help="Reference (DOI/URL/filename; slashes fine)"), + workspace_id: str = typer.Option(..., "--workspace-id"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit(client.records.get_reference(UUID(workspace_id), reference), json_flag) +``` + +Final `src/extralit/v2/cli/__init__.py`: + +```python +from __future__ import annotations + +import typer + + +def add_v2_commands(app: typer.Typer) -> None: + """Register v2 verbs at the TOP level of the extralit CLI (no `v2` prefix). + v2 owns these names; the v1 `schemas` subcommand is deliberately replaced.""" + from extralit.v2.cli import projection, questions, records, references, schemas, suggestions + + app.add_typer(schemas.app, name="schemas") + app.add_typer(records.app, name="records") + app.add_typer(questions.app, name="questions") + app.add_typer(suggestions.app, name="suggestions") + app.add_typer(projection.app, name="projection") + app.add_typer(references.app, name="references") +``` + +- [ ] **Step 4: Extend the registration assertion in `tests/unit/v2/cli/test_cli_schemas.py`** + +Update `test_top_level_registration` to loop over all six verbs: `("schemas", "records", "questions", "suggestions", "projection", "references")`. + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `uv run pytest tests/unit/v2/cli -v --disable-warnings` +Expected: all PASS, including the six-verb registration assertion. + +- [ ] **Step 6: Commit** + +```bash +git add src/extralit/v2/cli tests/unit/v2/cli +git commit -m "feat(sdk-v2): suggestions/projection/references CLI verbs complete the verb set" +``` + +--- + +### Task 15: Boundary, lazy-import, and startup gates + docs + +**Files:** +- Create: `tests/unit/v2/test_boundaries.py` +- Modify: `extralit/CLAUDE.md` (component docs) + +**Interfaces:** +- Consumes: the whole v2 package. +- Produces: CI-enforced guarantees — import wall, no heavy imports at v2 import time, CLI startup sanity. + +- [ ] **Step 1: Write the failing tests** + +`tests/unit/v2/test_boundaries.py`: + +```python +import re +import subprocess +import sys +from pathlib import Path + +SRC = Path(__file__).parents[3] / "src" / "extralit" +V2 = SRC / "v2" + +# The single allowed v1 import inside v2 (credentials helper outlives v1 retirement). +ALLOWED_V1_IMPORT = "extralit.client.login" +V1_IMPORT = re.compile(r"^\s*(?:from|import)\s+(extralit\.(?!v2\b)[\w.]*)", re.MULTILINE) +V2_IMPORT = re.compile(r"^\s*(?:from|import)\s+extralit\.v2[\w.]*", re.MULTILINE) + + +def test_v2_imports_no_v1_except_credentials(): + violations = [] + for path in V2.rglob("*.py"): + if path.name == "_generated.py": + continue + for match in V1_IMPORT.finditer(path.read_text()): + if match.group(1) != ALLOWED_V1_IMPORT: + violations.append(f"{path.relative_to(SRC)}: {match.group(0).strip()}") + assert not violations, f"v2 -> v1 imports outside the credentials exception:\n" + "\n".join(violations) + + +def test_v1_never_imports_v2_except_composition_root(): + violations = [] + for path in SRC.rglob("*.py"): + if V2 in path.parents or path == SRC / "cli" / "app.py": + continue + if V2_IMPORT.search(path.read_text()): + violations.append(str(path.relative_to(SRC))) + assert not violations, f"v1 files importing v2 (only cli/app.py may):\n" + "\n".join(violations) + + +HEAVY = ("pandas", "pandera", "datasets", "huggingface_hub") + + +def _heavy_after(imports: str) -> set: + code = ( + f"import sys; {imports}; " + f"print(','.join(sorted(m for m in {HEAVY!r} if m in sys.modules)))" + ) + proc = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True, check=True) + return set(filter(None, proc.stdout.strip().split(","))) + + +def test_v2_adds_no_heavy_imports(): + """Agents make many short CLI calls: importing extralit.v2 (incl. cli) must not add + heavy modules beyond what `import extralit` (the v1 package init) already drags in. + Delta-based on purpose: v1's own import weight is Phase 6 scope, measured at the + baseline (today: datasets + huggingface_hub via extralit/__init__.py).""" + baseline = _heavy_after("import extralit") + with_v2 = _heavy_after("import extralit; import extralit.v2; import extralit.v2.cli") + added = with_v2 - baseline + assert not added, f"extralit.v2 import added heavy modules: {sorted(added)}" +``` + +- [ ] **Step 2: Run tests** + +Run: `uv run pytest tests/unit/v2/test_boundaries.py -v --disable-warnings` +Expected: all 3 PASS immediately if Tasks 1–14 held the line; any failure names the violating file — fix it (usually a top-level heavy import that belongs inside a function). + +- [ ] **Step 3: Measure CLI startup and record the number** + +```bash +time uv run extralit schemas --help +uv run python -X importtime -c "import extralit.v2.cli" 2>&1 | tail -5 +``` + +The spec budget is < 300 ms for the v2 path. Note: `extralit.cli.app` still imports the v1 modules eagerly — if total startup exceeds the budget because of *v1* imports, record the measured split in the commit message; the v2 side must stay clean (the `test_v2_import_is_light` gate), and v1 startup is Phase 6 scope. + +- [ ] **Step 4: Full verification** + +```bash +uv run pytest tests/unit -v --disable-warnings +uv run ruff check +uv run ruff format --check +``` + +Expected: all green. + +- [ ] **Step 5: Update component docs** + +Append to `extralit/CLAUDE.md` a short v2 section: + +```markdown +## v2 SDK (`src/extralit/v2/`) + +- Parallel package for `/api/v2` (schema-centric). Import wall: v2 imports nothing from v1 + except `extralit.client.login`; only `cli/app.py` imports v2 (composition root). +- Wire types are GENERATED: `_api/openapi.json` (server `openapi-dump` snapshot) -> + `_api/_generated.py` via datamodel-codegen. Never hand-edit; regenerate with the command + in `tests/unit/v2/test_contract.py` and keep both in sync (drift-gated). +- `AsyncClient` is the real client; `Client` is a mechanical sync facade (background-thread + portal — works in Jupyter). CLI verbs register at TOP level (`extralit schemas|records|...`), + JSON-first (`--json` or non-TTY), errors as JSON on stderr (exit 0/1/2/3). +``` + +- [ ] **Step 6: Commit** + +```bash +git add tests/unit/v2/test_boundaries.py CLAUDE.md +git commit -m "test(sdk-v2): import-wall, lazy-import, and startup gates + docs" +``` + +--- + +## Deferred (from spec — do NOT implement in this plan) + +Responses write (submit/draft/discard), DataFrame/parquet/HF export, `rebuild-index`/admin verbs, webhooks, span questions, v1 retirement of the remaining subcommands, live-stack integration test (needs seeded backend — reuse `extralit-frontend/e2e/v2/seed/` approach in a follow-up). diff --git a/docs/superpowers/specs/2026-07-13-sdk-v2-redesign-design.md b/docs/superpowers/specs/2026-07-13-sdk-v2-redesign-design.md new file mode 100644 index 000000000..4f073f3b2 --- /dev/null +++ b/docs/superpowers/specs/2026-07-13-sdk-v2-redesign-design.md @@ -0,0 +1,223 @@ +# Python SDK v2 — schema-centric client, agentic CLI, async performance + +**Date:** 2026-07-13 +**Status:** Approved design, pending implementation plan +**Companions:** `2026-06-27-schema-centric-data-model-design.md` (server model), +`2026-07-09-v2-frontend-vertical-slice-design.md` (frontend precedent this design mirrors) + +## Problem + +The backend grew a second data model (`/api/v2`: schema-centric, versioned, +projection-based) alongside the Argilla-lineage one (`/api/v1`). The Python SDK +(`extralit/`, v0.6.1) is hardcoded to `/api/v1`: sync-only `httpx.Client`, Argilla +entities (Dataset → Record → Question → Response), and a human-oriented Rich CLI. +It cannot drive the v2 extraction loop at all, and three of its properties block the +intended consumers: + +1. **Wrong model.** No client for schemas/versions, version-pinned records, + column-bound questions, reference grouping, or projections. +2. **Not agentic.** CLI output is Rich tables only — unparseable by agents; no + stable JSON, no meaningful exit-code contract, heavy imports make every + short-lived invocation slow. +3. **Sequential I/O.** All HTTP is blocking and serial; pushing large record sets + or fanning out per-record reads requires user-written loops. + +## Decision summary + +| Decision | Choice | +|---|---| +| Coexistence | Parallel `extralit.v2` package; v1 untouched; one wheel | +| Contract | Committed `openapi-dump` snapshot → generated Pydantic DTOs (Approach A) | +| Transport | Async-native (`httpx.AsyncClient`) + sync facade | +| CLI | Top-level verbs (`extralit schemas`, `extralit records`, …) — v2 owns the CLI surface; JSON-first, lazy-imported | +| Primary workflow | LLM extraction pipeline (publish schema → bulk-upsert → suggestions → projections) | +| Scope | Vertical slice (below); responses-write, exports, admin verbs deferred | + +Alternatives rejected: **evolve-in-place** (recreates the name-collision / +entity-bending trap the frontend explicitly avoided — a v2 `Record` is not a v1 +`Record`); **fully generated client** via `openapi-python-client` (generated +ergonomics fight the agentic-CLI and pipeline goals on every regen); **hand-written +wire models** (silent drift against a quirky contract we already have a dump tool for). + +## 1. Package architecture + +``` +src/extralit/v2/ +├── __init__.py # exports AsyncClient, Client, domain classes +├── _api/ +│ ├── openapi.json # committed snapshot (server `openapi-dump`) +│ ├── _generated.py # datamodel-code-generator → Pydantic v2 DTOs +│ ├── _transport.py # httpx.AsyncClient wrapper: bearer auth + refresh, retries +│ └── _errors.py # error hierarchy + 422 normalizer +├── _sync.py # sync facade (background-thread event-loop portal) +├── models/ # hand-written domain classes (Schema, SchemaVersion, Record, …) +├── resources/ # Schemas, Records, Questions, Suggestions, Projections +└── cli/ # top-level CLI verbs (lazy-registered into cli/app.py) +``` + +Python namespacing removes the frontend's need for `V2`-prefixed names: +`extralit.v2.Record` cannot collide with `extralit.Record`. Entry points are +explicit — `from extralit.v2 import Client` (sync) / `AsyncClient` — never re-exported +from the top-level `extralit` namespace. + +**Import wall (reuse-don't-fork, with a hard boundary).** `extralit.v2` imports +nothing from v1 modules, with one documented exception: a shared credentials helper +(env vars + `~/.extralit/credentials.json`), which survives v1 retirement anyway. +v1 never imports v2. The boundary is grep-checkable and gated in CI. Phase 6 +retirement deletes v1 packages mechanically. `cli/app.py` is the composition root +(like the frontend's `plugins/3.di.ts`): it may import `extralit.v2.cli` to register +commands without counting as a boundary violation. + +## 2. Contract layer + +`_api/openapi.json` is a checked-in dump of the server's v2 OpenAPI schema, produced +by the existing server CLI (`uv run python -m extralit_server openapi-dump`). +`datamodel-code-generator` (dev dependency, `--output-model-type pydantic_v2.BaseModel`) +turns it into `_generated.py`. Resources type every wire payload from generated DTOs; +hand-written shapes never touch the wire. + +Two CI gates, mirroring the frontend's: + +- **No-drift:** regenerate from the committed snapshot and diff; any change fails. +- **Snapshot-vs-server:** the committed snapshot must match a fresh `openapi-dump` + from the server tree at the same monorepo revision. + +## 3. Transport & performance + +- **Async-native.** One `httpx.AsyncClient` per client instance: pooled connections, + transport `retries=5`, 60s default timeout — matching v1's proven settings. +- **Auth.** v2 routes accept both backends (verified: `security/authentication/ + provider.py` registers `APIKeyAuthenticationBackend` and + `BearerTokenAuthenticationBackend` on `get_current_user`). The SDK default is the + API-key header — same credentials as v1, no token lifecycle. Bearer JWT via + `POST /api/v2/token` (username/password, 30-min access / 30-day refresh) is also + supported; the transport transparently refreshes once on 401, then re-raises. +- **Sync facade.** `Client` mirrors every `AsyncClient` method by dispatching to a + background-thread event loop (a "portal"), not `asyncio.run` — so sync usage works + inside Jupyter, where a loop is already running. Sync mirrors are generated + mechanically from the async resource surface, not hand-maintained. +- **Bulk pipeline.** `records.bulk_upsert()` auto-chunks at the server's 500-item + cap and dispatches chunks with bounded concurrency (default semaphore of 4, + configurable) plus an optional progress callback. Deletes chunk at the 100-id cap. + Pushing 50k records is one SDK call. +- **Immutable-data caching.** Schema versions are immutable → cached in-client by + `(schema_id, version)`. Question name↔id maps cached per schema, invalidated on + question writes through the same client. + +## 4. Domain resources — the extraction loop + +The resource layer is the anti-corruption layer: every wire quirk lives here once, +so nothing above it knows about `snake_case` bodies, double-wrapping, or id-vs-name +keying. + +```python +async with AsyncClient(api_url=..., api_key=...) as xl: + schema = await xl.schemas.get_by_name(workspace_id, "clinical_trials") + version = await xl.schemas.publish(schema.id, pandera_schema, review_widgets={...}) + await xl.records.bulk_upsert(schema.id, rows, reference="10.1000/xyz") # auto-chunked + page = await xl.records.search(schema.id, text="tumor size", + filters=[("patient_age", "ge", 18)]) + await xl.suggestions.upsert(record.id, question="dosage", value=..., score=0.9, + agent="claude") + proj = await xl.projections.get(workspace_id, reference="10.1000/xyz") +``` + +Resources and the quirks they encapsulate (all confirmed against the server and the +frontend's PR #230 findings): + +- **Schemas** — create/get/list/update; `publish()` posts a Pandera body plus + out-of-band `review_widgets` (Pandera's `to_json()` drops column metadata); + versions list/get; `columns()` from the current version's cache. +- **Records** — `bulk_upsert` (idempotent on `external_id`, `metadata` is + patch-like), `search` (text + `(column, op, value)` filters with + `op ∈ {eq,in,ge,le}`; `total` is approximate — stale index ids are skipped and + FTS saturates, so pagination never promises exact counts), `list`, `delete`, + and `references.get()` for the cross-schema reference view. `bulk_upsert` + accepts list-of-dicts or a pandas DataFrame (pandas imported lazily inside that + path only). +- **Questions** — list/get per schema. Callers address questions by **name**; + the resource resolves to **id** via the cached map (suggestions key by id; + cells and response values key by name — getting this wrong silently detaches + provenance). +- **Suggestions** — upsert per (record, question) with `value`, `score`, `agent`, + `type`; list per record. +- **Projections** — `get(workspace_id, reference)` → cells of response ∨ suggestion + per question with `source` provenance. **Responses (read-only this slice)** — + `GET` returns literal `null` with 200 when absent (mapped to `None`, not an + error); values are double-wrapped `{name: {"value": …}}` and unwrapped here. + +Domain classes are thin Pydantic models with behavior (e.g. `SchemaVersion.find_column`, +`SearchPage.records`), constructed from DTOs by the resources — same layering the +frontend proved testable without a live backend. + +## 5. Agentic CLI + +v2 commands register at the **top level** of the existing `extralit` app — no `v2` +prefix in the interface — built over the sync facade: + +``` +extralit schemas list | get | create | publish | versions +extralit records upsert | search | list | delete +extralit questions list +extralit suggestions upsert +extralit projection get +extralit references get +``` + +The v2 model owns the CLI surface going forward. Consequence: the existing v1 +`extralit schemas` subcommand (workspace-file Pandera management) is **replaced** by +the v2 implementation — a deliberate breaking change; the CLI leads the v1→v2 +transition even while the SDK library keeps the v1 surface intact. All other v1 +subcommands (`datasets`, `users`, `workspaces`, `extraction`, …) remain untouched +until Phase 6. + +- **JSON-first.** Every command accepts `--json`; when stdout is not a TTY, JSON is + the default automatically. JSON shapes are the serialized DTOs — stable because + they are drift-gated by the contract layer. Rich tables remain the TTY default + for humans. +- **Exit-code and stream contract.** stdout carries data only; errors are structured + JSON on stderr (`{"error": {"type", "status", "detail"}}`). Exit codes: + 0 success, 1 API/runtime error, 2 usage error, 3 validation (422). +- **Non-interactive by construction.** Env vars (`EXTRALIT_API_URL`, + `EXTRALIT_API_KEY` or username/password) and the credentials file; no prompt is + ever emitted when stdin is not a TTY. +- **Piping.** `records upsert` reads JSONL from stdin or `--file`; `records search + --limit/--offset` page through cleanly for chaining. +- **Startup budget.** The v2 commands register into `cli/app.py` via lazy callbacks; + heavy deps (pandas, pandera, datasets) import only inside the commands that use + them. Budget: `extralit schemas --help` completes in < 300 ms on the Orin dev + host, verified with `python -X importtime` in a CI check. + +## 6. Error handling + +`V2APIError(status, detail)` base → `AuthError` (post-refresh 401), +`NotFoundError`, `ValidationError` (normalizes both server 422 body shapes — +`detail: str` and `detail: [{loc, msg}]`). The CLI maps this hierarchy onto the +exit-code contract. Search totals are documented as approximate rather than +pretending exactness. + +## 7. Testing + +- **Unit (no server):** pytest + pytest-asyncio + pytest-httpx. Resource tests pin + each wire quirk by name: double-wrap/unwrap, name↔id suggestion join, + both 422 shapes, null-response-200, bulk chunking + bounded concurrency + + mid-batch failure behavior, token refresh on 401. Sync-facade smoke tests run + inside a live event loop (Jupyter simulation). CLI tests via Typer's runner + assert `--json` output shapes, stderr error JSON, and exit codes. +- **CI gates:** codegen no-drift, snapshot-vs-server, import-boundary grep + (`extralit/v2` must not import v1 modules), CLI startup-time budget. +- **Integration (opt-in marker, live stack):** one end-to-end extraction-loop test + (publish → upsert → suggest → search → projection) reusing the deterministic + seeding approach from the frontend's `e2e/v2/`. + +## Scope + +**In this slice:** everything above — contract layer, transport + sync facade, +the five resources, the top-level CLI verbs with JSON-first output (including the +replacement of the v1 `schemas` subcommand), unit tests + CI gates. + +**Deferred by design:** responses *write* (submit/draft/discard — the review loop +belongs to a follow-up once the frontend's Phase 5 queue lands), DataFrame/parquet/HF +export, `rebuild-index` and other admin verbs, webhooks, span questions, and the +retirement of the remaining v1 CLI subcommands and SDK surface (Phase 6) this +boundary exists to enable. diff --git a/extralit-frontend/e2e/v2/draft-lifecycle.spec.ts b/extralit-frontend/e2e/v2/draft-lifecycle.spec.ts index d0eabb6db..df1073e62 100644 --- a/extralit-frontend/e2e/v2/draft-lifecycle.spec.ts +++ b/extralit-frontend/e2e/v2/draft-lifecycle.spec.ts @@ -6,7 +6,7 @@ import { createIsolatedRecord, expect, loadSeed, signIn, test } from "./fixtures // submit at the end never contaminates another spec's clean-suggestion precondition. test("draft persists in the form without touching the projection, then submits", async ({ page, request }) => { const seed = loadSeed(); - const { reference, recordId } = await createIsolatedRecord(request, `${seed.reference}-draft-lifecycle`); + const { reference, recordId } = await createIsolatedRecord(request, "10.2000/e2e-draft-lifecycle"); await signIn(page); await page.goto(`/references/${encodeURIComponent(reference)}?workspace_id=${seed.workspaceId}`); diff --git a/extralit-frontend/e2e/v2/fixtures.ts b/extralit-frontend/e2e/v2/fixtures.ts index f64e83fad..1289d67d6 100644 --- a/extralit-frontend/e2e/v2/fixtures.ts +++ b/extralit-frontend/e2e/v2/fixtures.ts @@ -32,6 +32,9 @@ export const apiToken = async (request: APIRequestContext): Promise => { // each seed one of these in beforeEach so they never share the single seed record — which // otherwise races in parallel and, in serial order, leaves a submitted response that breaks // the next spec's clean "Suggestion" precondition (roborev job 154). A reseed wipes the schema. +// The reference must NOT contain seed.reference as a substring: other specs assert on the +// shared record via non-exact getByText(seed.reference), and a superstring reference in the +// same schema substring-matches those assertions into strict-mode violations (roborev job 157). export const createIsolatedRecord = async ( request: APIRequestContext, reference: string diff --git a/extralit-frontend/e2e/v2/review-loop.spec.ts b/extralit-frontend/e2e/v2/review-loop.spec.ts index d15071961..b16b482ba 100644 --- a/extralit-frontend/e2e/v2/review-loop.spec.ts +++ b/extralit-frontend/e2e/v2/review-loop.spec.ts @@ -5,7 +5,7 @@ import { createIsolatedRecord, expect, loadSeed, signIn, test } from "./fixtures // Uses its own isolated record so it never shares mutable response state with other specs. test("converts a suggestion into a submitted response", async ({ page, request }) => { const seed = loadSeed(); - const { reference, recordId } = await createIsolatedRecord(request, `${seed.reference}-review-loop`); + const { reference, recordId } = await createIsolatedRecord(request, "10.2000/e2e-review-loop"); await signIn(page); await page.goto(`/references/${encodeURIComponent(reference)}?workspace_id=${seed.workspaceId}`); diff --git a/extralit/CLAUDE.md b/extralit/CLAUDE.md index 1b2c2c623..b40b0466e 100644 --- a/extralit/CLAUDE.md +++ b/extralit/CLAUDE.md @@ -47,3 +47,14 @@ src/extralit/ /models # Data models /utils # Utilities ``` + +## v2 SDK (`src/extralit/v2/`) + +- Parallel package for `/api/v2` (schema-centric). Import wall: v2 imports nothing from v1 + except `extralit.client.login`; only `cli/app.py` imports v2 (composition root). +- Wire types are GENERATED: `_api/openapi.json` (server `openapi-dump` snapshot) -> + `_api/_generated.py` via datamodel-codegen. Never hand-edit; regenerate with the command + in `tests/unit/v2/test_contract.py` and keep both in sync (drift-gated). +- `AsyncClient` is the real client; `Client` is a mechanical sync facade (background-thread + portal — works in Jupyter). CLI verbs register at TOP level (`extralit schemas|records|...`), + JSON-first (`--json` or non-TTY), errors as JSON on stderr (exit 0/1/2/3). diff --git a/extralit/pyproject.toml b/extralit/pyproject.toml index 338c83b51..9aa8db709 100644 --- a/extralit/pyproject.toml +++ b/extralit/pyproject.toml @@ -53,6 +53,7 @@ root = "./src" [tool.ruff] line-length = 120 +extend-exclude = ["src/extralit/v2/_api/_generated.py"] [tool.ruff.lint] select = [ @@ -115,6 +116,7 @@ dev = [ "mknotebooks >= 0.8.0", "pytest-retry>=1.5", "ty>=0.0.46", + "datamodel-code-generator>=0.26,<0.69", ] [project.scripts] diff --git a/extralit/src/extralit/cli/app.py b/extralit/src/extralit/cli/app.py index 95df2fc3d..18eb93f4e 100644 --- a/extralit/src/extralit/cli/app.py +++ b/extralit/src/extralit/cli/app.py @@ -9,7 +9,6 @@ info, login, logout, - schemas, training, users, whoami, @@ -50,13 +49,17 @@ def register_subcommands(): app.add_typer(info.app, name="info") app.add_typer(login.app, name="login") app.add_typer(logout.app, name="logout") - app.add_typer(schemas.app, name="schemas") app.add_typer(training.app, name="training") app.add_typer(users.app, name="users") app.add_typer(whoami.app, name="whoami") app.add_typer(workflows.app, name="workflows") app.add_typer(workspaces.app, name="workspaces") + # v2 owns the top-level verbs: schemas, records, questions, suggestions, projection, references. + from extralit.v2.cli import add_v2_commands # composition-root exception to the v1/v2 wall + + add_v2_commands(app) + register_subcommands() diff --git a/extralit/src/extralit/v2/__init__.py b/extralit/src/extralit/v2/__init__.py new file mode 100644 index 000000000..5ce235df4 --- /dev/null +++ b/extralit/src/extralit/v2/__init__.py @@ -0,0 +1,33 @@ +from extralit.v2._sync import Client +from extralit.v2.client import AsyncClient +from extralit.v2.models import ( + ProjectionCell, + ProjectionRecord, + ProjectionView, + Question, + Record, + ReferenceGroup, + ReferenceView, + Response, + Schema, + SchemaVersion, + SearchPage, + Suggestion, +) + +__all__ = [ + "AsyncClient", + "Client", + "ProjectionCell", + "ProjectionRecord", + "ProjectionView", + "Question", + "Record", + "ReferenceGroup", + "ReferenceView", + "Response", + "Schema", + "SchemaVersion", + "SearchPage", + "Suggestion", +] diff --git a/extralit/src/extralit/v2/_api/__init__.py b/extralit/src/extralit/v2/_api/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/extralit/src/extralit/v2/_api/_errors.py b/extralit/src/extralit/v2/_api/_errors.py new file mode 100644 index 000000000..e875de881 --- /dev/null +++ b/extralit/src/extralit/v2/_api/_errors.py @@ -0,0 +1,55 @@ +from __future__ import annotations + +from typing import Any + + +class V2APIError(Exception): + """Base error for /api/v2 calls.""" + + def __init__(self, status_code: int, detail: Any = None): + self.status_code = status_code + self.detail = detail + super().__init__(f"HTTP {status_code}: {detail!r}") + + +class AuthError(V2APIError): + """401/403, raised after any transparent token refresh has already been attempted.""" + + +class NotFoundError(V2APIError): + """404.""" + + +class ValidationError(V2APIError): + """422. The server emits two body shapes (detail: str | list[{loc, msg}]); `.errors` is normalized.""" + + def __init__(self, status_code: int, detail: Any = None): + super().__init__(status_code, detail) + self.errors = normalize_validation_detail(detail) + + +def normalize_validation_detail(detail: Any) -> list[dict]: + if detail is None: + return [] + if isinstance(detail, str): + return [{"loc": [], "msg": detail}] + if isinstance(detail, list): + out = [] + for item in detail: + if isinstance(item, dict): + out.append({"loc": list(item.get("loc", [])), "msg": str(item.get("msg", item))}) + else: + out.append({"loc": [], "msg": str(item)}) + return out + return [{"loc": [], "msg": str(detail)}] + + +def error_from_response(status_code: int, body: Any) -> V2APIError: + detail = body.get("detail") if isinstance(body, dict) else body + if status_code in (401, 403): + return AuthError(status_code, detail) + if status_code == 404: + return NotFoundError(status_code, detail) + if status_code == 422: + return ValidationError(status_code, detail) + return V2APIError(status_code, detail) diff --git a/extralit/src/extralit/v2/_api/_generated.py b/extralit/src/extralit/v2/_api/_generated.py new file mode 100644 index 000000000..3dc8fa4da --- /dev/null +++ b/extralit/src/extralit/v2/_api/_generated.py @@ -0,0 +1,275 @@ +# generated by datamodel-codegen: +# filename: openapi.json + +from __future__ import annotations + +from enum import Enum +from typing import Any, Optional, Union +from uuid import UUID + +from pydantic import AwareDatetime, BaseModel, Field, conint, constr + + +class BodyCreateTokenTokenPost(BaseModel): + password: str = Field(..., title="Password") + username: str = Field(..., title="Username") + + +class Source(Enum): + response = "response" + suggestion = "suggestion" + + +class ProjectionCell(BaseModel): + question_name: str = Field(..., title="Question Name") + source: Optional[Source] = Field(None, title="Source") + value: Any = Field(None, title="Value") + + +class ProjectionRecord(BaseModel): + cells: list[ProjectionCell] = Field(..., title="Cells") + record_id: UUID = Field(..., title="Record Id") + reference: str = Field(..., title="Reference") + schema_id: UUID = Field(..., title="Schema Id") + + +class ProjectionView(BaseModel): + records: list[ProjectionRecord] = Field(..., title="Records") + reference: str = Field(..., title="Reference") + total_records: int = Field(..., title="Total Records") + + +class QuestionType(Enum): + text = "text" + rating = "rating" + label_selection = "label_selection" + multi_label_selection = "multi_label_selection" + ranking = "ranking" + span = "span" + table = "table" + + +class QuestionUpdate(BaseModel): + columns: Optional[list[str]] = Field(None, title="Columns") + description: Optional[str] = Field(None, title="Description") + required: Optional[bool] = Field(None, title="Required") + settings: Optional[dict[str, Any]] = Field(None, title="Settings") + title: Optional[str] = Field(None, title="Title") + + +class Op(Enum): + eq = "eq" + in_ = "in" + ge = "ge" + le = "le" + + +class RecordFilter(BaseModel): + column: str = Field(..., title="Column") + op: Op = Field(..., title="Op") + value: Any = Field(..., title="Value") + + +class RecordSearchQuery(BaseModel): + filters: Optional[list[RecordFilter]] = Field(None, title="Filters") + limit: Optional[conint(ge=1, le=1000)] = Field(50, title="Limit") + offset: Optional[conint(ge=0)] = Field(0, title="Offset") + text: Optional[str] = Field(None, title="Text") + + +class RefreshTokenRequest(BaseModel): + refresh_token: str = Field(..., title="Refresh Token") + + +class ResponseStatus(Enum): + draft = "draft" + submitted = "submitted" + discarded = "discarded" + + +class ResponseUpsert(BaseModel): + status: ResponseStatus + values: Optional[dict[str, dict[str, Any]]] = Field(None, title="Values") + + +class SchemaCreate(BaseModel): + name: constr(min_length=1, max_length=200) = Field(..., title="Name") + settings: Optional[dict[str, Any]] = Field(None, title="Settings") + workspace_id: UUID = Field(..., title="Workspace Id") + + +class SchemaStatus(Enum): + draft = "draft" + published = "published" + + +class SchemaUpdate(BaseModel): + name: Optional[constr(min_length=1, max_length=200)] = Field(None, title="Name") + settings: Optional[dict[str, Any]] = Field(None, title="Settings") + + +class SchemaVersionCreate(BaseModel): + body: str = Field( + ..., + description="Pandera DataFrameSchema serialized via .to_json()", + title="Body", + ) + review_widgets: Optional[dict[str, dict[str, Any]]] = Field( + None, title="Review Widgets" + ) + + +class SchemaVersionRead(BaseModel): + checksum: str = Field(..., title="Checksum") + columns_cache: list[dict[str, Any]] = Field(..., title="Columns Cache") + etag: str = Field(..., title="Etag") + id: UUID = Field(..., title="Id") + inserted_at: AwareDatetime = Field(..., title="Inserted At") + object_key: str = Field(..., title="Object Key") + object_version_id: Optional[str] = Field(..., title="Object Version Id") + parent_version_id: Optional[UUID] = Field(..., title="Parent Version Id") + review_widgets: dict[str, dict[str, Any]] = Field(..., title="Review Widgets") + schema_id: UUID = Field(..., title="Schema Id") + version: int = Field(..., title="Version") + + +class SuggestionType(Enum): + model = "model" + human = "human" + selection = "selection" + + +class SuggestionUpsert(BaseModel): + agent: Optional[str] = Field(None, title="Agent") + question_id: UUID = Field(..., title="Question Id") + score: Optional[Union[float, list[float]]] = Field(None, title="Score") + type: Optional[SuggestionType] = None + value: Any = Field(..., title="Value") + + +class Token(BaseModel): + access_token: str = Field(..., title="Access Token") + refresh_token: Optional[str] = Field(None, title="Refresh Token") + token_type: Optional[str] = Field("bearer", title="Token Type") + + +class V2RecordStatus(Enum): + pending = "pending" + completed = "completed" + discarded = "discarded" + + +class QuestionCreate(BaseModel): + columns: list[str] = Field(..., min_length=1, title="Columns") + description: Optional[str] = Field(None, title="Description") + name: constr(min_length=1, max_length=200) = Field(..., title="Name") + required: Optional[bool] = Field(False, title="Required") + settings: Optional[dict[str, Any]] = Field(None, title="Settings") + title: constr(min_length=1) = Field(..., title="Title") + type: QuestionType + + +class QuestionRead(BaseModel): + columns: list[str] = Field(..., title="Columns") + description: Optional[str] = Field(..., title="Description") + id: UUID = Field(..., title="Id") + inserted_at: AwareDatetime = Field(..., title="Inserted At") + name: str = Field(..., title="Name") + required: bool = Field(..., title="Required") + schema_id: UUID = Field(..., title="Schema Id") + settings: dict[str, Any] = Field(..., title="Settings") + title: str = Field(..., title="Title") + type: QuestionType + updated_at: AwareDatetime = Field(..., title="Updated At") + + +class Questions(BaseModel): + items: list[QuestionRead] = Field(..., title="Items") + + +class RecordRead(BaseModel): + external_id: Optional[str] = Field(..., title="External Id") + fields: dict[str, Any] = Field(..., title="Fields") + id: UUID = Field(..., title="Id") + inserted_at: AwareDatetime = Field(..., title="Inserted At") + metadata: Optional[dict[str, Any]] = Field(None, title="Metadata") + reference: str = Field(..., title="Reference") + schema_id: UUID = Field(..., title="Schema Id") + schema_version_id: UUID = Field(..., title="Schema Version Id") + status: V2RecordStatus + updated_at: AwareDatetime = Field(..., title="Updated At") + + +class RecordUpsert(BaseModel): + external_id: Optional[str] = Field(None, title="External Id") + fields: dict[str, Any] = Field(..., title="Fields") + metadata: Optional[dict[str, Any]] = Field(None, title="Metadata") + reference: constr(min_length=1, max_length=500) = Field(..., title="Reference") + schema_version_id: Optional[UUID] = Field( + None, + description="Pin to a specific version; defaults to the schema's current_version_id", + title="Schema Version Id", + ) + status: Optional[V2RecordStatus] = None + + +class Records(BaseModel): + items: list[RecordRead] = Field(..., title="Items") + total: int = Field(..., title="Total") + + +class RecordsBulkUpsert(BaseModel): + items: list[RecordUpsert] = Field(..., max_length=500, min_length=1, title="Items") + + +class ReferenceGroup(BaseModel): + records: list[RecordRead] = Field(..., title="Records") + schema_id: UUID = Field(..., title="Schema Id") + schema_name: str = Field(..., title="Schema Name") + + +class ReferenceView(BaseModel): + groups: list[ReferenceGroup] = Field(..., title="Groups") + reference: str = Field(..., title="Reference") + total_records: int = Field(..., title="Total Records") + + +class ResponseRead(BaseModel): + id: UUID = Field(..., title="Id") + inserted_at: AwareDatetime = Field(..., title="Inserted At") + record_id: UUID = Field(..., title="Record Id") + status: ResponseStatus + updated_at: AwareDatetime = Field(..., title="Updated At") + user_id: UUID = Field(..., title="User Id") + values: Optional[dict[str, Any]] = Field(..., title="Values") + + +class SchemaRead(BaseModel): + current_version_id: Optional[UUID] = Field(..., title="Current Version Id") + id: UUID = Field(..., title="Id") + inserted_at: AwareDatetime = Field(..., title="Inserted At") + name: str = Field(..., title="Name") + settings: dict[str, Any] = Field(..., title="Settings") + status: SchemaStatus + updated_at: AwareDatetime = Field(..., title="Updated At") + workspace_id: UUID = Field(..., title="Workspace Id") + + +class Schemas(BaseModel): + items: list[SchemaRead] = Field(..., title="Items") + + +class SuggestionRead(BaseModel): + agent: Optional[str] = Field(..., title="Agent") + id: UUID = Field(..., title="Id") + inserted_at: AwareDatetime = Field(..., title="Inserted At") + question_id: UUID = Field(..., title="Question Id") + record_id: UUID = Field(..., title="Record Id") + score: Optional[Union[float, list[float]]] = Field(..., title="Score") + type: Optional[SuggestionType] + updated_at: AwareDatetime = Field(..., title="Updated At") + value: Any = Field(..., title="Value") + + +class Suggestions(BaseModel): + items: list[SuggestionRead] = Field(..., title="Items") diff --git a/extralit/src/extralit/v2/_api/_transport.py b/extralit/src/extralit/v2/_api/_transport.py new file mode 100644 index 000000000..71d23ea81 --- /dev/null +++ b/extralit/src/extralit/v2/_api/_transport.py @@ -0,0 +1,129 @@ +from __future__ import annotations + +import asyncio +from typing import Any, Optional + +import httpx + +from extralit.v2._api._errors import AuthError, error_from_response + +_API_PREFIX = "/api/v2" + + +def _safe_json(response: httpx.Response) -> Any: + try: + return response.json() + except ValueError: + return response.text + + +class AsyncTransport: + """One httpx.AsyncClient per client instance. Auth modes: api_key header (default, + no token lifecycle) or username/password -> bearer JWT with a single transparent + refresh on 401 (then AuthError).""" + + def __init__( + self, + api_url: str, + api_key: Optional[str] = None, + username: Optional[str] = None, + password: Optional[str] = None, + timeout: float = 60.0, + retries: int = 5, + extra_headers: Optional[dict] = None, + ): + self.api_url = api_url.rstrip("/") + self._api_key = api_key + self._username = username + self._password = password + self._access_token: Optional[str] = None + self._refresh_token: Optional[str] = None + self._auth_lock: Optional[asyncio.Lock] = None # lazily created on first use (loop-safe) + self._refresh_failed_for: Optional[str] = None # stale token whose refresh already failed + self._http = httpx.AsyncClient( + base_url=self.api_url, + timeout=timeout, + transport=httpx.AsyncHTTPTransport(retries=retries), + headers=extra_headers or {}, + ) + + def _auth_headers(self) -> dict: + if self._access_token: + return {"Authorization": f"Bearer {self._access_token}"} + if self._api_key: + return {"X-Extralit-Api-Key": self._api_key} + return {} + + async def _login(self) -> None: + response = await self._http.post( + f"{_API_PREFIX}/token", data={"username": self._username, "password": self._password} + ) + if response.status_code >= 400: + raise AuthError(response.status_code, _safe_json(response)) + payload = response.json() + self._access_token = payload["access_token"] + self._refresh_token = payload.get("refresh_token") + + async def _refresh(self) -> bool: + if not self._refresh_token: + return False + response = await self._http.post(f"{_API_PREFIX}/token/refresh", json={"refresh_token": self._refresh_token}) + if response.status_code >= 400: + return False + payload = response.json() + self._access_token = payload["access_token"] + self._refresh_token = payload.get("refresh_token", self._refresh_token) + return True + + def _get_auth_lock(self) -> asyncio.Lock: + """Return the per-instance auth lock, creating it lazily inside the running loop.""" + if self._auth_lock is None: + self._auth_lock = asyncio.Lock() + return self._auth_lock + + async def _ensure_logged_in(self) -> None: + """Login exactly once even under concurrent callers.""" + if self._access_token: + return + async with self._get_auth_lock(): + if not self._access_token: # re-check after acquiring lock + await self._login() + + async def _refresh_if_stale(self, stale_token: str) -> bool: + """Coalesce concurrent 401s: only the first waiter refreshes; the rest reuse the rotated + token. If that single refresh fails, later waiters don't re-hammer the auth endpoint.""" + async with self._get_auth_lock(): + if self._access_token != stale_token: + return True # another coroutine already refreshed while we waited for the lock + if self._refresh_failed_for == stale_token: + return False # a prior waiter already tried and failed to refresh this token + ok = await self._refresh() + if not ok: + self._refresh_failed_for = stale_token + return ok + + async def request( + self, + method: str, + path: str, + *, + params: Optional[dict] = None, + json: Optional[Any] = None, + ) -> Any: + if self._username and not self._api_key: + await self._ensure_logged_in() + response = await self._http.request( + method, f"{_API_PREFIX}{path}", params=params, json=json, headers=self._auth_headers() + ) + if response.status_code == 401 and self._access_token and await self._refresh_if_stale(self._access_token): + response = await self._http.request( + method, f"{_API_PREFIX}{path}", params=params, json=json, headers=self._auth_headers() + ) + if response.status_code >= 400: + raise error_from_response(response.status_code, _safe_json(response)) + if response.status_code == 204 or not response.content: + return None + return response.json() + + async def aclose(self) -> None: + await self._http.aclose() diff --git a/extralit/src/extralit/v2/_api/openapi.json b/extralit/src/extralit/v2/_api/openapi.json new file mode 100644 index 000000000..bb27a0dd0 --- /dev/null +++ b/extralit/src/extralit/v2/_api/openapi.json @@ -0,0 +1,4816 @@ +{ + "components": { + "schemas": { + "Body_create_token_token_post": { + "properties": { + "password": { + "title": "Password", + "type": "string" + }, + "username": { + "title": "Username", + "type": "string" + } + }, + "required": [ + "username", + "password" + ], + "title": "Body_create_token_token_post", + "type": "object" + }, + "ProjectionCell": { + "properties": { + "question_name": { + "title": "Question Name", + "type": "string" + }, + "source": { + "anyOf": [ + { + "enum": [ + "response", + "suggestion" + ], + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source" + }, + "value": { + "anyOf": [ + {}, + { + "type": "null" + } + ], + "title": "Value" + } + }, + "required": [ + "question_name" + ], + "title": "ProjectionCell", + "type": "object" + }, + "ProjectionRecord": { + "properties": { + "cells": { + "items": { + "$ref": "#/components/schemas/ProjectionCell" + }, + "title": "Cells", + "type": "array" + }, + "record_id": { + "format": "uuid", + "title": "Record Id", + "type": "string" + }, + "reference": { + "title": "Reference", + "type": "string" + }, + "schema_id": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + }, + "required": [ + "record_id", + "schema_id", + "reference", + "cells" + ], + "title": "ProjectionRecord", + "type": "object" + }, + "ProjectionView": { + "properties": { + "records": { + "items": { + "$ref": "#/components/schemas/ProjectionRecord" + }, + "title": "Records", + "type": "array" + }, + "reference": { + "title": "Reference", + "type": "string" + }, + "total_records": { + "title": "Total Records", + "type": "integer" + } + }, + "required": [ + "reference", + "records", + "total_records" + ], + "title": "ProjectionView", + "type": "object" + }, + "QuestionCreate": { + "properties": { + "columns": { + "items": { + "type": "string" + }, + "minItems": 1, + "title": "Columns", + "type": "array" + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description" + }, + "name": { + "maxLength": 200, + "minLength": 1, + "title": "Name", + "type": "string" + }, + "required": { + "default": false, + "title": "Required", + "type": "boolean" + }, + "settings": { + "title": "Settings", + "type": "object" + }, + "title": { + "minLength": 1, + "title": "Title", + "type": "string" + }, + "type": { + "$ref": "#/components/schemas/QuestionType" + } + }, + "required": [ + "name", + "title", + "type", + "columns" + ], + "title": "QuestionCreate", + "type": "object" + }, + "QuestionRead": { + "properties": { + "columns": { + "items": { + "type": "string" + }, + "title": "Columns", + "type": "array" + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description" + }, + "id": { + "format": "uuid", + "title": "Id", + "type": "string" + }, + "inserted_at": { + "format": "date-time", + "title": "Inserted At", + "type": "string" + }, + "name": { + "title": "Name", + "type": "string" + }, + "required": { + "title": "Required", + "type": "boolean" + }, + "schema_id": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + }, + "settings": { + "title": "Settings", + "type": "object" + }, + "title": { + "title": "Title", + "type": "string" + }, + "type": { + "$ref": "#/components/schemas/QuestionType" + }, + "updated_at": { + "format": "date-time", + "title": "Updated At", + "type": "string" + } + }, + "required": [ + "id", + "schema_id", + "name", + "title", + "description", + "type", + "columns", + "settings", + "required", + "inserted_at", + "updated_at" + ], + "title": "QuestionRead", + "type": "object" + }, + "QuestionType": { + "enum": [ + "text", + "rating", + "label_selection", + "multi_label_selection", + "ranking", + "span", + "table" + ], + "title": "QuestionType", + "type": "string" + }, + "QuestionUpdate": { + "properties": { + "columns": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Columns" + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description" + }, + "required": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Required" + }, + "settings": { + "anyOf": [ + { + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Settings" + }, + "title": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Title" + } + }, + "title": "QuestionUpdate", + "type": "object" + }, + "Questions": { + "properties": { + "items": { + "items": { + "$ref": "#/components/schemas/QuestionRead" + }, + "title": "Items", + "type": "array" + } + }, + "required": [ + "items" + ], + "title": "Questions", + "type": "object" + }, + "RecordFilter": { + "properties": { + "column": { + "title": "Column", + "type": "string" + }, + "op": { + "enum": [ + "eq", + "in", + "ge", + "le" + ], + "title": "Op", + "type": "string" + }, + "value": { + "title": "Value" + } + }, + "required": [ + "column", + "op", + "value" + ], + "title": "RecordFilter", + "type": "object" + }, + "RecordRead": { + "properties": { + "external_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "External Id" + }, + "fields": { + "title": "Fields", + "type": "object" + }, + "id": { + "format": "uuid", + "title": "Id", + "type": "string" + }, + "inserted_at": { + "format": "date-time", + "title": "Inserted At", + "type": "string" + }, + "metadata": { + "anyOf": [ + { + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Metadata" + }, + "reference": { + "title": "Reference", + "type": "string" + }, + "schema_id": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + }, + "schema_version_id": { + "format": "uuid", + "title": "Schema Version Id", + "type": "string" + }, + "status": { + "$ref": "#/components/schemas/V2RecordStatus" + }, + "updated_at": { + "format": "date-time", + "title": "Updated At", + "type": "string" + } + }, + "required": [ + "id", + "schema_id", + "schema_version_id", + "reference", + "external_id", + "fields", + "status", + "inserted_at", + "updated_at" + ], + "title": "RecordRead", + "type": "object" + }, + "RecordSearchQuery": { + "properties": { + "filters": { + "items": { + "$ref": "#/components/schemas/RecordFilter" + }, + "title": "Filters", + "type": "array" + }, + "limit": { + "default": 50, + "maximum": 1000.0, + "minimum": 1.0, + "title": "Limit", + "type": "integer" + }, + "offset": { + "default": 0, + "minimum": 0.0, + "title": "Offset", + "type": "integer" + }, + "text": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Text" + } + }, + "title": "RecordSearchQuery", + "type": "object" + }, + "RecordUpsert": { + "description": "One bulk-upsert item.\n\n`fields` and `reference` are always written. `metadata` and `status` are patch-like:\nwhen omitted (None) on an update they preserve the existing row's values (they cannot\nbe cleared via upsert); on insert they default to no metadata / `pending`.", + "properties": { + "external_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "External Id" + }, + "fields": { + "title": "Fields", + "type": "object" + }, + "metadata": { + "anyOf": [ + { + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Metadata" + }, + "reference": { + "maxLength": 500, + "minLength": 1, + "title": "Reference", + "type": "string" + }, + "schema_version_id": { + "anyOf": [ + { + "format": "uuid", + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Pin to a specific version; defaults to the schema's current_version_id", + "title": "Schema Version Id" + }, + "status": { + "anyOf": [ + { + "$ref": "#/components/schemas/V2RecordStatus" + }, + { + "type": "null" + } + ] + } + }, + "required": [ + "fields", + "reference" + ], + "title": "RecordUpsert", + "type": "object" + }, + "Records": { + "properties": { + "items": { + "items": { + "$ref": "#/components/schemas/RecordRead" + }, + "title": "Items", + "type": "array" + }, + "total": { + "title": "Total", + "type": "integer" + } + }, + "required": [ + "items", + "total" + ], + "title": "Records", + "type": "object" + }, + "RecordsBulkUpsert": { + "properties": { + "items": { + "items": { + "$ref": "#/components/schemas/RecordUpsert" + }, + "maxItems": 500, + "minItems": 1, + "title": "Items", + "type": "array" + } + }, + "required": [ + "items" + ], + "title": "RecordsBulkUpsert", + "type": "object" + }, + "ReferenceGroup": { + "properties": { + "records": { + "items": { + "$ref": "#/components/schemas/RecordRead" + }, + "title": "Records", + "type": "array" + }, + "schema_id": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + }, + "schema_name": { + "title": "Schema Name", + "type": "string" + } + }, + "required": [ + "schema_id", + "schema_name", + "records" + ], + "title": "ReferenceGroup", + "type": "object" + }, + "ReferenceView": { + "properties": { + "groups": { + "items": { + "$ref": "#/components/schemas/ReferenceGroup" + }, + "title": "Groups", + "type": "array" + }, + "reference": { + "title": "Reference", + "type": "string" + }, + "total_records": { + "title": "Total Records", + "type": "integer" + } + }, + "required": [ + "reference", + "groups", + "total_records" + ], + "title": "ReferenceView", + "type": "object" + }, + "RefreshTokenRequest": { + "description": "Refresh token request model", + "properties": { + "refresh_token": { + "title": "Refresh Token", + "type": "string" + } + }, + "required": [ + "refresh_token" + ], + "title": "RefreshTokenRequest", + "type": "object" + }, + "ResponseRead": { + "properties": { + "id": { + "format": "uuid", + "title": "Id", + "type": "string" + }, + "inserted_at": { + "format": "date-time", + "title": "Inserted At", + "type": "string" + }, + "record_id": { + "format": "uuid", + "title": "Record Id", + "type": "string" + }, + "status": { + "$ref": "#/components/schemas/ResponseStatus" + }, + "updated_at": { + "format": "date-time", + "title": "Updated At", + "type": "string" + }, + "user_id": { + "format": "uuid", + "title": "User Id", + "type": "string" + }, + "values": { + "anyOf": [ + { + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Values" + } + }, + "required": [ + "id", + "record_id", + "user_id", + "values", + "status", + "inserted_at", + "updated_at" + ], + "title": "ResponseRead", + "type": "object" + }, + "ResponseStatus": { + "enum": [ + "draft", + "submitted", + "discarded" + ], + "title": "ResponseStatus", + "type": "string" + }, + "ResponseUpsert": { + "properties": { + "status": { + "$ref": "#/components/schemas/ResponseStatus" + }, + "values": { + "anyOf": [ + { + "additionalProperties": { + "type": "object" + }, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Values" + } + }, + "required": [ + "status" + ], + "title": "ResponseUpsert", + "type": "object" + }, + "SchemaCreate": { + "properties": { + "name": { + "maxLength": 200, + "minLength": 1, + "title": "Name", + "type": "string" + }, + "settings": { + "title": "Settings", + "type": "object" + }, + "workspace_id": { + "format": "uuid", + "title": "Workspace Id", + "type": "string" + } + }, + "required": [ + "name", + "workspace_id" + ], + "title": "SchemaCreate", + "type": "object" + }, + "SchemaRead": { + "properties": { + "current_version_id": { + "anyOf": [ + { + "format": "uuid", + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Current Version Id" + }, + "id": { + "format": "uuid", + "title": "Id", + "type": "string" + }, + "inserted_at": { + "format": "date-time", + "title": "Inserted At", + "type": "string" + }, + "name": { + "title": "Name", + "type": "string" + }, + "settings": { + "title": "Settings", + "type": "object" + }, + "status": { + "$ref": "#/components/schemas/SchemaStatus" + }, + "updated_at": { + "format": "date-time", + "title": "Updated At", + "type": "string" + }, + "workspace_id": { + "format": "uuid", + "title": "Workspace Id", + "type": "string" + } + }, + "required": [ + "id", + "name", + "status", + "current_version_id", + "settings", + "workspace_id", + "inserted_at", + "updated_at" + ], + "title": "SchemaRead", + "type": "object" + }, + "SchemaStatus": { + "enum": [ + "draft", + "published" + ], + "title": "SchemaStatus", + "type": "string" + }, + "SchemaUpdate": { + "properties": { + "name": { + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Name" + }, + "settings": { + "anyOf": [ + { + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Settings" + } + }, + "title": "SchemaUpdate", + "type": "object" + }, + "SchemaVersionCreate": { + "properties": { + "body": { + "description": "Pandera DataFrameSchema serialized via .to_json()", + "title": "Body", + "type": "string" + }, + "review_widgets": { + "additionalProperties": { + "type": "object" + }, + "title": "Review Widgets", + "type": "object" + } + }, + "required": [ + "body" + ], + "title": "SchemaVersionCreate", + "type": "object" + }, + "SchemaVersionRead": { + "properties": { + "checksum": { + "title": "Checksum", + "type": "string" + }, + "columns_cache": { + "items": { + "type": "object" + }, + "title": "Columns Cache", + "type": "array" + }, + "etag": { + "title": "Etag", + "type": "string" + }, + "id": { + "format": "uuid", + "title": "Id", + "type": "string" + }, + "inserted_at": { + "format": "date-time", + "title": "Inserted At", + "type": "string" + }, + "object_key": { + "title": "Object Key", + "type": "string" + }, + "object_version_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Object Version Id" + }, + "parent_version_id": { + "anyOf": [ + { + "format": "uuid", + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Parent Version Id" + }, + "review_widgets": { + "additionalProperties": { + "type": "object" + }, + "title": "Review Widgets", + "type": "object" + }, + "schema_id": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + }, + "version": { + "title": "Version", + "type": "integer" + } + }, + "required": [ + "id", + "schema_id", + "version", + "object_key", + "object_version_id", + "etag", + "checksum", + "parent_version_id", + "columns_cache", + "review_widgets", + "inserted_at" + ], + "title": "SchemaVersionRead", + "type": "object" + }, + "Schemas": { + "properties": { + "items": { + "items": { + "$ref": "#/components/schemas/SchemaRead" + }, + "title": "Items", + "type": "array" + } + }, + "required": [ + "items" + ], + "title": "Schemas", + "type": "object" + }, + "SuggestionRead": { + "properties": { + "agent": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Agent" + }, + "id": { + "format": "uuid", + "title": "Id", + "type": "string" + }, + "inserted_at": { + "format": "date-time", + "title": "Inserted At", + "type": "string" + }, + "question_id": { + "format": "uuid", + "title": "Question Id", + "type": "string" + }, + "record_id": { + "format": "uuid", + "title": "Record Id", + "type": "string" + }, + "score": { + "anyOf": [ + { + "type": "number" + }, + { + "items": { + "type": "number" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Score" + }, + "type": { + "anyOf": [ + { + "$ref": "#/components/schemas/SuggestionType" + }, + { + "type": "null" + } + ] + }, + "updated_at": { + "format": "date-time", + "title": "Updated At", + "type": "string" + }, + "value": { + "title": "Value" + } + }, + "required": [ + "id", + "record_id", + "question_id", + "value", + "score", + "agent", + "type", + "inserted_at", + "updated_at" + ], + "title": "SuggestionRead", + "type": "object" + }, + "SuggestionType": { + "enum": [ + "model", + "human", + "selection" + ], + "title": "SuggestionType", + "type": "string" + }, + "SuggestionUpsert": { + "properties": { + "agent": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Agent" + }, + "question_id": { + "format": "uuid", + "title": "Question Id", + "type": "string" + }, + "score": { + "anyOf": [ + { + "type": "number" + }, + { + "items": { + "type": "number" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Score" + }, + "type": { + "anyOf": [ + { + "$ref": "#/components/schemas/SuggestionType" + }, + { + "type": "null" + } + ] + }, + "value": { + "title": "Value" + } + }, + "required": [ + "question_id", + "value" + ], + "title": "SuggestionUpsert", + "type": "object" + }, + "Suggestions": { + "properties": { + "items": { + "items": { + "$ref": "#/components/schemas/SuggestionRead" + }, + "title": "Items", + "type": "array" + } + }, + "required": [ + "items" + ], + "title": "Suggestions", + "type": "object" + }, + "Token": { + "description": "Token response model", + "properties": { + "access_token": { + "title": "Access Token", + "type": "string" + }, + "refresh_token": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Refresh Token" + }, + "token_type": { + "default": "bearer", + "title": "Token Type", + "type": "string" + } + }, + "required": [ + "access_token" + ], + "title": "Token", + "type": "object" + }, + "V2RecordStatus": { + "description": "v2 record status. Distinct from v1 RecordStatus: adds `discarded` and maps to its\nown PG enum type (v2_record_status_enum) so v1's record_status_enum is untouched.", + "enum": [ + "pending", + "completed", + "discarded" + ], + "title": "V2RecordStatus", + "type": "string" + } + }, + "securitySchemes": { + "APIKeyHeader": { + "in": "header", + "name": "X-Extralit-Api-Key", + "type": "apiKey" + }, + "HTTPBearer": { + "scheme": "bearer", + "type": "http" + } + } + }, + "info": { + "description": "Extralit Server API v2 (schema-centric)", + "title": "Extralit v2", + "version": "0.6.1" + }, + "openapi": "3.1.0", + "paths": { + "/projection/references/{reference}": { + "get": { + "operationId": "get_reference_projection_projection_references__reference__get", + "parameters": [ + { + "in": "path", + "name": "reference", + "required": true, + "schema": { + "title": "Reference", + "type": "string" + } + }, + { + "description": "Workspace to scope the view (required)", + "in": "query", + "name": "workspace_id", + "required": true, + "schema": { + "description": "Workspace to scope the view (required)", + "format": "uuid", + "title": "Workspace Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProjectionView" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Get Reference Projection", + "tags": [ + "v2: projection" + ] + } + }, + "/questions/{question_id}": { + "delete": { + "operationId": "delete_question_questions__question_id__delete", + "parameters": [ + { + "in": "path", + "name": "question_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Question Id", + "type": "string" + } + } + ], + "responses": { + "204": { + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Delete Question", + "tags": [ + "v2: questions" + ] + }, + "get": { + "operationId": "get_question_questions__question_id__get", + "parameters": [ + { + "in": "path", + "name": "question_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Question Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/QuestionRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Get Question", + "tags": [ + "v2: questions" + ] + }, + "put": { + "operationId": "update_question_questions__question_id__put", + "parameters": [ + { + "in": "path", + "name": "question_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Question Id", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/QuestionUpdate" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/QuestionRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Update Question", + "tags": [ + "v2: questions" + ] + } + }, + "/records/{record_id}/responses": { + "get": { + "operationId": "get_own_response_records__record_id__responses_get", + "parameters": [ + { + "in": "path", + "name": "record_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Record Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "anyOf": [ + { + "$ref": "#/components/schemas/ResponseRead" + }, + { + "type": "null" + } + ], + "title": "Response Get Own Response Records Record Id Responses Get" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Get Own Response", + "tags": [ + "v2: annotation" + ] + }, + "put": { + "operationId": "upsert_response_records__record_id__responses_put", + "parameters": [ + { + "in": "path", + "name": "record_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Record Id", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ResponseUpsert" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ResponseRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Upsert Response", + "tags": [ + "v2: annotation" + ] + } + }, + "/records/{record_id}/suggestions": { + "get": { + "operationId": "list_suggestions_records__record_id__suggestions_get", + "parameters": [ + { + "in": "path", + "name": "record_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Record Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Suggestions" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "List Suggestions", + "tags": [ + "v2: annotation" + ] + }, + "put": { + "operationId": "upsert_suggestion_records__record_id__suggestions_put", + "parameters": [ + { + "in": "path", + "name": "record_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Record Id", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SuggestionUpsert" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SuggestionRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Upsert Suggestion", + "tags": [ + "v2: annotation" + ] + } + }, + "/references/{reference}": { + "get": { + "description": "The document's project-level extraction view: all v2 records across every schema in\nthe workspace that share this `reference` (spec \u00a76), grouped per schema.\n\nAn unknown reference returns an empty view (200): the reference is a free-form join\nkey, not an entity, so \"no extractions yet\" is not an error.", + "operationId": "get_reference_view_references__reference__get", + "parameters": [ + { + "in": "path", + "name": "reference", + "required": true, + "schema": { + "title": "Reference", + "type": "string" + } + }, + { + "description": "Workspace to scope the cross-schema view (required)", + "in": "query", + "name": "workspace_id", + "required": true, + "schema": { + "description": "Workspace to scope the cross-schema view (required)", + "format": "uuid", + "title": "Workspace Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ReferenceView" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Get Reference View", + "tags": [ + "v2: records" + ] + } + }, + "/schemas": { + "get": { + "operationId": "list_schemas_schemas_get", + "parameters": [ + { + "description": "Workspace to list schemas for (required)", + "in": "query", + "name": "workspace_id", + "required": true, + "schema": { + "description": "Workspace to list schemas for (required)", + "format": "uuid", + "title": "Workspace Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Schemas" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "List Schemas", + "tags": [ + "v2: schemas" + ] + }, + "post": { + "operationId": "create_schema_schemas_post", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaCreate" + } + } + }, + "required": true + }, + "responses": { + "201": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Create Schema", + "tags": [ + "v2: schemas" + ] + } + }, + "/schemas/{schema_id}": { + "delete": { + "operationId": "delete_schema_schemas__schema_id__delete", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Delete Schema", + "tags": [ + "v2: schemas" + ] + }, + "get": { + "operationId": "get_schema_schemas__schema_id__get", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Get Schema", + "tags": [ + "v2: schemas" + ] + }, + "put": { + "operationId": "update_schema_schemas__schema_id__put", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaUpdate" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Update Schema", + "tags": [ + "v2: schemas" + ] + } + }, + "/schemas/{schema_id}/columns": { + "get": { + "operationId": "get_schema_columns_schemas__schema_id__columns_get", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "items": { + "type": "object" + }, + "title": "Response Get Schema Columns Schemas Schema Id Columns Get", + "type": "array" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Get Schema Columns", + "tags": [ + "v2: schemas" + ] + } + }, + "/schemas/{schema_id}/questions": { + "get": { + "operationId": "list_questions_schemas__schema_id__questions_get", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Questions" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "List Questions", + "tags": [ + "v2: questions" + ] + }, + "post": { + "operationId": "create_question_schemas__schema_id__questions_post", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/QuestionCreate" + } + } + }, + "required": true + }, + "responses": { + "201": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/QuestionRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Create Question", + "tags": [ + "v2: questions" + ] + } + }, + "/schemas/{schema_id}/records": { + "delete": { + "operationId": "delete_schema_records_schemas__schema_id__records_delete", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + }, + { + "description": "Comma-separated record ids to delete", + "in": "query", + "name": "ids", + "required": true, + "schema": { + "description": "Comma-separated record ids to delete", + "title": "Ids", + "type": "string" + } + } + ], + "responses": { + "204": { + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Delete Schema Records", + "tags": [ + "v2: records" + ] + }, + "get": { + "operationId": "list_schema_records_schemas__schema_id__records_get", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + }, + { + "in": "query", + "name": "offset", + "required": false, + "schema": { + "default": 0, + "minimum": 0, + "title": "Offset", + "type": "integer" + } + }, + { + "in": "query", + "name": "limit", + "required": false, + "schema": { + "default": 50, + "maximum": 1000, + "minimum": 1, + "title": "Limit", + "type": "integer" + } + }, + { + "in": "query", + "name": "status", + "required": false, + "schema": { + "anyOf": [ + { + "$ref": "#/components/schemas/V2RecordStatus" + }, + { + "type": "null" + } + ], + "title": "Status" + } + }, + { + "in": "query", + "name": "reference", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Reference" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Records" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "List Schema Records", + "tags": [ + "v2: records" + ] + } + }, + "/schemas/{schema_id}/records:bulk-upsert": { + "post": { + "operationId": "bulk_upsert_schema_records_schemas__schema_id__records_bulk_upsert_post", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/RecordsBulkUpsert" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Records" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Bulk Upsert Schema Records", + "tags": [ + "v2: records" + ] + } + }, + "/schemas/{schema_id}/records:search": { + "post": { + "description": "Full-text (BM25) + scalar-filter search over a schema's records.\n\nLance supplies matching record ids and scores; payloads are hydrated from Postgres\n(the source of truth) and returned in the engine's hit order. `total` is the engine's\ntotal match count, which may exceed the returned page.", + "operationId": "search_schema_records_schemas__schema_id__records_search_post", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/RecordSearchQuery" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Records" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Search Schema Records", + "tags": [ + "v2: records" + ] + } + }, + "/schemas/{schema_id}/versions": { + "get": { + "operationId": "list_schema_versions_schemas__schema_id__versions_get", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "items": { + "$ref": "#/components/schemas/SchemaVersionRead" + }, + "title": "Response List Schema Versions Schemas Schema Id Versions Get", + "type": "array" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "List Schema Versions", + "tags": [ + "v2: schemas" + ] + }, + "post": { + "operationId": "publish_schema_version_schemas__schema_id__versions_post", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaVersionCreate" + } + } + }, + "required": true + }, + "responses": { + "201": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaVersionRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Publish Schema Version", + "tags": [ + "v2: schemas" + ] + } + }, + "/schemas/{schema_id}/versions/{version}": { + "get": { + "operationId": "get_schema_version_schemas__schema_id__versions__version__get", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + }, + { + "in": "path", + "name": "version", + "required": true, + "schema": { + "title": "Version", + "type": "integer" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SchemaVersionRead" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Get Schema Version", + "tags": [ + "v2: schemas" + ] + } + }, + "/schemas/{schema_id}:rebuild-index": { + "post": { + "description": "Drop and repopulate the schema's Lance table from Postgres (the recovery path).\n\nUnlike the write-time sync hooks, this surfaces engine errors to the caller \u2014 the\noperator explicitly asked to rebuild. For large schemas the rebuild may take tens of\nseconds; consider running as a background job (via the CLI) if timeouts are a concern.", + "operationId": "rebuild_schema_index_schemas__schema_id__rebuild_index_post", + "parameters": [ + { + "in": "path", + "name": "schema_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Schema Id", + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "additionalProperties": { + "type": "integer" + }, + "title": "Response Rebuild Schema Index Schemas Schema Id Rebuild Index Post", + "type": "object" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "security": [ + { + "APIKeyHeader": [] + }, + { + "HTTPBearer": [] + } + ], + "summary": "Rebuild Schema Index", + "tags": [ + "v2: records" + ] + } + }, + "/token": { + "post": { + "operationId": "create_token_token_post", + "requestBody": { + "content": { + "application/x-www-form-urlencoded": { + "schema": { + "$ref": "#/components/schemas/Body_create_token_token_post" + } + } + }, + "required": true + }, + "responses": { + "201": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Token" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "summary": "Create Token", + "tags": [ + "Authentication" + ] + } + }, + "/token/refresh": { + "post": { + "description": "Refresh an access token using a valid refresh token.\nThis endpoint does not require database access, improving reliability.", + "operationId": "refresh_token_token_refresh_post", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/RefreshTokenRequest" + } + } + }, + "required": true + }, + "responses": { + "201": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Token" + } + } + }, + "description": "Successful Response" + }, + "400": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::MissingDatasetRecordsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Bad Request" + }, + "403": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ForbiddenOperationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Forbidden" + }, + "404": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityNotFoundError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Not Found" + }, + "409": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::EntityAlreadyExistsError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Conflict" + }, + "422": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "extralit.api.errors::ValidationError", + "params": { + "extra": "error parameters" + } + } + } + } + }, + "description": "Unprocessable Content" + }, + "500": { + "content": { + "application/json": { + "example": { + "detail": { + "code": "builtins.TypeError" + } + } + } + }, + "description": "Internal Server Error" + } + }, + "summary": "Refresh Token", + "tags": [ + "Authentication" + ] + } + } + } +} diff --git a/extralit/src/extralit/v2/_sync.py b/extralit/src/extralit/v2/_sync.py new file mode 100644 index 000000000..282887d17 --- /dev/null +++ b/extralit/src/extralit/v2/_sync.py @@ -0,0 +1,72 @@ +from __future__ import annotations + +import asyncio +import functools +import inspect +import threading + +from extralit.v2.client import AsyncClient +from extralit.v2.resources._base import ResourceBase + +_RESOURCE_NAMES = ("schemas", "questions", "records", "suggestions", "projections", "responses") + + +class _Portal: + """A background-thread event loop. Sync mirrors submit coroutines here, so they + work even when the calling thread already runs a loop (Jupyter).""" + + def __init__(self): + self._loop = asyncio.new_event_loop() + self._thread = threading.Thread(target=self._run, name="extralit-v2-portal", daemon=True) + self._thread.start() + + def _run(self) -> None: + asyncio.set_event_loop(self._loop) + self._loop.run_forever() + + def run(self, coroutine): + return asyncio.run_coroutine_threadsafe(coroutine, self._loop).result() + + def stop(self) -> None: + self._loop.call_soon_threadsafe(self._loop.stop) + self._thread.join() + + +class _SyncProxy: + """Wraps a resource: coroutine methods become sync calls through the portal; + everything else passes through unchanged. Mirrors are mechanical — never hand-written.""" + + def __init__(self, target: ResourceBase, portal: _Portal): + self._target = target + self._portal = portal + + def __getattr__(self, name: str): + attribute = getattr(self._target, name) + if inspect.iscoroutinefunction(attribute): + + @functools.wraps(attribute) + def call(*args, **kwargs): + return self._portal.run(attribute(*args, **kwargs)) + + return call + return attribute + + +class Client: + """Sync facade over AsyncClient — same constructor, same resource surface.""" + + def __init__(self, *args, **kwargs): + self._portal = _Portal() + self._async = AsyncClient(*args, **kwargs) + for name in _RESOURCE_NAMES: + setattr(self, name, _SyncProxy(getattr(self._async, name), self._portal)) + + def close(self) -> None: + self._portal.run(self._async.aclose()) + self._portal.stop() + + def __enter__(self) -> Client: + return self + + def __exit__(self, *exc_info) -> None: + self.close() diff --git a/extralit/src/extralit/v2/cli/__init__.py b/extralit/src/extralit/v2/cli/__init__.py new file mode 100644 index 000000000..4ade23886 --- /dev/null +++ b/extralit/src/extralit/v2/cli/__init__.py @@ -0,0 +1,16 @@ +from __future__ import annotations + +import typer + + +def add_v2_commands(app: typer.Typer) -> None: + """Register v2 verbs at the TOP level of the extralit CLI (no `v2` prefix). + v2 owns these names; the v1 `schemas` subcommand is deliberately replaced.""" + from extralit.v2.cli import projection, questions, records, references, schemas, suggestions + + app.add_typer(schemas.app, name="schemas") + app.add_typer(records.app, name="records") + app.add_typer(questions.app, name="questions") + app.add_typer(suggestions.app, name="suggestions") + app.add_typer(projection.app, name="projection") + app.add_typer(references.app, name="references") diff --git a/extralit/src/extralit/v2/cli/_context.py b/extralit/src/extralit/v2/cli/_context.py new file mode 100644 index 000000000..0cd06e945 --- /dev/null +++ b/extralit/src/extralit/v2/cli/_context.py @@ -0,0 +1,14 @@ +from __future__ import annotations + +from extralit.v2._sync import Client +from extralit.v2.cli._output import fail + + +def get_client() -> Client: + """Non-interactive by construction: args come from env or the credentials file; + a missing configuration is a structured error, never a prompt.""" + try: + return Client() + except ValueError as error: + fail(error) + raise # unreachable; keeps type-checkers happy diff --git a/extralit/src/extralit/v2/cli/_output.py b/extralit/src/extralit/v2/cli/_output.py new file mode 100644 index 000000000..34dad029a --- /dev/null +++ b/extralit/src/extralit/v2/cli/_output.py @@ -0,0 +1,55 @@ +from __future__ import annotations + +import functools +import json +import sys +from typing import Any + +import typer + +from extralit.v2._api._errors import V2APIError, ValidationError + + +def to_jsonable(data: Any) -> Any: + if hasattr(data, "model_dump"): + return data.model_dump(mode="json") + if isinstance(data, list): + return [to_jsonable(item) for item in data] + if isinstance(data, dict): + return {key: to_jsonable(value) for key, value in data.items()} + return data + + +def emit(data: Any, json_flag: bool) -> None: + """JSON-first: --json forces JSON; a non-TTY stdout (pipes, agents, CI) defaults to it. + Humans at a terminal get Rich output.""" + if json_flag or not sys.stdout.isatty(): + typer.echo(json.dumps(to_jsonable(data), default=str)) + return + from rich.console import Console # lazy: JSON path must not pay for rich + + Console().print(to_jsonable(data)) + + +def fail(error: Exception) -> None: + status = getattr(error, "status_code", None) + payload = { + "error": {"type": type(error).__name__, "status": status, "detail": str(getattr(error, "detail", error))} + } + typer.echo(json.dumps(payload, default=str), err=True) + raise typer.Exit(code=3 if isinstance(error, ValidationError) else 1) + + +def handle_errors(fn): + @functools.wraps(fn) + def wrapper(*args, **kwargs): + try: + return fn(*args, **kwargs) + except V2APIError as error: + fail(error) + except (ValueError, OSError) as error: + # Catches malformed UUID/JSON input (ValueError, JSONDecodeError) and + # missing/unreadable --file paths (OSError/FileNotFoundError) + fail(error) + + return wrapper diff --git a/extralit/src/extralit/v2/cli/projection.py b/extralit/src/extralit/v2/cli/projection.py new file mode 100644 index 000000000..ba3354112 --- /dev/null +++ b/extralit/src/extralit/v2/cli/projection.py @@ -0,0 +1,23 @@ +from __future__ import annotations + +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Read v2 projections (response-or-suggestion per question)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("get") +@handle_errors +def get_projection( + reference: str = typer.Argument(..., help="Reference (DOI/URL/filename; slashes fine)"), + workspace_id: str = typer.Option(..., "--workspace-id"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit(client.projections.get(UUID(workspace_id), reference), json_flag) diff --git a/extralit/src/extralit/v2/cli/questions.py b/extralit/src/extralit/v2/cli/questions.py new file mode 100644 index 000000000..068ae9e08 --- /dev/null +++ b/extralit/src/extralit/v2/cli/questions.py @@ -0,0 +1,19 @@ +from __future__ import annotations + +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Inspect v2 questions (column-bound)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("list") +@handle_errors +def list_questions(schema_id: str = typer.Argument(...), json_flag: bool = JSON_FLAG): + with get_client() as client: + emit(client.questions.list(UUID(schema_id)), json_flag) diff --git a/extralit/src/extralit/v2/cli/records.py b/extralit/src/extralit/v2/cli/records.py new file mode 100644 index 000000000..da4350941 --- /dev/null +++ b/extralit/src/extralit/v2/cli/records.py @@ -0,0 +1,100 @@ +from __future__ import annotations + +import json +import sys +from typing import Optional +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Manage v2 records (schema-version-pinned, reference-keyed)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +def _parse_filter(raw: str) -> tuple: + """col:op:value — value is JSON-decoded when possible ('age:ge:18' -> int 18).""" + column, op, value = raw.split(":", 2) + try: + value = json.loads(value) + except ValueError: + pass # keep as string + return (column, op, value) + + +def _read_jsonl(file: Optional[str]) -> list[dict]: + stream = sys.stdin if file in (None, "-") else open(file, encoding="utf-8") + try: + return [json.loads(line) for line in stream if line.strip()] + finally: + if stream is not sys.stdin: + stream.close() + + +@app.command("upsert") +@handle_errors +def upsert_records( + schema_id: str = typer.Argument(...), + file: Optional[str] = typer.Option(None, "--file", help="JSONL file of items; '-' or omitted reads stdin"), + reference: Optional[str] = typer.Option(None, "--reference", help="Reference applied to items lacking one"), + json_flag: bool = JSON_FLAG, +): + items = _read_jsonl(file) + with get_client() as client: + emit(client.records.bulk_upsert(UUID(schema_id), items, reference=reference), json_flag) + + +@app.command("search") +@handle_errors +def search_records( + schema_id: str = typer.Argument(...), + text: Optional[str] = typer.Option(None, "--text"), + filters: list[str] = typer.Option([], "--filter", help="col:op:value (op: eq|in|ge|le); repeatable"), + offset: int = typer.Option(0, "--offset"), + limit: int = typer.Option(50, "--limit"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit( + client.records.search( + UUID(schema_id), + text=text, + filters=[_parse_filter(raw) for raw in filters], + offset=offset, + limit=limit, + ), + json_flag, + ) + + +@app.command("list") +@handle_errors +def list_records( + schema_id: str = typer.Argument(...), + status: Optional[str] = typer.Option(None, "--status"), + reference: Optional[str] = typer.Option(None, "--reference"), + offset: int = typer.Option(0, "--offset"), + limit: int = typer.Option(50, "--limit"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit( + client.records.list(UUID(schema_id), status=status, reference=reference, offset=offset, limit=limit), + json_flag, + ) + + +@app.command("delete") +@handle_errors +def delete_records( + schema_id: str = typer.Argument(...), + ids: str = typer.Option(..., "--ids", help="Comma-separated record ids"), + json_flag: bool = JSON_FLAG, +): + record_ids = ids.split(",") + with get_client() as client: + client.records.delete(UUID(schema_id), record_ids) + emit({"deleted": len(record_ids)}, json_flag) diff --git a/extralit/src/extralit/v2/cli/references.py b/extralit/src/extralit/v2/cli/references.py new file mode 100644 index 000000000..466ec0674 --- /dev/null +++ b/extralit/src/extralit/v2/cli/references.py @@ -0,0 +1,23 @@ +from __future__ import annotations + +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Cross-schema reference views", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("get") +@handle_errors +def get_reference( + reference: str = typer.Argument(..., help="Reference (DOI/URL/filename; slashes fine)"), + workspace_id: str = typer.Option(..., "--workspace-id"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit(client.records.get_reference(UUID(workspace_id), reference), json_flag) diff --git a/extralit/src/extralit/v2/cli/schemas.py b/extralit/src/extralit/v2/cli/schemas.py new file mode 100644 index 000000000..1f01fa80e --- /dev/null +++ b/extralit/src/extralit/v2/cli/schemas.py @@ -0,0 +1,73 @@ +from __future__ import annotations + +import json +from pathlib import Path +from typing import Optional +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Manage v2 schemas (Pandera, versioned)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("list") +@handle_errors +def list_schemas( + workspace_id: str = typer.Option(..., "--workspace-id", help="Workspace UUID"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit(client.schemas.list(UUID(workspace_id)), json_flag) + + +@app.command("get") +@handle_errors +def get_schema(schema_id: str = typer.Argument(...), json_flag: bool = JSON_FLAG): + with get_client() as client: + emit(client.schemas.get(UUID(schema_id)), json_flag) + + +@app.command("create") +@handle_errors +def create_schema( + name: str = typer.Argument(...), + workspace_id: str = typer.Option(..., "--workspace-id"), + settings: Optional[str] = typer.Option(None, "--settings", help="Settings as a JSON object"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit( + client.schemas.create(UUID(workspace_id), name, settings=json.loads(settings) if settings else None), + json_flag, + ) + + +@app.command("publish") +@handle_errors +def publish_version( + schema_id: str = typer.Argument(...), + file: Path = typer.Option(..., "--file", help="Pandera DataFrameSchema JSON (schema.to_json() output)"), + review_widgets: Optional[str] = typer.Option(None, "--review-widgets", help="JSON: {column: widget config}"), + json_flag: bool = JSON_FLAG, +): + with get_client() as client: + emit( + client.schemas.publish( + UUID(schema_id), + file.read_text(), + review_widgets=json.loads(review_widgets) if review_widgets else None, + ), + json_flag, + ) + + +@app.command("versions") +@handle_errors +def list_versions(schema_id: str = typer.Argument(...), json_flag: bool = JSON_FLAG): + with get_client() as client: + emit(client.schemas.versions(UUID(schema_id)), json_flag) diff --git a/extralit/src/extralit/v2/cli/suggestions.py b/extralit/src/extralit/v2/cli/suggestions.py new file mode 100644 index 000000000..be3645d0d --- /dev/null +++ b/extralit/src/extralit/v2/cli/suggestions.py @@ -0,0 +1,44 @@ +from __future__ import annotations + +import json +from typing import Optional +from uuid import UUID + +import typer + +from extralit.v2.cli._context import get_client +from extralit.v2.cli._output import emit, handle_errors + +app = typer.Typer(help="Write v2 suggestions (per record x question)", no_args_is_help=True) + +JSON_FLAG = typer.Option(False, "--json", help="Force JSON output (auto when stdout is not a TTY)") + + +@app.command("upsert") +@handle_errors +def upsert_suggestion( + record_id: str = typer.Argument(...), + question_id: Optional[str] = typer.Option(None, "--question-id", help="Question UUID"), + question: Optional[str] = typer.Option(None, "--question", help="Question NAME (needs --schema-id)"), + schema_id: Optional[str] = typer.Option(None, "--schema-id", help="Schema UUID for name resolution"), + value: str = typer.Option(..., "--value", help="Suggested value as JSON (e.g. '\"120\"' or '[1,2]')"), + score: Optional[float] = typer.Option(None, "--score"), + agent: Optional[str] = typer.Option(None, "--agent"), + json_flag: bool = JSON_FLAG, +): + if question_id is None and question is None: + raise typer.BadParameter("pass --question-id or --question") + if question is not None and question_id is None and schema_id is None: + raise typer.BadParameter("--question (a name) requires --schema-id to resolve against") + with get_client() as client: + emit( + client.suggestions.upsert( + record_id, + question_id or question, + json.loads(value), + score=score, + agent=agent, + schema_id=UUID(schema_id) if schema_id else None, + ), + json_flag, + ) diff --git a/extralit/src/extralit/v2/client.py b/extralit/src/extralit/v2/client.py new file mode 100644 index 000000000..4cf10caea --- /dev/null +++ b/extralit/src/extralit/v2/client.py @@ -0,0 +1,64 @@ +from __future__ import annotations + +import os +from typing import Optional + +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.resources import Projections, Questions, Records, Responses, Schemas, Suggestions + + +def _credentials_fallback() -> tuple: + # The single, documented v1 import: the credentials file outlives v1 retirement. + from extralit.client.login import ExtralitCredentials + + if not ExtralitCredentials.exists(): + return None, None + try: + credentials = ExtralitCredentials.load() + return credentials.api_url, credentials.api_key + except (OSError, KeyError, ValueError): + return None, None + + +class AsyncClient: + """Async-native /api/v2 client. Resolution order for connection settings: + explicit args > EXTRALIT_API_URL / EXTRALIT_API_KEY env > ~/.extralit/credentials.json.""" + + def __init__( + self, + api_url: Optional[str] = None, + api_key: Optional[str] = None, + username: Optional[str] = None, + password: Optional[str] = None, + timeout: float = 60.0, + retries: int = 5, + ): + api_url = api_url or os.environ.get("EXTRALIT_API_URL") + api_key = api_key or os.environ.get("EXTRALIT_API_KEY") + if not api_url or (not api_key and not username): + file_url, file_key = _credentials_fallback() + api_url = api_url or file_url + if not api_key and not username: + api_key = file_key + if not api_url: + raise ValueError("api_url is required (argument, EXTRALIT_API_URL, or ~/.extralit/credentials.json)") + if not api_key and not username: + raise ValueError("credentials required: api_key or username/password") + self._transport = AsyncTransport( + api_url, api_key=api_key, username=username, password=password, timeout=timeout, retries=retries + ) + self.schemas = Schemas(self._transport) + self.questions = Questions(self._transport) + self.records = Records(self._transport) + self.suggestions = Suggestions(self._transport, self.questions) + self.projections = Projections(self._transport) + self.responses = Responses(self._transport) + + async def aclose(self) -> None: + await self._transport.aclose() + + async def __aenter__(self) -> AsyncClient: + return self + + async def __aexit__(self, *exc_info) -> None: + await self.aclose() diff --git a/extralit/src/extralit/v2/models.py b/extralit/src/extralit/v2/models.py new file mode 100644 index 000000000..1a889701c --- /dev/null +++ b/extralit/src/extralit/v2/models.py @@ -0,0 +1,81 @@ +from typing import Optional + +from pydantic import BaseModel + +from extralit.v2._api._generated import ( + ProjectionCell, + ProjectionRecord, + ProjectionView, + QuestionRead, + RecordRead, + ReferenceGroup, + ReferenceView, + ResponseRead, + SchemaRead, + SchemaVersionRead, + SuggestionRead, +) + +__all__ = [ + "ProjectionCell", + "ProjectionRecord", + "ProjectionView", + "Question", + "Record", + "ReferenceGroup", + "ReferenceView", + "Response", + "Schema", + "SchemaVersion", + "SearchPage", + "Suggestion", + "unwrap_response_values", + "wrap_response_values", +] + + +class Schema(SchemaRead): + pass + + +class SchemaVersion(SchemaVersionRead): + def find_column(self, name: str) -> Optional[dict]: + for column in self.columns_cache: + if column.get("name") == name: + return column + return None + + +class Record(RecordRead): + pass + + +class Question(QuestionRead): + pass + + +class Suggestion(SuggestionRead): + pass + + +def wrap_response_values(values: dict) -> dict: + """Server stores response values double-wrapped: {question_name: {"value": ...}}.""" + return {name: {"value": value} for name, value in values.items()} + + +def unwrap_response_values(values: Optional[dict]) -> dict: + return {name: cell.get("value") if isinstance(cell, dict) else cell for name, cell in (values or {}).items()} + + +class Response(ResponseRead): + @property + def unwrapped_values(self) -> dict: + return unwrap_response_values(self.values) + + +class SearchPage(BaseModel): + """One page of records. `total` is approximate: stale index ids are skipped and + FTS saturates (~10k) — never present it as an exact count.""" + + items: "list[Record]" + total: int diff --git a/extralit/src/extralit/v2/resources/__init__.py b/extralit/src/extralit/v2/resources/__init__.py new file mode 100644 index 000000000..3421d6dde --- /dev/null +++ b/extralit/src/extralit/v2/resources/__init__.py @@ -0,0 +1,7 @@ +from extralit.v2.resources._annotation import Responses, Suggestions +from extralit.v2.resources._projections import Projections +from extralit.v2.resources._questions import Questions +from extralit.v2.resources._records import Records +from extralit.v2.resources._schemas import Schemas + +__all__ = ["Projections", "Questions", "Records", "Responses", "Schemas", "Suggestions"] diff --git a/extralit/src/extralit/v2/resources/_annotation.py b/extralit/src/extralit/v2/resources/_annotation.py new file mode 100644 index 000000000..d8268b1fd --- /dev/null +++ b/extralit/src/extralit/v2/resources/_annotation.py @@ -0,0 +1,63 @@ +from __future__ import annotations + +import uuid +from typing import Any, Optional + +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.models import Response, Suggestion +from extralit.v2.resources._base import ResourceBase +from extralit.v2.resources._questions import Questions + + +def _as_question_id(question: Any) -> Optional[str]: + try: + return str(uuid.UUID(str(question))) + except (ValueError, AttributeError, TypeError): + return None + + +class Suggestions(ResourceBase): + def __init__(self, transport: AsyncTransport, questions: Questions): + super().__init__(transport) + self._questions = questions + + async def upsert( + self, + record: Any, + question: Any, + value: Any, + *, + score: Optional[Any] = None, + agent: Optional[str] = None, + type: Optional[str] = None, + schema_id: Optional[Any] = None, + ) -> Suggestion: + """Upsert one suggestion per (record, question). Suggestions key by question ID on + the wire, but callers may pass a question NAME — resolved via the record's schema.""" + record_id = getattr(record, "id", record) + question_id = _as_question_id(question) + if question_id is None: # a name: resolve against the record's schema + resolve_schema = getattr(record, "schema_id", None) or schema_id + if resolve_schema is None: + raise ValueError("resolving a question name requires a Record object or an explicit schema_id=") + question_id = str(await self._questions.id_for(resolve_schema, question)) + body: dict = {"question_id": question_id, "value": value} + if score is not None: + body["score"] = score + if agent is not None: + body["agent"] = agent + if type is not None: + body["type"] = type + payload = await self._transport.request("PUT", f"/records/{record_id}/suggestions", json=body) + return Suggestion.model_validate(payload) + + async def list(self, record_id) -> list[Suggestion]: + payload = await self._transport.request("GET", f"/records/{record_id}/suggestions") + return [Suggestion.model_validate(item) for item in payload["items"]] + + +class Responses(ResourceBase): + async def get(self, record_id) -> Optional[Response]: + """GET returns literal `null` with 200 (not 404) when no response exists yet.""" + payload = await self._transport.request("GET", f"/records/{record_id}/responses") + return None if payload is None else Response.model_validate(payload) diff --git a/extralit/src/extralit/v2/resources/_base.py b/extralit/src/extralit/v2/resources/_base.py new file mode 100644 index 000000000..7d958d65c --- /dev/null +++ b/extralit/src/extralit/v2/resources/_base.py @@ -0,0 +1,6 @@ +from extralit.v2._api._transport import AsyncTransport + + +class ResourceBase: + def __init__(self, transport: AsyncTransport): + self._transport = transport diff --git a/extralit/src/extralit/v2/resources/_projections.py b/extralit/src/extralit/v2/resources/_projections.py new file mode 100644 index 000000000..db7e51521 --- /dev/null +++ b/extralit/src/extralit/v2/resources/_projections.py @@ -0,0 +1,12 @@ +from extralit.v2.models import ProjectionView +from extralit.v2.resources._base import ResourceBase + + +class Projections(ResourceBase): + async def get(self, workspace_id, reference: str) -> ProjectionView: + """Response-or-suggestion per question for every record sharing this reference. + Slashes stay raw: the server route is /projection/references/{reference:path}.""" + payload = await self._transport.request( + "GET", f"/projection/references/{reference}", params={"workspace_id": str(workspace_id)} + ) + return ProjectionView.model_validate(payload) diff --git a/extralit/src/extralit/v2/resources/_questions.py b/extralit/src/extralit/v2/resources/_questions.py new file mode 100644 index 000000000..a2ab68528 --- /dev/null +++ b/extralit/src/extralit/v2/resources/_questions.py @@ -0,0 +1,37 @@ +from __future__ import annotations + +from uuid import UUID + +from extralit.v2._api._errors import NotFoundError +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.models import Question +from extralit.v2.resources._base import ResourceBase + + +class Questions(ResourceBase): + """Callers address questions by NAME; the server keys suggestions by question ID + (cells/response values key by name). This resource owns that join via a cached map.""" + + def __init__(self, transport: AsyncTransport): + super().__init__(transport) + self._maps: dict = {} # str(schema_id) -> {name: UUID} + + async def list(self, schema_id) -> list[Question]: + payload = await self._transport.request("GET", f"/schemas/{schema_id}/questions") + items = [Question.model_validate(item) for item in payload["items"]] + self._maps[str(schema_id)] = {q.name: q.id for q in items} + return items + + async def get(self, question_id) -> Question: + return Question.model_validate(await self._transport.request("GET", f"/questions/{question_id}")) + + async def id_for(self, schema_id, name: str) -> UUID: + key = str(schema_id) + if key not in self._maps or name not in self._maps[key]: + await self.list(schema_id) # refetch once: the question may be newly created + if name not in self._maps.get(key, {}): + raise NotFoundError(404, f"question named {name!r} not found in schema {schema_id}") + return self._maps[key][name] + + def invalidate(self, schema_id) -> None: + self._maps.pop(str(schema_id), None) diff --git a/extralit/src/extralit/v2/resources/_records.py b/extralit/src/extralit/v2/resources/_records.py new file mode 100644 index 000000000..4997e4485 --- /dev/null +++ b/extralit/src/extralit/v2/resources/_records.py @@ -0,0 +1,122 @@ +from __future__ import annotations + +import asyncio +from typing import Any, Callable, Optional + +from extralit.v2.models import Record, ReferenceView, SearchPage +from extralit.v2.resources._base import ResourceBase + +BULK_UPSERT_MAX_ITEMS = 500 # server cap (RECORDS_BULK_UPSERT_MAX_ITEMS) +DELETE_MAX_IDS = 100 # server cap (DELETE_RECORDS_LIMIT) + + +def _normalize_items(items: Any, reference: Optional[str]) -> list[dict]: + if hasattr(items, "to_dict") and hasattr(items, "columns"): # pandas.DataFrame, kept lazy + items = items.to_dict(orient="records") + normalized = [] + for item in items: + if "fields" in item: + entry = dict(item) + else: + fields = dict(item) + entry = {"fields": fields} + if "reference" in fields: + entry["reference"] = fields.pop("reference") + if reference is not None: + entry.setdefault("reference", reference) + if "reference" not in entry: + raise ValueError("every record needs a reference (per-item or via the reference= argument)") + normalized.append(entry) + return normalized + + +def _normalize_filters(filters: Optional[list]) -> list[dict]: + normalized = [] + for item in filters or []: + if isinstance(item, dict): + normalized.append({"column": item["column"], "op": item["op"], "value": item["value"]}) + else: + column, op, value = item + normalized.append({"column": column, "op": op, "value": value}) + return normalized + + +class Records(ResourceBase): + async def bulk_upsert( + self, + schema_id, + items: Any, + *, + reference: Optional[str] = None, + max_concurrency: int = 4, + on_progress: Optional[Callable] = None, + ) -> list[Record]: + """Idempotent on external_id; metadata is patch-like (omitted keys preserved). + Auto-chunks at the server's 500-item cap; chunks fly concurrently but the + returned list preserves input order.""" + normalized = _normalize_items(items, reference) + chunks = [normalized[i : i + BULK_UPSERT_MAX_ITEMS] for i in range(0, len(normalized), BULK_UPSERT_MAX_ITEMS)] + results: list = [None] * len(chunks) + total = len(normalized) + done = 0 + semaphore = asyncio.Semaphore(max_concurrency) + + async def _run(index: int, chunk: list[dict]) -> None: + nonlocal done + async with semaphore: + payload = await self._transport.request( + "POST", f"/schemas/{schema_id}/records:bulk-upsert", json={"items": chunk} + ) + results[index] = payload["items"] + done += len(chunk) + if on_progress: + on_progress(done, total) + + await asyncio.gather(*(_run(i, c) for i, c in enumerate(chunks))) + return [Record.model_validate(item) for chunk in results for item in chunk] + + async def search( + self, + schema_id, + *, + text: Optional[str] = None, + filters: Optional[list] = None, + offset: int = 0, + limit: int = 50, + ) -> SearchPage: + payload = await self._transport.request( + "POST", + f"/schemas/{schema_id}/records:search", + json={"text": text, "filters": _normalize_filters(filters), "offset": offset, "limit": limit}, + ) + return SearchPage(items=[Record.model_validate(i) for i in payload["items"]], total=payload["total"]) + + async def list( + self, + schema_id, + *, + offset: int = 0, + limit: int = 50, + status: Optional[str] = None, + reference: Optional[str] = None, + ) -> SearchPage: + params: dict = {"offset": offset, "limit": limit} + if status is not None: + params["status"] = status + if reference is not None: + params["reference"] = reference + payload = await self._transport.request("GET", f"/schemas/{schema_id}/records", params=params) + return SearchPage(items=[Record.model_validate(i) for i in payload["items"]], total=payload["total"]) + + async def delete(self, schema_id, ids: list) -> None: + id_strings = [str(record_id) for record_id in ids] + for start in range(0, len(id_strings), DELETE_MAX_IDS): + chunk = id_strings[start : start + DELETE_MAX_IDS] + await self._transport.request("DELETE", f"/schemas/{schema_id}/records", params={"ids": ",".join(chunk)}) + + async def get_reference(self, workspace_id, reference: str) -> ReferenceView: + # Slashes stay raw: the server route is /references/{reference:path}. + payload = await self._transport.request( + "GET", f"/references/{reference}", params={"workspace_id": str(workspace_id)} + ) + return ReferenceView.model_validate(payload) diff --git a/extralit/src/extralit/v2/resources/_schemas.py b/extralit/src/extralit/v2/resources/_schemas.py new file mode 100644 index 000000000..fe9506dcf --- /dev/null +++ b/extralit/src/extralit/v2/resources/_schemas.py @@ -0,0 +1,71 @@ +from __future__ import annotations + +from typing import Any, Optional + +from extralit.v2._api._errors import NotFoundError +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.models import Schema, SchemaVersion +from extralit.v2.resources._base import ResourceBase + + +class Schemas(ResourceBase): + def __init__(self, transport: AsyncTransport): + super().__init__(transport) + self._version_cache: dict = {} # (schema_id, version) -> SchemaVersion; versions are immutable + + async def create(self, workspace_id, name: str, settings: Optional[dict] = None) -> Schema: + payload = await self._transport.request( + "POST", + "/schemas", + json={"name": name, "workspace_id": str(workspace_id), "settings": settings or {}}, + ) + return Schema.model_validate(payload) + + async def list(self, workspace_id) -> list[Schema]: + payload = await self._transport.request("GET", "/schemas", params={"workspace_id": str(workspace_id)}) + return [Schema.model_validate(item) for item in payload["items"]] + + async def get(self, schema_id) -> Schema: + return Schema.model_validate(await self._transport.request("GET", f"/schemas/{schema_id}")) + + async def get_by_name(self, workspace_id, name: str) -> Schema: + for schema in await self.list(workspace_id): + if schema.name == name: + return schema + raise NotFoundError(404, f"schema named {name!r} not found in workspace {workspace_id}") + + async def update(self, schema_id, *, name: Optional[str] = None, settings: Optional[dict] = None) -> Schema: + body: dict = {} + if name is not None: + body["name"] = name + if settings is not None: + body["settings"] = settings + return Schema.model_validate(await self._transport.request("PUT", f"/schemas/{schema_id}", json=body)) + + async def publish(self, schema_id, schema: Any, review_widgets: Optional[dict] = None) -> SchemaVersion: + """Publish a new schema version. `schema` is a pandera DataFrameSchema (anything with + .to_json()) or the already-serialized JSON string. review_widgets ride out-of-band + because pandera's to_json() drops Column.metadata.""" + body = schema.to_json() if hasattr(schema, "to_json") else schema + payload = await self._transport.request( + "POST", + f"/schemas/{schema_id}/versions", + json={"body": body, "review_widgets": review_widgets or {}}, + ) + version = SchemaVersion.model_validate(payload) + self._version_cache[(str(schema_id), version.version)] = version + return version + + async def versions(self, schema_id) -> list[SchemaVersion]: + payload = await self._transport.request("GET", f"/schemas/{schema_id}/versions") + return [SchemaVersion.model_validate(item) for item in payload] + + async def get_version(self, schema_id, version: int) -> SchemaVersion: + key = (str(schema_id), version) + if key not in self._version_cache: + payload = await self._transport.request("GET", f"/schemas/{schema_id}/versions/{version}") + self._version_cache[key] = SchemaVersion.model_validate(payload) + return self._version_cache[key] + + async def columns(self, schema_id) -> list[dict]: + return await self._transport.request("GET", f"/schemas/{schema_id}/columns") diff --git a/extralit/tests/unit/cli/test_cli_schemas.py b/extralit/tests/unit/cli/test_cli_schemas.py deleted file mode 100644 index 666e2f249..000000000 --- a/extralit/tests/unit/cli/test_cli_schemas.py +++ /dev/null @@ -1,131 +0,0 @@ -from pathlib import Path -from unittest.mock import patch - -import pytest -from typer.testing import CliRunner - -from extralit.cli.app import app - - -@pytest.fixture -def runner(): - """Fixture providing a CLI runner.""" - return CliRunner() - - -@patch("rich.console.Console.print") -@pytest.mark.skip(reason="Test temporarily disabled") -def test_schemas_list(mock_print, runner): - """Test the 'list schemas' command functionality.""" - result = runner.invoke( - app, - [ - "schemas", - "list", - "--workspace", - "research", - ], - ) - - assert result.exit_code == 0 - mock_print.assert_called_once() - - # Since we're using mock data, just verify that print was called - # In a more complete test, we could mock the API call and check the output - - -@patch("rich.console.Console.print") -@pytest.mark.skip(reason="Test temporarily disabled") -def test_schemas_list_with_versions(mock_print, runner): - """Test the 'list schemas' command with versions flag.""" - result = runner.invoke(app, ["schemas", "list", "--workspace", "research", "--name", "customer-feedback"]) - - assert result.exit_code == 0 - mock_print.assert_called_once() - - -@patch("rich.console.Console.print") -@pytest.mark.skip(reason="Test temporarily disabled") -def test_schemas_list_with_csv_export(mock_print, runner): - """Test the 'list schemas' command with CSV export.""" - with runner.isolated_filesystem(): - result = runner.invoke( - app, - [ - "schemas", - "list", - "--workspace", - "research", - ], - ) - - assert result.exit_code == 0 - mock_print.assert_called_once() - - -@patch("extralit.cli.schemas.upload.upload_schemas") -@pytest.mark.skip(reason="Test temporarily disabled") -def test_schemas_upload(mock_upload_schemas, runner): - """Test the 'upload schemas' command functionality.""" - # Set up the mock to return a successful result - mock_upload_schemas.return_value = None # The function doesn't return anything - - with runner.isolated_filesystem(): - # Create a test directory with schema files - import os - - os.makedirs("schemas") - with open("schemas/schema1.json", "w") as f: - f.write('{"name": "test_schema"}') - - # Call the command with the appropriate parameters - result = runner.invoke( - app, - ["schemas", "upload", "--workspace", "research", "schemas", "--overwrite", "--exclude", "excluded_schema"], - ) - - # Verify the command executed successfully - assert result.exit_code == 0 - - # Verify the upload_schemas function was called with the correct parameters - mock_upload_schemas.assert_called_once() - args, kwargs = mock_upload_schemas.call_args - - # Check that the context was passed - assert args[0] is not None - # Check that the directory path was passed - assert isinstance(args[1], Path) - assert str(args[1]).endswith("schemas") - # Check that overwrite was set to True - assert args[2] is True - # Check that exclude was set correctly - assert args[3] == ["excluded_schema"] - - -@patch("rich.console.Console.print") -@pytest.mark.skip(reason="Test temporarily disabled") -def test_schemas_delete(mock_print, runner): - """Test the 'delete schema' command functionality.""" - # Simulate user confirming the deletion - result = runner.invoke(app, ["schemas", "delete", "--workspace", "research", "schema1"], input="y\n") - - assert result.exit_code == 0 - mock_print.assert_called_once() - - # Verify that Console.print was called - assert mock_print.called - - # For now, we'll just verify that the command completed successfully - # In a real test, we would need to mock the API call and verify the response - - -@patch("rich.console.Console.print") -def test_schemas_delete_nonexistent(mock_print, runner): - """Test deleting a schema that doesn't exist (should fail).""" - # Make print raise ValueError to simulate schema not found - mock_print.side_effect = ValueError("Schema not found") - - result = runner.invoke(app, ["schemas", "delete", "--workspace", "research", "nonexistent_schema"]) - - # Should exit with code 1 due to ValueError - assert result.exit_code == 1 diff --git a/extralit/tests/unit/v2/__init__.py b/extralit/tests/unit/v2/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/extralit/tests/unit/v2/cli/__init__.py b/extralit/tests/unit/v2/cli/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/extralit/tests/unit/v2/cli/test_cli_annotation.py b/extralit/tests/unit/v2/cli/test_cli_annotation.py new file mode 100644 index 000000000..dcaacc552 --- /dev/null +++ b/extralit/tests/unit/v2/cli/test_cli_annotation.py @@ -0,0 +1,76 @@ +import json +import uuid +from types import SimpleNamespace + +from typer.testing import CliRunner + +from extralit.cli.app import app as root_app +from extralit.v2.models import ProjectionView + +# Invoke through the mounted root app so the verb group name ("suggestions"/"projection") +# and its subcommand ("upsert"/"get") are both required — this matches real CLI usage. +# (Invoking a single-command Typer standalone promotes the command and drops its name, +# which is not how these verbs are actually invoked once mounted by add_v2_commands.) +runner = CliRunner() +RECORD_ID = str(uuid.uuid4()) +Q_ID = str(uuid.uuid4()) +WS = str(uuid.uuid4()) + + +class FakeClient(SimpleNamespace): + def __enter__(self): + return self + + def __exit__(self, *exc_info): + pass + + +def test_suggestions_upsert_decodes_json_value(monkeypatch): + calls = {} + + def upsert(record, question, value, score=None, agent=None, schema_id=None): + calls["args"] = (record, question, value, score, agent, schema_id) + return SimpleNamespace(model_dump=lambda mode: {"ok": True}) + + client = FakeClient(suggestions=SimpleNamespace(upsert=upsert)) + import extralit.v2.cli.suggestions as mod + + monkeypatch.setattr(mod, "get_client", lambda: client) + result = runner.invoke( + root_app, + [ + "suggestions", + "upsert", + RECORD_ID, + "--question-id", + Q_ID, + "--value", + '"120"', + "--score", + "0.9", + "--agent", + "claude", + ], + ) + assert result.exit_code == 0, result.output + record, question, value, score, agent, _schema_id = calls["args"] + assert (record, question, value, score, agent) == (RECORD_ID, Q_ID, "120", 0.9, "claude") + + +def test_suggestions_upsert_name_requires_schema_id(monkeypatch): + import extralit.v2.cli.suggestions as mod + + monkeypatch.setattr(mod, "get_client", lambda: FakeClient(suggestions=SimpleNamespace())) + result = runner.invoke(root_app, ["suggestions", "upsert", RECORD_ID, "--question", "size", "--value", '"x"']) + assert result.exit_code == 2 # usage error: --question needs --schema-id + + +def test_projection_get(monkeypatch): + view = ProjectionView(reference="10.1000/j.abc", records=[], total_records=0) + client = FakeClient(projections=SimpleNamespace(get=lambda ws, ref: view)) + import extralit.v2.cli.projection as mod + + monkeypatch.setattr(mod, "get_client", lambda: client) + result = runner.invoke(root_app, ["projection", "get", "10.1000/j.abc", "--workspace-id", WS]) + assert result.exit_code == 0, result.output + assert json.loads(result.stdout)["reference"] == "10.1000/j.abc" diff --git a/extralit/tests/unit/v2/cli/test_cli_records.py b/extralit/tests/unit/v2/cli/test_cli_records.py new file mode 100644 index 000000000..47b4265c0 --- /dev/null +++ b/extralit/tests/unit/v2/cli/test_cli_records.py @@ -0,0 +1,82 @@ +import json +import uuid +from datetime import datetime, timezone +from types import SimpleNamespace + +import pytest +from typer.testing import CliRunner + +from extralit.v2.cli.records import _parse_filter, app +from extralit.v2.models import Record, SearchPage + +runner = CliRunner() # click >= 8.2: stderr is separated by default (mix_stderr was removed) +SCHEMA_ID = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _record(): + return Record.model_validate( + { + "id": str(uuid.uuid4()), + "schema_id": SCHEMA_ID, + "schema_version_id": str(uuid.uuid4()), + "reference": "10.1000/xyz", + "external_id": None, + "fields": {"size": "120"}, + "metadata": None, + "status": "pending", + "inserted_at": NOW, + "updated_at": NOW, + } + ) + + +class FakeClient(SimpleNamespace): + def __enter__(self): + return self + + def __exit__(self, *exc_info): + pass + + +@pytest.fixture +def fake_client(monkeypatch): + calls = {} + + def upsert(schema_id, items, reference=None): + calls["upsert"] = (schema_id, items, reference) + return [_record()] + + def search(schema_id, text=None, filters=None, offset=0, limit=50): + calls["search"] = (schema_id, text, filters, offset, limit) + return SearchPage(items=[_record()], total=1) + + client = FakeClient(records=SimpleNamespace(bulk_upsert=upsert, search=search)) + import extralit.v2.cli.records as records_mod + + monkeypatch.setattr(records_mod, "get_client", lambda: client) + return calls + + +def test_parse_filter_json_decodes_value(): + assert _parse_filter("age:ge:18") == ("age", "ge", 18) + assert _parse_filter('label:in:["a","b"]') == ("label", "in", ["a", "b"]) + assert _parse_filter("country:eq:KE") == ("country", "eq", "KE") + + +def test_upsert_reads_jsonl_from_stdin(fake_client): + lines = '{"size": "120"}\n{"size": "135"}\n' + result = runner.invoke(app, ["upsert", SCHEMA_ID, "--reference", "10.1000/xyz"], input=lines) + assert result.exit_code == 0, result.output + _schema_id, items, reference = fake_client["upsert"] + assert items == [{"size": "120"}, {"size": "135"}] + assert reference == "10.1000/xyz" + assert json.loads(result.stdout)[0]["fields"] == {"size": "120"} + + +def test_search_passes_filters(fake_client): + result = runner.invoke(app, ["search", SCHEMA_ID, "--text", "tumor", "--filter", "age:ge:18", "--limit", "10"]) + assert result.exit_code == 0 + _, text, filters, _, limit = fake_client["search"] + assert text == "tumor" and filters == [("age", "ge", 18)] and limit == 10 + assert json.loads(result.stdout)["total"] == 1 diff --git a/extralit/tests/unit/v2/cli/test_cli_schemas.py b/extralit/tests/unit/v2/cli/test_cli_schemas.py new file mode 100644 index 000000000..e0d95dbfa --- /dev/null +++ b/extralit/tests/unit/v2/cli/test_cli_schemas.py @@ -0,0 +1,80 @@ +import json +import uuid +from datetime import datetime, timezone +from types import SimpleNamespace + +import click +import pytest +from typer.testing import CliRunner + +import extralit.v2.cli._context as context_mod +from extralit.v2._api._errors import ValidationError +from extralit.v2.cli.schemas import app +from extralit.v2.models import Schema + +# click >= 8.2 separates stderr by default; 8.1 (Python 3.9) requires mix_stderr=False. +runner = ( + CliRunner() if tuple(int(x) for x in click.__version__.split(".")[:2]) >= (8, 2) else CliRunner(mix_stderr=False) +) +WS = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _schema(name="trials"): + return Schema.model_validate( + { + "id": str(uuid.uuid4()), + "name": name, + "status": "draft", + "current_version_id": None, + "settings": {}, + "workspace_id": WS, + "inserted_at": NOW, + "updated_at": NOW, + } + ) + + +class FakeClient(SimpleNamespace): + def __enter__(self): + return self + + def __exit__(self, *exc_info): + pass + + +@pytest.fixture +def fake_client(monkeypatch): + client = FakeClient(schemas=SimpleNamespace(list=lambda workspace_id: [_schema()])) + monkeypatch.setattr(context_mod, "get_client", lambda: client) + # schemas.py imports get_client at call time via the module attribute: + import extralit.v2.cli.schemas as schemas_mod + + monkeypatch.setattr(schemas_mod, "get_client", lambda: client) + return client + + +def test_list_emits_json(fake_client): + result = runner.invoke(app, ["list", "--workspace-id", WS, "--json"]) + assert result.exit_code == 0 + items = json.loads(result.stdout) + assert items[0]["name"] == "trials" + + +def test_validation_error_exits_3(fake_client): + def boom(workspace_id): + raise ValidationError(422, "bad") + + fake_client.schemas.list = boom + result = runner.invoke(app, ["list", "--workspace-id", WS, "--json"]) + assert result.exit_code == 3 + assert json.loads(result.stderr)["error"]["type"] == "ValidationError" + + +def test_top_level_registration(): + from extralit.cli.app import app as root_app + + names = [t.name for t in root_app.registered_groups] + for verb in ("schemas", "records", "questions", "suggestions", "projection", "references"): + assert verb in names, f"{verb} not registered at top level" + assert names.count("schemas") == 1, "v1 schemas subcommand must be unregistered" diff --git a/extralit/tests/unit/v2/cli/test_output.py b/extralit/tests/unit/v2/cli/test_output.py new file mode 100644 index 000000000..598c16132 --- /dev/null +++ b/extralit/tests/unit/v2/cli/test_output.py @@ -0,0 +1,91 @@ +import json + +import click +import pytest +import typer +from typer.testing import CliRunner + +from extralit.v2._api._errors import V2APIError, ValidationError +from extralit.v2.cli._output import emit, fail, to_jsonable +from extralit.v2.models import SearchPage + +# click >= 8.2 separates stderr by default; 8.1 (Python 3.9) requires mix_stderr=False. +_runner = ( + CliRunner() if tuple(int(x) for x in click.__version__.split(".")[:2]) >= (8, 2) else CliRunner(mix_stderr=False) +) + + +def test_to_jsonable_handles_models_lists_dicts(): + page = SearchPage(items=[], total=3) + assert to_jsonable(page) == {"items": [], "total": 3} + assert to_jsonable([page]) == [{"items": [], "total": 3}] + assert to_jsonable({"a": 1}) == {"a": 1} + + +def test_emit_json_when_flag_set(capsys): + emit({"a": 1}, json_flag=True) + assert json.loads(capsys.readouterr().out) == {"a": 1} + + +def test_emit_json_when_not_a_tty(capsys): + emit({"a": 1}, json_flag=False) # pytest capture is not a tty -> auto-JSON + assert json.loads(capsys.readouterr().out) == {"a": 1} + + +def test_fail_validation_exits_3_with_stderr_json(capsys): + with pytest.raises(typer.Exit) as excinfo: + fail(ValidationError(422, "bad value")) + assert excinfo.value.exit_code == 3 + err = json.loads(capsys.readouterr().err) + assert err["error"]["status"] == 422 and err["error"]["type"] == "ValidationError" + + +def test_fail_api_error_exits_1(capsys): + with pytest.raises(typer.Exit) as excinfo: + fail(V2APIError(500, "kaboom")) + assert excinfo.value.exit_code == 1 + assert json.loads(capsys.readouterr().err)["error"]["detail"] == "kaboom" + + +def test_handle_errors_routes_value_error_to_structured_fail(monkeypatch): + """Malformed UUID must produce exit code 1 + structured stderr via handle_errors, not a traceback. + + get_client is monkeypatched so credentials resolution succeeds and the ValueError + provably originates from UUID('not-a-uuid') inside upsert_records (after JSONL is read). + """ + from types import SimpleNamespace + + import extralit.v2.cli.records as records_mod + from extralit.v2.cli.records import app as records_app + + class _FakeClient(SimpleNamespace): + def __enter__(self): + return self + + def __exit__(self, *_): + pass + + monkeypatch.setattr( + records_mod, "get_client", lambda: _FakeClient(records=SimpleNamespace(bulk_upsert=lambda *a, **k: [])) + ) + runner = _runner + # Pass valid JSONL via stdin; the UUID is what's malformed + result = runner.invoke(records_app, ["upsert", "not-a-uuid"], input='{"size": "10"}\n') + assert result.exit_code == 1 + err = json.loads(result.stderr) + assert err["error"]["type"] == "ValueError" + assert "UUID" in err["error"]["detail"] or "hexadecimal" in err["error"]["detail"] + + +def test_handle_errors_routes_oserror_to_structured_fail(): + """Missing --file path must produce exit code 1 + structured stderr, not a traceback.""" + import uuid + + from extralit.v2.cli.records import app as records_app + + runner = _runner + schema_id = str(uuid.uuid4()) + result = runner.invoke(records_app, ["upsert", schema_id, "--file", "/nonexistent/path/to/file.jsonl"]) + assert result.exit_code == 1 + err = json.loads(result.stderr) + assert err["error"]["type"] in ("FileNotFoundError", "OSError") diff --git a/extralit/tests/unit/v2/test_annotation_resources.py b/extralit/tests/unit/v2/test_annotation_resources.py new file mode 100644 index 000000000..1b060417a --- /dev/null +++ b/extralit/tests/unit/v2/test_annotation_resources.py @@ -0,0 +1,143 @@ +import json +import uuid +from datetime import datetime, timezone + +import pytest +import pytest_asyncio + +from extralit.v2._api._generated import Source +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.models import Record +from extralit.v2.resources import Projections, Questions, Responses, Suggestions + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" +SCHEMA_ID = str(uuid.uuid4()) +RECORD_ID = str(uuid.uuid4()) +Q_SIZE = str(uuid.uuid4()) +WS = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _suggestion(): + return { + "id": str(uuid.uuid4()), + "record_id": RECORD_ID, + "question_id": Q_SIZE, + "value": "120", + "score": 0.9, + "agent": "claude", + "type": "model", + "inserted_at": NOW, + "updated_at": NOW, + } + + +def _record_obj(): + return Record.model_validate( + { + "id": RECORD_ID, + "schema_id": SCHEMA_ID, + "schema_version_id": str(uuid.uuid4()), + "reference": "10.1000/xyz", + "external_id": None, + "fields": {}, + "metadata": None, + "status": "pending", + "inserted_at": NOW, + "updated_at": NOW, + } + ) + + +@pytest_asyncio.fixture +async def transport(): + t = AsyncTransport(API, api_key="k") + yield t + await t.aclose() + + +async def test_upsert_resolves_question_name_via_record_object(httpx_mock, transport): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", + json={ + "items": [ + { + "id": Q_SIZE, + "schema_id": SCHEMA_ID, + "name": "size", + "title": "Size", + "description": None, + "type": "text", + "columns": ["size"], + "settings": {}, + "required": False, + "inserted_at": NOW, + "updated_at": NOW, + } + ] + }, + ) + httpx_mock.add_response(method="PUT", url=f"{API}/api/v2/records/{RECORD_ID}/suggestions", json=_suggestion()) + questions = Questions(transport) + suggestions = Suggestions(transport, questions) + result = await suggestions.upsert(_record_obj(), "size", "120", score=0.9, agent="claude") + assert str(result.question_id) == Q_SIZE + body = json.loads(httpx_mock.get_requests()[-1].read()) + assert body["question_id"] == Q_SIZE and body["agent"] == "claude" and body["score"] == 0.9 + + +async def test_upsert_name_without_schema_raises(transport): + suggestions = Suggestions(transport, Questions(transport)) + with pytest.raises(ValueError, match="schema_id"): + await suggestions.upsert(RECORD_ID, "size", "120") + + +async def test_upsert_accepts_question_id_directly(httpx_mock, transport): + httpx_mock.add_response(method="PUT", url=f"{API}/api/v2/records/{RECORD_ID}/suggestions", json=_suggestion()) + suggestions = Suggestions(transport, Questions(transport)) + await suggestions.upsert(RECORD_ID, Q_SIZE, "120") # no questions fetch needed + assert len(httpx_mock.get_requests()) == 1 + + +async def test_response_get_maps_null_to_none(httpx_mock, transport): + httpx_mock.add_response(url=f"{API}/api/v2/records/{RECORD_ID}/responses", json=None) + assert await Responses(transport).get(RECORD_ID) is None + + +async def test_response_get_unwraps_values(httpx_mock, transport): + httpx_mock.add_response( + url=f"{API}/api/v2/records/{RECORD_ID}/responses", + json={ + "id": str(uuid.uuid4()), + "record_id": RECORD_ID, + "user_id": str(uuid.uuid4()), + "values": {"size": {"value": "135"}}, + "status": "submitted", + "inserted_at": NOW, + "updated_at": NOW, + }, + ) + response = await Responses(transport).get(RECORD_ID) + assert response.unwrapped_values == {"size": "135"} + + +async def test_projection_get(httpx_mock, transport): + httpx_mock.add_response( + url=f"{API}/api/v2/projection/references/10.1000/j.abc?workspace_id={WS}", + json={ + "reference": "10.1000/j.abc", + "records": [ + { + "record_id": RECORD_ID, + "schema_id": SCHEMA_ID, + "reference": "10.1000/j.abc", + "cells": [{"question_name": "size", "value": "120", "source": "suggestion"}], + } + ], + "total_records": 1, + }, + ) + view = await Projections(transport).get(WS, "10.1000/j.abc") + assert view.records[0].cells[0].source == Source.suggestion diff --git a/extralit/tests/unit/v2/test_boundaries.py b/extralit/tests/unit/v2/test_boundaries.py new file mode 100644 index 000000000..7651d8252 --- /dev/null +++ b/extralit/tests/unit/v2/test_boundaries.py @@ -0,0 +1,53 @@ +import re +import subprocess +import sys +from pathlib import Path + +SRC = Path(__file__).parents[3] / "src" / "extralit" +V2 = SRC / "v2" + +# The single allowed v1 import inside v2 (credentials helper outlives v1 retirement). +ALLOWED_V1_IMPORT = "extralit.client.login" +V1_IMPORT = re.compile(r"^\s*(?:from|import)\s+(extralit\.(?!v2\b)[\w.]*)", re.MULTILINE) +V2_IMPORT = re.compile(r"^\s*(?:from|import)\s+extralit\.v2[\w.]*", re.MULTILINE) + + +def test_v2_imports_no_v1_except_credentials(): + violations = [] + for path in V2.rglob("*.py"): + if path.name == "_generated.py": + continue + for match in V1_IMPORT.finditer(path.read_text()): + if match.group(1) != ALLOWED_V1_IMPORT: + violations.append(f"{path.relative_to(SRC)}: {match.group(0).strip()}") + assert not violations, "v2 -> v1 imports outside the credentials exception:\n" + "\n".join(violations) + + +def test_v1_never_imports_v2_except_composition_root(): + violations = [] + for path in SRC.rglob("*.py"): + if V2 in path.parents or path == SRC / "cli" / "app.py": + continue + if V2_IMPORT.search(path.read_text()): + violations.append(str(path.relative_to(SRC))) + assert not violations, "v1 files importing v2 (only cli/app.py may):\n" + "\n".join(violations) + + +HEAVY = ("pandas", "pandera", "datasets", "huggingface_hub") + + +def _heavy_after(imports: str) -> set: + code = f"import sys; {imports}; print(','.join(sorted(m for m in {HEAVY!r} if m in sys.modules)))" + proc = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True, check=True) + return set(filter(None, proc.stdout.strip().split(","))) + + +def test_v2_adds_no_heavy_imports(): + """Agents make many short CLI calls: importing extralit.v2 (incl. cli) must not add + heavy modules beyond what `import extralit` (the v1 package init) already drags in. + Delta-based on purpose: v1's own import weight is Phase 6 scope, measured at the + baseline (today: datasets + huggingface_hub via extralit/__init__.py).""" + baseline = _heavy_after("import extralit") + with_v2 = _heavy_after("import extralit; import extralit.v2; import extralit.v2.cli") + added = with_v2 - baseline + assert not added, f"extralit.v2 import added heavy modules: {sorted(added)}" diff --git a/extralit/tests/unit/v2/test_client.py b/extralit/tests/unit/v2/test_client.py new file mode 100644 index 000000000..e143cae87 --- /dev/null +++ b/extralit/tests/unit/v2/test_client.py @@ -0,0 +1,49 @@ +import json + +import pytest + +from extralit.v2 import AsyncClient + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" + + +async def test_explicit_args_and_resource_wiring(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w", json={"items": []}) + async with AsyncClient(api_url=API, api_key="k") as client: + assert await client.schemas.list("w") == [] + for name in ("schemas", "questions", "records", "suggestions", "projections", "responses"): + assert hasattr(client, name) + + +async def test_env_fallback(monkeypatch, httpx_mock): + monkeypatch.setenv("EXTRALIT_API_URL", API) + monkeypatch.setenv("EXTRALIT_API_KEY", "env-key") + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w", json={"items": []}) + async with AsyncClient() as client: + await client.schemas.list("w") + assert httpx_mock.get_requests()[0].headers["X-Extralit-Api-Key"] == "env-key" + + +async def test_credentials_file_fallback(monkeypatch, tmp_path): + monkeypatch.delenv("EXTRALIT_API_URL", raising=False) + monkeypatch.delenv("EXTRALIT_API_KEY", raising=False) + creds = tmp_path / "credentials.json" + creds.write_text(json.dumps({"api_url": API, "api_key": "file-key"})) + import extralit.client.login as login_mod + + monkeypatch.setattr(login_mod, "EXTRALIT_CREDENTIALS_FILE", creds) + client = AsyncClient() + assert client._transport._api_key == "file-key" + await client.aclose() + + +async def test_unresolvable_raises(monkeypatch): + monkeypatch.delenv("EXTRALIT_API_URL", raising=False) + monkeypatch.delenv("EXTRALIT_API_KEY", raising=False) + import extralit.client.login as login_mod + + monkeypatch.setattr(login_mod.ExtralitCredentials, "exists", classmethod(lambda cls: False)) + with pytest.raises(ValueError, match="api_url"): + AsyncClient() diff --git a/extralit/tests/unit/v2/test_contract.py b/extralit/tests/unit/v2/test_contract.py new file mode 100644 index 000000000..6b3b85614 --- /dev/null +++ b/extralit/tests/unit/v2/test_contract.py @@ -0,0 +1,109 @@ +import json +import subprocess +import sys +from pathlib import Path + +import pytest + +SDK_ROOT = Path(__file__).parents[3] +API_DIR = SDK_ROOT / "src" / "extralit" / "v2" / "_api" +SNAPSHOT = API_DIR / "openapi.json" +GENERATED = API_DIR / "_generated.py" +SERVER_DIR = SDK_ROOT.parent / "extralit-server" + +EXPECTED_MODELS = [ + "SchemaRead", + "Schemas", + "SchemaCreate", + "SchemaUpdate", + "SchemaVersionCreate", + "SchemaVersionRead", + "RecordUpsert", + "RecordsBulkUpsert", + "RecordRead", + "Records", + "RecordFilter", + "RecordSearchQuery", + "ReferenceGroup", + "ReferenceView", + "QuestionCreate", + "QuestionUpdate", + "QuestionRead", + "Questions", + "SuggestionUpsert", + "SuggestionRead", + "Suggestions", + "ResponseUpsert", + "ResponseRead", + "ProjectionCell", + "ProjectionRecord", + "ProjectionView", + "SchemaStatus", + "V2RecordStatus", + "QuestionType", + "SuggestionType", + "ResponseStatus", +] + + +def test_generated_models_importable(): + import extralit.v2._api._generated as gen + + missing = [name for name in EXPECTED_MODELS if not hasattr(gen, name)] + assert not missing, f"generated module lacks: {missing}" + + +def test_generated_matches_snapshot(tmp_path): + """No-drift gate: regenerating from the committed snapshot must be byte-identical.""" + out = tmp_path / "regen.py" + # Pin: datamodel-code-generator>=0.26,<0.69 (pyproject.toml); generated with 0.68.1 — a minor bump requires re-running codegen + proc = subprocess.run( + [ + sys.executable, + "-m", + "datamodel_code_generator", + "--input", + str(SNAPSHOT), + "--input-file-type", + "openapi", + "--output", + str(out), + "--output-model-type", + "pydantic_v2.BaseModel", + "--target-python-version", + "3.10", + "--use-double-quotes", + "--disable-timestamp", + "--no-use-union-operator", + "--formatters", + "black", + "isort", + ], + capture_output=True, + text=True, + ) + assert proc.returncode == 0, f"datamodel-codegen failed (rc={proc.returncode}):\n{proc.stderr}" + assert out.read_text() == GENERATED.read_text(), ( + "src/extralit/v2/_api/_generated.py drifted from openapi.json — rerun datamodel-codegen (see plan Task 1 Step 3)" + ) + + +@pytest.mark.slow +@pytest.mark.skipif(not SERVER_DIR.exists(), reason="server tree not present") +def test_snapshot_matches_server(): + """Snapshot-vs-server gate: committed snapshot must equal a fresh openapi-dump.""" + proc = subprocess.run( + ["uv", "run", "python", "-m", "extralit_server", "openapi-dump"], + cwd=SERVER_DIR, + capture_output=True, + text=True, + ) + assert proc.returncode == 0, f"openapi-dump failed (rc={proc.returncode}):\n{proc.stderr}" + # Slice from the first '{' to guard against any stdout preamble (banners, deprecation notices) + # that would cause json.loads to fail with JSONDecodeError unrelated to snapshot drift. + idx = proc.stdout.find("{") + assert idx != -1, f"no JSON object found in openapi-dump stdout:\n{proc.stdout}" + stdout = proc.stdout[idx:] + assert json.loads(stdout) == json.loads(SNAPSHOT.read_text()), ( + "openapi.json snapshot drifted from the server — re-dump it (see plan Task 1 Step 2)" + ) diff --git a/extralit/tests/unit/v2/test_errors.py b/extralit/tests/unit/v2/test_errors.py new file mode 100644 index 000000000..bbfe9660e --- /dev/null +++ b/extralit/tests/unit/v2/test_errors.py @@ -0,0 +1,39 @@ +from extralit.v2._api._errors import ( + AuthError, + NotFoundError, + V2APIError, + ValidationError, + error_from_response, + normalize_validation_detail, +) + + +def test_normalizes_string_detail(): + assert normalize_validation_detail("boom") == [{"loc": [], "msg": "boom"}] + + +def test_normalizes_fastapi_list_detail(): + detail = [{"loc": ["body", "items", 0, "reference"], "msg": "field required", "type": "missing"}] + assert normalize_validation_detail(detail) == [{"loc": ["body", "items", 0, "reference"], "msg": "field required"}] + + +def test_normalizes_none_and_junk(): + assert normalize_validation_detail(None) == [] + assert normalize_validation_detail({"weird": 1}) == [{"loc": [], "msg": "{'weird': 1}"}] + + +def test_error_from_response_maps_statuses(): + assert isinstance(error_from_response(401, {"detail": "nope"}), AuthError) + assert isinstance(error_from_response(403, {"detail": "nope"}), AuthError) + assert isinstance(error_from_response(404, {"detail": "gone"}), NotFoundError) + err = error_from_response(422, {"detail": "bad value"}) + assert isinstance(err, ValidationError) + assert err.errors == [{"loc": [], "msg": "bad value"}] + other = error_from_response(500, {"detail": "kaboom"}) + assert type(other) is V2APIError + assert other.status_code == 500 and other.detail == "kaboom" + + +def test_error_from_response_non_dict_body(): + err = error_from_response(502, "bad gateway") + assert err.detail == "bad gateway" diff --git a/extralit/tests/unit/v2/test_models.py b/extralit/tests/unit/v2/test_models.py new file mode 100644 index 000000000..a035104bc --- /dev/null +++ b/extralit/tests/unit/v2/test_models.py @@ -0,0 +1,77 @@ +import uuid +from datetime import datetime, timezone + +from extralit.v2.models import ( + Record, + Response, + SchemaVersion, + SearchPage, + unwrap_response_values, + wrap_response_values, +) + + +def _version_payload(**overrides): + payload = { + "id": str(uuid.uuid4()), + "schema_id": str(uuid.uuid4()), + "version": 1, + "object_key": "schemas/x/v1.json", + "object_version_id": None, + "etag": "e", + "checksum": "c", + "parent_version_id": None, + "columns_cache": [{"name": "size", "dtype": "str"}, {"name": "country", "dtype": "str"}], + "review_widgets": {}, + "inserted_at": datetime.now(timezone.utc).isoformat(), + } + payload.update(overrides) + return payload + + +def test_schema_version_find_column(): + version = SchemaVersion.model_validate(_version_payload()) + assert version.find_column("size") == {"name": "size", "dtype": "str"} + assert version.find_column("nope") is None + + +def test_wrap_unwrap_roundtrip(): + """Server double-wraps response values ({name: {"value": ...}}) on both PUT and GET.""" + values = {"size": "120", "country": ["KE", "UG"]} + wrapped = wrap_response_values(values) + assert wrapped == {"size": {"value": "120"}, "country": {"value": ["KE", "UG"]}} + assert unwrap_response_values(wrapped) == values + assert unwrap_response_values(None) == {} + + +def test_response_unwrapped_values(): + response = Response.model_validate( + { + "id": str(uuid.uuid4()), + "record_id": str(uuid.uuid4()), + "user_id": str(uuid.uuid4()), + "values": {"size": {"value": "135"}}, + "status": "submitted", + "inserted_at": datetime.now(timezone.utc).isoformat(), + "updated_at": datetime.now(timezone.utc).isoformat(), + } + ) + assert response.unwrapped_values == {"size": "135"} + + +def test_search_page_holds_records(): + record = { + "id": str(uuid.uuid4()), + "schema_id": str(uuid.uuid4()), + "schema_version_id": str(uuid.uuid4()), + "reference": "10.1000/xyz", + "external_id": None, + "fields": {"size": "120"}, + "metadata": None, + "status": "pending", + "inserted_at": datetime.now(timezone.utc).isoformat(), + "updated_at": datetime.now(timezone.utc).isoformat(), + } + page = SearchPage(items=[Record.model_validate(record)], total=1) + assert page.items[0].reference == "10.1000/xyz" + assert page.total == 1 diff --git a/extralit/tests/unit/v2/test_questions_resource.py b/extralit/tests/unit/v2/test_questions_resource.py new file mode 100644 index 000000000..a12dd4795 --- /dev/null +++ b/extralit/tests/unit/v2/test_questions_resource.py @@ -0,0 +1,64 @@ +import uuid +from datetime import datetime, timezone + +import pytest + +from extralit.v2._api._errors import NotFoundError +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.resources import Questions + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" +SCHEMA_ID = str(uuid.uuid4()) +Q_SIZE = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _question(qid, name): + return { + "id": qid, + "schema_id": SCHEMA_ID, + "name": name, + "title": name.title(), + "description": None, + "type": "text", + "columns": [name], + "settings": {}, + "required": False, + "inserted_at": NOW, + "updated_at": NOW, + } + + +@pytest.fixture +def questions(): + transport = AsyncTransport(API, api_key="k") + return Questions(transport) + + +async def test_list_and_id_for_uses_one_fetch(httpx_mock, questions): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", + json={"items": [_question(Q_SIZE, "size")]}, + ) + assert [q.name for q in await questions.list(SCHEMA_ID)] == ["size"] + assert str(await questions.id_for(SCHEMA_ID, "size")) == Q_SIZE # served from cache + assert len(httpx_mock.get_requests()) == 1 + + +async def test_id_for_refetches_once_then_raises(httpx_mock, questions): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", + json={"items": [_question(Q_SIZE, "size")]}, + ) + await questions.list(SCHEMA_ID) + q_new = str(uuid.uuid4()) + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", + json={"items": [_question(Q_SIZE, "size"), _question(q_new, "dosage")]}, + ) + assert str(await questions.id_for(SCHEMA_ID, "dosage")) == q_new # miss -> refetch -> hit + httpx_mock.add_response(url=f"{API}/api/v2/schemas/{SCHEMA_ID}/questions", json={"items": []}) + with pytest.raises(NotFoundError): + await questions.id_for(SCHEMA_ID, "ghost") diff --git a/extralit/tests/unit/v2/test_records_resource.py b/extralit/tests/unit/v2/test_records_resource.py new file mode 100644 index 000000000..957a3bf0a --- /dev/null +++ b/extralit/tests/unit/v2/test_records_resource.py @@ -0,0 +1,138 @@ +import json +import uuid +from datetime import datetime, timezone + +import pytest +import pytest_asyncio + +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.resources import Records + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" +SCHEMA_ID = str(uuid.uuid4()) +WS = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _record(i): + return { + "id": str(uuid.uuid4()), + "schema_id": SCHEMA_ID, + "schema_version_id": str(uuid.uuid4()), + "reference": "10.1000/xyz", + "external_id": str(i), + "fields": {"size": str(i)}, + "metadata": None, + "status": "pending", + "inserted_at": NOW, + "updated_at": NOW, + } + + +@pytest_asyncio.fixture +async def records(): + transport = AsyncTransport(API, api_key="k") + r = Records(transport) + yield r + await transport.aclose() + + +async def test_bulk_upsert_chunks_at_500_and_preserves_order(httpx_mock, records): + url = f"{API}/api/v2/schemas/{SCHEMA_ID}/records:bulk-upsert" + + def responder(request): + import httpx + + sent = json.loads(request.read())["items"] + assert len(sent) <= 500 + return httpx.Response(200, json={"items": [_record(item["external_id"]) for item in sent], "total": len(sent)}) + + for _ in range(3): + httpx_mock.add_callback(responder, method="POST", url=url) + + items = [{"fields": {"size": str(i)}, "reference": "10.1000/xyz", "external_id": str(i)} for i in range(1200)] + progress = [] + result = await records.bulk_upsert(SCHEMA_ID, items, on_progress=lambda done, total: progress.append((done, total))) + assert len(result) == 1200 + assert [r.external_id for r in result] == [str(i) for i in range(1200)] # input order preserved + assert len(httpx_mock.get_requests()) == 3 # 500 + 500 + 200 + assert progress[-1] == (1200, 1200) + + +async def test_bulk_upsert_bare_fields_and_shared_reference(httpx_mock, records): + url = f"{API}/api/v2/schemas/{SCHEMA_ID}/records:bulk-upsert" + httpx_mock.add_response(method="POST", url=url, json={"items": [_record(0)], "total": 1}) + await records.bulk_upsert(SCHEMA_ID, [{"size": "120"}], reference="10.1000/xyz") + sent = json.loads(httpx_mock.get_requests()[0].read())["items"] + assert sent == [{"fields": {"size": "120"}, "reference": "10.1000/xyz"}] + + +async def test_bulk_upsert_without_reference_raises(records): + with pytest.raises(ValueError, match="reference"): + await records.bulk_upsert(SCHEMA_ID, [{"size": "120"}]) + + +async def test_bulk_upsert_accepts_dataframe(httpx_mock, records): + pandas = pytest.importorskip("pandas") + url = f"{API}/api/v2/schemas/{SCHEMA_ID}/records:bulk-upsert" + httpx_mock.add_response(method="POST", url=url, json={"items": [_record(0), _record(1)], "total": 2}) + frame = pandas.DataFrame( + [ + {"size": "120", "reference": "10.1000/a"}, + {"size": "135", "reference": "10.1000/b"}, + ] + ) + await records.bulk_upsert(SCHEMA_ID, frame) + sent = json.loads(httpx_mock.get_requests()[0].read())["items"] + assert sent == [ + {"fields": {"size": "120"}, "reference": "10.1000/a"}, + {"fields": {"size": "135"}, "reference": "10.1000/b"}, + ] + + +async def test_search_normalizes_tuple_filters(httpx_mock, records): + url = f"{API}/api/v2/schemas/{SCHEMA_ID}/records:search" + httpx_mock.add_response(method="POST", url=url, json={"items": [_record(1)], "total": 41}) + page = await records.search(SCHEMA_ID, text="tumor", filters=[("age", "ge", 18)], limit=10) + assert page.total == 41 + body = json.loads(httpx_mock.get_requests()[0].read()) + assert body == { + "text": "tumor", + "filters": [{"column": "age", "op": "ge", "value": 18}], + "offset": 0, + "limit": 10, + } + + +async def test_list_passes_status_and_reference(httpx_mock, records): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/records?offset=0&limit=50&status=pending&reference=10.1000%2Fxyz", + json={"items": [], "total": 0}, + ) + page = await records.list(SCHEMA_ID, status="pending", reference="10.1000/xyz") + assert page.items == [] and page.total == 0 + + +async def test_delete_chunks_at_100(httpx_mock, records): + for _ in range(2): + httpx_mock.add_response( + method="DELETE", + url=__import__("re").compile(rf"{API}/api/v2/schemas/{SCHEMA_ID}/records\?ids=.*"), + status_code=204, + ) + await records.delete(SCHEMA_ID, [f"id-{i}" for i in range(150)]) + reqs = httpx_mock.get_requests() + assert len(reqs) == 2 + assert len(reqs[0].url.params["ids"].split(",")) == 100 + assert len(reqs[1].url.params["ids"].split(",")) == 50 + + +async def test_get_reference_keeps_slashes(httpx_mock, records): + httpx_mock.add_response( + url=f"{API}/api/v2/references/10.1000/j.abc?workspace_id={WS}", + json={"reference": "10.1000/j.abc", "groups": [], "total_records": 0}, + ) + view = await records.get_reference(WS, "10.1000/j.abc") + assert view.reference == "10.1000/j.abc" diff --git a/extralit/tests/unit/v2/test_schemas_resource.py b/extralit/tests/unit/v2/test_schemas_resource.py new file mode 100644 index 000000000..dfdd2dee4 --- /dev/null +++ b/extralit/tests/unit/v2/test_schemas_resource.py @@ -0,0 +1,111 @@ +import json +import uuid +from datetime import datetime, timezone + +import pytest + +from extralit.v2._api._errors import NotFoundError +from extralit.v2._api._transport import AsyncTransport +from extralit.v2.resources import Schemas + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" +WS = str(uuid.uuid4()) +SCHEMA_ID = str(uuid.uuid4()) +NOW = datetime.now(timezone.utc).isoformat() + + +def _schema(name="trials"): + return { + "id": SCHEMA_ID, + "name": name, + "status": "draft", + "current_version_id": None, + "settings": {}, + "workspace_id": WS, + "inserted_at": NOW, + "updated_at": NOW, + } + + +def _version(version=1): + return { + "id": str(uuid.uuid4()), + "schema_id": SCHEMA_ID, + "version": version, + "object_key": f"schemas/{SCHEMA_ID}/v{version}.json", + "object_version_id": None, + "etag": "e", + "checksum": "c", + "parent_version_id": None, + "columns_cache": [{"name": "size"}], + "review_widgets": {}, + "inserted_at": NOW, + } + + +@pytest.fixture +def schemas(): + transport = AsyncTransport(API, api_key="k") + yield Schemas(transport) + # Note: aclose() is handled by pytest-asyncio context manager + + +async def test_create_and_get(httpx_mock, schemas): + httpx_mock.add_response(method="POST", url=f"{API}/api/v2/schemas", status_code=201, json=_schema()) + created = await schemas.create(WS, "trials") + assert created.name == "trials" + body = httpx_mock.get_requests()[0].read() + assert b'"workspace_id"' in body and b'"trials"' in body + + +async def test_get_by_name_found_and_missing(httpx_mock, schemas): + listing = {"items": [_schema("other") | {"id": str(uuid.uuid4())}, _schema("trials")]} + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id={WS}", json=listing) + found = await schemas.get_by_name(WS, "trials") + assert str(found.id) == SCHEMA_ID + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id={WS}", json={"items": []}) + with pytest.raises(NotFoundError): + await schemas.get_by_name(WS, "trials") + + +async def test_publish_accepts_pandera_object_or_string(httpx_mock, schemas): + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/versions", + status_code=201, + json=_version(), + ) + + class FakePandera: # duck-type: anything with .to_json() + def to_json(self): + return '{"columns": {"size": {}}}' + + version = await schemas.publish(SCHEMA_ID, FakePandera(), review_widgets={"size": {"widget": "text"}}) + assert version.version == 1 + sent = json.loads(httpx_mock.get_requests()[0].read()) + assert set(sent) == { + "body", + "review_widgets", + } # review_widgets ride out-of-band, not merged into body + assert sent["review_widgets"] == {"size": {"widget": "text"}} + assert '"columns"' in sent["body"] # body is the pandera JSON string, kept as a string value + + +async def test_get_version_cached(httpx_mock, schemas): + httpx_mock.add_response(url=f"{API}/api/v2/schemas/{SCHEMA_ID}/versions/1", json=_version()) + v1 = await schemas.get_version(SCHEMA_ID, 1) + v1_again = await schemas.get_version(SCHEMA_ID, 1) # served from cache: no second request + assert v1_again is v1 + assert len(httpx_mock.get_requests()) == 1 + + +async def test_versions_and_columns(httpx_mock, schemas): + httpx_mock.add_response( + url=f"{API}/api/v2/schemas/{SCHEMA_ID}/versions", + json=[_version(1), _version(2)], + ) + httpx_mock.add_response(url=f"{API}/api/v2/schemas/{SCHEMA_ID}/columns", json=[{"name": "size"}]) + assert [v.version for v in await schemas.versions(SCHEMA_ID)] == [1, 2] + assert await schemas.columns(SCHEMA_ID) == [{"name": "size"}] diff --git a/extralit/tests/unit/v2/test_sync_client.py b/extralit/tests/unit/v2/test_sync_client.py new file mode 100644 index 000000000..c5715ed14 --- /dev/null +++ b/extralit/tests/unit/v2/test_sync_client.py @@ -0,0 +1,28 @@ +import asyncio + +from extralit.v2 import Client + +API = "http://test:6900" + + +def test_sync_mirror_calls_through(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w", json={"items": []}) + with Client(api_url=API, api_key="k") as client: + assert client.schemas.list("w") == [] + + +def test_sync_client_works_inside_running_loop(httpx_mock): + """Jupyter simulation: a loop is already running in the calling thread. + asyncio.run-based facades explode here; the portal must not.""" + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w", json={"items": []}) + + async def main(): + with Client(api_url=API, api_key="k") as client: + return client.schemas.list("w") + + assert asyncio.run(main()) == [] + + +def test_non_coroutine_attrs_pass_through(httpx_mock): + with Client(api_url=API, api_key="k") as client: + client.questions.invalidate("some-schema") # sync method on a resource: plain passthrough diff --git a/extralit/tests/unit/v2/test_transport.py b/extralit/tests/unit/v2/test_transport.py new file mode 100644 index 000000000..38df3d142 --- /dev/null +++ b/extralit/tests/unit/v2/test_transport.py @@ -0,0 +1,184 @@ +import asyncio + +import pytest + +from extralit.v2._api._errors import AuthError, NotFoundError, ValidationError +from extralit.v2._api._transport import AsyncTransport + +pytestmark = pytest.mark.asyncio + +API = "http://test:6900" + + +async def test_api_key_header_sent(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", json={"items": []}) + t = AsyncTransport(API, api_key="secret.key") + body = await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + assert body == {"items": []} + assert httpx_mock.get_requests()[0].headers["X-Extralit-Api-Key"] == "secret.key" + await t.aclose() + + +async def test_password_login_then_bearer(httpx_mock): + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/token", + status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", json={"items": []}) + t = AsyncTransport(API, username="u", password="p") + await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + token_req, api_req = httpx_mock.get_requests() + assert b"username=u" in token_req.content and b"password=p" in token_req.content + assert api_req.headers["Authorization"] == "Bearer AT1" + await t.aclose() + + +async def test_refresh_once_on_401_then_retry(httpx_mock): + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/token", + status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", status_code=401, json={"detail": "expired"}) + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/token/refresh", + status_code=201, + json={"access_token": "AT2", "refresh_token": "RT2"}, + ) + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", json={"items": []}) + t = AsyncTransport(API, username="u", password="p") + body = await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + assert body == {"items": []} + refresh_req = httpx_mock.get_requests()[2] + assert b"RT1" in refresh_req.content + assert httpx_mock.get_requests()[3].headers["Authorization"] == "Bearer AT2" + await t.aclose() + + +async def test_401_after_failed_refresh_raises_auth_error(httpx_mock): + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/token", + status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", status_code=401, json={"detail": "expired"}) + httpx_mock.add_response(method="POST", url=f"{API}/api/v2/token/refresh", status_code=401, json={"detail": "no"}) + t = AsyncTransport(API, username="u", password="p") + with pytest.raises(AuthError): + await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + await t.aclose() + + +async def test_api_key_401_raises_without_refresh(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", status_code=401, json={"detail": "bad key"}) + t = AsyncTransport(API, api_key="bad") + with pytest.raises(AuthError): + await t.request("GET", "/schemas", params={"workspace_id": "w1"}) + assert len(httpx_mock.get_requests()) == 1 # no refresh attempt in api-key mode + await t.aclose() + + +async def test_concurrent_requests_login_once(httpx_mock): + # Only one /token response is registered; a login stampede would need extra /token + # responses and error out. The count assertion pins the single-login behavior. + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/token", + status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + for _ in range(5): + httpx_mock.add_response(url=f"{API}/api/v2/schemas?workspace_id=w1", json={"items": []}) + t = AsyncTransport(API, username="u", password="p") + await asyncio.gather(*(t.request("GET", "/schemas", params={"workspace_id": "w1"}) for _ in range(5))) + token_posts = [r for r in httpx_mock.get_requests() if r.url.path == "/api/v2/token"] + assert len(token_posts) == 1 # single login despite 5 concurrent requests + await t.aclose() + + +async def test_concurrent_401s_refresh_once(httpx_mock): + # Fresh client logs in once, then 5 concurrent requests all 401 on the same token. + # _refresh_if_stale must coalesce them into a single /token/refresh; the rest reuse AT2. + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/token", + status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/token/refresh", + status_code=201, + json={"access_token": "AT2", "refresh_token": "RT2"}, + ) + + def schemas_by_token(request): + # Model the server by TOKEN, not registration order: the stale token 401s (forcing a + # single refresh), the rotated token 200s. Order-independent, so the concurrent + # interleaving of initial-vs-retry requests can't consume the wrong canned response. + import httpx + + if request.headers.get("Authorization") == "Bearer AT1": + return httpx.Response(401, json={"detail": "exp"}) + return httpx.Response(200, json={"items": []}) + + httpx_mock.add_callback(schemas_by_token, method="GET", url=f"{API}/api/v2/schemas?workspace_id=w1") + t = AsyncTransport(API, username="u", password="p") + await asyncio.gather(*(t.request("GET", "/schemas", params={"workspace_id": "w1"}) for _ in range(5))) + refresh_posts = [r for r in httpx_mock.get_requests() if r.url.path == "/api/v2/token/refresh"] + assert len(refresh_posts) == 1 # concurrent 401s coalesce into a single refresh + await t.aclose() + + +async def test_concurrent_401s_failed_refresh_coalesces(httpx_mock): + # Login succeeds (AT1); the token is then rejected (401) and the refresh ALSO fails (401), + # so the token is never rotated. All waiters must still coalesce into a SINGLE /token/refresh + # before raising AuthError, not stampede the auth endpoint once per concurrent request. + httpx_mock.add_response( + method="POST", + url=f"{API}/api/v2/token", + status_code=201, + json={"access_token": "AT1", "refresh_token": "RT1"}, + ) + + def refresh_fails(request): + import httpx + + return httpx.Response(401, json={"detail": "refresh dead"}) + + httpx_mock.add_callback(refresh_fails, method="POST", url=f"{API}/api/v2/token/refresh") + + def schemas_401(request): + import httpx + + return httpx.Response(401, json={"detail": "exp"}) + + httpx_mock.add_callback(schemas_401, method="GET", url=f"{API}/api/v2/schemas?workspace_id=w1") + + t = AsyncTransport(API, username="u", password="p") + results = await asyncio.gather( + *(t.request("GET", "/schemas", params={"workspace_id": "w1"}) for _ in range(5)), + return_exceptions=True, + ) + assert all(isinstance(r, AuthError) for r in results) # every request raises after the dead refresh + refresh_posts = [r for r in httpx_mock.get_requests() if r.url.path == "/api/v2/token/refresh"] + assert len(refresh_posts) == 1 # the failed refresh is coalesced, not stampeded per waiter + await t.aclose() + + +async def test_error_mapping_and_204(httpx_mock): + httpx_mock.add_response(url=f"{API}/api/v2/schemas/x", status_code=404, json={"detail": "gone"}) + httpx_mock.add_response(method="POST", url=f"{API}/api/v2/schemas", status_code=422, json={"detail": "bad"}) + httpx_mock.add_response(method="DELETE", url=f"{API}/api/v2/schemas/y/records?ids=a", status_code=204) + t = AsyncTransport(API, api_key="k") + with pytest.raises(NotFoundError): + await t.request("GET", "/schemas/x") + with pytest.raises(ValidationError): + await t.request("POST", "/schemas", json={}) + assert await t.request("DELETE", "/schemas/y/records", params={"ids": "a"}) is None + await t.aclose() diff --git a/extralit/uv.lock b/extralit/uv.lock index 89b2eea7b..af05b68d0 100644 --- a/extralit/uv.lock +++ b/extralit/uv.lock @@ -156,6 +156,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7f/9c/36c5c37947ebfb8c7f22e0eb6e4d188ee2d53aa3880f3f2744fb894f0cb1/anyio-4.12.0-py3-none-any.whl", hash = "sha256:dad2376a628f98eeca4881fc56cd06affd18f659b17a747d3ff0307ced94b1bb", size = 113362, upload-time = "2025-11-28T23:36:57.897Z" }, ] +[[package]] +name = "argcomplete" +version = "3.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/95/c0/c8e94135e66fabf89a120d9b4b123fe6993506beca6c1938a74c24cfa5fd/argcomplete-3.7.0.tar.gz", hash = "sha256:afde224f753f874807b1dc1414e883ab8fe0cda9c04807b6047dcb8e1ac23913", size = 73284, upload-time = "2026-06-30T22:28:22.249Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/12/f6/5b8ec087cd9cfa9449491ec83f76fb6b7006b4dff57d2ba8aaab330fe8e4/argcomplete-3.7.0-py3-none-any.whl", hash = "sha256:d8f0f22d2a8a7caa383be1e22b6caf1ecaf0ebd10d8f83cc125e36540c95830c", size = 42575, upload-time = "2026-06-30T22:28:20.547Z" }, +] + [[package]] name = "asttokens" version = "3.0.1" @@ -826,6 +835,55 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/21/0e/8459ca4413e1a21a06c97d134bfaf18adfd27cea068813dc0faae06cbf00/cssselect2-0.9.0-py3-none-any.whl", hash = "sha256:6a99e5f91f9a016a304dd929b0966ca464bcfda15177b6fb4a118fc0fb5d9563", size = 15453, upload-time = "2026-02-12T17:16:38.317Z" }, ] +[[package]] +name = "datamodel-code-generator" +version = "0.45.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "argcomplete", marker = "python_full_version < '3.10'" }, + { name = "black", marker = "python_full_version < '3.10'" }, + { name = "genson", version = "1.3.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "inflect", marker = "python_full_version < '3.10'" }, + { name = "isort", version = "6.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "jinja2", marker = "python_full_version < '3.10'" }, + { name = "packaging", marker = "python_full_version < '3.10'" }, + { name = "pydantic", marker = "python_full_version < '3.10'" }, + { name = "pyyaml", marker = "python_full_version < '3.10'" }, + { name = "tomli", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f5/45/993fb3f7f4005bfa167fa94929e5737c8f574dc77fb1a4ebca359a106b56/datamodel_code_generator-0.45.0.tar.gz", hash = "sha256:a493add7a59e5a9c1620deb508694bc84e2e2ca80da4bb58a75262a28b9b2c84", size = 554380, upload-time = "2025-12-19T00:13:13.157Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4d/15/b9b64a3576c3ac230c8f69806f32937364f837cda4facaa9b6d09dafd872/datamodel_code_generator-0.45.0-py3-none-any.whl", hash = "sha256:1c7065d3c9ebff7a0c967f2282e0472be971c721593c7e9cbe3f374f378749b8", size = 182003, upload-time = "2025-12-19T00:13:10.422Z" }, +] + +[[package]] +name = "datamodel-code-generator" +version = "0.68.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "argcomplete", marker = "python_full_version >= '3.10'" }, + { name = "black", marker = "python_full_version >= '3.10' and sys_platform != 'emscripten'" }, + { name = "genson", version = "1.4.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "inflect", marker = "python_full_version >= '3.10'" }, + { name = "isort", version = "8.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10' and sys_platform != 'emscripten'" }, + { name = "jinja2", marker = "python_full_version >= '3.10'" }, + { name = "pydantic", marker = "python_full_version >= '3.10'" }, + { name = "pyyaml", marker = "python_full_version >= '3.10'" }, + { name = "tomli", marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/8c/9d/9746c36d7b597235dae8806696fc158fa41a3f78912499f6cfebeb2b7833/datamodel_code_generator-0.68.1.tar.gz", hash = "sha256:3a9f0c501670b35d6ea1764b3ff6c552f1f594c7ee3ed9cab5b3b72f75ff092d", size = 1577260, upload-time = "2026-07-08T00:47:56.697Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f7/ed/03aab13a7b425a816f61c580fc9fb7ad8902fb4839a205dfb2e69a43fdd7/datamodel_code_generator-0.68.1-py3-none-any.whl", hash = "sha256:2784ad944f25579be874d40e7f28ee5e7ab2d4457a08612393c7fd02ea7b7f21", size = 425059, upload-time = "2026-07-08T00:47:54.903Z" }, +] + [[package]] name = "datasets" version = "3.6.0" @@ -948,6 +1006,8 @@ dev = [ { name = "build" }, { name = "cairosvg", version = "2.8.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, { name = "cairosvg", version = "2.9.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "datamodel-code-generator", version = "0.45.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "datamodel-code-generator", version = "0.68.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, { name = "flake8" }, { name = "ipython", version = "8.18.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, { name = "ipython", version = "8.38.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, @@ -998,6 +1058,7 @@ requires-dist = [ dev = [ { name = "build", specifier = ">=1.0.3" }, { name = "cairosvg", specifier = ">=2.7.1" }, + { name = "datamodel-code-generator", specifier = ">=0.26,<0.69" }, { name = "flake8", specifier = ">=5.0.4" }, { name = "ipython", specifier = ">=8.12.3" }, { name = "material-plausible-plugin", specifier = ">=0.2.0" }, @@ -1216,6 +1277,32 @@ http = [ { name = "aiohttp" }, ] +[[package]] +name = "genson" +version = "1.3.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/54/80/b3731c59ae3ccc162bd381517892c4641e1223249e19155f426f696dd208/genson-1.3.1.tar.gz", hash = "sha256:4ac23ffb6ff29bbde4909963e59af09c28acd57f15841208e9a5bcc5008e721b", size = 34958, upload-time = "2026-07-06T02:38:29.528Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/86/7e/42d450f118d635906538a5d7133964fca24d265f50aa044b2e933744e5c6/genson-1.3.1-py3-none-any.whl", hash = "sha256:dc2a997c86beedfbd81fa4419fe9d0456ef56a85241ba38e810111c8f0993c11", size = 21880, upload-time = "2026-07-06T02:38:28.13Z" }, +] + +[[package]] +name = "genson" +version = "1.4.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/f8/53/de162dc8e03fccd9ebe59d17c7812378fe8bd2b604f6b1b94d00165140ac/genson-1.4.0.tar.gz", hash = "sha256:bc7f1c1bae87a21ca44d81149aec95a3f4468d676de9b8b08caa064f3c50b3da", size = 47908, upload-time = "2026-07-06T08:21:50.331Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/16/02/767f744ab6d4cb7761e5008acc3d534b7a0481af62563d52e391fbcb2140/genson-1.4.0-py3-none-any.whl", hash = "sha256:03bc71bbe52defde70660cc4dcd1ea1097997da5a1cbb90a9dbd3acc7c9e1b65", size = 24484, upload-time = "2026-07-06T08:21:49.046Z" }, +] + [[package]] name = "ghp-import" version = "2.1.0" @@ -1415,6 +1502,20 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a4/ed/1f1afb2e9e7f38a545d628f864d562a5ae64fe6f7a10e28ffb9b185b4e89/importlib_resources-6.5.2-py3-none-any.whl", hash = "sha256:789cfdc3ed28c78b67a06acb8126751ced69a3d5f79c095a98298cd8a760ccec", size = 37461, upload-time = "2025-01-03T18:51:54.306Z" }, ] +[[package]] +name = "inflect" +version = "7.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "more-itertools", version = "10.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "more-itertools", version = "11.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "typeguard" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/78/c6/943357d44a21fd995723d07ccaddd78023eace03c1846049a2645d4324a3/inflect-7.5.0.tar.gz", hash = "sha256:faf19801c3742ed5a05a8ce388e0d8fe1a07f8d095c82201eb904f5d27ad571f", size = 73751, upload-time = "2024-12-28T17:11:18.897Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8a/eb/427ed2b20a38a4ee29f24dbe4ae2dafab198674fe9a85e3d6adf9e5f5f41/inflect-7.5.0-py3-none-any.whl", hash = "sha256:2aea70e5e70c35d8350b8097396ec155ffd68def678c7ff97f51aa69c1d92344", size = 35197, upload-time = "2024-12-28T17:11:15.931Z" }, +] + [[package]] name = "iniconfig" version = "2.1.0" @@ -1561,6 +1662,35 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/15/aa/0aca39a37d3c7eb941ba736ede56d689e7be91cab5d9ca846bde3999eba6/isodate-0.7.2-py3-none-any.whl", hash = "sha256:28009937d8031054830160fce6d409ed342816b543597cece116d966c6d99e15", size = 22320, upload-time = "2024-10-08T23:04:09.501Z" }, ] +[[package]] +name = "isort" +version = "6.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "importlib-metadata", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1e/82/fa43935523efdfcce6abbae9da7f372b627b27142c3419fcf13bf5b0c397/isort-6.1.0.tar.gz", hash = "sha256:9b8f96a14cfee0677e78e941ff62f03769a06d412aabb9e2a90487b3b7e8d481", size = 824325, upload-time = "2025-10-01T16:26:45.027Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7f/cc/9b681a170efab4868a032631dea1e8446d8ec718a7f657b94d49d1a12643/isort-6.1.0-py3-none-any.whl", hash = "sha256:58d8927ecce74e5087aef019f778d4081a3b6c98f15a80ba35782ca8a2097784", size = 94329, upload-time = "2025-10-01T16:26:43.291Z" }, +] + +[[package]] +name = "isort" +version = "8.0.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ef/7c/ec4ab396d31b3b395e2e999c8f46dec78c5e29209fac49d1f4dace04041d/isort-8.0.1.tar.gz", hash = "sha256:171ac4ff559cdc060bcfff550bc8404a486fee0caab245679c2abe7cb253c78d", size = 769592, upload-time = "2026-02-28T10:08:20.685Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/3e/95/c7c34aa53c16353c56d0b802fba48d5f5caa2cdee7958acbcb795c830416/isort-8.0.1-py3-none-any.whl", hash = "sha256:28b89bc70f751b559aeca209e6120393d43fbe2490de0559662be7a9787e3d75", size = 89733, upload-time = "2026-02-28T10:08:19.466Z" }, +] + [[package]] name = "jedi" version = "0.19.2" @@ -2188,6 +2318,32 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/54/40/2f69d9858c06d3fd9232fe7ead0c828cc3d64783edb98476782a7f125977/mknotebooks-0.8.0-py3-none-any.whl", hash = "sha256:4a9b998260c09bcc311455a19a44cc395a30ee82dc1e86e3316dd09f2445ebd3", size = 13230, upload-time = "2023-08-07T10:01:06.289Z" }, ] +[[package]] +name = "more-itertools" +version = "10.8.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ea/5d/38b681d3fce7a266dd9ab73c66959406d565b3e85f21d5e66e1181d93721/more_itertools-10.8.0.tar.gz", hash = "sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd", size = 137431, upload-time = "2025-09-02T15:23:11.018Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a4/8e/469e5a4a2f5855992e425f3cb33804cc07bf18d48f2db061aec61ce50270/more_itertools-10.8.0-py3-none-any.whl", hash = "sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b", size = 69667, upload-time = "2025-09-02T15:23:09.635Z" }, +] + +[[package]] +name = "more-itertools" +version = "11.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/de/1d/f4da6f02cdffe04d6362210b807146a26044c88d839208aec273bb0d9184/more_itertools-11.1.0.tar.gz", hash = "sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d", size = 145772, upload-time = "2026-05-22T14:14:29.909Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/3d/1087453384dbde46a8c7f9356eead2c58be8a7bf156bca40243377c85715/more_itertools-11.1.0-py3-none-any.whl", hash = "sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192", size = 72226, upload-time = "2026-05-22T14:14:28.824Z" }, +] + [[package]] name = "multidict" version = "6.7.0"