diff --git a/api.md b/api.md index 92b1af2..aaa2581 100644 --- a/api.md +++ b/api.md @@ -64,7 +64,7 @@ Methods: The `client.v2` sub-client targets LandingAI's next-generation ADE gateway, which lives on its own host (`api.ade.[env].landing.ai`) rather than the V1 host (`api.va.[env].landing.ai`). It is **additive**: `client.v2.*` is a separate surface from the top-level `client.*` (V1) methods documented above, and using it does not change any V1 behavior. See the [README](README.md#environments) for environment selection and usage examples. -`client.v2.parse_jobs`, `client.v2.extract_jobs`, and `client.v2.ground_jobs` all return a single, unified `Job` shape, even though the underlying parse/extract/ground job envelopes differ upstream -- `Job.raw` retains the full original envelope as an escape hatch for any field not surfaced on the typed model. +`client.v2.parse_jobs`, `client.v2.extract_jobs`, and `client.v2.build_schema_jobs` all return a single, unified `Job` shape, even though the underlying parse/extract/build-schema job envelopes differ upstream -- `Job.raw` retains the full original envelope as an escape hatch for any field not surfaced on the typed model. Types: @@ -73,10 +73,13 @@ from landingai_ade.types.v2 import ( Job, JobError, JobStatus, + V2BuildSchemaBilling, + V2BuildSchemaMetadata, + V2BuildSchemaResponse, + V2BuildSchemaWarning, V2ExtractBilling, V2ExtractMetadata, V2ExtractResult, - V2FileUploadResponse, V2GroundBilling, V2GroundMetadata, V2GroundResult, @@ -96,10 +99,10 @@ from landingai_ade.types.v2 import ( ) ``` -- Job -- unified job shape: `job_id`, `status` (JobStatus: `pending` / `processing` / `completed` / `failed` / `cancelled`), `created_at`, `completed_at`, `progress`, `result` (a `V2ParseResponse` for parse jobs, a `V2ExtractResult` for extract jobs, or `None` until completion), `error` (JobError), `raw` (the full original envelope as a `dict`), and the `.is_terminal` property. +- Job -- unified job shape: `job_id`, `status` (JobStatus: `pending` / `processing` / `completed` / `failed` / `cancelled`), `created_at`, `completed_at`, `progress`, `result` (a `V2ParseResponse` for parse jobs, a `V2ExtractResult` for extract jobs, a `V2BuildSchemaResponse` for build-schema jobs, or `None` until completion), `error` (JobError), `raw` (the full original envelope as a `dict`), and the `.is_terminal` property. - V2ParseResponse -- `markdown`, `structure`, `grounding`, `metadata` (V2ParseMetadata, which nests V2ParseBilling and carries `output_markdown_chars`, `range_units`, and `openapi_spec`). `structure` is a typed V2ParseStructure tree (`document` → V2ParsePageV2ParseElement); each node below the root carries its spatial data inline in a V2ParseNodeGrounding (`page`, V2ParseRange, V2ParseBox, normalized page coordinates), and leaf elements additionally carry an `atomic_grounding` list. With `options.inline_markdown`, each node also carries its `markdown` slice. The legacy top-level `grounding` tree (V2ParseGrounding → `V2ParseGroundingPage` → `V2ParseGroundingElement` → `V2ParseGroundingEntry`) is retained for older gateway responses. Element `type`/page `status` are permissive strings and unknown keys are retained. - V2ExtractResult -- `extraction`, `extraction_metadata`, `markdown`, `output_ref`, `schema_violation_error` (set when `strict=False` and the schema had unextractable fields), `warnings`, and `metadata` (V2ExtractMetadata, which carries `model_version`, `input_markdown_chars`, `output_extraction_chars`, `range_units`, `openapi_spec`, and nests V2ExtractBilling). -- V2FileUploadResponse -- `file_ref`. +- V2BuildSchemaResponse -- `extraction_schema` (the generated JSON Schema serialized as a string) and `metadata` (V2BuildSchemaMetadata: `job_id`, `duration_ms`, `openapi_spec`, `filename`/`org_id`/`version` (retained for compatibility), a `warnings` list of V2BuildSchemaWarning (`code`, `msg`), and nested V2BuildSchemaBilling). - V2GroundResult -- `grounding` (a tree mirroring the input `extraction_metadata`, each `{value, ranges}` leaf replaced by the list of `structure` blocks its ranges overlap) and `metadata` (V2GroundMetadata: `job_id`, `duration_ms`, `openapi_spec`, and nested V2GroundBilling). Methods: @@ -126,22 +129,22 @@ Methods: Same polling/timeout semantics as `parse_jobs.wait`. Extract jobs have no `cancelled` status, so `raise_on_failure` only ever triggers on `failed`. -- client.v2.ground(\*, extraction_metadata, structure) -> V2GroundResult +- client.v2.build_schema(\*, markdowns=..., markdown_urls=..., prompt=..., schema=...) -> V2BuildSchemaResponse - Synchronous ground. Maps each extracted field back to the `structure` blocks it was quoted from by overlapping `extraction_metadata` ranges against every block's inline `grounding.range`. Both `extraction_metadata` (e.g. `client.v2.extract(...).extraction_metadata`) and `structure` (e.g. `client.v2.parse(...).structure`) accept a `dict` or a pydantic model. Block ids resolve only against the `structure` supplied here, so pass the parse result the extraction actually came from. Raises `V2SyncTimeoutError` on a 504; use `ground_jobs` for long-running inputs. + Synchronous schema generation. Generate or edit a JSON Schema for extraction from one or more source Markdown documents and/or a natural-language `prompt`, optionally iterating on an existing `schema`. Provide at least one of `markdowns`, `markdown_urls`, `prompt`, or `schema`. `markdowns` entries are each either an inline markdown string or a file upload (`Path`/`bytes`/file object) -- the request is sent as `multipart/form-data` when any entry is a file, and as JSON otherwise. `schema` accepts a pydantic `BaseModel` subclass, a `dict`, or a JSON-encoded string -- all are coerced to a JSON Schema and sent as a JSON-encoded string. Raises `V2SyncTimeoutError` on a 504; use `build_schema_jobs` for long-running inputs. -- client.v2.ground_jobs.create(\*, extraction_metadata, structure, output_save_url=...) -> Job -- client.v2.ground_jobs.get(job_id) -> Job -- client.v2.ground_jobs.list(\*, page=..., page_size=..., status=...) -> JobList[Job] -- client.v2.ground_jobs.wait(job_id, \*, timeout=600, poll_interval=None, raise_on_failure=False) -> Job +- client.v2.build_schema_jobs.create(\*, markdowns=..., markdown_urls=..., prompt=..., schema=..., service_tier=...) -> Job +- client.v2.build_schema_jobs.get(job_id) -> Job +- client.v2.build_schema_jobs.list(\*, page=..., page_size=..., status=...) -> JobList[Job] +- client.v2.build_schema_jobs.wait(job_id, \*, timeout=600, poll_interval=None, raise_on_failure=False) -> Job - Same polling/timeout semantics as `parse_jobs.wait`. Ground jobs have no `cancelled` status, so `raise_on_failure` only ever triggers on `failed`. `output_save_url` (create only) delivers the result out-of-band and reports `output_url` on the completed job instead of an inline `result`. + Same polling/timeout semantics as `parse_jobs.wait`. Build-schema jobs have no `cancelled` status, so `raise_on_failure` only ever triggers on `failed`. -- client.v2.files.upload(\*, file) -> str +- client.v2.ground(\*, extraction_metadata, structure) -> V2GroundResult - Stages a file's bytes on the ADE data plane and returns a `file_ref` string for use in job inputs. Served on the ADE host under `/v1/files` (not `/v2/...`). Raises `LandingAiadeError` if the response has no `file_ref`. + Synchronous ground. Maps each extracted field back to the `structure` blocks it was quoted from by overlapping `extraction_metadata` ranges against every block's inline `grounding.range`. Both `extraction_metadata` (e.g. `client.v2.extract(...).extraction_metadata`) and `structure` (e.g. `client.v2.parse(...).structure`) accept a `dict` or a pydantic model. Block ids resolve only against the `structure` supplied here, so pass the parse result the extraction actually came from. `/v2/ground` is synchronous-only (no async jobs route). Notes: -- `parse_jobs.list` / `extract_jobs.list` / `ground_jobs.list` all return a `JobList` (a `list[Job]` subclass) carrying pagination metadata: `.has_more`, `.org_id`, `.page`, `.page_size`. +- `parse_jobs.list` / `extract_jobs.list` / `build_schema_jobs.list` all return a `JobList` (a `list[Job]` subclass) carrying pagination metadata: `.has_more`, `.org_id`, `.page`, `.page_size`. - All `client.v2.*` methods accept the usual `extra_headers`, `extra_query`, `extra_body`, and `timeout` overrides; sync methods additionally accept `save_to` (parse/extract only, not the job-creation methods) to write the response to disk, mirroring V1's `save_to`. diff --git a/docs/v2-testing.md b/docs/v2-testing.md index 4b6f04b..491aa12 100644 --- a/docs/v2-testing.md +++ b/docs/v2-testing.md @@ -8,8 +8,8 @@ what to check when the upstream spec (`specs/v2-aide.json`) changes. | Layer | Location | What it covers | | --- | --- | --- | -| Response models | `tests/test_v2_types.py` | Deserialization of `V2ParseResponse` / `V2ExtractResult` / `V2GroundResult` and their nested models from plain dicts, including unknown-key tolerance. | -| Job normalization | `tests/test_v2_normalize.py` | `normalize_parse_job` / `normalize_extract_job` / `normalize_ground_job`: envelope → unified `Job` (status, timestamps, `result`, `error`). | +| Response models | `tests/test_v2_types.py` | Deserialization of `V2ParseResponse` / `V2ExtractResult` / `V2BuildSchemaResponse` / `V2GroundResult` and their nested models from plain dicts, including unknown-key tolerance. | +| Job normalization | `tests/test_v2_normalize.py` | `normalize_parse_job` / `normalize_extract_job` / `normalize_build_schema_job`: envelope → unified `Job` (status, timestamps, `result`, `error`). | | Resource wiring | `tests/api_resources/v2/` | `respx`-mocked HTTP: host routing, multipart/JSON bodies, options serialization, job polling. No network. | | Live smoke | `tests/contract/test_v2_smoke.py` | End-to-end calls against staging (marked `contract`; skipped unless `LANDINGAI_ADE_STAGING_APIKEY` is set). | @@ -68,11 +68,33 @@ The async `extract_jobs.create` also accepts `output_save_url` (async jobs only) when set, the finished result is delivered to that URL and the completed job reports `output_url` (on `Job.raw`) instead of an inline `result`. +## Current build-schema-response shape + +`POST /v2/extract/build-schema` (and the completed `build_schema_jobs` result) +returns a `V2BuildSchemaResponse` with: + +- `extraction_schema` — the generated JSON Schema serialized as a **string** (VTRA + parity — a string, not an object). Parse it with `json.loads(...)` to get the + schema dict. +- `metadata` (`V2BuildSchemaMetadata`) — `job_id`, `duration_ms`, `openapi_spec`, + `filename` / `org_id` / `version` (retained for v1 compatibility, unpopulated in + this version), a `warnings` list of `V2BuildSchemaWarning` (`{code, msg}`, e.g. + code `nonconformant_schema`), and `billing` (`V2BuildSchemaBilling`). + +`client.v2.build_schema(...)` generates or edits a JSON Schema from one or more +source Markdown documents and/or a natural-language `prompt`, optionally iterating +on an existing `schema`. At least one of `markdowns`, `markdown_urls`, `prompt`, or +`schema` must be provided. `markdowns` entries are each either an inline markdown +string or a file upload (`Path`/`bytes`/file object); the request is sent as +`multipart/form-data` when any entry is a file and as JSON otherwise. `schema` +accepts a pydantic model, a `dict`, or a JSON string — all coerced to a JSON Schema +and sent as a JSON-encoded string. `build_schema_jobs.create` additionally accepts +`service_tier`. + ## Current ground-response shape -`POST /v2/ground` (and the completed `ground_jobs` result) returns a -`V2GroundResult` — a pure, stateless join that maps each extracted field back to -the `structure` blocks it was quoted from: +`POST /v2/ground` returns a `V2GroundResult` — a pure, stateless join that maps +each extracted field back to the `structure` blocks it was quoted from: - `grounding` — a tree mirroring the input `extraction_metadata`: nested objects and arrays keep their shape, and each `{value, ranges}` leaf is replaced by the @@ -83,13 +105,13 @@ the `structure` blocks it was quoted from: `client.v2.ground(...)` takes `extraction_metadata` and `structure`, each of which accepts a plain `dict` or a pydantic model (so a parse response's `.structure` can -be passed directly). `ground_jobs.create` additionally accepts `output_save_url`. +be passed directly). `/v2/ground` is synchronous-only (no async jobs route). ## Async job envelopes -`normalize_parse_job`, `normalize_extract_job`, and `normalize_ground_job` fold -the upstream job envelopes into the unified `Job`. All are tolerant of field-name -drift: +`normalize_parse_job`, `normalize_extract_job`, and `normalize_build_schema_job` +fold the upstream job envelopes into the unified `Job`. All are tolerant of +field-name drift: - The parse response lives under `result` (older envelopes used `data`). - Failures arrive as a structured `error` object (`{code, message}`); older parse diff --git a/examples/v2_smoke_test.py b/examples/v2_smoke_test.py index 5aefb55..9de96c2 100755 --- a/examples/v2_smoke_test.py +++ b/examples/v2_smoke_test.py @@ -43,7 +43,7 @@ from landingai_ade import LandingAIADE, AsyncLandingAIADE -ALL_CHECKS = ["files", "extract", "extract_jobs", "parse", "parse_jobs"] +ALL_CHECKS = ["extract", "extract_jobs", "parse", "parse_jobs"] # A tiny self-contained markdown document + schema so extract/files can run without any file. SAMPLE_MARKDOWN = "# Acme Inc. — Q1 Report\n\nTotal revenue for the quarter was **$1,250,000**.\n" @@ -116,7 +116,6 @@ def run_sync(args: argparse.Namespace, checks: List[str]) -> int: print(f"V1 base: {client.base_url} | V2 base: {client._v2_base_url}\n") results: dict[str, str] = {} - file_ref: Optional[str] = None def record(name: str, fn: Callable[[], Any]) -> None: print(f"── {name} ".ljust(60, "─")) @@ -130,15 +129,6 @@ def record(name: str, fn: Callable[[], Any]) -> None: traceback.print_exc() print() - if "files" in checks: - - def _files() -> Any: - nonlocal file_ref - file_ref = client.v2.files.upload(file=("doc.md", SAMPLE_MARKDOWN.encode(), "text/markdown")) - return f"file_ref={file_ref}" - - record("files.upload", _files) - if "extract" in checks: def _extract() -> Any: @@ -208,16 +198,6 @@ async def _main() -> int: print(f"[async] V1 base: {client.base_url} | V2 base: {client._v2_base_url}\n") results: dict[str, str] = {} - if "files" in checks: - print("── [async] files.upload ".ljust(60, "─")) - try: - ref = await client.v2.files.upload(file=("doc.md", SAMPLE_MARKDOWN.encode(), "text/markdown")) - results["files.upload"] = "PASS" - print(f" PASS file_ref={ref}\n") - except Exception as exc: # noqa: BLE001 - results["files.upload"] = "FAIL" - print(f" FAIL {type(exc).__name__}: {exc}\n") - if "extract" in checks: print("── [async] v2.extract ".ljust(60, "─")) try: @@ -262,9 +242,9 @@ def main() -> int: args = _make_parser().parse_args() checks = _selected(args.only) if args.use_async: - async_checks = [c for c in checks if c in {"files", "extract", "extract_jobs"}] + async_checks = [c for c in checks if c in {"extract", "extract_jobs"}] if async_checks != checks: - print("note: --async runs files/extract/extract_jobs only (parse checks are sync-only here)\n") + print("note: --async runs extract/extract_jobs only (parse checks are sync-only here)\n") return run_async(args, async_checks) return run_sync(args, checks) diff --git a/specs/_generated/v2_models.py b/specs/_generated/v2_models.py index 4f3b730..2d60b3d 100644 --- a/specs/_generated/v2_models.py +++ b/specs/_generated/v2_models.py @@ -16,10 +16,6 @@ class BaseElementOptions(BaseModel): markdown: Optional[bool] = Field(True, title='Markdown') -class BodyStageFileV1FilesPost(BaseModel): - file: str = Field(..., title='File') - - class Box(BaseModel): """ Axis-aligned bounding box in normalized page coordinates. @@ -53,6 +49,25 @@ class Box(BaseModel): ) +class BuildSchemaWarning(BaseModel): + """ + A structured warning from the schema-generation process (VTRA + ``ExtractWarning``). ``code`` classifies the warning (e.g. + ``nonconformant_schema``); ``msg`` is the human-readable description. + """ + + code: str = Field( + ..., + description='The type of warning, used to translate to a status code downstream.', + title='Code', + ) + msg: str = Field( + ..., + description='Human-readable description of the warning with more details.', + title='Msg', + ) + + class Type(Enum): """ The element type. Determines which optional fields appear on this element. @@ -213,6 +228,48 @@ class V2Billing(BaseModel): ) +class V2BuildSchemaMetadata(BaseModel): + """ + Response metadata for a v2 build-schema call (VTRA + ``BuildSchemaMetadata``). + """ + + billing: Optional[V2Billing] = Field( + None, + description='Billing summary: the service tier the request ran in and the credits charged.', + ) + duration_ms: Optional[int] = Field( + 0, + description='End-to-end request duration in milliseconds.', + title='Duration Ms', + ) + filename: Optional[str] = Field( + None, + description="Name of the first source document. Retained for v1 compatibility but NOT populated in this version — always null (the source is staged as an opaque ref, so the original name isn't carried through). Do not depend on it.", + title='Filename', + ) + job_id: Optional[str] = Field( + '', + description='Gateway job id (workflow id). Matches the billing row id in vision-agent.', + title='Job Id', + ) + openapi_spec: str = Field( + ..., + description='URL of the OpenAPI spec covering this API, for inspection and client generation.', + ) + org_id: Optional[str] = Field(None, description='Organization ID.', title='Org Id') + version: Optional[str] = Field( + None, + description='Model version used for generation. build-schema is version-free (no ``model``/``version`` input), so this is always null. Retained for v1 response-shape compatibility.', + title='Version', + ) + warnings: Optional[list[BuildSchemaWarning]] = Field( + None, + description='Structured warnings from the schema-generation process. Each is a ``{code, msg}`` object (e.g. code ``nonconformant_schema``).', + title='Warnings', + ) + + class V2ExtractMetadata(BaseModel): """ Response metadata for a v2 extract call. @@ -325,14 +382,6 @@ class V2WorkflowMetadata(BaseModel): ) -class ValidationError(BaseModel): - ctx: Optional[dict[str, Any]] = Field(None, title='Context') - input: Optional[Any] = Field(None, title='Input') - loc: list[Union[str, int]] = Field(..., title='Location') - msg: str = Field(..., title='Message') - type: str = Field(..., title='Error Type') - - class WorkflowDocumentInput(BaseModel): """ One declared document input in the ``inputs`` map. @@ -382,10 +431,6 @@ class WorkflowStepOptions(BaseModel): ) -class V1FilesPostResponse(RootModel[dict[str, str]]): - root: dict[str, str] - - class V2ExtractPostRequest(BaseModel): """ Input to V2ExtractOperationWorkflow. @@ -501,7 +546,82 @@ class V2ExtractPostResponse(BaseModel): ) -class V2ExtractJobsGetParametersQuery(BaseModel): +class V2ExtractBuildSchemaPostRequest(BaseModel): + """ + Input to V2BuildSchemaOperationWorkflow — the ``/v2/extract/build-schema`` + request body. + + Mirrors VTRA's ``BuildSchemaRequest``: generate a JSON Schema from one or + more source markdown documents and/or a natural-language ``prompt``, and/or + iterate on an existing ``schema``. At least one of ``markdowns`` / + ``markdown_urls`` / ``prompt`` / ``schema`` must be provided. + """ + + markdown_urls: Optional[list[str]] = Field( + None, + description='URLs to Markdown files to analyze for schema generation.', + title='Markdown Urls', + ) + markdowns: Optional[list[str]] = Field( + None, + description='Markdown files or inline content strings to analyze for schema generation. Multiple documents can be provided for better schema coverage.', + title='Markdowns', + ) + prompt: Optional[str] = Field( + None, + description='Instructions for how to generate or modify the schema.', + title='Prompt', + ) + schema_: Optional[str] = Field( + None, + alias='schema', + description='Existing JSON schema to iterate on or refine.', + title='Schema', + ) + + +class V2ExtractBuildSchemaPostRequest1(BaseModel): + markdown_urls: Optional[list[str]] = Field( + None, + description='URLs to Markdown files to analyze for schema generation. JSON-serialized string in form data.', + title='Markdown Urls', + ) + markdowns: Optional[list[Union[str, bytes]]] = Field( + None, description='Repeat the field for each file upload.' + ) + prompt: Optional[str] = Field( + None, + description='Instructions for how to generate or modify the schema. JSON-serialized string in form data.', + title='Prompt', + ) + schema_: Optional[str] = Field( + None, + alias='schema', + description='Existing JSON schema to iterate on or refine. JSON-serialized string in form data.', + title='Schema', + ) + + +class V2ExtractBuildSchemaPostResponse(BaseModel): + """ + Result returned by V2BuildSchemaOperationWorkflow — the + ``/v2/extract/build-schema`` response body (VTRA ``BuildSchemaResponse``). + + ``extraction_schema`` is the generated JSON Schema serialized as a STRING + (VTRA parity — the v1 field is a string, not an object). + """ + + extraction_schema: str = Field( + ..., + description='The generated JSON schema as a string.', + title='Extraction Schema', + ) + metadata: V2BuildSchemaMetadata = Field( + ..., description='The metadata for the schema generation process.' + ) + + +class V2ExtractBuildSchemaJobsGetParametersQuery(BaseModel): page: Optional[int] = Field( 0, description='Page number (0-indexed).', ge=0, title='Page' ) @@ -526,13 +646,13 @@ class Job(BaseModel): failure_reason: Optional[str] = None job_id: Optional[str] = Field( None, - description='The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.', + description='The unique identifier for this v2-build-schema job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.', ) model_version: Optional[str] = None status: Optional[Status1] = None -class V2ExtractJobsGetResponse(BaseModel): +class V2ExtractBuildSchemaJobsGetResponse(BaseModel): has_more: Optional[bool] = None jobs: Optional[list[Job]] = None page: Optional[int] = None @@ -548,6 +668,164 @@ class ServiceTier2(Enum): priority = 'priority' +class V2ExtractBuildSchemaJobsPostRequest(BaseModel): + """ + Input to V2BuildSchemaOperationWorkflow — the ``/v2/extract/build-schema`` + request body. + + Mirrors VTRA's ``BuildSchemaRequest``: generate a JSON Schema from one or + more source markdown documents and/or a natural-language ``prompt``, and/or + iterate on an existing ``schema``. At least one of ``markdowns`` / + ``markdown_urls`` / ``prompt`` / ``schema`` must be provided. + """ + + markdown_urls: Optional[list[str]] = Field( + None, + description='URLs to Markdown files to analyze for schema generation.', + title='Markdown Urls', + ) + markdowns: Optional[list[str]] = Field( + None, + description='Markdown files or inline content strings to analyze for schema generation. Multiple documents can be provided for better schema coverage.', + title='Markdowns', + ) + prompt: Optional[str] = Field( + None, + description='Instructions for how to generate or modify the schema.', + title='Prompt', + ) + schema_: Optional[str] = Field( + None, + alias='schema', + description='Existing JSON schema to iterate on or refine.', + title='Schema', + ) + service_tier: Optional[ServiceTier2] = Field( + None, + description='Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``.', + ) + + +class V2ExtractBuildSchemaJobsPostRequest1(BaseModel): + markdown_urls: Optional[list[str]] = Field( + None, + description='URLs to Markdown files to analyze for schema generation. JSON-serialized string in form data.', + title='Markdown Urls', + ) + markdowns: Optional[list[Union[str, bytes]]] = Field( + None, description='Repeat the field for each file upload.' + ) + prompt: Optional[str] = Field( + None, + description='Instructions for how to generate or modify the schema. JSON-serialized string in form data.', + title='Prompt', + ) + schema_: Optional[str] = Field( + None, + alias='schema', + description='Existing JSON schema to iterate on or refine. JSON-serialized string in form data.', + title='Schema', + ) + service_tier: Optional[ServiceTier2] = Field( + None, + description='Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``.', + ) + + +class V2ExtractBuildSchemaJobsPostResponse(BaseModel): + created_at: Optional[str] = None + job_id: Optional[str] = Field( + None, + description='The unique identifier for this v2-build-schema job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.', + ) + status: Optional[Status1] = None + + +class Error(BaseModel): + """ + Present once status is ``failed``. + """ + + code: Optional[str] = Field( + None, description='Stable error code (``internal_error`` when unmapped).' + ) + message: Optional[str] = None + + +class Result(BaseModel): + """ + Result returned by V2BuildSchemaOperationWorkflow — the + ``/v2/extract/build-schema`` response body (VTRA ``BuildSchemaResponse``). + + ``extraction_schema`` is the generated JSON Schema serialized as a STRING + (VTRA parity — the v1 field is a string, not an object). + """ + + extraction_schema: str = Field( + ..., + description='The generated JSON schema as a string.', + title='Extraction Schema', + ) + metadata: V2BuildSchemaMetadata = Field( + ..., description='The metadata for the schema generation process.' + ) + + +class V2ExtractBuildSchemaJobsJobIdGetResponse(BaseModel): + completed_at: Optional[str] = Field( + None, description='Present once the job is terminal.' + ) + created_at: Optional[str] = None + error: Optional[Error] = Field( + None, description='Present once status is ``failed``.' + ) + job_id: Optional[str] = Field( + None, + description='The unique identifier for this v2-build-schema job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.', + ) + progress: Optional[float] = Field( + None, + description='Job completion as a decimal from 0 (not started) to 1 (complete). Present while ``processing``.', + ge=0.0, + le=1.0, + ) + result: Optional[Result] = Field( + None, description='Present once status is ``completed``.' + ) + status: Optional[Status1] = None + + +class V2ExtractJobsGetParametersQuery(BaseModel): + page: Optional[int] = Field( + 0, description='Page number (0-indexed).', ge=0, title='Page' + ) + page_size: Optional[int] = Field( + 10, description='Number of items per page.', ge=1, le=100, title='Page Size' + ) + status: Optional[str] = Field( + None, description='Filter by job status.', title='Status' + ) + + +class Job1(BaseModel): + completed_at: Optional[str] = None + created_at: Optional[str] = None + failure_reason: Optional[str] = None + job_id: Optional[str] = Field( + None, + description='The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.', + ) + model_version: Optional[str] = None + status: Optional[Status1] = None + + +class V2ExtractJobsGetResponse(BaseModel): + has_more: Optional[bool] = None + jobs: Optional[list[Job1]] = None + page: Optional[int] = None + page_size: Optional[int] = None + + class V2ExtractJobsPostRequest(BaseModel): """ Input to V2ExtractOperationWorkflow. @@ -653,18 +931,7 @@ class V2ExtractJobsPostResponse(BaseModel): status: Optional[Status1] = None -class Error(BaseModel): - """ - Present once status is ``failed``. - """ - - code: Optional[str] = Field( - None, description='Stable error code (``internal_error`` when unmapped).' - ) - message: Optional[str] = None - - -class Result(BaseModel): +class Result1(BaseModel): """ Result returned by V2ExtractOperationWorkflow — the ``/v2/extract`` response body (``docs/extract-v2-proposal.md`` → Response). @@ -723,7 +990,7 @@ class V2ExtractJobsJobIdGetResponse(BaseModel): ge=0.0, le=1.0, ) - result: Optional[Result] = Field( + result: Optional[Result1] = Field( None, description='Present once status is ``completed`` and ``output_save_url`` was not set. When ``output_save_url`` was set, the result is delivered there and ``output_url`` is returned instead.', ) @@ -827,167 +1094,6 @@ class V2GroundPostResponse(BaseModel): ) -class V2GroundJobsGetParametersQuery(BaseModel): - page: Optional[int] = Field( - 0, description='Page number (0-indexed).', ge=0, title='Page' - ) - page_size: Optional[int] = Field( - 10, description='Number of items per page.', ge=1, le=100, title='Page Size' - ) - status: Optional[str] = Field( - None, description='Filter by job status.', title='Status' - ) - - -class Job1(BaseModel): - completed_at: Optional[str] = None - created_at: Optional[str] = None - failure_reason: Optional[str] = None - job_id: Optional[str] = Field( - None, - description='The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.', - ) - model_version: Optional[str] = None - status: Optional[Status1] = None - - -class V2GroundJobsGetResponse(BaseModel): - has_more: Optional[bool] = None - jobs: Optional[list[Job1]] = None - page: Optional[int] = None - page_size: Optional[int] = None - - -class V2GroundJobsPostRequest(BaseModel): - """ - Input to V2GroundOperationWorkflow — the ``/v2/ground`` request body. - - A pure, stateless join: each ``extraction_metadata`` leaf's ``ranges`` - (char offsets into the markdown both artifacts were produced from) is - overlapped against the ``grounding.range`` carried on every ``structure`` - block, and the matching blocks are returned. Nothing is stored server-side, - so block ids in the response resolve only against the ``structure`` tree - supplied here — pairing an extraction with the parse result it actually - came from is the caller's responsibility. - """ - - extraction_metadata: dict[str, Any] = Field( - ..., - description="The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown.", - examples=[ - { - 'invoice_number': { - 'ranges': [{'end': 31, 'start': 13}], - 'value': 'INV-042', - } - } - ], - title='Extraction Metadata', - ) - structure: dict[str, Any] = Field( - ..., - description='The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree.', - title='Structure', - ) - - -class V2GroundJobsPostRequest1(BaseModel): - extraction_metadata: dict[str, Any] = Field( - ..., - description="The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown. JSON-serialized string in form data.", - examples=[ - { - 'invoice_number': { - 'ranges': [{'end': 31, 'start': 13}], - 'value': 'INV-042', - } - } - ], - title='Extraction Metadata', - ) - structure: dict[str, Any] = Field( - ..., - description='The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree. JSON-serialized string in form data.', - title='Structure', - ) - - -class V2GroundJobsPostResponse(BaseModel): - created_at: Optional[str] = None - job_id: Optional[str] = Field( - None, - description='The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.', - ) - status: Optional[Status1] = None - - -class Result1(BaseModel): - """ - Result returned by V2GroundOperationWorkflow — the ``/v2/ground`` - response body. - - ``grounding`` MIRRORS the ``extraction_metadata`` tree: nested objects and - arrays keep their shape, and each ``{value, ranges}`` leaf is replaced by - the list of structure blocks its ranges overlap (the block-hit shape - documented on the field). It is NOT a flat map — a nested schema field like - ``issuer.name`` resolves to ``grounding["issuer"]["name"]``. - """ - - grounding: dict[str, Any] = Field( - ..., - description="A tree mirroring ``extraction_metadata``: nested objects and arrays keep their shape, and each ``{value, ranges}`` leaf is replaced by the list of blocks its ranges overlap, in reading order. Each entry carries ``block_id`` and ``type`` identifying the matched ``structure`` block, ``parent_id`` naming the enclosing block for nested blocks (table cells), and the block's own ``grounding`` object (``{page, range, box}``) verbatim. When the block itself carries ``atomic_grounding``, the entry also lists the overlapping subset as ``{index, page, range, box}`` objects, where ``index`` is the position in the block's own ``atomic_grounding`` array; ``[]`` means the block matched but no individual entry did, and the key is omitted for blocks that carry no ``atomic_grounding``. A leaf is ``null`` when its ``ranges`` was ``null`` (a synthesised value, with no supporting passage to look up) and ``[]`` when valid ranges overlapped no block (which usually indicates a mismatched extraction/structure pair).", - examples=[ - { - 'invoice_number': [ - { - 'atomic_grounding': [], - 'block_id': 'text-1', - 'grounding': { - 'box': { - 'xmax': 0.42, - 'xmin': 0.1, - 'ymax': 0.15, - 'ymin': 0.12, - }, - 'page': 1, - 'range': {'end': 31, 'start': 13}, - }, - 'type': 'text', - } - ] - } - ], - title='Grounding', - ) - metadata: V2GroundMetadata = Field( - ..., description='Request metadata (job_id, duration_ms, credit_usage).' - ) - - -class V2GroundJobsJobIdGetResponse(BaseModel): - completed_at: Optional[str] = Field( - None, description='Present once the job is terminal.' - ) - created_at: Optional[str] = None - error: Optional[Error] = Field( - None, description='Present once status is ``failed``.' - ) - job_id: Optional[str] = Field( - None, - description='The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.', - ) - progress: Optional[float] = Field( - None, - description='Job completion as a decimal from 0 (not started) to 1 (complete). Present while ``processing``.', - ge=0.0, - le=1.0, - ) - result: Optional[Result1] = Field( - None, description='Present once status is ``completed``.' - ) - status: Optional[Status1] = None - - class V2ParseJobsGetParametersQuery(BaseModel): page: Optional[int] = Field( 0, description='Page number (0-indexed).', ge=0, title='Page' @@ -1047,7 +1153,7 @@ class V2ParseJobsGetResponse(BaseModel): page_size: Optional[int] = Field(None, description='Items per page.') -class ServiceTier4(Enum): +class ServiceTier6(Enum): """ Async service tier (``POST /jobs`` only). ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``. """ @@ -1175,7 +1281,7 @@ class V2WorkflowJobsGetResponse(BaseModel): page_size: Optional[int] = None -class ServiceTier5(Enum): +class ServiceTier7(Enum): """ Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``. """ @@ -1302,10 +1408,6 @@ class Grounding(BaseModel): ) -class HTTPValidationError(BaseModel): - detail: Optional[list[ValidationError]] = Field(None, title='Detail') - - class PrebuiltWorkflowStep(BaseModel): """ A Phase 1 step: references a server-defined pipeline by name. @@ -1407,7 +1509,7 @@ class V2ParseJobsPostRequest(BaseModel): None, description='Public URL the full response is delivered to; the API response then carries ``output_url`` instead of inline data.', ) - service_tier: Optional[ServiceTier4] = Field( + service_tier: Optional[ServiceTier6] = Field( None, description='Async service tier (``POST /jobs`` only). ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``.', ) @@ -1557,7 +1659,7 @@ class V2WorkflowJobsPostRequest(BaseModel): ], title='Output', ) - service_tier: Optional[ServiceTier5] = Field( + service_tier: Optional[ServiceTier7] = Field( None, description='Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``.', ) @@ -1598,7 +1700,7 @@ class V2WorkflowJobsPostRequest1(BaseModel): ], title='Output', ) - service_tier: Optional[ServiceTier5] = Field( + service_tier: Optional[ServiceTier7] = Field( None, description='Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``.', ) diff --git a/specs/v2-aide.json b/specs/v2-aide.json index d28e2a5..d5aa8a0 100644 --- a/specs/v2-aide.json +++ b/specs/v2-aide.json @@ -44,20 +44,6 @@ "title": "BlocksOptions", "type": "object" }, - "Body_stage_file_v1_files_post": { - "properties": { - "file": { - "contentMediaType": "application/octet-stream", - "title": "File", - "type": "string" - } - }, - "required": [ - "file" - ], - "title": "Body_stage_file_v1_files_post", - "type": "object" - }, "Box": { "description": "Axis-aligned bounding box in normalized page coordinates.\n\nEvery value is a fraction of the page's width (`xmin`/`xmax`) or height\n(`ymin`/`ymax`) in `[0, 1]`, with at most 8 decimal places. To convert to\npixels, multiply by the dimensions of whatever raster of the page you are\ndrawing on. Coordinates are clamped and rounded at construction so the\nin-process value always equals the serialized one.", "properties": { @@ -91,6 +77,27 @@ "title": "Box", "type": "object" }, + "BuildSchemaWarning": { + "description": "A structured warning from the schema-generation process (VTRA\n``ExtractWarning``). ``code`` classifies the warning (e.g.\n``nonconformant_schema``); ``msg`` is the human-readable description.", + "properties": { + "code": { + "description": "The type of warning, used to translate to a status code downstream.", + "title": "Code", + "type": "string" + }, + "msg": { + "description": "Human-readable description of the warning with more details.", + "title": "Msg", + "type": "string" + } + }, + "required": [ + "code", + "msg" + ], + "title": "BuildSchemaWarning", + "type": "object" + }, "Document": { "properties": { "children": { @@ -314,19 +321,6 @@ "title": "Grounding", "type": "object" }, - "HTTPValidationError": { - "properties": { - "detail": { - "items": { - "$ref": "#/components/schemas/ValidationError" - }, - "title": "Detail", - "type": "array" - } - }, - "title": "HTTPValidationError", - "type": "object" - }, "Page": { "properties": { "children": { @@ -684,6 +678,90 @@ "title": "V2Billing", "type": "object" }, + "V2BuildSchemaMetadata": { + "description": "Response metadata for a v2 build-schema call (VTRA\n``BuildSchemaMetadata``).", + "properties": { + "billing": { + "anyOf": [ + { + "$ref": "#/components/schemas/V2Billing" + }, + { + "type": "null" + } + ], + "description": "Billing summary: the service tier the request ran in and the credits charged." + }, + "duration_ms": { + "default": 0, + "description": "End-to-end request duration in milliseconds.", + "title": "Duration Ms", + "type": "integer" + }, + "filename": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Name of the first source document. Retained for v1 compatibility but NOT populated in this version — always null (the source is staged as an opaque ref, so the original name isn't carried through). Do not depend on it.", + "title": "Filename" + }, + "job_id": { + "default": "", + "description": "Gateway job id (workflow id). Matches the billing row id in vision-agent.", + "title": "Job Id", + "type": "string" + }, + "openapi_spec": { + "description": "URL of the OpenAPI spec covering this API, for inspection and client generation.", + "type": "string" + }, + "org_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Organization ID.", + "title": "Org Id" + }, + "version": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Model version used for generation. build-schema is version-free (no ``model``/``version`` input), so this is always null. Retained for v1 response-shape compatibility.", + "title": "Version" + }, + "warnings": { + "description": "Structured warnings from the schema-generation process. Each is a ``{code, msg}`` object (e.g. code ``nonconformant_schema``).", + "items": { + "$ref": "#/components/schemas/BuildSchemaWarning" + }, + "title": "Warnings", + "type": "array" + } + }, + "required": [ + "openapi_spec" + ], + "title": "V2BuildSchemaMetadata", + "type": "object" + }, "V2ExtractMetadata": { "description": "Response metadata for a v2 extract call.", "properties": { @@ -862,46 +940,6 @@ "title": "V2WorkflowMetadata", "type": "object" }, - "ValidationError": { - "properties": { - "ctx": { - "title": "Context", - "type": "object" - }, - "input": { - "title": "Input" - }, - "loc": { - "items": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "integer" - } - ] - }, - "title": "Location", - "type": "array" - }, - "msg": { - "title": "Message", - "type": "string" - }, - "type": { - "title": "Error Type", - "type": "string" - } - }, - "required": [ - "loc", - "msg", - "type" - ], - "title": "ValidationError", - "type": "object" - }, "WorkflowDocumentInput": { "description": "One declared document input in the ``inputs`` map.\n\nA caller supplies exactly one of ``document`` or ``document_url``:\n\n* ``document`` — name of a multipart form part carrying the raw bytes. The\n gateway stages those bytes internally before the workflow runs.\n* ``document_url`` — public URL (SSRF-validated at submit; fetched and staged\n by the gateway before the workflow starts).", "properties": { @@ -975,52 +1013,6 @@ }, "openapi": "3.1.0", "paths": { - "/v1/files": { - "post": { - "description": "Stage bytes on the data plane; pass the returned ``file_ref`` in\njob inputs (the contract request types take ``*_ref`` fields).", - "operationId": "stage_file_v1_files_post", - "requestBody": { - "content": { - "multipart/form-data": { - "schema": { - "$ref": "#/components/schemas/Body_stage_file_v1_files_post" - } - } - }, - "required": true - }, - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "additionalProperties": { - "type": "string" - }, - "title": "Response Stage File V1 Files Post", - "type": "object" - } - } - }, - "description": "Successful Response" - }, - "422": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/HTTPValidationError" - } - } - }, - "description": "Validation Error" - } - }, - "summary": "Stage an input file", - "tags": [ - "files" - ] - } - }, "/v2/extract": { "post": { "description": "Extract structured data from a Markdown document according to a JSON schema, with character-span grounding into the source Markdown. Runs synchronously and returns the result inline.", @@ -1273,59 +1265,246 @@ ] } }, - "/v2/extract/jobs": { - "get": { - "description": "List your Extract jobs, newest first.", - "operationId": "v2-extract_list_jobs", - "parameters": [ - { - "description": "Page number (0-indexed).", - "in": "query", - "name": "page", - "required": false, - "schema": { - "default": 0, - "description": "Page number (0-indexed).", - "minimum": 0, - "title": "Page", - "type": "integer" - } - }, - { - "description": "Number of items per page.", - "in": "query", - "name": "page_size", - "required": false, - "schema": { - "default": 10, - "description": "Number of items per page.", - "maximum": 100, - "minimum": 1, - "title": "Page Size", - "type": "integer" - } - }, - { - "description": "Filter by job status.", - "in": "query", - "name": "status", - "required": false, - "schema": { - "anyOf": [ - { - "type": "string" + "/v2/extract/build-schema": { + "post": { + "description": "Generate or edit a JSON Schema for extraction from one or more source Markdown documents and/or a natural-language prompt. Runs synchronously and returns the result inline.", + "operationId": "v2-build-schema_run_sync", + "requestBody": { + "content": { + "application/json": { + "schema": { + "description": "Input to V2BuildSchemaOperationWorkflow — the ``/v2/extract/build-schema``\nrequest body.\n\nMirrors VTRA's ``BuildSchemaRequest``: generate a JSON Schema from one or\nmore source markdown documents and/or a natural-language ``prompt``, and/or\niterate on an existing ``schema``. At least one of ``markdowns`` /\n``markdown_urls`` / ``prompt`` / ``schema`` must be provided.", + "properties": { + "markdown_urls": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "default": null, + "description": "URLs to Markdown files to analyze for schema generation.", + "title": "Markdown Urls" + }, + "markdowns": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Markdown files or inline content strings to analyze for schema generation. Multiple documents can be provided for better schema coverage.", + "title": "Markdowns" + }, + "prompt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Instructions for how to generate or modify the schema.", + "title": "Prompt" + }, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Existing JSON schema to iterate on or refine.", + "title": "Schema" + } }, - { - "type": "null" - } - ], - "description": "Filter by job status.", - "title": "Status" - } - } - ], - "responses": { - "200": { + "title": "V2BuildSchemaRequest", + "type": "object" + } + }, + "multipart/form-data": { + "schema": { + "properties": { + "markdown_urls": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "default": null, + "description": "URLs to Markdown files to analyze for schema generation. JSON-serialized string in form data.", + "title": "Markdown Urls" + }, + "markdowns": { + "description": "Repeat the field for each file upload.", + "items": { + "anyOf": [ + { + "description": "Markdown files or inline content strings to analyze for schema generation. Multiple documents can be provided for better schema coverage.", + "type": "string" + }, + { + "description": "File upload.", + "format": "binary", + "type": "string" + } + ] + }, + "type": "array" + }, + "prompt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Instructions for how to generate or modify the schema. JSON-serialized string in form data.", + "title": "Prompt" + }, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Existing JSON schema to iterate on or refine. JSON-serialized string in form data.", + "title": "Schema" + } + }, + "type": "object" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "description": "Result returned by V2BuildSchemaOperationWorkflow — the\n``/v2/extract/build-schema`` response body (VTRA ``BuildSchemaResponse``).\n\n``extraction_schema`` is the generated JSON Schema serialized as a STRING\n(VTRA parity — the v1 field is a string, not an object).", + "properties": { + "extraction_schema": { + "description": "The generated JSON schema as a string.", + "title": "Extraction Schema", + "type": "string" + }, + "metadata": { + "$ref": "#/components/schemas/V2BuildSchemaMetadata", + "description": "The metadata for the schema generation process." + } + }, + "required": [ + "extraction_schema", + "metadata" + ], + "title": "V2BuildSchemaResponse", + "type": "object" + } + } + }, + "description": "v2-build-schema result" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." + } + }, + "summary": "ADE Extract", + "tags": [ + "Extract" + ] + } + }, + "/v2/extract/build-schema/jobs": { + "get": { + "description": "List your Extract jobs, newest first.", + "operationId": "v2-build-schema_list_jobs", + "parameters": [ + { + "description": "Page number (0-indexed).", + "in": "query", + "name": "page", + "required": false, + "schema": { + "default": 0, + "description": "Page number (0-indexed).", + "minimum": 0, + "title": "Page", + "type": "integer" + } + }, + { + "description": "Number of items per page.", + "in": "query", + "name": "page_size", + "required": false, + "schema": { + "default": 10, + "description": "Number of items per page.", + "maximum": 100, + "minimum": 1, + "title": "Page Size", + "type": "integer" + } + }, + { + "description": "Filter by job status.", + "in": "query", + "name": "status", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Filter by job status.", + "title": "Status" + } + } + ], + "responses": { + "200": { "content": { "application/json": { "schema": { @@ -1355,7 +1534,7 @@ ] }, "job_id": { - "description": "The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "description": "The unique identifier for this v2-build-schema job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", "type": "string" }, "model_version": { @@ -1408,41 +1587,47 @@ ] }, "post": { - "description": "Extract structured data from a Markdown document according to a JSON schema, with character-span grounding into the source Markdown. Runs asynchronously and returns a job ID; use it to poll for status and retrieve the result once processing completes.", - "operationId": "v2-extract_create_job", + "description": "Generate or edit a JSON Schema for extraction from one or more source Markdown documents and/or a natural-language prompt. Runs asynchronously and returns a job ID; use it to poll for status and retrieve the result once processing completes.", + "operationId": "v2-build-schema_create_job", "requestBody": { "content": { "application/json": { "schema": { - "description": "Input to V2ExtractOperationWorkflow.\n\nProvide the markdown as an inline ``markdown`` string, as a multipart file\npart named ``markdown`` (for large inputs — the gateway stages the upload\ninternally), or via a public ``markdown_url``. Exactly one source must be\nsupplied.", + "description": "Input to V2BuildSchemaOperationWorkflow — the ``/v2/extract/build-schema``\nrequest body.\n\nMirrors VTRA's ``BuildSchemaRequest``: generate a JSON Schema from one or\nmore source markdown documents and/or a natural-language ``prompt``, and/or\niterate on an existing ``schema``. At least one of ``markdowns`` /\n``markdown_urls`` / ``prompt`` / ``schema`` must be provided.", "properties": { - "markdown": { + "markdown_urls": { "anyOf": [ { - "type": "string" + "items": { + "type": "string" + }, + "type": "array" }, { "type": "null" } ], "default": null, - "description": "Markdown string to extract from, or a multipart FILE part carrying the markdown (large inputs — uploads are staged by the gateway). Can come from any source — LandingAI parse output, a third-party parser, or hand-authored text. When the markdown was produced by ``POST /v2/parse``, it ends with a ```` comment that the service reads automatically and echoes as ``metadata.doc_id``.", - "title": "Markdown" + "description": "URLs to Markdown files to analyze for schema generation.", + "title": "Markdown Urls" }, - "markdown_url": { + "markdowns": { "anyOf": [ { - "type": "string" + "items": { + "type": "string" + }, + "type": "array" }, { "type": "null" } ], "default": null, - "description": "URL to fetch the markdown from. Must be a public http(s) URL; private/loopback IPs are rejected at submit time.", - "title": "Markdown Url" + "description": "Markdown files or inline content strings to analyze for schema generation. Multiple documents can be provided for better schema coverage.", + "title": "Markdowns" }, - "model": { + "prompt": { "anyOf": [ { "type": "string" @@ -1452,22 +1637,10 @@ } ], "default": null, - "description": "The version of the model to use for extraction. Use ``extract-latest`` to use the latest version.", - "title": "Model" - }, - "options": { - "anyOf": [ - { - "$ref": "#/components/schemas/V2ExtractOptions" - }, - { - "type": "null" - } - ], - "default": null, - "description": "Extraction options (``strict``). Omit for defaults." + "description": "Instructions for how to generate or modify the schema.", + "title": "Prompt" }, - "output_save_url": { + "schema": { "anyOf": [ { "type": "string" @@ -1477,27 +1650,8 @@ } ], "default": null, - "description": "URL to save the result to — e.g. a presigned S3 PUT URL. Async jobs only. When set, the finished result is delivered (HTTP PUT) to this URL and the completed job reports ``output_url`` instead of an inline ``result``. Must be a public http(s) URL; private/loopback IPs are rejected at submit time.", - "title": "Output Save Url" - }, - "schema": { - "additionalProperties": true, - "description": "JSON Schema describing the fields to extract. The schema must be an object type with a ``properties`` map of field names to their types and descriptions.", - "example": { - "properties": { - "revenue": { - "description": "Q1 revenue figure", - "type": "string" - }, - "summary": { - "description": "Executive summary", - "type": "string" - } - }, - "type": "object" - }, - "title": "Schema", - "type": "object" + "description": "Existing JSON schema to iterate on or refine.", + "title": "Schema" }, "service_tier": { "anyOf": [ @@ -1515,43 +1669,47 @@ "description": "Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``." } }, - "required": [ - "schema" - ], - "title": "V2ExtractRequest", + "title": "V2BuildSchemaRequest", "type": "object" } }, "multipart/form-data": { "schema": { "properties": { - "markdown": { - "anyOf": [ - { - "description": "Markdown string to extract from, or a multipart FILE part carrying the markdown (large inputs — uploads are staged by the gateway). Can come from any source — LandingAI parse output, a third-party parser, or hand-authored text. When the markdown was produced by ``POST /v2/parse``, it ends with a ```` comment that the service reads automatically and echoes as ``metadata.doc_id``.", - "type": "string" - }, - { - "description": "File upload.", - "format": "binary", - "type": "string" - } - ] - }, - "markdown_url": { + "markdown_urls": { "anyOf": [ { - "type": "string" + "items": { + "type": "string" + }, + "type": "array" }, { "type": "null" } ], "default": null, - "description": "URL to fetch the markdown from. Must be a public http(s) URL; private/loopback IPs are rejected at submit time. JSON-serialized string in form data.", - "title": "Markdown Url" + "description": "URLs to Markdown files to analyze for schema generation. JSON-serialized string in form data.", + "title": "Markdown Urls" }, - "model": { + "markdowns": { + "description": "Repeat the field for each file upload.", + "items": { + "anyOf": [ + { + "description": "Markdown files or inline content strings to analyze for schema generation. Multiple documents can be provided for better schema coverage.", + "type": "string" + }, + { + "description": "File upload.", + "format": "binary", + "type": "string" + } + ] + }, + "type": "array" + }, + "prompt": { "anyOf": [ { "type": "string" @@ -1561,60 +1719,29 @@ } ], "default": null, - "description": "The version of the model to use for extraction. Use ``extract-latest`` to use the latest version. JSON-serialized string in form data.", - "title": "Model" + "description": "Instructions for how to generate or modify the schema. JSON-serialized string in form data.", + "title": "Prompt" }, - "options": { + "schema": { "anyOf": [ { - "$ref": "#/components/schemas/V2ExtractOptions" + "type": "string" }, { "type": "null" } ], "default": null, - "description": "Extraction options (``strict``). Omit for defaults. JSON-serialized string in form data." + "description": "Existing JSON schema to iterate on or refine. JSON-serialized string in form data.", + "title": "Schema" }, - "output_save_url": { + "service_tier": { "anyOf": [ { - "type": "string" - }, - { - "type": "null" - } - ], - "default": null, - "description": "URL to save the result to — e.g. a presigned S3 PUT URL. Async jobs only. When set, the finished result is delivered (HTTP PUT) to this URL and the completed job reports ``output_url`` instead of an inline ``result``. Must be a public http(s) URL; private/loopback IPs are rejected at submit time. JSON-serialized string in form data.", - "title": "Output Save Url" - }, - "schema": { - "additionalProperties": true, - "description": "JSON Schema describing the fields to extract. The schema must be an object type with a ``properties`` map of field names to their types and descriptions. JSON-serialized string in form data.", - "example": { - "properties": { - "revenue": { - "description": "Q1 revenue figure", - "type": "string" - }, - "summary": { - "description": "Executive summary", - "type": "string" - } - }, - "type": "object" - }, - "title": "Schema", - "type": "object" - }, - "service_tier": { - "anyOf": [ - { - "enum": [ - "standard", - "priority" - ], + "enum": [ + "standard", + "priority" + ], "type": "string" }, { @@ -1624,9 +1751,6 @@ "description": "Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``." } }, - "required": [ - "schema" - ], "type": "object" } } @@ -1646,7 +1770,7 @@ ] }, "job_id": { - "description": "The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "description": "The unique identifier for this v2-build-schema job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", "type": "string" }, "status": { @@ -1682,10 +1806,10 @@ ] } }, - "/v2/extract/jobs/{job_id}": { + "/v2/extract/build-schema/jobs/{job_id}": { "get": { "description": "Get the status of an async Extract job, including its result once the job has completed.", - "operationId": "v2-extract_get_job", + "operationId": "v2-build-schema_get_job", "parameters": [ { "description": "The identifier of the job to retrieve, as returned by the create-job request.", @@ -1729,16 +1853,9 @@ "type": "object" }, "job_id": { - "description": "The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "description": "The unique identifier for this v2-build-schema job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", "type": "string" }, - "output_url": { - "description": "The URL the result was delivered to. Present once the job has ``completed`` and ``output_save_url`` was set, instead of inline ``result``.", - "type": [ - "string", - "null" - ] - }, "progress": { "description": "Job completion as a decimal from 0 (not started) to 1 (complete). Present while ``processing``.", "maximum": 1, @@ -1748,66 +1865,30 @@ "result": { "anyOf": [ { - "description": "Result returned by V2ExtractOperationWorkflow — the ``/v2/extract``\nresponse body (``docs/extract-v2-proposal.md`` → Response).\n\n``extraction`` and ``extraction_metadata`` mirror each other structurally:\nleaf values in ``extraction`` are replaced by ``ExtractionFieldMetadata``\nobjects in ``extraction_metadata``.", + "description": "Result returned by V2BuildSchemaOperationWorkflow — the\n``/v2/extract/build-schema`` response body (VTRA ``BuildSchemaResponse``).\n\n``extraction_schema`` is the generated JSON Schema serialized as a STRING\n(VTRA parity — the v1 field is a string, not an object).", "properties": { - "extraction": { - "additionalProperties": true, - "description": "Extracted values conforming to the request ``schema``.", - "title": "Extraction", - "type": "object" - }, - "extraction_metadata": { - "additionalProperties": true, - "description": "Per-field metadata, mirroring ``extraction`` with leaf values replaced by ``{value, ranges}`` objects.", - "title": "Extraction Metadata", - "type": "object" - }, - "markdown": { - "description": "Echoed input markdown.", - "title": "Markdown", + "extraction_schema": { + "description": "The generated JSON schema as a string.", + "title": "Extraction Schema", "type": "string" }, "metadata": { - "$ref": "#/components/schemas/V2ExtractMetadata", - "description": "Request metadata (job_id, model_version, duration_ms, doc_id, billing)." - }, - "schema_violation_error": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "default": null, - "description": "Set when ``options.strict`` is false and the schema contained fields the model could not extract — the extraction is partial.", - "title": "Schema Violation Error" - }, - "warnings": { - "description": "Non-fatal warnings emitted during extraction.", - "items": { - "additionalProperties": true, - "type": "object" - }, - "title": "Warnings", - "type": "array" + "$ref": "#/components/schemas/V2BuildSchemaMetadata", + "description": "The metadata for the schema generation process." } }, "required": [ - "extraction", - "extraction_metadata", - "markdown", + "extraction_schema", "metadata" ], - "title": "V2ExtractResult", + "title": "V2BuildSchemaResponse", "type": "object" }, { "type": "null" } ], - "description": "Present once status is ``completed`` and ``output_save_url`` was not set. When ``output_save_url`` was set, the result is delivered there and ``output_url`` is returned instead." + "description": "Present once status is ``completed``." }, "status": { "enum": [ @@ -1852,157 +1933,10 @@ ] } }, - "/v2/ground": { - "post": { - "description": "Map extracted fields to the document blocks they were quoted from. Takes the extraction_metadata from an extract call and the structure tree from the parse call the markdown came from, and returns, for every extracted field, the overlapping blocks with their ids, page numbers, and bounding boxes. Runs synchronously and returns the result inline.", - "operationId": "v2-ground_run_sync", - "requestBody": { - "content": { - "application/json": { - "schema": { - "description": "Input to V2GroundOperationWorkflow — the ``/v2/ground`` request body.\n\nA pure, stateless join: each ``extraction_metadata`` leaf's ``ranges``\n(char offsets into the markdown both artifacts were produced from) is\noverlapped against the ``grounding.range`` carried on every ``structure``\nblock, and the matching blocks are returned. Nothing is stored server-side,\nso block ids in the response resolve only against the ``structure`` tree\nsupplied here — pairing an extraction with the parse result it actually\ncame from is the caller's responsibility.", - "properties": { - "extraction_metadata": { - "additionalProperties": true, - "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown.", - "example": { - "invoice_number": { - "ranges": [ - { - "end": 31, - "start": 13 - } - ], - "value": "INV-042" - } - }, - "title": "Extraction Metadata", - "type": "object" - }, - "structure": { - "additionalProperties": true, - "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree.", - "title": "Structure", - "type": "object" - } - }, - "required": [ - "extraction_metadata", - "structure" - ], - "title": "V2GroundRequest", - "type": "object" - } - }, - "multipart/form-data": { - "schema": { - "properties": { - "extraction_metadata": { - "additionalProperties": true, - "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown. JSON-serialized string in form data.", - "example": { - "invoice_number": { - "ranges": [ - { - "end": 31, - "start": 13 - } - ], - "value": "INV-042" - } - }, - "title": "Extraction Metadata", - "type": "object" - }, - "structure": { - "additionalProperties": true, - "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree. JSON-serialized string in form data.", - "title": "Structure", - "type": "object" - } - }, - "required": [ - "extraction_metadata", - "structure" - ], - "type": "object" - } - } - }, - "required": true - }, - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "description": "Result returned by V2GroundOperationWorkflow — the ``/v2/ground``\nresponse body.\n\n``grounding`` MIRRORS the ``extraction_metadata`` tree: nested objects and\narrays keep their shape, and each ``{value, ranges}`` leaf is replaced by\nthe list of structure blocks its ranges overlap (the block-hit shape\ndocumented on the field). It is NOT a flat map — a nested schema field like\n``issuer.name`` resolves to ``grounding[\"issuer\"][\"name\"]``.", - "properties": { - "grounding": { - "additionalProperties": true, - "description": "A tree mirroring ``extraction_metadata``: nested objects and arrays keep their shape, and each ``{value, ranges}`` leaf is replaced by the list of blocks its ranges overlap, in reading order. Each entry carries ``block_id`` and ``type`` identifying the matched ``structure`` block, ``parent_id`` naming the enclosing block for nested blocks (table cells), and the block's own ``grounding`` object (``{page, range, box}``) verbatim. When the block itself carries ``atomic_grounding``, the entry also lists the overlapping subset as ``{index, page, range, box}`` objects, where ``index`` is the position in the block's own ``atomic_grounding`` array; ``[]`` means the block matched but no individual entry did, and the key is omitted for blocks that carry no ``atomic_grounding``. A leaf is ``null`` when its ``ranges`` was ``null`` (a synthesised value, with no supporting passage to look up) and ``[]`` when valid ranges overlapped no block (which usually indicates a mismatched extraction/structure pair).", - "example": { - "invoice_number": [ - { - "atomic_grounding": [], - "block_id": "text-1", - "grounding": { - "box": { - "xmax": 0.42, - "xmin": 0.1, - "ymax": 0.15, - "ymin": 0.12 - }, - "page": 1, - "range": { - "end": 31, - "start": 13 - } - }, - "type": "text" - } - ] - }, - "title": "Grounding", - "type": "object" - }, - "metadata": { - "$ref": "#/components/schemas/V2GroundMetadata", - "description": "Request metadata (job_id, duration_ms, credit_usage)." - } - }, - "required": [ - "grounding", - "metadata" - ], - "title": "V2GroundResult", - "type": "object" - } - } - }, - "description": "v2-ground result" - }, - "422": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorResponse" - } - } - }, - "description": "Request validation failed." - } - }, - "summary": "ADE Ground", - "tags": [ - "Ground" - ] - } - }, - "/v2/ground/jobs": { + "/v2/extract/jobs": { "get": { - "description": "List your Ground jobs, newest first.", - "operationId": "v2-ground_list_jobs", + "description": "List your Extract jobs, newest first.", + "operationId": "v2-extract_list_jobs", "parameters": [ { "description": "Page number (0-indexed).", @@ -2081,7 +2015,7 @@ ] }, "job_id": { - "description": "The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "description": "The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", "type": "string" }, "model_version": { @@ -2128,82 +2062,230 @@ "description": "Request validation failed." } }, - "summary": "ADE List Ground Jobs", + "summary": "ADE List Extract Jobs", "tags": [ - "Ground" + "Extract" ] }, "post": { - "description": "Map extracted fields to the document blocks they were quoted from. Takes the extraction_metadata from an extract call and the structure tree from the parse call the markdown came from, and returns, for every extracted field, the overlapping blocks with their ids, page numbers, and bounding boxes. Runs asynchronously and returns a job ID; use it to poll for status and retrieve the result once processing completes.", - "operationId": "v2-ground_create_job", + "description": "Extract structured data from a Markdown document according to a JSON schema, with character-span grounding into the source Markdown. Runs asynchronously and returns a job ID; use it to poll for status and retrieve the result once processing completes.", + "operationId": "v2-extract_create_job", "requestBody": { "content": { "application/json": { "schema": { - "description": "Input to V2GroundOperationWorkflow — the ``/v2/ground`` request body.\n\nA pure, stateless join: each ``extraction_metadata`` leaf's ``ranges``\n(char offsets into the markdown both artifacts were produced from) is\noverlapped against the ``grounding.range`` carried on every ``structure``\nblock, and the matching blocks are returned. Nothing is stored server-side,\nso block ids in the response resolve only against the ``structure`` tree\nsupplied here — pairing an extraction with the parse result it actually\ncame from is the caller's responsibility.", + "description": "Input to V2ExtractOperationWorkflow.\n\nProvide the markdown as an inline ``markdown`` string, as a multipart file\npart named ``markdown`` (for large inputs — the gateway stages the upload\ninternally), or via a public ``markdown_url``. Exactly one source must be\nsupplied.", "properties": { - "extraction_metadata": { - "additionalProperties": true, - "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown.", - "example": { - "invoice_number": { - "ranges": [ - { - "end": 31, - "start": 13 - } - ], - "value": "INV-042" + "markdown": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" } - }, - "title": "Extraction Metadata", - "type": "object" + ], + "default": null, + "description": "Markdown string to extract from, or a multipart FILE part carrying the markdown (large inputs — uploads are staged by the gateway). Can come from any source — LandingAI parse output, a third-party parser, or hand-authored text. When the markdown was produced by ``POST /v2/parse``, it ends with a ```` comment that the service reads automatically and echoes as ``metadata.doc_id``.", + "title": "Markdown" }, - "structure": { + "markdown_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "URL to fetch the markdown from. Must be a public http(s) URL; private/loopback IPs are rejected at submit time.", + "title": "Markdown Url" + }, + "model": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "The version of the model to use for extraction. Use ``extract-latest`` to use the latest version.", + "title": "Model" + }, + "options": { + "anyOf": [ + { + "$ref": "#/components/schemas/V2ExtractOptions" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Extraction options (``strict``). Omit for defaults." + }, + "output_save_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "URL to save the result to — e.g. a presigned S3 PUT URL. Async jobs only. When set, the finished result is delivered (HTTP PUT) to this URL and the completed job reports ``output_url`` instead of an inline ``result``. Must be a public http(s) URL; private/loopback IPs are rejected at submit time.", + "title": "Output Save Url" + }, + "schema": { "additionalProperties": true, - "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree.", - "title": "Structure", + "description": "JSON Schema describing the fields to extract. The schema must be an object type with a ``properties`` map of field names to their types and descriptions.", + "example": { + "properties": { + "revenue": { + "description": "Q1 revenue figure", + "type": "string" + }, + "summary": { + "description": "Executive summary", + "type": "string" + } + }, + "type": "object" + }, + "title": "Schema", "type": "object" + }, + "service_tier": { + "anyOf": [ + { + "enum": [ + "standard", + "priority" + ], + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``." } }, "required": [ - "extraction_metadata", - "structure" + "schema" ], - "title": "V2GroundRequest", + "title": "V2ExtractRequest", "type": "object" } }, "multipart/form-data": { "schema": { "properties": { - "extraction_metadata": { + "markdown": { + "anyOf": [ + { + "description": "Markdown string to extract from, or a multipart FILE part carrying the markdown (large inputs — uploads are staged by the gateway). Can come from any source — LandingAI parse output, a third-party parser, or hand-authored text. When the markdown was produced by ``POST /v2/parse``, it ends with a ```` comment that the service reads automatically and echoes as ``metadata.doc_id``.", + "type": "string" + }, + { + "description": "File upload.", + "format": "binary", + "type": "string" + } + ] + }, + "markdown_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "URL to fetch the markdown from. Must be a public http(s) URL; private/loopback IPs are rejected at submit time. JSON-serialized string in form data.", + "title": "Markdown Url" + }, + "model": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "The version of the model to use for extraction. Use ``extract-latest`` to use the latest version. JSON-serialized string in form data.", + "title": "Model" + }, + "options": { + "anyOf": [ + { + "$ref": "#/components/schemas/V2ExtractOptions" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Extraction options (``strict``). Omit for defaults. JSON-serialized string in form data." + }, + "output_save_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "URL to save the result to — e.g. a presigned S3 PUT URL. Async jobs only. When set, the finished result is delivered (HTTP PUT) to this URL and the completed job reports ``output_url`` instead of an inline ``result``. Must be a public http(s) URL; private/loopback IPs are rejected at submit time. JSON-serialized string in form data.", + "title": "Output Save Url" + }, + "schema": { "additionalProperties": true, - "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown. JSON-serialized string in form data.", + "description": "JSON Schema describing the fields to extract. The schema must be an object type with a ``properties`` map of field names to their types and descriptions. JSON-serialized string in form data.", "example": { - "invoice_number": { - "ranges": [ - { - "end": 31, - "start": 13 - } - ], - "value": "INV-042" - } + "properties": { + "revenue": { + "description": "Q1 revenue figure", + "type": "string" + }, + "summary": { + "description": "Executive summary", + "type": "string" + } + }, + "type": "object" }, - "title": "Extraction Metadata", + "title": "Schema", "type": "object" }, - "structure": { - "additionalProperties": true, - "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree. JSON-serialized string in form data.", - "title": "Structure", - "type": "object" + "service_tier": { + "anyOf": [ + { + "enum": [ + "standard", + "priority" + ], + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Async service tier. ``priority`` runs in the fast lane at the sync billing rate; absent → ``standard``." } }, "required": [ - "extraction_metadata", - "structure" + "schema" ], "type": "object" } @@ -2224,7 +2306,7 @@ ] }, "job_id": { - "description": "The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "description": "The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", "type": "string" }, "status": { @@ -2254,16 +2336,16 @@ "description": "Request validation failed." } }, - "summary": "ADE Ground Jobs", + "summary": "ADE Extract Jobs", "tags": [ - "Ground" + "Extract" ] } }, - "/v2/ground/jobs/{job_id}": { + "/v2/extract/jobs/{job_id}": { "get": { - "description": "Get the status of an async Ground job, including its result once the job has completed.", - "operationId": "v2-ground_get_job", + "description": "Get the status of an async Extract job, including its result once the job has completed.", + "operationId": "v2-extract_get_job", "parameters": [ { "description": "The identifier of the job to retrieve, as returned by the create-job request.", @@ -2307,9 +2389,16 @@ "type": "object" }, "job_id": { - "description": "The unique identifier for this v2-ground job. Format: ``ground-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", + "description": "The unique identifier for this v2-extract job. Format: ``extract-<26-character Crockford base32 ULID>`` (``[0-9a-hjkmnp-tv-z]{26}`` tail). Opaque, server-minted, and stable for the life of the job — the same id is returned on the sync response, the async 202, and every poll. Treat it as opaque; older id formats remain accepted indefinitely and are never re-issued.", "type": "string" }, + "output_url": { + "description": "The URL the result was delivered to. Present once the job has ``completed`` and ``output_save_url`` was set, instead of inline ``result``.", + "type": [ + "string", + "null" + ] + }, "progress": { "description": "Job completion as a decimal from 0 (not started) to 1 (complete). Present while ``processing``.", "maximum": 1, @@ -2319,53 +2408,66 @@ "result": { "anyOf": [ { - "description": "Result returned by V2GroundOperationWorkflow — the ``/v2/ground``\nresponse body.\n\n``grounding`` MIRRORS the ``extraction_metadata`` tree: nested objects and\narrays keep their shape, and each ``{value, ranges}`` leaf is replaced by\nthe list of structure blocks its ranges overlap (the block-hit shape\ndocumented on the field). It is NOT a flat map — a nested schema field like\n``issuer.name`` resolves to ``grounding[\"issuer\"][\"name\"]``.", + "description": "Result returned by V2ExtractOperationWorkflow — the ``/v2/extract``\nresponse body (``docs/extract-v2-proposal.md`` → Response).\n\n``extraction`` and ``extraction_metadata`` mirror each other structurally:\nleaf values in ``extraction`` are replaced by ``ExtractionFieldMetadata``\nobjects in ``extraction_metadata``.", "properties": { - "grounding": { + "extraction": { "additionalProperties": true, - "description": "A tree mirroring ``extraction_metadata``: nested objects and arrays keep their shape, and each ``{value, ranges}`` leaf is replaced by the list of blocks its ranges overlap, in reading order. Each entry carries ``block_id`` and ``type`` identifying the matched ``structure`` block, ``parent_id`` naming the enclosing block for nested blocks (table cells), and the block's own ``grounding`` object (``{page, range, box}``) verbatim. When the block itself carries ``atomic_grounding``, the entry also lists the overlapping subset as ``{index, page, range, box}`` objects, where ``index`` is the position in the block's own ``atomic_grounding`` array; ``[]`` means the block matched but no individual entry did, and the key is omitted for blocks that carry no ``atomic_grounding``. A leaf is ``null`` when its ``ranges`` was ``null`` (a synthesised value, with no supporting passage to look up) and ``[]`` when valid ranges overlapped no block (which usually indicates a mismatched extraction/structure pair).", - "example": { - "invoice_number": [ - { - "atomic_grounding": [], - "block_id": "text-1", - "grounding": { - "box": { - "xmax": 0.42, - "xmin": 0.1, - "ymax": 0.15, - "ymin": 0.12 - }, - "page": 1, - "range": { - "end": 31, - "start": 13 - } - }, - "type": "text" - } - ] - }, - "title": "Grounding", + "description": "Extracted values conforming to the request ``schema``.", + "title": "Extraction", + "type": "object" + }, + "extraction_metadata": { + "additionalProperties": true, + "description": "Per-field metadata, mirroring ``extraction`` with leaf values replaced by ``{value, ranges}`` objects.", + "title": "Extraction Metadata", "type": "object" }, + "markdown": { + "description": "Echoed input markdown.", + "title": "Markdown", + "type": "string" + }, "metadata": { - "$ref": "#/components/schemas/V2GroundMetadata", - "description": "Request metadata (job_id, duration_ms, credit_usage)." + "$ref": "#/components/schemas/V2ExtractMetadata", + "description": "Request metadata (job_id, model_version, duration_ms, doc_id, billing)." + }, + "schema_violation_error": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Set when ``options.strict`` is false and the schema contained fields the model could not extract — the extraction is partial.", + "title": "Schema Violation Error" + }, + "warnings": { + "description": "Non-fatal warnings emitted during extraction.", + "items": { + "additionalProperties": true, + "type": "object" + }, + "title": "Warnings", + "type": "array" } }, "required": [ - "grounding", + "extraction", + "extraction_metadata", + "markdown", "metadata" ], - "title": "V2GroundResult", + "title": "V2ExtractResult", "type": "object" }, { "type": "null" } ], - "description": "Present once status is ``completed``." + "description": "Present once status is ``completed`` and ``output_save_url`` was not set. When ``output_save_url`` was set, the result is delivered there and ``output_url`` is returned instead." }, "status": { "enum": [ @@ -2404,7 +2506,154 @@ "description": "Request validation failed." } }, - "summary": "ADE Get Ground Jobs", + "summary": "ADE Get Extract Jobs", + "tags": [ + "Extract" + ] + } + }, + "/v2/ground": { + "post": { + "description": "Map extracted fields to the document blocks they were quoted from. Takes the extraction_metadata from an extract call and the structure tree from the parse call the markdown came from, and returns, for every extracted field, the overlapping blocks with their ids, page numbers, and bounding boxes. Runs synchronously and returns the result inline.", + "operationId": "v2-ground_run_sync", + "requestBody": { + "content": { + "application/json": { + "schema": { + "description": "Input to V2GroundOperationWorkflow — the ``/v2/ground`` request body.\n\nA pure, stateless join: each ``extraction_metadata`` leaf's ``ranges``\n(char offsets into the markdown both artifacts were produced from) is\noverlapped against the ``grounding.range`` carried on every ``structure``\nblock, and the matching blocks are returned. Nothing is stored server-side,\nso block ids in the response resolve only against the ``structure`` tree\nsupplied here — pairing an extraction with the parse result it actually\ncame from is the caller's responsibility.", + "properties": { + "extraction_metadata": { + "additionalProperties": true, + "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown.", + "example": { + "invoice_number": { + "ranges": [ + { + "end": 31, + "start": 13 + } + ], + "value": "INV-042" + } + }, + "title": "Extraction Metadata", + "type": "object" + }, + "structure": { + "additionalProperties": true, + "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree.", + "title": "Structure", + "type": "object" + } + }, + "required": [ + "extraction_metadata", + "structure" + ], + "title": "V2GroundRequest", + "type": "object" + } + }, + "multipart/form-data": { + "schema": { + "properties": { + "extraction_metadata": { + "additionalProperties": true, + "description": "The ``extraction_metadata`` object returned by ``POST /v2/extract`` (or the pipeline's extract step): a tree mirroring your extraction schema whose leaves are ``{value, ranges}`` objects, where ``ranges`` are ``{start, end}`` Unicode code point offsets into the parse markdown. JSON-serialized string in form data.", + "example": { + "invoice_number": { + "ranges": [ + { + "end": 31, + "start": 13 + } + ], + "value": "INV-042" + } + }, + "title": "Extraction Metadata", + "type": "object" + }, + "structure": { + "additionalProperties": true, + "description": "The ``structure`` tree from the parse response the extraction was produced from. Every block in the tree carries its ``grounding`` (``{page, range, box}``) inline; block ids in the response resolve against this exact tree. JSON-serialized string in form data.", + "title": "Structure", + "type": "object" + } + }, + "required": [ + "extraction_metadata", + "structure" + ], + "type": "object" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "description": "Result returned by V2GroundOperationWorkflow — the ``/v2/ground``\nresponse body.\n\n``grounding`` MIRRORS the ``extraction_metadata`` tree: nested objects and\narrays keep their shape, and each ``{value, ranges}`` leaf is replaced by\nthe list of structure blocks its ranges overlap (the block-hit shape\ndocumented on the field). It is NOT a flat map — a nested schema field like\n``issuer.name`` resolves to ``grounding[\"issuer\"][\"name\"]``.", + "properties": { + "grounding": { + "additionalProperties": true, + "description": "A tree mirroring ``extraction_metadata``: nested objects and arrays keep their shape, and each ``{value, ranges}`` leaf is replaced by the list of blocks its ranges overlap, in reading order. Each entry carries ``block_id`` and ``type`` identifying the matched ``structure`` block, ``parent_id`` naming the enclosing block for nested blocks (table cells), and the block's own ``grounding`` object (``{page, range, box}``) verbatim. When the block itself carries ``atomic_grounding``, the entry also lists the overlapping subset as ``{index, page, range, box}`` objects, where ``index`` is the position in the block's own ``atomic_grounding`` array; ``[]`` means the block matched but no individual entry did, and the key is omitted for blocks that carry no ``atomic_grounding``. A leaf is ``null`` when its ``ranges`` was ``null`` (a synthesised value, with no supporting passage to look up) and ``[]`` when valid ranges overlapped no block (which usually indicates a mismatched extraction/structure pair).", + "example": { + "invoice_number": [ + { + "atomic_grounding": [], + "block_id": "text-1", + "grounding": { + "box": { + "xmax": 0.42, + "xmin": 0.1, + "ymax": 0.15, + "ymin": 0.12 + }, + "page": 1, + "range": { + "end": 31, + "start": 13 + } + }, + "type": "text" + } + ] + }, + "title": "Grounding", + "type": "object" + }, + "metadata": { + "$ref": "#/components/schemas/V2GroundMetadata", + "description": "Request metadata (job_id, duration_ms, credit_usage)." + } + }, + "required": [ + "grounding", + "metadata" + ], + "title": "V2GroundResult", + "type": "object" + } + } + }, + "description": "v2-ground result" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "Request validation failed." + } + }, + "summary": "ADE Ground", "tags": [ "Ground" ] diff --git a/src/landingai_ade/lib/v2_errors.py b/src/landingai_ade/lib/v2_errors.py index 2c72747..6f2ba72 100644 --- a/src/landingai_ade/lib/v2_errors.py +++ b/src/landingai_ade/lib/v2_errors.py @@ -12,11 +12,12 @@ class V2SyncTimeoutError(LandingAiadeError): - """A synchronous /v2/parse or /v2/extract call exceeded the server wait window (504). + """A synchronous /v2 call exceeded the server wait window (504). - The server cancels the work; use the async jobs route - (`client.v2.parse_jobs.create(...)` / `client.v2.extract_jobs.create(...)`, then - `.wait(...)`) for long-running documents.""" + The server cancels the work; use the endpoint's matching async jobs route + (`client.v2.parse_jobs` / `extract_jobs` / `build_schema_jobs`), then + `.wait(...)`, for long-running inputs. The raised message names the route + for the specific endpoint that timed out.""" class JobWaitTimeoutError(LandingAiadeError): @@ -27,10 +28,19 @@ class JobFailedError(LandingAiadeError): """A job reached a terminal `failed`/`cancelled` state (raise_on_failure=True).""" -def raise_if_sync_timeout(exc: APIStatusError) -> None: +def raise_if_sync_timeout(exc: APIStatusError, *, jobs_resource: str = "") -> None: + """Re-raise a 504 as `V2SyncTimeoutError` pointing at this endpoint's async route. + + `jobs_resource` is the `client.v2.` async jobs resource to recommend for + the timed-out endpoint (e.g. ``"parse_jobs"``, ``"extract_jobs"``, + ``"build_schema_jobs"``). Optional and keyword-only so adding + it stays backward compatible; when omitted the message points generically at the + endpoint's async jobs route rather than naming a possibly-wrong resource. Always + pass it for a new endpoint so the remediation is specific. + """ if exc.response.status_code == 504: + route = f"`client.v2.{jobs_resource}.create(...)`" if jobs_resource else "the endpoint's async jobs route" raise V2SyncTimeoutError( "The synchronous request timed out (HTTP 504). The server cancels the work on " - "timeout — use the async jobs route (`client.v2.parse_jobs.create(...)` / " - "`client.v2.extract_jobs.create(...)`, then `.wait(...)`) for long-running documents." + f"timeout — use {route}, then `.wait(...)`, for long-running inputs." ) from exc diff --git a/src/landingai_ade/resources/v2/_normalize.py b/src/landingai_ade/resources/v2/_normalize.py index 70d2431..4f9bd0e 100644 --- a/src/landingai_ade/resources/v2/_normalize.py +++ b/src/landingai_ade/resources/v2/_normalize.py @@ -6,9 +6,20 @@ from ..._types import StrBytesIntFloat from ..._utils import parse_datetime -from ...types.v2 import Job, JobError, JobStatus, V2GroundResult, V2ExtractResult, V2ParseResponse - -__all__ = ["normalize_parse_job", "normalize_extract_job", "normalize_ground_job"] +from ...types.v2 import ( + Job, + JobError, + JobStatus, + V2ExtractResult, + V2ParseResponse, + V2BuildSchemaResponse, +) + +__all__ = [ + "normalize_parse_job", + "normalize_extract_job", + "normalize_build_schema_job", +] def _ts(value: Optional[Union[datetime, StrBytesIntFloat]]) -> Optional[datetime]: @@ -99,19 +110,19 @@ def normalize_extract_job(raw: Mapping[str, Any]) -> Job: ) -def normalize_ground_job(raw: Mapping[str, Any]) -> Job: +def normalize_build_schema_job(raw: Mapping[str, Any]) -> Job: status = _status(raw) payload = raw.get("result") # Build leniently (like the sync-response path) so unexpected upstream drift # doesn't fail construction. - result = V2GroundResult.construct(**cast(Dict[str, Any], payload)) if isinstance(payload, Mapping) else None + result = V2BuildSchemaResponse.construct(**cast(Dict[str, Any], payload)) if isinstance(payload, Mapping) else None error = None err = raw.get("error") if isinstance(err, Mapping): err = cast(Dict[str, Any], err) error = JobError(code=err.get("code"), message=err.get("message")) - elif raw.get("failure_reason"): # ground *list* uses failure_reason + elif raw.get("failure_reason"): # build-schema *list* uses failure_reason error = JobError(message=str(raw["failure_reason"])) return Job( diff --git a/src/landingai_ade/resources/v2/build_schema.py b/src/landingai_ade/resources/v2/build_schema.py new file mode 100644 index 0000000..8b87377 --- /dev/null +++ b/src/landingai_ade/resources/v2/build_schema.py @@ -0,0 +1,512 @@ +from __future__ import annotations + +import json +import time +from typing import Any, Dict, List, Type, Tuple, Union, Mapping, Callable, Optional, Sequence, cast +from typing_extensions import Literal + +import httpx +from pydantic import BaseModel + +from ._base import DEFAULT_WAIT_TIMEOUT, JobList, V2ResourceMixin, poll_until_terminal, apoll_until_terminal +from ..._types import Body, Omit, Query, Headers, NotGiven, FileTypes, omit, not_given +from ..._utils import is_given +from ...types.v2 import Job, V2BuildSchemaResponse +from ._normalize import normalize_build_schema_job +from ..._resource import SyncAPIResource, AsyncAPIResource +from ..._exceptions import APIStatusError +from ..._base_client import make_request_options +from ...lib.v2_errors import raise_if_sync_timeout +from ...lib.schema_utils import coerce_schema_to_dict + +__all__ = [ + "BuildSchemaResource", + "AsyncBuildSchemaResource", + "BuildSchemaJobsResource", + "AsyncBuildSchemaJobsResource", +] + + +def _prepare_build_schema( + markdowns: object, + markdown_urls: object, + prompt: object, + schema: object, + service_tier: object = omit, +) -> Tuple[Optional[List[FileTypes]], Optional[List[str]], Optional[str], Optional[str], Optional[str], bool]: + """Normalize the build-schema inputs and decide the wire encoding. + + Returns ``(markdowns, markdown_urls, prompt, schema, service_tier, use_multipart)``. + ``schema`` (a pydantic model / dict / JSON string) is coerced to a JSON-encoded + string, matching the contract's string-typed ``schema`` field. The request is + sent as ``multipart/form-data`` when any ``markdowns`` entry is a file upload + (bytes / path / file object / tuple), and as JSON otherwise. + """ + md_list: Optional[List[FileTypes]] = None + if is_given(markdowns) and markdowns is not None: + md_list = [markdowns] if isinstance(markdowns, str) else list(cast(Sequence[FileTypes], markdowns)) + + urls_list: Optional[List[str]] = None + if is_given(markdown_urls) and markdown_urls is not None: + # A bare string is one URL, not a sequence of single-character URLs. + urls_list = [markdown_urls] if isinstance(markdown_urls, str) else list(cast(Sequence[str], markdown_urls)) + + prompt_val: Optional[str] = cast(str, prompt) if (is_given(prompt) and prompt is not None) else None + + schema_val: Optional[str] = None + if is_given(schema) and schema is not None: + schema_val = json.dumps( + coerce_schema_to_dict(cast("Union[str, Mapping[str, object], Type[BaseModel]]", schema)) + ) + + tier_val: Optional[str] = cast(str, service_tier) if (is_given(service_tier) and service_tier is not None) else None + + # An empty list or a blank prompt carries no usable input; treat it as absent so + # the guard below rejects it here instead of sending a no-op request to the server. + if not md_list: + md_list = None + if not urls_list: + urls_list = None + if not prompt_val: + prompt_val = None + + if md_list is None and urls_list is None and prompt_val is None and schema_val is None: + raise ValueError("build_schema requires at least one of `markdowns`, `markdown_urls`, `prompt`, or `schema`.") + + # An inline markdown is a `str`; anything else (bytes / path / file object / + # tuple) is a file upload and forces multipart. + use_multipart = any(not isinstance(m, str) for m in (md_list or [])) + return md_list, urls_list, prompt_val, schema_val, tier_val, use_multipart + + +def _build_schema_json_body( + md_list: Optional[List[FileTypes]], + urls_list: Optional[List[str]], + prompt_val: Optional[str], + schema_val: Optional[str], + tier_val: Optional[str], +) -> Dict[str, Any]: + body: Dict[str, Any] = {} + if md_list is not None: + body["markdowns"] = md_list + if urls_list is not None: + body["markdown_urls"] = urls_list + if prompt_val is not None: + body["prompt"] = prompt_val + if schema_val is not None: + body["schema"] = schema_val + if tier_val is not None: + body["service_tier"] = tier_val + return body + + +def _build_schema_multipart( + md_list: Optional[List[FileTypes]], + urls_list: Optional[List[str]], + prompt_val: Optional[str], + schema_val: Optional[str], + tier_val: Optional[str], +) -> Tuple[Dict[str, Any], List[Tuple[str, FileTypes]]]: + # In multipart form data the list-valued fields are JSON-serialized strings, + # and every markdown (inline string or file) is a repeated ``markdowns`` part. + data: Dict[str, Any] = {} + if urls_list is not None: + data["markdown_urls"] = json.dumps(urls_list) + if prompt_val is not None: + data["prompt"] = prompt_val + if schema_val is not None: + data["schema"] = schema_val + if tier_val is not None: + data["service_tier"] = tier_val + files: List[Tuple[str, FileTypes]] = [("markdowns", m) for m in (md_list or [])] + return data, files + + +class BuildSchemaResource(V2ResourceMixin, SyncAPIResource): + def run( + self, + *, + markdowns: Optional[Sequence[FileTypes]] | Omit = omit, + markdown_urls: Optional[Sequence[str]] | Omit = omit, + prompt: Optional[str] | Omit = omit, + schema: Union[str, Mapping[str, object], Type[BaseModel], None] | Omit = omit, + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> V2BuildSchemaResponse: + """Generate or edit a JSON Schema for extraction synchronously against + `/v2/extract/build-schema`. + + At least one of `markdowns`, `markdown_urls`, `prompt`, or `schema` must be + provided. Raises `V2SyncTimeoutError` when the server times out the + synchronous request (HTTP 504); use the async jobs route for long-running + inputs in that case. + + Args: + markdowns: Markdown files or inline content strings to analyze for schema + generation. Each entry is either an inline markdown string or a file + upload (`Path`/`bytes`/file object). Multiple documents can be provided + for better schema coverage. + + markdown_urls: URLs to Markdown files to analyze for schema generation. + + prompt: Instructions for how to generate or modify the schema. + + schema: Existing JSON schema to iterate on or refine. Accepts a pydantic + `BaseModel` subclass, a dict, or a JSON-encoded string; it is coerced to + a JSON Schema and sent as a JSON-encoded string. + + extra_headers: Send extra headers + + extra_query: Add additional query parameters to the request + + extra_body: Add additional JSON properties to the request + + timeout: Override the client-level default timeout for this request, in seconds + """ + md_list, urls_list, prompt_val, schema_val, tier_val, use_multipart = _prepare_build_schema( + markdowns, markdown_urls, prompt, schema + ) + try: + if use_multipart: + data, files = _build_schema_multipart(md_list, urls_list, prompt_val, schema_val, tier_val) + extra_headers = {"Content-Type": "multipart/form-data", **(extra_headers or {})} + return self._post( + self._v2_url("/v2/extract/build-schema"), + body=data, + files=files, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=V2BuildSchemaResponse, + ) + body = _build_schema_json_body(md_list, urls_list, prompt_val, schema_val, tier_val) + return self._post( + self._v2_url("/v2/extract/build-schema"), + body=body, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=V2BuildSchemaResponse, + ) + except APIStatusError as exc: + raise_if_sync_timeout(exc, jobs_resource="build_schema_jobs") + raise + + +class AsyncBuildSchemaResource(V2ResourceMixin, AsyncAPIResource): + async def run( + self, + *, + markdowns: Optional[Sequence[FileTypes]] | Omit = omit, + markdown_urls: Optional[Sequence[str]] | Omit = omit, + prompt: Optional[str] | Omit = omit, + schema: Union[str, Mapping[str, object], Type[BaseModel], None] | Omit = omit, + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> V2BuildSchemaResponse: + """Async mirror of `BuildSchemaResource.run`. See there for full documentation.""" + md_list, urls_list, prompt_val, schema_val, tier_val, use_multipart = _prepare_build_schema( + markdowns, markdown_urls, prompt, schema + ) + try: + if use_multipart: + data, files = _build_schema_multipart(md_list, urls_list, prompt_val, schema_val, tier_val) + extra_headers = {"Content-Type": "multipart/form-data", **(extra_headers or {})} + return await self._post( + self._v2_url("/v2/extract/build-schema"), + body=data, + files=files, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=V2BuildSchemaResponse, + ) + body = _build_schema_json_body(md_list, urls_list, prompt_val, schema_val, tier_val) + return await self._post( + self._v2_url("/v2/extract/build-schema"), + body=body, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=V2BuildSchemaResponse, + ) + except APIStatusError as exc: + raise_if_sync_timeout(exc, jobs_resource="build_schema_jobs") + raise + + +class BuildSchemaJobsResource(V2ResourceMixin, SyncAPIResource): + def create( + self, + *, + markdowns: Optional[Sequence[FileTypes]] | Omit = omit, + markdown_urls: Optional[Sequence[str]] | Omit = omit, + prompt: Optional[str] | Omit = omit, + schema: Union[str, Mapping[str, object], Type[BaseModel], None] | Omit = omit, + service_tier: Optional[Literal["standard", "priority"]] | Omit = omit, + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> Job: + """Create an asynchronous build-schema job against `/v2/extract/build-schema/jobs`. + + Returns a normalized `Job` immediately (typically `pending`). Poll for + completion via `.get(job_id)`, or block until the job is terminal with + `.wait(job_id)`. + + Args: + markdowns: Markdown files or inline content strings to analyze for schema + generation. Each entry is either an inline markdown string or a file + upload (`Path`/`bytes`/file object). + + markdown_urls: URLs to Markdown files to analyze for schema generation. + + prompt: Instructions for how to generate or modify the schema. + + schema: Existing JSON schema to iterate on or refine. Accepts a pydantic + `BaseModel` subclass, a dict, or a JSON-encoded string. + + service_tier: Service tier for the job: ``standard`` or ``priority``. + + extra_headers: Send extra headers + + extra_query: Add additional query parameters to the request + + extra_body: Add additional JSON properties to the request + + timeout: Override the client-level default timeout for this request, in seconds + """ + md_list, urls_list, prompt_val, schema_val, tier_val, use_multipart = _prepare_build_schema( + markdowns, markdown_urls, prompt, schema, service_tier + ) + if use_multipart: + data, files = _build_schema_multipart(md_list, urls_list, prompt_val, schema_val, tier_val) + extra_headers = {"Content-Type": "multipart/form-data", **(extra_headers or {})} + raw = self._post( + self._v2_url("/v2/extract/build-schema/jobs"), + body=data, + files=files, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + else: + body = _build_schema_json_body(md_list, urls_list, prompt_val, schema_val, tier_val) + raw = self._post( + self._v2_url("/v2/extract/build-schema/jobs"), + body=body, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + return normalize_build_schema_job(cast(Mapping[str, Any], raw)) + + def get( + self, + job_id: str, + *, + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> Job: + """Get the current status of an async build-schema job by `job_id`.""" + if not job_id: + raise ValueError(f"Expected a non-empty value for `job_id` but received {job_id!r}") + raw = self._get( + self._v2_url(f"/v2/extract/build-schema/jobs/{job_id}"), + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + return normalize_build_schema_job(cast(Mapping[str, Any], raw)) + + def list( + self, + *, + page: int | Omit = omit, + page_size: int | Omit = omit, + status: Optional[str] | Omit = omit, + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> JobList: + """List async build-schema jobs associated with your API key, newest first.""" + query = { + key: value + for key, value in {"page": page, "page_size": page_size, "status": status}.items() + if is_given(value) and value is not None + } + raw = self._get( + self._v2_url("/v2/extract/build-schema/jobs"), + options=make_request_options( + extra_headers=extra_headers, + extra_query=extra_query, + extra_body=extra_body, + timeout=timeout, + query=query, + ), + cast_to=cast("type[Any]", object), + ) + env = cast(Mapping[str, Any], raw) + jobs = [normalize_build_schema_job(cast(Mapping[str, Any], item)) for item in env.get("jobs", [])] + return JobList.build(jobs, has_more=env.get("has_more"), page=env.get("page"), page_size=env.get("page_size")) + + def wait( + self, + job_id: str, + *, + timeout: float = DEFAULT_WAIT_TIMEOUT, + poll_interval: Optional[float] = None, + raise_on_failure: bool = False, + _monotonic: Optional[Callable[[], float]] = None, + ) -> Job: + """Block, polling `.get(job_id)` with backoff, until the job is terminal. + + Raises `JobWaitTimeoutError` if `timeout` seconds elapse before the job + reaches a terminal state, and `JobFailedError` if `raise_on_failure` is + set and the job ends failed with an error attached. Build-schema jobs have + no `cancelled` status. + + `_monotonic` is a test seam for injecting a fake clock; production + callers should leave it unset (defaults to `time.monotonic`). + """ + return poll_until_terminal( + lambda: self.get(job_id), + monotonic=_monotonic or time.monotonic, + sleep=self._sleep, + timeout=timeout, + poll_interval=poll_interval, + raise_on_failure=raise_on_failure, + ) + + +class AsyncBuildSchemaJobsResource(V2ResourceMixin, AsyncAPIResource): + async def create( + self, + *, + markdowns: Optional[Sequence[FileTypes]] | Omit = omit, + markdown_urls: Optional[Sequence[str]] | Omit = omit, + prompt: Optional[str] | Omit = omit, + schema: Union[str, Mapping[str, object], Type[BaseModel], None] | Omit = omit, + service_tier: Optional[Literal["standard", "priority"]] | Omit = omit, + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> Job: + """Async mirror of `BuildSchemaJobsResource.create`. See there for full documentation.""" + md_list, urls_list, prompt_val, schema_val, tier_val, use_multipart = _prepare_build_schema( + markdowns, markdown_urls, prompt, schema, service_tier + ) + if use_multipart: + data, files = _build_schema_multipart(md_list, urls_list, prompt_val, schema_val, tier_val) + extra_headers = {"Content-Type": "multipart/form-data", **(extra_headers or {})} + raw = await self._post( + self._v2_url("/v2/extract/build-schema/jobs"), + body=data, + files=files, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + else: + body = _build_schema_json_body(md_list, urls_list, prompt_val, schema_val, tier_val) + raw = await self._post( + self._v2_url("/v2/extract/build-schema/jobs"), + body=body, + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + return normalize_build_schema_job(cast(Mapping[str, Any], raw)) + + async def get( + self, + job_id: str, + *, + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> Job: + """Async mirror of `BuildSchemaJobsResource.get`. See there for full documentation.""" + if not job_id: + raise ValueError(f"Expected a non-empty value for `job_id` but received {job_id!r}") + raw = await self._get( + self._v2_url(f"/v2/extract/build-schema/jobs/{job_id}"), + options=make_request_options( + extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout + ), + cast_to=cast("type[Any]", object), + ) + return normalize_build_schema_job(cast(Mapping[str, Any], raw)) + + async def list( + self, + *, + page: int | Omit = omit, + page_size: int | Omit = omit, + status: Optional[str] | Omit = omit, + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> JobList: + """Async mirror of `BuildSchemaJobsResource.list`. See there for full documentation.""" + query = { + key: value + for key, value in {"page": page, "page_size": page_size, "status": status}.items() + if is_given(value) and value is not None + } + raw = await self._get( + self._v2_url("/v2/extract/build-schema/jobs"), + options=make_request_options( + extra_headers=extra_headers, + extra_query=extra_query, + extra_body=extra_body, + timeout=timeout, + query=query, + ), + cast_to=cast("type[Any]", object), + ) + env = cast(Mapping[str, Any], raw) + jobs = [normalize_build_schema_job(cast(Mapping[str, Any], item)) for item in env.get("jobs", [])] + return JobList.build(jobs, has_more=env.get("has_more"), page=env.get("page"), page_size=env.get("page_size")) + + async def wait( + self, + job_id: str, + *, + timeout: float = DEFAULT_WAIT_TIMEOUT, + poll_interval: Optional[float] = None, + raise_on_failure: bool = False, + _monotonic: Optional[Callable[[], float]] = None, + ) -> Job: + """Async mirror of `BuildSchemaJobsResource.wait`; sleeps via `anyio.sleep` instead of blocking.""" + return await apoll_until_terminal( + lambda: self.get(job_id), + monotonic=_monotonic or time.monotonic, + timeout=timeout, + poll_interval=poll_interval, + raise_on_failure=raise_on_failure, + ) diff --git a/src/landingai_ade/resources/v2/extract.py b/src/landingai_ade/resources/v2/extract.py index baf0a6c..4b5715e 100644 --- a/src/landingai_ade/resources/v2/extract.py +++ b/src/landingai_ade/resources/v2/extract.py @@ -120,7 +120,7 @@ def run( cast_to=V2ExtractResult, ) except APIStatusError as exc: - raise_if_sync_timeout(exc) + raise_if_sync_timeout(exc, jobs_resource="extract_jobs") raise if save_to: from ..._client import _save_response, _get_input_filename @@ -159,7 +159,7 @@ async def run( cast_to=V2ExtractResult, ) except APIStatusError as exc: - raise_if_sync_timeout(exc) + raise_if_sync_timeout(exc, jobs_resource="extract_jobs") raise if save_to: from ..._client import _save_response, _get_input_filename diff --git a/src/landingai_ade/resources/v2/files.py b/src/landingai_ade/resources/v2/files.py deleted file mode 100644 index a88b566..0000000 --- a/src/landingai_ade/resources/v2/files.py +++ /dev/null @@ -1,73 +0,0 @@ -# src/landingai_ade/resources/v2/files.py -from __future__ import annotations - -from typing import Mapping, cast - -import httpx - -from ._base import V2ResourceMixin -from ..._files import deepcopy_with_paths -from ..._types import Body, Query, Headers, NotGiven, FileTypes, not_given -from ..._utils import extract_files -from ...types.v2 import V2FileUploadResponse -from ..._resource import SyncAPIResource, AsyncAPIResource -from ..._exceptions import LandingAiadeError -from ..._base_client import make_request_options - -__all__ = ["FilesResource", "AsyncFilesResource"] - - -class FilesResource(V2ResourceMixin, SyncAPIResource): - def upload( - self, - *, - file: FileTypes, - extra_headers: Headers | None = None, - extra_query: Query | None = None, - extra_body: Body | None = None, - timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> str: - """Stage bytes on the data plane; returns a `file_ref` for use in job inputs.""" - body = deepcopy_with_paths({"file": file}, [["file"]]) - files = extract_files(cast(Mapping[str, object], body), paths=[["file"]]) - extra_headers = {"Content-Type": "multipart/form-data", **(extra_headers or {})} - response = self._post( - self._v2_url("/v1/files"), - body=body, - files=files, - options=make_request_options( - extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout - ), - cast_to=V2FileUploadResponse, - ) - if not response.file_ref: - raise LandingAiadeError(f"POST /v1/files did not return a file_ref (got: {response!r}).") - return response.file_ref - - -class AsyncFilesResource(V2ResourceMixin, AsyncAPIResource): - async def upload( - self, - *, - file: FileTypes, - extra_headers: Headers | None = None, - extra_query: Query | None = None, - extra_body: Body | None = None, - timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> str: - """Stage bytes on the data plane; returns a `file_ref` for use in job inputs.""" - body = deepcopy_with_paths({"file": file}, [["file"]]) - files = extract_files(cast(Mapping[str, object], body), paths=[["file"]]) - extra_headers = {"Content-Type": "multipart/form-data", **(extra_headers or {})} - response = await self._post( - self._v2_url("/v1/files"), - body=body, - files=files, - options=make_request_options( - extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout - ), - cast_to=V2FileUploadResponse, - ) - if not response.file_ref: - raise LandingAiadeError(f"POST /v1/files did not return a file_ref (got: {response!r}).") - return response.file_ref diff --git a/src/landingai_ade/resources/v2/ground.py b/src/landingai_ade/resources/v2/ground.py index bcc68c3..6c7ba4e 100644 --- a/src/landingai_ade/resources/v2/ground.py +++ b/src/landingai_ade/resources/v2/ground.py @@ -1,23 +1,19 @@ from __future__ import annotations -import time -from typing import Any, Dict, Union, Mapping, Callable, Optional, cast +from typing import Any, Dict, Union, Mapping import httpx from pydantic import BaseModel -from ._base import DEFAULT_WAIT_TIMEOUT, JobList, V2ResourceMixin, poll_until_terminal, apoll_until_terminal -from ..._types import Body, Omit, Query, Headers, NotGiven, omit, not_given +from ._base import V2ResourceMixin +from ..._types import Body, Query, Headers, NotGiven, omit, not_given from ..._utils import is_given from ..._compat import model_dump -from ...types.v2 import Job, V2GroundResult -from ._normalize import normalize_ground_job +from ...types.v2 import V2GroundResult from ..._resource import SyncAPIResource, AsyncAPIResource -from ..._exceptions import APIStatusError from ..._base_client import make_request_options -from ...lib.v2_errors import raise_if_sync_timeout -__all__ = ["GroundResource", "AsyncGroundResource", "GroundJobsResource", "AsyncGroundJobsResource"] +__all__ = ["GroundResource", "AsyncGroundResource"] def _as_object(value: Union[Mapping[str, object], BaseModel]) -> Dict[str, Any]: @@ -64,10 +60,6 @@ def run( extraction with the parse result it actually came from is the caller's responsibility. - Raises `V2SyncTimeoutError` when the server times out the synchronous - request (HTTP 504); use the async jobs route for long-running inputs in - that case. - Args: extraction_metadata: The `extraction_metadata` tree returned by `POST /v2/extract` (or `client.v2.extract(...).extraction_metadata`). @@ -86,274 +78,36 @@ def run( timeout: Override the client-level default timeout for this request, in seconds """ body = _build_ground_body(extraction_metadata, structure) - try: - return self._post( - self._v2_url("/v2/ground"), - body=body, - options=make_request_options( - extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout - ), - cast_to=V2GroundResult, - ) - except APIStatusError as exc: - raise_if_sync_timeout(exc) - raise - - -class AsyncGroundResource(V2ResourceMixin, AsyncAPIResource): - async def run( - self, - *, - extraction_metadata: Union[Mapping[str, object], BaseModel], - structure: Union[Mapping[str, object], BaseModel], - # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. - # The extra values given here take precedence over values defined on the client or passed to this method. - extra_headers: Headers | None = None, - extra_query: Query | None = None, - extra_body: Body | None = None, - timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> V2GroundResult: - """Async mirror of `GroundResource.run`. See there for full documentation.""" - body = _build_ground_body(extraction_metadata, structure) - try: - return await self._post( - self._v2_url("/v2/ground"), - body=body, - options=make_request_options( - extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout - ), - cast_to=V2GroundResult, - ) - except APIStatusError as exc: - raise_if_sync_timeout(exc) - raise - - -class GroundJobsResource(V2ResourceMixin, SyncAPIResource): - def create( - self, - *, - extraction_metadata: Union[Mapping[str, object], BaseModel], - structure: Union[Mapping[str, object], BaseModel], - output_save_url: Optional[str] | Omit = omit, - # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. - # The extra values given here take precedence over values defined on the client or passed to this method. - extra_headers: Headers | None = None, - extra_query: Query | None = None, - extra_body: Body | None = None, - timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> Job: - """Create an asynchronous ground job against `/v2/ground/jobs`. - - Returns a normalized `Job` immediately (typically `pending`). Poll for - completion via `.get(job_id)`, or block until the job is terminal with - `.wait(job_id)`. - - Args: - extraction_metadata: The `extraction_metadata` tree returned by - `POST /v2/extract`. Accepts a dict or a pydantic model. - - structure: The `structure` tree from the parse response the extraction was - produced from. Accepts a dict or a pydantic model. - - output_save_url: URL the result should be saved to (e.g. a presigned S3 PUT - URL) instead of being returned inline. When set, the completed job reports - `output_url` instead of an inline `result`. - - extra_headers: Send extra headers - - extra_query: Add additional query parameters to the request - - extra_body: Add additional JSON properties to the request - - timeout: Override the client-level default timeout for this request, in seconds - """ - body = _build_ground_body(extraction_metadata, structure, output_save_url) - raw = self._post( - self._v2_url("/v2/ground/jobs"), + return self._post( + self._v2_url("/v2/ground"), body=body, options=make_request_options( extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout ), - cast_to=cast("type[Any]", object), - ) - return normalize_ground_job(cast(Mapping[str, Any], raw)) - - def get( - self, - job_id: str, - *, - extra_headers: Headers | None = None, - extra_query: Query | None = None, - extra_body: Body | None = None, - timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> Job: - """Get the current status of an async ground job by `job_id`.""" - if not job_id: - raise ValueError(f"Expected a non-empty value for `job_id` but received {job_id!r}") - raw = self._get( - self._v2_url(f"/v2/ground/jobs/{job_id}"), - options=make_request_options( - extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout - ), - cast_to=cast("type[Any]", object), - ) - return normalize_ground_job(cast(Mapping[str, Any], raw)) - - def list( - self, - *, - page: int | Omit = omit, - page_size: int | Omit = omit, - status: Optional[str] | Omit = omit, - extra_headers: Headers | None = None, - extra_query: Query | None = None, - extra_body: Body | None = None, - timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> JobList: - """List async ground jobs associated with your API key, newest first.""" - query = { - key: value - for key, value in {"page": page, "page_size": page_size, "status": status}.items() - if is_given(value) and value is not None - } - raw = self._get( - self._v2_url("/v2/ground/jobs"), - options=make_request_options( - extra_headers=extra_headers, - extra_query=extra_query, - extra_body=extra_body, - timeout=timeout, - query=query, - ), - cast_to=cast("type[Any]", object), + cast_to=V2GroundResult, ) - env = cast(Mapping[str, Any], raw) - jobs = [normalize_ground_job(cast(Mapping[str, Any], item)) for item in env.get("jobs", [])] - return JobList.build(jobs, has_more=env.get("has_more"), page=env.get("page"), page_size=env.get("page_size")) - def wait( - self, - job_id: str, - *, - timeout: float = DEFAULT_WAIT_TIMEOUT, - poll_interval: Optional[float] = None, - raise_on_failure: bool = False, - _monotonic: Optional[Callable[[], float]] = None, - ) -> Job: - """Block, polling `.get(job_id)` with backoff, until the job is terminal. - Raises `JobWaitTimeoutError` if `timeout` seconds elapse before the job - reaches a terminal state, and `JobFailedError` if `raise_on_failure` is - set and the job ends failed with an error attached. Ground jobs have no - `cancelled` status. - - `_monotonic` is a test seam for injecting a fake clock; production - callers should leave it unset (defaults to `time.monotonic`). - """ - return poll_until_terminal( - lambda: self.get(job_id), - monotonic=_monotonic or time.monotonic, - sleep=self._sleep, - timeout=timeout, - poll_interval=poll_interval, - raise_on_failure=raise_on_failure, - ) - - -class AsyncGroundJobsResource(V2ResourceMixin, AsyncAPIResource): - async def create( +class AsyncGroundResource(V2ResourceMixin, AsyncAPIResource): + async def run( self, *, extraction_metadata: Union[Mapping[str, object], BaseModel], structure: Union[Mapping[str, object], BaseModel], - output_save_url: Optional[str] | Omit = omit, # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. # The extra values given here take precedence over values defined on the client or passed to this method. extra_headers: Headers | None = None, extra_query: Query | None = None, extra_body: Body | None = None, timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> Job: - """Async mirror of `GroundJobsResource.create`. See there for full documentation.""" - body = _build_ground_body(extraction_metadata, structure, output_save_url) - raw = await self._post( - self._v2_url("/v2/ground/jobs"), + ) -> V2GroundResult: + """Async mirror of `GroundResource.run`. See there for full documentation.""" + body = _build_ground_body(extraction_metadata, structure) + return await self._post( + self._v2_url("/v2/ground"), body=body, options=make_request_options( extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout ), - cast_to=cast("type[Any]", object), - ) - return normalize_ground_job(cast(Mapping[str, Any], raw)) - - async def get( - self, - job_id: str, - *, - extra_headers: Headers | None = None, - extra_query: Query | None = None, - extra_body: Body | None = None, - timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> Job: - """Async mirror of `GroundJobsResource.get`. See there for full documentation.""" - if not job_id: - raise ValueError(f"Expected a non-empty value for `job_id` but received {job_id!r}") - raw = await self._get( - self._v2_url(f"/v2/ground/jobs/{job_id}"), - options=make_request_options( - extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout - ), - cast_to=cast("type[Any]", object), - ) - return normalize_ground_job(cast(Mapping[str, Any], raw)) - - async def list( - self, - *, - page: int | Omit = omit, - page_size: int | Omit = omit, - status: Optional[str] | Omit = omit, - extra_headers: Headers | None = None, - extra_query: Query | None = None, - extra_body: Body | None = None, - timeout: float | httpx.Timeout | None | NotGiven = not_given, - ) -> JobList: - """Async mirror of `GroundJobsResource.list`. See there for full documentation.""" - query = { - key: value - for key, value in {"page": page, "page_size": page_size, "status": status}.items() - if is_given(value) and value is not None - } - raw = await self._get( - self._v2_url("/v2/ground/jobs"), - options=make_request_options( - extra_headers=extra_headers, - extra_query=extra_query, - extra_body=extra_body, - timeout=timeout, - query=query, - ), - cast_to=cast("type[Any]", object), - ) - env = cast(Mapping[str, Any], raw) - jobs = [normalize_ground_job(cast(Mapping[str, Any], item)) for item in env.get("jobs", [])] - return JobList.build(jobs, has_more=env.get("has_more"), page=env.get("page"), page_size=env.get("page_size")) - - async def wait( - self, - job_id: str, - *, - timeout: float = DEFAULT_WAIT_TIMEOUT, - poll_interval: Optional[float] = None, - raise_on_failure: bool = False, - _monotonic: Optional[Callable[[], float]] = None, - ) -> Job: - """Async mirror of `GroundJobsResource.wait`; sleeps via `anyio.sleep` instead of blocking.""" - return await apoll_until_terminal( - lambda: self.get(job_id), - monotonic=_monotonic or time.monotonic, - timeout=timeout, - poll_interval=poll_interval, - raise_on_failure=raise_on_failure, + cast_to=V2GroundResult, ) diff --git a/src/landingai_ade/resources/v2/parse.py b/src/landingai_ade/resources/v2/parse.py index 9aa5b13..e5624ef 100644 --- a/src/landingai_ade/resources/v2/parse.py +++ b/src/landingai_ade/resources/v2/parse.py @@ -135,7 +135,7 @@ def run( cast_to=V2ParseResponse, ) except APIStatusError as exc: - raise_if_sync_timeout(exc) + raise_if_sync_timeout(exc, jobs_resource="parse_jobs") raise if save_to: from ..._client import _save_response, _get_input_filename @@ -182,7 +182,7 @@ async def run( cast_to=V2ParseResponse, ) except APIStatusError as exc: - raise_if_sync_timeout(exc) + raise_if_sync_timeout(exc, jobs_resource="parse_jobs") raise if save_to: from ..._client import _save_response, _get_input_filename diff --git a/src/landingai_ade/resources/v2/v2.py b/src/landingai_ade/resources/v2/v2.py index 17b67b5..92c7f2e 100644 --- a/src/landingai_ade/resources/v2/v2.py +++ b/src/landingai_ade/resources/v2/v2.py @@ -1,7 +1,7 @@ # src/landingai_ade/resources/v2/v2.py from __future__ import annotations -from typing import TYPE_CHECKING, Type, Union, Mapping, Optional +from typing import TYPE_CHECKING, Type, Union, Mapping, Optional, Sequence from pathlib import Path import httpx @@ -10,14 +10,19 @@ from ._base import V2ResourceMixin from ..._types import Body, Omit, Query, Headers, NotGiven, FileTypes, omit, not_given from ..._compat import cached_property -from ...types.v2 import V2GroundResult, V2ExtractResult, V2ParseResponse +from ...types.v2 import V2GroundResult, V2ExtractResult, V2ParseResponse, V2BuildSchemaResponse from ..._resource import SyncAPIResource, AsyncAPIResource if TYPE_CHECKING: - from .files import FilesResource, AsyncFilesResource from .parse import ParseResource, ParseJobsResource, AsyncParseResource, AsyncParseJobsResource - from .ground import GroundResource, GroundJobsResource, AsyncGroundResource, AsyncGroundJobsResource + from .ground import GroundResource, AsyncGroundResource from .extract import ExtractResource, ExtractJobsResource, AsyncExtractResource, AsyncExtractJobsResource + from .build_schema import ( + BuildSchemaResource, + BuildSchemaJobsResource, + AsyncBuildSchemaResource, + AsyncBuildSchemaJobsResource, + ) __all__ = ["V2Resource", "AsyncV2Resource"] @@ -25,19 +30,12 @@ class V2Resource(SyncAPIResource, V2ResourceMixin): """Container for the V2 (ADE) surface: ``client.v2.``. - ``files``, ``parse``, ``extract``, and ``ground`` are wired up; each sub-resource does its - own lazy import inside its cached property body -- mirroring + ``parse``, ``extract``, ``build_schema``, and ``ground`` are wired up; each sub-resource + does its own lazy import inside its cached property body -- mirroring ``LandingAIADE.parse_jobs`` -- so that this module keeps importing standalone - regardless of which sub-resources exist yet. Remaining job-polling resources - are attached by later tasks following the same pattern. + regardless of which sub-resources exist yet. """ - @cached_property - def files(self) -> FilesResource: - from .files import FilesResource - - return FilesResource(self._client) - @cached_property def _parse(self) -> ParseResource: from .parse import ParseResource @@ -123,16 +121,48 @@ def extract( ) @cached_property - def _ground(self) -> GroundResource: - from .ground import GroundResource + def _build_schema(self) -> BuildSchemaResource: + from .build_schema import BuildSchemaResource - return GroundResource(self._client) + return BuildSchemaResource(self._client) @cached_property - def ground_jobs(self) -> GroundJobsResource: - from .ground import GroundJobsResource + def build_schema_jobs(self) -> BuildSchemaJobsResource: + from .build_schema import BuildSchemaJobsResource + + return BuildSchemaJobsResource(self._client) - return GroundJobsResource(self._client) + def build_schema( + self, + *, + markdowns: Optional[Sequence[FileTypes]] | Omit = omit, + markdown_urls: Optional[Sequence[str]] | Omit = omit, + prompt: Optional[str] | Omit = omit, + schema: Union[str, Mapping[str, object], Type[BaseModel], None] | Omit = omit, + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> V2BuildSchemaResponse: + """Generate or edit a JSON Schema for extraction synchronously. See ``BuildSchemaResource.run`` for full documentation.""" + return self._build_schema.run( + markdowns=markdowns, + markdown_urls=markdown_urls, + prompt=prompt, + schema=schema, + extra_headers=extra_headers, + extra_query=extra_query, + extra_body=extra_body, + timeout=timeout, + ) + + @cached_property + def _ground(self) -> GroundResource: + from .ground import GroundResource + + return GroundResource(self._client) def ground( self, @@ -160,12 +190,6 @@ def ground( class AsyncV2Resource(AsyncAPIResource, V2ResourceMixin): """Async mirror of :class:`V2Resource`.""" - @cached_property - def files(self) -> AsyncFilesResource: - from .files import AsyncFilesResource - - return AsyncFilesResource(self._client) - @cached_property def _parse(self) -> AsyncParseResource: from .parse import AsyncParseResource @@ -251,16 +275,48 @@ async def extract( ) @cached_property - def _ground(self) -> AsyncGroundResource: - from .ground import AsyncGroundResource + def _build_schema(self) -> AsyncBuildSchemaResource: + from .build_schema import AsyncBuildSchemaResource - return AsyncGroundResource(self._client) + return AsyncBuildSchemaResource(self._client) @cached_property - def ground_jobs(self) -> AsyncGroundJobsResource: - from .ground import AsyncGroundJobsResource + def build_schema_jobs(self) -> AsyncBuildSchemaJobsResource: + from .build_schema import AsyncBuildSchemaJobsResource + + return AsyncBuildSchemaJobsResource(self._client) - return AsyncGroundJobsResource(self._client) + async def build_schema( + self, + *, + markdowns: Optional[Sequence[FileTypes]] | Omit = omit, + markdown_urls: Optional[Sequence[str]] | Omit = omit, + prompt: Optional[str] | Omit = omit, + schema: Union[str, Mapping[str, object], Type[BaseModel], None] | Omit = omit, + # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs. + # The extra values given here take precedence over values defined on the client or passed to this method. + extra_headers: Headers | None = None, + extra_query: Query | None = None, + extra_body: Body | None = None, + timeout: float | httpx.Timeout | None | NotGiven = not_given, + ) -> V2BuildSchemaResponse: + """Async mirror of :meth:`V2Resource.build_schema`.""" + return await self._build_schema.run( + markdowns=markdowns, + markdown_urls=markdown_urls, + prompt=prompt, + schema=schema, + extra_headers=extra_headers, + extra_query=extra_query, + extra_body=extra_body, + timeout=timeout, + ) + + @cached_property + def _ground(self) -> AsyncGroundResource: + from .ground import AsyncGroundResource + + return AsyncGroundResource(self._client) async def ground( self, diff --git a/src/landingai_ade/types/v2/__init__.py b/src/landingai_ade/types/v2/__init__.py index 54fd8e6..d56d455 100644 --- a/src/landingai_ade/types/v2/__init__.py +++ b/src/landingai_ade/types/v2/__init__.py @@ -26,4 +26,9 @@ V2ExtractBilling as V2ExtractBilling, V2ExtractMetadata as V2ExtractMetadata, ) -from .file_upload_response import V2FileUploadResponse as V2FileUploadResponse +from .build_schema_response import ( + V2BuildSchemaBilling as V2BuildSchemaBilling, + V2BuildSchemaWarning as V2BuildSchemaWarning, + V2BuildSchemaMetadata as V2BuildSchemaMetadata, + V2BuildSchemaResponse as V2BuildSchemaResponse, +) diff --git a/src/landingai_ade/types/v2/build_schema_response.py b/src/landingai_ade/types/v2/build_schema_response.py new file mode 100644 index 0000000..a598211 --- /dev/null +++ b/src/landingai_ade/types/v2/build_schema_response.py @@ -0,0 +1,57 @@ +from __future__ import annotations + +from typing import List, Optional + +from ..._models import BaseModel + +__all__ = [ + "V2BuildSchemaWarning", + "V2BuildSchemaBilling", + "V2BuildSchemaMetadata", + "V2BuildSchemaResponse", +] + + +class V2BuildSchemaWarning(BaseModel): + """A structured warning from the schema-generation process.""" + + # The type of warning (e.g. `nonconformant_schema`), used to translate to a + # status code downstream. Required and non-null per the spec. + code: str + # Human-readable description of the warning with more details. Required and + # non-null per the spec. + msg: str + + +class V2BuildSchemaBilling(BaseModel): + """Billing summary: the service tier the request ran in and the credits charged.""" + + service_tier: Optional[str] = None + total_credits: Optional[float] = None + + +class V2BuildSchemaMetadata(BaseModel): + """Response metadata for a v2 build-schema call.""" + + # Gateway job id (workflow id). Matches the billing row id in vision-agent. + job_id: Optional[str] = None + duration_ms: Optional[int] = None + # Name of the first source document. Retained for v1 compatibility but not + # populated in this version -- always None. + filename: Optional[str] = None + org_id: Optional[str] = None + # Model version used for generation. build-schema is version-free, so this is + # always None; retained for v1 response-shape compatibility. + version: Optional[str] = None + # URL of the OpenAPI spec covering this API. Required and non-null per the spec. + openapi_spec: str + # Structured warnings from the schema-generation process. + warnings: Optional[List[V2BuildSchemaWarning]] = None + billing: Optional[V2BuildSchemaBilling] = None + + +class V2BuildSchemaResponse(BaseModel): + # The generated JSON schema serialized as a string (VTRA parity -- the field + # is a string, not an object). + extraction_schema: str + metadata: V2BuildSchemaMetadata diff --git a/src/landingai_ade/types/v2/file_upload_response.py b/src/landingai_ade/types/v2/file_upload_response.py deleted file mode 100644 index 393a182..0000000 --- a/src/landingai_ade/types/v2/file_upload_response.py +++ /dev/null @@ -1,13 +0,0 @@ -from __future__ import annotations - -from typing import Optional - -from ..._models import BaseModel - -__all__ = ["V2FileUploadResponse"] - - -class V2FileUploadResponse(BaseModel): - """`POST /v1/files` returns an open string map; `file_ref` is the key we consume.""" - - file_ref: Optional[str] = None diff --git a/tests/api_resources/v2/test_build_schema.py b/tests/api_resources/v2/test_build_schema.py new file mode 100644 index 0000000..35e506e --- /dev/null +++ b/tests/api_resources/v2/test_build_schema.py @@ -0,0 +1,263 @@ +from __future__ import annotations + +import json +from typing import Any, Dict + +import httpx +import respx +import pytest +from pydantic import Field, BaseModel + +from landingai_ade import LandingAIADE +from landingai_ade.types.v2 import JobStatus, V2BuildSchemaResponse +from landingai_ade.lib.v2_errors import V2SyncTimeoutError + +APIKEY = "My Apikey" +BUILD_SCHEMA_BODY: Dict[str, Any] = { + "extraction_schema": json.dumps({"type": "object", "properties": {"revenue": {"type": "string"}}}), + "metadata": {"job_id": "bs1", "duration_ms": 5, "openapi_spec": "https://api.ade.landing.ai/openapi.json"}, +} + + +class Invoice(BaseModel): + revenue: str = Field(description="Q1 revenue") + + +@respx.mock +def test_build_schema_sync_json_body_with_prompt() -> None: + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/extract/build-schema").mock( + return_value=httpx.Response(200, json=BUILD_SCHEMA_BODY) + ) + result = client.v2.build_schema(prompt="Extract the revenue figure") + assert isinstance(result, V2BuildSchemaResponse) + assert json.loads(result.extraction_schema)["type"] == "object" + assert result.metadata.job_id == "bs1" + req = json.loads(route.calls.last.request.content) + assert req["prompt"] == "Extract the revenue figure" + assert route.calls.last.request.headers["content-type"].startswith("application/json") + + +@respx.mock +def test_build_schema_sync_json_markdowns_and_urls() -> None: + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/extract/build-schema").mock( + return_value=httpx.Response(200, json=BUILD_SCHEMA_BODY) + ) + client.v2.build_schema(markdowns=["# doc one", "# doc two"], markdown_urls=["https://x/y.md"]) + req = json.loads(route.calls.last.request.content) + # Inline-string markdowns stay a JSON array (no file → JSON, not multipart). + assert req["markdowns"] == ["# doc one", "# doc two"] + assert req["markdown_urls"] == ["https://x/y.md"] + assert route.calls.last.request.headers["content-type"].startswith("application/json") + + +@respx.mock +def test_build_schema_sync_coerces_schema_to_json_string() -> None: + # `schema` (a pydantic model / dict / JSON string) is coerced to a JSON Schema + # and sent as a JSON-encoded *string* per the contract's string-typed field. + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/extract/build-schema").mock( + return_value=httpx.Response(200, json=BUILD_SCHEMA_BODY) + ) + client.v2.build_schema(schema=Invoice) + req = json.loads(route.calls.last.request.content) + assert isinstance(req["schema"], str) + parsed = json.loads(req["schema"]) + assert parsed["type"] == "object" and "revenue" in parsed["properties"] + + +@respx.mock +def test_build_schema_sync_multipart_when_markdown_is_a_file() -> None: + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/extract/build-schema").mock( + return_value=httpx.Response(200, json=BUILD_SCHEMA_BODY) + ) + client.v2.build_schema(markdowns=[b"# uploaded doc"], prompt="go") + assert route.calls.last.request.headers["content-type"].startswith("multipart/form-data") + sent = route.calls.last.request.content + assert b'name="markdowns"' in sent + assert b"# uploaded doc" in sent + assert b'name="prompt"' in sent + + +@respx.mock +def test_build_schema_sync_multipart_serializes_urls_as_json_string() -> None: + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/extract/build-schema").mock( + return_value=httpx.Response(200, json=BUILD_SCHEMA_BODY) + ) + client.v2.build_schema(markdowns=[b"# doc"], markdown_urls=["https://x/y.md"]) + sent = route.calls.last.request.content + # markdown_urls is a JSON-serialized string form field in multipart. + assert b'["https://x/y.md"]' in sent + + +@respx.mock +def test_build_schema_requires_at_least_one_input() -> None: + # No route registered: @respx.mock keeps this hermetic so a regression fails + # loudly on an unmocked request instead of hitting the network. + client = LandingAIADE(apikey=APIKEY) + with pytest.raises(ValueError, match="at least one"): + client.v2.build_schema() + + +@respx.mock +def test_build_schema_empty_inputs_count_as_absent() -> None: + # Empty containers / a blank prompt carry no usable input and must raise here, + # not slip through to the API as a no-op request. + client = LandingAIADE(apikey=APIKEY) + with pytest.raises(ValueError, match="at least one"): + client.v2.build_schema(markdowns=[], markdown_urls=[], prompt="") + + +@respx.mock +def test_build_schema_markdown_url_string_is_one_url() -> None: + # A bare string is a single URL, not a sequence of one-character URLs. + client = LandingAIADE(apikey=APIKEY) + route = respx.post("https://api.ade.landing.ai/v2/extract/build-schema").mock( + return_value=httpx.Response(200, json=BUILD_SCHEMA_BODY) + ) + client.v2.build_schema(markdown_urls="https://x/y.md") + req = json.loads(route.calls.last.request.content) + assert req["markdown_urls"] == ["https://x/y.md"] + + +@respx.mock +def test_build_schema_job_create_requires_at_least_one_input() -> None: + client = LandingAIADE(apikey=APIKEY) + with pytest.raises(ValueError, match="at least one"): + client.v2.build_schema_jobs.create() + + +@respx.mock +def test_build_schema_sync_504() -> None: + client = LandingAIADE(apikey=APIKEY, max_retries=0) + respx.post("https://api.ade.landing.ai/v2/extract/build-schema").mock( + return_value=httpx.Response(504, json={"detail": "x"}) + ) + with pytest.raises(V2SyncTimeoutError): + client.v2.build_schema(prompt="x") + + +@respx.mock +def test_build_schema_job_create_and_get() -> None: + client = LandingAIADE(apikey=APIKEY) + respx.post("https://api.ade.landing.ai/v2/extract/build-schema/jobs").mock( + return_value=httpx.Response( + 202, json={"job_id": "bs1", "status": "pending", "created_at": "2026-01-01T00:00:00Z"} + ) + ) + job = client.v2.build_schema_jobs.create(prompt="x", service_tier="priority") + assert job.job_id == "bs1" and job.status is JobStatus.PENDING + + respx.get("https://api.ade.landing.ai/v2/extract/build-schema/jobs/bs1").mock( + return_value=httpx.Response( + 200, + json={ + "job_id": "bs1", + "status": "completed", + "created_at": "2026-01-01T00:00:00Z", + "completed_at": "2026-01-01T00:00:09Z", + "result": BUILD_SCHEMA_BODY, + }, + ) + ) + done = client.v2.build_schema_jobs.get("bs1") + assert done.status is JobStatus.COMPLETED + assert isinstance(done.result, V2BuildSchemaResponse) + assert json.loads(done.result.extraction_schema)["type"] == "object" + + +def test_build_schema_job_create_sends_service_tier(monkeypatch: pytest.MonkeyPatch) -> None: + client = LandingAIADE(apikey=APIKEY) + captured: Dict[str, Any] = {} + + def fake_post(path: str, *, cast_to: Any, body: Any = None, files: Any = None, options: Any = None) -> Any: # noqa: ARG001 + captured["body"] = body + return {"job_id": "bs1", "status": "pending"} + + monkeypatch.setattr(client.v2.build_schema_jobs, "_post", fake_post) + client.v2.build_schema_jobs.create(prompt="x", service_tier="priority") + assert captured["body"]["service_tier"] == "priority" + + +@respx.mock +def test_build_schema_job_get_failed_maps_error_object() -> None: + client = LandingAIADE(apikey=APIKEY) + respx.get("https://api.ade.landing.ai/v2/extract/build-schema/jobs/bs2").mock( + return_value=httpx.Response( + 200, json={"job_id": "bs2", "status": "failed", "error": {"code": "internal_error", "message": "boom"}} + ) + ) + job = client.v2.build_schema_jobs.get("bs2") + assert job.status is JobStatus.FAILED and job.error is not None and job.error.code == "internal_error" + + +@respx.mock +def test_build_schema_job_get_empty_job_id_raises() -> None: + client = LandingAIADE(apikey=APIKEY) + with pytest.raises(ValueError): + client.v2.build_schema_jobs.get("") + + +@respx.mock +def test_build_schema_job_list_carries_envelope() -> None: + client = LandingAIADE(apikey=APIKEY) + respx.get("https://api.ade.landing.ai/v2/extract/build-schema/jobs").mock( + return_value=httpx.Response( + 200, + json={ + "jobs": [{"job_id": "bs1", "status": "completed"}], + "page": 0, + "page_size": 10, + "has_more": True, + }, + ) + ) + jobs = client.v2.build_schema_jobs.list() + assert len(jobs) == 1 and jobs[0].job_id == "bs1" and jobs[0].status is JobStatus.COMPLETED + assert jobs.has_more is True and jobs.page == 0 and jobs.page_size == 10 + + +@respx.mock +def test_build_schema_job_list_status_none_omits_query_param() -> None: + client = LandingAIADE(apikey=APIKEY) + route = respx.get("https://api.ade.landing.ai/v2/extract/build-schema/jobs").mock( + return_value=httpx.Response(200, json={"jobs": [], "has_more": False}) + ) + client.v2.build_schema_jobs.list(status=None) + assert "status" not in route.calls.last.request.url.params + + +@respx.mock +@pytest.mark.asyncio +async def test_async_build_schema_sync_ok() -> None: + from landingai_ade import AsyncLandingAIADE + + client = AsyncLandingAIADE(apikey=APIKEY) + respx.post("https://api.ade.landing.ai/v2/extract/build-schema").mock( + return_value=httpx.Response(200, json=BUILD_SCHEMA_BODY) + ) + result = await client.v2.build_schema(prompt="x") + assert isinstance(result, V2BuildSchemaResponse) + assert result.metadata.job_id == "bs1" + + +@respx.mock +@pytest.mark.asyncio +async def test_async_build_schema_job_create_and_get() -> None: + from landingai_ade import AsyncLandingAIADE + + client = AsyncLandingAIADE(apikey=APIKEY) + respx.post("https://api.ade.landing.ai/v2/extract/build-schema/jobs").mock( + return_value=httpx.Response(202, json={"job_id": "bs1", "status": "pending"}) + ) + respx.get("https://api.ade.landing.ai/v2/extract/build-schema/jobs/bs1").mock( + return_value=httpx.Response(200, json={"job_id": "bs1", "status": "completed", "result": BUILD_SCHEMA_BODY}) + ) + created = await client.v2.build_schema_jobs.create(prompt="x") + assert created.status is JobStatus.PENDING + fetched = await client.v2.build_schema_jobs.get("bs1") + assert fetched.status is JobStatus.COMPLETED + assert isinstance(fetched.result, V2BuildSchemaResponse) diff --git a/tests/api_resources/v2/test_files.py b/tests/api_resources/v2/test_files.py deleted file mode 100644 index e3e376f..0000000 --- a/tests/api_resources/v2/test_files.py +++ /dev/null @@ -1,32 +0,0 @@ -from __future__ import annotations - -import httpx -import respx -import pytest - -from landingai_ade import LandingAIADE, AsyncLandingAIADE - -APIKEY = "My Apikey" - - -@respx.mock -def test_files_upload_returns_ref_and_hits_v2_host() -> None: - client = LandingAIADE(apikey=APIKEY, environment="production") - route = respx.post("https://api.ade.landing.ai/v1/files").mock( - return_value=httpx.Response(200, json={"file_ref": "fr_1"}) - ) - ref = client.v2.files.upload(file=b"markdown bytes") - assert route.called - assert route.calls.last.request.headers["authorization"] == "Bearer My Apikey" - assert ref == "fr_1" - - -@respx.mock -@pytest.mark.asyncio -async def test_async_files_upload() -> None: - client = AsyncLandingAIADE(apikey=APIKEY, environment="staging") - respx.post("https://api.ade.staging.landing.ai/v1/files").mock( - return_value=httpx.Response(200, json={"file_ref": "fr_2"}) - ) - ref = await client.v2.files.upload(file=b"bytes") - assert ref == "fr_2" diff --git a/tests/api_resources/v2/test_ground.py b/tests/api_resources/v2/test_ground.py index 8828076..0ee3d6a 100644 --- a/tests/api_resources/v2/test_ground.py +++ b/tests/api_resources/v2/test_ground.py @@ -5,11 +5,9 @@ import httpx import respx -import pytest from landingai_ade import LandingAIADE -from landingai_ade.types.v2 import JobStatus, V2GroundResult -from landingai_ade.lib.v2_errors import V2SyncTimeoutError +from landingai_ade.types.v2 import V2GroundResult APIKEY = "My Apikey" @@ -59,98 +57,6 @@ def test_ground_sync_parses_billing_metadata() -> None: assert result.metadata.billing.total_credits == 2.5 -@respx.mock -def test_ground_sync_504() -> None: - client = LandingAIADE(apikey=APIKEY, max_retries=0) - respx.post("https://api.ade.landing.ai/v2/ground").mock(return_value=httpx.Response(504, json={"detail": "x"})) - with pytest.raises(V2SyncTimeoutError): - client.v2.ground(extraction_metadata=EXTRACTION_METADATA, structure=STRUCTURE) - - -@respx.mock -def test_ground_job_create_and_get() -> None: - client = LandingAIADE(apikey=APIKEY) - respx.post("https://api.ade.landing.ai/v2/ground/jobs").mock( - return_value=httpx.Response( - 202, json={"job_id": "ground-1", "status": "pending", "created_at": "2026-01-01T00:00:00Z"} - ) - ) - job = client.v2.ground_jobs.create(extraction_metadata=EXTRACTION_METADATA, structure=STRUCTURE) - assert job.job_id == "ground-1" and job.status is JobStatus.PENDING - - respx.get("https://api.ade.landing.ai/v2/ground/jobs/ground-1").mock( - return_value=httpx.Response( - 200, - json={ - "job_id": "ground-1", - "status": "completed", - "created_at": "2026-01-01T00:00:00Z", - "completed_at": "2026-01-01T00:00:09Z", - "result": GROUND_BODY, - }, - ) - ) - done = client.v2.ground_jobs.get("ground-1") - assert done.status is JobStatus.COMPLETED - assert isinstance(done.result, V2GroundResult) - assert done.result.metadata.job_id == "ground-1" - - -@respx.mock -def test_ground_job_create_sends_output_save_url() -> None: - client = LandingAIADE(apikey=APIKEY) - route = respx.post("https://api.ade.landing.ai/v2/ground/jobs").mock( - return_value=httpx.Response(202, json={"job_id": "ground-2", "status": "pending"}) - ) - client.v2.ground_jobs.create( - extraction_metadata=EXTRACTION_METADATA, - structure=STRUCTURE, - output_save_url="https://example.com/put", - ) - req = json.loads(route.calls.last.request.content) - assert req["output_save_url"] == "https://example.com/put" - - -@respx.mock -def test_ground_job_get_failed_maps_error_object() -> None: - client = LandingAIADE(apikey=APIKEY) - respx.get("https://api.ade.landing.ai/v2/ground/jobs/g2").mock( - return_value=httpx.Response( - 200, json={"job_id": "g2", "status": "failed", "error": {"code": "internal_error", "message": "boom"}} - ) - ) - job = client.v2.ground_jobs.get("g2") - assert job.status is JobStatus.FAILED and job.error is not None and job.error.code == "internal_error" - - -@respx.mock -def test_ground_job_get_empty_job_id_raises() -> None: - client = LandingAIADE(apikey=APIKEY) - with pytest.raises(ValueError): - client.v2.ground_jobs.get("") - - -@respx.mock -def test_ground_job_list_carries_envelope() -> None: - client = LandingAIADE(apikey=APIKEY) - respx.get("https://api.ade.landing.ai/v2/ground/jobs").mock( - return_value=httpx.Response( - 200, - json={ - "jobs": [{"job_id": "ground-1", "status": "completed"}], - "page": 0, - "page_size": 10, - "has_more": True, - }, - ) - ) - jobs = client.v2.ground_jobs.list() - assert len(jobs) == 1 and jobs[0].job_id == "ground-1" and jobs[0].status is JobStatus.COMPLETED - assert jobs.has_more is True - assert jobs.page == 0 - assert jobs.page_size == 10 - - @respx.mock def test_ground_accepts_pydantic_structure() -> None: # `structure` may be passed as a pydantic model (e.g. a parse response's diff --git a/tests/contract/test_v2_smoke.py b/tests/contract/test_v2_smoke.py index 50664c6..98e1319 100644 --- a/tests/contract/test_v2_smoke.py +++ b/tests/contract/test_v2_smoke.py @@ -1,6 +1,7 @@ from __future__ import annotations import os +import json from typing import Iterator from pathlib import Path @@ -8,7 +9,13 @@ from pydantic import Field, BaseModel from landingai_ade import LandingAIADE -from landingai_ade.types.v2 import JobStatus, V2GroundResult, V2ExtractResult, V2ParseResponse +from landingai_ade.types.v2 import ( + JobStatus, + V2GroundResult, + V2ExtractResult, + V2ParseResponse, + V2BuildSchemaResponse, +) pytestmark = pytest.mark.contract @@ -34,12 +41,6 @@ def staging_client() -> Iterator[LandingAIADE]: yield client -def test_files_upload(staging_client: LandingAIADE) -> None: - file_ref = staging_client.v2.files.upload(file=("doc.md", SAMPLE_MARKDOWN.encode(), "text/markdown")) - assert isinstance(file_ref, str) - assert file_ref - - def test_extract_sync(staging_client: LandingAIADE) -> None: res = staging_client.v2.extract(schema=RevenueSchema, markdown=SAMPLE_MARKDOWN) assert isinstance(res, V2ExtractResult) @@ -57,6 +58,28 @@ def test_extract_jobs(staging_client: LandingAIADE) -> None: assert isinstance(done.result, V2ExtractResult) +def test_build_schema_sync(staging_client: LandingAIADE) -> None: + # Generate a JSON Schema from a source markdown document and a prompt. + res = staging_client.v2.build_schema( + markdowns=[SAMPLE_MARKDOWN], + prompt="Capture the total revenue figure and the company name.", + ) + assert isinstance(res, V2BuildSchemaResponse) + assert isinstance(res.extraction_schema, str) and res.extraction_schema + # The generated schema is a JSON Schema serialized as a string. + assert isinstance(json.loads(res.extraction_schema), dict) + + +def test_build_schema_jobs(staging_client: LandingAIADE) -> None: + job = staging_client.v2.build_schema_jobs.create( + markdowns=[SAMPLE_MARKDOWN], + prompt="Capture the total revenue figure and the company name.", + ) + done = staging_client.v2.build_schema_jobs.wait(job.job_id, timeout=300) + assert done.status is JobStatus.COMPLETED + assert isinstance(done.result, V2BuildSchemaResponse) + + def test_parse_sync(staging_client: LandingAIADE) -> None: pdf = Path(__file__).parent / "sample.pdf" resp = staging_client.v2.parse(document=pdf) @@ -96,19 +119,6 @@ def test_ground_sync(staging_client: LandingAIADE) -> None: assert grounded.metadata.job_id -def test_ground_jobs(staging_client: LandingAIADE) -> None: - parsed = staging_client.v2.parse(document=Path(__file__).parent / "sample.pdf") - assert parsed.structure is not None - extracted = staging_client.v2.extract(schema=RevenueSchema, markdown=parsed.markdown or "") - job = staging_client.v2.ground_jobs.create( - extraction_metadata=extracted.extraction_metadata, - structure=parsed.structure, - ) - done = staging_client.v2.ground_jobs.wait(job.job_id, timeout=300) - assert done.status is JobStatus.COMPLETED - assert isinstance(done.result, V2GroundResult) - - def test_parse_jobs(staging_client: LandingAIADE) -> None: pdf = Path(__file__).parent / "sample.pdf" job = staging_client.v2.parse_jobs.create(document=pdf) diff --git a/tests/test_v2_environment.py b/tests/test_v2_environment.py index 0dcc2c9..59a1e9d 100644 --- a/tests/test_v2_environment.py +++ b/tests/test_v2_environment.py @@ -67,9 +67,8 @@ def test_v2_attribute_exists() -> None: @respx.mock def test_v2_subclient_routes_to_v2_host() -> None: c = LandingAIADE(apikey=APIKEY, environment="production") - route = respx.post("https://api.ade.landing.ai/v1/files").mock( - return_value=httpx.Response(200, json={"file_ref": "ref-123"}) + route = respx.post("https://api.ade.landing.ai/v2/ground").mock( + return_value=httpx.Response(200, json={"grounding": {}, "metadata": {"job_id": "g", "duration_ms": 1}}) ) - ref = c.v2.files.upload(file=b"hello") + c.v2.ground(extraction_metadata={}, structure={"type": "document", "children": []}) assert route.called - assert ref == "ref-123" diff --git a/tests/test_v2_errors.py b/tests/test_v2_errors.py index 1ded801..7cf9b22 100644 --- a/tests/test_v2_errors.py +++ b/tests/test_v2_errors.py @@ -20,12 +20,14 @@ def _status_error(code: int) -> APIStatusError: def test_raise_if_sync_timeout_converts_504() -> None: - with pytest.raises(V2SyncTimeoutError): - raise_if_sync_timeout(_status_error(504)) + with pytest.raises(V2SyncTimeoutError) as exc_info: + raise_if_sync_timeout(_status_error(504), jobs_resource="build_schema_jobs") + # The remediation names the endpoint's own async route, not a hardcoded parse/extract one. + assert "build_schema_jobs" in str(exc_info.value) def test_raise_if_sync_timeout_ignores_other_codes() -> None: - raise_if_sync_timeout(_status_error(500)) # returns without raising + raise_if_sync_timeout(_status_error(500), jobs_resource="parse_jobs") # returns without raising def test_error_hierarchy() -> None: diff --git a/tests/test_v2_normalize.py b/tests/test_v2_normalize.py index dce3c88..aed1ba6 100644 --- a/tests/test_v2_normalize.py +++ b/tests/test_v2_normalize.py @@ -4,11 +4,16 @@ from typing import Any, Dict from datetime import datetime, timezone -from landingai_ade.types.v2 import JobStatus, V2GroundResult, V2ExtractResult, V2ParseResponse +from landingai_ade.types.v2 import ( + JobStatus, + V2ExtractResult, + V2ParseResponse, + V2BuildSchemaResponse, +) from landingai_ade.resources.v2._normalize import ( normalize_parse_job, - normalize_ground_job, normalize_extract_job, + normalize_build_schema_job, ) @@ -145,34 +150,42 @@ def test_normalize_extract_job_unknown_status_defaults_to_pending() -> None: assert job.raw["status"] == "some_brand_new_status" -def test_normalize_ground_job_iso_and_result() -> None: +def test_normalize_build_schema_job_iso_and_result() -> None: raw: Dict[str, Any] = { - "job_id": "ground-1", + "job_id": "extract-bs1", "status": "completed", "created_at": "2026-01-02T03:04:05Z", "completed_at": "2026-01-02T03:04:09Z", "result": { - "grounding": {"invoice_number": []}, - "metadata": {"job_id": "ground-1", "duration_ms": 10}, + "extraction_schema": '{"type": "object", "properties": {}}', + "metadata": {"job_id": "extract-bs1", "duration_ms": 10, "openapi_spec": "https://x/openapi.json"}, }, } - job = normalize_ground_job(raw) + job = normalize_build_schema_job(raw) assert job.status is JobStatus.COMPLETED assert job.created_at is not None and job.created_at.year == 2026 - assert isinstance(job.result, V2GroundResult) - assert job.result.metadata.job_id == "ground-1" + assert isinstance(job.result, V2BuildSchemaResponse) + assert job.result.metadata.job_id == "extract-bs1" + assert job.result.extraction_schema.startswith("{") -def test_normalize_ground_job_error_object() -> None: - raw = {"job_id": "g2", "status": "failed", "error": {"code": "internal_error", "message": "boom"}} - job = normalize_ground_job(raw) +def test_normalize_build_schema_job_error_object() -> None: + raw = {"job_id": "bs2", "status": "failed", "error": {"code": "internal_error", "message": "boom"}} + job = normalize_build_schema_job(raw) assert job.status is JobStatus.FAILED assert job.error is not None and job.error.code == "internal_error" -def test_normalize_ground_job_minimal_create_envelope_defaults_to_pending() -> None: - raw = {"job_id": "ground-x"} - job = normalize_ground_job(raw) - assert job.job_id == "ground-x" +def test_normalize_build_schema_job_minimal_create_envelope_defaults_to_pending() -> None: + raw = {"job_id": "extract-bs-x"} + job = normalize_build_schema_job(raw) + assert job.job_id == "extract-bs-x" assert job.status is JobStatus.PENDING assert job.result is None + + +def test_normalize_build_schema_job_smoke() -> None: + raw = {"job_id": "bs-x", "status": "completed", "result": {"extraction_schema": "{}"}} + job = normalize_build_schema_job(raw) + assert job.job_id == "bs-x" + assert job.status is JobStatus.COMPLETED diff --git a/tests/test_v2_types.py b/tests/test_v2_types.py index 7140817..3656c53 100644 --- a/tests/test_v2_types.py +++ b/tests/test_v2_types.py @@ -10,7 +10,7 @@ V2GroundResult, V2ExtractResult, V2ParseResponse, - V2FileUploadResponse, + V2BuildSchemaResponse, ) @@ -269,5 +269,28 @@ def test_ground_result_retains_unknown_fields() -> None: assert r.to_dict()["surprise"] == 1 -def test_file_upload_response() -> None: - assert V2FileUploadResponse(file_ref="abc").file_ref == "abc" +def test_build_schema_response_builds_from_dicts() -> None: + r = V2BuildSchemaResponse( + extraction_schema='{"type": "object", "properties": {"revenue": {"type": "string"}}}', + metadata={ # type: ignore[arg-type] + "job_id": "extract-bs1", + "duration_ms": 7, + "openapi_spec": "https://api.example/openapi.json", + "warnings": [{"code": "nonconformant_schema", "msg": "heads up"}], + "billing": {"service_tier": "priority", "total_credits": 1.5}, + }, + ) + assert isinstance(r.extraction_schema, str) + assert r.metadata.job_id == "extract-bs1" + assert r.metadata.warnings is not None and r.metadata.warnings[0].code == "nonconformant_schema" + assert r.metadata.billing is not None and r.metadata.billing.total_credits == 1.5 + + +def test_build_schema_response_retains_unknown_fields() -> None: + r = V2BuildSchemaResponse( + extraction_schema="{}", + # `openapi_spec` is required and non-null per the spec. + metadata={"job_id": "bs", "duration_ms": 1, "openapi_spec": "https://x/openapi.json"}, # type: ignore[arg-type] + surprise=1, # type: ignore[call-arg] + ) + assert r.to_dict()["surprise"] == 1