Fix: parse --workflow-dag-simple JSON before posting provenance

[mitchest]

Summary

--workflow-dag-simple is never JSON-parsed, so the CLI posts it to the API as a string where the API requires an object. Every provenance write that passes this flag has failed with a 422 since app v0.3.0 was deployed on 2026-05-19. This parses the option in the three provenance commands, matching how --workflow-dag is already handled.

The bug

In cli_provenance.py, the two DAG options are treated inconsistently:

workflow_dag_parsed = loads(workflow_dag) if workflow_dag else read_steps()   # parsed
...
workflow_dag_simple=workflow_dag_simple,   # raw string, straight through

The raw string is threaded into get_provenance() (provenance.py:200) and dropped into the payload verbatim, so the API receives:

"workflowDagSimple": "{\"description\": \"Prepare the Global 10-meter Seagrass dataset...\"}"

The app declares workflowDagSimple as an object (packages/schemas/src/crud.ts:61, applied to input at :239), producing:

422 Validation Error
  path: workflowDagSimple
  message: Invalid input: expected object, received string
  code: invalid_type

c812fdf ("Add simple workflow visualisation data", 2026-05-04) is the only commit that has ever touched this field, and it shipped without the parse. The path has therefore never worked.

Why it wasn't caught

The database has 7 dataset / 5 geometries / 17 product runs where workflow_dag_simple is a valid object, which makes the feature look healthy. Those rows are not from the CLI. They're all dated 2026-04-29 and their images (0.0.0-485-g31b2df1 and similar) predate the feature at commit 492, so the CLI could not have written them. Their provenance_json->'workflowDagSimple' is also absent while the column is populated — consistent with a direct backfill, likely to give the product-output sidebar UI something to render.

Two things then hid the breakage:

1. No run has been recorded since strict validation deployed. Zero rows exist in dataset_run, geometries_run, or product_run created after 2026-05-19. The failure is total, not intermittent.
2. The provenance JSON is written before the DB post, so a .json in S3 does not mean the write succeeded. The aus-states geometry run on 2026-05-25 wrote its provenance to S3 with a string and never landed in the database — it failed exactly this way and went unnoticed.

Changes

• cli_provenance.py — parse the option in _write_dataset_provenance, _write_geometry_provenance, and _write_product_provenance:

  workflow_dag_simple_parsed = (
      loads(workflow_dag_simple) if workflow_dag_simple else None
  )

  and pass the parsed value through. Type hint on _meta_provenance corrected str | None → dict | None; docstring entry added.

• provenance.py — same type-hint correction on get_provenance.

This also fixes the provenance JSON written to S3, which currently stores a string where the backfilled DB rows hold objects.

Verification

Ran the real csdr provenance dataset command inside csdr-cloud-spatial:latest with requests.post intercepted (no DB or S3 writes), using the seagrass template's actual workflow-dag-simple value:

	workflowDagSimple on the wire	Against app zod schema
Before	str → JSON string	422, invalid_type — matches the reported error verbatim
After	dict → JSON object	passes

All 37 templates that pass --workflow-dag-simple validate against the current schema once parsed, so no template changes are needed. ruff format --check and ruff check pass.

Not yet verified: a live 200 against data.dev.sd17.org, since that writes a real dataset_run row to dev. Suggest confirming via a seagrass-global dataset run after the image builds.

Reviewer question

This assumes the intended contract is CLI parses, API receives an object, consistent with --workflow-dag and with the objects already in the database. The alternative — loosening the app schema to accept a JSON string — would leave the existing object rows inconsistent, so I've gone the other way. Flagging in case the original design intended otherwise.

Blast radius

Fixes provenance DB writes for all datasets, geometries, and products. Requires a new image build before the workflows will succeed. No migration needed; existing rows are already objects and stay valid.