feat(deployment): reject config where metadata and file category names overlap

[corneliusroemer-agent]

While reviewing #6816 I realized that things would break silently if file category names match metadata fields. Similar issues arise if metadata fields match common-metadata (reserved) fields like "accession".

The best way to validate this is in the helm chart as it's the only place where common-metadata vs normal metadata is defined, and it also knows about file categories. I first wanted to put the validation in the backend but that would mean duplicating the reserved names.

What I did not cover

perSegment fields get expanded to name_L / name_M / name_S by the SILO and website templates, so a file category named e.g. length_L could still collide with an expanded name. Catching that would mean duplicating the segment-expansion logic inside the validator, which did not feel worth it.

I also left earliestReleaseDate exactly as it is. It is the one name that is both injected by the backend and declared in organism metadata. Earliest release date should really go in common-metadata but its per-organism nature makes that a little involved so keeping it out of this PR for now.

Claude description

Released data, the website config and the SILO database config each flatten three separate sources into one set of metadata keys: the commonMetadata fields the chart adds to every organism, the organism's own schema.metadata (after metadataAdd is merged in), and the file categories from schema.files and submissionDataTypes.files.categories.

Every one of those merges silently lets one source win over another, so an overlap never raises an error anywhere. A field just quietly disappears, or a real metadata value gets replaced by the file-list JSON string, and in the SILO database config you additionally get two metadata entries with the same name. This is an implicit requirement of code we already have, and nothing currently checks it.

This came up while reviewing #6816, which adds a rawReads file category — but it applies to any file category or metadata field, so it is worth having independently of that PR.

Why in helm rather than the backend

I tried this in the backend first and it is the wrong place. The backend config deliberately does not receive commonMetadata, because schema.metadata doubles as the whitelist of fields that may be submitted — ProcessedSequenceEntryValidator rejects any metadata key outside it. Handing the backend commonMetadata would make accession, submitter and friends acceptable as pipeline-supplied metadata, so it cannot see the list it would need to check against. Doing it there means hand-mirroring the reserved names in Kotlin, which drifts: my first attempt already missed versionComment.

In the chart all three lists are in scope at once, so there is no duplicated list to keep in sync.

What I did not cover

perSegment fields get expanded to name_L / name_M / name_S by the SILO and website templates, so a file category named e.g. length_L could still collide with an expanded name. Catching that would mean duplicating the segment-expansion logic inside the validator, which did not feel worth it — happy to add it if you disagree.

I also left earliestReleaseDate exactly as it is. It is the one name that is both injected by the backend and declared in organism metadata, which initially looked like it needed a special case. It does not: because the rule compares organism metadata against commonMetadata, and earliestReleaseDate is not in commonMetadata, there is nothing to exempt — and a file category colliding with it is still caught, via the organism-metadata side of the second rule. So no config migration is needed.

For the reviewer

The first rule only forbids organism metadata overlapping commonMetadata. The second forbids file categories overlapping either list — I included categories-vs-commonMetadata as well as categories-vs-organism-metadata since it costs nothing and the website and SILO configs really do merge all three, but say if you want it narrowed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TfNESPRGqoCmoh7Dr16Mt8

🚀 Preview: Add preview label to enable