fix(sql): wire sql.engine/allowFullScan/maxSortRows/maxHashRows to real config

[kriszyp]

sqlEngine/config.ts read sql.engine/allowFullScan/maxSortRows/maxHashRows from globalThis.harperConfig, which nothing in production ever assigned — only three unit-test files set it as scaffolding. A value under sql: in harperdb-config.yaml never reached the SQL engine; sql.engine only appeared to work because it also has a HARPER_SQL_ENGINE env fallback, and the other three caps had no fallback at all and were permanently stuck at their compiled-in defaults. This registers the four keys with the real config layer (CONFIG_PARAMS + configUtils.getConfigValue(), the same accessor every other config domain uses) and deletes the dead globalThis branch, plus adds Joi validation for the section so a malformed value fails loudly instead of silently keeping the default.

Making sql: a validated core section collided with the fact that Harper's root config namespace also holds application entries: deploy_component project=sql package=… wrote sql: {package: …}, reported success, and made the next boot fail config validation with no recovery but hand-editing the YAML. So sql is now a reserved component name, refused at every ingress that creates a component under it, and an application already deployed under the name is grandfathered by a shape-switched schema plus a rename warning.

For the human reviewer

Framing: the planning gate returned better-alternative-exists and the alternative was adopted, rewriting the chosen approach: the reservation is a shared predicate in utility/componentNames.ts (the module the CLI and server already share for name derivation) applied to every component-creation ingress, rather than a check on the two obvious operations. That is what caught the set_configuration sql_package=x bypass — findUnrecognizedParams() deliberately admits arbitrary <component>_package/_port names and updateConfigValue() maps them straight into a root entry, so the deploy guard alone would not have held.

1. Reserve only sql, or every core root section? Only sql is reserved. Every other section (storage, node, replication, models…) inherits the top-level allowUnknown: true, so a component deployed under one of those names validates and boots today — it silently replaces that section's settings instead of failing, which is a real but different bug. Widening the set is additive and cheap later; each widening is a new rejection for someone who has such a component deployed now. Left for a follow-up rather than decided here (not yet filed as an issue).
2. A second, absolute reservation next to the existing force-overridable protected-core-component list. deploy_component already refuses graphql, http, … with a 409 that force: true overrides. This adds a distinct rule with a 400 that force cannot override, because there is no core component to overwrite — forcing would only let the operator write config that breaks their next boot. Two mechanisms with different force semantics now coexist; collapsing them later changes what force means for operators.
3. A grandfathered sql application is frozen for deploys, editable for migration. set_component_file and set_env_value refuse the name only when the project directory does not exist yet — a filesystem probe inside a Joi validator, and a second "is this legacy?" heuristic beside the config-key one. The uniform alternative (refuse always, migrate through drop/redeploy) is simpler but leaves an operator unable to patch the app they are being told to migrate.
4. A grandfathered sql application is frozen for redeploys. It keeps loading, but it cannot be redeployed or reconfigured — deploy_component refuses the name unconditionally, force included. That is deliberate friction to force the rename, and the boot warning is the only migration guidance shipped. The alternative (allow redeploy while the entry exists) keeps the ambiguity alive indefinitely.
5. A mixed entry is rejected rather than grandfathered. An entry carrying both an application key and an engine setting fails validation naming the key; false/null, componentLoader's spelling for a disabled component, is accepted so an operator who had already turned such an app off still boots. The pre-push review asked for historical mixed entries to be accepted and new ones blocked in mutation validation instead; declined, because "historical" is only distinguishable at a mutation — a hand-edited YAML, a restored backup or a replicated write reaching boot validation has no prior state to diff against, so the rule would be unenforceable exactly where the ambiguity does its damage. No supported path produces a mixed entry: deploy_component rebuilds the whole entry from request fields, and sql.engine was unreachable in production before this PR.
6. Sibling key shape, not sql.engine.*. sql.engine ships as its own scalar mode string, with allowFullScan/maxSortRows/maxHashRows as siblings under sql — not nested under engine as sqlEngine/PLAN.md (now corrected) used to describe. This matches SqlEngineConfig's pre-existing shape and every existing test's scaffolding. Reversible only before this ships: once a scalar sql.engine is in a customer's YAML, moving to a sql.engine.* section later is a breaking rename.
7. Config resolved per call site, not once per statement. getSqlEngineConfig() is still read fresh at each of its ~5 call sites (router, scan validator, hash-join/hash-aggregate build, sort) rather than resolved once into a per-statement snapshot. The alternative — attaching a resolved config to SqlEngineContext — only covers the physical-execution half of the pipeline; the router and the optimizer's validateScannable rule run before any SqlEngineContext exists, so it would need new plumbing invented for those two call sites too. Cost of not doing this: a set_configuration write landing mid-query could theoretically see allowFullScan/caps change between planning and execution within one statement — accepted, since config is restart-required everywhere else in Harper already, and the per-call cost is a handful of map lookups per statement, not per row.
8. sql.* Joi validation uses convert: false. validateConfig() only writes Joi's coerced value back into configDoc for threads/componentsRoot/logging/storage/operationsApi — never sql — so leaving Joi's default convert: true on would have accepted a quoted allowFullScan: "true" and then silently dropped it (the accessor's typeof guard rejects the still-unconverted string). convert: false makes that combination fail loudly at boot/set_configuration instead.
9. Registering sql_engine also exposes the bare env var SQL_ENGINE. Every CONFIG_PARAMS entry is reachable as an unprefixed environment variable on harper run (bin/run.ts's assignCMDENVVariables), and SQL_ENGINE is also a common external convention (Django, other ORM tooling) that could plausibly already be set, unrelated to Harper, in a shared .env/compose file — with the new Joi enum validation, such a collision now makes harper run abort instead of silently no-op. Discussed directly with the task owner: accepted as consistent with ~150 other existing bare CONFIG_PARAMS names carrying the same class of risk; a general "opt a param out of the bare-env-var scan" mechanism is left for a follow-up rather than special-cased here (not yet filed as an issue).
10. set_configuration sql_* still requires a restart, like every other config param: the write lands in the file without updating the in-memory flat config. The engine flag is the kind of knob an operator might expect to flip live during a cutover, but no config domain in Harper propagates live across threads.
11. Not covered by the reservation, deliberately: a components-root directory named sql that already exists (it has no root-config entry, so nothing fails validation; its operator finds out at the next redeploy, which is refused), and HARPER_BUILTIN_COMPONENTS=sql, which registers a trusted plugin by name and bypasses operation validation by design.

Verification

• npm run build, npm run lint:required, npm run format:write clean.
• npm run test:unit:main: 5334 passing, 1 failing — a pre-existing domain-socket-path-length assertion that resolves a relative rootPath against process.cwd() and fails from any agent worktree, identical on origin/main. npm run test:unit:resources 2004 passing / 0 failing; npm run test:unit:bin 252 passing.
• npm run test:integration -- "integrationTests/deploy/**/*.test.ts": 61 passing / 0 failing — real deploys still run through the new validator unchanged.
• Fails-on-base for the reservation: emptying RESERVED_COMPONENT_NAMES turns all 10 reservation assertions red across unitTests/components/reservedComponentName.test.js, unitTests/config/setConfigurationSql.test.js and the deploy handler cases; the grandfather schema cases fail on the previous commit, where a sql: {package} entry is rejected with 'sql.package' is not allowed. The four sql.* accessor assertions in unitTests/sqlEngine/router.test.js stay red on origin/main.
• Reservation coverage: deploy_component with an explicit sql, a canonicalized sql.tgz, a package-derived sql, the payload form, and force: true (400, through the real handler, which derives project before validating); add_component; set_component_file and set_env_value against a pinned temporary components root, covering both sides of the grandfather check; set_configuration sql_package/sql_port; and the CLI deploy setup=true name check.
• Config schema matrix: well-formed settings, a bad engine enum, a quoted allowFullScan, non-positive/non-integer row caps, an unknown key, {}, false/null (componentLoader's disabled-component spelling, which the closed settings schema would otherwise have failed at boot — found by the pre-push review), an application-shaped entry, and a mixed entry.
• Not proven end-to-end: no test drives a sql_* value from a set_configuration body through castConfigValue/flattenConfig to getSqlEngineConfig(); each side is covered in isolation, and a break there fails loudly at validation rather than silently. The boot rename warning is covered only through its predicate — validateConfig() is not exported and AGENTS.md forbids new sinon/rewire uses, so the one logger.warn call behind the tested predicate has no direct test. No API-to-restart integration test was added.

Complexity: medium

_{Review-Coverage: authored=claude; ran=codex,gemini; declined=cursor-grok,cursor-composer,domain; rounds=9 @ 6a66d45}

_{Human-Review-Need: 3 @ 6a66d45}

[gemini-code-assist]

Code Review

This pull request flattens the SQL engine configuration parameters (such as engine, allowFullScan, maxSortRows, and maxHashRows) under the main configuration utility instead of nesting them under globalThis.harperConfig. It updates configuration resolution, validation schemas, and associated unit tests to use these new flat keys. Feedback on the changes suggests importing plain 'node:assert' instead of 'node:assert/strict' in the new test file to comply with the repository's linting rules.

[claude]

Reviewed; no blockers found.