You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-up to the #94 comprehensive fix (PR forthcoming). The steady-state (converged) cluster path is closed: prepare_fk_side_effects fails the delete closed when a set_null (direct or cross-owned) would null a schema-typed field, and the constraint API rejects such definitions. This tracks a narrow transient residual an adversarial review surfaced.
The window
The runtime guard (reject_typed_set_null in crates/mqdb-cluster/src/cluster/node_controller/db_ops.rs) runs on the coordinating node and fails open when that node's schema is not yet synced (schema_get returns None). The executing primary for a RemoteSetNull handles it as a plain JsonDbOp::Update in handle_json_update_local (and execute_local_cascade_set_null) and does not re-check the schema. So during a node-join / snapshot-sync window — a coordinator that already holds the FK constraint but has not yet imported the broadcast schema — a RemoteSetNull can reach a remote primary that does hold the typed schema and write the null with no second check.
Mitigation already in place
DB_SCHEMA and DB_CONSTRAINT are broadcast in the same set (replication_ops.rs), and snapshots assert schema uniformity across nodes. A node lacking the schema very likely also lacks the FK constraint/reverse index, in which case it produces no set_null effect at all. So the window is narrow and not provably reachable — but schema and constraint are separate stores with no proven ordered/atomic arrival, and the referencing child records are partitioned data a primary can hold independently, so it is not provably empty.
Options
Remote-leg defense-in-depth (partial): invoke a schema type-check on the executing node inside handle_json_update_local / execute_local_cascade_set_null when __mqdb_fk_expected is present, refusing to write null into a typed field. Closes the remote sub-case. Interacts with cascade-ack/retry semantics (a refused RemoteSetNull leaves the cascade un-acked → dangling reference rather than corruption) — needs care so it does not create a stuck-retry state. The local-leg (coordinator is the primary) shares the coordinator's schema view, so a same-node re-check does not help it.
Full bootstrap-ordering fix: do not process FK-cascade deletes that produce set_null effects until the node's schema catalog is confirmed synced. This is the airtight closure and the same class as cluster unique: empty-map sync self-seal + bootstrap founding speed-up #106; it should carry a TLA model + real-cluster smoke.
Acceptance
A model (extend the cluster unique/fk specs or a new one) demonstrating the skew window and the chosen fix's safety.
Real-cluster smoke: a joining node coordinating an FK set_null delete before schema import does not corrupt a typed field, and does not wedge the cascade.
No regression to steady-state cluster delete throughput.
This is a restart/join-window residual only; steady-state clusters are unaffected.
Follow-up to the #94 comprehensive fix (PR forthcoming). The steady-state (converged) cluster path is closed:
prepare_fk_side_effectsfails the delete closed when aset_null(direct or cross-owned) would null a schema-typed field, and the constraint API rejects such definitions. This tracks a narrow transient residual an adversarial review surfaced.The window
The runtime guard (
reject_typed_set_nullincrates/mqdb-cluster/src/cluster/node_controller/db_ops.rs) runs on the coordinating node and fails open when that node's schema is not yet synced (schema_getreturnsNone). The executing primary for aRemoteSetNullhandles it as a plainJsonDbOp::Updateinhandle_json_update_local(andexecute_local_cascade_set_null) and does not re-check the schema. So during a node-join / snapshot-sync window — a coordinator that already holds the FK constraint but has not yet imported the broadcast schema — aRemoteSetNullcan reach a remote primary that does hold the typed schema and write the null with no second check.Mitigation already in place
DB_SCHEMAandDB_CONSTRAINTare broadcast in the same set (replication_ops.rs), and snapshots assert schema uniformity across nodes. A node lacking the schema very likely also lacks the FK constraint/reverse index, in which case it produces noset_nulleffect at all. So the window is narrow and not provably reachable — but schema and constraint are separate stores with no proven ordered/atomic arrival, and the referencing child records are partitioned data a primary can hold independently, so it is not provably empty.Options
handle_json_update_local/execute_local_cascade_set_nullwhen__mqdb_fk_expectedis present, refusing to write null into a typed field. Closes the remote sub-case. Interacts with cascade-ack/retry semantics (a refusedRemoteSetNullleaves the cascade un-acked → dangling reference rather than corruption) — needs care so it does not create a stuck-retry state. The local-leg (coordinator is the primary) shares the coordinator's schema view, so a same-node re-check does not help it.set_nulleffects until the node's schema catalog is confirmed synced. This is the airtight closure and the same class as cluster unique: empty-map sync self-seal + bootstrap founding speed-up #106; it should carry a TLA model + real-cluster smoke.Acceptance
set_nulldelete before schema import does not corrupt a typed field, and does not wedge the cascade.This is a restart/join-window residual only; steady-state clusters are unaffected.