Skip to content

add_index replaces instead of merging, de-registering an entity's earlier indexed fields #112

Description

@fabracht

Discovered while investigating PR #111 (#90). IndexManager::add_index replaces the per-entity index definition rather than merging, so registering a second indexed/unique field on the same entity silently unregisters the first field's index.

Root cause

crates/mqdb-core/src/index.rs:

pub fn add_index(&mut self, definition: IndexDefinition) {
    self.indexes.insert(definition.entity.clone(), definition); // <- replace, not merge
}

self.indexes is a HashMap<String, IndexDefinition> keyed by entity, and get_indexed_fields returns only the current definition's fields. So the second add_index clobbers the first.

Reachable through the normal API (no guard)

  • Database::add_index(entity, fields) (crates/mqdb-agent/src/database/schema_ops.rs) and add_unique_constraint(entity, fields) (which calls add_index, schema_ops.rs:126) each create a fresh IndexDefinition::new(entity, fields) and call manager.add_index — no merge.
  • Public MQTT admin endpoints $DB/_admin/index/{entity}/add and $DB/_admin/constraint/{entity}/add pass fields straight through (crates/mqdb-agent/src/agent/handlers.rs).

Empirically confirmed: add_index(users,[email]) then add_index(users,[name]) (or two add_unique_constraint calls on email then username) leaves get_indexed_fields("users") == ["name"] and is_field_indexed("users","email") == false, while the live idx/users/email/… entries remain on disk.

Consequences

  1. Query degradation: after the second registration, an equality/range filter on the first field (email) no longer takes the index path (list_with_filtersis_field_indexed is false) and falls back to a full scan. Results stay correct; performance regresses silently.
  2. Orphaned index entries: add_index only writes entries for the new field set and never removes the old field's entries, so idx/users/email/… lingers with no registered definition. (Uniqueness enforcement is unaffected — that runs off the constraint manager, which keeps both constraints.)
  3. This is the root cause behind the narrow purge_stale_index_entries scan to indexed fields #90 self-heal narrowing that was dropped from PR use uuid v7 for time-ordered server-generated ids #111: a narrowed stale-index scan keyed on get_indexed_fields misses the orphaned field's entries.

Proposed fix

Make an entity's index definition the union of all registered fields: add_index should merge fields into the existing IndexDefinition (dedup) rather than replace it, and remove/drop should subtract. Then get_indexed_fields reflects every field with live entries, is_field_indexed stays true for all registered fields, and a field-scoped stale-index scan (a future re-attempt of #90) becomes correct.

Acceptance

  • Registering indexes/unique constraints on two different fields of one entity keeps both fields in get_indexed_fields and both is_field_indexed true.
  • Equality filters on either field use the index path.
  • Test covering the two-field registration (both add_index and add_unique_constraint paths).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions