Skip to content

feat(context): transient (non-persisted) EventContext entries - #150

Draft
nicolasburtey wants to merge 2 commits into
mainfrom
feat/transient-event-context
Draft

feat(context): transient (non-persisted) EventContext entries#150
nicolasburtey wants to merge 2 commits into
mainfrom
feat/transient-event-context

Conversation

@nicolasburtey

@nicolasburtey nicolasburtey commented Jul 14, 2026

Copy link
Copy Markdown
Member

Motivation

lana-bank#6740 records request IP + user agent on audit entries by carrying them through es_entity::EventContext. Side effect: every repo with event_context = true copies that metadata — personal data under GDPR — into the immutable context column of every event persisted during the request.

This PR adds a context entry flavor that behaves exactly like today's context data in-process but is never serialized out of the process.

Design

  • ContextData now holds two maps: persisted and transient (field was already private — non-breaking).
  • Serialization is the enforcement point: the manual Serialize impl emits only the persisted map, in the same plain-map wire format as before. Deserialize reads a plain map into persisted. Consequences:
    • Old DB rows decode unchanged; new rows are byte-identical minus transient keys. No migration, no format versioning.
    • The sqlx Encode impl and the derived repo persist path (data_for_storing()) exclude transient entries with zero changes — any future serde path is safe by construction.
  • EventContext::insert_transient — mirror of insert, writes to the transient map.
  • ContextData::lookup stays unified: transient first, then persisted (transient shadows on collision). Consumers work unchanged regardless of which insert produced the key, making downstream cutover painless.
  • Propagation (fork / seed / WithEventContext / per-poll re-seeding) is untouched — it clones ContextData wholesale, which is exactly what carries transient entries across polls/tasks/threads.

Erasure design rule (for the docs)

  • Payload PII (domain state, must be readable until erasure) → per-entity data key + codec (forgettable-payloads direction). Recoverable until shredded.
  • Context PII (request metadata, observability) → transient context. Never persisted; nothing to shred later.

Testing

  • Unit: transient insert/lookup roundtrip, shadowing, serde exclusion, legacy wire-format roundtrip.
  • Boundary tests extended: transient entries survive fork, thread seed, tokio::spawn via WithEventContext (current-thread + multi-thread).
  • New Postgres integration test asserts at the DB level that the context column contains the persisted key and lacks the transient one (create + update paths).
  • CI change: nextest now runs with --features event-context — previously the context persistence path was never integration-tested in CI. The macro codegen snapshot test is feature-aware accordingly.

Versioning

Additive API, unchanged wire format → 0.10.x minor bump (0.10.41).

Out of scope (follow-up)

  • #[es_event_context(transient(arg))] macro syntax.

🤖 Generated with Claude Code


Note

Medium Risk
Touches event-context serialization and persistence paths used on every append; wire format is unchanged but behavior for PII-bearing metadata changes materially.

Overview
Adds EventContext::insert_transient so request-scoped metadata (e.g. client IP, user agent) propagates like normal context in-process but never lands in the immutable events context column.

ContextData is split into persisted vs transient maps; a custom Serialize impl still emits only the persisted map (same JSON wire format as before), so sqlx encoding, repo persist via data_for_storing(), and existing DB rows stay compatible without migration. lookup checks transient first (shadowing persisted keys); Deserialize loads a plain map into persisted only.

Docs cover persisted vs transient context and PII/erasure guidance. CI nextest runs with --features event-context; macro snapshot test matches that default. New Postgres integration test asserts transient keys are absent from user_events.context on create and update.

Reviewed by Cursor Bugbot for commit 7fc397b. Bugbot is set up for automated code reviews on this repo. Configure here.

Add EventContext::insert_transient for context entries that behave
exactly like persisted entries in-process — they survive fork(),
thread hand-off via seed(), and async boundaries via WithEventContext
— but are never serialized out of the process.

ContextData now holds two maps (persisted + transient). The manual
Serialize impl emits only the persisted map, keeping the wire format
byte-identical to the previous plain-map format: old DB rows decode
unchanged and no migration is needed. Deserialize reads the plain map
into persisted entries. Because the sqlx Encode impl and
data_for_storing() go through Serialize, the derived repo persist
path excludes transient entries with zero changes — the guarantee
lives on the type, not on call-site discipline.

ContextData::lookup stays unified: it checks transient entries first,
then persisted ones (transient shadows on collision), so consumers
work unchanged regardless of which insert produced the key.

Motivation: request-scoped personal data (client IP / user agent)
carried through the event context for audit purposes must not be
copied into immutable event streams.

CI now runs nextest with --features event-context so the context
persistence path is integration-tested; the macro codegen snapshot
test is feature-aware accordingly. A new integration test asserts at
the database level that the context column contains persisted keys
and lacks transient ones.

Out of scope (follow-up): #[es_event_context(transient(arg))] macro
syntax.
"Must not be persisted forever" read as "never persisted". The actual
design: consumers may still read a transient entry and store it
deliberately in a mutable table that can honor erasure requests (e.g.
an audit table); transient only guarantees the event-context machinery
never fans it out into append-only event streams as a side effect.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant