Document per-query and per-principal memory limits#462
Open
puzpuzpuz wants to merge 5 commits into
Open
Conversation
This was referenced Jun 5, 2026
|
🚀 Build success! Latest successful preview: https://preview-462--questdb-documentation.netlify.app/docs/ Commit SHA: 1d8fbfc
|
- show.md: SHOW SERVICE ACCOUNTS <user|group> returns name, grant_option and SHOW GROUPS <user> returns name, external_alias; only the bare listings carry the new memory_limit column - meta.md: query_activity example projects memory_used/memory_limit - rbac.md: alphabetize ALTER entries in the SQL commands reference - alter-user.md: add the --- separator before Syntax for parity with siblings - capacity-planning.md: link the RAM section to the memory limits docs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror questdb/questdb-enterprise@fb4e050: the effective per-principal memory limit resolves by strict precedence (user's own limit -> group limit -> global workload limit) rather than the most-restrictive value. A more specific limit, when set, fully overrides the broader one and binds even when larger, so a per-entity override can raise a principal's ceiling above the workload limit, not only lower it. Service accounts never inherit group limits. Updated rbac.md, alter-user/group/service-account.md, show.md, meta.md, and cairo-engine.md accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mory limit Reviewer note on questdb-enterprise#1039: the memory-limits section named only mat-view refresh and WAL apply as running under internal (non-principal) contexts, but UPDATE applied through the WAL runs under the allow-all root context too (OperationExecutor.getRootContext, reports limit 0), so a large UPDATE is not capped by a SET MEMORY LIMIT override. Name UPDATE explicitly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… the gaps Three corrections, each checked against the Enterprise implementation rather than inferred: - `UPDATE` was described as never picking up a per-principal limit. That is false for a non-WAL table: `UpdateOperation.apply()` passes the caller's own SqlExecutionContext to `UpdateOperatorImpl.executeUpdate`, which calls `queryRegistry.register(...)`, which acquires a QUERY tracker from that context's security context. So a non-WAL UPDATE IS capped, and an operator sizing one from the old text would meet an unexpected OOM abort. Only the WAL path -- the default table type -- runs under the WAL apply job's tracker. Replaced with a section that splits what a per-principal limit covers from what it does not. - `SHOW GROUPS <user>` and `SHOW SERVICE ACCOUNTS <user | group>` were shown without a `memory_limit` column, and the prose said so explicitly. They carry it: AbstractShowLinkedEntitiesCursorFactory adds it to both GROUP_METADATA and METADATA, pinned by ShowAclTest. Since these are exactly the results that break a positional client, the omission hid half the breaking change. - The example tables disagreed with each other on the same entity's limit; made them consistent. Documents four things the page did not mention at all: - `COPY ... TO` exports are capped by the issuing principal's limit, so a limit must be sized for exports and not only for interactive queries. - `SET MEMORY LIMIT` is self-escalating. It takes no entity name, so its holder can set any principal's limit including its own, and a per-entity limit overrides the workload limit rather than tightening it -- a non-admin holding it can raise its own ceiling above `cairo.query.memory.limit.bytes`. Plus the `GRANT ALL` expansion, which means an upgrade does not confer it. - The breaking change: five results gain a column on upgrade, and positional clients must be updated. - `sys.acl_entities` stores the value, readable only by the built-in admin. Also notes that `memory_limit` means the effective limit in `SHOW USERS` but the entity's own limit in `SHOW GROUPS` / `SHOW SERVICE ACCOUNTS`, and that an external user's inherited limit refreshes at next login rather than live. Verified with `yarn build`, which is configured to throw on broken links and anchors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documents the new memory-limit feature across configuration, monitoring, and access control.
OSS — per-workload memory limits
configuration/cairo-engine.md: new "Memory limits" section covering the three reloadable workload limits (cairo.query.memory.limit.bytes,cairo.mat.view.refresh.memory.limit.bytes,cairo.wal.apply.memory.limit.bytes), breach behavior, and the best-effort coverage caveat. Cross-links added from the WAL and materialized-views config pages.query/functions/meta.md:query_activitygains thememory_usedandmemory_limitcolumns.Enterprise — per-principal memory limits
query/sql/acl/alter-user.md,alter-service-account.md, and a newalter-group.md(+ sidebar):SET MEMORY LIMIT { <size> | UNLIMITED }.security/rbac.md: new "Memory limits" section — how limits resolve, what a per-principal limit does and does not cover, how to inspect limits, theSET MEMORY LIMITpermission, andALTER GROUPin the command list.query/sql/show.md: thememory_limitcolumn onSHOW USERS,SHOW GROUPSandSHOW GROUPS <user>, andSHOW SERVICE ACCOUNTSandSHOW SERVICE ACCOUNTS <user | group>.Limit resolution — override, not min. Limits resolve by strict precedence: a principal's own limit → (users only) the most-restrictive limit among its groups → the global
cairo.query.memory.limit.bytesworkload limit. A0/UNLIMITEDvalue at a level means "not set" and falls through to the next. A more specific limit, when set, fully overrides the broader one and binds even when larger, so a per-user or per-group override can raise a principal's ceiling above the workload limit, not only lower it. Service accounts never inherit group limits. (Earlier drafts of these docs described a most-restrictive "min" behavior; revised per questdb/questdb-enterprise@fb4e050 "Make per-entity memory limits override, not min".)Corrections made after checking against the implementation
Two statements in earlier revisions of this PR were wrong, and both were verified against the Enterprise code rather than re-reasoned:
UPDATEwas described as never picking up a per-principal limit. False for a non-WAL table.UpdateOperation.apply()passes the caller's ownSqlExecutionContexttoUpdateOperatorImpl.executeUpdate, which callsqueryRegistry.register(...), which acquires a QUERY-workload tracker from that context's security context — so a non-WALUPDATEis capped bySET MEMORY LIMIT. An operator sizing one from the old text would have met an unexpected OOM abort. Only the WAL path (the default table type) runs under the WAL apply job's tracker. Replaced with a "What a per-principal limit covers" section that splits the two.SHOW GROUPS <user>andSHOW SERVICE ACCOUNTS <user | group>were shown without amemory_limitcolumn, with prose saying so explicitly. They carry it —AbstractShowLinkedEntitiesCursorFactoryadds it to both metadata shapes, pinned byShowAclTest. Since these are among the results that break a positional client, the omission hid half the breaking change.Example tables that disagreed with each other on the same entity's limit were also made consistent.
Behaviour the docs previously did not mention at all
COPY ... TOexports are capped by the issuing principal's limit. The export runs under that principal's own context on the query workload, so a limit must be sized for the largest single thing the principal runs, exports included — not only for its interactive queries.SET MEMORY LIMITis self-escalating. It takes no entity name, so its holder can set any principal's limit including its own; combined with override-not-min, a non-admin holding it can raise its own ceiling abovecairo.query.memory.limit.bytes. Documented as a warning, together with theGRANT ALLexpansion — a principal grantedALLbefore the upgrade does not acquire the new permission.SHOW USERS,SHOW GROUPSandSHOW GROUPS <user>,SHOW SERVICE ACCOUNTSandSHOW SERVICE ACCOUNTS <user | group>), plusSELECT *onsys.acl_entities. Positional clients must be updated; name-based clients are unaffected.sys.acl_entitiesstores the value, and is readable only by the built-in admin — a principal holdingDATABASE ADMINis still denied.memory_limitmeans different things per statement: the effective limit inSHOW USERS, the entity's own limit inSHOW GROUPS/SHOW SERVICE ACCOUNTS, since neither inherits one.Dependencies
Documents:
Web-console syntax highlighting for the new clause: questdb/sql-parser#27
Verification
yarn build, which this repo configures to throw on broken links, broken markdown links, and broken anchors.