Skip to content

Support for RegoVM-based OPA engine for real await function call - #225

Open
imlk0 wants to merge 8 commits into
mainfrom
regovm
Open

Support for RegoVM-based OPA engine for real await function call#225
imlk0 wants to merge 8 commits into
mainfrom
regovm

Conversation

@imlk0

@imlk0 imlk0 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

This PR adds an opt-in regorus-regovm Cargo feature that switches the OPA engine to regorus's VM backend, which drives host functions through the suspendable __builtin_host_await loop instead of the sync Extension bridge.

regovm natively supports async extension calls (RVPS lookups, artifact-server queries), so it runs on single-threaded wasm32 and avoids nesting the tokio runtime (microsoft/regorus#667, microsoft/regorus#730, microsoft/regorus#363). Since it still has some syntax gaps vs the interpreter, the interpreter stays the default and regorus-regovm is opt-in.

The VM path also caches compiled programs across evaluations (keyed by policy id + content hash) so repeated appraisals skip parse/compile, and gates regorus's arc feature behind regorus-regovm so the interpreter keeps the faster Rc.

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@imlk0
imlk0 marked this pull request as ready for review August 24, 2026 08:54
@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

…tion

`build_extensions` splices each caller-supplied extension key raw into a
Rego rule head (`{key}(arg) := v if { ... }`) and a string literal
(`"{key}"`). A key carrying a newline, comment, quote, or brace could
inject Rego source — appending `allow := true` would flip a
`default allow := false`. regorus ships no identifier validator for this,
so validate at the interpolation point itself.

Add `is_valid_rego_extension_name`: a dotted path of Rego identifiers
(each segment `[A-Za-z_][A-Za-z0-9_]*`, dot-separated, not a reserved
keyword). Dotted names are accepted because both backends resolve them —
rego.v1 admits a ref-headed function definition (regovm) and the Engine
resolves a dotted `add_extension` path (interpreter). `build_extensions`
now returns `Result` and rejects any name outside this set, so the gate
is unbypassable for the source-generation path.

Characterize the contract with non-ignored tests:
- regovm accepts a dotted wrapper name (compiles + end-to-end eval);
- interpreter accepts a dotted extension name via `add_extension`;
- the injection name is rejected before interpolation, keeping
  `default allow := false` intact.

On this branch `build_extensions`/`build_extensions_module` are
`#[cfg(feature = "regorus-regovm")]` (the interpreter path uses
`Engine::add_extension`, which does not interpolate source, so it is not
an injection surface). The validator and its helpers are gated to match,
so the interpreter lib build carries no dead code (CI clippy `-D warnings`).

Co-Authored-By: Claude <noreply@anthropic.com>
imlk0 added a commit that referenced this pull request Sep 3, 2026
…ion path

The PR #225 benchmark review (jialez0, comment on eval_bench) noted the
bench tests a separately-copied cache impl structurally divergent from the
production ProgramCache/resolve_programs, so bench-pass does not imply the
production cache is correct — and at the time no non-bench test exercised
the production cache path. ab2b56d fixed the rule-set-change bug and added
one regression through OPAInMemory::evaluate; the remaining stale-source /
hash-mismatch branch of resolve_programs was still untested.

Add three production-path tests pinning every cache branch:

- set_policy_drops_program_cache_slot (in_memory.rs): OPAInMemory::set_policy
  must eagerly remove() the policy_id's slot, not leave it for the hash
  check to evict lazily. White-box on the real ProgramCache.
- evaluate_picks_up_external_policy_file_change (fs.rs): an external
  {policy_id}.rego overwrite (no set_policy -> no eager eviction) must be
  picked up by the next evaluate; under regorus-regovm this exercises
  resolve_programs' hash-mismatch branch directly through the fs read-fresh
  path.
- common_evaluate_invalidates_cache_when_policy_source_changes (mod.rs):
  call common_evaluate with the same policy_id but different source; assert
  the cache auto-invalidates and the second appraisal reflects the new
  policy.

Also derive Debug on CachedPolicy: OPA's #[derive(Debug)] (fs.rs) only
compiles under the previously-untested fs+regorus-regovm feature combo
(no CI matrix combines them) because CachedPolicy lacked Debug. Safe since
regorus::rvm::Program already derives Debug; unblocks the fs+regovm build.

Co-Authored-By: Claude <noreply@anthropic.com>
@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

imlk0 and others added 5 commits September 4, 2026 09:43
A cache hit served only the rules the first appraisal compiled; a later
request for a rule the policy defines but the cache lacked was skipped as
"not defined". On a hit, compile just the missing rules and merge them in.
Regression-tested via OPAInMemory::evaluate with two disjoint rule sets.

Co-Authored-By: Claude <noreply@anthropic.com>
…on_functions

The regovm path now validates extension names in build_extensions, so the
note telling callers to validate upstream is misleading.

Co-Authored-By: Claude <noreply@anthropic.com>
Drop the regorus-interpreter/regorus-regovm mutual exclusion: regorus-regovm
is the single opt-in feature (default stays the interpreter). Gate regorus's
`arc` behind it -- the VM needs Send regorus types (compiler errors reach an
anyhow/Send boundary), the interpreter keeps Rc (futures block_on'd locally,
never Send) and is ~12% faster on the default EAR policy.

Co-Authored-By: Claude <noreply@anthropic.com>
The compiled program is data-independent (data.x -> LoadData reads the VM's
runtime store, set per-eval), so the real data was never baked into bytecode.
Pass an empty object -- it just satisfies compile_with_entrypoint's
prepare_for_eval, which requires a valid object, and keeps the cached program
safe to reuse across different data.

Co-Authored-By: Claude <noreply@anthropic.com>
Pin the cache branches via OPAInMemory::evaluate / common_evaluate:
set_policy eagerly drops the slot, an external .rego overwrite is picked up
on the next eval, and a same-id different-source appraisal auto-invalidates.
Also derive Debug on CachedPolicy (unblocks the fs+regovm build) and fmt the
new tests.

Co-Authored-By: Claude <noreply@anthropic.com>
@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@imlk0
imlk0 marked this pull request as ready for review September 4, 2026 02:36
@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@imlk0 imlk0 changed the title Switch OPA engine to RegoVM for real await function call Support for RegoVM-based OPA engine for real await function call Sep 4, 2026
@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants