diff --git a/CHANGELOG.md b/CHANGELOG.md index a8e81d0..ecdf866 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,34 @@ All notable changes to this project will be documented in this file. ## Unreleased +## [0.0.2] - 2026-08-27 + +### Changed + +- Pinned bundled native vllm.cpp to tag `v0.0.2`, commit `7020de93652ca920424a10ac5255b34810dd2f24`, moving the stable C contract from ABI 10 with 19 functions to ABI 17 with 35 functions; system libraries must match the new ABI, layouts, signatures, and exports. +- Model construction now obtains native defaults and overlays only explicit Rust settings, preserving helper defaults including `block_size=32`, `max_num_seqs=32`, and `gpu_memory_utilization=0.92`. +- Hardened deterministic backend configuration, binding/export/package checks, package curation, and multi-architecture Triton packaging: all six vendored AOT trees are embedded, exact-SM dispatched, and paired with portable fallback on other accepted targets. +- Raised the safe crate unpacked package limit to 512 KiB while retaining its 40-file and 128-KiB compressed limits; sys limits remain 1,400 files, 40 MiB unpacked, and 6 MiB compressed. + +### Added + +- `Device::{Auto, Cpu, Cuda}` selection and KV-memory controls with native precedence `num_blocks > kv_cache_memory_bytes > gpu_memory_utilization` and profile/fallback sizing. +- Pre-tokenized `Engine::complete_tokens` with owned token output, optional copied completion metadata, and explicit truncation reporting. +- Thread-local, exclusive `TranscriptionEngine` and `EmbeddingEngine` owners with borrowed inputs and Rust-owned transcription and row-major embedding results. +- Separate `VideoEngine` checkpoint-set ownership, blocking generation parameters/results, and standalone `VideoMuxParams`/`VideoMuxArgv` composition that never executes ffmpeg. +- Model-free acceptance gates for docs, tests, ABI/layout/signature/exports, native fixtures, sanitizers, link modes, packages, downstream consumers, publish dry-run, and exact MSRV. + +### Compatibility + +- Both Rust crates are lockstep version `0.0.2`; `vllm-cpp` depends on exactly `vllm-cpp-sys =0.0.2`. +- ABI-10 system libraries are incompatible. ABI 17 has no task query, so task-specific owners cannot introspect a checkpoint at load time and native wrong-task `InvalidArgument` errors remain authoritative. + +### Known limitations + +- Native Linux x86_64 CPU is the supported runtime family. The native `vllm_server_main` entry point remains raw-only; the safe crate exposes no tokenizer, task query, raw handle, HTTP-server wrapper, ffmpeg execution, or general process execution. +- Video generation writes filesystem artifacts and may leave partial output. No successful Rust MiniMax-H3 generation fixture is claimed for this candidate. +- Exact-candidate prepared-Qwen inference/sanitizers, native-only TSan, Miri, Linux ARM64, Apple ARM64, Vulkan, CUDA/CUTLASS/Triton, Metal/MLX, and accelerator runtime lanes were not run. + ## [0.0.1] - 2026-08-22 ### Added diff --git a/Cargo.lock b/Cargo.lock index a0f1e6c..3aeb195 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1057,7 +1057,7 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" [[package]] name = "vllm-cpp" -version = "0.0.1" +version = "0.0.2" dependencies = [ "clap", "hf-hub", @@ -1068,7 +1068,7 @@ dependencies = [ [[package]] name = "vllm-cpp-sys" -version = "0.0.1" +version = "0.0.2" dependencies = [ "cmake", ] diff --git a/Cargo.toml b/Cargo.toml index 92e5343..ddb026f 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,11 +3,11 @@ members = ["vllm-cpp", "vllm-cpp-sys"] resolver = "2" [workspace.package] -version = "0.0.1" +version = "0.0.2" edition = "2021" license = "MIT OR Apache-2.0" repository = "https://github.com/querymt/vllm-cpp-rs" rust-version = "1.85" [workspace.dependencies] -vllm-cpp-sys = { version = "=0.0.1", path = "vllm-cpp-sys", default-features = false } +vllm-cpp-sys = { version = "=0.0.2", path = "vllm-cpp-sys", default-features = false } diff --git a/README.md b/README.md index cd7d798..a46a14b 100644 --- a/README.md +++ b/README.md @@ -7,11 +7,11 @@ Rust bindings for [vllm.cpp](https://github.com/mudler/vllm.cpp), organized as: ## Status -> **Work in progress:** these bindings track a pinned vllm.cpp revision and are still catching up with upstream development. APIs, supported features, and backend behavior may lag behind the latest vllm.cpp release; check the pinned commit and compatibility notes before adopting them. +> **Work in progress:** these bindings track a pinned vllm.cpp release and may lag later upstream APIs or backend behavior. Check the exact native identity and support boundary before adoption. -The safe crate provides a cloneable engine API for local model loading, blocking completion and streaming, non-blocking concurrent requests, structured output, and raw-JSON chat. It also provides an always-available synchronous Hugging Face resolver for standalone GGUF files and runtime-complete sparse Safetensors snapshots, plus a Clap-based interactive chat example using those APIs. An optional `serde` feature adds `serde_json::Value` chat helpers. The sys crate provides checked-in generated FFI declarations with C/Rust layout checks and coverage for all 19 exported C symbols. +The safe crate covers text completion, streaming, raw-JSON and optional serde chat, concurrent requests, structured output, custom logits processing, pre-tokenized completion, transcription, embeddings, video generation, and standalone video-mux argument composition. It also includes a synchronous Hugging Face resolver and text-focused examples. The sys crate exposes the complete 35-function stable C boundary at ABI 17 with checked-in generated bindings and C/Rust conformance checks. -Linux x86_64 CPU builds support bundled static, bundled dynamic, system static, and system dynamic linking. Bundled CPU builds also target Linux aarch64 and Apple ARM64. Experimental bundled builds expose Linux x86_64/aarch64 build configuration for CUDA, external CUTLASS, Triton AOT, and Vulkan, plus Apple ARM64 Metal and external MLX configuration. Accelerator features are build integration surfaces, not runtime-support claims. vllm.cpp is pinned at `34aedfbe8ed9779697905541a62e2160ccfd9c05`, which exposes C ABI version 10. +The native source is independently pinned to vllm.cpp tag `v0.0.2`, commit `7020de93652ca920424a10ac5255b34810dd2f24`. Native Linux x86_64 CPU is the supported runtime target, including bundled/system and static/dynamic linking. Linux ARM64, Apple ARM64, CUDA, external CUTLASS, Triton AOT, Vulkan, Metal, and external MLX are configured build or optional validation surfaces; they are not accelerator runtime-support claims. ## Prerequisites @@ -43,17 +43,21 @@ git submodule update --init --recursive ## Safe API -The packaged [`vllm-cpp` guide](vllm-cpp/README.md) covers local and Hugging Face model resolution, safe ownership, callbacks, concurrency, features, link modes, and deployment. The [`vllm-cpp-sys` guide](vllm-cpp-sys/README.md) documents the raw ABI and native build boundary. +The packaged [`vllm-cpp` guide](vllm-cpp/README.md) covers the full API, ownership, callbacks, filesystem effects, link modes, and deployment. The [`vllm-cpp-sys` guide](vllm-cpp-sys/README.md) documents the unsafe ABI and native build boundary. -`Engine::load` accepts a native-compatible model directory or standalone GGUF. `HuggingFaceModel` synchronously resolves into the normal Hugging Face cache before engine construction, defaulting to the Hub's mutable `main` revision; `.revision(...)` can pin a branch, tag, or commit. GGUF mode selects one safe root file. Safetensors mode pins downloads to repository metadata's commit SHA and retrieves only native runtime requirements: root configuration/tokenizer files and either unsharded weights or an index plus all root shards. Every inference example accepts a bare or explicit local path and both Hub artifact forms with optional `--revision`. Cached downloads are reused. Retrieval does not prove model/backend compatibility. +`Engine::load` accepts a native-compatible model directory or standalone GGUF. `HuggingFaceModel` synchronously resolves a GGUF or runtime-complete sparse Safetensors snapshot into the normal cache; retrieval does not prove native model, task, or backend compatibility. -`EngineBuilder` owns model settings and converts them to temporary C strings only for the load call. `SamplingParams` owns stop strings, structured constraints, and optional `Send + Sync` custom logits processors. Processor panics are contained before the C boundary and reported through Rust errors; processor-backed generation must be bounded because ABI v10 has no callback abort channel. Processor state remains registered only through the blocking call or asynchronous request lifetime; stale native invocations after cleanup become no-ops. `version()` copies the linked native diagnostic version string. Completion and chat strings are copied into Rust values before the matching native free function runs. +Text `Engine` is a cloneable `Send + Sync` RAII owner. It provides blocking completion, streaming, chat, structured output, custom logits processing, and `complete_tokens`, plus non-blocking `Request` submission. Requests retain an `Arc` to the engine, are `Send` but not `Sync`, and expose completion probes, cancellation, waiting, and copied diagnostics. Callback panics are contained before crossing C; asynchronous callbacks run on a native delivery thread, and callback-thread wait/free is rejected or deferred under the stable lifecycle contract. -`Engine` is `Clone + Send + Sync`; each `Request` retains the shared engine until native callback delivery has joined. A request is `Send` but deliberately not `Sync`. `submit` returns before generation finishes, and `Request` provides `is_done`, idempotent `cancel`, `wait`, and copied `native_error` diagnostics. `wait` classifies completion as `Completed`, `StoppedByCallback`, or `Cancelled`; an explicit asynchronous `Stop` is classified as `StoppedByCallback` even when returned for the terminal event. +`EngineBuilder` starts from `vllm_model_params_default()` and overlays only explicitly selected settings, preserving native helper defaults such as `block_size=32`, `max_num_seqs=32`, and `gpu_memory_utilization=0.92`. Text/task [`Device`](vllm-cpp/src/params.rs) numbering is `Auto=0`, `Cpu=1`, and `Cuda=2`; explicit CUDA never silently falls back. KV sizing precedence is `num_blocks` over `kv_cache_memory_bytes` over `gpu_memory_utilization` and its native profile/fallback path. -All streaming callbacks receive copied UTF-8 deltas. Blocking callbacks may borrow stack data; their panics are caught before the C boundary and resumed only after the native call returns. Asynchronous callbacks must be `Send + 'static`, run on a native delivery thread, and report panic as `Error::CallbackPanicked` from `wait`. Waiting for or freeing a request from its own callback thread is prohibited by ABI v10: `wait` returns `Error::RequestCallbackThread`, while drop transfers cleanup to a prestarted reaper that owns the request, callback, and engine until native free/cancel/join completes. Chat methods accept raw OpenAI-compatible request JSON; enable `serde` for `serde_json::Value` request and response helpers. `SchedulerPolicy::Priority` selects the native queue. Raw and serde chat request JSON can carry a `priority` field that the native OpenAI-compatible path parses and submits. Direct completion, completion streaming, and `Request` submissions currently default to priority zero and tie by arrival; caller-selected priorities for those direct APIs require a future C ABI/API change. +`complete_tokens` borrows caller-provided token IDs for one blocking call. Its output capacity limits only reported/copied IDs, not native generation; `truncated` compares the copied count with native completion metadata. `include_completion = false` suppresses the Rust metadata copy, while the hidden native metadata request still occurs so truncation remains accurate. -See [the examples guide](vllm-cpp/examples/README.md) for ordinary Linux and optional Nix setup, commands for every example, and the interactive chat CLI's local/Hub model forms and generation options. Release-facing changes are recorded in the [changelog](CHANGELOG.md), and maintainers use the manual [release process](RELEASING.md). +`TranscriptionEngine` and `EmbeddingEngine` are separate, non-cloneable, conservatively thread-local RAII owners. Their operations take `&mut self`, block, borrow call inputs, and copy results into Rust-owned values; embeddings are row-major and preserve input order. ABI 17 has no task-introspection function, so none of the task-specific owners can prove a checkpoint's task at load time. Native wrong-task `InvalidArgument` diagnostics remain authoritative. + +`VideoEngine` separately owns a MiniMax-H3 checkpoint set and performs exclusive blocking generation. Video device numbering is `Cpu=0` and `Cuda=1`, with no `Auto`; explicit CUDA never falls back. Generation writes frame/audio artifacts, may leave stale or partial output, and trusts caller paths without sandboxing. `VideoMuxParams` and `VideoMuxArgv` only compose ordered `OsString` argument boundaries. This crate never executes ffmpeg, an HTTP server, or another process; `vllm_server_main` remains available only through the raw crate. + +The high-level crate intentionally adds no tokenizer, task-query, raw-handle, process-execution, or HTTP-server wrapper. See [the examples guide](vllm-cpp/examples/README.md) for the intentionally text-focused binaries. Release-facing changes are recorded in the [changelog](CHANGELOG.md), and maintainers use the manual [release process](RELEASING.md). ## Build and Test @@ -72,59 +76,41 @@ Set `CMAKE_BUILD_PARALLEL_LEVEL` to control native parallelism. The default bund ## Experimental Backend Builds -Backend features are bundled-only and mutually exclusive with `system`; CUDA and Vulkan are also mutually exclusive. CUDA/CUTLASS/Triton/Vulkan target Linux x86_64/aarch64, while Metal/MLX require exact `aarch64-apple-darwin`. Backend features do not enable `bundled`: normal default-feature commands may use `--features cuda`, while `--no-default-features` callers must include it explicitly, for example `--features bundled,cuda`. Use a fresh `CARGO_TARGET_DIR` for every backend and link mode. +Backend features are bundled-only and mutually exclusive with `system`; CUDA and Vulkan also conflict. CUDA/CUTLASS/Triton/Vulkan target Linux x86_64/aarch64, while Metal/MLX require exact `aarch64-apple-darwin`. Features do not imply runtime support and do not enable `bundled` for `--no-default-features` callers. Use a fresh `CARGO_TARGET_DIR` for each backend/link combination. -- `cuda` requires `VLLM_CPP_CUDA_ARCHITECTURES` equal to `80`, `86`, `87`, `89`, `90a`, `100a`, `103a`, `110`, `120a`, `121a`, or `120a;121a`. Leave this variable unset when `cuda` is disabled, including CPU and system builds. -- `cuda-cutlass` implies `cuda`, requires an explicit canonical `VLLM_CPP_CUTLASS_DIR` containing CUTLASS >=4.5.0, disables fetching, and rejects `103a` and `110`. Plain CUDA uses a nonexistent sentinel CUTLASS root so an ambient checkout cannot alter the build. -- `triton-aot` implies `cuda`, enables only checked-in AOT artifacts for one of `80`, `86`, `89`, `90a`, `100a`, or `121a`, and forces regeneration off. -- `vulkan` uses packaged Khronos headers and checked-in SPIR-V. It does not link a Vulkan SDK library; the native library opens the runtime loader dynamically. -- `metal` enables the native Metal backend on Apple ARM64 and links Apple's `Metal` and `Foundation` frameworks. Its MSL is compiled at runtime. -- `mlx` implies `metal` and requires canonical `MLX_ROOT` containing `include/mlx/array.h`, `lib/libmlx.dylib`, and `lib/mlx.metallib`. MLX remains an external dependency: Cargo neither fetches nor packages it and emits no machine-local rpath. +- `cuda` requires `VLLM_CPP_CUDA_ARCHITECTURES` equal to `80`, `86`, `87`, `89`, `90a`, `100a`, `103a`, `110`, `120a`, `121a`, or `120a;121a`. +- `cuda-cutlass` requires caller-provided CUTLASS >=4.5.0, disables fetching, and rejects `103a` and `110`. +- `triton-aot` packages and embeds all six checked-in AOT trees: `sm_80`, `sm_86`, `sm_89`, `sm_90a`, `sm_100a`, and `sm_121a`. Runtime dispatch selects only an exact SM match; other accepted CUDA targets, including `87`, `103a`, `110`, and `120a`, retain the portable C++/CUDA fallback. Regeneration remains disabled for consumer builds. +- `vulkan` uses packaged headers and SPIR-V and opens the runtime loader dynamically. +- `metal` links the Apple Metal and Foundation frameworks on Apple ARM64. +- `mlx` implies `metal` and requires an external `MLX_ROOT`; Cargo neither fetches nor packages MLX and emits no machine-local rpath. -For example: +A configuration example, not candidate runtime evidence: ```console nix develop .#cuda -VLLM_CPP_CUDA_ARCHITECTURES=120a \ +VLLM_CPP_CUDA_ARCHITECTURES=80 \ CARGO_TARGET_DIR=target/cuda-static \ cargo build --locked --release --features cuda -VLLM_CPP_CUDA_ARCHITECTURES=120a \ - CARGO_TARGET_DIR=target/cuda-dynamic \ - cargo build --locked --release --features cuda,dynamic-link - -nix develop .#vulkan -CARGO_TARGET_DIR=target/vulkan-static cargo build --locked --release --features vulkan - -# Apple ARM64 only -CARGO_TARGET_DIR=target/metal-static cargo build --locked --release --features metal -MLX_ROOT=/absolute/path/to/mlx CARGO_TARGET_DIR=target/mlx-static \ - cargo build --locked --release --features mlx ``` -Static CUDA links the exact `cudart`, `cublasLt`, and, for Triton, CUDA driver locations selected by CMake. Static Apple builds link `libc++`; Metal adds the `Metal` and `Foundation` frameworks, while MLX adds its canonical `lib` search path before `dylib=mlx`. Dynamic builds rely on the shared native library's transitive dependencies instead of repeating them through Cargo. Deploy `libvllm.so`/`libvllm.dylib` and optional toolkit/MLX libraries through normal loader paths. +Static CUDA links toolkit libraries selected by CMake. Static Apple builds link `libc++` and the selected frameworks/providers. Dynamic builds rely on the native shared library's transitive dependencies. Deploy `libvllm.so`/`libvllm.dylib` and optional toolkit/MLX libraries through normal loader paths. -Compilation does not establish runtime correctness. Known native evidence blockers remain: CUDA teardown can SIGSEGV after otherwise successful tests; CUDA bf16 testing has a numerical tolerance failure; CUTLASS concurrent output differs from the non-concurrent path; Vulkan attention/model runtime is incomplete; and MLX is an external, numerically distinct provider without release-lane model evidence. No accelerator runtime support is claimed here. +Compilation does not establish runtime correctness. CUDA/CUTLASS/Triton, Vulkan, Metal, and MLX remain experimental build surfaces; no accelerator runtime support is claimed. -## Test Model and Sanitizers +## Optional Model and Sanitizer Recipes -Model-backed tests use Apache-2.0 `Qwen/Qwen3-0.6B` at pinned revision `c1899de289a04d12100db370d81485cdf75e47ca`. Explicitly resolve its complete Safetensors snapshot into the standard Hugging Face cache, then run exactly 18 blocking and request-lifecycle model tests serially, including choice and JSON-Schema structured-output enforcement: +The required Linux x86_64 CPU candidate gate is model-free and uses committed native fixtures. An optional prepared-model lane uses Apache-2.0 `Qwen/Qwen3-0.6B` at revision `c1899de289a04d12100db370d81485cdf75e47ca`; the model is never included in repository or crate packages. No ordinary test or instrumentation recipe downloads a model implicitly. ```console model=$(just setup-test-model) VLLM_CPP_TEST_MODEL="$model" \ cargo test --locked -p vllm-cpp --release --test qwen3 -- --test-threads=1 -``` - -`just setup-test-model` is the only explicit test-fixture acquisition step. It uses `HuggingFaceModel` with the immutable revision above, honors normal `HF_HOME` and Hugging Face authentication, reuses the standard cache, and prints the resolved directory. The approximately 1.5 GB model is not included in repository or crate packages. Ordinary tests, sanitizers, and TSan never resolve or download models: `VLLM_CPP_TEST_MODEL` must name an externally prepared model directory. Model-backed tests skip with an explanatory message when it is unset; when set, tests and instrumentation recipes require it to be a directory. - -AddressSanitizer, UndefinedBehaviorSanitizer, and leak detection run the full safe/request/model suites with native instrumentation. The Linux x86_64 GCC ThreadSanitizer lane runs selected request lifecycle tests individually and instruments native C++ only; it does not claim race coverage for Rust or the Rust standard library. Callback-thread self-drop remains in the normal and ASan/leak suites because its handoff uses uninstrumented Rust synchronization. - -```console just sanitizers "$model" just tsan "$model" ``` -`VLLM_CPP_SANITIZE` is a bundled-build test input. System mode rejects it because Cargo cannot infer whether an externally built native library carries matching instrumentation. +These commands are optional evidence and are recorded only when rerun against the exact candidate. The TSan recipe instruments native C++ only and does not claim Rust standard-library race coverage. `VLLM_CPP_SANITIZE` is a bundled-build test input; system mode rejects it. ## Link Modes @@ -149,13 +135,15 @@ The package gate validates deterministic inventories for both crates, package me `just publish-dry-run` performs a sys-then-safe workspace packaging dry-run without uploading; it uses `--no-verify` to avoid the pre-publication registry cycle. As required by [RELEASING.md](RELEASING.md), after `vllm-cpp-sys` is available from crates.io, run the full `cargo publish -p vllm-cpp --locked --dry-run` verification before publishing the safe crate. -## Platform and Backend Validation +## Validation Boundary + +Mandatory candidate evidence is Linux x86_64 CPU and model-free: formatting, lint, docs, workspace tests, generated-binding/header/layout/signature/ABI/exact-export checks, all four CPU link modes, native C API fixtures, ASan/UBSan/leak checks over committed fixtures, package extraction/downstream checks, publication dry-run, and exact MSRV validation. -The manual `platforms` workflow provides exact Rust 1.85.0, Linux ARM64 CPU, Apple ARM64 CPU, Apple ARM64 Metal compile/link, and Mesa llvmpipe Vulkan jobs without duplicating ordinary Linux x86_64 CPU CI. The Vulkan job requires a real llvmpipe device and `storageBuffer16BitAccess`, then runs native backend/op gates; its scope is backend/op checking, not attention or model-inference support. The hosted Metal job checks compile/link only, not runtime correctness. +Prepared Qwen inference/sanitizers, native-only TSan, successful Rust MiniMax-H3 generation, Miri, Linux ARM64, Apple ARM64, Vulkan, CUDA/CUTLASS/Triton, Metal/MLX, and accelerator runtime are optional or deferred lanes. The manual platform workflows are configuration, not exact-candidate evidence unless separately dispatched and recorded. ## Support -The supported runtime target is native Linux x86_64 CPU. Maintainer tests cover the four bundled/system static/dynamic CPU link modes plus bundled blocking and concurrent request inference with the pinned Qwen fixture. Sanitizer evidence covers native ASan/UBSan/leak detection and selected native-only GCC TSan lifecycle paths as described above. The manual Linux ARM64 and Apple ARM64 CPU jobs are configured for model-free build/test coverage. CUDA/CUTLASS/Triton/Vulkan/Metal/MLX remain experimental surfaces with the evidence boundaries and limitations listed above; CPU is the only supported runtime family. +Native Linux x86_64 CPU is the supported runtime family. All accelerator features remain experimental build/configuration surfaces. Cross-platform compile or workflow configuration alone does not establish runtime support. ## Licensing and Affiliation diff --git a/RELEASING.md b/RELEASING.md index cae7c3b..d4f0ab9 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -1,59 +1,90 @@ # Releasing -Releases are prepared and published manually. The repository does not tag, publish, or create a GitHub release automatically. A successful local candidate or dry-run is not a release; crates.io publication is an irreversible registry action. +Releases are prepared and published manually. A candidate pass or dry-run does not authorize a Git tag, crates.io upload, merge, push, or GitHub release. Publication is an irreversible registry action and requires separate maintainer authorization. ## Prepare a candidate -1. Start from the reviewed release commit and verify the branch, `HEAD`, and intended remote identity. -2. Require a clean worktree and index, including initialized submodules: +1. Start from the exact independently reviewed release commit. Require a clean root worktree and index and a clean, detached native submodule: ```console test -z "$(git status --short --untracked-files=all)" git diff --quiet git diff --cached --quiet git submodule status --recursive - test "$(git -C vllm-cpp-sys/vllm.cpp rev-parse HEAD)" = 34aedfbe8ed9779697905541a62e2160ccfd9c05 + test "$(git -C vllm-cpp-sys/vllm.cpp symbolic-ref -q HEAD || true)" = "" test -z "$(git -C vllm-cpp-sys/vllm.cpp status --short --untracked-files=all)" ``` -3. Confirm the release version in the workspace manifest, both normalized package manifests, `Cargo.lock`, and the pinned native `project(vllm_cpp VERSION ...)` declaration. Both crates and the native CMake project must use the same version, and `vllm-cpp` must depend on exactly that `vllm-cpp-sys` version. The CMake project declaration is the native release version authority; do not derive the crate version from `git describe` or the nearest native tag. -4. Confirm the native gitlink is `34aedfbe8ed9779697905541a62e2160ccfd9c05`, `VLLM_ABI_VERSION` is 10 in the pinned public C header and checked-in bindings, and generated bindings have no drift. -5. Move the relevant entries from `Unreleased` to a dated version section. Describe only validated support; preserve known backend/runtime blockers. -6. Audit dual-license metadata, crate license files, `NOTICE`, `THIRD_PARTY.md`, imported license texts, and the package inventory. Do not publish models, fixtures, build output, caches, SDKs, external CUTLASS trees, or repository-local paths. -7. Run the complete maintainer validation from the pinned development shell. At minimum run formatting, lint, model-free tests, docs, sys conformance, all CPU link modes, package extraction/downstream tests, and the exact MSRV gate. +2. Verify native identity directly, never with `git describe`: -## Inspect and dry-run + ```console + native=vllm-cpp-sys/vllm.cpp + test "$(git rev-parse HEAD:vllm-cpp-sys/vllm.cpp)" = 7020de93652ca920424a10ac5255b34810dd2f24 + test "$(git -C "$native" rev-parse HEAD)" = 7020de93652ca920424a10ac5255b34810dd2f24 + test "$(git -C "$native" rev-parse 'refs/tags/v0.0.2^{}')" = 7020de93652ca920424a10ac5255b34810dd2f24 + test "$(git -C "$native" rev-parse 'HEAD^{tree}')" = 28df226f0ef9924e67d563c3bef4712d0e628c5a + ``` + +3. Use `cargo metadata --locked --no-deps --format-version 1` and normalized package manifests to require both Rust crates at `0.0.2` and the safe dependency requirement exactly `=0.0.2`. Confirm both local package records in `Cargo.lock`. Separately parse native `project(vllm_cpp VERSION 0.0.2 LANGUAGES CXX)` from `CMakeLists.txt`. Rust and native versions are independent release identities that happen to both be `0.0.2` here; equality is not a universal policy. +4. Require `VLLM_ABI_VERSION == 17` in the pinned header and generated bindings and exactly 35 stable C functions. Run binding-drift, C11/C++20 header, every C/Rust layout and signature, runtime ABI, all-function link, and exact dynamic-export checks. ABI-10 system libraries are incompatible. +5. Keep `Unreleased` empty above the dated release entry. Describe only validated support and preserve known limitations. +6. Audit root and crate dual-license metadata, `LICENSE-MIT`, `LICENSE-APACHE`, native `LICENSE`/`NOTICE`, `THIRD_PARTY.md`, and every imported license text against the exact package inventories. Reject models, media fixtures, build output, caches, SDKs, external CUTLASS trees, internal records, and repository-local paths. + +## Validate -Build fresh archives; do not trust old files under `target/package`: +Run the mandatory Linux x86_64 CPU gates from the pinned shell: ```console -cargo package -p vllm-cpp-sys --locked --list -cargo package -p vllm-cpp --locked --list -just package-test +env -u VLLM_CPP_TEST_MODEL nix develop -c just ci +nix develop .#msrv -c just msrv +cargo check --locked --workspace --all-targets --features vllm-cpp/serde +RUSTDOCFLAGS='-D warnings' cargo doc --locked --workspace --no-deps --features vllm-cpp/serde +git diff --check +``` + +`just ci` includes formatting, warnings-denied lint/docs, model-free workspace tests, generated bindings and ABI conformance, four CPU link modes, the native C API fixture, model-free ASan/UBSan/leak checks, package/extracted/downstream validation, and no-upload publish dry-run. Run the exact MSRV gate separately so stable-toolchain success cannot mask it. + +Prepared-Qwen inference/sanitizers, native-only TSan, successful Rust MiniMax-H3 generation, Miri, Linux ARM64, Apple ARM64, Vulkan, CUDA/CUTLASS/Triton, Metal/MLX, and accelerator runtime are optional or deferred. Record one only when it ran against the exact candidate; configured workflows and older results are not candidate evidence. + +## Inspect packages + +Use two fresh, separate `CARGO_TARGET_DIR` values; do not trust existing `target/package` files: + +```console +CARGO_NET_OFFLINE=true CARGO_TARGET_DIR=target/package-release-a just package-test +CARGO_NET_OFFLINE=true CARGO_TARGET_DIR=target/package-release-b just package-test just publish-dry-run ``` -Inspect both sorted inventories and extracted normalized `Cargo.toml` files. Confirm the packages contain their READMEs, dual licenses, notices and provenance where applicable, source, tests, examples, and every required native/backend input. Confirm extracted builds and independent downstream consumers pass offline and that the safe consumer resolves the extracted sys crate rather than the workspace. +For both runs, retain: + +- sorted `cargo package --list` inventories and archive basenames; +- normalized `Cargo.toml` manifests, including exact `vllm-cpp-sys =0.0.2`; +- file counts and unpacked/compressed sizes, enforcing sys limits of 1,400 files, 40 MiB unpacked, and 6 MiB compressed and safe limits of 40 files, 512 KiB unpacked, and 128 KiB compressed; +- complete license/notice/provenance inventories and checks that current READMEs, source, tests, examples, and required native/backend inputs are present; +- SHA-256 for both archives from each run and successful extracted/offline builds, all four extracted sys link modes, and independent sys and safe downstream consumers. + +Require identical sorted inventories and semantically identical normalized manifests between runs. Compare archive hashes and record both outcomes, but do not assume byte identity: Cargo-generated `.cargo_vcs_info.json`, archive metadata, or timestamps may differ. Claim reproducible bytes only when both hashes actually match and the comparison explains the metadata involved. -`just publish-dry-run` uses Cargo's workspace dry-run in sys-first order without uploading. The preceding package gate provides the full extracted/offline verification; the workspace command uses `--no-verify` to avoid a registry-resolution cycle before the exact sys version exists on crates.io. After sys is published, run `cargo publish -p vllm-cpp --locked --dry-run` and require its full verification to pass before the safe upload. +`just publish-dry-run` uses Cargo's sys-first workspace order with `--no-verify` and never uploads. It cannot provide the safe crate's full registry-resolution verification before exact sys `0.0.2` is available from crates.io. Check both crate-version slots are available before any future upload; do not reserve or publish them during candidate preparation. ## Publish -Only an authorized maintainer should publish, from the exact reviewed commit with a clean worktree and index. Verify crates.io credentials and ownership, then publish one crate at a time: +Only a separately authorized maintainer may publish from the exact reviewed commit with a clean root and detached submodule. Publish sys first: ```console cargo publish -p vllm-cpp-sys --locked -# Wait until crates.io serves the exact sys version. +# Wait until crates.io serves exact vllm-cpp-sys 0.0.2. cargo publish -p vllm-cpp --locked --dry-run cargo publish -p vllm-cpp --locked ``` -The sys crate must be accepted and available from crates.io before publishing the safe crate because the safe archive declares an exact registry dependency. After both uploads, verify the registry metadata, package contents, docs.rs results, and a clean downstream build. Create the Git tag and release notes only for the exact published commit and version. +The full safe dry-run must resolve registry sys `0.0.2` before the safe upload. After both uploads, verify registry metadata, archives, docs.rs, licenses, and a clean downstream build. Create a tag and GitHub release only after separate authorization and only for the exact published commit. ## Abort and recovery -- Before an upload succeeds, abort on any mismatch, validation failure, unexpected file, dirty state, changed lockfile, changed native pin/ABI, or inaccurate release note. Fix the issue in a separately reviewed commit and restart the checklist. -- After crates.io accepts a version, that version cannot be replaced or deleted. Never rebuild a different archive under the same version. -- If the sys crate publishes but the safe crate fails, stop and diagnose. Retry the unchanged safe version only when the failure is transient and the exact reviewed archive remains valid; otherwise prepare a new coordinated version. -- Yank a published version only when leaving it selectable would harm users. Yanking prevents new resolution but does not erase the crate, undo existing lockfiles, or make the version reusable. Record the reason publicly and publish a corrected new version. -- Never use `cargo yank` as an ordinary abort mechanism, and never publish merely to test credentials or packaging. +- Before upload, abort on any mismatch, failed gate, unexpected file, dirty state, changed lockfile, native identity/ABI/export drift, inaccurate support statement, or unavailable version. Fix it in a separately reviewed commit and restart. +- Accepted crates.io versions cannot be replaced or deleted. Never rebuild different bytes under the same version. +- If sys publishes but safe fails, stop and diagnose. Retry unchanged safe bytes only for a transient failure; otherwise prepare a new coordinated version. +- Yank only when leaving a version selectable would harm users. Yanking does not erase the crate or make the version reusable. +- Never use `cargo yank` as an ordinary abort mechanism or publish merely to test credentials or packaging. diff --git a/vllm-cpp/Cargo.toml b/vllm-cpp/Cargo.toml index 652c576..e0e52fb 100644 --- a/vllm-cpp/Cargo.toml +++ b/vllm-cpp/Cargo.toml @@ -5,7 +5,7 @@ edition.workspace = true license.workspace = true repository.workspace = true rust-version.workspace = true -description = "Safe model inference, streaming, chat, and concurrent requests for vllm.cpp" +description = "Safe vllm.cpp completion, streaming, chat, requests, token completion, transcription, embeddings, video, and mux argv APIs" readme = "README.md" documentation = "https://docs.rs/vllm-cpp" keywords = ["llm", "inference", "vllm", "vllm-cpp", "machine-learning"] diff --git a/vllm-cpp/README.md b/vllm-cpp/README.md index a06fc04..ab6f431 100644 --- a/vllm-cpp/README.md +++ b/vllm-cpp/README.md @@ -1,6 +1,6 @@ # vllm-cpp -Safe Rust API for the stable [vllm.cpp](https://github.com/mudler/vllm.cpp) C boundary. The crate owns native resources, checks ABI compatibility before model loading, and provides blocking completion/streaming/chat plus concurrent requests. Use `vllm-cpp-sys` directly only when an application needs the unsafe raw ABI. +Safe Rust API for the stable [vllm.cpp](https://github.com/mudler/vllm.cpp) C boundary. The crate owns native resources, checks ABI compatibility before loading, and covers text completion/streaming/chat/requests, pre-tokenized completion, transcription, embeddings, video generation, and mux argument composition. Use `vllm-cpp-sys` only for the unsafe raw ABI. ## Quick use @@ -41,15 +41,26 @@ In the repository checkout, `just setup-test-model` explicitly resolves `Qwen/Qw ## API and ownership -- `EngineBuilder` configures and loads a model. `Engine` is `Clone + Send + Sync`; clones share one reference-counted native engine. -- `SamplingParams` owns stop strings, structured constraints, and an optional `Send + Sync` custom logits processor. The processor receives generated token IDs and a mutable logits row each decode step; panics are contained and returned as `Error::LogitsProcessorPanicked`. Processor-backed generation must have a finite `max_tokens` bound because ABI v10 cannot abort from that callback. Processor state remains registered only through the blocking call or asynchronous request lifetime; stale native invocations after cleanup become no-ops. -- Completion, chat, error, and stream text is copied into Rust-owned values before native storage is released or reused. -- Blocking `complete`, `complete_stream`, `chat_json`, and `chat_stream_json` calls keep borrowed callbacks alive only for the call. Callback panics are caught before crossing C and resumed after the native call returns. -- `Engine::submit` returns a `Request` before generation finishes. A request retains its engine and callback until native free/join completes, is `Send`, and is deliberately not `Sync`. -- Asynchronous callbacks run on a native delivery thread and must be `Send + 'static`. `wait` reports callback panics as `Error::CallbackPanicked`; waiting or freeing from that same callback thread is prohibited by ABI v10, so callback-thread drop transfers cleanup to a prestarted reaper. -- Dropping a live request cancels and joins it. `cancel` is idempotent, `wait` reports the request outcome, and `native_error` copies the request-owned diagnostic after completion into an owned Rust `String`; the native storage remains valid until the request is dropped or freed. +| Surface | Owner and contract | +|---|---| +| Text | `EngineBuilder` loads `Engine`; `Engine` is `Clone + Send + Sync` and shares one `Arc`-retained native handle. Completion, streaming, chat, structured output, logits processors, and `complete_tokens` are blocking; `submit` returns a non-blocking `Request`. | +| Tokens | `complete_tokens` borrows prompt IDs for the call and returns `TokenCompletion`. Output capacity limits reported IDs, not generation. `truncated` comes from native completion metadata; `include_completion = false` suppresses only the Rust metadata copy. | +| Transcription | `TranscriptionEngine` is non-cloneable and neither `Send` nor `Sync`; `transcribe(&mut self, TranscriptionInput)` blocks, borrows a WAV path or PCM slice, and returns Rust-owned optional text and token IDs. | +| Embeddings | `EmbeddingEngine` has the same conservative thread-local/exclusive contract. `embed(&mut self, ...)` blocks and returns a Rust-owned row-major `EmbeddingResult` preserving input order. | +| Video | `VideoEngineBuilder` loads a separate checkpoint set. Non-cloneable `VideoEngine` is neither `Send` nor `Sync`; `generate(&mut self, ...)` is blocking and exclusive and returns Rust-owned paths, dimensions, rates, counts, and mux argv. | +| Mux | `compose_video_mux_argv(&VideoMuxParams)` returns owned `VideoMuxArgv` argument boundaries. It performs no filesystem I/O and never locates or executes ffmpeg. | + +`EngineBuilder` obtains native defaults first and overlays only explicit Rust settings. Unset values preserve the helper defaults, including `block_size=32`, `max_num_seqs=32`, and `gpu_memory_utilization=0.92`. `Device` uses `Auto=0`, `Cpu=1`, and `Cuda=2`; explicit CUDA never falls back. KV-memory precedence is `num_blocks > kv_cache_memory_bytes > gpu_memory_utilization` and its native profile/fallback path. Video uses independent `VideoDevice` numbering, `Cpu=0` and `Cuda=1`, with no automatic mode. + +All native completion, chat, transcription, embedding, video, stream, argv, and diagnostic data is copied into Rust-owned values before native storage is released or reused. Blocking calls borrow their input storage only until return. Blocking callback panics are contained before C and resumed afterward. Custom logits processors are `Send + Sync`, may run on native worker threads, and report contained panic through `Error::LogitsProcessorPanicked`. + +`Engine::submit` returns a `Request` that retains its engine, callback, and optional logits processor until native free/join completes. A request is `Send` but not `Sync`; dropping a live request cancels and joins it. Asynchronous callbacks are `Send + 'static` and run on a native delivery thread. Callback-thread wait/free is rejected or delegated to the cleanup reaper under the stable ABI lifecycle contract. + +ABI 17 exposes no task query. Loading `Engine`, `TranscriptionEngine`, or `EmbeddingEngine` does not inspect or infer the checkpoint task; native wrong-task `Error::InvalidArgument` diagnostics remain authoritative. Video model format, partition, task, media, and capability checks are also native authority. + +Video generation creates or truncates frame/audio artifacts, may leave stale files or partial output after failure, and has no cancellation, timeout, quota, rollback, or sandbox. Paths are trusted as supplied; Rust does not canonicalize, confine, reject symlinks, or clean outputs. On Unix, paths and mux arguments preserve raw bytes through `OsString`; non-Unix native conversion requires valid UTF-8. Mux arguments remain separate process arguments and must not be shell-joined. This crate does not execute ffmpeg, a server, or any other process. -`SchedulerPolicy::Priority` selects the native priority queue. Raw and serde chat request JSON can carry a `priority` field that the native OpenAI-compatible path parses and submits. Direct completion, completion streaming, and `Request` submissions currently default to priority zero and tie by arrival; caller-selected priorities for those direct APIs require a future C ABI/API change. +`SchedulerPolicy::Priority` selects the native queue. Raw and serde chat JSON may carry `priority`; direct completion, streaming, and `Request` submission currently use priority zero and tie by arrival. ## Features and linking @@ -72,13 +83,13 @@ Hugging Face resolution is not a Cargo feature: synchronous `hf-hub` support is ## ABI and deployment -This crate is tied to the exact same `vllm-cpp-sys` crate version and the pinned vllm.cpp commit `34aedfbe8ed9779697905541a62e2160ccfd9c05`. Model loading requires exact C ABI version 10 before any versioned struct crosses FFI. `version()` copies the linked library's diagnostic native version string, while `abi_version()` remains the compatibility authority. A system library must implement the same ABI; the consumer build checks for its header, while maintainer conformance tests check layout and symbols. +This crate uses the exact matching `vllm-cpp-sys =0.0.2`, whose bundled source is independently pinned to vllm.cpp tag `v0.0.2`, commit `7020de93652ca920424a10ac5255b34810dd2f24`. The checked-in bindings cover all 35 stable C functions at ABI 17. Loading checks exact runtime ABI equality before any versioned struct crosses FFI. `version()` is diagnostic; `abi_version()` is the compatibility authority. A system library must implement ABI 17 with matching layouts and signatures; ABI-10 libraries are incompatible. Static bundled builds include the native archive in the application link. Dynamic bundled or system builds do not deploy `libvllm.so`/`libvllm.dylib`: install it and its backend/toolkit dependencies in a loader-visible location using `LD_LIBRARY_PATH`, `DYLD_LIBRARY_PATH`, rpath supplied by the application, or the system loader configuration. System mode uses `VLLM_CPP_ROOT`; `VLLM_CPP_LIB_DIR` can choose a nonstandard library directory. System static linking also requires the matching `libblake3_vendored.a` through `VLLM_CPP_BLAKE3_LIB_DIR` or the selected vllm library directory. ## Support boundary -The supported runtime tier is native Linux x86_64 CPU, covering bundled/system and static/dynamic link modes. Linux ARM64 and Apple ARM64 CPU have manual hosted jobs configured for model-free build/test coverage. CUDA, external CUTLASS, Triton AOT, Vulkan, Metal, and MLX are experimental build/configuration surfaces. The hosted Metal job checks compilation/linking only; Vulkan software-device gates check backend/ops, not attention or model inference; MLX remains external and has no release-lane model/runtime evidence. Known native blockers include CUDA teardown failure, CUDA bf16 numerical tolerance failure, CUTLASS concurrent-output differences, incomplete Vulkan attention/model runtime, and MLX's numerically distinct provider behavior. CPU remains the only supported runtime family. +The supported runtime tier is native Linux x86_64 CPU, covering bundled/system and static/dynamic link modes. Mandatory candidate evidence is model-free and includes native fixtures, sanitizers, exact link/export checks, packages, and downstream consumers. Prepared-Qwen inference/sanitizers, TSan, successful Rust MiniMax-H3 generation, Miri, Linux ARM64, Apple ARM64, Vulkan, CUDA/CUTLASS/Triton, Metal/MLX, and accelerator runtime are optional or deferred unless rerun against the exact candidate. Accelerator features remain experimental build/configuration surfaces; CPU is the only supported runtime family. See the repository [changelog](https://github.com/querymt/vllm-cpp-rs/blob/main/CHANGELOG.md), [release process](https://github.com/querymt/vllm-cpp-rs/blob/main/RELEASING.md), and [root support details](https://github.com/querymt/vllm-cpp-rs#support) for the current release boundary. diff --git a/vllm-cpp/examples/README.md b/vllm-cpp/examples/README.md index cca4186..94b7933 100644 --- a/vllm-cpp/examples/README.md +++ b/vllm-cpp/examples/README.md @@ -1,6 +1,6 @@ # examples -five examples exercise the safe `vllm-cpp` api: four use fixed prompts and settings, and `chat` is an interactive command-line application. one additional maintainer utility prepares the pinned test fixture: +five intentionally text-focused examples exercise the safe `vllm-cpp` api: four use fixed prompts and settings, and `chat` is an interactive command-line application. one additional maintainer utility prepares the optional pinned text fixture: | example | behavior | |---|---| @@ -11,6 +11,19 @@ five examples exercise the safe `vllm-cpp` api: four use fixed prompts and setti | [`structured`](structured.rs) | extracts a fixed weather report under a JSON Schema | | [`setup_test_model`](setup_test_model.rs) | resolves the pinned Qwen test fixture for `just setup-test-model`; not a general inference CLI | +The library also supports pre-tokenized completion without adding another binary: + +```rust,no_run +use vllm_cpp::{Engine, SamplingParams}; + +let engine = Engine::load("/path/to/model")?; +let result = engine.complete_tokens(&[1, 2, 3], &SamplingParams::greedy(), 32, true)?; +println!("{:?} truncated={}", result.token_ids, result.truncated); +# Ok::<(), vllm_cpp::Error>(()) +``` + +`max_output_tokens` limits copied IDs, not native generation; truncation is determined from native completion metadata. Transcription, embedding, and video need task-specific checkpoints, media, and substantial resources, so their rustdoc/API documentation is authoritative rather than unvalidated runnable examples. Mux composition returns argument boundaries only and never executes ffmpeg. + The four fixed examples (`complete`, `stream`, `concurrent`, and `structured`) accept the same manual model-source forms: ```console @@ -91,7 +104,7 @@ CMAKE_GENERATOR=Ninja cargo run --locked --release -p vllm-cpp --example chat -- `--prompt/-p ` and `--file/-f ` are mutually exclusive optional first user messages; prompt files must be UTF-8. `--system ` adds a retained system message. Generation options are `--max-tokens` (default `256`, maximum `2147483647`), `--temperature` (default `0.7`), `--top-p` (default `1`), `--top-k` (default `0`), `--min-p` (default `0`), and optional `--seed`. Responses stream by default; `--no-stream` selects blocking `Engine::chat_json` output. The CLI maintains the complete user/assistant history, submits it on each turn, and prints only assistant content rather than raw response JSON. Native role, reasoning, tool-call, and finish metadata is ignored; a valid response with no content is stored as an empty assistant message. -At `user>` enter `/clear` to retain the system message while clearing other history, or `/quit`/`/exit` to stop. EOF also exits cleanly. A per-turn request or response error is reported as `chat: `, the attempted turn is removed from history, and the prompt continues; terminal input/output errors and model startup failures still exit. This high-level example intentionally exposes only controls supported by the existing chat request and engine APIs; it does not add low-level model, device, context, batch, token, or timing controls from other runtimes. +At `user>` enter `/clear` to retain the system message while clearing other history, or `/quit`/`/exit` to stop. EOF also exits cleanly. A per-turn request or response error is reported as `chat: `, the attempted turn is removed from history, and the prompt continues; terminal input/output errors and model startup failures still exit. This high-level example intentionally exposes a small chat-oriented subset. The library provides device, memory, pre-tokenized completion, and additional model controls that this CLI deliberately omits; the example does not attempt to mirror every native or library option. ## optional nix shell @@ -108,7 +121,7 @@ replace `complete` with another example and its arguments from the table. plain `cuda` is the baseline accelerator feature. read the root [experimental backend build details](https://github.com/querymt/vllm-cpp-rs#experimental-backend-builds) before using it: cuda is a bundled linux experimental/build-only integration surface, and successful compilation does not guarantee model inference or backend correctness. a non-nix build needs a compatible cuda toolkit and driver installation in addition to the ordinary linux prerequisites. -set `VLLM_CPP_CUDA_ARCHITECTURES` to an architecture supported by this crate and the target gpu, and use a fresh `CARGO_TARGET_DIR` for the backend/link combination. for example, the tested RTX 5080 setup uses `120a`; do not use that value for unrelated hardware: +set `VLLM_CPP_CUDA_ARCHITECTURES` to an architecture supported by this crate and the target GPU, and use a fresh `CARGO_TARGET_DIR` for the backend/link combination. The value below is only a configuration example; it is not candidate runtime evidence and must match the actual target: ```console arch=120a @@ -131,7 +144,7 @@ CMAKE_GENERATOR=Ninja \ other accelerator features have stricter limits: - `cuda-cutlass` is an optional cuda variant; follow the [exact external cutlass prerequisites and known blockers](https://github.com/querymt/vllm-cpp-rs#experimental-backend-builds) before selecting it. -- `triton-aot` requires a target architecture with matching checked-in artifacts; `120a` is not supported by those artifacts. +- `triton-aot` packages all six checked-in AOT trees (`80`, `86`, `89`, `90a`, `100a`, and `121a`). Runtime dispatch uses only an exact-SM tree; accepted targets without one, including `120a`, retain the portable CUDA fallback. - `vulkan` is currently for backend build/testing work. its model attention path is absent, so it cannot run these full-model examples. ## troubleshooting diff --git a/vllm-cpp/src/request.rs b/vllm-cpp/src/request.rs index 91b02f1..aac005e 100644 --- a/vllm-cpp/src/request.rs +++ b/vllm-cpp/src/request.rs @@ -27,7 +27,7 @@ pub enum RequestOutcome { Completed, /// The Rust callback returned [`StreamControl::Stop`]. /// - /// ABI v10 treats this as an explicit stop even for the terminal event. + /// ABI 17 treats this as an explicit stop even for the terminal event. StoppedByCallback, /// Rust requested cancellation before completion was observable. Cancelled, @@ -282,7 +282,7 @@ impl AsyncCallbackState { } fn record_delivery_thread(&self) { - // ABI v10 invokes user_data only from this request's single library-owned + // ABI 17 invokes user_data only from this request's single library-owned // delivery thread. Retain its ID through cleanup instead of marking only // an active trampoline, so every possible Rust re-entry from that thread // remains ineligible for synchronous wait/free.