AX Engine is a Mac-first LLM inference runtime for Apple Silicon. Install
with Homebrew, download a curated model, and serve OpenAI-compatible endpoints
locally — with a repo-owned MLX path for Gemma, Qwen, and GLM, first-class MTP,
multi-model serving with exact-prompt prefix reuse, and peer-backed benchmarks
against mlx-lm, MTPLX, and OMLX.
NVIDIA/CUDA fleet serving lives in AX Serving. AX Engine remains the local Apple Silicon runtime and no longer ships the former vLLM or TensorRT provider bridges, runtime package, container, or CUDA qualification scripts.
Browse AutomatosX serve-ready chat / coding / embedding snapshots in the AutomatosX model collection on Hugging Face. Additional native families (GLM 4.7 Flash, Nemotron Omni, Unlimited-OCR, Whisper, MiniCPM-V, and others) are documented under Supported Models.
Requires macOS 26 (Tahoe)+ on Apple Silicon (M2 or newer). For compact single models (Qwen 3.5 9B 4-bit preferred; 6-bit also fits), 16 GB unified memory is enough — including base Mac mini M4 16 GB. Prefer 4-bit for headroom. For multi-model serving, longer contexts, and larger packs (27B/35B class), plan on 32 GB+ (64 GB recommended).
- Faster speculative decode — AutomatosX chat snapshots bundle their MTP sidecar or assistant weights, so one standard download is serve-ready; AX speeds up 14 of 15 exact same-package 6-bit MTP rows (1.68× geometric mean; 0.88×–2.56× range). In the newest Qwen3.6 peer campaign, The latest AXQ MTP campaign compares AX Engine with MTPLX 2.9.0 and OMLX 0.6.4 on Qwen3.8/Qwen3.6; Gemma4 assistant-MTP peer lanes are unsupported where the peers reject the AXQ contract
- Faster single-model serving — on the path users actually measure (streaming OpenAI chat), AX Engine 6.13.1 leads a peer MLX serving engine 0.4.3 in 8/8 Qwen 3.6 decode cells, with +12.9% matrix-wide geometric-mean throughput and ~21–24% MoE wins (2026-08-06, M5 Max) — see Performance
- Strong direct decode on Apple Silicon — the fresh v6.13.3 snapshot wins
30/30 comparable decode cells against a separate-run
mlx_lm0.31.3 reference (+4.6% geometric mean) - Multi-model on one process — keep a scoped set of Qwen 3.5/3.6,
Qwen3-Coder-Next, Gemma 4, and embedding models resident (
load_mode=add), route by requestmodel(chat + embeddings together), with fair Metal turn arbitration, memory preflight, and optional idle eviction. Exact-prompt prefix reuse is the S1 differentiator: official dual-model campaign (Qwen stream + Gemma 13.8k prefill) clears all locked gates at 5.03× median throughput vs a multi-process peer MLX server — see Performance and Server: Multi-model - You own the stack you serve — AX runs the MLX graph, KV/runtime, and
OpenAI-compatible server for supported Gemma / Qwen / GLM (and other direct
families);
mlx-lmandllama.cppstay optional compatibility adapters - Engine-owned scheduling and KV — continuous batched decode
(certification-gated), chunked prefill, preempt-and-recompute, and a paged
KV ledger with cross-request prefix sharing run inside the engine, with a
published
ax_runtime_*saturation contract for fleet routers — see Scheduling and KV runtime - Native media and speech — image/video chat, mixed image+audio reasoning, OCR, and Whisper transcription/translation run through repo-owned MLX graphs with capability-gated OpenAI endpoints (checkpoint-authoritative; see media table below)
- Clear fleet boundary — AX Engine owns local Apple Silicon inference; AX Serving owns fleet orchestration and NVIDIA/CUDA workers
- Claims you can audit — public rows ship with checked-in artifacts (route, model snapshot, sampler, accept rate, provenance)
brew install defai-digital/tap/ax-engine
ax-engine doctorHomebrew is the primary install path for the CLI, server, and bench tools.
The self-contained release formula installs the release's pinned
libmlx.dylib, libjaccl.dylib, and precompiled mlx.metallib; it does not
build MLX from source. End users therefore do not need Python, Xcode, or the
Metal Toolchain.
Use the wheel for Python applications that import ax_engine, optional Python
integrations, or systems where Homebrew is unavailable. Install it in a virtual
environment:
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install --upgrade "ax-engine[download]>=7.3.0,<8"
ax-engine doctorThe wheel also exposes ax-engine and ax-engine-server and bundles the bench
binary used by diagnostics. If both Homebrew and pip are installed, an active
virtual environment normally wins on PATH; use which -a ax-engine to see
every copy and prefer one installation channel in each shell. See
Getting Started for the full channel comparison and
troubleshooting.
Option A — interactive TUI (pick a model, download, serve, chat):
ax-engine tui
Option B — serve Qwen 3.8 27B AXQ 6-bit MTP, then request from another
terminal. qwen3.8-27b:axq is the pinned AutomatosX 6-bit MTP pack (same
checkpoint as ax-qwen3.8-27b). The command reuses the cached snapshot when
present and downloads it otherwise. Listen defaults to 127.0.0.1:31418:
ax-engine serve qwen3.8-27b:axq
curl http://127.0.0.1:31418/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model":"qwen3.8-27b","messages":[{"role":"user","content":"Say hello in one sentence."}],"max_tokens":64}'Option C — coding model (resolve + serve):
ax-engine serve ax-qwen3-coder-next --port 31418Python wheel, source builds, and troubleshooting: Getting Started.
ax-engine download --list and the TUI expose the curated public
AutomatosX model collection
only — not every community MLX weight. Qwen 3.8 27B AXQ (6-bit MTP default),
Qwen 3.6, Qwen 3.5, and Gemma 4 variants published there (plain 4-bit/6-bit,
QAT, OptiQ, AXQ where available) are first-class serve targets. Other native families (for example GLM 4.7 Flash, Nemotron
Omni, Unlimited-OCR, Whisper, MiniCPM-V) use the repo-owned runtime via serve
aliases, presets, or manual model directories; they are not all AutomatosX-managed
packages. Full matrix:
Supported Models.
Qwen 3.8 Super-class (2.4T) is experimental only, not a production target.
Those packs can technically load through the SSD expert-stream path
(--stream-experts, default auto), but local inference is too slow even at
2-bit to recommend or certify. Start local serving on Qwen 3.8 27B AXQ
6-bit MTP (qwen3.8-27b:axq).
Recommended starting packages (serve-ready, match published benches):
| Goal | Alias / family | Why |
|---|---|---|
| Default dense chat + MTP | qwen3.8-27b:axq (pinned AXQ 6-bit MTP) |
Production-size Qwen 3.8 27B; AutomatosX AXQ 6-bit with MTP sidecar |
| Fastest MoE chat + MTP | ax-qwen3.6-35b-a3b (4-bit or 6-bit MTP) |
Strongest serving and MTP peer decode rows |
| Dense chat + MTP (3.6) | ax-qwen3.6-27b (6-bit MTP preferred) |
High same-package MTP speedup; solid serving |
| AXQ evaluation candidate | qwen3.6-27b:axq (pinned 6-bit) |
Qwen 3.6 27B AXQ candidate; explicit until its checkpoint certification gates pass |
| Vision MoE Instruct AXQ | ax-qwen3-vl-30b / ax-qwen3-vl-30b-4bit |
Qwen3-VL 30B-A3B Instruct AXQ packs; candidate, no MTP |
| Holo3 GUI-agent AXQ | holo3-35b / holo3-35b:axq |
Qwen3.5-class 35B-A3B MoE; Tier 1 certified text path; no MTP |
| Ornith coding AXQ | ornith-35b / ornith-35b:axq |
Qwen3.5-class 35B-A3B MoE coding agent; Tier 1 AXQ; no MTP |
| GPT-OSS AXQ | gpt-oss-20b:axq / gpt-oss-120b:axq |
AutomatosX AXQ; bare aliases stay mlx-community MXFP4-Q4 |
| Nemotron 3 Nano AXQ | nemotron-3-nano / nemotron-3-nano:axq |
nemotron_h 30B-A3B; development AXQ |
| Muse-Glimmer image-text AXQ | muse-glimmer-30b / muse-glimmer-30b:axq |
Meta dense 30B image-text agent; ATEM chat; development AXQ; no MTP |
| Multimodal chat + MTP | ax-gemma4-12b / 26B / 31B Assistant-MTP |
Image/audio/video + assistant draft package |
| Coding agent | ax-qwen3-coder-next |
Coding-focused MoE; multi-model friendly |
| Embeddings | ax-embeddinggemma-300m or Qwen3-Embedding aliases |
Batched ingest scale in full results |
Repositories ending in -MTP or -Assistant-MTP already contain the prepared
sidecar or assistant artifacts and model-manifest.json. Download them with
the standard flow; do not run download-mtp afterward.
Native multimodal and speech support is checkpoint-authoritative: AX advertises
only modalities whose required tower tensors are present in
model-manifest.json.
| Model family | Inputs | Native API surface | Current scope |
|---|---|---|---|
| Qwen3-VL; visual Qwen3.5; Qwen 3.6 | Image, video | Chat/generate | Conv3D visual patches, full ViT/merger, MRoPE, multi-image/video ordering; Qwen 3.6 27B image/video smoke-tested on M3 Max |
| Standard Gemma 4 E4B/26B/31B | Image, video | Chat/generate | Full bidirectional ViT, 2-D RoPE, spatial pooling, checkpoint standardization, and vision projection; catalogued sizes are E4B/26B/31B. E2B still loads from an explicit directory. Conformer audio is not yet native |
| Gemma 4 unified 12B | Image, audio, video | Chat/generate | Encoder-free image/audio connector and sampled per-frame video path; requires the unified connector roles |
| MiniCPM-V 4.6 | One or more images | Chat/generate | Dynamic SigLIP grid, mid-tower merger, OCR/document prompts |
| Nemotron 3 Nano Omni | Image, audio, or both | Chat/generate | RADIO vision plus Parakeet audio with ordered mixed-media spans |
| Unlimited-OCR | Image | Native processed-input runtime; delegated OpenAI OCR profile | Full prefill KV is protected; only generated-token KV uses the decode ring |
| Whisper large-v3-turbo | Audio | /v1/audio/transcriptions, /v1/audio/translations, Rust SDK |
WAV/MP3 to 16 kHz; multilingual transcribe/translate; text-generation routes fail closed |
GLM-OCR is not supported.
Download helpers inspect the source tensor index and automatically rebuild
older packaged manifests that omitted a declared Qwen or Gemma vision tower.
For an existing manual model directory, run
ax-engine-bench generate-manifest --force /path/to/model.
The default Hugging Face cache layout is
models--AutomatosX--<repository>/snapshots/<revision>. Use the shorter
ax-* aliases shown by ax-engine download --list; for example:
ax-engine serve qwen3.8-27b:axq
ax-engine serve qwen3.8-27b:axq --offline # require the pinned 6-bit MTP cacheAliases, hardware sizing, and legacy MTP packaging targets: Supported Models · Qwen 3.6 27B AXQ certification · Hardware FAQ · CLI.
AX Engine 6.13.5 completed an 8-hour endurance test with 8.87 hours of continuous measured runtime for the pinned Qwen 3.6 27B AXQ 6-bit model on a 64 GB M4 Pro Mac mini: 437/437 requests succeeded, the owned server did not restart, and no retained-RSS growth, swap, lifecycle-drain failure, or performance guardrail breach was observed. The operator stopped this run to add deeper vLLM-style leak attribution before restarting the full 72-hour qualification, so it is short-duration evidence, not a 72-hour pass. Read the 8-hour endurance report or use the reusable AXQ endurance utility and detached launch procedure to verify another local AXQ package.
One process can keep several allowlisted models loaded and route each
request by model (OpenAI, gRPC, Ollama, Anthropic). Add mode is limited to
Qwen 3.5 9B, Qwen 3.6 27B/35B, Qwen3-Coder-Next, Gemma 4 12B/26B/31B, and the
EmbeddingGemma 300M / Qwen3-Embedding 0.6B–8B embedding models (chat +
embeddings from one process); AutomatosX AX- package names resolve to the
same targets. Each model owns its own session and scheduler while a process
arbiter fair-rotates Metal turns (no fused cross-model batch).
# After a first model is already serving on :31418
curl -s http://127.0.0.1:31418/v1/model/load -H 'content-type: application/json' -d '{
"model_id": "gemma-4-12b-it",
"model_path": "/path/to/gemma-4-12b-artifacts",
"load_mode": "add",
"make_default": false
}'
curl -s http://127.0.0.1:31418/v1/chat/completions -H 'content-type: application/json' -d '{
"model": "gemma-4-12b-it",
"messages": [{"role": "user", "content": "Hi"}],
"max_tokens": 32
}'Full contract (load/unload, memory preflight, idle eviction, metrics labels): Server: Multi-model serving · Supported Models.
Token-level scheduling is owned by the engine, not a gateway: each step the scheduler builds a decode-first, token-budgeted batch with chunked prefill and mixed prefill+decode routes, backed by a paged logical KV ledger. Full design: Scheduler · KV Cache · Serving Invariants.
- Continuous batched decode — structurally eligible decode requests share
one batched forward (default on;
AX_MLX_BATCHED_DECODE=0is the kill switch) behind a fail-closed bit-exact certification gate; host-sampled batching is a separate opt-in - Paged KV ledger with prefix sharing — per-request block tables, ref-counted cross-request prefix reuse, tiered eviction, and an optional disk-durable prefix cache that survives restarts
- Pressure handling — KV memory-pressure throttling, preempt-and-recompute
(newest in-flight prefill only, never decode), and server admission control
(global and per-model concurrency caps →
429) - Fleet telemetry contract —
/metricspublishes versionedax_runtime_*saturation series (KV utilization, queue depth, batch headroom, TTFT p95, decode tok/s, error rate) that AX Serving and other routers consume for node selection; token scheduling itself stays on-box
Current dense batched-decode ceiling. On the strict M5 Max projection probe, the default Shared policy reaches 328.9 aggregate tok/s at B=8 (4.01× its B=1 throughput), versus 102.6 tok/s / 1.25× for the RowExact fallback. The paired Shared/RowExact ratio is 3.20× with five of five wins and identical full-cohort greedy hashes. This is a dense Llama-3.1-8B, 32-token-prefill microbenchmark—not an end-to-end serving, long-prompt, or MoE claim. Production continuous-batching claims still require matching serving evidence—see Batched decode ceiling and Long Context claim boundaries.
Why people try AX Engine: faster serving and speculative decode on Apple Silicon, plus multi-model that peers usually need multiple processes for. Results are session-separated — do not mix multi-model (S1), single-client serving, MTP, direct, or embedding rows, and do not mix M3 Max vs M5 Max absolute tok/s.
| Session | Peers | Headline | Host / when |
|---|---|---|---|
| Single-client serving | AX Engine · peer MLX serving engine 0.4.3 | 8/8 decode wins · MoE ~21–24% faster · GM decode +12.9% | M5 Max · 2026-08-06 · AX 6.13.1 |
| Multi-model (S1) | AX one process · multi-process peer MLX server | All locked gates · thr 5.03× | M5 Max · 2026-08-06 |
| MTP generation | AX · MTPLX · OMLX | Latest AXQ campaign: AX exact MTP vs MTPLX 2.9.0 and OMLX 0.6.4 on Qwen3.8/Qwen3.6; Gemma4 assistant-MTP peer lanes are unsupported | M5 Max · 2026-08-31 · AX 7.2.0 |
| Direct generation | AX · mlx-lm | AX 30/30 decode wins vs separate-run mlx-lm · +4.6% GM | M5 Max · 2026-08-07 · separate runs |
| Embeddings | AX · mlx-lm / mlx-embeddings | Qwen 18/18 wins, +1.56% GM; EmbeddingGemma 6/6, +7.99% GM | M5 Max · 2026-08-07 · same-session paired |
| Qwen3.8 direct refresh | AX · mlx-lm | AXQ 6-bit AX direct decode 34.04–34.59 tok/s across p128–p2048 | M5 Max 128 GB · 2026-08-30 · v7.2.0 refresh |
| Qwen3.8 same-package MTP refresh | AX direct · AX exact sampled MTP | 1.32–1.36× MTP/direct decode across the three workload suites | M5 Max 128 GB · 2026-08-30 · v7.2.0 refresh |
Full tables, charts, and methodology: Performance Results · Benchmarks · Claim boundaries.
Important
Prefill/TTFT peer rows require the same resolved libmlx on both sides.
Some Homebrew or low-deployment-target MLX builds omit M5 GEMM paths and look
~3–4× slower. Details:
Performance Results.
Streaming OpenAI /v1/chat/completions — the comparison users run when they
open a server and time chat. AX Engine 6.13.1 vs peer MLX serving engine
0.4.3, Apple M5 Max 128 GB, Qwen 3.6 27B / 35B-A3B at 4-bit and 6-bit,
~512 and ~2k prompt targets, 256 gen tokens, temperature 0.
| Model | p512 decode (AX / peer) | p2048 decode (AX / peer) |
|---|---|---|
| Qwen3.6 27B 4-bit | 34.40 / 32.32 (+6.4%) | 33.88 / 32.01 (+5.9%) |
| Qwen3.6 27B 6-bit | 24.59 / 23.94 (+2.7%) | 23.97 / 23.35 (+2.7%) |
| Qwen3.6 35B-A3B 4-bit | 159.10 / 129.06 (+23.3%) | 156.89 / 126.60 (+23.9%) |
| Qwen3.6 35B-A3B 6-bit | 128.79 / 106.67 (+20.7%) | 126.90 / 105.04 (+20.8%) |
AX wins 8 of 8 decode cells; geometric-mean decode advantage is 12.9% (dense 27B 4.4%, 35B-A3B MoE 22.2%). Effective prefill and TTFT split 4/8 and are roughly neutral in the matrix-wide geometric mean, so they are not headline wins. Full prefill/TTFT tables, methodology, provenance, and caveats:
Serving peer detail · Performance Results: serving
One AX process co-serves Qwen interactive stream + Gemma 13.8k prefill with exact-prompt prefix reuse against a multi-process peer MLX server (2026-08-06, M5 Max). All locked gates pass every rep; median throughput ratio 5.03× (TTFT and stream-gap p95 also win). Detail: S1 results.
This is the current AXQ campaign on df-macbookpro-m5 (Apple M5 Max, 128 GB,
macOS 26.6.2). It uses the repository flappy prompt suite, four prompt
cases, 256 generated tokens, greedy sampling, two warmups, five measured
repetitions, three-second cooldowns, and disabled prefix-cache/n-gram
stacking. Values are the median decode throughput over 20 measured runs.
The requested Qwen3.6 25B and Gemma4 35B labels do not correspond to published AutomatosX AXQ packs. The measured pack mappings are Qwen3.6 27B and Gemma4 31B, respectively. Exact raw artifacts and runtime caveats are in the campaign result.
| AXQ model | AX Engine 7.2.0 | MTPLX 2.9.0 | OMLX 0.6.4 | Readout |
|---|---|---|---|---|
| Qwen3.8 27B 6-bit | 45.05 tok/s | 46.68 tok/s | 37.04 tok/s | AX exact MTP; MTPLX accepted 100% of drafted tokens; OMLX text-only staging |
| Qwen3.6 27B 6-bit (requested 25B) | 45.53 tok/s | 46.86 tok/s | 38.39 tok/s | AX exact MTP; MTPLX accepted 99.51% of drafted tokens; OMLX text-only staging |
| Gemma4 31B 6-bit (requested 35B) | 22.49 tok/s | unsupported | unsupported | AX assistant-MTP depth 2; peers rejected the AXQ vision/assistant contract |
| Gemma4 26B-A4B 6-bit | 112.70 tok/s | unsupported | unsupported | AX assistant-MTP depth 2; one AX telemetry row was incomplete |
The OMLX Qwen rows use BatchedEngine with mtp_enabled and an imported AXQ
MTP sidecar; its VLM loader rejected this AXQ vision-key layout, so these are
text-only OMLX measurements. MTPLX rejected both Gemma packs because they
declare an MTP layer but do not ship MTPLX-compatible root MTP weights. No
unsupported lane is replaced with direct-mode throughput.
Per-runtime raw artifacts and the full contract: AXQ MTP peer campaign.
The default Qwen3.8 package was rerun on df-macbookpro-m5 (Apple M5 Max,
128 GB) using
AutomatosX/AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP,
pinned to snapshot 3e290738e96972307c6aeb9934ab170ca0eae1c1. Direct rows use
the mlx_lm.benchmark-compatible random-token contract with 128 generated
tokens; MTP rows use the real flappy, long_code, and python_modules_long
workload suites with 1,000 generated tokens.
| Prompt / workload | AX direct decode | AX direct prefill | AX direct TTFT |
|---|---|---|---|
| 128 prompt tokens | 34.59 tok/s | 617.3 tok/s | 207.4 ms |
| 512 prompt tokens | 34.45 tok/s | 876.2 tok/s | 584.3 ms |
| 2,048 prompt tokens | 34.04 tok/s | 953.8 tok/s | 2,147.1 ms |
| Workload suite | AX direct decode | AX exact sampled MTP decode | MTP/direct |
|---|---|---|---|
flappy |
32.06 tok/s | 43.27 tok/s | 1.35× |
long_code |
31.97 tok/s | 43.38 tok/s | 1.36× |
python_modules_long |
32.10 tok/s | 42.50 tok/s | 1.32× |
All 11 MTP rows were correctness-eligible, with no direct fallback or
optimistic verification steps recorded. The refresh was captured from the
v7.2.0 release binary at commit 3cea9def, but the benchmark host recorded
tracked runtime changes in its worktree; treat these numbers as refresh
evidence pending a clean-build rerun. Raw artifacts:
mlx-lm reference,
AX direct,
AX MTP.
The fresh v6.13.3 direct snapshot improves over the dated, tracked-dirty
v6.12.0 benchmark snapshot by 2.7% decode and 25.3% prefill on the
36-cell geometric mean while lowering TTFT by 20.2%. Against a fresh but
separate-run mlx_lm 0.31.3 snapshot, AX wins all 30 comparable decode cells
(+4.6% geometric mean), while prefill is 10.6% lower and TTFT is
11.9% higher. This is cross-run evidence, not a same-session peer
benchmark or a clean release-to-release comparison.
The fresh same-session embedding matrix is positive but not one uniform-sized win: Qwen3-Embedding wins all 18 sustained-ingest shapes with a +1.56% geometric mean (near parity to modestly faster), while EmbeddingGemma wins all six shapes with +7.99% geometric mean throughput.
Non-speculative decode/prefill/TTFT (Gemma 4 and Qwen 3.6 box plots from fresh
separate-run AX and mlx_lm snapshots), embedding ingest scale,
DiffusionGemma, and historical composites live under docs so this README
stays on the numbers that decide “is AX faster for me?”:
| Topic | Where |
|---|---|
| Direct: Gemma 4 / Qwen 3.6 charts | Performance Results: Direct |
| Embeddings (Qwen3 + EmbeddingGemma) | Performance Results: Embeddings |
| Gemma 4 12B case study | v6.8.2 case study |
| How to interpret a row | Performance |
| Reproduce a session | Benchmarks |
How to read headline metrics
- Decode (tok/s, higher is better) is the main interactive metric.
- Serving and MTP sessions answer different questions; pick the table that matches how you run the engine.
- Prefill / TTFT are cold-prompt cost; AX does not claim a matrix-wide prefill lead on every retained historical direct overlay.
Most clients use the OpenAI-compatible HTTP server. Python also has an in-process session API.
| SDK | Docs |
|---|---|
| Rust | docs/sdk/rust.md |
| Python | docs/sdk/python.md |
| JavaScript / TypeScript | docs/sdk/javascript.md |
| Go | docs/sdk/go.md |
| Ruby | docs/sdk/ruby.md |
| Swift | docs/sdk/swift.md |
| Mojo (experimental) | docs/sdk/mojo.md |
ax-engine serve is the normal entrypoint (see Quick Start). Default listen is
127.0.0.1:31418 (not AX Serving’s 18080). Port map, LAN bind, and
Serving vs Engine:
curl http://127.0.0.1:31418/v1/runtimeAuth, streaming, embeddings, Ollama-shaped routes: Server · API Compatibility · OpenClaw. Fleet / NVIDIA serving: AX Serving.
| Need | Read |
|---|---|
| Docs hub | docs/README.md |
| Install and first request | Getting Started |
| Ports, bind host, Engine vs Serving | Ports |
| Models and MTP packages | Supported Models · MTP Docs |
| Hardware / FAQ | FAQ |
| Full performance tables | Performance Results |
| Serving peer (newest) | Serving peer detail |
| Reproduce benchmarks | Benchmarks |
| Server / API / SDKs | Server · API · OpenClaw · SDKs |
| Fleet / NVIDIA (AX Serving) | AX Serving |
| Architecture | Architecture |
| Scheduler / KV internals | Scheduler · KV Cache · Serving Invariants |
cargo build --workspace
cargo test --quiet
cargo clippy --all-targets --all-features -- -D warnings
cargo fmt --check
maturin develop
python -m unittest discover -s python/tests -vCrate layout and conventions: AGENTS.md · Architecture.
- Qwen3.5 long-prompt prefill can trail upstream MLX references on longer prompts
- Raw HuggingFace / mlx-community snapshots load directly:
model-manifest.jsonis auto-generated fromconfig.json+ safetensors headers on first load and weight sanitization is auto-detected, somlx_lm.convertis not required - N-gram acceleration is workload-dependent, not a raw kernel speedup
- NVIDIA/CUDA serving requires the separate AX Serving product
Details: FAQ limitations.
Issues, wishlist items, reproducible benchmarks, and docs feedback are welcome. Unsolicited code PRs for runtime, kernel, scheduler, or performance paths are generally not accepted — open an issue first. See CONTRIBUTING.md.
- Website: automatosx.com
- Discord: Join us
- Email: enquiry@defai.digital
AX Engine is grateful to the open-source foundations, benchmarking peers, and community contributors listed in Acknowledgments.
Apache License, Version 2.0. See LICENSE.
Copyright (c) 2026 DEFAI Private Limited