Add Windows ARM64 and Qualcomm Hexagon NPU support - #40
Open
roozbehid wants to merge 2 commits into
Open
Conversation
Owner
|
Okay, this is really nice. Your results with the fast attention path suggest that there might be a possibility for fast attention on other platforms if we solve the precision issues, that would make runs much faster. I'll try to merge ASAP. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds native Windows ARM64 support and makes the TRELLIS.2 pipeline usable with
partial-coverage accelerators such as the Qualcomm Hexagon NPU.
The implementation adds:
--backend HTP|Vulkan|CPU.--vulkan-fallback.Companion ggml correctness/build PR: ggml-org/ggml#1601
This Trellis PR deliberately leaves the ggml submodule revision unchanged. The companion
ggml changes are required for the tested Qualcomm HTP F16 and Adreno Q4 Vulkan behavior
until they are incorporated into the submodule.
Backend selection
On Snapdragon systems, Vulkan and Hexagon can both register as GPU-class devices.
Selecting based only on reported memory is therefore ambiguous.
The new option chooses the backend explicitly:
trellis-devices --initlists registered backends and verifies that each device caninitialize.
Multi-backend scheduling
The Hexagon backend intentionally supports only a subset of ggml operations. Graphs now
run through
ggml_backend_schedwhen HTP is selected, allowing unsupported nodes toexecute on fallback backends.
Default HTP order:
Optional mixed-backend order:
Enable the latter with:
The equivalent
TRELLIS_VULKAN_FALLBACK=1environment variable remains available forexisting automation.
FlashAttention policy
HTP defaults to F16 K/V with fast accumulation. Other backends retain BF16 K/V with F32
accumulation by default.
Overrides:
--fa-fast: force the fast F16 path--fa-f32: force BF16 K/V with F32 accumulation--no-fa: disable FlashAttentionA 12-step res512 correctness gate produced:
This is a 3.53x end-to-end improvement. Deterministic renders preserved the same
silhouette, topology, and material placement.
HTP + Vulkan fallback result
A matching res512, one-step, geometry-only A/B produced:
The mixed configuration was 37.9% faster. Both runs produced byte-identical voxel PLY
output. Vulkan fallback remains opt-in pending testing across more inputs and full sampler
counts.
Windows ARM64 build
The build helper imports the Visual Studio ARM64 environment and configures clang for
arm64-pc-windows-msvc.The trimmed Hexagon SDK described in the official llama.cpp Windows Snapdragon
documentation is supported. The script reads the Hexagon Tools path from
hexagon_sdk.jsonand supplies the compatibility values required by the current ggmlHexagon CMake integration.
Windows still requires properly signed HTP Ops libraries and catalog files before the NPU
driver will load them.
Validation
Tested on:
Build validation after rebasing onto current
pwilkin/main:trellis-devicesratherthan asserting
Scope and cleanup
The branch contains no port diary, machine-specific status file, temporary probe
executable, hard-coded developer path, or ggml submodule change.