Qualcomm AI Engine Direct - Gemma4 Text model enablement

[DannyYuyang-quic]

Summary

Gemma4 Text model enablement

Test plan

python examples/qualcomm/oss_scripts/llama/llama.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --temperature 0 --model_mode kv--max_seq_len 1024 --prefill_ar_len 128 --decoder_model gemma4-e2b --prompt "I would like to learn python, could you teach me with a simple example?" --calib_tasks wikitext --calib_limit 1 --embedding-quantize '4,32'

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20979

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 16 Awaiting Approval

As of commit 507a79d with merge base 5bfbe1d ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

• Build Aarch64 Linux Wheels (gh)
• Build documentation (gh)
• Build Linux Wheels (gh)
• Build macOS Wheels (gh)
• Build Presets (gh)
• Build Windows Wheels (gh)
• Lint (gh)
• periodic (gh)
• pull (gh)
• Test ARM Backend (gh)
• Test CoreML Backend (gh)
• Test QNN Backend (gh)
• Test QNN Windows MSVC build (gh)
• Test Vulkan Backend (gh)
• Test WebGPU Backend (gh)
• Test XNNPACK Backend (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[DannyYuyang-quic]

@psiddh Hi,
I've put up two draft PRs for Gemma4 text model and would like your view on which approach fits the long-term direction.

• This PR(Qualcomm AI Engine Direct - Gemma4 Text model enablement #20979) keeps Gemma4 standalone as its own module, matching how Gemma4 is currently structured on mainline (i.e. not going through the shared/unified flow).

• Qualcomm AI Engine Direct - Gemma4 Text model enablement (Unified Architecture) #20980 migrates it onto the existing static_llama unified architecture, reusing shared model components like the decoder, feedforward and attention, etc.

We noticed mainline's Gemma4 doesn't go through the usual unified flow, even its convert_weights and config key names differ (doesn't match ModelArgs). Maybe because Gemma4's architecture diverges so much from the existing models (YOCO KV-sharing, per-layer head_dim, zero-pad RoPE, per-layer embeddings), and that's why mainline kept it standalone rather than folding it into the unified flow? That's what left us unsure which convention I should follow.

So my question is: for models that differ significantly from the current unified flow, do we prefer keeping them as standalone implementations? Or is the long-term direction to make all models standalone?

I'm not sure if I'm thinking about this the right way, but if this hasn't been decided yet, we preference for Gemma 4 on the QNN backend would be to keep it standalone for now (as in this PR). Once more Gemma 4-like architectures (e.g., YOCO or per-layer embedding variants) are introduced, perhaps around 4-5 models, it may make sense to revisit the design and refactor them into a unified architecture. What do you think?

cc: @shewu-quic @winskuo-quic @haowhsu-quic @chenweng-quic @abhinaykukkadapu