Fix LoRA IndexError when syncing weights to vLLM for Qwen models in GRPO colocate mode

[jinchenyu]

patched_load_adapter uses the model's hf_to_vllm_mapper as-is when building a LoRAModel from in-memory tensors, but for models like Qwen (whose mapper stacks q_proj/k_proj/v_proj into a single qkv_proj name for base-weight loading), this collapses the three distinct LoRA tensor names into one, silently overwriting each other. The result is a malformed LoRAModel that later raises
"IndexError: tuple index out of range" from
ColumnParallelLinearWithLoRA.set_lora() when the LoRA adapter is synced into the vLLM engine in GRPO colocate mode.

vllm's own worker_manager._load_adapter already calls get_unstacked_mapper() before using the mapper for LoRA name parsing, so that constituent module names survive instead of being rewritten to the stacked vLLM name. Apply the same fix here.

PR type

• Bug Fix
• New Feature
• Document Updates
• More Models or Datasets Support

PR information

Problem

In GRPO's vllm_mode=colocate with vllm_enable_lora=true, syncing a LoRA adapter into the vLLM engine for Qwen-family models (Qwen2/Qwen3, whose base weights fuse q_proj/k_proj/v_proj into a single qkv_proj module) raises:

IndexError: tuple index out of range

at ColumnParallelLinearWithLoRA.set_lora() on the very first LoRA sync.

Reproduced with vllm==0.26.0.

Root cause

Qwen2Model.hf_to_vllm_mapper (inherited by Qwen3) defines a stacked mapping used for loading base checkpoint weights:

hf_to_vllm_mapper = WeightsMapper(
    orig_to_new_stacked={
        ".q_proj": (".qkv_proj", "q"),
        ".k_proj": (".qkv_proj", "k"),
        ".v_proj": (".qkv_proj", "v"),
        ...
    }
)

patched_load_adapter passes this mapper as-is via weights_mapper into LoRAModel.from_lora_tensors, which uses it to parse LoRA tensor names (parse_fine_tuned_lora_name). Because the mapper rewrites q_proj/k_proj/v_proj all to the same qkv_proj name, the three distinct LoRA A/B tensors collapse into a single dict entry and overwrite each other. The resulting LoRAModel ends up with a plain 2D tensor instead of the list of per-shard tensors that PackedLoRALayerWeights/set_lora() expect, so indexing into it raises the IndexError.

This is purely a name-collision bug in the LoRA-name-parsing path — it does not depend on model size, only on the model family defining a stacked hf_to_vllm_mapper (all Qwen2/Qwen3 sizes share the same mapper structure).

Note: hf_to_vllm_mapper is computed once and shared by both branches of patched_load_adapter — the in-memory from_lora_tensors path (used by GRPO colocate weight sync) and the on-disk from_local_checkpoint path — so this fix benefits both, not just the tensor-sync path.

Fix

vllm's own worker_manager._load_adapter avoids exactly this problem by calling hf_to_vllm_mapper.get_unstacked_mapper() before using the mapper for LoRA name parsing — this drops the QKV/MLP-fusion stacked maps while keeping genuine renames/prefixes, so q_proj/k_proj/v_proj survive as distinct names for the LoRA manager to pack correctly. This PR applies the same call in patched_load_adapter.

get_unstacked_mapper() was introduced in a relatively recent vllm version; this fix guards with hasattr(hf_to_vllm_mapper, 'get_unstacked_mapper') so it stays a no-op (falling back to the original mapper) on older vllm versions that predate this method, rather than raising AttributeError. For model families whose hf_to_vllm_mapper has no stacked maps to begin with, get_unstacked_mapper() is itself a no-op, so this change carries no regression risk for non-Qwen models.

Experiment results

Ran the LoRA + colocate GRPO smoke test (Qwen3-0.6B, gsm8k, vllm_enable_lora=true, vllm_mode=colocate, 3 steps) with vllm==0.26.0. Note: the bug is a pure name-mapping collision independent of model scale — the same hf_to_vllm_mapper code path is shared by every Qwen2/Qwen3 size, so a smoke-scale model is sufficient to reproduce and verify the fix.

• Before the fix: IndexError: tuple index out of range at ColumnParallelLinearWithLoRA.set_lora() on the first LoRA sync into vLLM.

• After the fix: training completes all 3 steps without error, checkpoint saves successfully, loss/reward values are sane (no NaN):

  Train: 100%|██████████| 3/3 [00:06<00:00, 2.09s/it]
  {'train_runtime': '6.255', 'train_loss': '7.11e-06', ...}
  [INFO:swift] Saving model checkpoint to .../checkpoint-3