Skip to content

vllm 0.23.0版本 安装此插件后，启动时报错，如何解决? #8

Open
#10
Open
vllm 0.23.0版本 安装此插件后，启动时报错，如何解决?#8
#10

Description

@huangcheng1x
huangcheng1x
opened on Sep 1, 2026

在国产化算力环境下，已有环境vllm 0.23.0 启动推理Qwen3.5 系列模型正常推理情况下，安装Spark-plugin完成。

python -c "from importlib.metadata import entry_points; \
print([(e.name, e.value) for e in entry_points(group='vllm.general_plugins')])"
[('spark2_5', 'vllm_spark2_5_plugin:register'), ('dl_models_plugin', 'vllm.plugins.dl_models_plugin.vllm_add_models:register'), ('lora_filesystem_resolver', 'vllm.plugins.lora_resolvers.filesystem_resolver:register_filesystem_resolver'), ('lora_hf_hub_resolver', 'vllm.plugins.lora_resolvers.hf_hub_resolver:register_hf_hub_resolver')]

启动过程中报错，是否能解决？

vllm serve /data/models/Spark-X2.5-4B/ --trust-remote-code --enable-auto-tool-choice --tool-call-parser spark25 --chat-template /data/models/Spark-X2.5-4B/chat_template.jinja --served-model-name Spark2_5
INFO 09-01 14:47:05 [__init__.py:43] Available plugins for group vllm.platform_plugins:
INFO 09-01 14:47:05 [__init__.py:45] - dl -> vllm.plugins.dl_platform_plugin:register
INFO 09-01 14:47:05 [__init__.py:48] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 09-01 14:47:06 [__init__.py:238] Platform plugin dl is activated
[transformers] `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead.
INFO 09-01 14:47:12 [dl_patch_config.py:61] patch in dl_platform
INFO 09-01 14:47:12 [dl_patch_config.py:61] patch in dl_platform
INFO 09-01 14:47:12 [dl_patch_config.py:61] patch in dl_platform
(APIServer pid=4353) INFO 09-01 14:47:12 [api_utils.py:339] 
(APIServer pid=4353) INFO 09-01 14:47:12 [api_utils.py:339]        █     █     █▄   ▄█
(APIServer pid=4353) INFO 09-01 14:47:12 [api_utils.py:339]  ▄▄ ▄█ █     █     █ ▀▄▀ █  version 0.23.0
(APIServer pid=4353) INFO 09-01 14:47:12 [api_utils.py:339]   █▄█▀ █     █     █     █  model   /data/models/Spark-X2.5-4B/
(APIServer pid=4353) INFO 09-01 14:47:12 [api_utils.py:339]    ▀▀  ▀▀▀▀▀ ▀▀▀▀▀ ▀     ▀
(APIServer pid=4353) INFO 09-01 14:47:12 [api_utils.py:339] 
(APIServer pid=4353) INFO 09-01 14:47:12 [dl_patch_config.py:61] patch in dl_platform
(APIServer pid=4353) INFO 09-01 14:47:12 [api_utils.py:273] non-default args: {'model_tag': '/data/models/Spark-X2.5-4B/', 'chat_template': '/data/models/Spark-X2.5-4B/chat_template.jinja', 'enable_auto_tool_choice': True, 'tool_call_parser': 'spark25', 'model': '/data/models/Spark-X2.5-4B/', 'trust_remote_code': True, 'served_model_name': ['Spark2_5']}
(APIServer pid=4353) INFO 09-01 14:47:12 [dl_patch_config.py:61] patch in dl_platform
(APIServer pid=4353) WARNING 09-01 14:47:12 [envs.py:2088] Unknown vLLM environment variable detected: VLLM_RPC_TIMEOUT
(APIServer pid=4353) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'full_attention', 'sliding_attention'}
(APIServer pid=4353) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'full_attention', 'sliding_attention'}
(APIServer pid=4353) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'full_attention', 'sliding_attention'}
(APIServer pid=4353) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'full_attention', 'sliding_attention'}
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] Error in inspecting model architecture 'Spark2_5ForCausalLM'
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] Traceback (most recent call last):
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 1397, in _run_in_subprocess
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     returned.check_returncode()
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/lib/python3.12/subprocess.py", line 502, in check_returncode
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     raise CalledProcessError(self.returncode, self.args, self.stdout,
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'vllm.model_executor.models.registry']' returned non-zero exit status 1.
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] 
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] The above exception was the direct cause of the following exception:
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] 
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] Traceback (most recent call last):
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 960, in _try_inspect_model_cls
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     return model.inspect_model_cls()
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]            ^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/logging_utils/log_time.py", line 21, in _wrapper
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     result = func(*args, **kwargs)
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]              ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 921, in inspect_model_cls
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     mi = _run_in_subprocess(
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]          ^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 1400, in _run_in_subprocess
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     raise RuntimeError(
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] RuntimeError: Error raised in subprocess:
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] <frozen runpy>:128: RuntimeWarning: 'vllm.model_executor.models.registry' found in sys.modules after import of package 'vllm.model_executor.models', but prior to execution of 'vllm.model_executor.models.registry'; this may result in unpredictable behaviour
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] [transformers] `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead.
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] Traceback (most recent call last):
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "<frozen runpy>", line 198, in _run_module_as_main
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "<frozen runpy>", line 88, in _run_code
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 1423, in <module>
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     _run()
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 1416, in _run
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     result = fn()
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]              ^^^^
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 922, in <lambda>
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     lambda: _ModelInfo.from_model_cls(self.load_model_cls())
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]                                       ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 935, in load_model_cls
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     mod = importlib.import_module(self.module_name)
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     return _bootstrap._gcd_import(name[level:], package, level)
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/data/models/Spark-plugin-main/src/vllm_spark2_5_plugin/spark2_5.py", line 323, in <module>
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     class Spark2_5ForCausalLM(nn.Module, SupportsPP):
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]   File "/data/models/Spark-plugin-main/src/vllm_spark2_5_plugin/spark2_5.py", line 329, in Spark2_5ForCausalLM
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]     hf_to_vllm_mapper = WeightsMapper(
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962]                         ^^^^^^^^^^^^^^
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] TypeError: WeightsMapper.__init__() got an unexpected keyword argument 'orig_to_new_stacked'
(APIServer pid=4353) ERROR 09-01 14:47:23 [registry.py:962] 
(APIServer pid=4353) Traceback (most recent call last):
(APIServer pid=4353)   File "/usr/local/bin/vllm", line 6, in <module>
(APIServer pid=4353)     sys.exit(main())
(APIServer pid=4353)              ^^^^^^
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/main.py", line 103, in main
(APIServer pid=4353)     args.dispatch_function(args)
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/serve.py", line 148, in cmd
(APIServer pid=4353)     uvloop.run(run_server(args))
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=4353)     return __asyncio.run(
(APIServer pid=4353)            ^^^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/lib/python3.12/asyncio/runners.py", line 195, in run
(APIServer pid=4353)     return runner.run(main)
(APIServer pid=4353)            ^^^^^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=4353)     return self._loop.run_until_complete(task)
(APIServer pid=4353)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=4353)     return await main
(APIServer pid=4353)            ^^^^^^^^^^
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 665, in run_server
(APIServer pid=4353)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 679, in run_server_worker
(APIServer pid=4353)     async with build_async_engine_client(
(APIServer pid=4353)                ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=4353)     return await anext(self.gen)
(APIServer pid=4353)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 99, in build_async_engine_client
(APIServer pid=4353)     async with build_async_engine_client_from_engine_args(
(APIServer pid=4353)                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=4353)     return await anext(self.gen)
(APIServer pid=4353)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 123, in build_async_engine_client_from_engine_args
(APIServer pid=4353)     vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=4353)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 1755, in create_engine_config
(APIServer pid=4353)     model_config = self.create_model_config()
(APIServer pid=4353)                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 1578, in create_model_config
(APIServer pid=4353)     return ModelConfig(
(APIServer pid=4353)            ^^^^^^^^^^^^
(APIServer pid=4353)   File "/usr/local/lib/python3.12/dist-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=4353)     s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=4353) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=4353)   Value error, Model architectures ['Spark2_5ForCausalLM'] failed to be inspected. Please check the logs for more details. [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]
(APIServer pid=4353)     For further information visit https://errors.pydantic.dev/2.13/v/value_error
root@iFLYTEK-AIPC:/data/models# python3 - <<'EOF'
from vllm.model_executor.models.utils import WeightsMapper
import inspect

print(inspect.getsource(WeightsMapper))
EOF

INFO 09-01 15:01:46 [__init__.py:43] Available plugins for group vllm.platform_plugins:
INFO 09-01 15:01:46 [__init__.py:45] - dl -> vllm.plugins.dl_platform_plugin:register
INFO 09-01 15:01:46 [__init__.py:48] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 09-01 15:01:46 [__init__.py:238] Platform plugin dl is activated
^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/utils.py", line 25, in <module>
    from vllm.model_executor.model_loader.reload import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/__init__.py", line 11, in <module>
    from vllm.model_executor.model_loader.base_loader import BaseModelLoader
  File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 12, in <module>
    from vllm.model_executor.model_loader.reload import finalize_layerwise_processing
  File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/reload/__init__.py", line 29, in <module>
    from .layerwise import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/reload/layerwise.py", line 14, in <module>
    from vllm.model_executor.model_loader.weight_utils import default_weight_loader
  File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/weight_utils.py", line 63, in <module>
    from fastsafetensors import SafeTensorsFileLoader, SingleGroup
  File "/usr/local/lib/python3.12/dist-packages/fastsafetensors/__init__.py", line 7, in <module>
    from .auto_loader import AutoLoader
  File "/usr/local/lib/python3.12/dist-packages/fastsafetensors/auto_loader.py", line 5, in <module>
    from .common import init_logger
  File "/usr/local/lib/python3.12/dist-packages/fastsafetensors/common.py", line 12, in <module>
    from .dlpack import from_cuda_buffer
  File "/usr/local/lib/python3.12/dist-packages/fastsafetensors/dlpack.py", line 116, in <module>
    class DLTensor(ctypes.Structure):
KeyboardInterrupt
^C
root@iFLYTEK-AIPC:/data/models# python -c "from importlib.metadata import entry_points; \
print([(e.name, e.value) for e in entry_points(group='vllm.general_plugins')])"
[('spark2_5', 'vllm_spark2_5_plugin:register'), ('dl_models_plugin', 'vllm.plugins.dl_models_plugin.vllm_add_models:register'), ('lora_filesystem_resolver', 'vllm.plugins.lora_resolvers.filesystem_resolver:register_filesystem_resolver'), ('lora_hf_hub_resolver', 'vllm.plugins.lora_resolvers.hf_hub_resolver:register_hf_hub_resolver')]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions