Hi！I attempted to run it locally, but encountered an error. I once tried vram_limit, but unfortunately it failed. This is very unfavorable for downstream devices. Could you please optimize the code to automatically be compatible with multiple GPUs by allocating weights to different GPUs? Or, the model can be imported to the CPU and then GPU can be utilized as needed, similar to the idea of sparse MoE? Additionally, the error message is as follows
[] Loading input image from ./assets/data/crocodile/0.png...
[] Loading DepthAnything3 Prior Models...
[INFO ] using SwiGLU layer as FFN
Loading weights from local directory
-> Extracting 3D Features and Intrinsics...
[INFO ] Processed Images Done taking 0.033123016357421875 seconds. Shape: torch.Size([1, 3, 462, 840])
[INFO ] Model Forward Pass Done. Time: 0.8708021640777588 seconds
[INFO ] Conversion to Prediction Done. Time: 0.034888267517089844 seconds
-> Extracting Depth Map...
[INFO ] using SwiGLU layer as FFN
[INFO ] using MLP layer as FFN
Loading weights from local directory
[INFO ] Processed Images Done taking 0.02515101432800293 seconds. Shape: torch.Size([1, 3, 462, 840])
[INFO ] Model Forward Pass Done. Time: 0.30040574073791504 seconds
[INFO ] Conversion to Prediction Done. Time: 0.0004968643188476562 seconds
[] Calculating Target Pose -> Yaw: -30.0°, Pitch: -30.0°, Radius: 2.0
[] Loading Qwen-Image-Edit Pipeline...
2026-07-26 11:57:02,432 | INFO | modelscope_hub.download | No files to download for Qwen/Qwen-Image-Edit@master
Loading models from: [
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00006-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00009-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00008-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00002-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00001-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00004-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00007-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00005-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00003-of-00009.safetensors"
]
[W726 11:59:07.707538121 CUDACachingAllocator.cpp:508] expandable_segments: memory mapping failed with OOM on device 0 while trying to map 20971520 bytes (free: 20316160, total: 47665709056).
[W726 11:59:07.707659316 CUDACachingAllocator.cpp:508] expandable_segments: memory mapping failed with OOM on device 0 while trying to map 20971520 bytes (free: 20316160, total: 47665709056).
Traceback (most recent call last):
File "/workspace/MetaView/src/inference.py", line 227, in
main()
File "/workspace/MetaView/src/inference.py", line 181, in main
pipe = MetaViewPipeline.from_pretrained(
File "/workspace/MetaView/src/MetaView_pipeline.py", line 61, in from_pretrained
model_pool = pipe.download_and_load_models(model_configs, vram_limit)
File "/workspace/MetaView/diffsynth/diffusion/base_pipeline.py", line 288, in download_and_load_models
model_pool.auto_load_model(
File "/workspace/MetaView/diffsynth/models/model_loader.py", line 70, in auto_load_model
model = self.load_model_file(config, path, vram_config, vram_limit=vram_limit)
File "/workspace/MetaView/diffsynth/models/model_loader.py", line 40, in load_model_file
model = load_model(
File "/workspace/MetaView/diffsynth/core/loader/model.py", line 55, in load_model
model = model.to(dtype=torch_dtype, device=device)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1383, in to
return self._apply(convert)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 933, in _apply
module._apply(fn)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 933, in _apply
module._apply(fn)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 933, in _apply
module._apply(fn)
[Previous line repeated 1 more time]
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 964, in _apply
param_applied = fn(param)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1369, in convert
return t.to(
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 18.00 MiB. GPU 0 has a total capacity of 44.39 GiB of which 19.38 MiB is free. Including non-PyTorch memory, this process has 44.37 GiB memory in use. Of the allocated memory 43.81 GiB is allocated by PyTorch, and 49.34 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf)
Hi！I attempted to run it locally, but encountered an error. I once tried vram_limit, but unfortunately it failed. This is very unfavorable for downstream devices. Could you please optimize the code to automatically be compatible with multiple GPUs by allocating weights to different GPUs? Or, the model can be imported to the CPU and then GPU can be utilized as needed, similar to the idea of sparse MoE? Additionally, the error message is as follows
[] Loading input image from ./assets/data/crocodile/0.png...
[] Loading DepthAnything3 Prior Models...
[INFO ] using SwiGLU layer as FFN
Loading weights from local directory
-> Extracting 3D Features and Intrinsics...
[INFO ] Processed Images Done taking 0.033123016357421875 seconds. Shape: torch.Size([1, 3, 462, 840])
[INFO ] Model Forward Pass Done. Time: 0.8708021640777588 seconds
[INFO ] Conversion to Prediction Done. Time: 0.034888267517089844 seconds
-> Extracting Depth Map...
[INFO ] using SwiGLU layer as FFN
[INFO ] using MLP layer as FFN
Loading weights from local directory
[INFO ] Processed Images Done taking 0.02515101432800293 seconds. Shape: torch.Size([1, 3, 462, 840])
[INFO ] Model Forward Pass Done. Time: 0.30040574073791504 seconds
[INFO ] Conversion to Prediction Done. Time: 0.0004968643188476562 seconds
[] Calculating Target Pose -> Yaw: -30.0°, Pitch: -30.0°, Radius: 2.0
[] Loading Qwen-Image-Edit Pipeline...
2026-07-26 11:57:02,432 | INFO | modelscope_hub.download | No files to download for Qwen/Qwen-Image-Edit@master
Loading models from: [
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00006-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00009-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00008-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00002-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00001-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00004-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00007-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00005-of-00009.safetensors",
"./models/Qwen/Qwen-Image-Edit/transformer/diffusion_pytorch_model-00003-of-00009.safetensors"
]
[W726 11:59:07.707538121 CUDACachingAllocator.cpp:508] expandable_segments: memory mapping failed with OOM on device 0 while trying to map 20971520 bytes (free: 20316160, total: 47665709056).
[W726 11:59:07.707659316 CUDACachingAllocator.cpp:508] expandable_segments: memory mapping failed with OOM on device 0 while trying to map 20971520 bytes (free: 20316160, total: 47665709056).
Traceback (most recent call last):
File "/workspace/MetaView/src/inference.py", line 227, in
main()
File "/workspace/MetaView/src/inference.py", line 181, in main
pipe = MetaViewPipeline.from_pretrained(
File "/workspace/MetaView/src/MetaView_pipeline.py", line 61, in from_pretrained
model_pool = pipe.download_and_load_models(model_configs, vram_limit)
File "/workspace/MetaView/diffsynth/diffusion/base_pipeline.py", line 288, in download_and_load_models
model_pool.auto_load_model(
File "/workspace/MetaView/diffsynth/models/model_loader.py", line 70, in auto_load_model
model = self.load_model_file(config, path, vram_config, vram_limit=vram_limit)
File "/workspace/MetaView/diffsynth/models/model_loader.py", line 40, in load_model_file
model = load_model(
File "/workspace/MetaView/diffsynth/core/loader/model.py", line 55, in load_model
model = model.to(dtype=torch_dtype, device=device)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1383, in to
return self._apply(convert)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 933, in _apply
module._apply(fn)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 933, in _apply
module._apply(fn)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 933, in _apply
module._apply(fn)
[Previous line repeated 1 more time]
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 964, in _apply
param_applied = fn(param)
File "/opt/conda/envs/metaview/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1369, in convert
return t.to(
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 18.00 MiB. GPU 0 has a total capacity of 44.39 GiB of which 19.38 MiB is free. Including non-PyTorch memory, this process has 44.37 GiB memory in use. Of the allocated memory 43.81 GiB is allocated by PyTorch, and 49.34 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf)