MultiDiffusion: High-Resolution Image Generation with Pre-trained Models

Generate seamless panoramic and ultra-high-resolution images with state-of-the-art diffusion models.

MultiDiffusion is a technique that enables text-to-image diffusion models to generate images at resolutions far beyond their training size — producing wide-format and ultra-high-res outputs without visible seams or tiling artifacts. This repository implements the technique across multiple state-of-the-art models under a unified, modular architecture.

---

How It Works

The core of the method lives in the denoising loop (using also utils/common.py) and in each model's custom pipeline. The process unfolds in four stages at every denoising timestep:

1. Tile generation: the target panoramic canvas is divided into overlapping windows (tiles), each matching the model's native resolution (e.g. 1024×1024).
2. Per-tile denoising: at every timestep, each tile is denoised independently through the model's transformer or UNet.
3. Blending: overlapping tile outputs are averaged pixel-by-pixel via value and count tensors, eliminating seam artifacts.
4. Variable stride: the stride parameter evolves during denoising: smaller in early steps (more overlap → coherent global structure) and larger in later steps (less overlap → sharp local details).

---

Repository Structure

MultiDiffusion-diffusers/
    ├── inference_*.py          # 7 inference entry-point scripts
    ├── pipelines/              # 7 custom pipeline implementations
    ├── utils/                  # Shared tiling, blending, and helper utilities
    └── outputs/                # Sample output images

---

Supported Models

Script	Model
inference_flux.py	FLUX.1-dev
inference_flux2.py	FLUX.2-dev
inference_flux2_kb.py	FLUX.2-klein (4B / 9B)
inference_flux2_kb_img2img.py	FLUX.2-klein (4B / 9B) — image-to-image
inference_flux_cn.py	FLUX.1 + ControlNet
inference_sdxl.py	stable-diffusion-xl-base-1.0
inference_sd3.py	stable-diffusion-3-medium-diffusers

---

Key Parameters

Parameter	Description
height / width	Dimensions of each individual tile (default 1024x1024)
total_height / total_width	Final output canvas size (e.g. 3072×3072)
view_batch_size	Number of tiles processed together (memory/speed trade-off)
stride_start / stride_end	Tile grid step at the beginning and end of denoising
guidance_scale	Prompt adherence strength (typical range: 3.5–7.5)
num_inference_steps	Total denoising steps (typical range: 20–50)

---

Advanced Features

ControlNet — condition generation on structural signals such as Canny edge maps, depth maps, or pose skeletons.

Image-to-image — feed an existing canvas image and refine it at high resolution while preserving its overall composition.

Memory optimization — CPU offload, VAE tiling, and sliced VAE decoding allow generation on GPUs with limited VRAM.

---

Installation

git clone https://github.com/your-username/MultiDiffusion-repo.git
cd MultiDiffusion-repo/MultiDiffusion-diffusers

pip install -r requirements.txt

Note: a CUDA-capable GPU is strongly recommended. For very large outputs (4096×4096 and above), 24 GB VRAM or CPU offload is advised.

---

Usage

Each model has a dedicated inference script. Below are representative examples.

Text-to-image — FLUX.2-klein 9B (specify the 9B model inside the code)

python inference_flux2_kb.py

Image-to-image — FLUX.2-klein 9B

python inference_flux2_kb_img2img.py 

Text-to-image — Stable Diffusion 3

python inference_sd3.py 

---

Results

All images below were generated at 2048×2048 resolution on a single GPU using the MultiDiffusion tiling pipeline.

FLUX.2-klein 9B — Text-to-image

Prompt: "A vast alien mountain range under twin nebulae, glowing purple energy rivers winding through jagged peaks, teal and gold bioluminescent mist, floating ice crystals, ultra-detailed fantasy landscape"

---

FLUX.2-klein 9B — Image-to-image

The model takes a lower-resolution or lower-detail image as input and refines it at high resolution, preserving the overall scene composition while adding sharpness and detail.

Input	Output

---

Stable Diffusion 3 — Text-to-image

Prompt: "Aerial view of a dense tropical rainforest, infinite monstera and palm canopy, vivid emerald and lime greens, soft dappled sunlight"

---

Dependencies

Package	Version	Role
torch + torchvision	2.11.0+cu128 / 0.26.0+cu128	Core deep learning framework and image transforms
diffusers	0.37.1	Diffusion model pipelines (Hugging Face)
transformers	5.5.0	Text encoders — CLIP, T5, SigLIP
huggingface_hub	1.11.0	Model download and Hub utilities
helpers	0.2.0	FLUX pipeline helper utilities
safetensors	0.7.0	Efficient model weight serialization
Pillow / numpy	12.2.0 / 2.4.4	Image I/O and array operations
opencv-python	4.13.0.92	ControlNet preprocessing (Canny, depth)
matplotlib	3.10.9	Visualization and debug image display
tqdm	4.67.3	Progress bars for tile decoding
scipy	1.17.1	Scientific computing (transitive dependency)
torch_xla	2.9.0	XLA / TPU support (optional)

Install all dependencies with:

pip install -r requirements.txt

---

License

This project is released under the Apache 2.0 License. See LICENSE for details.