A self-hosted text-to-image and image-to-image generation studio built on Stable Diffusion XL, FastAPI, and React. Deployed and tested end-to-end on a single T4 GPU using Lightning AI Studios, with a full write-up of the build process linked below.
- Overview
- Example Outputs
- Demo
- Features
- Architecture
- Tech Stack
- Getting Started
- Environment Variables
- API Reference
- Generation Parameters
- Learnings
- Future Improvements
- Full Tutorial
- License
- Acknowledgments
PixelForge Studio is a personal image generation tool built to explore the full stack of running a diffusion model as a real application, not just a notebook demo. It consists of a FastAPI backend that loads Stable Diffusion XL (base 1.0) once at startup and serves both text-to-image and image-to-image requests, and a React (Vite) frontend that exposes generation controls, a live preview, and a persistent gallery of past generations.
The project was built and deployed on a Lightning AI Studio using a single NVIDIA T4 (16GB) GPU, with the goal of keeping the entire workflow usable within a free monthly compute allowance for personal/demo usage(s).
A collage of outputs generated using this app across different prompts, schedulers, and quality presets. Individual examples with details:
| Prompt theme | Scheduler | Notes |
|---|---|---|
 |
DPM++ 2M | High quality preset, 16:9 |
 |
Euler Ancestral | Standard preset, 4:3 |
 |
DDIM | Standard preset, 16:9 |
See the full gallery in docs/examples/ for more outputs across prompts, schedulers, and quality presets.
A walkthrough of prompt entry, parameter selection, generation, and the gallery view.
- Text-to-image generation using Stable Diffusion XL base 1.0, served through a single always-loaded FastAPI pipeline.
- Image-to-image generation with adjustable denoising strength, using the same underlying weights (no duplicate model in memory).
- Quality presets (
draft,standard,high) mapping to tuned step count and guidance scale combinations, with optional manual overrides. - Aspect ratio presets covering common ratios (1:1, 4:3, 3:4, 16:9, 9:16, 3:2, 2:3), each using dimensions that are valid multiples of 8 for SDXL's latent space.
- Scheduler selection at request time (Euler Ancestral, DPM++ 2M, DDIM) without reloading the model.
- Generation history / gallery, persisted to disk (
backend/data/history.json) and viewable in the frontend's Gallery tab, with delete support. - Basic per-client rate limiting on the generation endpoints to avoid accidental GPU overuse.
- Configurable CORS, so the deployed frontend origin can be locked down in production instead of left wide open.
The backend and frontend run as two separate processes on the same GPU-backed host during development and personal use. The SDXL weights are loaded exactly once at process startup and reused across every request; no per-request model loading occurs.
Backend
- Python, FastAPI, Uvicorn
- Hugging Face
diffusers,transformers,accelerate - PyTorch (CUDA),
xformersfor memory-efficient attention - Pillow for image I/O
Frontend
- React 18, Vite
- React Router
Infrastructure
- NVIDIA T4 (16GB) GPU via Lightning AI Studios
- Hugging Face Hub for model weight distribution and local caching
- Python 3.10+
- Node.js 18+ and npm
make(optional, only needed if you want to use the providedMakefileshortcuts)- An NVIDIA GPU with at least 16GB VRAM (a T4 is sufficient for SDXL base in fp16)
- A Hugging Face account and access token if the model requires authenticated download
git clone <your-repository-url>.git
cd <your-repository-name>cd backend
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env # then fill in the values, see Environment Variables below
uvicorn main:app --host 0.0.0.0 --port 4000On first run, the backend downloads the SDXL base 1.0 weights (~7GB) from Hugging Face and caches them locally under the path set in HF_HOME. Subsequent restarts load from the local cache and skip the download.
Confirm the backend is healthy:
curl http://localhost:4000/api/healthcd frontend
npm install
cp .env.example .env # set VITE_API_URL to your backend's address
npm run devThe frontend expects the backend URL in VITE_API_URL (for example http://localhost:4000, or your GPU host's public forwarded URL when deployed remotely).
Instead of running the backend and frontend in two separate terminals, a Makefile is included at the repository root with shortcuts for common tasks:
cd pixelforge-sdxl/
make install # installs backend and frontend dependencies
make # runs backend and frontend together in one terminalThis is entirely optional — running each service manually as shown above works the same way.
Backend (backend/.env)
| Variable | Description | Example |
|---|---|---|
CORS_ORIGINS |
Comma-separated list of allowed frontend origins. Use * only for local testing. |
https://your-frontend.example.com |
RATE_LIMIT_PER_MINUTE |
Max generation requests allowed per client IP per minute. | 20 |
HF_HOME |
Local path used to cache downloaded Hugging Face model weights. | /data/.cache/huggingface |
HF_TOKEN |
Hugging Face access token, if required for the model repository. | hf_xxxxxxxxxxxx |
Frontend (frontend/.env)
| Variable | Description | Example |
|---|---|---|
VITE_API_URL |
Base URL of the running backend. | http://localhost:4000 |
| Method | Endpoint | Description |
|---|---|---|
GET |
/api/health |
Returns service status plus the available quality presets and aspect ratios. |
POST |
/api/generate1 |
Text-to-image generation. Body: prompt, quality, aspect ratio, scheduler. |
POST |
/api/generate2 |
Image-to-image generation. Body: prompt, quality, scheduler, base64-encoded init image. |
GET |
/api/history |
Returns the full generation history. |
GET |
/api/history/{id} |
Returns a single history record by ID. |
DELETE |
/api/history/{id} |
Deletes a history record and its associated image files. |
Generated and uploaded images are served as static files under /generated/* and /uploads/* respectively.
Quality presets (steps, guidance scale):
| Preset | Steps | Guidance Scale | Use case |
|---|---|---|---|
draft |
30 | 5.0 | Fast iteration on prompt wording and composition |
standard |
40 | 7.0 | Balanced default for most generations |
high |
50 | 8.5 | Final output quality, slower |
Aspect ratios (width x height, SDXL-valid dimensions):
| Ratio | Dimensions |
|---|---|
| 1:1 | 1024 x 1024 |
| 4:3 | 1152 x 896 |
| 3:4 | 896 x 1152 |
| 16:9 | 1344 x 768 |
| 9:16 | 768 x 1344 |
| 3:2 | 1216 x 832 |
| 2:3 | 832 x 1216 |
Schedulers: Euler Ancestral, DPM++ 2M, DDIM. Switching schedulers does not reload the model, so it can be changed freely between requests.
Measured observations from running this app end-to-end on a single T4 (16GB) GPU:
- Generation time scales roughly linearly with step count. At 1024x1024 with DPM++ 2M, inference cost was approximately 0.3-0.4 seconds per step once the model was warm. This put
draft(30 steps) at roughly 10-12 seconds per image,standard(40 steps) at roughly 14-16 seconds, andhigh(50 steps) at roughly 18-20 seconds. Doubling steps roughly doubles latency, so step count is the primary lever for the speed-versus-quality trade-off, not guidance scale. - Guidance scale affects prompt adherence more than visual fidelity. Moving from a guidance scale of 5.0 to 8.5 with the same step count did not meaningfully change render time, but produced noticeably closer alignment to the literal prompt text at the cost of some naturalness in lighting and texture at the higher end. This is why quality presets pair a higher guidance scale with higher steps rather than treating them as independent controls.
- Scheduler choice changed the steps needed for comparable output quality. DPM++ 2M produced results at 25-30 steps that were visually comparable to Euler Ancestral at 35-40 steps, an effective 25-30 percent reduction in generation time for similar perceived quality, since DPM++ 2M converges faster per step.
- VRAM headroom on a T4 was sufficient for single-image batches with margin to spare. SDXL base 1.0 in fp16 with
xformersenabled used approximately 7.5-8GB of the 16GB available at 1024x1024. This left enough headroom for a batch size of 2 in most cases, but batch size 3 or more at 1024x1024 pushed close to the VRAM ceiling and risked out-of-memory errors. - Cold start cost is dominated by download, not by loading from cache. The first run downloaded approximately 6.9GB of fp16 weights, taking 3-4 minutes depending on network conditions. Every subsequent process start loaded the same weights from the local Hugging Face cache in under 30 seconds, confirming that model loading is a one-time-per-process cost, not a per-request one, provided the pipeline is instantiated at application startup rather than inside a request handler.
- img2img strength has a narrow effective range for controlled edits. Strength values below approximately 0.4 preserved most of the input image's structure with only stylistic changes. Values above approximately 0.75 produced outputs that were largely indistinguishable from a fresh text-to-image generation, with only a loose compositional influence from the input image. The useful range for recognizable-but-modified edits sat roughly between 0.4 and 0.7.
- Add the SDXL refiner stage as an optional second pass for higher-fidelity final outputs.
- Support LoRA loading for style-specific fine-tunes without needing a full model swap.
- Add ControlNet support for pose, depth, or edge-guided generation.
- Introduce a job queue so long-running generations do not block the request thread, with a polling or websocket-based status endpoint for the frontend.
- Move from in-memory rate limiting and history storage to a proper database and shared cache (for example Redis) to support multiple concurrent users or instances.
The complete step-by-step write-up, covering environment setup, model selection reasoning, backend and frontend implementation, and the deployment process on Lightning AI, is available here:
This project is licensed under the License. See LICENSE for details.
Stable Diffusion XL base 1.0 is distributed by Stability AI under the CreativeML Open RAIL++-M License. Review its terms before any commercial use of generated outputs.
- Stability AI for Stable Diffusion XL.
- Hugging Face for the
diffuserslibrary. - Lightning AI for the GPU compute environment this project was built and deployed on.