UT3G

just a repo where i store information about the UT3G

Install

Start by setting up Tinygrad

Then we want to install TinyGPU

With the 7900xtx we use ENV

Testing with:

DEV=AMD:HIP uv run python3 -m tinygrad.device

Optimisation

Speed, batch_size, finetuning, add features (add video on yolov)

BEAM=2 DEV=AMD:HIP uv run examples/yolov8_video.py video.m4
DEV=AMD:HIP uv run examples/yolov8_video.py 

Simple mnist:

PYTHONPATH="." DEBUG=2 DEV=AMD:HIP uv run python3 examples/beautiful_mnist.py

Yolov:

PYTHONPATH="." DEBUG=2 DEV=AMD:HIP uv run python3 examples/yolov8.py "~/.tools/wallpapers/Japanese_garden.jpg" m

Yolov8 has different variants, you can choose from ['n', 's', 'm', 'l', 'x']

fast-llama-gpt2

In order to pass the template correctly to pi, you need jinja2:

uv pip install jinja2

you need the huggingface_cli: curl -LsSf https://hf.co/cli/install.sh | bash then we can download with link from huggingface: hf download hf://unsloth/Qwen3.6-35B-A3B-GGUF/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf and then we can use ~/.cache/huggingface/hub/model-name/snapshots/git-hash/model-Quantized

! Test --benchmark, --warmup, BEAM vs JITBEAM [0,1,2,3,4] ! Also need to understand how to manage max_context lenght? set higher number go big ? are smaller context to give? ! how to have 0 Context addition query during thinking? look into [pi]

JITBEAM=2 DEBUG=2 AM_RESET=1 AM_DEBUG=2 DEV=AMD:HIP uv run python3 -m tinygrad.llm --model "/Users/plagache/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-GGUF/snapshots/a483e9e6cbd595906af30beda3187c2663a1118c/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" --serve --max_context=65536

New model drop out With DEBUG=2 at first to see the different Optimisation then we remove it and add the --max_context=262144 131072 or 65536 respectively multiple of 1024 * 256 128 or 64 This quantization Qwen3.8-27B-IQ4_XS.gguf

JITBEAM=2 DEBUG=2 AM_RESET=1 AM_DEBUG=2 DEV=AMD:HIP uv run python3 -m tinygrad.llm --model "/Users/plagache/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/b62a80264f8b0c1bb849ee1c9c487415ebeca194/Qwen3.8-27B-IQ4_XS.gguf" --serve --max_context=65536

Then you have very small models that we would want to Agent.

JITBEAM=2 DEBUG=2 AM_RESET=1 AM_DEBUG=2 DEV=AMD:HIP uv run python3 -m tinygrad.llm --model "/Users/plagache/.cache/huggingface/hub/models--unsloth--Qwen3-0.6B-GGUF/snapshots/50968a4468ef4233ed78cd7c3de230dd1d61a56b/Qwen3-0.6B-IQ4_XS.gguf" --serve --max_context=262144

BEAM=2 AM_RESET=1 AM_DEBUG=2 DEV=AMD:HIP uv run python3 -m tinygrad.llm -m "qwen3.6:35b-a3b" --serve

DEBUG=2 AM_RESET=1 BEAM=2 GMMU=0 DEV=AMD:HIP uv run python3 -m tinygrad.llm -m "qwen3.5:0.8b" --benchmark 32

Difference JITBEAM and BEAM ? What is GMMU? seems to be GPU Memory Management Unit? in Tinygrad it will bypass the GMMU of the provider

Development

test World Model test simulation Quel World?, pour farming? Quel Model?

Todo

• Mount 7900xtx on ut3g
• plug everything in the PSU
• flash firmware
  • we actually didn't need to flash a specifique firmware
  • re-flashed the base USB4 firmware
• mnist examples to appreciates the speed and Viz UI
• yolov example
• Qwen3.5_0.4b.gguf from 200tok/s to 250tok/s
• Qwen3.5_4b.gguf from 5tok/s to 105tok/s
• Qwen3.27b.gguf running at 20tok/s with JITBEAM=2
• plug Local Qwen in pi
• Simple push T world model rewrite in tinygrad
• yolov on video, look at roryclear Examples
  • the idea is to cut the video in multiple frame, and feed the frame one by one
  • then recreating the video with the list of frame processed by yolov
  • Packed frame with batch_size, git apply staged.patch in tinygrad
  • in his implementation roryclear is creating Camera stream object that express its setup
• finetune yolov with rugby dataset
  • think of other architecture that could learn from the rugby model

TinyRack

for the TinyGpu idea from: https://fangpenlin.com/posts/2025/11/26/tinyrack-a-3d-printable-modular-rack-for-mini-server/ https://fangpenlin.com/posts/2026/01/12/manufacturing-as-code-is-the-future/

3d Tool https://github.com/gumyr/build123d

https://www.printables.com/model/1494272-tinyrack-an-open-source-modular-customizable-mini We bought the Mac Mini M4 base

need to make a setup that we can moove and transport

mining rig gpu case was the search i made to find rack for 6-8 GPU's wire rack

Current

longeur 80

largeur 35

hauteur 40

2 etage

Desired

3 etage

35 - 90

moin de 1m10 - 1m

https://www.ikea.com/fr/fr/p/omar-etagere-acier-zingue-10069763/

and some 3D printed ideas https://www.printables.com/model/1113251-apple-mac-mini-m4-vertical-holder-with-hubssd#preview.506eY https://www.printables.com/model/1438192-mac-mini-m4-stand-with-storage https://www.printables.com/model/1576269-openclaw-claude-code-mac-mini-m4-enclosure

Archives

NO lsusb anymore its a usb4 plug and power the 7900xtx

lsusb

should return: Bus 00X Device 00X: ID add1:0001 tiny custom v0.1 then you need to:

uv run pcie_bringup.py

should return: *** PCIe link is UP! ***

CUSTOM=1 DEBUG=2 AM_RESET=1 GMMU=0 DEV=USB+AMD uv run python3 pcie/test_sram_verify.py