int-llm-postscript

Bit-exact, integer-only MicroGPT training and inference in pure PostScript.

This repository runs a character-level GPT (1 transformer layer, 32 dimensions, 4 heads, block size 8) entirely in PostScript and reproduces the int-llm C Q16.48 integer oracle byte for byte:

$ make verify
./gpt_int --load model.mgw > out_c.txt
gs -q -dBATCH -dNODISPLAY -dNOSAFER microgpt_infer.ps > out_ps.txt
cmp out_c.txt out_ps.txt
BIT-EXACT: PASS — C oracle and PostScript agree byte-for-byte (20/20 samples)

Both print the same 485 bytes: the Machin-formula Pi banner (Pi = 3.141592653589782, integer-truncation warts and all) and the same 20 sampled names — kayla, daia, lee, … karin — 122 forward passes through the transformer, every sampled token identical because every one of the ~1.8 million Q16.48 multiplies rounds identically.

Runtime: about 2 seconds in Ghostscript for the full 20-sample run.

Integer-only execution

PostScript has floating-point exp/ln/sqrt built in. This program uses none of them — no reals ever touch the computation. Everything is signed 64-bit integer arithmetic (add, mul, idiv, bitshift, xor, and), exactly mirroring the C oracle's fp_math.h semantics:

C oracle (fp_math.h)	PostScript port
fp_mul: 128-bit product, asr 48	signed floor-split into 24-bit limbs; carry chain keeps every intermediate < 2^63
fp_div: 128-bit (a<<48)/b, truncating	binary restoring division, remainder as two 31-bit limbs
fp_inv_sqrt: range-reduce + 8 Newton steps	same iteration, asr 49 done as floor-shift chain
fp_exp: dyadic (1 + x/2^14)^(2^14)	same 14 squarings through the exact fp_mul
xorshift64 PRNG (<<13, >>7, <<17)	state as two 32-bit halves; logical shifts reconstructed
RMSNorm, softmax, attention, sampling	line-by-line port of inference_forward / mgpt_generate_sample

The model file model.mgw (weights, tokenizer, and the PRNG state — all int64) is parsed directly by the PostScript program: MGW header, config, tensor index, little-endian two's-complement decode, the lot.

The one thing PostScript brings that C doesn't: the interpreter's integers must be 64-bit (Ghostscript's are), and its mul silently promotes to real on overflow — so the limb arithmetic is designed so that no intermediate can exceed 2^63 - 1. Any violation would instantly break the byte-exact gate.

Quick start

# Check the wide-Q16.48 path against the committed golden output.
make verify-ps

# Run either stored-weight path directly.
make run
make run-f12

Requires Ghostscript (gs) with 64-bit PostScript integers (any recent build). The supplied commands use -dNOSAFER for local model, dataset, and checkpoint I/O. That option disables Ghostscript's safety sandbox, so run only trusted repository contents and checkpoints.

The C-backed verification and training targets expect an int-llm checkout at ../int-llm. Set INTLLM=/path/to/int-llm to use another location. The reported release gates used int-llm commit 0b4b6d0.

Verification

Target	Purpose	Typical runtime
make verify-ps	Check standalone wide-Q16.48 inference against the committed transcript	~2 seconds
make verify	Compare wide-Q16.48 PostScript inference with the C oracle byte-for-byte	~2 seconds
make verify-f12	Compare F12 stored-weight inference with C and pin the checkpoint SHA-256	~2 seconds
make verify-train	Compare all 5,000 training steps, samples, and the wide checkpoint with C	~62 minutes
make verify-train-f12	Repeat the full training gate and verify F12-at-export	~45 minutes

The training targets also require input.txt. If it is absent, the Makefile uses the adjacent int-llm checkout's dataset download script.

Training

microgpt_train.ps trains the model from scratch entirely in PostScript: whole-sequence forward/backward, Adam with gradient clipping and bias correction, a CORDIC cosine learning-rate schedule, MGW v1 serialization, and the same 20-sample inference pass. All model math and optimizer state use Q16.48 signed integer arithmetic.

# train, save model_trained.mgw, sample, and compare both stdout and model
# byte-for-byte with a fresh 5000-step C oracle run:
make verify-train

# identical Q16.48 training/Adam, with F12 applied only while exporting:
make verify-train-f12

# run the PostScript trainer directly (input.txt must already exist):
gs -q -dBATCH -dNODISPLAY -dNOSAFER microgpt_train.ps

On Ghostscript 10.04.0, the full verification gate took 3,711.91 seconds (about 61m 52s). It produced 5,000 identical loss lines, 20 identical samples, and a byte-identical 115,576-byte model_trained.mgw. The fresh trained file also matched the committed model.mgw exactly (SHA-256 466cfe9dba7b888cdaa23dedf4b10351826795793448c8e95dcb0f7a61ed33eb).

Uniform-F12 stored weights

The F12 path changes checkpoint storage precision, not the runtime number system. uniform-f12.mgw is still the ordinary 115,576-byte MGW v1 container with signed int64 payloads, and inference still performs Q16.48 arithmetic. Each of the nine weight tensors is rounded to a multiple of 2^36 raw units (12 fractional bits, halfway cases away from zero); tokenizer.uchars and rng.state remain unchanged. This is deliberately not the packed-int16 MGWI format.

The committed checkpoint comes from int-llm-precision-ladder and has SHA-256 742cbd6d0b750bf3d164a23d97390171e3fe545ee9d87a2b0e843d3d8d1ae9f4. The _f12.ps files are tiny entry wrappers: they select the model/export mode and run the same inference or training core, so none of the transformer, optimizer, or fixed-point implementation is duplicated. During F12 training, the in-memory weights, gradients, Adam state, and all 5,000 updates remain bit-exact Q16.48; rounding happens only as the finished weight payload is written to model_trained_f12.mgw.

On Ghostscript 10.04.0, the full make verify-train-f12 gate took 2,707.29 seconds (about 45m 7s). Its 5,000 loss lines and 20 samples matched the fresh wide-Q16.48 C run byte-for-byte, and the exported checkpoint matched uniform-f12.mgw byte-for-byte at the pinned SHA-256 above.

Model trust and provenance

The loaders target the two committed, SHA-pinned MGW v1 files; they are not hardened parsers for adversarial checkpoints. Do not substitute untrusted PostScript or model files when running with -dNOSAFER.

466cfe9dba7b888cdaa23dedf4b10351826795793448c8e95dcb0f7a61ed33eb  model.mgw
742cbd6d0b750bf3d164a23d97390171e3fe545ee9d87a2b0e843d3d8d1ae9f4  uniform-f12.mgw

Files

• microgpt_infer.ps — inference, complete: Q16.48 fixed-point library, MGW model loader, transformer forward pass with KV cache, sampler. Pure PostScript.
• microgpt_infer_f12.ps — tiny entry wrapper selecting the canonical F12 checkpoint for the shared Q16.48 inference core.
• microgpt_train.ps — complete bit-exact Q16.48 training, MGW writer, and post-training sampler in pure integer PostScript.
• microgpt_train_f12.ps — tiny entry wrapper enabling F12-at-export for the shared Q16.48 trainer.
• model.mgw — trained MicroGPT weights from int-llm (14,272 parameters, names dataset), including tokenizer and the exact xorshift64 seed state.
• uniform-f12.mgw — the same trained weights rounded to the uniform F12 storage grid, still in the MGW v1 int64 container.
• expected_output.txt — the golden 485 bytes both implementations print.
• Makefile — the verification gate.
• LICENSE — Apache License 2.0.

Lineage

• Andrej Karpathy's microgpt — the algorithm.
• int-llm — the Q16.48 integer-only C implementation whose behavior is the oracle here.
• int-llm-postscript — the same computation expressed in the language your laser printer speaks.

Created by Nenad Mićić as part of the int-llm family of reproducible integer inference experiments.

License

Apache-2.0 © 2026 Nenad Mićić. See LICENSE.