Replace Node/Socket.IO/Pug with a measured Zig epoll server

[JanTuck]

Executive summary

This PR replaces the production Node.js/Express/Socket.IO/Pug server with a single, static, Linux-native Zig executable and takes the opportunity to finish the game around it: real lobby isolation, bounded capacity, compact binary networking, deterministic connection lifecycle, password-protected lobbies, a Classical mode, a redesigned lobby/game UI, a full-viewport authoritative board, and a deployment image small enough to be mistaken for a rounding error.

The short version is:

• one Zig executable serves HTTP, WebSockets, embedded assets, lobby state, and the simulation;
• zero Node.js runtime dependencies are required in production;
• raw RFC 6455 WebSockets replace Socket.IO/Engine.IO;
• versioned binary snapshots replace repeated JSON object graphs;
• one edge-triggered epoll reactor plus six game workers served the measured 12,000-player run;
• that run held 12,000/12,000 players, with zero join failures, zero sampled disconnects, 20.39 MiB RSS, and a stable 66.675 / 68.225 / 69.359 ms p50/p95/p99 delivery cadence;
• the game now has actual isolated lobbies, capacity limits, lifecycle cleanup, passwords, Classical mode, and protocol/security validation instead of a hopeful collection of global maps and console logs;
• start-docker.sh builds the Zig executable on the host and publishes a compiler-free, runtime-only scratch image on port 9687.

This is intentionally a large PR. It is not a cosmetic language port. It is the result of repeatedly profiling, benchmarking, deleting, bounding, and adversarially testing the complete server/client path until the remaining optimization ideas were either noise-scale, harmful elsewhere, or not worth their complexity.

Friendly-fire disclaimer

Jan and I are friends, so I am going to describe the old stack with the tenderness it deserves: very little.

Node.js is not intrinsically incapable of serving a multiplayer game. It is, however, spectacularly good at letting a tiny project accumulate an HTTP framework, a WebSocket abstraction layered over another WebSocket abstraction, a template engine, two body parsers, cookie middleware, logging middleware, error middleware, transitive dependency archaeology, and a node_modules directory with the gravitational pull of a small moon—all before the snake has moved one cell.

The actual problem was not JavaScript syntax. The problem was using a general-purpose web application stack on a fixed-cadence, stateful, high-fan-out simulation hot path and then representing the hottest data as verbose object-shaped JSON. Zig makes the new architecture explicit and enforceable, but the gains come from the architecture: bounded state, raw sockets, compact protocols, controlled ownership, fewer syscalls, fewer allocations, fewer copies, fewer threads, and fewer bytes.

A loving autopsy of the old Node stack

The original package.json needed these production dependencies:

body-parser
cookie-parser
debug
express
http-errors
morgan
pug
rcolor
socket.io

That is an impressive committee for a server whose essential responsibilities are:

1. serve a handful of static files;
2. accept a player;
3. move snakes 15 times per second;
4. tell browsers what moved.

Instead, the old server booted Express, created a separate HTTP server, attached Socket.IO, configured a Pug view directory, installed Morgan, installed body-parser twice in spirit (urlencoded and JSON), installed cookie-parser despite the game not needing cookie state, served two static roots, forwarded errors through http-errors, and finally rendered the error through Pug.

The Pug situation

The Pug usage deserves its own archaeological plaque.

The game itself was served as static HTML from client/. Pug was configured as the application view engine, but its meaningful runtime role was primarily to render the error path through views/error.pug, views/layout.pug, and views/layout_error.pug. In other words, an entire template compiler and its dependency tree were invited into production so a 404 could put on formalwear.

This is not progressive enhancement. This is making an error page pass through a templating language while the actual application walks around it and uses plain HTML.

The new server responds directly with bounded static or generated bodies. There is no template compiler, no dual static/template rendering model, no view lookup, no template runtime, and no opportunity for views/layout_error.pug to become the most architecturally pampered file in the repository.

Pug has not been “optimized.” Pug has been given the greatest optimization available to dead weight: deletion.

The Express middleware buffet

The previous process paid for middleware and abstractions the game did not need:

• cookie-parser was registered without a cookie-backed game design.
• body-parser was an extra dependency even though modern Express already provides body parsing helpers.
• Morgan logged requests in development format.
• debug remained in the dependency surface without being the production observability design.
• http-errors manufactured an object so Express could pass it through middleware so Pug could turn it back into HTML.
• two express.static roots and bespoke sendFile routes overlapped responsibilities.
• GET /generateid logged req.body, which is an especially avant-garde interpretation of both GET semantics and lobby creation.
• join requests and the entire lobby map were printed to stdout, turning routine traffic and internal state into accidental logging policy.

The Zig server has an explicit route table and parser limits. It handles the small protocol this application actually owns. Static assets are embedded at compile time. There is no middleware conveyor belt and no runtime filesystem dependency.

Global state pretending to be lobbies

The old code declared global SOCKET_LIST, LOBBY_LIST, BOT_LIST, FOOD_LIST, and one global food. LOBBY_LIST contained "12345" -> null, while connected players lived in the one global socket map. io.emit('updateFood', ...) broadcast the shared food update to every connection.

That is not lobby isolation. That is one room wearing a lobby-shaped nametag.

The new implementation has independent lobby ownership, player lists, PRNG state, food, bonus apples, drops, golden apples, timers, metrics, passwords, mode flags, and worker assignment. A player joins exactly one lobby. Roster and world updates are scoped to that lobby. Empty generated lobbies are reaped after a bounded idle lifetime. Capacity is enforced globally and per lobby.

The game loop and fan-out shape

The old simulation used one global setInterval(..., 1000 / 15), walked every socket, built a full segment array containing player objects and snake arrays, then emitted that whole structure to every socket through Socket.IO. That means the hottest operation in the application asked JavaScript to repeatedly walk object graphs, serialize field names and numbers to text, pass them through Socket.IO/Engine.IO framing, allocate intermediary data, and make every browser parse the same verbosity back into objects.

It also mixed mutation, collision handling, deletion, movement, and publication in one loop. After removing a player for a collision, the old loop could continue into player.updatePosition() and append that player to the outgoing segment for the current tick. This is exactly the kind of lifecycle ambiguity that disappears when ownership and state transitions have to be written down explicitly.

The new server uses:

• a 15 Hz authoritative simulation;
• one edge-triggered epoll reactor for HTTP and WebSocket I/O;
• lazy game workers packing up to 128 lobbies per worker;
• stable, bounded lobby player arrays;
• collision indexing over the canonical board;
• immutable shared snapshot frames with exact reference accounting;
• direct nonblocking writes when sockets are ready;
• bounded per-connection queues only when backpressure requires them;
• writev batching for coalesced HTTP pipelines;
• explicit ownership and teardown rules for connections, memberships, lobbies, frames, and worker assignments.

At 750 active lobbies, the final measured server required six game workers, not 750 lobby threads and certainly not two threads per client.

Why Zig is the right production language here

Again, “Zig fast, Node bad” is not a sufficient engineering argument. The useful question is what the language lets this particular server guarantee.

Deterministic data and ownership

The game has small, known caps and fixed wire formats. Zig lets those constraints exist as actual types, compile-time checks, stack arrays, bounded buffers, and explicit ownership instead of conventions spread across mutable JavaScript objects.

Examples:

• the browser-supported lobby cap is 16 players;
• the canonical board is 128 x 72 cells, 9,216 cells total;
• cell coordinates fit exactly in u8 wire fields;
• snapshot cell counts fit in u16;
• join passwords are at most 64 UTF-8 bytes;
• WebSocket application frames, input buffers, output bytes, and queued item counts are independently bounded;
• worker scratch arrays are fixed at the maximum lobby population;
• the collision index is compile-time checked against the worker stack budget.

These are executable invariants, not comments begging future code to behave.

Direct access to the Linux primitives we actually need

The production server is Linux-native and uses epoll. One reactor owns descriptors and connection I/O. Accepted sockets inherit TCP_NODELAY from the listener. Final descriptor close performs the kernel's epoll cleanup, avoiding redundant per-connection syscalls. Ready sockets use direct nonblocking writes; retained output storage is used only when the peer actually applies backpressure.

There is no libuv abstraction to negotiate with, no event-emitter layer in the hot path, and no Socket.IO protocol transforming one binary-capable transport into an event-shaped compatibility product.

A static deployment artifact

Zig produces one stripped, asset-embedded executable. The Dockerfile does not compile it. start-docker.sh builds on the host, verifies that the ELF has no dynamic interpreter, then copies only that executable into scratch.

The container runs:

• as UID/GID 65532;
• read-only;
• with all Linux capabilities dropped;
• with no-new-privileges;
• on container port 9687;
• with an HTTP smoke check before reporting success.

No Node runtime. No npm. No shell. No package manager. No source tree. No node_modules. No Pug compiler quietly meditating in production in case a 404 needs emotional support.

Network protocol: stop mailing the same field names 15 times per second

The hot path no longer uses Socket.IO events or JSON snapshots.

Client input

Direction and visibility input use tiny fixed binary packets. Lobby join uses one bounded binary packet carrying lobby ID, username, and optional password lengths plus bytes. Parsers reject fragmentation, reserved bits, invalid UTF-8, noncanonical WebSocket lengths, malformed close frames, oversized payloads, partial application packets, and trailing garbage.

Snapshot v4

The server sends a compact, versioned binary snapshot:

• immutable roster metadata is sent separately;
• periodic keyframes carry complete recoverable state;
• deltas carry direction/mode transitions instead of resending absolute body coordinates;
• kind and player count share one byte;
• bonus count, drop count, and golden presence share one byte;
• sequence continuity is wrapping-u16 exact;
• the browser validates the entire variable-length frame before committing visible state;
• hidden tabs receive reduced-cadence recovery keyframes instead of full foreground fan-out.

For the representative 16-player, 18-cell stream, snapshot v4 averages 29.40 bytes/frame, down from 34.40 bytes/frame in v3, a further 14.5% reduction. Against v2, direction deltas reduced moving-stream bytes by 47–49% for short snakes.

The older format-selection microbenchmark shows why the original JSON object stream had to go:

Representative 16-player tick	Payload	Encode	Parse
Legacy JSON objects	7,164 B	21.4 us	33.2 us
Binary snapshot v1	725 B	1.366 us	0.310 us

That is roughly 89.9% fewer repeated bytes, 93.6% less encode time, and 99.1% less parse time in that microbenchmark. The production v4 format is smaller still for common delta frames.

JSON remains where it belongs: infrequent control messages such as initialization, roster changes, errors, and activity feed entries. /debug/stats uses Zig's typed std.json serializer; manual serialization was measured and rejected when it did not justify its complexity.

Measured production-shaped result

The final capacity run was measured on Linux, AMD Ryzen 7 5700G, Zig 0.16.0, -O ReleaseFast -fstrip. Server and load clients ran over loopback. Sixteen Node load-generator processes opened the same raw WebSocket protocol as browsers and parsed every snapshot.

At 12,000 concurrent players across 750 lobbies:

Metric	Result
Players held	12,000 / 12,000
Join failures / sampled disconnects	0 / 0
Snapshots received	181,931.2/s
Client cadence p50 / p95 / p99	66.675 / 68.225 / 69.359 ms
Process CPU	115.18%
RSS	20.39 MiB
VM	25.54 MiB
Application egress	32.30 MB/s
Average measured frame	177.53 B
Lobby tick p50 / p95 / p99	0.186 / 0.226 / 0.244 ms
Game workers / process threads	6 / 7 total

That is approximately 1,782 bytes of process RSS per connected player, including shared server state. Kernel socket memory and load-generator memory are not included.

These numbers are documented with methodology and caveats in docs/BENCHMARKS.md. I am deliberately not inventing a fake current Node-versus-Zig percentage: the original Node stack was not rebuilt around the exact raw binary protocol and rerun side by side. The defensible claim is that the current Zig implementation meets the measured capacity above, while the old Node/Socket.IO/JSON architecture was structurally paying for work that the current protocol deletes.

Additional measured optimizations

This was not one heroic rewrite followed by a victory lap. The implementation went through repeated measured passes.

HTTP parsing and output

The original incremental parser candidate compacted the receive buffer after every request, producing quadratic copies for large pipelines. The final parser advances an offset and compacts once; response segments drain in writev batches of up to 64.

For 6,240 minimal pipelined requests:

• original: 129.133 ms;
• one input compaction: 39.096 ms;
• input compaction plus output batching: 7.289 ms.

That is 94.4% faster than the original parser shape, while preserving response order, bodies, partial trailing requests, HTTP/1.0, half-close behavior, and backpressure.

Lobby reaping

Ordered removal of 4,095 expired lobby entries shifted 8,382,465 later map entries—at least 191.9 MiB of key/value movement. Allocation-free swap removal reduced the measured reap from 14,712.030 us to 34.555 us, a 425.76x speedup.

Lobby player container

A capped 16-player lobby does not need a string hash map when the hot operations are stable iteration, append, and known-pointer removal. Replacing it with a pointer list reduced requested backing memory by 80.9%, made fill 3.49x faster, and made ordered remove/refill 5.88x faster.

Snapshot and tick scratch

Bounded stack arrays removed 22,500 arena-allocation API calls per second at 750 lobbies. Realtime sampling was hoisted from each lobby to each packed worker tick. The focused benchmark measured:

• scratch work: 77.0% reduction;
• realtime sampling: 99.2% reduction.

Connection churn

The server configures listener TCP_NODELAY once and relies on final close for epoll deregistration, removing one setsockopt and one epoll_ctl per connection: 24,000 syscalls across a 12,000-connection establish/teardown cycle.

Maintenance reclaims poisoned, idle, and heartbeat-expired connections using fixed stack batches rather than allocating a temporary list under the exact pressure condition where allocation is least welcome.

Output queues

Backpressured queues now cap live bytes and item count, compact released descriptor prefixes, and release oversized idle descriptor arrays. Previously, a slowly progressing connection could retain consumed metadata indefinitely—roughly 31 MiB/day per connection at 15 snapshots/s even though the bytes themselves had already gone out.

Correctness and security work included

The migration is not “fast server, same bugs.” It adds or hardens:

• real lobby isolation;
• global, per-lobby, and lobby-count capacity limits;
• lazy worker allocation and release;
• exact player/lobby lifecycle teardown;
• periodic heartbeat and timeout handling;
• generic authentication failure responses that do not reveal protected lobby existence;
• optional lobby passwords stored as salted SHA-256 digests and compared in constant time;
• exact UTF-8 validation and a 64-byte password limit;
• immutable Classical mode, disabling supply drops, bonus apples, and golden apples;
• WebSocket canonical framing and bounded parser state;
• malformed/partial packet rejection;
• HTTP request/body/input limits;
• bounded WebSocket application, input, output-byte, and output-item queues;
• security headers (nosniff, frame denial);
• transactionally validated snapshots;
• correct wrapping sequence recovery;
• collision bounds covering the full canonical board;
• central-half player spawning so a new player has reaction room instead of materializing one keypress from a wall;
• growth behavior that remains unchanged on allocation failure;
• same-socket rejoin after death;
• hidden-tab delivery throttling with foreground resynchronization.

Client and UX work

The server rewrite is accompanied by a complete client pass rather than leaving a fast backend attached to a 2019 scaffold.

• Landing page redesigned as a restrained game/control console.
• HUD rebuilt for readable player state, standings, lobby mode, event feed, and responsive layouts.
• Game-over state is now an accessible DOM dialog with focus management and a real Retry control, with a canvas fallback.
• The authoritative board is 128 x 72 cells / 2048 x 1152 logical pixels.
• The client restores the legacy full-viewport canvas behavior: the entire authoritative board maps to the viewport, so there are no fake playable margins and every visible edge is the actual collision edge.
• In-world labels and pickups are more readable.
• Left/right steering follows the requested horizontal behavior:
  • moving right: Left turns up, Right turns down;
  • moving left: Left turns down, Right turns up;
  • moving vertically: Left and Right remain absolute;
  • rapid turns mirror the server's two-turn queue.
• Password handoff survives create/join/reload/reconnect without putting the password in the URL.
• Classical/Arcade mode is visible in the HUD.
• Reduced-motion behavior is honored.
• The render loop sleeps on the join screen and after game-over particles drain instead of requesting 60 idle callbacks every second forever.

GSAP was removed in favor of the Web Animations API. Startup JavaScript dropped by 58.4% in the measured graph at that stage, removing a 72,925-byte render-blocking dependency used for four transitions.

Repository cleanup

The repository is now honest about what it ships:

• retired Node, Bun, Go, and Rust server implementations are gone;
• dead overlay/layout modules are gone;
• the unused Pug/public error-site parallel universe is gone;
• unused image assets are gone;
• .idea/deployment.xml is gone;
• benchmark source remains tracked;
• generated binaries, caches, profiles, and result captures are ignored;
• the asset manifest contains only served production assets;
• benchmark history and caveats remain documented instead of being rewritten into marketing.

Node still exists as an optional development/load-test tool because it is useful there. This PR is not a religious purge. It is a production-runtime eviction.

Verification

The final branch passes:

• 39 Zig unit/integration tests;
• transport protocol tests;
• password/create/join/reconnect interaction tests;
• production snapshot decoder tests, including board-edge and malformed-frame cases;
• horizontal-aware steering tests;
• native motion/HUD tests;
• render-loop lifecycle tests;
• embedded client dependency/asset graph tests;
• black-box HTTP, raw WebSocket, lobby, movement, collision/death, isolation, and same-socket rejoin parity;
• ReleaseFast build;
• runtime-only Docker build and HTTP smoke test.

The benchmark suite additionally covers capacity, 4,000 idle sockets, malformed traffic, connection churn, paused readers, memory recovery, lobby lifecycle, snapshot encoding, HTTP pipelines, lobby reaping, worker balance, and player-container behavior.

Review guide

The easiest review order is:

1. README.md — architecture and how to run it.
2. docs/SPEC.md — exact HTTP/WebSocket/game protocol and ownership rules.
3. docs/BENCHMARKS.md — methodology, measurements, rejected ideas, and caveats.
4. servers/zig/src/config.zig — centralized bounds and protocol constants.
5. servers/zig/src/main.zig — reactor, lifecycle, HTTP, lobby/workers, and game loop.
6. servers/zig/src/websocket.zig — RFC 6455 parser/writer.
7. servers/zig/src/snapshot.zig — snapshot v4 encoder.
8. servers/zig/src/collision.zig and model.zig — simulation data and collision behavior.
9. client/js/transport.js, snapshot.js, and rendering.js — browser side of the protocol.
10. start-docker.sh and Dockerfile — deployment boundary.

Operational notes

• Production target is Linux because the server uses epoll.
• Default HTTP/WebSocket port in the Docker flow is 9687.
• Terminate HTTPS at a reverse proxy and forward to the container.
• Node is needed only for the optional benchmark/test drivers.
• The server defaults to 100 global players for conservative local operation; capacity is configurable through documented SNEK_* environment variables.
• SNEK_DEBUG=1 exposes benchmark statistics and should be treated as an explicit operational choice.

Final verdict on the old stack

The Node version was a perfectly recognizable prototype: quick to begin, easy to poke, and held together by enough npm packages to qualify for municipal funding. That is fine for proving the game exists.

It was not a good final production architecture for a multiplayer simulation whose dominant operation is “serialize and fan out bounded state 15 times per second.” Express was doing routing theater, Socket.IO was charging a compatibility tax, JSON was repeatedly mailing field names to browsers that already knew the schema, global maps were cosplaying as lobbies, and Pug was sitting in the corner compiling an error page with the confidence of a technology central to the product.

The Zig server does less. That is the point.

It allocates less, copies less, parses less, formats less, locks less, schedules fewer threads, performs fewer syscalls, sends fewer bytes, retains less state, and ships less software. It also now has the tests and measurements to prove where those claims are real—and the documentation is explicit where a comparison would not be apples-to-apples.

Please merge this before Pug notices we changed the locks.

[DarkEyeDragon]

10/10 AI Slop

[JanTuck]

10/10 AI Slop

10/10 AI slop, but the slop holds 12k players in 20MB of RAM so unfortunately I'm keeping it