Implement hardware-rendered graphics

[Zizin13]

Should decompile some of the 3dfx exe and see how it was done originally before redoing it from scratch.

[schlangz]

So far I couldn't find the proper glide library - did you have any luck on this?

[Zizin13]

I did get 3dfx running in dosbox before though I don't remember the steps off the top of my head. I don't think decompilation of the exe will help with this anymore, @deevus has done a lot of work refactoring the rendering code to work well with hardware acceleration and hot swapping between hardware and original software rendering modes while in game. Hardware rendering will probably just be all new code at this point.

[schlangz]

For some reason I found it EXTREMELY interesting today trying to figure out how the game renders its engine on the original 3Dfx hardware, so I spent some hours on fatfx.exe.

I was able to automap most of Glide SDK in IDA

From what I remember is that you guys have a pdb file for the original function names?

[deevus]

The debug symbols were only for a specific version. AFAIK there are no debug symbols for the 3dfx version.

[Zizin13]

Yep, the hash of the one exe released with symbols is in the readme. I've attached it here. FATAL.zip wdump.exe included in open watcom is the software you need to extract the symbols, or you can use @fonic's wcdatool that puts the wdump output into nicely formatted disassembly files. Here is an old video I made about it https://youtu.be/bG9tEZOSrQg

[schlangz]

Lession learned: never ever build open watcom inside WSL again... it took over half a day to complete.

Thanks for the fatal.exe, I extracted the symbols and applied the function names in IDA.
Unfortunately IDA overlooked some functions in its control flow but I still managed to automap 480 functions (from what I can see there are only 27 missing, not too bad).

I went ahead and mapped the functions from fatal.exe to fatfx.exe using Diaphora Plugin for IDA and without going less than 95% probability I was able to automap 409 functions from fatal.exe

The 3Dfx stuff:

• _sst1_* (Glide SDK)
• Glide_* (Glide SDK or new ROLLER functions exclusive to FATFX)
• Engine_* (fantasy names, either unmapped ROLLER functions or new functions exclusive to FATFX that are involved in using 3Dfx code, subject for renaming once identified)

[schlangz]

I added some basic bilinear filtering to polytex.c at the point where texels are sampled in polyt(). The implementation works reasonably well for vehicle textures, but several limitations became apparent due to the original renderer's architecture.

Bilinear off:

Bilinear on:

The problem is that the original renderer does not operate on RGB pixels.
Instead texture samples are palette indices: uint8_t texel = pTex[iV * 256 + iU];

For bilinear filtering we need:

• Sample four neighboring palette indices
• Convert each palette index to RGB
• Perform bilinear interpolation in RGB space
• Convert the interpolated RGB value back to the nearest palette entry

This requires a call to nearest_colour() for every filtered pixel, which is extremely expensive.
The software renderer was designed for nearest-neighbor sampling.

Bilinear filtering adds:

• 4 texture fetches per pixel
• 4 palette lookups per pixel
• Multiple floating-point operations
• Palette remapping back to an 8-bit index

The result is horribly slower than the original renderer, but with a lookup-cache to reduce the cost of repeated palette searches and the performance was fine. However as you can see in the screenshots above not all textures benefit from bilinear filtering.

Because the renderer is palette-based, high-contrast textures (track markings, sky textures) often lose sharpness and are suffering from palette remapping artifacts. The texture system does not store textures as independent linear 64×64 blocks, instead setmapsel() arranges textures into large 256-pixel-wide atlases:

• 64×64 textures are grouped 4 per row
• 32×32 textures are grouped 8 per row

Incorrect clamping causes texels from neighboring textures to bleed into each other.

GPUs provide bilinear filtering essentially for free, so for the ultimate hardware accelerated rendering solution we need:

• convert textures from 8-bit palette index to RGBA8888
• current mapsel system has to be replaced (or bypassed) by GPU texture objects
• instead of rasterizing scanlines in polyt() the renderer would submit triangles/quads with UV coordinates to the GPU

The GPU would perform clipping, interpolation and bilinear filtering automatically.
All the code is there in fatfx.exe but I wonder if reverse-engineering would be really faster than a new implementation.

I think in the meantime the sweetspot would be only activating bilinear filtering for cars.

Full bilinear:

Car-only bilinear:

What do you guys think?

[deevus]

I assume this is still software rendered since you said it is a change to polytex.c.
With regards to the GPU rendering pipeline, there is a bit of work remaining, however it can be staged. I was thinking something like this:

1. Start by blitting the full scrbuf to the GPU. Basically the software renderer but we're just rendering a full-screen texture to the GPU.
2. Move UI rendering out, so that scrbuf is only the game content. Render UI elements using GPU on top.
3. Pick some geometry that isn't GPU rendered e.g. player car.
4. Render it
5. Go to step 3

[schlangz]

Yes it's still all software rendering, I just wanted to see if there is something that could be done fast with low effort before the real implementation of the GPU render is complete.
I like your idea of migrating individual rendering components from the software renderer to the GPU incrementally.

Forgot to upload the polytex.c

[schlangz]

...cleaning up comments a little bit...

I have now mapped almost every function from fatal.exe to fatfx.exe in IDA and gave every Glide involved function a meaningful name, so I decided to let claude do the comparison between fatal.exe software renderer and fatfx.exe 3Dfx hardware renderer (I'm a bit lazy lately).

FATAL.EXE Software Renderer

Initialization

three_d_init_screen_ (0x34D28)
  ├── svgacpy_VESAmode_()          ← try 640×400 VESA first
  │   └── [success] → VESA linear framebuffer, 640×400, 8-bit
  └── [fail] → func2_setmodex_(128)  ← fall back to Mode X
      └── int 10h mode 13h → VGA sequencer+CRTC → 320×200 unchained (4 planes)


Per-Frame Render Path

three_d_updatescreen_ (0x35988)
  ├── viewport calc (view size setting → src rect in 8-bit framebuffer)
  ├── func2_clear_border_()         ← clears letterbox bars with 8-bit fill
  ├── three_d_draw_road_()          ← kicks off geometry pipeline:
  │   └── drawtrk3_DrawTrack3_ (0x42298)
  │       ├── drawtrk3_CalcVisibleTrack_()  ← frustum cull track segments
  │       ├── per-polygon loop:
  │       │   ├── drawtrk3_dodivide_()      ← geometric screen-space subdivision
  │       │   │   └── [optionally] drawtrk3_subdivide_()
  │       │   ├── polytex_POLYTEX_ (0x91178)   ← textured polygon
  │       │   │   ├── polytex_twpolym_()    ← affine span rasterizer (two-plane)
  │       │   │   └── polytex_polym_()      ← SOFTWARE SPAN RASTERIZER
  │       │   │       └── writes 8-bit indexed pixels → framebuffer
  │       │   └── polyf_POLYFLAT_ (0x95598)   ← flat-shaded polygon
  │       │       ├── polyf_twpoly_()
  │       │       └── polyf_poly_()         ← watcom_memset rows → framebuffer (8-bit fill)
  │       └── tower_DrawTower_()
  │           └── car_DisplayCar_()
  │               └── → polytex_POLYTEX_ / polyf_POLYFLAT_  (same rasterizer)
  ├── building_DrawBuilding_()      ← → polytex_POLYTEX_ / polyf_POLYFLAT_
  └── building_init_animate_ads_()

three_d_game_copypic_ (0x39C78)    ← HUD overlay + panel render into 8-bit framebuffer
  ├── func3_DrawCar_()              ← minimap/car sprites drawn into framebuffer
  ├── func2_print_*()               ← text rendered as 8-bit blits into framebuffer
  └── func2_test_panel_()           ← options UI into framebuffer

three_d_copypic_ (0x34B78)         ← PRESENT (CPU blit to VGA)
  └── svgacpy_svgacopy_()
      ├── [VESA] → memcpy 8-bit framebuffer → VESA linear ptr
      └── [Mode X] → write 4×VGA plane blit (sequencer port 0x3C4)

FATFX.EXE Glide Hardware Renderer

Initialization

three_d_init_screen_ (0x3BB6C)
  ├── svgacpy_VESAmode_()           ← still negotiates a video mode
  ├── func2_setmodex_()             ← VESA fallback (mostly unused in 3Dfx path)
  └── sets viewport floats:
      flt_D09B8=115.0, flt_D09C8=159.0   ← half-width/height for Glide clip

Engine_GlideInitHW_ (0x7FFB4)       ← 3Dfx HW init
  ├── Engine_DPMISimInt10_(0x3D00)  ← DPMI simulate int 10h for VESA detect
  ├── Engine_GlideInitVMode_ (0x7FF80)
  └── Engine_GlideSetMode3_ (0x8006C)

grGlideInit (0xB1558)               ← Glide SDK init
  ├── reads env vars (sst1InitGetEnv_)
  ├── registers Engine_GlideRenderScene_ as buffer-swap callback
  └── sets init flag MEMORY[0x2C6E80]

Engine_GlideLoadSprites_ (0x7EE30)  ← ONE-TIME asset upload (called from play_game_init_)
  ├── loads 13 sprite files (palette + bitmap)
  ├── Engine_GlideBuildPalette_ (0x7B2FC)  ← build color lookup tables:
  │   ├── word_116EF8[256]   = RGB565    (R5G6B5, no alpha — backgrounds)
  │   ├── dword_11544E[256]  = ARGB1555  (A=1, opaque — textures)
  │   └── dword_1159B0[256]  = xRGB8888  (32-bit, system-side ops)
  └── Engine_GlideUploadTexture_ (0x7C7DC)  ← per sprite:
      ├── converts 8-bit pixels → ARGB1555 via dword_11544E LUT
      ├── transparency: palette index 0 → alpha=0; else alpha=1 (bit 15)
      └── guTexAllocateMemory(format=11 = GR_TEXFMT_ARGB_1555, 256px wide strips)
          └── guTexDownloadMipMap()  ← DMA upload to TMU VRAM

Engine_GlideUploadTexture_ (0x7C7DC) ← also called per-track for track textures
  └── same 8-bit → ARGB1555 + guTexAllocateMemory(11) + guTexDownloadMipMap

Per-Frame Render Path

three_d_updatescreen_ (0x3C818)           ← FATFX game update loop
  ├── Engine_DrawZoom_ (0x41B1C)          ← zoom/transition effect (palette copy, network-aware)
  ├── three_d_draw_road_()                ← builds draw list (CPU-side)
  │   └── drawtrk3_DrawTrack3_ (0x49424)
  │       └── drawtrk3_DrawTrack3__body_ (0x49C90)  ← main polygon loop
  │           ├── drawtrk3_CalcVisibleTrack_()
  │           ├── drawtrk3_GlideSubdivide_ (0x9832C)  ★ FATFX-ONLY
  │           │   └── 3D subdivision WITH UV texture coords + double precision
  │           │       (replaces dodivide_ which had no UV)
  │           ├── polytex_POLYTEX_ (0x9ADC4)   ← [dword_9620 ≠ 0 → Glide path]
  │           │   ├── Engine_GlideDrawPoly_ (0x7DD18)
  │           │   │   ├── Engine_ClipPolygon_ (0x7E86C)
  │           │   │   └── grDrawPolygonVertexList (0xB46FC)  ← GPU submit
  │           │   └── Engine_GlidePolyFlipped_ (0x7C96C)
  │           │       └── grDrawPolygonVertexList
  │           └── polyf_POLYFLAT_ (0x9F648)    ← [dword_9620 ≠ 0 → Glide path]
  │               └── Engine_GlideDrawFlatPolyFull_ (0x7B4E8)
  │                   ├── Engine_ClipPolygon_
  │                   ├── grConstantColorValue()
  │                   └── grDrawPolygonVertexList
  ├── three_d_game_copypic_ (0x41464)     ← HUD/UI overlay (Glide text + panel)
  │   ├── func2_test_panel_()
  │   └── func2_prt_centrecol_()
  └── Engine_PresentFrame_ (0x41290)      ← present frame
      ├── Engine_DrawZoom_()
      ├── Engine_GlideDrawHUD_(112) × 2   ← HUD border boxes
      └── Engine_SwapBuffers_ (0x7B3E8)   ← triggers grBufferSwap

grBufferSwap (0xB0E20)                    ← calls registered callback:
  └── Engine_GlideRenderScene_ (0xC3000)  ← PER-FRAME GPU RENDER
      ├── grColorCombineFunction(1)        ← DECAL (texture color direct)
      ├── grAlphaBlendFunction(4,0,4,0)    ← alpha blend ONE/ZERO
      ├── grDepthBufferFunction(3)         ← LEQUAL z-test
      ├── per-view loop (splitscreen support via loc_425AC):
      │   ├── grGammaCorrectionValue()     ← per-view gamma (fade in/out)
      │   ├── grBufferClear(0, 0, 0xFFFF)  ← clear color=0, depth=max
      │   ├── PASS 2: Glide_SetupVertexTransform_ + Glide_RenderFrame_
      │   │          (far/background geometry, depth prepass)
      │   ├── PASS 0: Glide_SetupVertexTransform_ + Glide_RenderFrame_
      │   │          (main opaque geometry)
      │   ├── grDepthMask(1) + grDepthBufferFunction(7=ALWAYS)
      │   ├── Engine_GlideDrawCars_()      ← car polys (depth always-write)
      │   ├── grDepthBufferFunction(3=LEQUAL)
      │   ├── PASS 1: Glide_SetupVertexTransform_ + Glide_RenderFrame_
      │   │          (transparent/alpha geometry — first subpass)
      │   ├── grAlphaBlendFunction(4,4,4,0)  ← additive alpha blend
      │   ├── PASS 1 (alpha): Glide_RenderFrame_  (additive alpha layer)
      │   └── grBufferSwap(2) + sub_B2768() (PCI reselect)
      └── 2× clear+swap flush at end of scene

Comparison

┌───────────────────┬────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────┐
│      Aspect       │                       FATAL.EXE                        │                       FATFX.EXE                       │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Pixel format      │ 8-bit indexed (256-color palette)                      │ ARGB1555 (textures) + RGB565 (fill)                   │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Framebuffer       │ Software buffer in RAM                                 │ Glide double-buffer in TMU VRAM                       │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Present           │ CPU blit via svgacpy_svgacopy_ (or Mode X planes)      │ grBufferSwap → callback fires GPU render              │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Polygon submit    │ Software span rasterizer writes pixels to char* buffer │ grDrawPolygonVertexList / grDrawTriangle              │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Texture coords    │ Affine only (no UV per vertex) — polytex_polym_        │ UV per vertex via drawtrk3_GlideSubdivide_            │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Track subdivision │ dodivide_ — screen-space geometric only                │ GlideSubdivide_ — 3D with UV + double precision       │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Render passes     │ Single pass (painter's algorithm)                      │ 3 passes: far geo → opaque → transparent              │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Car drawing       │ func3_DrawCar_ (SW)                                    │ Engine_GlideDrawCars_ (Glide tris, depth always-pass) │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Transparency      │ Palette color 0 = background color skip                │ alpha=0 in ARGB1555 (bit 15 clear)                    │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ HUD/text          │ 8-bit pixel blit into framebuffer                      │ Engine_GlideDrawHUD_ via grDrawTriangle               │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Gamma             │ Fixed (palette DAC)                                    │ Per-view grGammaCorrectionValue (fade support)        │
├───────────────────┼────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Splitscreen       │ Not evident                                            │ Loop over loc_425AC views in GlideRenderScene         │
└───────────────────┴────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────┘

Pixel Format Detail

FATAL.EXE — 8-bit palettized:
framebuffer[y][x] = palette_index (0–255)
VGA DAC: palette[i] = {R6, G6, B6}   ← 6-bit per channel
Output: 256 colors from 18-bit palette

FATFX.EXE — ARGB1555 (GR_TEXFMT_ARGB_1555 = format 11 in Glide 2.x):
15  14 13 12 11 10  |  9  8  7  6  5  |  4  3  2  1  0
 A  |     R (5)     |      G (5)      |      B (5)
 1  = opaque
 0  = transparent (palette index 0)

Built at load time from the 8-bit game palette by Engine_GlideBuildPalette_ (0x7B2FC):
// ARGB1555 (stored in dword_11544E — used for textures + sprites):
pixel = 0x8000                      // alpha = 1 (opaque, always)
      | (palette[i].R & 0x3E) << 9  // R[5:1] → bits [14:10]
      | (palette[i].G & 0x3E) << 4  // G[5:1] → bits [9:5]
      | (palette[i].B & 0x3E) >> 1; // B[5:1] → bits [4:0]
// For transparent pixel (index 0): clear bit 15 → alpha = 0

// RGB565 (stored in word_116EF8 — used for solid fill/background):
pixel = (palette[i].R & 0x3E) << 10  // R[5:1] → bits [15:11]
      | (palette[i].G & 0x3F) << 5   // G[5:0] → bits [10:5]  ← 6-bit G!
      | (palette[i].B & 0x3E) >> 1;  // B[5:1] → bits [4:0]

It's actually ARGB1555 (1-bit alpha + 5R + 5G + 5B = 16 bits), only 1 alpha bit (fully opaque or fully transparent, no blending levels).
That's why the transparency system is binary (palette index 0 = invisible, everything else = solid).

Also I was curious why VGA functions reside in sound.c module, this is the answer I got:

This is a well-known DOS game development pattern — palette functions ended up in the sound module because both need the same thing: a reliable periodic timer interrupt.

In DOS, you don't have an OS scheduler. To get smooth palette fades, you need to step the DAC values at a fixed rate — typically 50 or 70 times per second. 
The sound system already reprograms the PIT (8253 timer, IRQ 0) to fire at a high rate for audio mixing. 
The sound_tickhandler_ (0xE6D bytes — enormous) is that IRQ 0 callback. 
Once you have a timer firing reliably, it's natural to drive palette fades from it too.

Look at what else lives in the sound_ module — it's not really a sound module, it's the interrupt-driven platform layer:

- sound_tickhandler_ — the actual IRQ 0 handler; drives everything below
- sound_fade_palette_ (0x375 bytes) — steps the palette one increment toward a target; called from tickhandler each tick
- sound_set_palette_, sound_resetpal_, sound_blankpal_ — DAC control
- sound_ReadJoys_, sound_check_joystickpresence_, sound_updatejoy_ — joystick polling (also timing-sensitive in DOS — joystick timing requires a known delay)
- sound_readuserdata_, sound_readsoundconfig_ — reads settings that cover audio and controls
- sound_claim_ticktimer_ / sound_release_ticktimer_ — the functions that install/remove the IRQ 0 hook

The joystick being there is the second giveaway: DOS joystick reading works by timing how long a capacitor takes to discharge (ISA game port), so you need the same timer you use for audio. 
The programmer handling IRQ 0 for SOS naturally owned joystick reading too.

So the module should really be named something like platform_ or irq_ — it's the low-level interrupt substrate. 
The name sound_ stuck because SOS (the audio library it wraps) was probably the first thing integrated, and everything else got added around it.

[schlangz]

I have a working GPU hardware renderer implementation for ROLLER, @Zizin13 @deevus would you be interested in my code?

[Zizin13]

@schlangz sure, open a PR and make sure it is able to merge into the master and I'll check it out

[schlangz]

I'm new to Git and Github (my professional life only worked with CVS, SVN and Perforce), from what I understood is I have to create my own fork, commit my changes there and then submit a PR to ROLLER? In case of bugfixes and changes (I'm not happy with the clouds yet) how does it work? I fix it in my fork and then again submit a PR to ROLLER?

Attached some eye-candy with 8xMSAA and anisotropic filtering for motivation:

[Zizin13]

Yep that's exactly how it works. My current job uses git/bitbucket but I used perforce at my previous job too, has its advantages as well as disadvantages compared to git. I would add a texture filtering combobox in the debug menu that allows users to select nearest-neighbor. I think that will fix the clouds and give it the chunky whiplash look, that's what I used in track editor.

[schlangz]

I have that, but the clouds look weird with any setting :)