Skip to content

Add the GLB and glTF 3D model FileDef families - #5782

Draft
lukemelia wants to merge 16 commits into
mainfrom
cs-12243-3d-model-formats-glb-gltf-stl-3mf
Draft

Add the GLB and glTF 3D model FileDef families#5782
lukemelia wants to merge 16 commits into
mainfrom
cs-12243-3d-model-formats-glb-gltf-stl-3mf

Conversation

@lukemelia

@lukemelia lukemelia commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Part of CS-12243 (3D model formats). Stacked on #5658 (StlDef/ThreeMfDef, CS-12053/CS-12054) — base this PR against that branch so the diff shows only the glTF additions. Per that decision, GLB/glTF are added in #5658's existing style (Three.js ThreeDModelDef leaves), not a separate shared-shell rebuild.

What

Adds the two glTF forms so .glb and .gltf get the same interactive orbit viewer and inspector as STL/3MF.

  • GlbDef / GltfDef — thin ThreeDModelDef leaves. They inherit the shared Three.js viewer (whose GLTFLoader default path already renders both the binary and JSON forms), the fitted thumbnail, and the isolated body. Each adds a gltfMetadata field and an extension-guarded, size-capped extractAttributes, mirroring StlDef.
  • gltf-meta-extractor.ts — reads metadata from the glTF structure itself rather than a geometry scan: a glTF describes its accessors, so vertex/triangle counts, mesh/material/node/animation/texture counts, and bounding-box dimensions come straight from the header — the JSON of a .gltf, or the JSON chunk of a .glb (the BIN chunk is never touched). This directly delivers the ticket's "vertex/triangle counts, bounding box." Non-glTF bytes → undefined → FileContentMismatchError → graceful fallback.
  • Routing .glb → GlbDef, .gltf → GltfDef.

Tests

  • Extractor unit tests (added to model-meta-extractor-test.ts): both container forms, indexed / non-indexed / triangle-strip counts, bounding box, and the non-glTF fallback — all fixtures built in-code, no binaries.
  • Code-ref routing assertions for GlbDef/GltfDef.

Validation

  • ember-tsc --noEmit (host): 0 errors; base template-lint clean.
  • Extractor logic cross-checked in Node against the in-code fixtures (21/21).
  • Did not boot the browser locally — CI runs the QUnit suite.

Notes / limitations

  • A .gltf that references external .bin/texture resources renders its metadata fine but the live viewer (parsing with an empty resource path) will fall back to the thumbnail for the 3D scene; self-contained .gltf (embedded base64) and .glb render fully.
  • glTF distances are nominally meters but files routinely ignore that, so dimensions is presented unitless (matching STL's honesty), and displayUnit stays inherited-empty.

🤖 Generated with Claude Code

FadhlanR and others added 14 commits August 12, 2026 18:34
@FadhlanR @claude
Add StlDef and ThreeMfDef FileDef subclasses for 3D model files
c041209 
Introduce a shared ModelDef base (extends FileDef) with two leaves:
StlDef (.stl) and ThreeMfDef (.3mf). Each extracts format metadata in
extractAttributes (pure-JS parsers ported from the handoff realm) and
renders a deterministic inline SVG silhouette across atom/fitted/
embedded/isolated — no WebGL, prerender-safe.

- ModelDef: shared Model3DInfoField scene facts, silhouette preview,
  and a thumbnailUrl seam for the shaded-PNG follow-up (CS-12401).
- StlDef: ASCII/binary detection, facet/normal/degenerate counts,
  color-data flag, bounding-box extents.
- ThreeMfDef: OPC ZIP unzip (fflate), model-part + slicer-config parse
  (objects, materials, plates, print parts, extruders, bounds).
- Register .stl/.3mf in file-def-code-ref; add fflate dependency.

Shaded raster PNG previews are intentionally deferred to CS-12401; the
SVG silhouette is the preview until then.

CS-12053, CS-12054

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FadhlanR
Fix fflate runtime shim and glint template type error
79d3028 
- Register `fflate` as an async module shim in host externals.ts and add
  it to host deps. Bare imports in realm-served base modules resolve only
  via the shim registry, not node_modules, so the extractor's
  `import 'fflate'` would 404 at runtime without this.
- Render 3MF materialNames via a joined getter instead of interpolating a
  string[] directly (glint: string[] is not a ContentValue).
- prettier formatting.

CS-12053, CS-12054
@FadhlanR
Render live WebGL 3D viewer in embedded/isolated
531ef3d 
isolated + embedded now mount an interactive three.js orbit viewer
(ModelViewer + renderModel modifier), matching the handoff prototype.
three.js + OrbitControls + STL/3MFLoader load lazily from a CDN
(esm.sh) at client render time only — the sanctioned Boxel pattern for
card libraries — so the engine never runs during extraction/indexing.

- fitted keeps the pure SVG silhouette (collection-tile budget).
- The silhouette is the loading placeholder and the fallback whenever
  WebGL/the CDN engine is unavailable (e.g. prerender), so the static
  prerendered HTML degrades gracefully.
- Full GPU teardown on unmount (dispose + forceContextLoss).

CS-12053, CS-12054
@FadhlanR
Fix cross-origin CORS on model viewer byte fetch
cbee366 
The WebGL viewer's `fetch(url, { credentials: 'include' })` was blocked
cross-origin (host 4200 → realm 4201): a credentialed CORS request is
illegal against the realm's wildcard Access-Control-Allow-Origin, so the
STL/3MF bytes never loaded and every model fell back to the silhouette.

Drop `credentials: 'include'` — realm auth is carried by an Authorization
header injected by the host auth service worker on the GET (the same path
that lets <img src> load realm images), not by cookies. Verified in the
browser: complex STL and moderate 3MF now render in the live orbit viewer.

CS-12053, CS-12054
@FadhlanR
Add STL/3MF example files to experiments realm
0c50c37 
Six sample model files (stl + three-mf, simple/moderate/complex) under
experiments/model-samples/ so StlDef/ThreeMfDef can be exercised manually
in the app — each is indexed directly as its FileDef subtype.

CS-12053, CS-12054
@FadhlanR
Match handoff-realm isolated styling for model FileDefs
c88878a 
Rework the isolated view to mirror the handoff prototype: a header bar
with a mono extension pill, a bordered live-viewer stage, and a two-column
(stage + property inspector) body that collapses to one column on narrow
containers. Metadata reads as grouped "3D model" / "STL mesh" / "3MF
package" sections with uppercase mono labels; the viewer hint is a small
mono chip bottom-right.

Also aligns tokens with the design conventions: semantic role tokens
(--foreground/--muted-foreground/--card/--muted/--border) instead of
numbered palette values, rem units, and no `font` shorthand.

CS-12053, CS-12054
@FadhlanR
Box the isolated inspector with per-row separators
c0aa212 
Match the handoff realm's inspector: wrap the property groups in a
bordered, rounded --card panel and separate rows with hairline top
borders (first row none), with mono uppercase group headings. Applied to
the shared "3D model" group and the leaf STL/3MF groups.

CS-12053, CS-12054
@FadhlanR
Render live 3D in fitted (client-side), lazy + non-interactive
35f4473 
Fitted now mounts the WebGL viewer instead of only the silhouette, but:
- lazy via IntersectionObserver — the engine boots only when the tile is
  on-screen, so a grid of many models doesn't exhaust the browser's WebGL
  context budget; off-screen tiles keep the silhouette.
- non-interactive — no orbit controls, no gesture trapping, no hint, so
  fitted tiles render a static model and the enclosing grid scrolls
  normally.

During prerender/indexing the engine never loads (no CDN/WebGL), so fitted
still falls back to the silhouette there. Verified in the browser across
badge/strip tile sizes.

CS-12053, CS-12054
@FadhlanR @claude
Address review: bounded parsers, size cap, dedup, unit tests
e2e16f3 
Extract the STL/3MF metadata parsers into pure `.ts` modules
(stl-meta-extractor / three-mf-meta-extractor, mirroring
png-meta-extractor) so they're unit-testable without the card-api
harness, and harden them per review:

- STL parsing is now a single streaming pass over the DataView with no
  per-vertex array, so index-time memory stays flat regardless of model
  size; degeneracy is counted per facet as read, fixing the misaligned
  count when non-finite vertices were dropped.
- parseThreeMf returns undefined for any unparseable input (non-ZIP,
  no model part, malformed XML) so the leaf falls back cleanly to base
  FileDef; removed a redundant modelPart regex clause.
- Both leaves take an optional fileSizeLimitBytes, defaulting to the
  realm's standard DEFAULT_FILE_SIZE_LIMIT_BYTES. Over the cap they skip
  the parse and return base attributes only (keeping the 3D type + live
  viewer), rather than throwing and demoting to a plain file.
- Share Model3dData and getExtension; replace the ~40-line inspector
  style block duplicated across the three isolated templates with a
  data-driven ModelInspectorSection component.
- Document that model3d.vertices counts vertex records as the format
  stores them (not comparable across formats).
- Add unit tests for parseStl, parseThreeMf, and silhouettePath.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FadhlanR @claude
Add end-to-end file-extract acceptance test for STL/3MF
808e947 
Drives the real render/file-extract route (the indexer's path) over an
ASCII STL and a real 3MF ZIP served from a test realm, asserting the
file-meta search doc carries the parsed model3d / stlMetadata /
threeMfMetadata, plus the mismatch-fallback for a .stl whose bytes
aren't STL.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FadhlanR @claude
Fix test-assets build: move silhouettePath to a card-api-free module
4514529 
The "Build test web assets" (rolldown) job failed with MISSING_EXPORT
cascading from card-api.gts: the unit test value-imported silhouettePath
from model-file-def.gts, which pulled model-file-def -> card-api into the
host test bundle, where card-api's long-standing loose type imports
(CodeRef/ToolContext/LocalPath/SingleCardDocument imported without the
`type` keyword) become hard rolldown errors.

Move silhouettePath into a pure model-silhouette.ts (no card-api
dependency) and import the test from there; model-file-def re-exports it
for back-compat. No host test now value-imports a card-api-chained .gts,
so card-api is no longer bundled into the test assets.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FadhlanR @claude
Simplify 3D previews: static cube fitted, header-only extraction, cli…
3073b30 
…ent-side size

Rework STL/ThreeMf FileDefs per review feedback:

- Rename ModelDef -> ThreeDModelDef (and model-file-def.gts ->
  three-d-model-def.gts) so the base class no longer reads like a shared
  base type.
- Fitted view now renders a static cube icon + filename (mirroring the audio
  FileDef) instead of a live per-tile WebGL viewer, so a grid of tiles can no
  longer exhaust the browser's WebGL context budget. The shaded-thumbnail seam
  (`thumbnailUrl`) is kept for CS-12401.
- Drop the deterministic SVG silhouette (model-silhouette.ts) and the
  index-time bounding box (sizeX/Y/Z). Physical dimensions now come from the
  live client-side viewer, which reads the true, transform-correct bounds off
  the loaded geometry — so the 3MF-transform approximation disappears too.
- STL extraction is header-only: it reads the binary header (facet count,
  COLOR= flag) or the ASCII prologue (solid name) and never scans the facet
  body.
- 3MF extraction is a bounded, DOM-free prologue read: fflate's filter
  decompresses only the `.model` + `model_settings.config` entries (skipping
  embedded thumbnails/textures and refusing oversized entries as a ZIP-bomb
  backstop), and metadata is regex-read from the model part's prologue rather
  than DOM-parsing the geometry. Also validates the 3MF `<model>` core root so
  unrelated `.model` XML is rejected.
- Thread the realm's configured fileSizeLimitBytes through
  FileDefAttributesExtractor -> the leaf extractAttributes, so the size cap
  tracks the write-path limit instead of a hard-coded default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FadhlanR @claude
Gate the 3D viewer on prerender-vs-live; release WebGL contexts off-s…
b97f29e 
…creen

Reframe the WebGL/static split around render context rather than format, and
fix the context-budget leak the review flagged:

- Prerender gate: `ModelViewer`'s modifier no-ops when
  `globalThis.__boxelRenderContext` is set (server-side indexing/prerender), so
  WebGL and the CDN engine import are never started there — the static thumbnail
  is the deterministic prerender representation, instead of relying on a doomed
  CDN import failing.
- Viewport gate: the viewer now lazy-boots when the element scrolls on-screen
  and disposes its WebGL context (renderer.dispose + forceContextLoss) when it
  scrolls off, re-booting from cached bytes on re-entry. So an embedded strip of
  models holds a context only for the visible ones and never exhausts the
  browser's ~16-context budget — the actual fix to the "contexts grow
  monotonically" finding.

Fitted stays on the static thumbnail (a grid can show more tiles than the
context cap at once; pooled-renderer live fitted is tracked in CS-12401).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ylm @claude
Add the GLB and glTF 3D model FileDef families
2d96c8c 
Extends the 3D model family (StlDef/ThreeMfDef, CS-12053/CS-12054) with the two
glTF forms, so `.glb` and `.gltf` files get the same interactive viewer and
inspector as the other model formats.

GlbDef and GltfDef are thin ThreeDModelDef leaves — they inherit the shared
Three.js orbit viewer, whose GLTFLoader default path already renders both the
binary and JSON forms, plus the fitted thumbnail and the isolated body. Each
adds a `gltfMetadata` field and an extension-guarded, size-capped
`extractAttributes`, mirroring StlDef.

The metadata is read from the glTF structure itself rather than a geometry scan:
a glTF describes its accessors, so `gltf-meta-extractor.ts` reads the vertex and
triangle counts, mesh/material/node/animation/texture counts, and the
bounding-box dimensions straight from the header — the JSON of a `.gltf`, or the
JSON chunk of a `.glb` (its BIN chunk is never touched). Returns undefined for
non-glTF bytes, which the leaf turns into a FileContentMismatchError so a
mislabeled file falls back to the plain FileDef.

Routes `.glb` → GlbDef and `.gltf` → GltfDef, and adds unit tests for the
extractor (both container forms, indexed/non-indexed/strip triangle counts,
bounds, and the non-glTF fallback) plus code-ref routing assertions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Preview deployments

Host Test Results

    1 files  ±  0      1 suites  ±0   36m 39s ⏱️ + 30m 30s
1 142 tests +933  1 135 ✅ +927  7 💤 +6  0 ❌ ±0 
1 147 runs  +938  1 140 ✅ +932  7 💤 +6  0 ❌ ±0 

Results for commit cd4adf3. ± Comparison against earlier commit 2d96c8c.

Realm Server Test Results

    1 files  ± 0      1 suites  ±0   17m 51s ⏱️ + 3m 47s
2 174 tests +55  2 174 ✅ +55  0 💤 ±0  0 ❌ ±0 
2 254 runs  +56  2 254 ✅ +56  0 💤 ±0  0 ❌ ±0 

Results for commit cd4adf3. ± Comparison against earlier commit 2d96c8c.

@FadhlanR
FadhlanR force-pushed the cs-12053-cs-12054-model-filedef-subclasses branch from b97f29e to b30b584 Compare August 14, 2026 09:41
@lukemelia
Merge remote-tracking branch 'origin/cs-12053-cs-12054-model-filedef-…
cd4adf3 
…subclasses' into cs-12243-3d-model-formats-glb-gltf-stl-3mf

# Conflicts:
#	packages/base/stl-model-def.gts
#	packages/base/three-d-model-def.gts
#	packages/base/three-mf-def.gts
#	packages/host/tests/acceptance/model-file-extract-test.gts
#	packages/host/tests/unit/file-def-code-ref-test.ts
#	packages/host/tests/unit/model-meta-extractor-test.ts
#	packages/runtime-common/file-def-code-ref.ts
@lukemelia

lukemelia commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

[Claude Code 🤖] The merge from the base branch was more than marker cleanup, because the base's squashed commit rewrote the 3D family onto the shared file-format shells while this branch's GLB/glTF leaves were written against the earlier hand-rolled architecture. The merge commit (cd4adf3) ports them:

  • GlbDef/GltfDef are now thin ThreeDModelDef leaves following the same pattern as StlDef/ThreeMfDef: one extension-guarded, size-capped extractAttributes projecting the header read onto the shared model3d field via model3dAttributes. The leaf-local GltfMetadataField and isolated-body override are gone.
  • The shared SerializedModel3d/Model3dMetadataField gain the glTF header facts (vertices, nodes, animations, textures, dimensions) — consistent with the field's header-only doctrine, since a glTF header enumerates its scene graph directly, unlike STL/3MF where geometry is deliberately left to the live viewer.
  • The isolated shell's hasModel3d gate also accepts vertices/dimensions, so a glTF without a mesh count still shows its inspector section.
  • The acceptance test gains a glTF round-trip case asserting the model3d projection (format label with folded version, counts, and bounding-box dimensions); the extractor unit tests carry over unchanged.

Typechecks pass across runtime-common and host; template-lint is clean on the touched .gts files.

@lukemelia
lukemelia changed the base branch from cs-12053-cs-12054-model-filedef-subclasses to main August 18, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lukemelia @FadhlanR