Skip to content

Mac app: release-to-desktop as the only mode, with working panes and picking - #839

Merged
ecto merged 5 commits into
mainfrom
claude/mac-desktop-app-a0cc3c
Aug 29, 2026
Merged

Mac app: release-to-desktop as the only mode, with working panes and picking#839
ecto merged 5 commits into
mainfrom
claude/mac-desktop-app-a0cc3c

Conversation

@ecto

@ecto ecto commented Aug 28, 2026

Copy link
Copy Markdown
Owner

The Mac app now launches straight into release-to-desktop — parts floating over the desktop in a borderless transparent overlay — and has no other mode to reach. That meant everything the studio window carried had to find a new home, and the interaction layer had to work over the desktop rather than inside a window.

Mode

The WindowGroup's window hides itself the moment AppKit attaches it and hands the model to the overlay controller. Previously a Task.sleep(for: .seconds(1)) raced the window into existence, so the app showed the studio window for a second before switching — the flash that started this. The in-window transparency path, the return pill, the mode toggle and the VCAD_RELEASE opt-out are gone.

ViewportView is deliberately left in place, unpresented: it is still the home of the shared RealityKit scene code (environments, materials, gizmo and mesh construction) the released view builds on. Untangling that is its own change.

Chrome

  • Menu bar — New/Open/Recent/Examples, Save/Save As/Revert, Export STL/USDZ, Undo/Redo, and a View menu for panels, zebra and camera presets. These lived in the title bar's document menu and were simply unreachable once release mode hid it. The model moved to app scope so the commands and the overlay edit the same document.
  • Object Inspector — branches on the document instead of always showing sandbox rows; live scrub editors for the selected feature's parameters (Cube W/D/H, Fillet radius, pattern count, …), the swatch material menu scoped to the node's part, solve time and the pick readout. Docked to a right rail rather than chasing the selected part around the viewport. The editors are shared with the studio inspector as FeatureParamEditors, so a newly editable op appears in both at once.
  • Components palette — rebuilt in the Borland C++ Builder idiom, HIG-styled: speedbar separated from the palette proper, icon-only tiles on a grid that doesn't move between tabs, and a hint line that explains whatever the pointer is over (orange when a tool is blocked). Create tiles now arm — the next click in the scene places the primitive where you pointed, on the surface under the cursor or the ground plane.
  • Feature tree is a closable tool window like every other pane; the gripper gets the full Receipt in the inspector's slot; the document name and status strip return as a pill.

Picking and mouse

  • Hover — parts and assembly instances light under the pointer, the cursor states what a click will do, and the surface readout reaches the inspector. It rides the raycast the pass-through test was already running on every mouse-moved, so it costs no second raycast.
  • Assembly instances were entirely inert. The renderer draws inst0…instN for an assembly while every hit test matched part… only, so no link of a robot could be hovered or clicked. They now hover, select, drive the tree, and expose their part def's parameters — editing the definition moves every instance of it.
  • Tree sync both ways — hovering geometry lights its row and hovering a row lights the geometry it drew (all of it: one row, four elbows on a quadruped); selecting scrolls the row into view.
  • Scroll to dolly, double-click to frame, right-click menu, occlusion cycling on repeated clicks in the same spot, ⌘-drag marquee, Esc to deselect (an empty-space click passes through to the desktop, so it cannot deselect).
  • Picking was mirrored about the horizontal midline: SwiftUI hands out top-left points, ARView.hitTest wants the view's bottom-left space, and the click and hover paths disagreed about which needed the flip. Invisible on a single centred body — you hit the same thing either way — and obvious on a tall robot.
  • The gizmo was never rescaled in this view, so its handles were sized for whatever camera distance the scene happened to be built at.

Performance

  • Selecting no longer rebuilds the scene. highlightedParts was part of the geometry key, so every click re-evaluated the document, re-tessellated every part, rebuilt every entity and regenerated every collider — to change one material's emissive value. Instrumented at 73–83 ms per click on a 3,884-triangle plate, and it scales with the model. Selection and visibility are appearance, not geometry; they repaint in place now.
  • Every single click was waiting out the double-click interval. A separate SpatialTapGesture(count: 2) forced SwiftUI to disambiguate, so ~0.2 ms of selection work sat behind ~half a second of nothing. Framing reads clickCount off the event being handled instead. Worth noting the timing log is what found this — it proved the handler was innocent and pointed at the gesture layer.
  • Timing is available behind VCAD_PERF=1 (writes /tmp/vcad_perf.log); it is off and free otherwise.

Dock icon

The tile follows the viewport: it snapshots the ARView and frames the model by its alpha silhouette rather than cropping the centre of the frame, so zoom and pan cannot change the picture — the icon is a portrait of the document, not a screenshot. Capped at 2 Hz and gated on a scene key, so a still scene stops feeding the Dock identical frames.

Notes for review

  • The FFI change (first commit) touches shared kernel surface: vcad_scene_from_json / _json_in now clear and record vcad_last_error on UTF-8, parse and evaluate failures. The ABI is unchanged — callers that ignore errors keep working — but the diagnosis was previously dropped on the floor, which is how a schema-stale example presented as a renderer bug.
  • examples/robot-arm-2dof.vcad is reformatted wholesale by the key rename; the substantive change is snake_case → camelCase on the assembly keys.
  • The Swift files carried uncommitted work from before this session, so those commits are not a clean per-change history.
  • Verified by driving the running app: hover readout, click selection on parts and instances, the arm loading at 996 triangles, and the parse error surfacing in the pill. The Dock icon's framing has been reasoned through and its alpha scan verified with a standalone harness, but I have not looked at the rendered tile.
  • Known open item: one unexplained app exit during testing, with no crash report and no unified-log entry. Not reproduced; idle memory is flat.

🤖 Generated with Claude Code


Added since the first review pass

Per-triangle face ids (d817ead3)

Groundwork for sub-element selection — picking a face is the prerequisite for sketching on one, filleting an edge, or measuring between two.

The tessellator already tagged each triangle with its surface kind for the face-kind debugger, which says a triangle came from a plane but not which plane. TriangleMesh.face_ids now carries the face's ordinal within its shell alongside face_kinds, threaded through every pass that rewrites the triangle list — merge, T-junction heal, weld (which drops degenerates), and fan fill, whose bridging triangles belong to no face and are tagged unknown rather than guessed. Both tessellation entry points tag, so either produces a pickable mesh.

Carried to the app through EvaluatedMesh (the render bake unindexes but never reorders triangles, so per-triangle tags stay aligned) and a new vcad_scene_part_face_ids, which hands back the array only when it matches the mesh being drawn — a partial or stale array would point hover at the wrong face — and null when a part has no B-rep, so callers treat face picking as an optional capability. The mesh disk-cache format is bumped to 2, since a v1 record has no id block and reading its trailing bytes as one would hand the viewport garbage faces.

These ids are a handle for a session's hover and selection, not a durable reference: any edit that changes the B-rep renumbers them. Persisting a face across edits is the topological-naming problem and wants its own scheme.

Tests: a cube resolves to six distinct ids covering every triangle; a cylinder's caps and side stay in lockstep with face_kinds (the failure mode being an array that drifts one triangle and highlights the neighbour); the FFI hands back one id per triangle of the mesh it drew.

The Swift half — ray→triangle→face, the face highlight, the inspector row — is not in this PR.

A document is an app instance (a4fffdae)

⌘N opens another copy of the app, so each document gets its own Dock tile (already rendering that document's viewport) and its own ⌘-Tab entry.

On macOS the Dock tile and ⌘-Tab entry belong to the application, not its windows — Pages with six documents open still shows one of each — so this cannot be a second window. Release-to-desktop has no window of its own anyway, so "another window" was already a fiction; another instance gives each document its own tile, menu bar, and crash domain.

⇧⌘N keeps the reset-this-document behaviour. Open/Recents/Examples reuse the current instance only when it holds an untouched sandbox, the way a Mac app reuses its empty Untitled window. The document arrives as a launch argument rather than VCAD_OPEN, since the environment is inherited by anything the instance spawns and a stale value would silently reopen the wrong file. Recents re-read the stored list before appending, or each instance would write back the list as it looked when it launched and drop what the others opened.

Verified with three instances live: separate processes, each rendering its own document, two overlays compositing over the desktop at once with the transparent one passing the mouse through to the one behind.

Known gap: instances anchor their chrome to the same screen corners, so two documents' inspectors stack on top of each other. Cascading the chrome per instance is the fix and is not in this PR.

ecto and others added 2 commits August 28, 2026 19:48
`examples/robot-arm-2dof.vcad` used snake_case assembly keys (`part_defs`,
`part_def_id`, `ground_instance_id`, `parent_instance_id`) where the IR
declares camelCase, so the kernel refused to parse it. Renamed to the
current schema; it now evaluates to 996 triangles / 100x100x335 mm.

The worse half was that the failure was silent. `vcad_scene_from_json`
and `vcad_scene_from_json_in` returned null without touching the
`vcad_last_error` channel that already exists for the simulation entry
points, so callers had a null handle and no reason for it. serde names
the offending field and line ("missing field `partDefId` at line 107")
and that diagnosis was being dropped at the FFI boundary — leaving the
Swift app to draw a populated feature tree over an empty viewport, which
reads as a renderer bug rather than a schema mismatch.

Both entry points now clear the slot on entry and record UTF-8, parse,
and evaluate failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…d picking

The Mac app now launches straight into release-to-desktop and has no
other mode to reach. Everything the studio window used to carry had to
find a home, and the interaction layer had to work over the desktop
rather than inside a window.

Mode
- The WindowGroup's window hides itself the moment AppKit attaches it and
  hands the model to the overlay controller, replacing a 1s sleep that
  raced the window into existence and showed the studio window first.
- Removed the in-window transparency path, the return pill, the mode
  toggle and the VCAD_RELEASE opt-out.

Chrome
- Document actions live in the real menu bar (New/Open/Recent/Examples,
  Save/Save As/Revert, Export STL/USDZ, Undo/Redo, and a View menu for
  the panels, zebra and camera presets) — they were unreachable once the
  title bar went away. The model moved to app scope so the commands and
  the overlay edit the same document.
- Object Inspector: branches on the document instead of always showing
  sandbox rows, gained live parameter editors for the selected feature
  (shared with the studio inspector via FeatureParamEditors so the two
  cannot drift), the swatch material menu, solve time and the pick
  readout. Docked to a right rail rather than chasing the selected part
  around the viewport.
- Components palette rebuilt in the Borland idiom: speedbar separated
  from the palette proper, icon-only tiles on a stable grid, a hint line
  that explains whatever the pointer is over (orange when a tool is
  blocked), and click-to-place — a Create tile arms, the next click in
  the scene puts the primitive where you pointed.
- Feature tree is a closable tool window like every other pane; the
  gripper gets the full Receipt in the inspector's slot; document name
  and status return as a pill.

Picking and mouse
- Hover: parts and assembly instances light under the pointer, with the
  cursor stating intent and the surface readout published to the
  inspector. It rides the raycast the pass-through test was already
  doing every mouse-moved, so it costs no extra work.
- Instances were entirely inert (the renderer draws inst0..instN for an
  assembly, every hit test matched part... only): they now hover, select,
  drive the tree, and expose their part def's parameters.
- Hover and selection sync both ways with the feature tree, and the
  selected row scrolls into view.
- Scroll to dolly, double-click to frame, right-click menu, occlusion
  cycling on repeated clicks, Cmd-drag marquee, Esc to deselect (an
  empty-space click belongs to the desktop, so it cannot deselect).
- Fixed picking being mirrored about the horizontal midline: SwiftUI
  hands out top-left points, ARView.hitTest wants the view's bottom-left
  space, and the two paths disagreed about which needed flipping.
- Fixed the gizmo never being rescaled in this view, so its handles were
  sized for whatever camera distance the scene was built at.

Performance
- Selecting a part no longer rebuilds the scene. highlightedParts was
  part of the geometry key, so every click re-evaluated the document,
  re-tessellated, rebuilt entities and regenerated colliders to change
  one emissive value — measured 73-83ms per click on a 3.9k-triangle
  plate. Selection and visibility now repaint in place.
- Removed the separate double-click gesture, which forced SwiftUI to
  disambiguate and left every single click waiting out the double-click
  interval; framing reads clickCount off the event instead.
- Timing is available behind VCAD_PERF=1.

Dock icon
- The tile follows the viewport: it snapshots the ARView, frames the
  model by its alpha silhouette (so zoom and pan cannot change the
  picture), and updates at most twice a second when something changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

4 Skipped Deployments
Project Deployment Actions Updated (UTC)
mecheval Ignored Ignored Aug 29, 2026 3:20pm
vcad Ignored Ignored Aug 29, 2026 3:20pm
vcad-docs Ignored Ignored Aug 29, 2026 3:20pm
vcad-mcp Ignored Ignored Aug 29, 2026 3:20pm

Request Review

ecto and others added 3 commits August 28, 2026 22:23
Groundwork for sub-element selection: the viewport can tell you which
PART you clicked, but not which face of it — and picking a face is the
prerequisite for sketching on one, filleting an edge, or measuring
between two.

The tessellator already tagged each triangle with its surface KIND
(plane, cylinder, ...) for the face-kind debugger. That says a triangle
came from a plane; it cannot say which plane. `TriangleMesh.face_ids`
now carries the face's ordinal within its shell alongside `face_kinds`,
threaded through every pass that rewrites the triangle list — merge,
T-junction heal, weld (which drops degenerates), and fan fill, whose
bridging triangles belong to no face and are tagged FACE_ID_UNKNOWN.
Both tessellation entry points tag, so either produces a pickable mesh.

Carried out to the app through EvaluatedMesh (the render bake unindexes
but never reorders triangles, so per-triangle tags stay aligned) and a
new `vcad_scene_part_face_ids` FFI entry point, which hands back the id
array only when it matches the mesh being drawn — a partial or stale
array would point hover at the wrong face. Null when a part has no tags
at all (a frozen or imported mesh has no B-rep to name faces in), so
callers treat face picking as an optional capability.

The mesh disk-cache format is bumped to 2: a v1 record has no id block,
and reading its trailing bytes as one would hand the viewport garbage.

These ids are a handle for a session's hover and selection, NOT a
durable reference — any edit that changes the B-rep renumbers them.
Persisting a face across edits is the topological-naming problem and
wants its own scheme.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
⌘N now opens another copy of the app rather than resetting the current
document, and each copy shows up as its own Dock tile (already rendering
that document's viewport) and its own ⌘-Tab entry.

Why a process and not a window: on macOS the Dock tile and the ⌘-Tab
entry belong to the APPLICATION, not to its windows — Pages with six
documents open still shows one of each. `NSApp.dockTile` customises that
single tile, it does not mint new ones, and there is no supported route
to a second tile short of a second app bundle, which would mean shipping
a decoy bundle pretending to be vcad and would still be one tile per
bundle rather than per document. Release-to-desktop has no window of its
own anyway — the app IS the parts floating over the desktop — so "another
window" was already a fiction, while another instance gives each document
its own tile, ⌘-Tab entry, menu bar, and crash domain.

- ⌘N opens an instance; ⇧⌘N resets the current document, so that
  capability is not lost.
- Open / Recents / Examples route through AppInstance.opening, which
  reuses THIS instance only when it holds an untouched sandbox — the way
  a Mac app reuses its empty Untitled window — and spawns otherwise. The
  open panel takes a multiple selection now that several documents at
  once is a sensible thing to ask for.
- Finder double-click, drops and `open file.vcad` go through
  application(_:open:): macOS hands those to a RUNNING instance, so that
  was the path that would otherwise replace the document you are looking
  at. Implementing it also tells AppKit this app opens documents, which
  suppresses the untitled window at launch — and this app's whole UI
  hangs off that window — so applicationShouldOpenUntitledFile says yes
  explicitly.
- The document arrives as a launch ARGUMENT, not via VCAD_OPEN: the
  environment is inherited by anything the instance spawns, and a stale
  value two launches later would silently reopen the wrong file.
- Recents re-read the stored list before appending. Each instance held
  the list as it looked when IT launched, so writing that stale copy back
  would drop every document the other instances opened meanwhile. Still
  last-writer-wins, but the window is the write rather than the lifetime
  of the process.

Verified with three instances live: separate processes each with their
own window, each rendering its own document, two overlays compositing
over the desktop at once with the transparent one passing the mouse
through to the one behind.

Known gap: instances anchor their chrome to the same screen corners, so
two documents' inspectors stack on top of each other. Cascading the
chrome per instance is the fix and is not in this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`cargo build --workspace` does not build example targets, so the new
field compiled everywhere the workspace build looks and broke six
example/integration files that CI's `cargo test --workspace` does build.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ecto
ecto merged commit 6f92c78 into main Aug 29, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant