Mac app: release-to-desktop as the only mode, with working panes and picking - #839
Merged
Conversation
`examples/robot-arm-2dof.vcad` used snake_case assembly keys (`part_defs`,
`part_def_id`, `ground_instance_id`, `parent_instance_id`) where the IR
declares camelCase, so the kernel refused to parse it. Renamed to the
current schema; it now evaluates to 996 triangles / 100x100x335 mm.
The worse half was that the failure was silent. `vcad_scene_from_json`
and `vcad_scene_from_json_in` returned null without touching the
`vcad_last_error` channel that already exists for the simulation entry
points, so callers had a null handle and no reason for it. serde names
the offending field and line ("missing field `partDefId` at line 107")
and that diagnosis was being dropped at the FFI boundary — leaving the
Swift app to draw a populated feature tree over an empty viewport, which
reads as a renderer bug rather than a schema mismatch.
Both entry points now clear the slot on entry and record UTF-8, parse,
and evaluate failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…d picking The Mac app now launches straight into release-to-desktop and has no other mode to reach. Everything the studio window used to carry had to find a home, and the interaction layer had to work over the desktop rather than inside a window. Mode - The WindowGroup's window hides itself the moment AppKit attaches it and hands the model to the overlay controller, replacing a 1s sleep that raced the window into existence and showed the studio window first. - Removed the in-window transparency path, the return pill, the mode toggle and the VCAD_RELEASE opt-out. Chrome - Document actions live in the real menu bar (New/Open/Recent/Examples, Save/Save As/Revert, Export STL/USDZ, Undo/Redo, and a View menu for the panels, zebra and camera presets) — they were unreachable once the title bar went away. The model moved to app scope so the commands and the overlay edit the same document. - Object Inspector: branches on the document instead of always showing sandbox rows, gained live parameter editors for the selected feature (shared with the studio inspector via FeatureParamEditors so the two cannot drift), the swatch material menu, solve time and the pick readout. Docked to a right rail rather than chasing the selected part around the viewport. - Components palette rebuilt in the Borland idiom: speedbar separated from the palette proper, icon-only tiles on a stable grid, a hint line that explains whatever the pointer is over (orange when a tool is blocked), and click-to-place — a Create tile arms, the next click in the scene puts the primitive where you pointed. - Feature tree is a closable tool window like every other pane; the gripper gets the full Receipt in the inspector's slot; document name and status return as a pill. Picking and mouse - Hover: parts and assembly instances light under the pointer, with the cursor stating intent and the surface readout published to the inspector. It rides the raycast the pass-through test was already doing every mouse-moved, so it costs no extra work. - Instances were entirely inert (the renderer draws inst0..instN for an assembly, every hit test matched part... only): they now hover, select, drive the tree, and expose their part def's parameters. - Hover and selection sync both ways with the feature tree, and the selected row scrolls into view. - Scroll to dolly, double-click to frame, right-click menu, occlusion cycling on repeated clicks, Cmd-drag marquee, Esc to deselect (an empty-space click belongs to the desktop, so it cannot deselect). - Fixed picking being mirrored about the horizontal midline: SwiftUI hands out top-left points, ARView.hitTest wants the view's bottom-left space, and the two paths disagreed about which needed flipping. - Fixed the gizmo never being rescaled in this view, so its handles were sized for whatever camera distance the scene was built at. Performance - Selecting a part no longer rebuilds the scene. highlightedParts was part of the geometry key, so every click re-evaluated the document, re-tessellated, rebuilt entities and regenerated colliders to change one emissive value — measured 73-83ms per click on a 3.9k-triangle plate. Selection and visibility now repaint in place. - Removed the separate double-click gesture, which forced SwiftUI to disambiguate and left every single click waiting out the double-click interval; framing reads clickCount off the event instead. - Timing is available behind VCAD_PERF=1. Dock icon - The tile follows the viewport: it snapshots the ARView, frames the model by its alpha silhouette (so zoom and pan cannot change the picture), and updates at most twice a second when something changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Groundwork for sub-element selection: the viewport can tell you which PART you clicked, but not which face of it — and picking a face is the prerequisite for sketching on one, filleting an edge, or measuring between two. The tessellator already tagged each triangle with its surface KIND (plane, cylinder, ...) for the face-kind debugger. That says a triangle came from a plane; it cannot say which plane. `TriangleMesh.face_ids` now carries the face's ordinal within its shell alongside `face_kinds`, threaded through every pass that rewrites the triangle list — merge, T-junction heal, weld (which drops degenerates), and fan fill, whose bridging triangles belong to no face and are tagged FACE_ID_UNKNOWN. Both tessellation entry points tag, so either produces a pickable mesh. Carried out to the app through EvaluatedMesh (the render bake unindexes but never reorders triangles, so per-triangle tags stay aligned) and a new `vcad_scene_part_face_ids` FFI entry point, which hands back the id array only when it matches the mesh being drawn — a partial or stale array would point hover at the wrong face. Null when a part has no tags at all (a frozen or imported mesh has no B-rep to name faces in), so callers treat face picking as an optional capability. The mesh disk-cache format is bumped to 2: a v1 record has no id block, and reading its trailing bytes as one would hand the viewport garbage. These ids are a handle for a session's hover and selection, NOT a durable reference — any edit that changes the B-rep renumbers them. Persisting a face across edits is the topological-naming problem and wants its own scheme. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
⌘N now opens another copy of the app rather than resetting the current document, and each copy shows up as its own Dock tile (already rendering that document's viewport) and its own ⌘-Tab entry. Why a process and not a window: on macOS the Dock tile and the ⌘-Tab entry belong to the APPLICATION, not to its windows — Pages with six documents open still shows one of each. `NSApp.dockTile` customises that single tile, it does not mint new ones, and there is no supported route to a second tile short of a second app bundle, which would mean shipping a decoy bundle pretending to be vcad and would still be one tile per bundle rather than per document. Release-to-desktop has no window of its own anyway — the app IS the parts floating over the desktop — so "another window" was already a fiction, while another instance gives each document its own tile, ⌘-Tab entry, menu bar, and crash domain. - ⌘N opens an instance; ⇧⌘N resets the current document, so that capability is not lost. - Open / Recents / Examples route through AppInstance.opening, which reuses THIS instance only when it holds an untouched sandbox — the way a Mac app reuses its empty Untitled window — and spawns otherwise. The open panel takes a multiple selection now that several documents at once is a sensible thing to ask for. - Finder double-click, drops and `open file.vcad` go through application(_:open:): macOS hands those to a RUNNING instance, so that was the path that would otherwise replace the document you are looking at. Implementing it also tells AppKit this app opens documents, which suppresses the untitled window at launch — and this app's whole UI hangs off that window — so applicationShouldOpenUntitledFile says yes explicitly. - The document arrives as a launch ARGUMENT, not via VCAD_OPEN: the environment is inherited by anything the instance spawns, and a stale value two launches later would silently reopen the wrong file. - Recents re-read the stored list before appending. Each instance held the list as it looked when IT launched, so writing that stale copy back would drop every document the other instances opened meanwhile. Still last-writer-wins, but the window is the write rather than the lifetime of the process. Verified with three instances live: separate processes each with their own window, each rendering its own document, two overlays compositing over the desktop at once with the transparent one passing the mouse through to the one behind. Known gap: instances anchor their chrome to the same screen corners, so two documents' inspectors stack on top of each other. Cascading the chrome per instance is the fix and is not in this commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`cargo build --workspace` does not build example targets, so the new field compiled everywhere the workspace build looks and broke six example/integration files that CI's `cargo test --workspace` does build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Mac app now launches straight into release-to-desktop — parts floating over the desktop in a borderless transparent overlay — and has no other mode to reach. That meant everything the studio window carried had to find a new home, and the interaction layer had to work over the desktop rather than inside a window.
Mode
The
WindowGroup's window hides itself the moment AppKit attaches it and hands the model to the overlay controller. Previously aTask.sleep(for: .seconds(1))raced the window into existence, so the app showed the studio window for a second before switching — the flash that started this. The in-window transparency path, the return pill, the mode toggle and theVCAD_RELEASEopt-out are gone.ViewportViewis deliberately left in place, unpresented: it is still the home of the shared RealityKit scene code (environments, materials, gizmo and mesh construction) the released view builds on. Untangling that is its own change.Chrome
FeatureParamEditors, so a newly editable op appears in both at once.Picking and mouse
inst0…instNfor an assembly while every hit test matchedpart…only, so no link of a robot could be hovered or clicked. They now hover, select, drive the tree, and expose their part def's parameters — editing the definition moves every instance of it.ARView.hitTestwants the view's bottom-left space, and the click and hover paths disagreed about which needed the flip. Invisible on a single centred body — you hit the same thing either way — and obvious on a tall robot.Performance
highlightedPartswas part of the geometry key, so every click re-evaluated the document, re-tessellated every part, rebuilt every entity and regenerated every collider — to change one material's emissive value. Instrumented at 73–83 ms per click on a 3,884-triangle plate, and it scales with the model. Selection and visibility are appearance, not geometry; they repaint in place now.SpatialTapGesture(count: 2)forced SwiftUI to disambiguate, so ~0.2 ms of selection work sat behind ~half a second of nothing. Framing readsclickCountoff the event being handled instead. Worth noting the timing log is what found this — it proved the handler was innocent and pointed at the gesture layer.VCAD_PERF=1(writes/tmp/vcad_perf.log); it is off and free otherwise.Dock icon
The tile follows the viewport: it snapshots the ARView and frames the model by its alpha silhouette rather than cropping the centre of the frame, so zoom and pan cannot change the picture — the icon is a portrait of the document, not a screenshot. Capped at 2 Hz and gated on a scene key, so a still scene stops feeding the Dock identical frames.
Notes for review
vcad_scene_from_json/_json_innow clear and recordvcad_last_erroron UTF-8, parse and evaluate failures. The ABI is unchanged — callers that ignore errors keep working — but the diagnosis was previously dropped on the floor, which is how a schema-stale example presented as a renderer bug.examples/robot-arm-2dof.vcadis reformatted wholesale by the key rename; the substantive change is snake_case → camelCase on the assembly keys.🤖 Generated with Claude Code
Added since the first review pass
Per-triangle face ids (
d817ead3)Groundwork for sub-element selection — picking a face is the prerequisite for sketching on one, filleting an edge, or measuring between two.
The tessellator already tagged each triangle with its surface kind for the face-kind debugger, which says a triangle came from a plane but not which plane.
TriangleMesh.face_idsnow carries the face's ordinal within its shell alongsideface_kinds, threaded through every pass that rewrites the triangle list — merge, T-junction heal, weld (which drops degenerates), and fan fill, whose bridging triangles belong to no face and are tagged unknown rather than guessed. Both tessellation entry points tag, so either produces a pickable mesh.Carried to the app through
EvaluatedMesh(the render bake unindexes but never reorders triangles, so per-triangle tags stay aligned) and a newvcad_scene_part_face_ids, which hands back the array only when it matches the mesh being drawn — a partial or stale array would point hover at the wrong face — and null when a part has no B-rep, so callers treat face picking as an optional capability. The mesh disk-cache format is bumped to 2, since a v1 record has no id block and reading its trailing bytes as one would hand the viewport garbage faces.These ids are a handle for a session's hover and selection, not a durable reference: any edit that changes the B-rep renumbers them. Persisting a face across edits is the topological-naming problem and wants its own scheme.
Tests: a cube resolves to six distinct ids covering every triangle; a cylinder's caps and side stay in lockstep with
face_kinds(the failure mode being an array that drifts one triangle and highlights the neighbour); the FFI hands back one id per triangle of the mesh it drew.The Swift half — ray→triangle→face, the face highlight, the inspector row — is not in this PR.
A document is an app instance (
a4fffdae)⌘N opens another copy of the app, so each document gets its own Dock tile (already rendering that document's viewport) and its own ⌘-Tab entry.
On macOS the Dock tile and ⌘-Tab entry belong to the application, not its windows — Pages with six documents open still shows one of each — so this cannot be a second window. Release-to-desktop has no window of its own anyway, so "another window" was already a fiction; another instance gives each document its own tile, menu bar, and crash domain.
⇧⌘N keeps the reset-this-document behaviour. Open/Recents/Examples reuse the current instance only when it holds an untouched sandbox, the way a Mac app reuses its empty Untitled window. The document arrives as a launch argument rather than
VCAD_OPEN, since the environment is inherited by anything the instance spawns and a stale value would silently reopen the wrong file. Recents re-read the stored list before appending, or each instance would write back the list as it looked when it launched and drop what the others opened.Verified with three instances live: separate processes, each rendering its own document, two overlays compositing over the desktop at once with the transparent one passing the mouse through to the one behind.
Known gap: instances anchor their chrome to the same screen corners, so two documents' inspectors stack on top of each other. Cascading the chrome per instance is the fix and is not in this PR.