visionOS 版を出せる状態にして、ついでにスクショ撮影を3機種とも自動化する - #56
Merged
Conversation
The public text described a product that no longer exists. It was written when visionOS was the iPad app in a window (#11) — "the same three panes in a window: the blocks to pick from, the program you are building, and the tortoise drawing it" — and #53 replaced that with a viewer: no editing at all, the drawing on a real table, the tortoise walking on the paper. None of it had been pushed, so the only cost is rewriting it before it goes up. visionOS gets its own metadata directory rather than a corrected paragraph in the shared one. A Vision Pro shopper is shown the visionOS description and nothing else, so a shared description opens by telling that shopper to drag blocks into a program and change numbers where they sit — the one thing this app cannot do there. What it does do is worth its own text: a drawing that lands on the table, a tortoise standing on the paper rather than a cursor on a screen, and the blocks and the Swift code in two more windows at once, which is the version of the idea iPad and Mac cannot give because a screen has to choose. Three fields in those directories are app-level in App Store Connect rather than version-level — name, subtitle and privacy URL belong to the app — so every lane writes the same ones and whichever runs last decides them for all three listings. Two directories that disagreed would make the app's name depend on lane order, silently, which is exactly the kind of thing metadata_check exists to refuse: it now fails when they drift. The shared release notes and the site keep their Vision Pro sections and say the true thing instead. The site's heading went with them — "iPad, Mac and Vision Pro, one app" was the claim being corrected, not the wording.
The eight new captures came out RGBA — fully opaque, every pixel, so the alpha channel carried nothing — and App Store Connect refuses a screenshot that has one. metadata_check caught all eight before anything reached Apple, which is the whole reason it reads the PNG header itself rather than trusting deliver to say so partway through an upload. So: drop the channel, which is lossless here, and run the same oxipng pass the rest of the images take. The RGB is byte-identical to what came off the screen, and the files land about a quarter smaller than they arrived — and smaller than the ones they replace, which the extra channel had been inflating too. site/shots/mac-*.png are downscaled copies of two of these, so they are regenerated from the new ones rather than left showing the previous build's UI. The pipeline is the one in the release skill (`magick -resize -dither None -colors 256 -strip`, then oxipng); rerunning it against the *old* captures reproduced the committed files byte for byte, which is what makes it safe to rerun against the new ones.
Same shape as the Mac reshoot: eight new iPad captures and a replacement for the one Mac shot whose window sat 14px off the other seven, all nine RGBA with every pixel fully opaque, all nine refused by metadata_check before Apple ever saw them. Dropping the channel is lossless — the RGB is byte-identical — and with the usual oxipng pass the iPad files land about 45% smaller than they arrived. The Mac set is now uniform: all eight put the window at the same four edges, so the outlier that was left in the previous commit is gone rather than documented. site/shots/ipad-*.png and mac-en-1.png are downscaled copies of seven of these, regenerated for the same reason as last time. The iPad pipeline was checked the same way before being trusted: rerunning it against the old captures reproduced all six committed files byte for byte.
Every capture this project shoots arrives with an alpha channel and App Store Connect refuses every one of them. That is not carelessness at the export step and it will not be fixed there: the Mac shots are composed in Figma, the iPad ones come from the simulator's screenshot button, the visionOS ones from simctl io, and none of the three can be told to write RGB. So it is a step that has to happen on every reshoot, forever — which is a script, not a thing to remember. `ruby Tools/screenshots.rb` drops the channel, runs the oxipng pass over what it flattened, and regenerates site/shots. The last part is the actual reason this is a script: those are downscaled copies of seven of the captures, so a reshoot that stops at appstore/ leaves the website showing the previous build's UI. That was missed twice; it cannot be missed now. Two things it refuses to do. It will not composite: a capture with real transparency stops the run, because the background it would need is whatever the picture was shot against and this script does not know that. And it does not re-optimise a set that is already flat unless asked with --all, since oxipng at -o max costs minutes and is not bit-for-bit reproducible, so a gratuitous pass rewrites files by a few dozen bytes with the pixels untouched. MetadataCheck.alpha? is now public and is what the script asks. A fixer that disagreed with the gate about what counts as alpha would be worse than no fixer: it would report success on files CI still rejects.
The simulator was written off as unable to show the drawing at all, and it can. The symptom was real — blank sheet, nil currentTortoiseState, no tortoise — but the diagnosis was wrong: ViewAttachmentComponent hosts here perfectly well. What broke it was -TBPlace placing by calling place(.inFront) *after* the load, which flips sitsOnTable, which is the .id() on the immersive space's RealityView. The scene was demolished and rebuilt at launch and the attachment did not come back. "Opening a drawing puts it down" made setting the preference first the natural shape, and that alone fixed it. That matters now because the App Store screenshots have to come from somewhere, and a headset is the wrong somewhere: a real room cannot be framed the same way twice, and it is someone's home. So the launch line describes the picture instead. -TBSample picks the drawing (square|star|spiral|tree) and -TBDraw runs it to a fraction of its length and stops there, which is what makes a capture reproducible — and lets a shot show the tortoise mid-line rather than only a finished picture. The seek waits for the canvas to attach before it goes out, because TortoisePlayer.seek is a no-op until then and the attachment cannot happen before the immersive space has opened and rendered. currentTortoiseState turning non-nil is that moment and there is nothing to await on, so it polls, with a deadline so a run where the space never opens ends rather than spins. One simulator quirk survives and is now written down: queryDeviceAnchor reports the identity transform and calls it tracked, so pose() rejects it and placement takes the fixed fallback 1.2m along -Z from the space's origin. That is in front of the camera on a fresh boot and nowhere useful once the camera has been flown around, which cost two confusing captures before it was clear.
-TBSheet side,reach,drop, because nothing in the simulator can reach out and pinch the sheet bigger or drag it further off, and eight captures (four drawings, two languages) have to be framed the same way. It overrides the two constants the placement is built from rather than the position they produce, so a forced framing still lands in front of the camera and turned to face it. The first attempt did the opposite — it replaced the no-pose fallback, on the strength of the note saying the simulator reports the identity transform and calls it tracked. That note is about the *first* reading; pose() retries for three seconds and gets a real one, so the aimed branch is what runs here, exactly as on a headset. The override compiled, ran, and did nothing, and two captures came back byte-identical to the control before that was clear. The fresh-boot advice from the previous commit goes with it: the sheet is aimed at the camera, so where the camera is does not matter. What does happen is that a run occasionally opens with no sheet at all and a relaunch fixes it, which is now written down as "look at the capture" rather than being rediscovered as a mysterious framing rule.
Tools/visionos-shots.rb takes the shot list — drawing, how far through it, how the sheet sits — and produces every capture in both languages, then hands the result to Tools/screenshots.rb so the alpha comes off and the check runs. A reshoot is now editing a table and running a command, which is what makes the framing worth tuning at all: each attempt costs a minute instead of an afternoon. The language comes from -AppleLanguages on the launch line rather than from the simulator's own setting, so both locales come out of one run with no reboot between them and nothing left switched afterwards. The flake gets handled rather than described. Some launches open the immersive space with no attachment, and from outside that is indistinguishable from a slow one — waiting on EntityLoad, which is only the USDZ arriving, files pictures of empty rooms. So the app says which it is: TBReady once the canvas has attached and the drawing is on the paper, TBNotReady if it gives up. The script waits for that line from *this* process id, and relaunches rather than photographing what it finds. Three attempts, because the flake has never needed a second one twice.
Left the system, every window opens straight ahead, so the blocks landed over the remote that asked for them and the code landed over both. "Three surfaces at once, because a headset never has to choose" was the whole argument for this platform, and the only arrangement that does not deliver it is the one the app shipped with. So the program group asks for the remote's .leading and the code group for its .trailing, reading order deciding the sides: the blocks are what the drawing is made from, the code is what they become. The remote gets an id of its own so WindowPlacementContext can be asked which window to sit beside — unusual for a first window, and the only handle available. This nearly went unfixed. Asked whether window positions could be adjusted, I said no without checking, and the plan was to shoot the App Store captures around the overlap. defaultWindowPlacement has been there since visionOS 2. What is genuinely unavailable is absolute positioning — every CGPoint and UnitPoint initialiser is @available(visionOS, unavailable) — so a placement can name another window and a side, which is all this needs. The captures follow: three drawings matching the iPad and Mac listings, three windows in every frame, one sheet size for all of them. 0.8m of paper ate the bottom of the side windows, which is the arrangement these pictures exist to show, so 0.7 is a ceiling rather than a preference. The spiral stops at 46 of 82 commands on purpose — a finished picture is what the other listings already say, and what only this platform can say is that the tortoise is standing on the paper drawing it.
The English star had two blocks windows in it, one of them up by the ceiling. visionOS restores an app's windows, so each launch inherited wherever the last one left them and then opened its own on top — which is also what had the framing wandering between captures that were meant to be identical, and quite possibly what the "no sheet" runs were as well. Uninstalling before each attempt takes the scene state with it. Every shot now starts from the same empty room, and the six captures agree with each other across both languages. The spiral is drawn to the end like the other two. The half-drawn version made its own argument — the tortoise standing on a line still growing under it — but it also put a visibly unfinished picture in a set of three, and a shopper reading a row of screenshots has no way to know that was deliberate.
`ruby Tools/ipad-shots.rb` produces all eight — four drawings, two languages — and hands them to Tools/screenshots.rb. The same idea as the visionOS rig, but a UI test rather than launch arguments, because iPad needs the device rotated and simctl cannot turn one: there is no command for it, and driving the Simulator's menu means granting keystroke permission to whatever runs the script. XCUIDevice rotates in a line, and the same mechanism then presses play and switches panes — so the app keeps no screenshot-only code at all. Five things cost an hour each and are written down where they bit. The scrubber is *disabled* until a program has been run, so dragging it to the end does nothing — silently, producing a capture that looks perfectly well made of an empty canvas. adjust(toNormalizedSliderPosition:) does nothing either, and neither does ⌘R, the simulator having no hardware keyboard. The play button is tapped instead, addressed as `play.fill`: SwiftUI passes an SF Symbol's name through as the accessibility identifier, so it is the same in both languages while its label is not. Nothing here matches on a label. XCTAttachment(screenshot:) writes the framebuffer as it is held — portrait — and leaves the rotation to a flag, so a landscape capture arrives 2064×2752 on its side. The rotation is baked in by redrawing the image once. Orientation has to be set *after* the launch; set before, it is silently undone. TEST_RUNNER_ variables have to be on xcodebuild's own environment; as build settings after the command they are accepted, ignored, and arrive nowhere. And the documents are seeded into the device's tmp rather than the app's container, because preparing a run reinstalls the app and a reinstall gives it a new container. The app is uninstalled before each group of shots. Opening a document from outside the app's folder imports a copy under a deduplicated name, against a history that outlives deleting the files — the spiral came back titled `spiral-1` in its own title bar. Uninstalling resets it, so the shots are grouped to open each drawing once. The status bar is pinned to 9:41 for the run. Without it the captures carry whatever the clock said, and a reshoot never matches the set it joins.
The screenshot material had outgrown its home. It is now three platforms, two rigs, a shared pass, and about a dozen traps whose common shape is that they hand back a capture which looks perfectly well made and shows the wrong thing — an empty canvas, a portrait frame, an empty room, a window near the ceiling. That is a body of knowledge with its own trigger: reshooting happens because the UI changed, which is not the same afternoon as cutting a tag. So `screenshots` covers producing the pictures and `release` covers pushing them, with the boundary drawn where it falls naturally: App Store Connect's vocabulary, deliver's platform spellings and the upload reconciliation stay with the listing, while the rigs, the flatten-and-optimise pass and the site derivation move out. Both point at each other rather than restating, so they cannot drift. The release skill's description no longer claims screenshots, so work on them routes to the skill that has the traps rather than the one that has the tags.
Groundwork for shooting the macOS captures the same way. The target moves to SDKROOT = auto with an explicit platform list, which is all it took — no new target, so no hand-editing of the object graph and none of the hazards that come with it. The test code turned out to be nearly portable already: XCUISystem.open, the `play.fill` identifier and the segmented control all exist on macOS, and the one thing that does not is rotation, XCUIDevice having no orientation there. A Mac window is never held sideways, so that and the redraw that bakes an iPhone's orientation flag into the pixels are both #if os(iOS). Both test builds are green; nothing shoots on the Mac yet.
`ruby Tools/macos-shots.rb` finishes the set: all three platforms now come out
of a command. The capture is the *window alone*, taken with
`XCUIElement.screenshot()`, composited onto
`appstore/screenshot-sources/macos-plate-{en,ja}.png` — a desktop and a menu
bar drawn once per language. That is what keeps the picture independent of the
machine: no wallpaper, menu extra or clock of the Mac's own ever reaches it.
The plates carry no shadow; it is generated at composite time. So the window
can move or change size without the artwork being redrawn — which is not
hypothetical, since the previous set had one capture whose window sat 14px off
the other seven. The window is now fully in frame, centred under the menu bar,
rather than running off the bottom edge as the hand-made set did.
Four things the Mac does differently, each of which cost a run to find.
`XCUIElement.screenshot()` hands back the window's bounding box **fully
opaque, with the rounded corners filled near-black** — composite that and the
window wears four black wedges. The corners are flood-filled to transparent
rather than masked with a drawn radius: the shape is macOS's own continuous
curve, not a circle, and the wedges are the only near-black regions touching
the corners.
**macOS reopens the windows it had when it quit**, so the second shot's launch
restored the first shot's drawing and opened its own beside it. Two windows
means two transports, and `play.fill` stops being a single element — reported
as "Multiple matching elements found", which says nothing about restoration.
Every window is closed after the launch.
**The same `Picker(.segmented)` is a different element on each platform**: a
SegmentedControl of buttons on iOS, a RadioGroup of radio buttons in the
toolbar on macOS. Looking for the iOS one on a Mac times out saying only that
there is no toggle.
And **macOS UI testing needs Xcode to hold the Accessibility permission**.
Without it every run fails with "Timed out while enabling automation mode",
which mentions neither Xcode nor permissions.
The saved window state is thrown away before each run, since macOS restores a
window's frame in preference to the app's `defaultSize` — that is what makes
2560×1600 reproducible.
The skill described macOS as hand-composed in Figma, which stopped being true one commit ago. Figma now draws the plate — the desktop and the menu bar, once per language — and the window is captured and composited by a script, so the part a person does is artwork rather than assembly. The four ways the Mac differs from the iPad go in with it, since they are the part that will be met again: an opaque window screenshot with black corner wedges, window restoration turning one play button into two, the same picker being a RadioGroup rather than a SegmentedControl, and Accessibility permission announcing its absence as a timeout that mentions neither.
The previous commit updated the skill's body but not the two places that route to it — CLAUDE.md's one-liner and the skill's own description, both of which still listed two rigs out of three. A description that does not mention the Mac is a description that does not get loaded when the Mac capture is the thing that looks wrong.
The platform section had said the right thing since the text was fixed — "open one and it lies on the table in front of you, with the tortoise standing on the paper" — and illustrated it with `mac-en-1.png`, a Mac window. A reader was being told the drawing comes off the screen while looking at a screen. The picture only existed as of this branch; now it does, so the figure is the table. `DERIVED` gains a visionOS row, and loses the Mac star that the section was using: with the figure swapped, `mac-*-1.png` was referenced nowhere and the pass would have kept regenerating it forever. The Mac's remaining job on the page is the code pane. One shot, not three. Every visitor downloads both languages' images — that is what the quantize pass is for — and the drawing on the table is the whole of what the platform adds. Net effect on site/shots is +100KB: two files in at 184KB, two out at 85KB. The room's gradients quantize worse than flat app UI does, and there is faint banding in the ceiling if you look for it; at the size it is shown, it does not read.
temoki
changed the title
CLAUDE.md had reached 619 lines and the visionOS viewer was the largest single thing in it — a product that four files implement, documented in a file every task reads. It now lives at `App/Views/Viewer/CLAUDE.md`, and the four files moved into that directory with it: `App/` is a synchronized folder, so the move needed no project edit. A nested CLAUDE.md rather than a skill, deliberately. A skill is reached by its description, which works when someone says "work on the viewer" and not when they open TableCanvas.swift; a path does both. The viewer is also a separate product surface from the editor, so the directory earns its keep either way. Two passages shrank on the way, both because they had somewhere better to be. The 3D tortoise's reasoning is in Tools/tortoise-model/README.md, so what stays is the four-point contract app code may assume. And the screenshot rig is the `screenshots` skill now; what stays is the part that is app code rather than tooling — that -TBSheet overrides the constants the aimed placement is built from, since the simulator does report a usable head pose and an override written against the no-pose fallback does nothing at all. Root CLAUDE.md is 459 lines. The new file is excluded from the app target the same way `Views/CLAUDE.md` is: a nested CLAUDE.md inside a buildable folder ships in the bundle otherwise, silently, with the build at EXIT=0. Verified by looking in the built product — all three platform builds green, no CLAUDE.md in Contents/Resources.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
これは何
visionOS 版を公開できる状態にするための一式です。作業中にスクリーンショット撮影が3プラットフォームとも自動化され、そちらのほうが分量としては大きくなりました。
1. Vision Pro のストア文面と、それが指していた不具合
release_notes.txtの Vision Pro の段落は、visionOS が「iPad アプリを窓に入れたもの」だった #11 の頃のままでした。#53 がこれをビューワーに置き換えています。3ペインもなく、編集もできず、代わりに絵が机の上に出て 🐢 が紙の上を歩く。一度も push されていなかったので、直す以外のコストはありません。
visionOS の説明文は専用ディレクトリに分けました（
appstore/metadata-visionos/）。Vision Pro の App Store では、その客に見えるのは visionOS 版の説明文だけです。共通の文面だと冒頭から「ブロックをドラッグして並べ、数字をその場で直し…」と、そこでは唯一できないことを説明することになります。そして書いていて気づいたのですが、アプリ自体にも同じ実装漏れがありました。「ヘッドセットなら3つ同時に見られる」と言いながら、3つのウィンドウを全部同じ場所に重ねて開いていました。
defaultWindowPlacementで、ブロックを左・リモコンを中央・コードを右に配置しています（読む順で決めました。ブロックは絵の「もと」、コードはその「なれの果て」）。罠をひとつチェックにしました
name/subtitle/privacy_urlは ASC ではバージョンではなくアプリに属するフィールドです。どのレーンも同じものを書くので、最後に走ったレーンが3つのリスティング全部を決めます。2つのディレクトリで食い違うとアプリ名がレーンの実行順で決まる — しかも静かに。metadata_checkが落とします。2. スクリーンショットの自動化
3プラットフォームすべてがコマンド1つになりました。
発端は、毎回アルファチャンネルが付いてくることでした。 Figma も シミュレータも
simctl ioも RGB では書き出せず、App Store Connect はそれを拒否します。手で直すたびにサイトの派生画像を作り直し忘れる（2回やりました）ので、スクリプトにしました。方式はプラットフォームで違い、理由があります。visionOS はランチ引数（
-TBSample/-TBDraw/-TBSheet）。iPad と Mac は UI テストで、決め手は回転でした —simctlに回転コマンドはなく、Simulator のメニューを叩くにはシェルにキー入力送信の権限を渡すことになります。XCUIDeviceなら1行です。macOS はウィンドウだけを撮って台紙に合成するので、マシンの壁紙もメニューエクストラも時計も写りません。見つけた罠
共通して「完璧に撮れた、間違ったものの写真」を返してくるのが厄介でした。全部
screenshotsスキルに書いてあります。Disabled。ドラッグもadjust(toNormalizedSliderPosition:)も ⌘R も無言で何もせず、空のキャンバスが撮れますXCTAttachment(screenshot:)は縦のまま書き出す（横向きで撮っても 2064×2752 の横倒し）launch()の後。前だと静かに戻されますTEST_RUNNER_は xcodebuild の環境変数として。引数として置くと受理され、無視され、どこにも届きませんplay.fillが複数になりますPicker(.segmented)が iOS ではSegmentedControl、macOS ではRadioGroupドキュメントの誤りも1つ直しました。 CLAUDE.md は「visionOS シミュレータは
ViewAttachmentComponentをホストしない」と書いていましたが、します。症状は本物でしたが原因は別で、-TBPlaceが load の 後 にplace()を呼んでsitsOnTableを反転させ、それがRealityViewの.id()だったためシーンごと作り直されていました。3. ドキュメントの分解
screenshotsスキルを新設しました。撮影は「リリースする」のとは別のタイミング（UI が変わった日）に走るので、releaseから切り出しています。境界は作る／押し込むで、互いを指すだけで重複させていません。visionOS ビューワーの記述をコードの隣へ移しました。
CLAUDE.mdは619行に達していて、その中で最大の塊が「4ファイルで実装されている製品」の説明でした。ソース（TableCanvas/ViewerWindow/ProgramWindow/CodeWindow）をApp/Views/Viewer/に移し、App/Views/Viewer/CLAUDE.mdを置いています。スキルではなくネストした CLAUDE.md にしたのは、スキルは説明文から到達するためです。「ビューワーを直して」には効きますが、
TableCanvas.swiftを直接開いた時に効くとは限りません。パスなら両方に効きます。移動のついでに2箇所を畳みました。3Dカメは
Tools/tortoise-model/README.mdに理屈があるので契約4点だけ、撮影フラグはscreenshotsスキルへ譲ってアプリコードの話だけ残しています。結果は 619 → 459行。新しい CLAUDE.md は
PBXFileSystemSynchronizedBuildFileExceptionSetに追加済みで、バンドルに同梱されていないことを成果物の中を見て確認しました（同期フォルダ内の CLAUDE.md は、放っておくとContents/Resources/に入ります。ビルドは EXIT=0 のまま）。変更
appstore/metadata-visionos/appstore/screenshots/Tools/{ipad,macos,visionos}-shots.rbTools/screenshots.rbsite/shots派生TortoiseBlocksUITests/App/TortoiseBlocksApp.swiftApp/Views/Viewer/CLAUDE.mdfastlane/site/index.html,site/shots/.claude/skills/screenshots/確認したこと
ruby fastlane/metadata_check.rb→appstore/ looks sendable.このPRに入っていないもの
リポジトリ外の3つです。
buildDistributionAudienceは APP_STORE_ELIGIBLE）metadata_diffが "No editable xros version" で止まります）pages.ymlの手動実行（プライバシーポリシー §6 がまだ載っていません）