Release 1.7.0 — Flutter 3.47.1 and the archive-mode guard - #66
Conversation
The Runner project runs no Dart build. "Embed App.framework" and "Copy flutter_assets" copy whatever the last `flutter-tvos build/run` staged into tvos/Flutter, and the engine is the Flutter.xcframework that same run copied in, so Xcode's CONFIGURATION never had any influence on the Flutter payload. Archiving Release after a debug run therefore shipped the debug/JIT engine and a kernel_blob.bin inside a release app. That build runs from Xcode, because a development signature permits the JIT pages the VM needs. Installed from TestFlight or the App Store, where the distribution signature carries no get-task-allow, the VM cannot get executable pages: the app launches, the launch screen paints, and no Flutter frame ever arrives. Verified end to end - two builds from one source, same Release configuration, differing only in what was staged: the clean one runs, the one archived after a debug run shows a blank screen. Nothing upstream catches it either, altool --validate-app and App Store processing both accept it. Generated.xcconfig now records FLUTTER_BUILD_MODE, and the app template gained a "Check Flutter build mode" phase - first in the target, ahead of anything that copies a payload in - that fails the build on a mismatch and names the command to run. A profile payload under Release only warns, since the CLI drives profile builds through that configuration and they do run; an unset FLUTTER_BUILD_MODE from an older CLI warns rather than fails. project.pbxproj is written once at create time and never rewritten, so existing projects keep archiving unguarded until regenerated. Device builds now warn when the phase is absent.
The existing assertions only prove the phase exists and still contains the right strings. An inverted condition, or a case pattern that stopped matching Release, would leave all of them intact and still archive an app that cannot start - which is the whole failure this phase is meant to prevent. Extract the phase's shellScript from both pbxproj files, unescape it, and run it under /bin/sh against a throwaway project directory: Release staged for debug fails and names the fix, Release staged for release and Debug staged for debug pass, profile under Release and an unset FLUTTER_BUILD_MODE only warn, a release build with no App.framework staged fails, and an unrecognized configuration is skipped rather than guessed at. Checked against a mutant: inverting the mode comparison in the template fails five of the seven cases, while the example's untouched copy stays green.
Moves the pinned SDK to 3.47.1 and the artifacts to the engine built against it, and cuts the release with the archive-mode guard already on this branch. Nothing tvOS-facing moved between 3.47.0 and 3.47.1: none of the files the tvOS patch set touches changed, `shell/platform/darwin` is untouched, and the five Dart SDK files behind the platform-identity patches are identical blobs at both revisions. The engine was rebuilt anyway, because `dart_revision` moved and AOT snapshots are keyed to the Dart SDK hash — profile and release builds against the previous artifacts would fail to load. The two pins move together for that reason. Verified before publishing: 118 engine unit tests green on both tvOS 17.5 and 26.5, and the full build-and-run matrix green on the 17.5 simulator and a physical Apple TV 4K across debug, profile and release — launch, VM service, hot reload and restart, platform identity and FFI symbol export in each. All six artifacts pass their checks, including origin signing on all four tvOS engines.
DenisovAV
left a comment
There was a problem hiding this comment.
Reviewed with five agents plus Codex, in parallel. Two of them extracted the guard script out of the pbxproj and ran it across a matrix; one validated the phase against real xcodebuild 26.5. Findings below are deduplicated, and I verified the load-bearing ones against the source myself.
The diagnosis in the PR body is excellent — the 13,833,216 vs 102,709,504 table, and the observation that nothing upstream of the device flags it, is the kind of write-up that makes a bug impossible to argue with. The phase mechanics are right too: exit 1 from a PBXShellScriptBuildPhase genuinely fails archive, buildActionMask/runOnlyForDeploymentPostprocessing/alwaysOutOfDate are all load-bearing and all correct, object IDs are unique in both files, and the script has no grep/sed pipeline that could rot into an empty string. The tests execute the real script rather than matching its text, which is above the bar.
What follows is where the guard can be green while the bad archive ships, and one place where it fails a build that was fine.
Blocking
1. The remediation this PR prescribes is a no-op
lib/commands/tvos_runner.dart:40
if (!templateDir.existsSync() || targetDir.existsSync()) {
return;
}flutter-tvos create . returns immediately — silently, before the Generating tvOS runner... status — whenever tvos/ exists. That is true for every project the warning targets. --overwrite does not reach this code. Nothing else in lib/ ever writes project.pbxproj.
The advice appears three times in this PR: the new warning text (application.dart:1089-1091), doc/publish-app.md:44, and CHANGELOG.md:37-38. So the user sees the warning, runs the command, gets no error and no output, rebuilds, sees the identical warning — and either concludes the tool is broken or believes they are now protected and archives anyway.
Four of six reviewers found this independently. It is pre-existing behaviour — three earlier _warnIf* migrations give the same broken advice — but this is the first one where following it is the only thing standing between the user and a dead submission.
Either splice the phase into an existing pbxproj (a blind re-render would clobber DEVELOPMENT_TEAM, bundle id and user-added phases, so it has to be surgical), or make renderTvosRunner say out loud that it declined, and change the three texts to advice that works today.
2. The marker is written in the middle of staging, so an interrupted build leaves it lying
lib/build_targets/application.dart — 421 _copyFlutterFramework, 424 _copyFlutterAssets, 427 _generatePluginRegistrant, 431 compileAotSnapshot (skipped for debug), 435 _generateXcconfigs, 445 _generateSwiftPackages.
By line 435 the payload has already been replaced. Anything that throws or is interrupted between 421 and 435 leaves Generated.xcconfig describing a payload that is no longer on disk.
Concretely, and needing no failure at all beyond a Ctrl-C: last build was --release, so the marker says release and a release App.framework is staged. Run --debug. Line 421 swaps in the JIT engine, 424 stages kernel_blob.bin, 431 is skipped so the release App.framework survives, and the user interrupts at 427. Marker still release. Archive Release: mode check passes, App.framework check passes, debug engine ships. Issue #65 verbatim, with the guard's blessing.
The window on the other side is real too — 445 is where the engine symlink is actually created, two steps after the marker.
Three of the reviewers proposed three different orderings for the fix; none of them work alone. Writing earlier turns the failure into a false positive, which is safer but still wrong; writing later leaves the old marker over a new payload. The shape that holds is two-phase: invalidate the key at the top of build() — write unknown, or delete it — and write the real value only once every staging step has succeeded. That depends on finding 3.
3. "Cannot determine the mode" passes, and its only backstop is already defeated
Script: STAGED="${FLUTTER_BUILD_MODE}"; if empty → warning: → exit 0.
The PR justifies this as "an older CLI", but it is also what you get after flutter-tvos clean (which deletes Generated.xcconfig, clean.dart:28), on a fresh clone or CI checkout since that file is gitignored, and on any configuration whose baseConfigurationReference is not Debug.xcconfig/Release.xcconfig.
The backstop is [ ! -d "${PROJECT_DIR}/Flutter/App.framework" ]. I checked: nothing in a normal build ever deletes App.framework — only clean.dart:26 does. The AOT step is skipped for debug, and _copyFlutterFramework deletes only Flutter.framework. So a release App.framework survives indefinitely across debug builds. An agent ran the combination: CONFIGURATION=Release, mode unset, stale App.framework present → exit 0, one warning: line in a multi-thousand-line archive log.
For EXPECTED=release, an undeterminable mode has to be exit 1. Better still, check the payload rather than its label, since a label can go stale (finding 2) while the payload cannot — fail a Release build when Flutter/flutter_assets/kernel_blob.bin exists. Your own PR body identifies that file as the debug tell.
4. FLUTTER_BUILD_MODE is upstream's user-facing override, and a target-level value defeats the guard silently
application.dart:1759. Upstream reads ${FLUTTER_BUILD_MODE:-${CONFIGURATION}} and treats it as the mode the user wants — which is exactly why upstream never writes it into Generated.xcconfig. It is the standard advice for Flutter users with flavors or custom configurations to set it themselves.
Generated.xcconfig is only the base configuration, so a target-level definition shadows it. An agent verified this on Xcode 26.5 with the same Generated.xcconfig in both cases:
target-level override present: GUARD mode=[release]
no target-level override: GUARD mode=[debug]
A user who carried that habit over makes the guard compare their declared intent against itself. It always matches, always passes, and looks green — on precisely the archive it exists to stop. FLUTTER_STAGED_BUILD_MODE removes the collision and costs nothing.
Worth fixing before release
5. The guard checks the mode but not the platform. flutter-tvos build tvos --simulator --release, then Product ▸ Archive for a device: marker release, configuration Release, App.framework present — green — and the archive embeds a simulator-SDK App.framework and the simulator engine slice. Same failure class, same green light. Recording buildInfo.sdkName and comparing it against ${PLATFORM_NAME} in the same phase closes it.
6. Unrecognized configurations disable the guard at note: severity. Staging, AppStore, QA, Prod all fall through to exit 0. note: is Xcode's lowest level — absent from the issue navigator, dropped by xcbeautify and most CI filters. The guard announces its own disablement in the one severity nobody reads, to the population most likely to need it. warning: at minimum, plus an explicit opt-out setting so unusual projects silence it deliberately rather than by accident.
7. It hard-fails in the direction where nothing is broken, with a message that is wrong there. Debug or Profile configuration with a release payload → exit 1, and the error says the app "would hang on a blank screen when installed from TestFlight" — a Debug-configured build is never submitted, and an AOT payload under a Debug shell runs fine. The trigger is ordinary: build tvos --release, then ⌘R in Xcode, or Product ▸ Profile. The property that actually matters is JIT-vs-AOT, not string equality: the only fatal pairing is a debug payload under a non-Debug configuration. Inverting it — hard-fail on the JIT/AOT mismatch, warn on every other disagreement — would also retire the hand-carved profile-under-Release exception.
8. The suite green-lights deleting half the guard. An agent mutated the script two ways: narrow enforcement to Release only, and delete the Profile* arm entirely. Both pass all seven tests on both pbxproj files. Uncovered: Debug + release payload, Debug + profile payload, Profile configuration with anything, and CONFIGURATION unset. Three one-line tests against the existing runGuard helper kill both mutants.
Nothing asserts the chain the guard depends on either — Generated.xcconfig → #included by Debug/Release.xcconfig → baseConfigurationReference on each configuration. runGuard injects the variable straight into the process environment, so if someone drops the include or adds a configuration without a base reference, the guard degrades to warn-only and all fourteen cases stay green.
Smaller
--jit-releaseyieldsjit_release, which thecasecannot map, so the CLI's own build hard-fails advising--release.tvos_device.dart:460already rejects it forrun;buildshould too.pbxprojLacksBuildModeGuardgreps the whole file for the phase name. A phase object left in the file but unwired frombuildPhasesreports "guarded". The tests already assert the two things that matter — thebuildPhasesreference and its position ahead ofEmbed App.framework; the runtime predicate should assert the same._warnIfMissingBuildModeGuardreturns silently on an unreadable pbxproj — a binary-plist project file gets neither the guard nor the warning about not having it. AprintTracecosts nothing.- Every CLI
--profilebuild trips the profile-under-Release warning, becauseapplication.dart:471drives profile through-configuration Release. It is swallowed for CLI builds (stdout is only printed on non-zero exit), so it only ever reaches people building from Xcode — as noise on a supported path. - "First in the target" is true for the template but not the example, where CocoaPods' manifest check sits at index 0 — and every generated project converges on that after the first
pod install. Harmless, but the comment, CHANGELOG and PR body all state it. "Before any phase that copies a payload" is both true and the property that matters. inputPathsis inert underalwaysOutOfDate = 1— verified, the phase runs even when the declared input does not exist. Worth a comment saying it is documentation-only, since a reader reasonably assumes a missing file would be noticed.doc/architecture.md's Build Flow section still lists the xcconfig step without mentioningFLUTTER_BUILD_MODEor the guard.
Where I land
Findings 1 through 4 each reduce the guard to decorative in a scenario that needs no unusual setup, and 1 actively persuades the user they are protected when they are not. I would hold on those. 5 through 8 decide whether the phase is trusted or learned-to-ignore, which for a tripwire is close to the same question.
None of this is an argument against the change. The bug is real, the diagnosis is unusually good, and a tripwire is the proportionate shape for it given Xcode runs no Dart build here. It is worth being explicit somewhere that this detects the hazard rather than removing it — the coupling itself only goes away with per-configuration staging directories or Xcode invoking the tool, and that is a much bigger change than this PR should carry.
Review found the guard green in scenarios that need no unusual setup. All four are real; each is addressed here. **The remediation was a no-op.** `renderTvosRunner` returns as soon as `tvos/` exists — before its own status line, and with no `overwrite` parameter for `--overwrite` to reach — so `flutter-tvos create .` did nothing, silently, for every project the warning targets. The user cleared no warning and archived believing they were protected. It now says it declined and why, and the three places that gave that advice (the warning, doc/publish-app.md, CHANGELOG) lead with the mitigation that works today: run the release build immediately before each archive. **The marker was written mid-staging.** `_generateXcconfigs` ran at step 5, after the engine, assets and registrant had already been replaced. A build interrupted in that window left a marker describing a payload no longer on disk: release build, then Ctrl-C a debug build after the JIT engine and kernel_blob were staged but before the marker moved, and the guard blesses issue #65 verbatim. `build()` now writes `unknown` before touching anything and the real value only once staging has finished, so the failure mode is a rejected build rather than an accepted one. **An undeterminable mode passed.** Empty is the state after `flutter-tvos clean`, on a fresh checkout (Generated.xcconfig is gitignored) and with an older CLI — not just the last of those. Its backstop was the App.framework check, which cannot help: nothing in a normal build deletes App.framework, so a release one survives any number of debug builds. Under Debug this still warns; under anything else it now fails. A payload check runs first and is immune to a stale marker entirely: a `kernel_blob.bin` under a non-debug configuration proves the payload is JIT whatever the marker claims. **The marker collided with upstream's override.** `FLUTTER_BUILD_MODE` is what upstream tells users to set themselves, and `Generated.xcconfig` is only the base configuration, so a target-level definition shadowed it and the guard compared the user's declared intent against itself — matching every time. It is now `FLUTTER_STAGED_BUILD_MODE`, which nothing else writes. Also, being cheap next to the above: the phase records `FLUTTER_STAGED_SDK` and fails when a simulator payload is built for a device, the unrecognized- configuration path announces itself at `warning:` rather than `note:` (which xcbeautify and most CI filters drop) and gained an explicit opt-out, and the suite gained the cases that were missing — Debug with a release or profile payload, the Profile configuration, an unset CONFIGURATION, a stale marker over a JIT payload, and SDK mismatch. Deleting the Profile arm or the payload check now fails tests; both previously passed everything. 396 tests green, analyzer clean.
The new payload check failed a profile build that should have passed, and it was right to: `build/tvos/` is shared across modes and nothing there removes `kernel_blob.bin`, which only a debug build writes. `copyFlutterAssetsTree` mirrors that directory faithfully, so a profile or release build run after any debug build staged a 43 MB stale debug kernel and shipped it inside the AOT app. The engine ignores it there, which is why this went unnoticed — it is dead weight in the bundle rather than a crash, and it hands out the app's Dart kernel with a release build. Dropped at staging time rather than by cleaning the shared output directory, which incremental builds rely on.
Extracted from _copyFlutterAssets so the behaviour is testable without constructing a build: the removal is what keeps the new payload check from failing legitimate AOT builds, so it should not rest on an e2e run noticing.
|
All four blocking findings were real — I verified each against the source before touching anything, and none of them needed an unusual setup to reach. Fixed in Blocking1. The remediation was a no-op. Confirmed, and worse than described: the early return sits above the 2. The marker was written mid-staging. 3. An undeterminable mode passed. Now fatal under any non-debug configuration, warning under Debug. The 4. The marker collided with upstream's override. Now Also in: The guard caught something on its first runWorth reporting on its own, because it is not in your list and nobody had noticed it. With the phase in place, a profile build failed — correctly. The engine ignores it there, which is exactly why it survived: not a crash, just dead weight in the bundle and the app's Dart kernel handed out with a release build. Fixed at staging time rather than by cleaning the shared output directory, which incremental builds rely on.
Verification398 unit tests and the analyzer are clean. The full build-and-run matrix is green again on the tvOS 17.5 simulator and a physical Apple TV 4K — 35 checks across debug, profile and release, including hot reload and restart, platform identity and FFI symbol export. That matrix is what surfaced the kernel blob, and it is green with the guard active rather than with it bypassed. Not in this PRFinding 7 (inverting the hard-fail to JIT-vs-AOT rather than string equality) and the seven smaller items. 7 is the one I would take next — it retires the hand-carved profile-under-Release exception, and the payload check added here already moves in that direction, so the two want doing together rather than piecemeal. You also noted that three earlier And your closing point deserves recording somewhere permanent rather than in a review thread: this detects the hazard, it does not remove it. The coupling goes away only with per-configuration staging directories or Xcode invoking the tool. |
DenisovAV
left a comment
There was a problem hiding this comment.
Re-reviewed against the code rather than the commit messages. All four blocking findings are closed, and three of them by a better fix than the one I proposed.
1 — the remediation that did nothing. renderTvosRunner now says out loud that it declined (tvos_runner.dart:44-58), and all three texts that prescribed the command were corrected. The warning now leads with the mitigation that actually works ("Run flutter-tvos build tvos --release immediately before every archive. That is the whole mitigation, and it works today.") and states plainly that create . will not add the phase to an existing tvos/. doc/publish-app.md:44 matches.
2 — the marker written mid-staging. Two-phase, as prescribed: step 0 writes unknown before anything mutates tvos/Flutter, step 5 writes the real value once staging has succeeded, and the script treats unknown as "a build began staging and did not finish". Steps 6–7 run after the marker but touch project wiring rather than the payload, so the marker is written exactly when the thing it describes is complete. Correct call.
3 — undeterminable mode passing. Now exit 1 for anything non-Debug, with the warning:-only path kept for Debug where the cost of being wrong is a launch failure in your hand. The App.framework check is re-scoped exactly right: its presence proves nothing, its absence still proves an AOT build cannot work.
4 — the FLUTTER_BUILD_MODE collision. Renamed to FLUTTER_STAGED_BUILD_MODE. Upstream's user-facing override can no longer make the guard compare a declared intent against itself.
Findings 5, 6 and most of 8 are closed too — the FLUTTER_STAGED_SDK / PLATFORM_NAME check, warning: plus an explicit FLUTTER_STAGED_BUILD_MODE_CHECK=off opt-out, and tests for the four combinations that previously let a mutated script stay green (Debug+release, Debug+profile, Profile+anything, CONFIGURATION unset).
The structural change is what makes this hold. Findings 2, 3 and 4 were three different ways to desynchronise the marker from reality; rather than patching three holes, the guard now decides on the artifact first — kernel_blob.bin on disk — and treats the marker as a more precise second opinion. A marker can go stale; a payload cannot. That retires the whole class.
stripJitPayload is a good catch that wasn't in the review: build/tvos/ being shared across modes meant a debug kernel_blob.bin was mirrored forward into AOT builds, shipping dead weight and the app's Dart kernel inside a release build.
Approving. Two follow-ups, neither blocking:
- Finding 7 — you kept the hard failure for a Debug configuration over a release payload, and the tests make that deliberate, so that is your call. My actual objection was the message, and it is fixed: the TestFlight/blank-screen wording now appears only in the
kernel_blobcheck, where it is true. - Finding 8, second half — the chain the guard depends on is still unasserted:
Generated.xcconfig→#included byDebug/Release.xcconfig→baseConfigurationReferenceon each configuration. Zero references acrosstest/.runGuardinjects the variable straight into the process environment, so dropping the include or adding a configuration without a base reference degrades the guard to warn-only with every test still green. Worth its own issue.
Every guard test injects FLUTTER_STAGED_BUILD_MODE straight into the process
environment, so all of them stay green even when the chain that delivers it in a
real build is broken:
Generated.xcconfig
-> #include'd by Flutter/Debug.xcconfig and Flutter/Release.xcconfig
-> baseConfigurationReference on each configuration of the Runner target
Drop the include, or add a configuration without a base reference, and the phase
sees an unset marker — a hard failure for release, a warning for debug — with
nothing failing until someone archives.
The xcconfig body moved into `buildModeXcconfig` so the include is assertable at
all. The pbxproj half walks the Runner *target*'s configuration list (the
project-level configurations carry no base xcconfig, by design), requires a
baseConfigurationReference on each, and resolves it to one of the two Flutter
xcconfigs rather than to any file.
Verified against mutants: deleting a baseConfigurationReference and repointing
one at an unrelated file both fail these tests, and neither did before.
401 tests, analyzer clean.
|
Closed the second half of finding 8 here rather than filing it — You were right that nothing asserted the chain, and right about why it mattered: Two halves:
Since the first half of finding 8 was that tests can bless a broken guard, I checked these do not repeat it. Deleting a 401 tests, analyzer clean. On finding 7: agreed, and thanks for separating the message from the behaviour — the TestFlight/blank-screen wording now appears only in the |
The existing release/flutter-3.32.8 branch carries the same port done on an older base (1.5.0 era, 17 commits behind dev). This branch redoes it on current dev instead, so the line picks up the archive-mode guard from #66 and the rest of the mainline work rather than a three-week-old snapshot. Its documentation is not discarded: the release-line banner was carried over in the previous commit, rebased onto the mainline's current README. Its code changes are superseded by the equivalent ones here — the two ports converged independently on the same shape. Recording the merge so the line's history is continuous and the diff shows only what actually changes.
Release 1.7.0. Two things ship: the archive-mode guard from #65, and Flutter 3.47.1 with engine artifacts rebuilt against it.
An archive no longer silently ships the wrong mode
Closes #65.
The Runner project runs no Dart build.
Embed App.frameworkandCopy flutter_assetscopy whatever the lastflutter-tvos build/runstaged intotvos/Flutter, and the engine is theFlutter.xcframeworkthat same run copied in — so Xcode'sCONFIGURATIONhas never had any influence on the Flutter payload. Archive Release right after a release build and all is well; let a debug build or run happen in between and Product → Archive packages the debug/JIT engine and akernel_blob.bininto a Release-configured app.That build runs from Xcode, because a development signature permits the JIT pages the VM needs. Installed from TestFlight or the App Store, where the distribution signature carries no
get-task-allow, the VM can't get executable pages at all: the app launches, the launch screen paints, and the first Flutter frame never arrives — a blank screen, no crash, no log.Verified on hardware — two builds from one source, same
-configuration Release, same signing, differing only in what was staged when the archive ran:build tvos --releasebuild tvos --debugNothing upstream of the device flags the bad one:
altool --validate-appclean, upload clean, App Store processingVALID, no ITMS warning, and Xcode saysARCHIVE SUCCEEDED.The fix.
Generated.xcconfigrecordsFLUTTER_BUILD_MODE, and the app template gains a "Check Flutter build mode" phase, first in the target so it runs before anything copies a payload in. It fails the build on a mismatch and names the command to run. A profile payload under Release only warns (the CLI drives profile builds through that configuration and they do run), and an unsetFLUTTER_BUILD_MODEfrom an older CLI warns rather than fails.Existing projects keep their old phase list —
project.pbxprojis written once atcreatetime and never rewritten on build — so they get a warning naming the risk instead. Regenerate the tvOS project (flutter-tvos create .) to pick up the guard itself.Flutter 3.47.1
Nothing tvOS-facing moved between 3.47.0 and 3.47.1: none of the files the tvOS patch set touches changed,
shell/platform/darwinis untouched, and the five Dart SDK files behind the platform-identity patches are identical blobs at both revisions. Upstream's changes land influtter_tools, the Linux and Windows embedders, and an Impeller compiler path fix.The engine was rebuilt regardless, because
dart_revisionmoved (da6595cd→852b3e36) and AOT snapshots are keyed to the Dart SDK hash — a profile or release build compiled against the previous artifacts fails to load with an SDK-hash mismatch. For the same reason the CLI and the engine move together;flutter-tvos precachepulls the pair.Verification
Engine unit tests: 118 cases, 0 failures, on both tvOS 17.5 (the deployment floor) and 26.5.
Full build-and-run matrix against the signed artifacts, on the tvOS 17.5 simulator and a physical Apple TV 4K — 35 checks, all green:
sim-debugdevice-debugdevice-profiledevice-releasePlatform identity resolves correctly in every scenario (
operatingSystem == "tvos",isIOSandisTvOSboth true,defaultTargetPlatformiOS), and the ten FFI symbols are exported fromRunner.debug.dylibin debug and fromRunnerdirectly under AOT.Artifact checks pass on all six variants: correct load-command platforms, the on-device JIT RWX fix present, the platform-identity patch in both host SDKs, the profile
_NetworkProfilingfix retained, and all four tvOS engines origin-signed with a Developer ID certificate and a secure timestamp — which is what keeps apps clear of ITMS-91065 at App Store review.