fix(deploy): stop worker bundles starting extra workers and unstick migrate deploy on fresh databases - #627
Merged
Merged
Conversation
…worker Job names now live in lib/queueNames.ts. The notification service imported JOB_CREATE_NOTIFICATION from the notification worker module, so esbuild inlined that worker into every bundle that sends a notification, and inside a CommonJS bundle `require.main === module` is true for every inlined module. Every worker process therefore also booted a notification worker, and the budget-alert worker booted lazily inside the LLM-backed workers. The worker build now fails if any bundle inlines a second entry point, and the image smoke test covers the seven workers it was missing. The run-ready check used a fixed job id with removeOnFail: false. BullMQ never re-adds an id it already holds, including one in the failed set, so a single failure blocked every later readiness check for that run. The check now uses a deduplication id with a 5s TTL and leaves failed jobs to the queue's default retention.
…tabase migrations The entrypoint called zenstack and tsx through npx. npm's first run in a new container asks the registry for a newer version and, without internet egress, waits out that request before the migration step starts. The globally installed binaries are now called directly, and Prisma's telemetry beacon is switched off. The migrate step also sets DEBUG=prisma:engines. On arm64 pods the first deploy against an empty database has hung indefinitely before the schema engine starts, and enabling that debug namespace is the only known way past it. The Helm migrate job gets the same two settings.
The production image pinned the global @zenstackhq/cli at 3.8.0 while the project's devDependency had moved to 3.9.3. The base stage now records the version its install resolved, and the production stage installs exactly that, so the CLI that runs migrate deploy at boot cannot drift from the one the project develops and tests against.
Contributor
Author
|
🎉 This PR is included in version 1.0.3 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
therealbrad
added a commit
that referenced
this pull request
Sep 8, 2026
…igrate deploy on fresh databases (#627) * fix(workers): stop each worker bundle from starting the notification worker Job names now live in lib/queueNames.ts. The notification service imported JOB_CREATE_NOTIFICATION from the notification worker module, so esbuild inlined that worker into every bundle that sends a notification, and inside a CommonJS bundle `require.main === module` is true for every inlined module. Every worker process therefore also booted a notification worker, and the budget-alert worker booted lazily inside the LLM-backed workers. The worker build now fails if any bundle inlines a second entry point, and the image smoke test covers the seven workers it was missing. The run-ready check used a fixed job id with removeOnFail: false. BullMQ never re-adds an id it already holds, including one in the failed set, so a single failure blocked every later readiness check for that run. The check now uses a deduplication id with a 5s TTL and leaves failed jobs to the queue's default retention. * fix(docker): keep container boot off the network and unstick fresh-database migrations The entrypoint called zenstack and tsx through npx. npm's first run in a new container asks the registry for a newer version and, without internet egress, waits out that request before the migration step starts. The globally installed binaries are now called directly, and Prisma's telemetry beacon is switched off. The migrate step also sets DEBUG=prisma:engines. On arm64 pods the first deploy against an empty database has hung indefinitely before the schema engine starts, and enabling that debug namespace is the only known way past it. The Helm migrate job gets the same two settings. * fix(docker): install the ZenStack CLI version the lockfile resolves The production image pinned the global @zenstackhq/cli at 3.8.0 while the project's devDependency had moved to 3.9.3. The base stage now records the version its install resolved, and the production stage installs exactly that, so the CLI that runs migrate deploy at boot cannot drift from the one the project develops and tests against. (cherry picked from commit 49afe64)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Three fixes for 1.0.3.
Worker bundles started the notification worker.
lib/services/notificationService.tsimported a job-name constant fromworkers/notificationWorker.ts, so esbuild inlined that worker into 22 of 23 bundles, and inside a CommonJS bundlerequire.main === moduleis true for every inlined module. Every worker process also booted a notification worker, the LLM-backed workers lazily booted the budget-alert worker, and their SIGTERM handlers raced toprocess.exit. Job names now live inlib/queueNames.ts, the worker build fails if any bundle inlines a second entry point, and the image smoke test covers the seven workers it was missing.Run-ready check blocked itself after one failure. The check used a fixed job id with
removeOnFail: false; BullMQ never re-adds an id it already holds, so one failed job silenced every later readiness check for that run. It now uses a deduplication id with a 5s TTL.Container boot depended on the network. The entrypoint went through
npx, whose first run in a new container waits on an npm registry request when the pod has no egress. The globally installedzenstackandtsxare now called directly, Prisma's telemetry beacon is off, and the migrate step carries theDEBUG=prisma:enginesworkaround for the fresh-database hang seen on arm64 pods. The Helm migrate job gets the same settings.The image's ZenStack CLI had drifted from the project's. The Dockerfile pinned the global
@zenstackhq/cliat 3.8.0 while the devDependency is 3.9.3. The base stage now records the version its install resolved and the production stage installs exactly that, so the CLI that runsmigrate deployat boot always matches the one the project develops against.Related Issue
N/A
Type of Change
How Has This Been Tested?
Full precommit green. Worker build gate: every bundle has exactly one start guard; the same gate against the previous tree reports 29 embedded entry points. Image smoke test loads all 24 bundles. Entrypoint run end-to-end in the v1.0.0 image on arm64 against a fresh database with no internet egress: migrations, triggers, extensions, and app start in 3s.
Test Configuration: