feat(aws): add Lightsail CloudFormation template#142
Draft
defangdevs wants to merge 2 commits into
Draft
Conversation
Add `aws/lightsail-template.yaml`, a separate (not a toggle) template that runs the same services.agent-box on AWS Lightsail for one flat monthly bundle price — folding compute, SSD, static IPv4, and a multi-TB transfer allowance into a single line item (no separate EBS/IPv4 charges). Lightsail has no NixOS blueprint and CloudFormation can't boot a custom image, so the box launches the Ubuntu 24.04 blueprint and converts itself to NixOS in-place with nixos-infect (pinned by commit). PROVIDER=lightsail is first-class in nixos-infect and imports the same virtualisation/amazon-image.nix the EC2 template uses, so the platform layer is the proven one; the baked configuration.nix is the EC2 config minus Spot/NAT64 plus the Lightsail bits (amazon-image import + GRUB on nvme). selfUpdate, the web-password-hash activation script, the first-boot WaitCondition signal, and the amazon-init disable all carry over. Design notes: - No VPC/subnet/IGW/SG/EIP/IAM — Lightsail manages networking; firewall is the Networking.Ports block (443 always, 22 when DebugSsh=true). - A static IP is always attached (free while attached, stable across stop/start). It attaches after instance creation, so to avoid a dependency cycle the box discovers its settled public IPv4 at runtime for the sslip.io hostname; Outputs report the same address via GetAtt. - Debug access is root SSH with the Lightsail default key (no SSM); infect logs at /var/log/agent-box-infect.log. Validation: cfn-lint passes and the generated configuration.nix parses as valid Nix; aws-ci.yml now lints both templates. The end-to-end nixos-infect bootstrap still needs a live deploy-test (documented as a follow-up, along with S3 publish + a Launch button), so this ships as a draft. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sw7j9FYnbChZJbYArxbhuo
lionello
reviewed
Jul 22, 2026
Collaborator
There was a problem hiding this comment.
add at least one E2E test
lionello
approved these changes
Jul 22, 2026
Addresses the review ask to include at least one end-to-end test for the Lightsail template rather than deferring it. aws-ci.yml only lints; the existing deploy-test.yml deploys the EC2 template only. Adds a standalone deploy-test-lightsail.yml (create -> wait -> assert -> smoke -> destroy). Lightsail is IPv4-native and GH runners are IPv4-only, so this single leg runs the full browser-terminal smoke suite (auth 200, unauth 401, tabbed workspace, ttyd HTML, WebSocket upgrade, live-tmux via scripts/ws_smoke.py). Reuses the OIDC + module-pin conventions; drops the EC2-only bits (ipv4/ipv6 matrix, Spot, SSM, serial console) since the FirstBootDone WaitCondition already gates CREATE_COMPLETE on the first nixos-rebuild switch. DebugSsh=false keeps 22 closed on the public box. BundleId/WebPassword are passed via the step env so the shell never expands the "$12" price annotation as a positional parameter (which would fail AllowedValues validation). Also adds the new workflow to aws-ci.yml trigger paths for symmetry with deploy-test.yml. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wx9PVLmDbzS35JCN5N85i6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
aws/lightsail-template.yaml— a separate template (not a toggle on the EC2 one) that runs the sameservices.agent-boxon AWS Lightsail, priced as one flat monthly bundle. Motivated by a cost comparison: Lightsail folds compute + SSD + a static IPv4 + a multi-TB transfer allowance into a single price, with no separate EBS/public-IPv4 line items.t4g.smallsmall_3_0$13.4/mo on-demand-equiv ($12.6 measured on Spot)How it works (nixos-infect)
Lightsail has no NixOS blueprint and
AWS::Lightsail::Instancecan't boot a custom image, so the box launches the Ubuntu 24.04 blueprint and converts itself to NixOS in-place withnixos-infect(pinned by commit).UserDatapre-writes/etc/nixos/configuration.nixbefore running nixos-infect; the script'smakeConf()early-returns when that file exists, so it never clobbers our config.PROVIDER=lightsailis first-class in nixos-infect: it relabels root tonixosand importsvirtualisation/amazon-image.nix— the same module the EC2 template uses. So the platform layer is the proven one; only the delivery is new.amazon-image.niximport, GRUB on/dev/nvme0n1).selfUpdate, the web-password-hash activation script, the first-bootWaitConditionsignal, disk-GC, and theamazon-initdisable all carry over unchanged.Differences from the EC2 template
Networking.Portsblock (443 always, 22 whenDebugSsh=true).PublicIpv4/Nat64; no Spot → noUseSpot; fixed bundle disk → noRootVolumeSize.sslip.iohostname;Outputsreport the same address viaGetAtt StaticIp.IpAddress.DebugSsh=trueopens 22), not SSM. Infect logs at/var/log/agent-box-infect.log.Validation
cfn-lintpasses on both templates (aws-ci.ymlnow lints both).configuration.nixparses as valid Nix (rendered theFn::Sub,${!x}→${x}, substituted params + sed placeholders,nix-instantiate --parse).Known risk to validate on first launch
The runtime IP-settle poll (
sleep 150+ stability check) assumes the static IP attaches within the sleep window. The margin is large (attach ≈ 1–2 min; the NixOS closure build + reboot takes 10+ min), but a live run should confirm the bakedsslip.iohost matches the attached static IP.Follow-ups (deferred to keep this diff focused)
deploy-test.yml(create → assertWebURLreachable over IPv4 — runners are IPv4-only and Lightsail is IPv4-native, so this can smoke the live URL — → tear down).publish-template.ymlcurrently publishes onlytemplate.yamland scopes the bucket policy to that one object; a second template needs both objects covered by one policy (they must not fight overPutBucketPolicy).Deploy instructions (CLI) and the full rationale are in
aws/README.md→ "Lightsail variant".🤖 Generated with Claude Code