Skip to content

feat(cli): make migrate in-flight budget configurable, raise default to 50 GB - #233

Merged
designcode merged 3 commits into
mainfrom
feat/add-limit-options-in-cli-migrate
Aug 4, 2026
Merged

feat(cli): make migrate in-flight budget configurable, raise default to 50 GB#233
designcode merged 3 commits into
mainfrom
feat/add-limit-options-in-cli-migrate

Conversation

@designcode

@designcode designcode commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

What

tigris buckets migrate capped in-flight migrations — objects scheduled server-side but not yet confirmed — at a fixed 10 GB. Any single object larger than that could only run once the queue had drained, so a bucket of multi-gigabyte files migrated close to serially.

  • Default in-flight byte budget raised 10 GB → 50 GB, so several large objects can transfer concurrently.
  • New --max-in-flight-gb overrides it per run, accepting 1–100 (default 50). Out-of-range and non-numeric values are rejected before discovery lists the bucket rather than silently clamped, so the flag always means what it says. The option registers as [value], so a bare --max-in-flight-gb arrives as boolean true — that is an error, not Number(true) = 1 GB.
  • The budget moved from a module constant onto MigrationState.maxInFlightBytes, since it is now per-run configuration that both capacity checks read. atCapacity and shouldFlushBatch keep their signatures.

MAX_IN_FLIGHT_OBJECTS (1,000) and SCHEDULE_BATCH_SIZE (50) are unchanged.

Commits

Commit Change
7a36d89 feat(cli) — the change described above.
ef4690f chore(repo)biome.json $schema 2.5.4 → 2.5.6 via biome migrate --write. The @biomejs/biome devDep floats on ^2.4.13, so the binary had moved ahead of the pin and every biome check printed a migration notice. The schema URL was the only change; no deprecated config to migrate.
ffb8b14 docs(cli) — regenerated packages/cli/README.md. It is generated from specs.yaml by pnpm updatedocs and had gone stale, so this also picks up tigris init, buckets rebase, buckets merge, --default-tier, stdin input for objects put, and --snapshot-version on objects restore/restore-info. The buckets migrate section came out byte-identical to the version hand-written in 7a36d89, confirming those docs were already correct.

Notes for reviewers

This raise only helps large-object runs. The byte cap and the object cap cross over at budget / 1000, so the crossover moved from 10 MB to 50 MB average object size. Below that, MAX_IN_FLIGHT_OBJECTS = 1000 is still the binding constraint and this change has no effect there. If we later want to lift the object cap too, CONCURRENCY should move with it — drainCompleted polls in windows of CONCURRENCY, so a larger in-flight set stretches a full sweep proportionally.

Spec/code default drift guard. The spec's default: '50' is what commander actually supplies at runtime, so if it drifted from DEFAULT_MAX_IN_FLIGHT_GB the constant would become dead code and the CLI would quietly run the spec's number instead. A test asserts the two resolve to the same budget.

Test plan

  • tsc --noEmit clean
  • 36 tests in test/lib/buckets/migrate.test.ts pass — 12 new (parse bounds, fractional values, bare flag, budget overrides changing capacity decisions, spec-default drift guard). Two existing byte-budget cases were retargeted, since 22.8 GB no longer exceeds the budget: the original bug-report scenario is kept against an explicit 10 GB budget, with a new counterpart proving the same pair now batches freely at 50 GB.
  • Full CLI suite: 834 passed, 214 skipped (integration tests, no credentials)
  • Biome clean across 342 files
  • Smoke-tested the built CLI: --help renders the flag; 500, 0.5, abc and a bare --max-in-flight-gb each fail with the right message and exit 1, before any network or auth call

Release

Changeset included — minor for @tigrisdata/cli and tigris (kept in lockstep by the Changesets fixed group). The regenerated README ships with that release; the biome bump is repo tooling and never reaches npm.

🤖 Generated with Claude Code

…light-gb

`tigris buckets migrate` capped in-flight migrations — objects scheduled
server-side but not yet confirmed — at a fixed 10 GB. Any single object
larger than that could only run once the queue had drained, so a bucket of
multi-gigabyte files migrated close to serially.

- Raise the default budget to 50 GB, letting several large objects transfer
  concurrently.
- Add `--max-in-flight-gb` to override it per run, accepting 1 to 100.
  Out-of-range and non-numeric values are rejected before discovery lists
  the bucket rather than clamped, so the flag always means what it says.
  The option registers as `[value]`, so a bare `--max-in-flight-gb` arrives
  as boolean `true` — that is an error, not `Number(true)` = 1 GB.
- Move the budget from a module constant onto `MigrationState`, since it is
  now per-run configuration that both capacity checks read; `atCapacity`
  and `shouldFlushBatch` keep their signatures.
- Guard the spec default against the in-code default in tests. The spec's
  `default: '50'` is what commander supplies at runtime, so drift would
  make the constant dead code and quietly change the budget.

`MAX_IN_FLIGHT_OBJECTS` (1,000) and `SCHEDULE_BATCH_SIZE` are unchanged. The
two caps cross over at `budget / 1000`, so the raise only loosens runs
averaging more than 50 MB per object; below that the object cap still binds.

Assisted-by: Claude Opus 5 via Claude Code

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown

Greptile Summary

The PR raises the default migration in-flight byte budget from 10 GB to 50 GB and adds a validated per-run override.

  • Adds --max-in-flight-gb with an accepted range of 1–100 GB.
  • Stores the resolved budget in MigrationState and uses it in both batching and capacity checks.
  • Updates generated CLI documentation, release metadata, and migration tests for defaults, validation, and custom-budget behavior.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness or security failures identified.

The new option is correctly wired through Commander, validated before network access, initialized on the only production migration state, and consistently enforced by both batching and in-flight capacity checks.

Important Files Changed

Filename Overview
packages/cli/src/lib/buckets/migrate.ts Adds strict budget parsing and consistently applies the resolved per-run byte cap throughout migration scheduling.
packages/cli/src/specs.yaml Registers the new optional-value CLI argument with the intended 50 GB default.
packages/cli/test/lib/buckets/migrate.test.ts Covers parsing boundaries, bare flags, fractional values, default drift, and default/custom capacity behavior.
packages/cli/README.md Documents the new flag, accepted range, default, and an example invocation.
.changeset/cli-migrate-max-in-flight-gb.md Records the CLI feature and synchronized minor releases.

Reviews (1): Last reviewed commit: "feat(cli): raise migrate in-flight budge..." | Re-trigger Greptile

designcode and others added 2 commits August 4, 2026 15:23
The @biomejs/biome devDep floats on ^2.4.13, so a lockfile refresh moved the
binary to 2.5.6 while $schema stayed pinned at 2.5.4, making every `biome
check` print a migration notice. Applied with `biome migrate --write`; the
schema URL was the only change.

Assisted-by: Claude Opus 5 via Claude Code

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
packages/cli/README.md is generated from specs.yaml by `pnpm updatedocs` and
had not been regenerated since several commands landed. Documents `tigris
init`, `buckets rebase`, `buckets merge`, `--default-tier`, stdin input for
`objects put`, and `--snapshot-version` on `objects restore` and
`restore-info`.

No behavior change. The `buckets migrate` section is unchanged, which confirms
the hand-applied docs in 7a36d89 matched the generator exactly.

Assisted-by: Claude Opus 5 via Claude Code

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@designcode designcode changed the title feat(cli): raise migrate in-flight budget to 50 GB and add --max-in-flight-gb feat(cli): make migrate in-flight budget configurable, raise default to 50 GB Aug 4, 2026
@designcode
designcode merged commit 8c25b57 into main Aug 4, 2026
2 checks passed
@designcode
designcode deleted the feat/add-limit-options-in-cli-migrate branch August 4, 2026 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants