Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/ci_run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ on:
required: true
type: string
profile:
description: 'Dependency profile declared in ci/dependency-profiles.json'
description: 'Dependency profile declared in dependencies.toml'
required: true
type: string
enable_reporting:
Expand Down Expand Up @@ -128,11 +128,11 @@ jobs:
with:
node-version: 'lts/*'

- name: "CHECK: {Resolve dependency profile}"
- name: "CHECK: {Resolve dependencies}"
id: dependencies
run: |
METADATA="$RUNNER_TEMP/ci-dependencies.json"
python3 scripts/resolve-ci-dependencies.py resolve \
METADATA="$RUNNER_TEMP/dependencies.json"
python3 scripts/resolve-dependencies.py resolve \
--profile '${{ inputs.profile }}' \
--output "$METADATA" \
--github-output "$GITHUB_OUTPUT" \
Expand Down Expand Up @@ -213,7 +213,7 @@ jobs:
- name: "CHECK: {Verify dependency checkouts}"
id: verified-dependencies
run: |
python3 scripts/resolve-ci-dependencies.py verify \
python3 scripts/resolve-dependencies.py verify \
--metadata "$CI_DEPENDENCY_METADATA" \
--github-output "$GITHUB_OUTPUT"

Expand Down
19 changes: 13 additions & 6 deletions README_ADVANCED.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,10 @@ branch = "main"

### How Defaults Work

Defaults are defined in code (see [`src/config.rs`](src/config.rs) `Config::default()`) and written to `config.toml` during `init`. This means:
Operational defaults are defined in code (see [`src/config.rs`](src/config.rs)
`Config::default()`), while dependency locations come from
[`dependencies.toml`](dependencies.toml). They are written to
`config.toml` during `init`. This means:

- **First-time setup:** Running `foc-devnet init` creates `config.toml` with current defaults from code
- **Updating defaults:** When a new version of `foc-devnet` includes updated defaults (e.g., newer Lotus version), run `foc-devnet clean --all` then `foc-devnet init` to regenerate `config.toml` with the new defaults
Expand Down Expand Up @@ -820,7 +823,9 @@ port_range_count = 100
- **[multicall3](https://github.com/mds1/multicall3)** - Multicall3 contract
### Dependent Version Strategy

Default versions for these repositories are defined in code (see [`src/config.rs`](src/config.rs) `Config::default()`).
Default versions for these repositories are defined in
[`dependencies.toml`](dependencies.toml) and consumed by
`Config::default()`.

**Version specification methods:**
- **Latest tag** (`latesttag`, `latesttag:<selector>`, `latesttag:<url>:<selector>`): Resolved once at `init` time via `git ls-remote` and pinned as a concrete `GitTag` in `config.toml`. Use a glob selector to scope which tags are considered, e.g. `latesttag:v*` or `latesttag:pdp/v*`. Bare `latesttag` matches all tags.
Expand Down Expand Up @@ -1330,15 +1335,17 @@ Reports are written to `~/.foc-devnet/state/latest/scenario_report.md`.

Scenarios run automatically in CI after the devnet starts. On nightly runs (or manual dispatch with `reporting` enabled), failures automatically create a GitHub issue with a full report.

CI resolves compatibility-sensitive dependencies from `ci/dependency-profiles.json`.
CI resolves compatibility-sensitive dependencies from `dependencies.toml`.
Pull requests use the pinned `default` profile, while nightly `stability` runs use
the latest final releases and nightly `frontier` runs pin current development
branch heads to immutable commits. Nightly CI also runs manifest-declared mixed
profiles such as `stability-frontier-curio`, where all dependencies come from
`stability` except the named component, which comes from `frontier`. In
`stability`, PDP comes from the stable filecoin-services checkout's bundled
submodule; in `frontier`, PDP is pinned as an independent repo. The resolved
submodule; in `frontier`, PDP is pinned as an independent repo. Mixed profiles
that override filecoin-services pin PDP to the configured bundled PDP commit so
the run does not unintentionally test two moving contract sources. The resolved
metadata path is exposed to scenarios as `CI_DEPENDENCY_METADATA`; Synapse SDK
and filecoin-pin also receive their exact source, version/ref, and commit
through `SYNAPSE_SDK_*` and `FILECOIN_PIN_*` environment variables. PDP receives
and filecoin-pin receive their exact source, version/ref, and commit through
`SYNAPSE_SDK_*` and `FILECOIN_PIN_*` environment variables. PDP receives
`PDP_SOURCE`, `PDP_REF`, and `PDP_COMMIT` when dependency profiles are resolved.
3 changes: 2 additions & 1 deletion build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,8 @@ fn main() {
println!("cargo:rerun-if-changed=.git/HEAD");
println!("cargo:rerun-if-changed=.git/refs/heads/");

// Re-run if MockUSDFC contract files change
// Re-run if embedded defaults or MockUSDFC contract files change
println!("cargo:rerun-if-changed=dependencies.toml");
println!("cargo:rerun-if-changed=contracts/MockUSDFC/src/MockUSDFC.sol");
println!("cargo:rerun-if-changed=contracts/MockUSDFC/script/Deploy.s.sol");
println!("cargo:rerun-if-changed=contracts/MockUSDFC/script/Verify.s.sol");
Expand Down
162 changes: 82 additions & 80 deletions ci/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# CI Dependency Profiles

`dependency-profiles.json` is the central manifest for CI dependency selection.
Its resolver is located in `scripts/resolve-ci-dependencies.py`.
`dependencies.toml` is the central manifest for CI dependency selection.
Its resolver is located in `scripts/resolve-dependencies.py`.

## Profiles

The manifest declares valid profiles in its top-level `profiles` object:
The manifest declares valid profiles in its top-level `profiles` tables:

- `default`: used by PR CI, a known-working set of client versions.
- `stability`: used by nightly CI to test stable releases.
Expand All @@ -19,72 +19,88 @@ The manifest declares valid profiles in its top-level `profiles` object:
- `stability-frontier-pdp`: used by nightly CI to test stable releases except
PDP, which is resolved from `frontier`.

Each component must define a selection for every component profile referenced by
the top-level profile definitions. Today those component selections are
`default`, `stability`, and `frontier`.

Top-level profile definitions have a `base` component profile and can override
specific components:

```json
{
"stability-frontier-curio": {
"base": "stability",
"components": {
"curio": "frontier"
}
}
}
Each dependency must define a selection for every profile selection referenced
by the top-level profile definitions. Today those selections are `default`,
`stability`, and `frontier`.

Profile definitions have a `base` selection and can override specific
dependencies:

```toml
[profiles.stability-frontier-curio]
base = "stability"

[profiles.stability-frontier-curio.components]
curio = "frontier"
```

In that example, Curio resolves from its `frontier` selection while every other
component resolves from `stability`. A profile is valid only if it is explicitly
dependency resolves from `stability`. A profile is valid only if it is explicitly
declared in `profiles`; for example, `stability-frontier-filecoin-pin` does not
exist unless added there.

## Component Fields
The root manifest is also embedded by foc-devnet itself. Rust `Config::default()`
reads the runtime dependency locations it needs from the manifest's `default`
selections, so CI and local defaults share one source of truth. Scenario helpers
read their own dependency entries from the same manifest.

Top-level component fields:
## Dependency Fields

Top-level dependency fields:

- `repository`: Git repository URL.
- `npm_package`: npm package name, for components that are resolved through npm
- `npm_package`: npm package name, for dependencies that are resolved through npm
metadata.
- `default`, `stability`, `frontier`: component profile selections.
- `default`, `stability`, `frontier`: dependency profile selections.

Dependencies use named TOML tables so each dependency stays easy to review and
edit:

```toml
[dependencies.lotus]
repository = "https://github.com/filecoin-project/lotus.git"

default = { strategy = "git_tag", tag = "v1.36.1" }
stability = { strategy = "git_tag", tag = "v*" }
frontier = { strategy = "git_branch", branch = "master" }
```

Profile selections always have a `strategy`. Some strategies require additional
fields.

### `config_default`
### `bundled`

Use the compiled `Config::default()` value and pass no runtime override to
`foc-devnet init`.
Use a git submodule bundled with a specific dependency selection.

```json
{
"strategy": "config_default"
}
```toml
strategy = "bundled"
bundle = "filecoin-services@stability"
path = "service_contracts/lib/pdp"
```

The `bundle` field uses `<dependency>@<selection>`. When the active bundled
dependency selection matches that reference, the resolver omits an init override
and lets foc-devnet use the bundled submodule. When a mixed profile changes the
bundled dependency away from that selection, the resolver emits an explicit
`gitcommit:...` override for the bundled commit so the run does not test two
moving contract sources by accident.

### `git_commit`

Use an exact Git commit SHA.

```json
{
"strategy": "git_commit",
"commit": "fadc836e65804311aca3bd2276861acabe42313f"
}
```toml
strategy = "git_commit"
commit = "fadc836e65804311aca3bd2276861acabe42313f"
```

### `git_branch`

Resolve a branch head to an immutable commit SHA before the run starts.

```json
{
"strategy": "git_branch",
"branch": "master"
}
```toml
strategy = "git_branch"
branch = "master"
```

The resolved metadata records both the branch name and the exact commit.
Expand All @@ -93,51 +109,41 @@ The resolved metadata records both the branch name and the exact commit.

Resolve a Git tag to an immutable commit SHA. `tag` can be an exact tag:

```json
{
"strategy": "git_tag",
"tag": "v1.2.3"
}
```toml
strategy = "git_tag"
tag = "v1.2.3"
```

`tag` can also be a pattern. Pattern selections choose the latest matching tag:

```json
{
"strategy": "git_tag",
"tag": "v*"
}
```toml
strategy = "git_tag"
tag = "v*"
```

By default, pattern selections exclude prerelease tags such as `-rc`, `-alpha`,
`-beta`, and development tags. Set `include_prereleases` to include them:

```json
{
"strategy": "git_tag",
"tag": "v*",
"include_prereleases": true
}
```toml
strategy = "git_tag"
tag = "v*"
include_prereleases = true
```

### `npm_version`

Resolve an npm version, range, or dist-tag to a concrete package version.

```json
{
"strategy": "npm_version",
"version": "1.0.1"
}
```toml
strategy = "npm_version"
version = "1.0.1"
```

The `version` field can also be an npm dist-tag:

```json
{
"strategy": "npm_version",
"version": "latest"
}
```toml
strategy = "npm_version"
version = "latest"
```

The resolver records the concrete package version selected at resolution time
Expand All @@ -149,17 +155,13 @@ Some profile selections can include an optional `overrides` object. Each entry
maps a package name to a `version` and a `reason` explaining why the override
exists:

```json
{
"strategy": "git_tag",
"tag": "synapse-sdk-v1.0.1",
"overrides": {
"nanoid": {
"version": "3.3.13",
"reason": "nanoid 5.x is ESM-only and breaks the CJS build"
}
}
}
```toml
strategy = "git_tag"
tag = "synapse-sdk-v1.0.1"

[dependencies.synapse-sdk.default.overrides.nanoid]
version = "3.3.13"
reason = "nanoid 5.x is ESM-only and breaks the CJS build"
```

Overrides are explicit profile policy. Both `version` and `reason` are required
Expand All @@ -182,8 +184,8 @@ Current consumers:

## Current Boundary

`resolve-ci-dependencies.py` resolves metadata. It does **not** install
components.
`resolve-dependencies.py` resolves metadata. It does **not** install
dependencies.

Installation currently lives in three places (which consume the resolved
metadata):
Expand Down
Loading