Publish preflight for use with Trusted Publishing

[tarcieri]

If we adopted a convention around release branch names, we could add an additional job to preflight releases with cargo publish --dry-run to help ensure the Trusted Publishing job will work when the release is tagged and we're not left with unpublishable releases that have already been tagged

[newpavlov]

Is it motivated by the ctutils v0.4.1 failure? (I am not sure how it happened, did you publish it manually first by habit?)

I am not sure the suggested complication is worth the trouble. It's certainly annoying to have failed CI runs and deployments, but I would prefer to have a way to somehow delete failed runs.

[tarcieri]

Yes. The problem was it has a dependency on the in-workspace cmov, and depended on changes in cmov which hadn't yet been published. These don't manifest until it comes time to publish.

I was able to work around the new issue by retroactively publishing cmov. That made it possible for me to manually publish ctutils without having to tag a new version (though I should probably do that anyway, but at least there's a release corresponding to the tag). However, it wasn't possible to use the Trusted Publishing workflow to do the publishing after cmov v0.5.3 had been released, which I'm a little confused by, as I thought publishing vendored but did not honor Cargo.lock but rather tried to use the latest versions of everything.

The really annoying thing is the failure doesn't manifest until you've actually tagged a version, and while I lucked out here, in other situations it could result in a version being tagged which is unpublishable.

[newpavlov]

Maybe we could somehow configure for --dry-run to run as part of release PR?

in other situations it could result in a version being tagged which is unpublishable

Since we do not (currently) use immutable releases, we always can delete tag from unpublishable commit and push it to a new one.

[tarcieri]

Maybe we could somehow configure for --dry-run to run as part of release PR?

That's exactly what I'm suggesting

Since we do not (currently) use immutable releases, we always can delete tag from unpublishable commit and push it to a new one.

I personally think of tags as being immutable regardless and that you should never retag. I would just do another release at that point, and there would be no actual release corresponding to the version number that was tagged but botched. But if we can avoid it by actually testing publish with --dry-run, it could avoid the problem in advance.

[newpavlov]

That's exactly what I'm suggesting

Ah, I misunderstood you then. I thought your suggestion was to add a pre-release branch and some kind of automated merge from it to master after --dry-run has succeeded.

So your suggestion is to run --dry-run for crate-name on branches named like release/crate-name-vx.y.z? I think we could try it. Though it would mean that mass-release PRs like RustCrypto/hashes#812 would not trigger it, but I guess it's fine.

[tarcieri]

Yes, though I suppose in the interim I could try to run cargo publish --dry-run myself prior to tagging. Having it in CI would be nice though.

[newpavlov]

If we are to use some kind of labeling of release PRs, then it may be worth to also create release tags using CI job after the publish job has succeeded (with accompanying branch and tag protection rules). In other words, the publish job would be triggered by merge of PRs marked with a "release" label, not by release tags. It would be a pretty involved config, but should be doable.

[newpavlov]

Should we use labels or origin branch name to trigger the workflows? The former looks like a better option on the first glance since it should allow to release multiple crates in one PR, on the other hand, we risk polluting the list of labels.

It would be nice to see a project in the wild which does something similar.