Skip to content

operator: image registry override cannot express non-azure registries #574

Description

The operator resolves component images with a hardcoded /azure/ path segment, but release images are published flat under the org. The two only agree because the GitHub org is literally named azure. For any other org, mirror, or fork there is no value of UNBOUNDED_IMAGE_REGISTRY that produces a correct reference.

Two independent paths

The operator's own image is baked at render time:

CONTAINER_REGISTRY ?= ghcr.io/azure                                              Makefile:13
UNBOUNDED_OPERATOR_IMAGE ?= $(CONTAINER_REGISTRY)/unbounded-operator:$(VERSION_TAG)  Makefile:85
  -> --set OperatorImage=...                                                     Makefile:1077
  -> image: {{ .OperatorImage }}                          deploy/unbounded-operator/04-deployment.yaml.tmpl:49

Component images are resolved at runtime from a separate value:

UNBOUNDED_OPERATOR_IMAGE_REGISTRY ?= ghcr.io                                     Makefile:88
  -> --set ImageRegistry=...                                                     Makefile:1078
  -> UNBOUNDED_IMAGE_REGISTRY: "ghcr.io"                  deploy/unbounded-operator/03-configmap.yaml.tmpl:24
  -> envFrom -> --image-registry default                  cmd/unbounded-operator/main.go:76
  -> Config.Image()                                       internal/operator/component/env.go:74-76
func (c Config) Image(repository string) string {
    return strings.TrimRight(c.ImageRegistry, "/") + "/azure/" + repository + ":" + c.ImageTag
}

The defaults are ghcr.io/azure and ghcr.io: different variables, no relationship. release.yaml:805-809 passes only CONTAINER_REGISTRY, so UNBOUNDED_OPERATOR_IMAGE_REGISTRY always stays at its default.

Images are published flat

.github/workflows/release.yaml:365,753 push to ${REGISTRY}/<component>:<TAG> where REGISTRY = ghcr.io/${{ github.repository_owner }}. hack/cmd/release-bom/main.go:30-42 lists the same flat names. There is no /azure/ segment beyond the org itself.

Why it works today

source of azure
ghcr.io/azure/unbounded-operator:v0.3.0 CONTAINER_REGISTRY
ghcr.io/azure/machina:v0.3.0 string literal in Config.Image

Same string, two unrelated origins. Config.Image treats the org name as a fixed namespace inside the registry. It isn't.

Reproduction

Fork to org myorg and run the release workflow. Images publish to ghcr.io/myorg/<component>:<TAG>. Then:

UNBOUNDED_IMAGE_REGISTRY Resolves to
ghcr.io (default) ghcr.io/azure/machina:<TAG> wrong org
ghcr.io/myorg ghcr.io/myorg/azure/machina:<TAG> phantom /azure/

No setting yields ghcr.io/myorg/machina. The operator's own image is correct (it came from CONTAINER_REGISTRY), so it starts healthy and then deploys components pointing elsewhere.

Impact

Silent wrong-image pull is the main concern. If the tag also exists upstream the pull succeeds and the cluster runs upstream binaries with a fork's operator: healthy-looking, and a direct violation of the version-lockstep invariant the operator design depends on. If the tag doesn't exist upstream you get ImagePullBackOff, which is at least loud.

Air-gapped mirrors need a phantom directory. Mirroring to registry.corp.internal/unbounded/ requires the layout <prefix>/azure/<component>, because Config.Image appends /azure/ unconditionally. Undocumented and unguessable.

The two config fields use different conventions. UNBOUNDED_IMAGE_REGISTRY is a prefix that gets a path appended; the Deployment image: is a complete reference. Users applying unbounded-operator-<TAG>.yaml directly must edit both and know they differ. (envFrom is read once at pod start, so a post-apply edit also needs kubectl rollout restart deployment/unbounded-operator.)

This blocks #461: offline installation cannot be cleanly supported while the registry setting can't express a normal mirror layout.

Suggested fix

Treat the setting as a full image-repository prefix and drop the literal:

func (c Config) Image(repository string) string {
    return strings.TrimRight(c.ImageRegistry, "/") + "/" + repository + ":" + c.ImageTag
}

Default becomes ghcr.io/azure, identical to CONTAINER_REGISTRY, so UNBOUNDED_OPERATOR_IMAGE_REGISTRY can default from it instead of drifting. One concept, one value. Forks work; mirrors use the obvious layout.

Migration is required and must not be skipped. Existing clusters store a bare ghcr.io, and cmd/kubectl-unbounded/app/install.go:284-338 deliberately preserves it across re-runs. Under the new semantics that resolves to ghcr.io/machina. Either normalize on read (a registry with no path segment means <host>/azure) or rewrite the value explicitly during install.

Suggested tests

  • Config.Image table test covering bare host, host+org, host+multi-segment path, trailing slash.
  • Migration test: a preserved bare ghcr.io still resolves to the correct upstream images after upgrade.
  • Consider extending internal/operator/manifests_guard_test.go to assert the rendered operator ConfigMap registry and the rendered operator image share a prefix, which would have caught this.

Not included

The raw-kubectl apply UX gap (no supported way to generate a customized manifest without building from source) is real but separate. kubectl unbounded install --dry-run -o yaml would address it and gets easier once both fields use the same convention. Can be filed separately.

Context

Found while documenting the control plane upgrade workflow in #567.

Metadata

Metadata

Labels

bugSomething isn't workinggoPull requests that update go code

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions