feat: add TransformStripCRDSchema for reduced cache memory usage

[rafal-jan]

Description of your changes

Fixes #1056

I have:

• Read and followed Crossplane's contribution process.
• Run ./nix.sh flake check to ensure this PR is ready for review.
• Added or updated unit tests.
• Linked a PR or a docs tracking issue to document this change.
• Added backport release-x.y labels to auto-backport this PR.

Need help with this checklist? See the cheat sheet.

This PR introduces a new cache optimization function, TransformStripCRDSchema, to reduce the memory footprint of Crossplane providers that watch CustomResourceDefinition (CRD) objects.

By default, the controller-runtime cache stores the entire schema for every watched object. For clusters with a massive number of CRDs storing the OpenAPI v3 validation schemas in the informer cache leads to significant memory bloat (hundreds of megabytes per provider pod).

TransformStripCRDSchema is a cache.TransformFunc that safely strips the following fields from CRD objects before they enter the informer cache:

• Spec.Versions[].Schema (OpenAPI v3 validation schemas)
• ObjectMeta.ManagedFields
• The kubectl.kubernetes.io/last-applied-configuration annotation

Providers can leverage this by wiring it into their manager's cache.Options.ByObject configuration. Since most controllers (like the CRD gate) only require basic metadata, group/version/kind names, and status conditions to function correctly, stripping these heavy fields is completely safe.

How has this been tested?

• Added comprehensive unit tests in pkg/reconciler/customresourcesgate/cache_test.go verifying that schemas, specific annotations, and managed fields are correctly stripped while leaving all other data and non-CRD objects intact.
• Validated via a local test bed using provider-template. Testing a provider against a cluster with 2,045 Upjet AWS CRDs showed controller memory usage dropping from ~630 MiB down to ~83 MiB after applying this cache transform.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a CRD cache transform that removes OpenAPI schemas and selected metadata while preserving non-CRD objects, with table-driven tests covering normal and edge cases.

Changes

CRD cache transform

Layer / File(s)	Summary
CRD transform behavior pkg/reconciler/customresourcesgate/cache.go	Adds TransformStripCRDSchema, which removes version schemas, managed fields, and the last-applied annotation from CRDs while passing through other objects unchanged.
Transform validation pkg/reconciler/customresourcesgate/cache_test.go	Adds table-driven coverage for multiple versions, empty versions, nil annotations, unrelated annotation preservation, and non-CRD passthrough.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: bobh66

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	This PR addresses the CRD cache optimization, but #1056 also requires disabling Secret caching, which is not implemented here.	Add the Secret cache bypass or clarify the linked issue scope so the PR fully covers the required memory optimizations.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	The changes are focused on the CRD cache transform and its tests, with no obvious unrelated additions.
Breaking Changes	✅ Passed	Only a unit test assertion changed; no exported Go APIs were removed, renamed, or had signature or behavior changes.
Title check	✅ Passed	The title is descriptive, under 72 characters, and matches the cache memory optimization change.
Description check	✅ Passed	The description is clearly related to the CRD cache transform and its tests, so it satisfies the lenient check.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/reconciler/customresourcesgate/cache_test.go`:
- Around line 241-243: Update the error comparison in the test around
TransformStripCRDSchema to pass cmpopts.EquateErrors() to cmp.Diff, and add the
corresponding github.com/google/go-cmp/cmp/cmpopts import while preserving the
existing error assertion.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0faf12c1-76d1-41f3-a1f9-fa0317e2e768

📥 Commits

Reviewing files that changed from the base of the PR and between 3346a95 and 43d5ef8.

📒 Files selected for processing (2)

• pkg/reconciler/customresourcesgate/cache.go
• pkg/reconciler/customresourcesgate/cache_test.go

[bobh66]

This LGTM but I'm far from an expert in this area, so might want someone else to take a look.

[jonasz-lasut]

Hi @rafal-jan thank you for contributing the PR!

I've done a short safety verification against consumers of this code - crossplane-runtime and upjet like we've discussed on slack.

• Confirmed (code): the gate reconciler
  (crossplane-runtime/pkg/reconciler/customresourcesgate/reconciler.go:40-86) reads
  only Spec.Group, Spec.Versions[].Name/.Served, Spec.Names.Kind,
  Status.Conditions (Established), and the deletion timestamp — all preserved by
  the transform.
• Confirmed (code): the gate is the only consumer of cached CRDs in
  crossplane-runtime. pkg/xcrd and pkg/xpkg generate/lint CRDs from package
  contents; neither reads the informer cache.
• Confirmed (code): upjet's CRD storage-version migrator is unaffected twice over:
  cmd/crdmigrator/main.go:102 builds a direct client (client.New, no cache),
  and UpdateCRDStorageVersion (upjet/pkg/config/crd_migrator.go:221-238) re-Gets
  the CRD and patches status with client.MergeFrom, so only the storedVersions
  diff goes over the wire — it cannot write a stripped schema back even if handed a
  cached client.
• Falsified: the concern that something watches Secrets. No
  Secret watch, Owns, or informer registration exists in upjet or
  crossplane-runtime. All consumers are point reads/writes: sensitive-parameter
  resolution (upjet/pkg/controller/api.go:57-63), credential extraction
  (crossplane-runtime/pkg/resource/providerconfig.go), and connection-secret
  publishing (crossplane-runtime/pkg/reconciler/managed/api.go, Get + Apply). The
  generated upjet controller watches only the managed-resource kind
  (pkg/pipeline/templates/controller.go.tmpl:153). External secret stores — the
  historical Secret-heavy subsystem — were removed in runtime v2.3.

From my perspective it's ok to get this merged in the current state

[jbw976]

/backport

[github-actions]

Successfully created backport PR for release-2.3:

• [Backport release-2.3] feat: add TransformStripCRDSchema for reduced cache memory usage #1109