fix(e2e): restore readiness gates for runtime, auth, and localization by schultzp2020 · Pull Request #5083 · redhat-developer/rhdh

schultzp2020 · 2026-07-10T16:22:48Z

Summary

Restores and hardens RHDH E2E readiness after #5004 / #5022 broke nightlies, then deepens auth/runtime/operator install so defaults (especially lightspeed) cannot recreate the same failure class.

Instance readiness — globalSetup classifies BASE_URL as unset | router-stub | instance-url; runtime jobs keep a predicted URL + RUNTIME_AUTO_DEPLOY=true; auth leaves BASE_URL empty so the harness self-deploys.
Localization — drop smoke-test deps that asserted English "Welcome back!".
Shared operator install profiles — OperatorInstallProfile + DynamicPluginsProfile own apiVersion, flavours: [], includes: [], and BACKEND_SECRET for runtime and auth adapters.
Auth deploy — AuthInstanceDeployer stages created → HTTP /healthcheck → catalog synced; config liveness, IdP waits, cookie/session isolation, and OIDC autologout hardenings for reconcile races.
CI operator CRs — pin flavours: [] on rhdh-start*.yaml (same lightspeed invariant as Playwright); unit test locks the seam; fail-closed wait for customization-provider S2I so Deployment image is never " ".
Restore e2e-tests/.lintstagedrc.js and document /test ? / /test <job> for agents/contributors.

flowchart TB
  subgraph globalSetup["Playwright globalSetup"]
    A[classify BASE_URL] --> B{mode?}
    B -->|router-stub, no auto-deploy| C[no-op]
    B -->|RUNTIME_AUTO_DEPLOY| D[ensureRuntimeDeployed]
    D --> E[set instance BASE_URL]
    E --> F["GET /healthcheck"]
    B -->|instance-url| F
    B -->|unset, no auto-deploy| C
  end

  subgraph authDeploy["AuthInstanceDeployer"]
    P[OperatorInstallProfile + DynamicPluginsProfile] --> Q[apply CR / CMs]
    Q --> R[wait Deployment created]
    R --> S["HTTP /healthcheck ≤ 600s"]
    S --> T[wait catalog synced]
  end

  subgraph ciOperator["CI operator nightlies"]
    CR["rhdh-start*.yaml flavours: []"] --> DP[install-dynamic-plugins]
    DP --> Ready[showcase / showcase-rbac]
  end

  globalSetup -.->|auth BASE_URL empty| authDeploy

Why auth / operator were still failing after the gate fix

Auth YAML omitted flavours / includes, so the operator reintroduced defaults (e.g. lightspeed) and readiness stayed on HTTP 503. Runtime already encoded those invariants; profiles close the drift.
Auth tests then hit reconcile races (sessionDuration leakage, progressbar/main gates, aborted navigations, OIDC refresh-cookie timing) — hardened in follow-up commits.
Operator nightly CrashLooped install-dynamic-plugins with a lightspeed-core sidecar because CI start CRs lacked flavours: []. Pinned to match OperatorInstallProfile.

Follow-up (only if operator nightly still red)

flavours: [] should remove the lightspeed-core sidecar. Operator CI still uses helm value-file includes: [dynamic-plugins.default.yaml]. Helm is already green with that list — leave it alone unless a retest CrashLoops without lightspeed-core, then pin includes: [] (or disable the bad OCI) on the operator path.

Test plan

Rebased onto main; tip is 823bbd91. Re-trigger nightlies after the force-push:

cd e2e-tests && yarn test:unit
cd e2e-tests && yarn lint / yarn fmt:check
/test e2e-ocp-helm-nightly
/test e2e-ocp-helm-localization-nightly
/test e2e-ocp-operator-nightly — expect no lightspeed-core; no CrashLoop on showcase / showcase-rbac
/test e2e-ocp-operator-auth-providers-nightly — OIDC autologout + provider flows green
Localization: no smoke "Welcome back!" failure
showcase-runtime progresses past globalSetup
auth-providers: deploy reaches HTTP ready / synced

codecov · 2026-07-10T16:24:40Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.77%. Comparing base (1af7158) to head (ba327e0).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5083      +/-   ##
==========================================
- Coverage   55.39%   54.77%   -0.62%     
==========================================
  Files         122      110      -12     
  Lines        2365     2147     -218     
  Branches      544      537       -7     
==========================================
- Hits         1310     1176     -134     
+ Misses       1049      970      -79     
+ Partials        6        1       -5

Flag	Coverage Δ
rhdh	`54.77% <ø> (-0.62%)`	⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1af7158...ba327e0. Read the comment docs.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2026-07-10T16:29:36Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-07-10T16:34:16Z

The container image build workflow finished with status: cancelled.

schultzp2020 · 2026-07-10T16:35:45Z

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly
/test e2e-ocp-helm-localization-nightly
/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-10T17:06:42Z

Image was built and published successfully. It is available at:

schultzp2020 · 2026-07-10T18:01:25Z

/test e2e-ocp-operator-nightly
/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-10T18:34:05Z

Image was built and published successfully. It is available at:

schultzp2020 · 2026-07-10T20:11:01Z

/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-10T20:14:09Z

The container image build workflow finished with status: cancelled.

schultzp2020 · 2026-07-10T20:17:26Z

/test e2e-ocp-operator-auth-providers-nightly

schultzp2020 · 2026-07-10T20:40:19Z

/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-10T20:40:55Z

The container image build workflow finished with status: cancelled.

github-actions · 2026-07-10T21:09:46Z

Image was built and published successfully. It is available at:

schultzp2020 · 2026-07-10T22:52:32Z

/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-10T22:55:10Z

The container image build workflow finished with status: cancelled.

schultzp2020 · 2026-07-10T22:59:20Z

/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-10T23:28:38Z

Image was built and published successfully. It is available at:

schultzp2020 · 2026-07-11T01:51:22Z

/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-11T02:23:32Z

Image was built and published successfully. It is available at:

zdrapela · 2026-07-13T06:49:15Z

/test e2e-ocp-helm

zdrapela · 2026-07-13T06:49:25Z

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

zdrapela · 2026-07-13T08:21:25Z

/agentic-review

zdrapela · 2026-07-13T13:58:18Z

/agentic-review

schultzp2020 · 2026-07-13T15:15:30Z

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly
/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-13T15:15:51Z

The container image build and publish workflows were skipped (either due to [skip-build] tag or no relevant changes with existing image).

schultzp2020 · 2026-07-13T17:54:30Z

/test e2e-ocp-operator-auth-providers-nightly

Co-authored-by: Cursor <cursoragent@cursor.com>

Auth fixtures looked for yamls under rhdh-deployment/ after the folder split. Operator runtime was missing BACKEND_SECRET/auth.keys and the homepage plugin, leaving readiness at HTTP 503. Co-authored-by: Cursor <cursoragent@cursor.com>

prepareProvider updates ConfigMaps before createBackstageDeployment. Capturing reconcile baseline must tolerate a missing deployment. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

waitForRolloutStart treated a missing Deployment as a hard error, so auth prepareProvider failed immediately after applying the Backstage CR. Co-authored-by: Cursor <cursoragent@cursor.com>

Extract OperatorInstallProfile and DynamicPluginsProfile so auth and runtime stop drifting on flavours, includes, and BACKEND_SECRET. Stage auth deploy readiness as Available → HTTP → synced behind AuthInstanceDeployer. Co-authored-by: Cursor <cursoragent@cursor.com>

Stop dual BACKEND_SECRET injection, stage created→HTTP→synced so 503s surface before Available timeouts, always publish instance BASE_URL after runtime deploy, and align auth YAML fixtures with the install profile. Co-authored-by: Cursor <cursoragent@cursor.com>

Auth CI leaves BASE_URL empty so globalSetup no-ops, but rhdhContext ignored describe-level baseURL and login goto("/") failed with invalid URL. Co-authored-by: Cursor <cursoragent@cursor.com>

Centralize helm/operator URL naming, scope RUNTIME_AUTO_DEPLOY so it cannot stomp later projects, wait for authenticated global-header after OAuth, and kill port-forward process groups cleanly. Co-authored-by: Cursor <cursoragent@cursor.com>

Fix catalog sync log races, assert cookieMaxAge for refresh tokens, stop Azure redirect churn, tolerate failed-login alerts, and re-order OIDC+GitHub secondary auth around reconcile. Co-authored-by: Cursor <cursoragent@cursor.com>

Force pod restart and Deployment Ready after ConfigMap updates so resolvers, sessionDuration, and autologout actually load; switch cookie tests to the correct Backstage key and harden post-reconcile navigation/IdP waits. Co-authored-by: Cursor <cursoragent@cursor.com>

Default reconcile waits HTTP-only after mounted-config proof, shares cookie and label helpers, returns authenticated|error from login, and moves GitHub provider sign-out into the Settings POM. Co-authored-by: Cursor <cursoragent@cursor.com>

Playwright TimeoutError from request.get aborted expect.poll after one attempt, so post-reconcile HTTP readiness never used its full budget. Co-authored-by: Cursor <cursoragent@cursor.com>

The catalog identity-error regex had a ';u' typo that broke expected-error cases; also reuse healthy auth namespaces on worker restart and harden IdP popup/login outcome locality so nightlies stop burning the 2h budget. Co-authored-by: Cursor <cursoragent@cursor.com>

Worker-restart reuse was dying on AlreadyExists when recreating secrets, and sessionDuration cases inherited broken resolvers from earlier login tests. Co-authored-by: Cursor <cursoragent@cursor.com>

…tions Requiring role=main before loader settle burned the readiness budget on stuck progressbar paints; also retry ERR_ABORTED and enrich secret k8s errors. Co-authored-by: Cursor <cursoragent@cursor.com>

…ap errors Make waitForAppReady observe the CI stuck-progressbar paint, reset the page before goto retries, and surface ConfigMap k8s failures with status detail. Co-authored-by: Cursor <cursoragent@cursor.com>

Pin autologout resolver/config, poll refresh-cookie absence after reload, and wrap config-liveness/wait API failures so CI no longer shows opaque HttpErrors. Co-authored-by: Cursor <cursoragent@cursor.com>

Operator nightlies CrashLoop when the default lightspeed flavour injects OCI pulls; pin flavours: [] on start CRs to match OperatorInstallProfile. Also fail closed until the customization-provider S2I build leaves a non-whitespace Deployment image. Co-authored-by: Cursor <cursoragent@cursor.com>

Restore #4791 lightspeed disabled overrides in showcase helm values so install-dynamic-plugins cannot fail on unresolved {{inherit}}. Raise catalog API timeouts and poll ingestion checks so OIDC entity sync can finish through Routes without cascading skips. Co-authored-by: Cursor <cursoragent@cursor.com>

Stop burning the 120s ingestion budget on HTTP/shape errors, drop nested expect.poll wrappers, and align CI lightspeed overrides with post-#5021 enabled:false so helm/auth nightlies fail for the right reason. Co-authored-by: Cursor <cursoragent@cursor.com>

sonarqubecloud · 2026-07-16T14:52:53Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

schultzp2020 · 2026-07-16T14:53:08Z

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly
/test e2e-ocp-helm-localization-nightly
/test e2e-ocp-operator-auth-providers-nightly

github-actions · 2026-07-16T15:22:02Z

The container image build workflow finished with status: failure.

openshift-ci · 2026-07-16T16:31:19Z

@schultzp2020: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-ocp-helm	`ba327e0`	link	true	`/test e2e-ocp-helm`
ci/prow/e2e-ocp-operator-nightly	`ba327e0`	link	false	`/test e2e-ocp-operator-nightly`
ci/prow/e2e-ocp-operator-auth-providers-nightly	`ba327e0`	link	false	`/test e2e-ocp-operator-auth-providers-nightly`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci · 2026-07-23T09:31:52Z

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

zdrapela · 2026-07-23T11:23:56Z

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly
/test e2e-ocp-helm-localization-nightly
/test e2e-ocp-operator-auth-providers-nightly

openshift-ci Bot requested review from christoph-jerolimov and debsmita1 July 10, 2026 16:22

schultzp2020 force-pushed the fix/e2e-readiness-and-localization-gates branch from 62f7502 to 0953344 Compare July 10, 2026 22:54

zdrapela reviewed Jul 13, 2026

View reviewed changes

Comment thread .ci/pipelines/jobs/ocp-operator.sh Outdated

Comment thread .ci/pipelines/jobs/ocp-nightly.sh Outdated

Comment thread e2e-tests/playwright/utils/runtime-deploy.ts Outdated

Comment thread e2e-tests/.lintstagedrc.js Outdated

schultzp2020 and others added 21 commits July 16, 2026 09:52

style(e2e): oxfmt instance-readiness unit test

09252c3

Co-authored-by: Cursor <cursoragent@cursor.com>

fix(e2e): allow auth config updates before first deploy

e4d0a0c

prepareProvider updates ConfigMaps before createBackstageDeployment. Capturing reconcile baseline must tolerate a missing deployment. Co-authored-by: Cursor <cursoragent@cursor.com>

refactor(e2e): rename continue param to continueToken

782e62d

Co-authored-by: Cursor <cursoragent@cursor.com>

fix(e2e): poll for auth deployment until operator creates it

b280702

waitForRolloutStart treated a missing Deployment as a hard error, so auth prepareProvider failed immediately after applying the Backstage CR. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(e2e): bind auth harness URL into worker-scoped Playwright context

7235b18

Auth CI leaves BASE_URL empty so globalSetup no-ops, but rhdhContext ignored describe-level baseURL and login goto("/") failed with invalid URL. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(e2e): keep healthcheck polling through Route timeouts

53d25d9

Playwright TimeoutError from request.get aborted expect.poll after one attempt, so post-reconcile HTTP readiness never used its full budget. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(e2e): isolate sessionDuration resolvers and upsert reuse secrets

3024fe9

Worker-restart reuse was dying on AlreadyExists when recreating secrets, and sessionDuration cases inherited broken resolvers from earlier login tests. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(e2e): drop hard main gate and retry aborted post-reconcile naviga…

fd638a6

…tions Requiring role=main before loader settle burned the readiness budget on stuck progressbar paints; also retry ERR_ABORTED and enrich secret k8s errors. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(e2e): settle role=progressbar, narrow abort retries, wrap ConfigM…

8034de6

…ap errors Make waitForAppReady observe the CI stuck-progressbar paint, reset the page before goto retries, and surface ConfigMap k8s failures with status detail. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(e2e): harden OIDC autologout isolation and reconcile k8s errors

649c386

Pin autologout resolver/config, poll refresh-cookie absence after reload, and wrap config-liveness/wait API failures so CI no longer shows opaque HttpErrors. Co-authored-by: Cursor <cursoragent@cursor.com>

schultzp2020 force-pushed the fix/e2e-readiness-and-localization-gates branch from 823bbd9 to ba327e0 Compare July 16, 2026 14:52

openshift-ci Bot added the needs-rebase label Jul 23, 2026

zdrapela mentioned this pull request Jul 23, 2026

[blocked] chore(e2e): Cloud SQL runtime E2E Auth Proxy (Helm + Operator) #5143

Open

5 tasks

Uh oh!

Conversation

schultzp2020 commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why auth / operator were still failing after the gate fix

Follow-up (only if operator nightly still red)

Test plan

Uh oh!

codecov Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

schultzp2020 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

schultzp2020 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

schultzp2020 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

schultzp2020 commented Jul 10, 2026

Uh oh!

schultzp2020 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

schultzp2020 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

schultzp2020 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

schultzp2020 commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 11, 2026

Uh oh!

zdrapela commented Jul 13, 2026

Uh oh!

zdrapela commented Jul 13, 2026

Uh oh!

zdrapela commented Jul 13, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zdrapela commented Jul 13, 2026

Uh oh!

schultzp2020 commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

schultzp2020 commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sonarqubecloud Bot commented Jul 16, 2026

Quality Gate passed

Uh oh!

schultzp2020 commented Jul 16, 2026

Uh oh!

github-actions Bot commented Jul 16, 2026

Uh oh!

openshift-ci Bot commented Jul 16, 2026

Uh oh!

openshift-ci Bot commented Jul 23, 2026

schultzp2020 commented Jul 10, 2026 •

edited

Loading

codecov Bot commented Jul 10, 2026 •

edited

Loading

schultzp2020 commented Jul 11, 2026 •

edited

Loading

schultzp2020 commented Jul 13, 2026 •

edited

Loading

schultzp2020 commented Jul 13, 2026 •

edited

Loading