feat: expose publishNotReadyAddresses for broker services (#244) - #1
Open
gomitrah wants to merge 12 commits into
Open
feat: expose publishNotReadyAddresses for broker services (#244)#1gomitrah wants to merge 12 commits into
gomitrah wants to merge 12 commits into
Conversation
gomitrah
force-pushed
the
feat/expose-publish-not-ready-addresses
branch
2 times, most recently
from
July 17, 2026 08:24
fd22b43 to
a85e688
Compare
gomitrah
force-pushed
the
feat/expose-publish-not-ready-addresses
branch
from
July 22, 2026 07:12
a85e688 to
b73c540
Compare
amuraru
force-pushed
the
feat/expose-publish-not-ready-addresses
branch
from
August 6, 2026 16:45
b73c540 to
256012d
Compare
* feat/Broker version extraction improvements * Added PodMonitor for Prometheus metrics * Skip JMX version fetch when broker status is already current (adobe#249) * fixed legacy label for alertmanager peerauthentication * Fixed missing version on brokers due to jmx exporter regression Reduced logging verbosity Filtered out status and metadatta.managedFields from watched events Co-authored-by: Razvan Dobre <dobre@adobe.com> Co-authored-by: Adrian Coman <acoman@Adrians-MacBook-Pro.local> Co-authored-by: hvan <hvan@redhat.com> Co-authored-by: Ha Van <red83362@adobe.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…adobe#308) The comment on the graceful-downscale guard in reconcileKafkaPodDelete described the opposite of what the code does. Clarify that a controller-only node has no Cruise Control broker and is therefore deleted directly (branch skipped), while broker-only/combined/non-KRaft nodes wait for the graceful downscale to finish. No behavior change. (cherry picked from commit 677a297) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test(e2e): print a running "[k/N] specs done" progress counter
The e2e suite runs --ginkgo.v but Ginkgo only prints the spec count
up front ("Will run N of N specs") and a final summary, so CI logs
give no indication of progress while the ~45m run is in flight.
Add a ReportBeforeSuite hook to capture the total spec count and a
ReportAfterEach hook that prints "[k/N specs done] <spec>: <state>
(took <duration>)" after every spec, so progress and per-spec timing
can be read straight off the CI log.
…atches, and workflow tooling (adobe#315) fix(renovate): close coverage gaps for docker image versions in samples, patches, and workflow tooling Renovate's helm-values manager only matches files literally named values.yaml/values.yml with a nested {repository,tag} shape, kustomize only reads the kustomization.yaml 'images:' transformer (unused here), and the kubernetes manager has no default file patterns and isn't configured - so every flat "repo:tag" image string in config/samples/**, docs/benchmarks/infrastructure/**, tests/e2e/templates/*.tmpl, config/base/manager/manager.yaml, config/overlays/**, and hack/kafka-test-pod/manifest_certmanager.yaml was silently unwatched (the existing "config samples docker images" packageRule referenced helm-values and was a no-op). Confirmed via drift already present: several samples were still pinned to jmx-javaagent:1.4.0 and cruise-control:3.0.3-adbe-20250804, both stale against the annotated defaults in api/v1beta1/kafkacluster_types.go. Add explicit '# renovate:' comment annotations (the pattern already used for Makefile/Dockerfile _VERSION vars and kafkacluster_types.go) plus matching customManagers, and repoint the dead packageRule at custom.regex. Also cover previously-unwatched build tooling: an unpinned `go install ...@latest` in the kind-create composite action, a bare Helm CLI version in e2e-test.yaml, and scripts/install_kustomize.sh's lowercase `version` var (renamed to KUSTOMIZE_VERSION to match the _VERSION regex convention). Separately, verified with a live local Renovate run (RENOVATE_PLATFORM=local, dry-run) against ghcr.io that the default docker versioning scheme truncates ghcr.io/adobe/koperator/kafka's <scala>-<kafka> tag format (e.g. 2.13-3.9.1) to just "2.13", silently dropping the actual Kafka version - this already affected the pre-existing, correctly-annotated DefaultKafkaImage constant, so no real version-bump PR was ever possible for this image, only digest pins. Added a compatibility-anchored regex versioning packageRule (same technique already used for cruise-control/zookeeper-operator's -adbe/-adobe suffix) so the Kafka release itself is now compared correctly. The kraft sample's -jdk<version> suffixed variant is deliberately excluded (needs a second compatibility slot a single regex can't express) and keeps today's non-broken-but-limited default versioning. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
…adobe#317) The 3.9.2 Kafka image was published as 2.13-3.9.2-jdk21.0.11 instead of the plain 2.13-3.9.2 shape prior releases used, so the packageRule's matchCurrentValue/versioning regex (scoped to the plain shape only) couldn't see it as a valid version. Renovate silently had nothing to bump, and PR adobe#302 shipped with DefaultKafkaImage still on 3.9.1. Extend the versioning regex to parse an optional -jdk<major>.<interim>.<update> suffix, mapping jdk major/update onto the regex versioning module's build/revision slots so both tag shapes compare correctly against each other. This does not gate jdk-major jumps the way 'compatibility' gates the scala prefix - that's deferred to the same operator-side migration already planned for the apache/kafka 4.x bump. Since this changes which tags are valid for the ghcr.io image line, manually cut over the two DefaultKafkaImage/defaultKafkaImage constants to the now-current 2.13-3.9.2-jdk21.0.11 tag; Renovate will track further -jdk<version> bumps automatically going forward. The ~20 config/samples/**/docs/benchmarks/** references still pinned to 3.9.1 are left for Renovate's own follow-up PR (same customManager/packageRule now covers them) rather than hand-edited here to avoid conflicting with that automated PR. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Delay headless/non-headless Service cleanup while any running broker pod is no longer listed in spec.brokers. Such pods are waiting for Cruise Control removal and are outside per-broker reconciliation, so switching addressing can delete their only Service. Update broker-scaling e2e coverage to patch spec.brokers directly instead of applying independent manifests, avoiding unrelated spec drift during removal/upscale scenarios. Add the KRaft broker scaling path and remove the universal capacity default from the KRaft sample so capacity.json changes are actually exercised. Avoid passing namespaces to cluster-scoped CRD deletes in e2e uninstall cleanup, removing noisy kubectl warnings without changing generic delete behavior. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
generateRandomClusterID used base64.URLEncoding, producing a 24-character padded id (e.g. "…=="). Kafka 3.9's Uuid.fromString happens to accept it (length <= 24 and its URL decoder tolerates padding), but that is a non-canonical form: Kafka itself emits the 22-character padding-free encoding via Base64.getUrlEncoder().withoutPadding(), and newer Kafka versions reject longer strings. Switch to base64.RawURLEncoding so freshly generated cluster IDs match Kafka's canonical form. Existing clusters are unaffected: their id is already persisted in KafkaCluster.Status.ClusterID and reused verbatim. Strengthen TestGenerateClusterID to assert the id decodes to exactly 16 bytes and is 22 characters long. (cherry picked from commit ed420fc) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#310) During a rolling upgrade the operator deleted the next pod as soon as the previous one left the Pending/Terminating phase (getPodsInTerminatingOrPendingState), and its only health gate (AllOfflineReplicas/OutOfSyncReplicas) reflects partition/data-plane health, which controller-only nodes never appear in. The KafkaClient exposes no metadata-quorum API, so a controller could be restarted before the previously restarted one had rejoined the quorum, risking loss of quorum majority. The controller PodDisruptionBudget does not help here because the operator deletes pods directly, bypassing the PDB. Make controller readiness reflect quorum membership and gate the rolling upgrade on it: - The controller readiness probe now execs the kraft healthcheck in a new "readiness" mode (KRAFT_HEALTH_CHECK_MODE=readiness), which is fail-closed: the pod is Ready only once the controller reports a leader/follower raft state. Liveness keeps the previous fail-open behaviour so a slow or briefly unavailable JMX exporter never restarts a healthy controller. This replaces the previous bare TCP-socket readiness check (and the now-unused findControllerListenerPort helper). - handleRollingUpgrade now refuses (requeues) to delete a controller pod while any other KRaft controller is not Ready (controllersBlockingRollingUpgrade -> notReadyControllerBrokerIDs). The gate fires only when the pod being rolled is itself a controller: broker-only restarts cannot cost the quorum its majority and stay gated by the data-plane health check. The pod being reconciled is excluded so an unhealthy controller can still be replaced. Operators can bypass via KRAFT_HEALTH_CHECK_SKIP if the metric is unavailable clusterwide. Add unit tests for notReadyControllerBrokerIDs and controllersBlockingRollingUpgrade. (cherry picked from commit 0a3dc02c60d183f50d4da8d7473057b5de796f3c) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
amuraru
force-pushed
the
feat/expose-publish-not-ready-addresses
branch
from
August 20, 2026 07:32
256012d to
17f5c61
Compare
amuraru
force-pushed
the
feat/expose-publish-not-ready-addresses
branch
from
August 24, 2026 08:27
17f5c61 to
a5ac3a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a
publishNotReadyAddressesfield to theKafkaClusterCRD spec and propagates it to thenon-headless broker Services created by the operator: the per-broker Services (
kafka-{id}) andkafka-all-broker.Type of Change
Checklist