Skip to content

[Studio] feat: rip 2 proxy admin grpc - #117

Open
zhaohai666 wants to merge 4 commits into
apache:mainfrom
zhaohai666:feature/rip-2-proxy-admin-grpc
Open

[Studio] feat: rip 2 proxy admin grpc#117
zhaohai666 wants to merge 4 commits into
apache:mainfrom
zhaohai666:feature/rip-2-proxy-admin-grpc

Conversation

@zhaohai666

Copy link
Copy Markdown

RIP-2: Add ProxyAdminService gRPC interface surface to admin.proto

Summary

This PR implements the RIP-2 (Proxy Admin gRPC Interface Surface) control-plane
contract for the RocketMQ Proxy, expressed at the protocol-definition layer
(rocketmq-apis). It adds a new, independent gRPC service
ProxyAdminService to apache/rocketmq/v2/admin.proto, covering online client
query, runtime configuration, connection control, rate-limit/quota observation,
POP/batch consumption diagnostics, and a server-streaming route-event
subscription.

The service is intentionally separate from the data-plane MessagingService
and from the existing broker-facing Admin service, and it covers both gRPC
and Remoting clients
connected to the Proxy. This PR is the API contract only
(message/service definitions); the server-side binding lives in the Proxy
implementation repo (e.g. rocketmq-proxy).

Background

RIP-2 introduces a dedicated administrative interface for the Proxy over gRPC so
that operators and consoles can inspect and manage the clients/state attached to
a Proxy at runtime, without overloading the data plane. It complements — rather
than extends — the existing Admin service, which targets the broker control
plane.

Design Decisions (RIP-2)

  • D1 — Service placement (Option B): a dedicated ProxyAdminService, kept
    separate from MessagingService and the broker-facing Admin.
  • D2 — Authorization (ACL 2.0): least-privilege resource types under
    proxy.admin.*, one per sub-module:
    • proxy.admin.clientListClients / DescribeClient / ListClientsByGroup / ListClientsByTopic
    • proxy.admin.configDescribeProxyConfig / UpdateProxyConfig
    • proxy.admin.connection (high privilege) — KickClient / DisconnectChannel
    • proxy.admin.quota (high privilege) — DescribeQuota / UpdateQuota
    • proxy.admin.routeDescribeRouteTopology / SubscribeRouteEvents
    • Credentials are carried in the gRPC request metadata (Authorization
      header via the RIP-1 AUTH-01 AuthCallCredentials mechanism), not in the
      request message body — auth stays decoupled from the data plane and reuses the
      5.0 ACL 2.0 baseline. A global proxyAdminEnable kill switch disables the
      whole surface.
  • D3 — Multi-proxy semantics: each Proxy returns its local view, tagged
    with proxy_endpoint + epoch so a dashboard/CLI can deduplicate across a
    Proxy cluster. ProxyScope (LOCAL / CLUSTER / PROXY_ID) is wired into the
    aggregate queries.
  • D4 — Pagination: cursor-based (next_token) pagination for the highly
    dynamic client lists; offset-based (page_num/page_size) for diagnostic
    snapshots (M3/M4). No full unpaginated dumps.
  • D5 — Dual-protocol coverage: both gRPC and Remoting clients are included,
    distinguished by the protocol field of ClientInstance.

Added RPCs (14 total)

Milestone RPC Purpose
M1 — Online client query ListClients Filter + paginate all connected clients (by role/group/topic time window).
DescribeClient Full detail of a single client: settings, subscriptions, publish settings, recent heartbeats, auth status, consume progress, network info.
ListClientsByGroup Clients belonging to a consumer/producer group.
ListClientsByTopic Clients attached to a topic.
M2 — Config & connection DescribeProxyConfig Read the Proxy runtime config (ProxyRuntimeConfig).
UpdateProxyConfig Controlled runtime config update (ACL proxy.admin.config).
KickClient Force-disconnect a client by client_id (ACL proxy.admin.connection).
DisconnectChannel Force-disconnect a transport channel by channel_id.
DescribeQuota Rate-limit/quota policy visualization by namespace/topic/group.
UpdateQuota Controlled quota adjustment (ACL proxy.admin.quota).
M3 — POP diagnostics DescribePopReceiptHandles POP receipt-handle diagnostics incl. lock view, ACK/NACK counters, NOT_CONSUME_YET reasons.
M4 — Batch diagnostics DescribeBatchConsumeDiagnostics Batch-consumption diagnostics incl. pull-window state.
Stream SubscribeRouteEvents Server-streaming Proxy→Broker route-change events.
M2 — Route observation DescribeRouteTopology Static snapshot: Proxy→Broker links, failover state, load distribution, region-affinity.

Key new message/enum types

  • ClientInstance, ClientDetail, ClientFilter, PublishSettings,
    HeartbeatRecord, AuthStatus, ClientConsumeProgress, NetworkInfo
  • ProxyRuntimeConfig, ProxyScope, ProxyClientIdentity
  • QuotaPolicy, QuotaDimension
  • PopReceiptHandleGroupSummary, PopReceiptHandleInfo, PopLockView
  • BatchConsumeGroupSummary, BatchConsumeClientDiagnostics, BatchPullWindowState
  • RouteChangeEvent, ProxyBrokerLink, LoadBalanceInfo
  • Enums: ClientRole (PRODUCER/PUSH_CONSUMER/SIMPLE_CONSUMER),
    ClientProtocol (GRPC/REMOTING), RouteChangeEventType

Existing apache.rocketmq.v2 types (Status, Resource, Settings,
SubscriptionEntry, Language, Permission, MessageModel) from
definition.proto are reused to stay consistent with the rest of the API.

Validation

  • protoc 25.3 compiles apache/rocketmq/v2/admin.proto cleanly (with
    definition.proto and the well-known types Timestamp/Duration).
  • No conflicts with the existing Admin service.

Versioning & Changelog

  • Bumped protocol version: java/VERSION 2.2.02.3.0 (backward-compatible minor).
  • Added entry 15 to ChangeLog.md documenting the RIP-2 addition.

Commits on this branch

  • 60cafc1 RIP-2: add ProxyAdminService gRPC interface surface to admin.proto
  • 976b7e2 RIP-2: align ProxyAdminService with competition spec (gap fixes)
  • 15183c4 RIP-2: align ProxyAdminService with review feedback
  • 2cd7e56 RIP-2: revert items not required by the competition spec

Notes / Follow-ups

  • This repository is the protocol contract; the actual service binding and
    ProxyStartup wiring must be implemented in the Proxy implementation repo.
  • OTel self-monitoring metrics and the ACL 2.0 enforcement hook are implementation
    concerns; the contract already documents the proxy.admin.* resource vocabulary
    for those integrations.
  • A bespoke ProxyAdminErrorCode taxonomy and a concrete page_num/page_size
    • max-100 form were explored but reverted, as the competition spec does not
      mandate them (standard gRPC status codes + Status.message are used for errors;
      cursor pagination is used per D4).

Add a dedicated, independent ProxyAdminService (design decision D1 Option B)
to apache/rocketmq/v2/admin.proto, covering the RIP-2 control-plane
interface for the RocketMQ Proxy over gRPC:

- M1 Online client query: ListClients, DescribeClient, ListClientsByGroup,
  ListClientsByTopic (cursor-based pagination, dual gRPC/Remoting coverage)
- M2 Runtime config & connection management: GetProxyConfig,
  UpdateProxyConfig, DisconnectClient
- M3 POP receipt handle diagnostics: DescribePopReceiptHandles
- M4 Batch consumption diagnostics: DescribeBatchConsumeDiagnostics
- Server-streaming route change subscription: SubscribeRouteEvents

Reuses existing apache.rocketmq.v2 types (Status, Resource, Settings,
SubscriptionEntry, Language, Permission, MessageModel) from definition.proto
and the existing Admin service for a consistent protocol surface.
Address the capability gaps found against the official RIP-2 competition spec:

P0 naming alignment:
- GetProxyConfig -> DescribeProxyConfig (consistent Describe* prefix)
- DisconnectClient -> KickClient; add DisconnectChannel(channel_id) for
  transport-level connection control

P1 M1 minimum-set fields:
- ClientInstance.topics (associate topics, in addition to groups)
- PublishSettings + ClientDetail.publish_settings (producer publish info)
- ClientFilter.connected_before (time-window upper bound)
- ProxyScope wired into ListClients/ByGroup/ByTopic requests (D3)

P1 new modules (M2 planning):
- Quota visualization: DescribeQuota/UpdateQuota + QuotaPolicy/QuotaDimension
  (ACL resource proxy.admin.quota)
- Route & load-balance observation: DescribeRouteTopology + ProxyBrokerLink/
  LoadBalanceInfo (ACL resource proxy.admin.route)

P2 diagnostics & ACL:
- POP lock view (PopLockView), ACK/NACK counters, NOT_CONSUME_YET reason
- Batch pull window state (BatchPullWindowState)
- Enumerate proxy.admin.* ACL resource vocabulary in service doc & per-RPC

protoc 25.3 validation passes; no stale references remain.
- Pagination: ListClients/ByGroup/ByTopic now use page_num/page_size
  (server-enforced max 100) per RIP-2 D4, replacing the cursor-based
  next_token + page_size(1000) scheme; responses echo total/page_num/page_size.
  (M3/M4 diagnostics already used page_num/page_size, now consistent.)
- ACL 2.0 carrier: D2 clarifies credentials ride gRPC request metadata
  (Authorization header, RIP-1 AUTH-01 AuthCallCredentials), NOT the message
  body, matching RIP-2's decoupled-auth requirement.
- Dedicated error model (D6): add ProxyAdminErrorCode enum (11 RIP-2-specific
  codes) surfaced via an optional error_code field on every response, replacing
  ad-hoc Status.message parsing.
- Bump protocol version java/VERSION 2.2.0 -> 2.3.0.
- Add RIP-2 entry (15) to ChangeLog.md.

protoc 25.3 validation passes; no stale next_token/1000 references remain.
Roll back the two additions from the previous review pass that the RIP-2
competition spec does NOT mandate:

- Drop the dedicated ProxyAdminErrorCode enum and the optional error_code
  field on every response (RIP-2 D6). Transport-level errors keep using
  standard gRPC status codes + Status.message; the spec does not ask for a
  bespoke error-code taxonomy.
- Revert ListClients/ByGroup/ByTopic pagination from page_num/page_size
  (max 100) back to the cursor-based next_token + page_size scheme (D4),
  since the spec only requires pagination (filter-pushdown, no full dump)
  without prescribing a concrete paging form. M3/M4 diagnostics keep their
  existing page_num/page_size, unchanged.

Kept (spec-mandated or user-requested):
- ACL 2.0 credentials ride gRPC request metadata (Authorization header,
  RIP-1 AUTH-01), not the message body.
- java/VERSION 2.3.0 (user-requested bump).
- ChangeLog entry 15 trimmed to reflect the above.

protoc 25.3 validation passes; no stale ProxyAdminErrorCode/error_code remain.
@zhaohai666 zhaohai666 changed the title Feature/rip 2 proxy admin grpc [Studio] feat: rip 2 proxy admin grpc Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant