Skip to content

[ISSUE 57] add rocketmq api #63

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
linjiemiao wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[ISSUE 57] add rocketmq api #63

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6b7520f
add rocketmq api
linjiemiao Sep 27, 2020
5b9c061
[ISSUE #39] Add RocketMQ Console CRD and Controller (#53)
liuruiyiyang Sep 23, 2020
7ee06fb
1. pick console api
linjiemiao Sep 28, 2020
1f91011
Merge branch 'master' into issue-57-dev
linjiemiao Sep 28, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
219 changes: 102 additions & 117 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,117 +177,103 @@ data:

---
apiVersion: rocketmq.apache.org/v1alpha1
kind: Broker
kind: Rocketmq
metadata:
# name of broker cluster
name: broker
namespace: default
name: rocketmq-1
spec:
# size is the number of the broker cluster, each broker cluster contains a master broker and [replicaPerGroup] replica brokers.
size: 1
# nameServers is the [ip:port] list of name service
nameServers: ""
# replicaPerGroup is the number of each broker cluster
replicaPerGroup: 1
# brokerImage is the customized docker image repo of the RocketMQ broker
brokerImage: apacherocketmq/rocketmq-broker:4.5.0-alpine-operator-0.3.0
# imagePullPolicy is the image pull policy
imagePullPolicy: Always
# resources describes the compute resource requirements and limits
resources:
requests:
memory: "2048Mi"
cpu: "250m"
limits:
memory: "12288Mi"
cpu: "500m"
# allowRestart defines whether allow pod restart
allowRestart: true
# storageMode can be EmptyDir, HostPath, StorageClass
storageMode: EmptyDir
# hostPath is the local path to store data
hostPath: /data/rocketmq/broker
# scalePodName is [Broker name]-[broker group number]-master-0
scalePodName: broker-0-master-0
# env defines custom env, e.g. BROKER_MEM
env:
- name: BROKER_MEM
valueFrom:
configMapKeyRef:
broker:
# size is the number of the broker cluster, each broker cluster contains a master broker and [replicaPerGroup] replica brokers.
size: 1
# nameServers is the [ip:port] list of name service
nameServers: ""
# replicaPerGroup is the number of each broker cluster
replicaPerGroup: 1
# brokerImage is the customized docker image repo of the RocketMQ broker
brokerImage: apacherocketmq/rocketmq-broker:4.5.0-alpine-operator-0.3.0
# imagePullPolicy is the image pull policy
imagePullPolicy: Always
# resources describes the compute resource requirements and limits
resources:
requests:
memory: "2048Mi"
cpu: "250m"
limits:
memory: "12288Mi"
cpu: "500m"
# allowRestart defines whether allow pod restart
allowRestart: true
# storageMode can be EmptyDir, HostPath, StorageClass
storageMode: EmptyDir
# hostPath is the local path to store data
hostPath: /data/rocketmq/broker
# scalePodName is [Broker name]-[broker group number]-master-0
scalePodName: broker-0-master-0
# env defines custom env, e.g. BROKER_MEM
env:
- name: BROKER_MEM
valueFrom:
configMapKeyRef:
name: broker-config
key: BROKER_MEM
# volumes defines the broker.conf
volumes:
- name: broker-config
configMap:
name: broker-config
key: BROKER_MEM
# volumes defines the broker.conf
volumes:
- name: broker-config
configMap:
name: broker-config
items:
- key: broker-common.conf
path: broker-common.conf
# volumeClaimTemplates defines the storageClass
volumeClaimTemplates:
- metadata:
name: broker-storage
spec:
accessModes:
- ReadWriteOnce
storageClassName: rocketmq-storage
resources:
requests:
storage: 8Gi
---
apiVersion: rocketmq.apache.org/v1alpha1
kind: NameService
metadata:
name: name-service
namespace: default
spec:
# size is the the name service instance number of the name service cluster
size: 1
# nameServiceImage is the customized docker image repo of the RocketMQ name service
nameServiceImage: apacherocketmq/rocketmq-nameserver:4.5.0-alpine-operator-0.3.0
# imagePullPolicy is the image pull policy
imagePullPolicy: Always
# hostNetwork can be true or false
hostNetwork: true
# Set DNS policy for the pod.
# Defaults to "ClusterFirst".
# Valid values are 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'.
# DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy.
# To have DNS options set along with hostNetwork, you have to specify DNS policy
# explicitly to 'ClusterFirstWithHostNet'.
dnsPolicy: ClusterFirstWithHostNet
# resources describes the compute resource requirements and limits
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "1024Mi"
cpu: "500m"
# storageMode can be EmptyDir, HostPath, StorageClass
storageMode: EmptyDir
# hostPath is the local path to store data
hostPath: /data/rocketmq/nameserver
# volumeClaimTemplates defines the storageClass
volumeClaimTemplates:
- metadata:
name: namesrv-storage
spec:
accessModes:
- ReadWriteOnce
storageClassName: rocketmq-storage
resources:
requests:
storage: 1Gi

---
apiVersion: rocketmq.apache.org/v1alpha1
kind: Console
metadata:
name: console
namespace: default
spec:
items:
- key: broker-common.conf
path: broker-common.conf
# volumeClaimTemplates defines the storageClass
volumeClaimTemplates:
- metadata:
name: broker-storage
spec:
accessModes:
- ReadWriteOnce
storageClassName: rocketmq-storage
resources:
requests:
storage: 8Gi
nameService:
# size is the the name service instance number of the name service cluster
size: 1
# nameServiceImage is the customized docker image repo of the RocketMQ name service
nameServiceImage: apacherocketmq/rocketmq-nameserver:4.5.0-alpine-operator-0.3.0
# imagePullPolicy is the image pull policy
imagePullPolicy: Always
# hostNetwork can be true or false
hostNetwork: true
# Set DNS policy for the pod.
# Defaults to "ClusterFirst".
# Valid values are 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'.
# DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy.
# To have DNS options set along with hostNetwork, you have to specify DNS policy
# explicitly to 'ClusterFirstWithHostNet'.
dnsPolicy: ClusterFirstWithHostNet
# resources describes the compute resource requirements and limits
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "1024Mi"
cpu: "500m"
# storageMode can be EmptyDir, HostPath, StorageClass
storageMode: EmptyDir
# hostPath is the local path to store data
hostPath: /data/rocketmq/nameserver
# volumeClaimTemplates defines the storageClass
volumeClaimTemplates:
- metadata:
name: namesrv-storage
spec:
accessModes:
- ReadWriteOnce
storageClassName: rocketmq-storage
resources:
requests:
storage: 1Gi
console:
# nameServers is the [ip:port] list of name service
nameServers: ""
# consoleDeployment define the console deployment
Expand Down Expand Up @@ -324,9 +310,8 @@ The yaml defines the RocketMQ name server and broker cluster scale, the [ip:port

```
$ kubectl apply -f example/rocketmq_v1alpha1_rocketmq_cluster.yaml
broker.rocketmq.apache.org/broker created
nameservice.rocketmq.apache.org/name-service created
console.rocketmq.apache.org/console created
configmap/broker-config created
rocketmq.rocketmq.apache.org/rocketmq-1 created
```

The name server cluster will be created first, after all name server cluster is in running state, the operator will create the broker cluster.
Expand All @@ -335,12 +320,12 @@ Check the status:

```
$ kubectl get pods -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
broker-0-master-0 1/1 Running 0 71s 10.1.5.91 docker-desktop <none> <none>
broker-0-replica-1-0 1/1 Running 0 71s 10.1.5.92 docker-desktop <none> <none>
console-5c4c9d5757-jnsbq 1/1 Running 0 71s 10.1.5.93 docker-desktop <none> <none>
name-service-0 1/1 Running 0 78s 192.168.65.3 docker-desktop <none> <none>
rocketmq-operator-758bb9c774-jrfw4 1/1 Running 0 106s 10.1.5.90 docker-desktop <none> <none>
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
rocketmq-1-broker-0-master-0 1/1 Running 0 27s 10.1.2.27 docker-desktop <none> <none>
rocketmq-1-broker-0-replica-1-0 1/1 Running 0 27s 10.1.2.28 docker-desktop <none> <none>
rocketmq-1-name-service-0 1/1 Running 0 27s 192.168.65.3 docker-desktop <none> <none>
rocketmq-1-console-56564554ff-xqv8j 1/1 Running 0 27s 192.168.65.4 docker-desktop <none> <none>
rocketmq-operator-76b4b9f4db-x52mz 1/1 Running 0 3h25m 10.1.2.17 docker-desktop <none> <none>
```

Using the default yaml, we can see that there are 2 name-server Pods and 1 master broker 1 replica(slave) broker running on the k8s cluster.
Expand Down
65 changes: 65 additions & 0 deletions deploy/crds/rocketmq_v1alpha1_rocketmq_crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
apiVersion: apiextensions.k8s.io/v1beta1

@RockteMQ-AI RockteMQ-AI Sep 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRD uses apiextensions.k8s.io/v1beta1 which has been deprecated since Kubernetes 1.16 and removed in 1.22. Consider migrating to apiextensions.k8s.io/v1 for forward compatibility.

kind: CustomResourceDefinition
metadata:
name: rocketmqs.rocketmq.apache.org
spec:
group: rocketmq.apache.org
names:
kind: Rocketmq
listKind: RocketmqList
plural: rocketmqs
singular: rocketmq
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
apiVersion:
description: "APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources"
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
broker:
description: Broker defines rocketmq broker spec info

@RockteMQ-AI RockteMQ-AI Aug 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CRD validation schema defines broker, nameService, and console as type: object with no sub-properties. This provides zero validation for nested fields, meaning any malformed spec will be accepted by the API server. At minimum, required sub-fields should be enumerated, or the full nested schemas from the Broker/NameService/Console CRDs should be inlined.

@RockteMQ-AI RockteMQ-AI Sep 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CRD does not require broker or nameService and provides no nested schema. Invalid or misspelled cluster configuration is accepted and reconciled into empty child CRs instead of being rejected at admission.

type: object
nameService:
description: NameService defines rocketmq name service spec info
type: object

@RockteMQ-AI RockteMQ-AI Sep 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The broker, nameService, and console fields under spec are declared as type: object with no nested properties or validation. This means any arbitrary YAML is accepted under these fields with no schema enforcement. Users will get no validation feedback for misspelled or missing required fields. Either add property-level validation or use x-kubernetes-preserve-unknown-fields explicitly to document the intent.

console:
description: Console defines rocketmq console deployment info
type: object
status:
properties:
nameService:
description: 'INSERT ADDITIONAL STATUS FIELD - define observed state
of cluster Important: Run "operator-sdk generate k8s" to regenerate
code after modifying this file Add custom validation using kubebuilder
tags: https://book-v1.book.kubebuilder.io/beyond_basics/generating_crd.html
NameService is the name of name service'
type: string
broker:
description: 'INSERT ADDITIONAL STATUS FIELD - define observed state
of cluster Important: Run "operator-sdk generate k8s" to regenerate
code after modifying this file Add custom validation using kubebuilder
tags: https://book-v1.book.kubebuilder.io/beyond_basics/generating_crd.html
Brokers is the name of broker'
type: string
required:
- nameService
- broker
type: object
version: v1alpha1
versions:
- name: v1alpha1
served: true
storage: true
1 change: 1 addition & 0 deletions example/rocketmq_v1alpha1_cluster_service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ metadata:
spec:
type: NodePort
selector:
name_service_cr: ${rocketmq-name}-name-service

@RockteMQ-AI RockteMQ-AI Aug 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The selector name_service_cr: ${rocketmq-name}-name-service uses a shell variable placeholder ${rocketmq-name} that is not substituted by kubectl. Users must manually replace this value, but this is not documented and will silently result in a Service with no matching pods if applied as-is.

@RockteMQ-AI RockteMQ-AI Sep 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Service now requires name_service_cr=${rocketmq-name}-name-service, but generated Console pods only receive app and console-cr labels. The Service will have no endpoints; kubectl also does not expand this placeholder.

app: rocketmq-console
ports:
- port: 8080
Expand Down
Loading