Skip to content

[ISSUE #65]Add RocketMQ-on-DLeger group mode deploy #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
haifzhu wants to merge 5 commits into
base: master
Choose a base branch
from
Open

[ISSUE #65]Add RocketMQ-on-DLeger group mode deploy #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
915821a
Add RocketMQ-on-DLeger group mode deploy
haifzhu Sep 26, 2021
90eac73
Add RocketMQ-on-DLeger group mode deploy
haifzhu Sep 26, 2021
428c3b4
add affinity and tolerations
haifzhu Feb 15, 2022
b192258
修改crds
haifzhu Feb 15, 2022
6575669
add env to nameserver
haifzhu Feb 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,8 @@ spec:
size: 1
# nameServers is the [ip:port] list of name service
nameServers: ""
# Whether enable rocketmq-on-dleger group deploy

@RockteMQ-AI RockteMQ-AI Aug 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

README documents enableDLeger: false as optional with an implicit default, but the CRD schema marks the field as required and does not declare a default value. The documented behavior does not match the schema.

enableDLeger: false
# replicaPerGroup is the number of each broker cluster
replicaPerGroup: 1
# brokerImage is the customized docker image repo of the RocketMQ broker
Expand Down
796 changes: 772 additions & 24 deletions deploy/crds/rocketmq_v1alpha1_broker_crd.yaml
View file
Open in desktop

Large diffs are not rendered by default.

787 changes: 768 additions & 19 deletions deploy/crds/rocketmq_v1alpha1_nameservice_crd.yaml
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions docs/cn/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,8 @@ spec:
nameServers: 192.168.130.33:9876
# replicationMode is the broker replica sync mode, can be ASYNC or SYNC
replicationMode: ASYNC
# Whether enable rocketmq-on-dleger group deploy
enableDLeger: false
# replicaPerGroup is the number of replica broker in each group
replicaPerGroup: 1
# brokerImage is the customized docker image repo of the RocketMQ broker
Expand Down
6 changes: 5 additions & 1 deletion example/rocketmq_v1alpha1_broker_cr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ spec:
size: 1
# nameServers is the [ip:port] list of name service
nameServers: ""
# Whether enable rocketmq-on-dleger group deploy
enableDLeger: false
# replicaPerGroup is the number of each broker cluster
replicaPerGroup: 1
# brokerImage is the customized docker image repo of the RocketMQ broker
Expand Down Expand Up @@ -67,6 +69,8 @@ spec:
configMapKeyRef:
name: broker-config
key: BROKER_MEM
- name: TZ
value: Asia/Shanghai
# volumes defines the broker.conf
volumes:
- name: broker-config
Expand All @@ -82,7 +86,7 @@ spec:
annotations:
volume.beta.kubernetes.io/storage-class: rocketmq-storage
spec:
accessModes: [ "ReadWriteOnce" ]
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 8Gi
7 changes: 6 additions & 1 deletion example/rocketmq_v1alpha1_console_cr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,9 @@ spec:
- name: console
image: apacherocketmq/rocketmq-console:2.0.0
ports:
- containerPort: 8080
- containerPort: 8080
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPTS
value: -Drocketmq.namesrv.addr=rocketmq-name-service:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false
15 changes: 14 additions & 1 deletion example/rocketmq_v1alpha1_nameservice_cr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@ spec:
# To have DNS options set along with hostNetwork, you have to specify DNS policy
# explicitly to 'ClusterFirstWithHostNet'.
dnsPolicy: ClusterFirstWithHostNet
env:
- name: TZ
value: Asia/Shanghai
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: nameserver
operator: In
values:
- true
tolerations: []
# resources describes the compute resource requirements and limits
resources:
requests:
Expand All @@ -52,7 +65,7 @@ spec:
annotations:
volume.beta.kubernetes.io/storage-class: rocketmq-storage
spec:
accessModes: [ "ReadWriteOnce" ]
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 1Gi
34 changes: 33 additions & 1 deletion example/rocketmq_v1alpha1_rocketmq_cluster.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,24 @@ spec:
size: 1
# nameServers is the [ip:port] list of name service
nameServers: ""
# Whether enable rocketmq-on-dleger group deploy
enableDLeger: false
# replicaPerGroup is the number of each broker cluster
replicaPerGroup: 1
# brokerImage is the customized docker image repo of the RocketMQ broker
brokerImage: apacherocketmq/rocketmq-broker:4.5.0-alpine-operator-0.3.0
# imagePullPolicy is the image pull policy
imagePullPolicy: Always
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: broker
operator: In
values:
- "true"
tolerations: []
# resources describes the compute resource requirements and limits
resources:
requests:
Expand All @@ -70,6 +82,8 @@ spec:
configMapKeyRef:
name: broker-config
key: BROKER_MEM
- name: TZ
value: Asia/Shanghai
# volumes defines the broker.conf
volumes:
- name: broker-config
Expand Down Expand Up @@ -111,6 +125,19 @@ spec:
# To have DNS options set along with hostNetwork, you have to specify DNS policy
# explicitly to 'ClusterFirstWithHostNet'.
dnsPolicy: ClusterFirstWithHostNet
env:
- name: TZ
value: Asia/Shanghai
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: nameserver
operator: In
values:
- true
tolerations: []
# resources describes the compute resource requirements and limits
resources:
requests:
Expand Down Expand Up @@ -165,4 +192,9 @@ spec:
- name: console
image: apacherocketmq/rocketmq-console:2.0.0
ports:
- containerPort: 8080
- containerPort: 8080
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPTS
value: -Drocketmq.namesrv.addr=rocketmq-name-service:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false
11 changes: 11 additions & 0 deletions images/broker/alpine/brokerGenConfig.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,17 @@ function create_config() {
if [ $BROKER_ID != 0 ]; then
sed -i 's/brokerRole=.*/brokerRole=SLAVE/g' $BROKER_CONFIG_FILE
fi

# Enable RocketMQ-on-DLedger Group

@RockteMQ-AI RockteMQ-AI Sep 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shell test [ ! -z $ENABLE_DLEGER ] is unquoted. If ENABLE_DLEGER is unset (not just empty), this becomes [ ! -z ] which evaluates to true (tests the -z flag itself as a non-empty string), enabling DLedger unintentionally. Quote it: [ ! -z "$ENABLE_DLEGER" ] or use [ -n "$ENABLE_DLEGER" ].

if [ ! -z $ENABLE_DLEGER ]; then
echo "enableDLegerCommitLog=true" >> $BROKER_CONFIG_FILE
echo "dLegerGroup=$BROKER_NAME" >> $BROKER_CONFIG_FILE
echo "dLegerPeers=$DLEGER_PEERS" >> $BROKER_CONFIG_FILE
echo "dLegerSelfId=$DLEGER_SELF_ID" >> $BROKER_CONFIG_FILE
if [ ! -z $DLEGER_THREAD_NUM ]; then
echo "sendMessageThreadPoolNums=$DLEGER_THREAD_NUM" >> $BROKER_CONFIG_FILE
fi
fi
}

create_config
Expand Down
6 changes: 5 additions & 1 deletion pkg/apis/rocketmq/v1alpha1/broker_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,18 @@ type BrokerSpec struct {
Size int `json:"size"`

@RockteMQ-AI RockteMQ-AI Aug 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No test changes detected alongside source modifications. Consider adding tests to cover the changes.

// NameServers defines the name service list e.g. 192.168.1.1:9876;192.168.1.2:9876
NameServers string `json:"nameServers,omitempty"`
// Whether enable rocketmq-on-dleger group deploy, false in default
EnableDLeger bool `json:"enableDLeger"`
// ReplicaPerGroup each broker cluster's replica number
ReplicaPerGroup int `json:"replicaPerGroup"`
// BaseImage is the broker image to use for the Pods
BrokerImage string `json:"brokerImage"`
// ImagePullPolicy defines how the image is pulled
ImagePullPolicy corev1.PullPolicy `json:"imagePullPolicy"`
// AllowRestart defines whether allow pod restart
AllowRestart bool `json:"allowRestart"`
AllowRestart bool `json:"allowRestart"`

@RockteMQ-AI RockteMQ-AI Sep 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Affinity is declared as corev1.Affinity (value type) without omitempty. An empty Affinity struct will always be serialized into the StatefulSet pod spec even when the user sets none, and the CRD makes it required. Use *corev1.Affinity (pointer) with json:"affinity,omitempty" so it is nil when unset.

Affinity corev1.Affinity `json:"affinity"`

@RockteMQ-AI RockteMQ-AI Sep 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tolerations and other new fields (Env in NameService) also lack omitempty in their JSON tags. This forces users to provide empty arrays in their CRs and makes the CRD schema unnecessarily strict.

Tolerations []corev1.Toleration `json:"tolerations"`
// Resources describes the compute resource requirements
Resources corev1.ResourceRequirements `json:"resources"`
// StorageMode can be EmptyDir, HostPath, StorageClass
Expand Down
5 changes: 4 additions & 1 deletion pkg/apis/rocketmq/v1alpha1/nameservice_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,10 @@ type NameServiceSpec struct {
// HostNetwork can be true or false
HostNetwork bool `json:"hostNetwork"`
// dnsPolicy defines how a pod's DNS will be configured
DNSPolicy corev1.DNSPolicy `json:"dnsPolicy"`
DNSPolicy corev1.DNSPolicy `json:"dnsPolicy"`
Env []corev1.EnvVar `json:"env"`
Affinity corev1.Affinity `json:"affinity"`
Tolerations []corev1.Toleration `json:"tolerations"`
// Resources describes the compute resource requirements
Resources corev1.ResourceRequirements `json:"resources"`
// StorageMode can be EmptyDir, HostPath, StorageClass
Expand Down
18 changes: 18 additions & 0 deletions pkg/constants/constants.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,18 @@ const (
// EnvBrokerName is the container environment variable name of broker name
EnvBrokerName = "BROKER_NAME"

// EnvBrokerIp is the container environment variable name of broker register ip
EnvBrokerIp = "BROKER_IP"

// EnvEnableDLeger is the container environment variable name of enable dLeger
EnvEnableDLeger = "ENABLE_DLEGER"

// EnvDLegerPeers is the container environment variable name of dLeger peers
EnvDLegerPeers = "DLEGER_PEERS"

// EnvDLegerSelfId is the container environment variable name of dLeger self id
EnvDLegerSelfId = "DLEGER_SELF_ID"

// LogMountPath is the directory of RocketMQ log files
LogMountPath = DataPath + "/logs"

Expand Down Expand Up @@ -94,6 +106,12 @@ const (
// BrokerMainContainerPortName is the main port name of broker container
BrokerMainContainerPortName = "main"

// BrokerDlegerContainerPort is the dleger port number of broker container
BrokerDlegerContainerPort = 40911

// BrokerDlegerContainerPortName is the dleger port name of broker container
BrokerDlegerContainerPortName = "dleger"

// BrokerHighAvailabilityContainerPort is the high availability port number of broker container
BrokerHighAvailabilityContainerPort = 10912

Expand Down
Loading