[ISSUEE 60] Add features to the broker and nameserver #61
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,180 @@ | ||
| # Licensed to the Apache Software Foundation (ASF) under one or more | ||
| # contributor license agreements. See the NOTICE file distributed with | ||
| # this work for additional information regarding copyright ownership. | ||
| # The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| # (the "License"); you may not use this file except in compliance with | ||
| # the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: broker-config | ||
| data: | ||
| # BROKER_MEM sets the broker JVM, if set to "" then Xms = Xmx = max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB)) | ||
| BROKER_MEM: " -Xms2g -Xmx2g -Xmn1g " | ||
| broker-common.conf: | | ||
| # brokerClusterName, brokerName, brokerId are automatically generated by the operator and do not set it manually!!! | ||
| deleteWhen=04 | ||
| fileReservedTime=48 | ||
| flushDiskType=ASYNC_FLUSH | ||
| # set brokerRole to ASYNC_MASTER or SYNC_MASTER. DO NOT set to SLAVE because the replica instance will automatically be set!!! | ||
| brokerRole=ASYNC_MASTER | ||
|
|
||
| --- | ||
| apiVersion: rocketmq.apache.org/v1alpha1 | ||
| kind: Broker | ||
| metadata: | ||
| # name of broker cluster | ||
| name: broker | ||
| spec: | ||
| # size is the number of the broker cluster, each broker cluster contains a master broker and [replicaPerGroup] replica brokers. | ||
| size: 1 | ||
| # nameServers is the [ip:port] list of name service | ||
| nameServers: "" | ||
| # replicaPerGroup is the number of each broker cluster | ||
| replicaPerGroup: 0 | ||
| # brokerImage is the customized docker image repo of the RocketMQ broker | ||
| brokerImage: apacherocketmq/rocketmq-broker:4.5.0-alpine-operator-0.3.0 | ||
| # imagePullPolicy is the image pull policy | ||
| imagePullPolicy: Always | ||
| # resources describes the compute resource requirements and limits | ||
| resources: | ||
| requests: | ||
| memory: "2048Mi" | ||
| cpu: "250m" | ||
| limits: | ||
| memory: "12288Mi" | ||
| cpu: "500m" | ||
| # allowRestart defines whether allow pod restart | ||
| allowRestart: true | ||
| # storageMode can be EmptyDir, HostPath, StorageClass | ||
| storageMode: EmptyDir | ||
| # hostPath is the local path to store data | ||
| hostPath: /tmp/data/rocketmq/broker | ||
| # scalePodName is [Broker name]-[broker group number]-master-0 | ||
| scalePodName: broker-0-master-0 | ||
| # env defines custom env, e.g. BROKER_MEM | ||
| env: | ||
| - name: BROKER_MEM | ||
| valueFrom: | ||
| configMapKeyRef: | ||
| name: broker-config | ||
| key: BROKER_MEM | ||
| # volumes defines the broker.conf | ||
| volumes: | ||
| - name: broker-config | ||
| configMap: | ||
| name: broker-config | ||
| items: | ||
| - key: broker-common.conf | ||
| path: broker-common.conf | ||
| # volumeClaimTemplates defines the storageClass | ||
| volumeClaimTemplates: | ||
| - metadata: | ||
| name: broker-storage | ||
| spec: | ||
| accessModes: | ||
| - ReadWriteOnce | ||
| resources: | ||
| requests: | ||
| storage: 8Gi | ||
| selector: | ||
| matchLabels: | ||
| app: broker-storage-pv | ||
| --- | ||
| apiVersion: rocketmq.apache.org/v1alpha1 | ||
| kind: NameService | ||
| metadata: | ||
| name: name-service | ||
| spec: | ||
| # size is the the name service instance number of the name service cluster | ||
| size: 1 | ||
| # nameServiceImage is the customized docker image repo of the RocketMQ name service | ||
| nameServiceImage: apacherocketmq/rocketmq-nameserver:4.5.0-alpine-operator-0.3.0 | ||
| # imagePullPolicy is the image pull policy | ||
| imagePullPolicy: Always | ||
| # hostNetwork can be true or false | ||
| hostNetwork: true | ||
| podAnnotations: | ||
| prometheus.io/path: /metrics | ||
| prometheus.io/port: "5557" | ||
| prometheus.io/scrape: "true" | ||
| securityContext: | ||
| allowPrivilegeEscalation: true | ||
|
There was a problem hiding this comment.
|
||
| runAsUser: 0 | ||
|
There was a problem hiding this comment. The NameService securityContext sets allowPrivilegeEscalation: true, runAsUser: 0, runAsGroup: 0. This example is intended as a copyable template, and running the name server as root with privilege escalation enabled is a poor security pattern for an Apache-shipped reference. Consider a non-root runAsUser (the alpine image supports a dedicated UID) and allowPrivilegeEscalation: false, or at minimum document the risk and make it opt-in. There was a problem hiding this comment. Example configures NameService to run as root ( |
||
| runAsGroup: 0 | ||
| affinity: | ||
| nodeAffinity: | ||
| requiredDuringSchedulingIgnoredDuringExecution: | ||
| nodeSelectorTerms: | ||
| - matchExpressions: | ||
| - key: kubernetes.io/hostname | ||
| operator: In | ||
| values: | ||
| - azure-k8s-7 | ||
| - azure-k8s-1 | ||
| - azure-k8s-3 | ||
| - azure-k8s-2 | ||
| podAntiAffinity: | ||
| preferredDuringSchedulingIgnoredDuringExecution: | ||
| - weight: 100 | ||
|
There was a problem hiding this comment. The example sets |
||
| podAffinityTerm: | ||
| labelSelector: | ||
| matchExpressions: | ||
| - key: app | ||
| operator: In | ||
| values: | ||
| - name_service | ||
| topologyKey: kubernetes.io/hostname | ||
| priorityClassName: "" | ||
| imagePullSecrets: | ||
| - name: registry-secret | ||
| nodeSelector: | ||
| azure: "true" | ||
| tolerations: | ||
| - key: "key1" | ||
| operator: "Equal" | ||
| value: "value1" | ||
| effect: "NoSchedule" | ||
| - key: "key2" | ||
| operator: "Equal" | ||
| value: "value1" | ||
| effect: "NoExecute" | ||
| # Set DNS policy for the pod. | ||
| # Defaults to "ClusterFirst". | ||
| # Valid values are 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'. | ||
| # DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy. | ||
| # To have DNS options set along with hostNetwork, you have to specify DNS policy | ||
| # explicitly to 'ClusterFirstWithHostNet'. | ||
| dnsPolicy: ClusterFirstWithHostNet | ||
| # resources describes the compute resource requirements and limits | ||
| resources: | ||
| requests: | ||
| memory: "512Mi" | ||
| cpu: "250m" | ||
| limits: | ||
| memory: "1024Mi" | ||
| cpu: "500m" | ||
| # storageMode can be EmptyDir, HostPath, StorageClass | ||
| storageMode: EmptyDir | ||
| # hostPath is the local path to store data | ||
| hostPath: /data/rocketmq/nameserver | ||
| # volumeClaimTemplates defines the storageClass | ||
| volumeClaimTemplates: | ||
| - metadata: | ||
| name: namesrv-storage | ||
| spec: | ||
| accessModes: | ||
| - ReadWriteOnce | ||
| storageClassName: rocketmq-storage | ||
| resources: | ||
| requests: | ||
| storage: 1Gi | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -401,8 +401,15 @@ func (r *ReconcileBroker) getBrokerStatefulSet(broker *rocketmqv1alpha1.Broker, | |
| Template: corev1.PodTemplateSpec{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Labels: ls, | ||
| Annotations: broker.Spec.PodAnnotations, | ||
|
There was a problem hiding this comment. The new pod-scheduling fields are wired into getBrokerStatefulSet (Annotations, Affinity, SecurityContext, ImagePullSecrets, Tolerations, NodeSelector, PriorityClassName) and similarly into nameservice_controller.go, but no test changes accompany this PR. There is no coverage verifying that CR-spec values propagate to the rendered StatefulSet PodTemplateSpec, nor that nil/empty values (e.g. unset Affinity, empty PriorityClassName) render safely. Add reconciler tests asserting these fields pass through correctly. There was a problem hiding this comment.
|
||
| }, | ||
|
There was a problem hiding this comment. All six new PodSpec fields ( |
||
| Spec: corev1.PodSpec{ | ||
| Affinity: broker.Spec.Affinity, | ||
| SecurityContext: broker.Spec.SecurityContext, | ||
| ImagePullSecrets: broker.Spec.ImagePullSecrets, | ||
| Tolerations: broker.Spec.Tolerations, | ||
| NodeSelector: broker.Spec.NodeSelector, | ||
| PriorityClassName: broker.Spec.PriorityClassName, | ||
|
Comment on lines
+407
to
+412
Member
There was a problem hiding this comment. You need to write functions for each of them and handle nil value scenarios, by directly adding this you are forcing the user at all times to specify these values, the operator will crash if you don't pass in tolerations.
Contributor
There was a problem hiding this comment. +1 This pr added features that help enrich operator, hope that corresponding null check would be supplemented. |
||
| Containers: []corev1.Container{{ | ||
| Resources: broker.Spec.Resources, | ||
| Image: broker.Spec.BrokerImage, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new complex Kubernetes types (affinity, securityContext, tolerations, imagePullSecrets items) are declared as bare type: object/item type: object with no OpenAPI v3 properties. This provides no server-side validation, so malformed scheduling rules are accepted silently and only surface as Pod creation errors later. For a cleaner CRD (and better kubectl explain output), define the full schema via x-kubernetes-preserve-unknown-fields or the concrete property trees. Same applies to rocketmq_v1alpha1_nameservice_crd.yaml.