rocketmq在Kubernetes(k8s)中的集群配置,2m-2s-async:多Master多Slave模式,异步复制

目标 rocketmq 集群

2m-2s-async:多Master多Slave模式,异步复制

宿主机IP: 192.168.0.254

fox.风

源码 地址 https://github.com/foxiswho/k8s-nacos-sentinel-rocketmq-zipkin-elasticsearch-redis-mysql

配置目录

mkdir -p /www/k8s/foxdev/rocketmq/logs-1/
mkdir -p /www/k8s/foxdev/rocketmq/store-1/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-1a/
mkdir -p /www/k8s/foxdev/rocketmq/store-1a/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-1as/
mkdir -p /www/k8s/foxdev/rocketmq/store-1as/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-1as2/
mkdir -p /www/k8s/foxdev/rocketmq/store-1as2/commitlog

mkdir -p /www/k8s/foxdev/rocketmq/logs-2/
mkdir -p /www/k8s/foxdev/rocketmq/store-2/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-2b/
mkdir -p /www/k8s/foxdev/rocketmq/store-2b/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-2bs/
mkdir -p /www/k8s/foxdev/rocketmq/store-2bs/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-2bs2/
mkdir -p /www/k8s/foxdev/rocketmq/store-2bs2/commitlog

chmod -R 777 /www/k8s/foxdev/rocketmq/*

logs开头目录为 日志目录

store开头目录为 数据目录

-1 表示 rocketmq nameserver

-1a 表示 rocketmq broker

-1as 表示 rocketmq broker slave

logs-ui 为 rocketmq web ui 管理 日志目录

其他以此类推

相关说明

nodePort 后面表示 外部端口
port 后面表示 内部端口

集群1端口如下

spec:
  type: NodePort
  ports:
    - port: 9876
      targetPort: 9876
      nodePort: 30876
      name: rmqser
    - port: 30911
      targetPort: 30911
      nodePort: 30911
      name: rmqbroker
    - port: 30921
      targetPort: 30921
      nodePort: 30921
      name: rmqbrokerslave
    - port: 30931
      targetPort: 30931
      nodePort: 30931
      name: rmqbrokerslave2

集群2端口如下

spec:
  type: NodePort
  ports:
    - port: 9876
      targetPort: 9876
      nodePort: 30877
      name: rmqser2
    - port: 30961
      targetPort: 30961
      nodePort: 30961
      name: rmqbroker2
    - port: 30971
      targetPort: 30971
      nodePort: 30971
      name: rmqbrokerslave2
    - port: 30981
      targetPort: 30981
      nodePort: 30981
      name: rmqbrokerslave22

web ui 端口 和 管理员与密码

端口 30879 ,管理员账户和密码如下

admin
admin

账户和密码 在 配置文件在 rocketmq/conf/users.properties 文件中修改,

# This file supports hot change, any change will be auto-reloaded without Console restarting.
# Format: a user per line, username=password[,N] #N is optional, 0 (Normal User); 1 (Admin)

# Define Admin
admin=admin,1

第一个admin 表示 用户名

第二个admin 表示 密码

1 表示 管理员 角色

集群1配置

broker 配置 broker-a.conf

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


#所属集群名字
brokerClusterName=DefaultCluster

#broker名字,注意此处不同的配置文件填写的不一样,如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-a

#0 表示Master,>0 表示Slave
brokerId=0

#nameServer地址,分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876

#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);,解决方式2 brokerIP1 设置宿主机IP,不要使用docker 内部IP
brokerIP1=192.168.0.254

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭 !!!这里仔细看是false,false,false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true

#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口
listenPort=30911
#haService中使用  默认值为:listenPort + 1
#haListenPort=30912
#主要用于slave同步master  listenPort - 2
#fastListenPort=30909

#删除文件时间点,默认凌晨4点
deleteWhen=04

#文件保留时间,默认48小时
fileReservedTime=120

#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536

#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000

#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER

#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

broker-slave 配置 broker-a-s.conf

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


#所属集群名字
brokerClusterName=DefaultCluster

#broker名字,注意此处不同的配置文件填写的不一样,如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-a

#0 表示Master,>0 表示Slave
brokerId=1

#nameServer地址,分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876

#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);,解决方式2 brokerIP1 设置宿主机IP,不要使用docker 内部IP
brokerIP1=192.168.0.254

#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭 !!!这里仔细看是false,false,false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true

#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口
listenPort=30921
#haService中使用  默认值为:listenPort + 1
#haListenPort=31912
#主要用于slave同步master  listenPort - 2
#fastListenPort=31909

#删除文件时间点,默认凌晨4点
deleteWhen=04

#文件保留时间,默认48小时
fileReservedTime=120

#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536

#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000



#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

broker-slave 配置 broker-a-s2.conf

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


#所属集群名字
brokerClusterName=DefaultCluster

#broker名字,注意此处不同的配置文件填写的不一样,如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-a

#0 表示Master,>0 表示Slave
brokerId=2

#nameServer地址,分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876

#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);,解决方式2 brokerIP1 设置宿主机IP,不要使用docker 内部IP
brokerIP1=192.168.0.254

#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭 !!!这里仔细看是false,false,false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true

#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口
listenPort=30931
#haService中使用  默认值为:listenPort + 1
#haListenPort=31912
#主要用于slave同步master  listenPort - 2
#fastListenPort=31909

#删除文件时间点,默认凌晨4点
deleteWhen=04

#文件保留时间,默认48小时
fileReservedTime=120

#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536

#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000



#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

k8s pod

rocketmq1-pod.yml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rocketmq
  labels:
    app: rocketmq

spec:
  serviceName: rocketmq
  replicas: 1
  selector:
    matchLabels:
      app: rocketmq
  template:
    metadata:
      labels:
        app: rocketmq
    spec:
      containers:
        - name: rocketmq
          image: foxiswho/rocketmq:4.8.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 450m
              memory: 2000Mi
            requests:
              cpu: 400m
              memory: 2000Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT
              value: -Duser.home=/home/rocketmq
            - name: JAVA_OPT_EXT
              value:  -Xmn256m
          command: ["sh","-c","mqnamesrv"]
          ports:
            - containerPort: 9876
          volumeMounts:
            - name: vol-logs
              mountPath: /home/rocketmq/logs
            - name: vol-store
              mountPath: /home/rocketmq/store
        - name: rmqbroker
          image: foxiswho/rocketmq:4.8.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 440m
              memory: 2000Mi
            requests:
              cpu: 400m
              memory: 2000Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT_EXT
              value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
          command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-a.conf" ]
          ports:
            - containerPort: 30909
            - containerPort: 30911
            - containerPort: 30912
          volumeMounts:
            - name: vol-logs-a
              mountPath: /home/rocketmq/logs
            - name: vol-store-a
              mountPath: /home/rocketmq/store
            - name: vol-conf
              mountPath: /etc/rocketmq/
        - name: rmqbroker-slave
          image: foxiswho/rocketmq:4.8.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 450m
              memory: 2000Mi
            requests:
              cpu: 400m
              memory: 2000Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT_EXT
              value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
          command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-a-s.conf" ]
          ports:
            - containerPort: 30919
            - containerPort: 30921
            - containerPort: 30922
          volumeMounts:
            - name: vol-logs-as
              mountPath: /home/rocketmq/logs
            - name: vol-store-as
              mountPath: /home/rocketmq/store
            - name: vol-conf
              mountPath: /etc/rocketmq/
        - name: rmqbroker-slave2
          image: foxiswho/rocketmq:4.8.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 450m
              memory: 2000Mi
            requests:
              cpu: 400m
              memory: 2000Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT_EXT
              value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
          command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-a-s2.conf" ]
          ports:
            - containerPort: 30929
            - containerPort: 30931
            - containerPort: 30932
          volumeMounts:
            - name: vol-logs-as2
              mountPath: /home/rocketmq/logs
            - name: vol-store-as2
              mountPath: /home/rocketmq/store
            - name: vol-conf
              mountPath: /etc/rocketmq/
      volumes:
        - name: vol-logs
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-1
            type: Directory
        - name: vol-store
          hostPath:
            path: /www/k8s/foxdev/rocketmq/store-1
            type: Directory
        - name: vol-conf
          hostPath:
            path: /www/k8s/foxdev/rocketmq/conf
            type: Directory
        - name: vol-logs-a
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-1a
            type: Directory
        - name: vol-store-a
          hostPath:
            path: /www/k8s/foxdev/rocketmq/store-1a
            type: Directory
        - name: vol-logs-as
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-1as
            type: Directory
        - name: vol-store-as
          hostPath:
            path: /www/k8s/foxdev/rocketmq/store-1as
            type: Directory
        - name: vol-logs-as2
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-1as2
            type: Directory
        - name: vol-store-as2
          hostPath:
            path: /www/k8s/foxdev/rocketmq/store-1as2
            type: Directory

k8s service

rocketmq1-service.yml

apiVersion: v1
kind: Service
metadata:
  name: rocketmq
  labels:
    app: rocketmq
spec:
  type: NodePort
  ports:
    - port: 9876
      targetPort: 9876
      nodePort: 30876
      name: rmqser
    - port: 30911
      targetPort: 30911
      nodePort: 30911
      name: rmqbroker
    - port: 30921
      targetPort: 30921
      nodePort: 30921
      name: rmqbrokerslave
    - port: 30931
      targetPort: 30931
      nodePort: 30931
      name: rmqbrokerslave2
  selector:
    app: rocketmq

集群2配置

broker 配置 broker-b.conf

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


#所属集群名字
brokerClusterName=DefaultCluster

#broker名字,注意此处不同的配置文件填写的不一样,如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-b

#0 表示Master,>0 表示Slave
brokerId=0

#nameServer地址,分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876

#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);,解决方式2 brokerIP1 设置宿主机IP,不要使用docker 内部IP
brokerIP1=192.168.0.254

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭 !!!这里仔细看是false,false,false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true

#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口
listenPort=30961
#haService中使用  默认值为:listenPort + 1
#haListenPort=30912
#主要用于slave同步master  listenPort - 2
#fastListenPort=30909

#删除文件时间点,默认凌晨4点
deleteWhen=04

#文件保留时间,默认48小时
fileReservedTime=120

#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536

#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000

#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER

#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

broker-slave 配置 broker-b-s.conf

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


#所属集群名字
brokerClusterName=DefaultCluster

#broker名字,注意此处不同的配置文件填写的不一样,如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-b

#0 表示Master,>0 表示Slave
brokerId=1

#nameServer地址,分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876

#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);,解决方式2 brokerIP1 设置宿主机IP,不要使用docker 内部IP
brokerIP1=192.168.0.254

#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭 !!!这里仔细看是false,false,false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true

#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口
listenPort=30971
#haService中使用  默认值为:listenPort + 1
#haListenPort=30912
#主要用于slave同步master  listenPort - 2
#fastListenPort=30909

#删除文件时间点,默认凌晨4点
deleteWhen=04

#文件保留时间,默认48小时
fileReservedTime=120

#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536

#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000



#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

broker-slave 配置 broker-b-s2.conf

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


#所属集群名字
brokerClusterName=DefaultCluster

#broker名字,注意此处不同的配置文件填写的不一样,如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-b

#0 表示Master,>0 表示Slave
brokerId=2

#nameServer地址,分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876

#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);,解决方式2 brokerIP1 设置宿主机IP,不要使用docker 内部IP
brokerIP1=192.168.0.254

#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE

#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4

#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭 !!!这里仔细看是false,false,false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true

#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true

#Broker 对外服务的监听端口
listenPort=30981
#haService中使用  默认值为:listenPort + 1
#haListenPort=30912
#主要用于slave同步master  listenPort - 2
#fastListenPort=30909

#删除文件时间点,默认凌晨4点
deleteWhen=04

#文件保留时间,默认48小时
fileReservedTime=120

#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824

#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000

#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536

#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000



#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

k8s pod

rocketmq2-pod.yml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rocketmq2
  labels:
    app: rocketmq2

spec:
  serviceName: rocketmq2
  replicas: 1
  selector:
    matchLabels:
      app: rocketmq2
  template:
    metadata:
      labels:
        app: rocketmq2
    spec:
      containers:
        - name: rocketmq
          image: foxiswho/rocketmq:4.8.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 450m
              memory: 2000Mi
            requests:
              cpu: 400m
              memory: 2000Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT
              value: -Duser.home=/home/rocketmq
            - name: JAVA_OPT_EXT
              value:  -Xmn256m
          command: ["sh","-c","mqnamesrv"]
          ports:
            - containerPort: 9876
          volumeMounts:
            - name: vol-logs
              mountPath: /home/rocketmq/logs
            - name: vol-store
              mountPath: /home/rocketmq/store
        - name: rmqbroker
          image: foxiswho/rocketmq:4.8.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 440m
              memory: 2000Mi
            requests:
              cpu: 400m
              memory: 2000Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT_EXT
              value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
          command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-b.conf" ]
          ports:
            - containerPort: 30959
            - containerPort: 30961
            - containerPort: 30962
          volumeMounts:
            - name: vol-logs-a
              mountPath: /home/rocketmq/logs
            - name: vol-store-a
              mountPath: /home/rocketmq/store
            - name: vol-conf
              mountPath: /etc/rocketmq/
        - name: rmqbroker-slave
          image: foxiswho/rocketmq:4.8.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 450m
              memory: 2000Mi
            requests:
              cpu: 400m
              memory: 2000Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT_EXT
              value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
          command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-b-s.conf" ]
          ports:
            - containerPort: 30969
            - containerPort: 30971
            - containerPort: 30972
          volumeMounts:
            - name: vol-logs-as
              mountPath: /home/rocketmq/logs
            - name: vol-store-as
              mountPath: /home/rocketmq/store
            - name: vol-conf
              mountPath: /etc/rocketmq/
        - name: rmqbroker-slave2
          image: foxiswho/rocketmq:4.8.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 450m
              memory: 2000Mi
            requests:
              cpu: 400m
              memory: 2000Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT_EXT
              value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
          command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-b-s2.conf" ]
          ports:
            - containerPort: 30979
            - containerPort: 30981
            - containerPort: 30982
          volumeMounts:
            - name: vol-logs-as2
              mountPath: /home/rocketmq/logs
            - name: vol-store-as2
              mountPath: /home/rocketmq/store
            - name: vol-conf
              mountPath: /etc/rocketmq/
      volumes:
        - name: vol-logs
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-2
            type: Directory
        - name: vol-store
          hostPath:
            path: /www/k8s/foxdev/rocketmq/store-2
            type: Directory
        - name: vol-conf
          hostPath:
            path: /www/k8s/foxdev/rocketmq/conf
            type: Directory
        - name: vol-logs-a
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-2b
            type: Directory
        - name: vol-store-a
          hostPath:
            path: /www/k8s/foxdev/rocketmq/store-2b
            type: Directory
        - name: vol-logs-as
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-2bs
            type: Directory
        - name: vol-store-as
          hostPath:
            path: /www/k8s/foxdev/rocketmq/store-2bs
            type: Directory
        - name: vol-logs-as2
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-2bs2
            type: Directory
        - name: vol-store-as2
          hostPath:
            path: /www/k8s/foxdev/rocketmq/store-2bs2
            type: Directory

k8s service

rocketmq2-service.yml

apiVersion: v1
kind: Service
metadata:
  name: rocketmq2
  labels:
    app: rocketmq2
spec:
  type: NodePort
  ports:
    - port: 9876
      targetPort: 9876
      nodePort: 30877
      name: rmqser2
    - port: 30961
      targetPort: 30961
      nodePort: 30961
      name: rmqbroker2
    - port: 30971
      targetPort: 30971
      nodePort: 30971
      name: rmqbrokerslave2
    - port: 30981
      targetPort: 30981
      nodePort: 30981
      name: rmqbrokerslave22
  selector:
    app: rocketmq2

UI

pod

ui-rc.yml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rmqui
  labels:
    app: rmqui

spec:
  serviceName: rmqui
  replicas: 1
  selector:
    matchLabels:
      app: rmqui
  template:
    metadata:
      labels:
        app: rmqui
    spec:
      containers:
        - name: rmqui
          image: pongsky/rocketmq-console-ng
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 450m
              memory: 1024Mi
            requests:
              cpu: 400m
              memory: 768Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPTS
              value: -Drocketmq.config.namesrvAddr=rocketmq:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false -Duser.home=/root -Drocketmq.config.dataPath=/tmp/rocketmq-console/data -Xms768m -Xmx768m
          ports:
            - containerPort: 8080
          volumeMounts:
            - name: vol-logs
              mountPath: /root/logs
            - name: vol-conf
              mountPath: /tmp/rocketmq-console/data/users.properties
      volumes:
        - name: vol-logs
          hostPath:
            path: /www/k8s/foxdev/rocketmq/logs-ui
            type: Directory
        - name: vol-conf
          hostPath:
            path: /www/k8s/foxdev/rocketmq/conf/users.properties
            type: File

service

ui-service.yml

apiVersion: v1
kind: Service
metadata:

  name: rmqui
  labels:
    app: rmqui
spec:
  type: NodePort
  ports:
    - port: 8080
      targetPort: 8080
      nodePort: 30879
  selector:
    app: rmqui

创建执行

kubectl create -f  ui-rc.yml
kubectl create -f  ui-service.yml

kubectl create -f  rocketmq1-pod.yml
kubectl create -f  rocketmq1-service.yml

kubectl create -f  rocketmq2-pod.yml
kubectl create -f  rocketmq2-service.yml

至此创建完成

已标记关键词 清除标记
©️2020 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页