一、准备工作
1、拉取kafka镜像
docker pull wurstmeister/kafka
docker tag docker.io/wurstmeister/kafka kafka
docker rmi docker.io/wurstmeister/kafka
2、拉取kafka可视化管理工具镜像
docker pull sheepkiller/kafka-manager
docker tag docker.io/sheepkiller/kafka-manager kafka-manager
docker rmi docker.io/sheepkiller/kafka-manager
3、安装docker-compose工具
# 升级 pip
pip3 install --upgrade pip
# 指定 docker-compose 版本安装
pip install docker-compose==1.22
# 验证是否安装成功,有返回值,说明安装成功
docker-compose -v
4、创建相关文件夹
mkdir -p /data/docker-compose/kafka
mkdir -p /data/docker-data/kafka
5、创建网络,用于kafka和zookeeper共享一个网络段
docker network create --driver bridge zookeeper_kafka_net
6、构建zookeeper集群
kafka集群需要用到zookeeper集群,因此需要先构建zookeeper集群,请查看文章
docker快速搭建zookeeper集群
二、使用docker-compose编排kafka集群
1、创建docker-compose.yml
cd /data/docker-compose/kafka
vi docker-compose.yml
version: '3'
services:
kafka1:
image: kafka
restart: always
container_name: kafka1
hostname: kafka1
ports:
- 9091:9092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ADVERTISED_HOST_NAME: kafka1
KAFKA_ADVERTISED_PORT: 9091
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.78.200:9091
KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
volumes:
- /data/docker-data/kafka1/docker.sock:/var/run/docker.sock
- /data/docker-data/kafka1/data:/kafka
external_links:
- zoo1
- zoo2
- zoo3
kafka2:
image: kafka
restart: always
container_name: kafka2
hostname: kafka2
ports:
- 9092:9092
environment:
KAFKA_BROKER_ID: 2
KAFKA_ADVERTISED_HOST_NAME: kafka2
KAFKA_ADVERTISED_PORT: 9092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.78.200:9092
KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
volumes:
- /data/docker-data/kafka2/docker.sock:/var/run/docker.sock
- /data/docker-data/kafka2/data:/kafka
external_links:
- zoo1
- zoo2
- zoo3
kafka3:
image: kafka
restart: always
container_name: kafka3
hostname: kafka3
ports:
- 9093:9092
environment:
KAFKA_BROKER_ID: 3
KAFKA_ADVERTISED_HOST_NAME: kafka3
KAFKA_ADVERTISED_PORT: 9093
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.78.200:9093
KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
volumes:
- /data/docker-data/kafka3/docker.sock:/var/run/docker.sock
- /data/docker-data/kafka3/data:/kafka
external_links:
- zoo1
- zoo2
- zoo3
kafka-manager:
image: kafka-manager
restart: always
container_name: kafka-manager
hostname: kafka-manager
ports:
- 9010:9000
links:
- kafka1
- kafka2
- kafka3
external_links:
- zoo1
- zoo2
- zoo3
environment:
ZK_HOSTS: zoo1:2181,zoo2:2181,zoo3:2181
注:links是引入当前docker-compose内部的service,external_links 引入的是当前docker-compose外部的service
2、执行构建
docker-compose up -d
3、将所有kafka和zookeeper加入一个网络
docker network connect zookeeper_kafka_net zoo1
docker network connect zookeeper_kafka_net zoo2
docker network connect zookeeper_kafka_net zoo3
docker network connect zookeeper_kafka_net kafka1
docker network connect zookeeper_kafka_net kafka2
docker network connect zookeeper_kafka_net kafka3
docker network connect zookeeper_kafka_net kafka-manager
之所以这样指定网络而不是在编排文件中指定网络为一个网络段的原因是因为考虑到后面可能kafka和zookeeper分布在6台机器上,到时使用docker swarm构建docker 集群,网络处理就很方便,不用修改编排文件
4、重启这组服务
因为为每台容器都新增了一个网络,如果不重启,容器用的还是之前的网络,就会导致kafka和kafka-manager是ping不通zookeeper的三个容器的
docker-compose restart
三、验证结果
浏览器访问kafka-manager管理界面
192.168.78.200:9010
文章来源:https://www.toymoban.com/news/detail-541939.html
说明kafka集群构建好了,并且kafka集群已经完成了和zookeeper集群的互通文章来源地址https://www.toymoban.com/news/detail-541939.html
到了这里,关于docker快速搭建kafka集群的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!