手把手教你快速在生产环境搭建Doris集群
一、部署规划
组件分布规划
| 节点 | node4 | node5 | node6 | node7 | node8 | Node9 | Node10 | Node11 |
|---|---|---|---|---|---|---|---|---|
| 服务 | FE（follower） BE |
FE（follower） BE |
FE（follower） BE |
FE（observer） BE |
BE Broker |
BE Broker |
BE | BE |
| Ip | 192.168.22.34 | 192.168.22.35 | 192.168.22.36 | 192.168.22.37 | 192.168.22.20 | 192.168.22.16 | 192.168.22.17 | 192.168.22.18 |
二、部署准备
安装包准备
根据自己集群的软硬件配置，选择合适的版本进行编译，这里我下载编译doris1.2.2版本源码，推荐使用Docker
https://doris.apache.org/zh-CN/docs/install/source-install/compilation
安装目录准备
1.部署目录规划
为了方便后期升级维护，数据存储，元数据和日志尽量不要放在安装包内。
| FE | 部署目录：/opt/module/doris/fe 日志目录：/var/log/doris/fe 元数据目录：/mnt/disk4/doris/meta |
|---|---|
| BE | 部署目录：/opt/module/doris/be 日志目录：/var/log/doris/be 数据存储目录：/mnt/disk4/doris/storage，/mnt/disk5/doris/storage，/mnt/disk6/doris/storage |
| Broker | 部署目录：/opt/module/doris/apache_hdfs_broker |
2.数据存储目录创建
在BE上创建数据存储目录，根据规划node4到node11
[root@node4 hadoop]# mkdir -p /mnt/disk4/doris/storage
[root@node4 hadoop]# mkdir -p /mnt/disk5/doris/storage
[root@node4 hadoop]# mkdir -p /mnt/disk6/doris/storage
3.元数据存储目录创建
FE上需要创建元数据存储目录，node4到node7
[root@node4 hadoop]# mkdir -p /mnt/disk4/doris/meta
4.修改数据和元数据文件目录归属
管理Doris时，我们不能使用root用户直接管理，我们使用具有管理权限的hadoop用户进行管理
node4到node11，元数据和数据权限一起修改
[root@node4 hadoop]# chown -R hadoop:hadoop /mnt/disk4/doris
[root@node4 hadoop]# chown -R hadoop:hadoop /mnt/disk5/doris
[root@node4 hadoop]# chown -R hadoop:hadoop /mnt/disk6/doris
5.日志目录创建
BE的日志目录，node4-node11
[root@node4 hadoop]# mkdir -p /var/log/doris/be
FE的日志目录，node4-node7
[root@node4 hadoop]# mkdir -p /var/log/doris/fe
6.修改日志目录权限归属
[root@node4 hadoop]# chown -R hadoop:hadoop /var/log/doris
三、部署安装
配置文件
1.修改FE配置文件
vim /opt/module/doris-1.2.2/fe/conf/fe.conf
日志存储目录
LOG_DIR = /var/log/doris/fe
修改jvm参数，-Xmx调整为16G
JAVA_OPTS="-Xmx16384m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"
元数据存储目录
meta_dir = /mnt/disk4/doris/meta
绑定ip
priority_networks = 192.168.22.34/24
2.修改be配置
配置java环境变量
由于从 1.2 版本开始支持 Java UDF 函数，BE 依赖于 Java 环境。所以要预先配置 JAVA_HOME 环境变量，也可以在 start_be.sh 启动脚本第一行添加 export JAVA_HOME=your_java_home_path 来添加环境变量。
vim /opt/module/doris-1.2.2/be/bin/start_be.sh
第一行添加
export JAVA_HOME=/usr/java/jdk
配置be，修改配置文件
vim /opt/module/doris-1.2.2/be/conf/be.conf
日志存储目录
sys_log_dir = /var/log/doris/be
默认的8040端口和hadoop端口冲突了修改为18040
# ports for admin, web, heartbeat service
be_port = 9060
webserver_port = 18040
heartbeat_service_port = 9050
brpc_port = 8060
修改绑定ip
priority_networks = 192.168.22.34/24
修改存储目录
路径和硬盘类型间用“，”隔开，多路径用“；”隔开，我这里时机械硬盘medium值为HDD，固态硬盘使用SSD
storage_root_path = /mnt/disk4/doris/storage,medium:HDD;/mnt/disk5/doris/storage,medium:HDD;/mnt/disk6/doris/storage,medium:HDD
3.Broker配置
把hdfs配置文件复制到Broker目录下
mv hdfs-site.xml /opt/module/doris-1.2.2/apache_hdfs_broker/conf/
4.分发与修改配置
4.1.分发
到node4-node11
xsync /opt/module/doris-1.2.2
4.2.修改绑定ip
修改其余节点的绑定ip
node4-node7
vim /opt/module/doris-1.2.2/fe/conf/fe.conf
fe改为对应的绑定ip
...
be修改为对应绑定ip
node8-node11
vim /opt/module/doris-1.2.2/be/conf/be.conf
be修改为对应绑定ip
...
5.添加组件和启动集群
5.1.启动第一个FE
[hadoop@node4 doris]$ fe/bin/start_fe.sh --daemon
查看一下进出是否存在
[hadoop@node4 ~]$ jps | grep PaloFe
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
221813 dorisFE
5.2安装mysql客户端
集群安装mysql客户端
使用客户端连接Fe
mysql -h node4 -P 9030 -uroot
查看fe节点
mysql> show frontends;
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| Name | IP | EditLogPort | HttpPort | QueryPort | RpcPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| 192.168.22.34_9010_1679021376875 | 192.168.22.34 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | true | 1095009859 | true | true | 144 | 2023-03-17 10:57:15 | true | | doris-1.2.2-rc01-Unknown | Yes |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
第一次连接，没有密码，这里我们修改root密码
SET PASSWORD FOR 'root' = PASSWORD('000000');
5.3.添加其余的FE节点
ALTER SYSTEM ADD FOLLOWER "192.168.22.35:9010";
ALTER SYSTEM ADD FOLLOWER "192.168.22.36:9010";
ALTER SYSTEM ADD OBSERVER "192.168.22.37:9010";
再查看
show frontends
可以看到其他节点FE已经添加上了，只是连不上，因为我们还没有启动
mysql> show frontends;
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+-------+-------+-------------------+---------------------+----------+--------------------------------------------------------------+--------------------------+------------------+
| Name | IP | EditLogPort | HttpPort | QueryPort | RpcPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+-------+-------+-------------------+---------------------+----------+--------------------------------------------------------------+--------------------------+------------------+
| 192.168.22.36_9010_1679022090994 | 192.168.22.36 | 9010 | 8030 | 0 | 0 | FOLLOWER | false | 1095009859 | false | false | 0 | NULL | true | java.net.ConnectException: 拒绝连接 (Connection refused) | NULL | No |
| 192.168.22.34_9010_1679021376875 | 192.168.22.34 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | true | 1095009859 | true | true | 232 | 2023-03-17 11:01:57 | true | | doris-1.2.2-rc01-Unknown | Yes |
| 192.168.22.35_9010_1679022090967 | 192.168.22.35 | 9010 | 8030 | 0 | 0 | FOLLOWER | false | 1095009859 | false | false | 0 | NULL | true | java.net.ConnectException: 拒绝连接 (Connection refused) | NULL | No |
| 192.168.22.37_9010_1679022093117 | 192.168.22.37 | 9010 | 8030 | 0 | 0 | OBSERVER | false | 1095009859 | false | false | 0 | NULL | false | java.net.ConnectException: 拒绝连接 (Connection refused) | NULL | No |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+-------+-------+-------------------+---------------------+----------+--------------------------------------------------------------+--------------------------+------------------+
4 rows in set (0.02 sec)
5.4.启动其他FE
注意：第一次启动需指定–helper参数，后续再启动无需指定此参数
node5-node7
分别进入对应节点的fe目录
cd /opt/module/doris-1.2.2/fe
分别执行如下命令
bin/start_fe.sh --helper 192.168.22.34:9010 --daemon
分别查看FE进程
[hadoop@node7 fe]$ jps | grep PaloFe
12644 PaloFe
第三次使用mysql客户端查看
show frontends
结果如下
mysql> show frontends;
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| Name | IP | EditLogPort | HttpPort | QueryPort | RpcPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| 192.168.22.36_9010_1679022090994 | 192.168.22.36 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | false | 1095009859 | true | true | 357 | 2023-03-17 11:09:03 | true | | doris-1.2.2-rc01-Unknown | No |
| 192.168.22.34_9010_1679021376875 | 192.168.22.34 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | true | 1095009859 | true | true | 359 | 2023-03-17 11:09:03 | true | | doris-1.2.2-rc01-Unknown | Yes |
| 192.168.22.35_9010_1679022090967 | 192.168.22.35 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | false | 1095009859 | true | true | 357 | 2023-03-17 11:09:03 | true | | doris-1.2.2-rc01-Unknown | No |
| 192.168.22.37_9010_1679022093117 | 192.168.22.37 | 9010 | 8030 | 9030 | 9020 | OBSERVER | false | 1095009859 | true | true | 357 | 2023-03-17 11:09:03 | false | | doris-1.2.2-rc01-Unknown | No |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
注意看4个FE全部处于激活状态
5.5.添加BE
mysql客户端执行对应的如下语句
ALTER SYSTEM ADD BACKEND "192.168.22.34:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.35:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.36:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.37:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.20:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.16:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.17:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.18:9050";
6.6.启动其他BE
node4-node11
进入BE目录
cd /opt/module/doris/be
启动
/be/bin/start_be.sh --daemon
提示(若没有如下提示，跳过)
Please set vm.max_map_count to be 2000000 under root using 'sysctl -w vm.max_map_count=2000000'.
根据提示，分别在be节点执行
sysctl -w vm.max_map_count=2000000
再在各个be节点，启动be。
查看状况
mysql客户端
SHOW backends;
mysql> SHOW backends;
+-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| BackendId | Cluster | IP | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole |
+-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| 11006 | default_cluster | 192.168.22.16 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:14:29 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 10.567 TB | 10.911 TB | 3.15 % | 3.24 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:37","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11007 | default_cluster | 192.168.22.17 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:14:29 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 10.575 TB | 10.911 TB | 3.08 % | 3.11 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:35","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11008 | default_cluster | 192.168.22.18 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:14:30 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 10.562 TB | 10.911 TB | 3.20 % | 3.24 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:35","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11001 | default_cluster | 192.168.22.34 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:12 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 7.731 TB | 10.744 TB | 28.05 % | 28.51 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:25","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11002 | default_cluster | 192.168.22.35 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:13 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 7.832 TB | 10.744 TB | 27.11 % | 27.34 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:35","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11003 | default_cluster | 192.168.22.36 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:12 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 8.204 TB | 10.744 TB | 23.64 % | 24.17 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:41","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11004 | default_cluster | 192.168.22.37 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:14 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 8.424 TB | 10.744 TB | 21.59 % | 23.50 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:33","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11005 | default_cluster | 192.168.22.20 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:16 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 7.340 TB | 10.744 TB | 31.68 % | 31.77 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:35","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
+-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
查看进程
[hadoop@node4 be]$ ps -ef | grep doris_be
hadoop 52260 1 0 17:23 pts/2 00:00:08 /opt/module/doris/be/lib/doris_be
6.7.添加Broker
mySql
ALTER SYSTEM ADD BROKER broker8 "192.168.22.38:8000";
ALTER SYSTEM ADD BROKER broker9 "192.168.22.39:8000";
6.8.启动broker
node8和node9上启动
[hadoop@node8 doris]$ cd apache_hdfs_broker/
[hadoop@node8 apache_hdfs_broker]$ bin/start_broker.sh --daemon
查看进程
[hadoop@node8 apache_hdfs_broker]$ jps | grep BrokerBootstrap
37003 BrokerBootstrap
6.9.编写集群启停脚本
群起脚本
#!/bin/bash
# use-method: starrocks.sh start|stop|restart
fe_hosts=(node4 node5 node6 node7)
be_hosts=(node4 node5 node6 node7 node8 node9 node10 node11)
broker_hosts=(node8 kafka1)
case $1 in
"start"){
for i in ${fe_hosts[*]}
do
echo "=================== start $i's fe service ================"
ssh $i "source /etc/profile.d/my_env.sh ;cd /opt/module/starRocks;./fe/bin/start_fe.sh --daemon"
done
for i in ${be_hosts[*]}
do
echo "=================== start $i's be service ================"
ssh $i "source /etc/profile.d/my_env.sh ;cd /opt/module/starRocks;./be/bin/start_be.sh --daemon"
done
for i in ${broker_hosts[*]}
do
echo "=================== start $i's broker service ================"
# ssh $i "source /etc/profile.d/my_env.sh ;cd /opt/module/starRocks;./apache_hdfs_broker/bin/start_broker.sh --daemon"
done
};;
"stop"){
for i in ${broker_hosts[*]}
do
echo "=================== stop $i's broker service ================"
ssh $i "/opt/module/starRocks/apache_hdfs_broker/bin/stop_broker.sh"
done
for i in ${be_hosts[*]}
do
echo "=================== stop $i's be service ================"
ssh $i "/opt/module/starRocks/be/bin/stop_be.sh"
done
for i in ${fe_hosts[*]}
do
echo "=================== stop $i's fe service ================"
ssh $i "/opt/module/starRocks/fe/bin/stop_fe.sh"
done
};;
"status"){
for i in ${fe_hosts[*]}
do
echo "=================== status $i's fe service ================"
ssh $i "jps | grep StarRocksFE"
done
for i in ${be_hosts[*]}
do
echo "=================== status $i's be service ================"
ssh $i "ps -ef | grep starrocks_be | grep -v 'grep'"
done
for i in ${broker_hosts[*]}
do
echo "=================== status $i's broker service ================"
ssh $i "jps | grep BrokerBootstrap | grep -v ‘JAVA_TOOL_OPTIONS’"
done
};;
"restart")
starrocks-cluster.sh stop
sleep 2
starrocks-cluster.sh start
;;
*)
echo "Parameter ERROR!!! starrocks.sh start|stop|restart"
;;
esac
添加脚本执行权限
sudo +x doris-cluster.sh
6.10.web界面
be的web界面
http://node4.bigdata.59wanmei.com:18040/
http://node5.bigdata.59wanmei.com:18040/
http://node6.bigdata.59wanmei.com:18040/
http://node7.bigdata.59wanmei.com:18040/
。。。
fe的web界面
http://node4.bigdata.59wanmei.com:8030/home
http://node5.bigdata.59wanmei.com:8030/home
http://node6.bigdata.59wanmei.com:8030/home
http://node7.bigdata.59wanmei.com:8030/home
四、账号权限管理
1.添加一个账号
mysql -h node4 -P 9030 -uroot -p'000000'
mysql客户端执行
set password=password('000000');
创建数据库
create database test_db;
创建一个普通用户
CREATE USER test IDENTIFIED BY 'test000000';
给当前用户bus_log库的一个只读权限文章来源：https://www.toymoban.com/news/detail-432010.html
GRANT SELECT_PRIV ON test_db.* TO test;
好了，集群搭建完成。文章来源地址https://www.toymoban.com/news/detail-432010.html
到了这里，关于手把手教你快速在生产环境搭建Doris集群附集群启停管理脚本的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！
![1. [手把手教你搭建] 之 在linux上搭建java环境](https://imgs.yssmx.com/Uploads/2024/01/410755-1.png)
