0 背景
本文基于Ambari集群搭建最新版本的海豚调度dolphinscheduler-3.1.3版本，后续会尝试整合到Ambari中。
1 安装准备
安装dolphinscheduler需要在环境中安装如下依赖
① JDK8
下载JDK (1.8+)，安装并配置 JAVA_HOME 环境变量，并将其下的 bin 目录追加到 PATH 环境变量中
② 数据库：PostgreSQL (8.2.15+) 或者 MySQL (5.7+)，两者任选其一即可，如 MySQL 则需要 JDBC Driver 8.0.16
③ 注册中心：ZooKeeper (3.4.6+)
2 安装集群
| 节点 | ip | 服务角色 |
|---|---|---|
| Ambari-1 | 192.168.x.31 | ds-master ds-api ds-alert |
| Ambari-2 | 192.168.x.32 | ds-master |
| Ambari-3 | 192.168.x.33 | ds-worker |
| Ambari-4 | 192.168.x.34 | ds-worker |
| Ambari-5 | 192.168.x.35 | ds-worker |
2.1 配置dolphinscheduler用户
新建dolphinscheduler用户，并且一定要配置 sudo 免密.
# 创建用户需使用 root 登录
useradd dolphinscheduler
# 添加密码
echo "dolphinscheduler" | passwd --stdin dolphinscheduler
# 配置 sudo 免密
sed -i '$adolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL' /etc/sudoers
sed -i 's/Defaults requirett/#Defaults requirett/g' /etc/sudoers
# 修改目录权限，使得部署用户对二进制包解压后的 apache-dolphinscheduler-*-bin 目录有操作权限
chown -R dolphinscheduler:dolphinscheduler apache-dolphinscheduler-*-bin
配置免密
su dolphinscheduler
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
集群之间完成免密
ssh-copy-id -i ~/.ssh/id_rsa.pub dolphinscheduler@主机
2.2 配置mysql
DolphinScheduler 元数据存储在关系型数据库中，故需创建相应的数据库和用户。注：这里根据Mysql安装版本的不同选择不同的执行方式。
mysql5.6/5.7
mysql -uroot -p
mysql> CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
# 修改 {user} 和 {password} 为你希望的用户名和密码
mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%' IDENTIFIED BY '{password}';
mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost' IDENTIFIED BY '{password}';
mysql> flush privileges;
mysql8:
创建数据库并指定编码格式
mysql> CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
开启权限
# 修改 {user} 和 {password} 为你希望的用户名和密码
mysql> CREATE USER '{user}'@'%' IDENTIFIED BY '{password}';
mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%';
mysql> CREATE USER '{user}'@'localhost' IDENTIFIED BY '{password}';
mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost';
mysql> FLUSH PRIVILEGES;
执行过程中可能出现如下报错：
ERROR 1819 (HY000): Your password does not satisfy the current policy requirements
执行如下命令查看mysql密码要求规则
show variables like 'validate_password%';
修改校验登记为LOW,重新设置即可
set global validate_password.policy=LOW;
2.3 启动zk
2.4 修改ds配置文件
2.4.1 mysql的JDBC驱动放入ds的lib目录下
包括 api-server/libs 和 alert-server/libs 和 master-server/libs 和 worker-server/libs
cp ../mysql-connector-java-8.0.16.jar ./lib
cp ../mysql-connector-java-8.0.16.jar ./api-server/libs/
cp ../mysql-connector-java-8.0.16.jar ./alert-server/libs/
cp ../mysql-connector-java-8.0.16.jar ./master-server/libs/
cp ../mysql-connector-java-8.0.16.jar ./worker-server/libs/
2.4. 2 修改bin/env目录下的install_env.sh配置文件
我们仅需修改INSTALL MACHINE，DolphinScheduler ENV、Database、Registry Server
2.4.3 修改dolphinscheduler_env.sh 文件
bin/env/dolphinscheduler_env.sh 中配置JAVA_HOME、数据库、注册中心等相关配置
2.6 初始化数据库
bash tools/bin/upgrade-schema.sh
2.7 启动ds
sh install.sh
报错如下：
9:32:47.555 [main] INFO org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
09:32:47.561 [main] INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 1048575 Bytes
09:32:47.578 [main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
ZooKeeper -server host:port -client-configuration properties-file cmd args
addWatch [-m mode] path # optional mode is one of [PERSISTENT, PERSISTENT_RECURSIVE] - default is PERSISTENT_RECURSIVE
addauth scheme auth
close
config [-c] [-w] [-s]
connect host:port
create [-s] [-e] [-c] [-t ttl] path [data] [acl]
delete [-v version] path
deleteall path [-b batch size]
delquota [-n|-b|-N|-B] path
get [-s] [-w] path
getAcl [-s] path
getAllChildrenNumber path
getEphemerals path
history
listquota path
ls [-s] [-w] [-R] path
printwatches on|off
quit
reconfig [-s] [-v version] [[-file path] | [-members serverID=host:port1:port2;port3[,…]]] | [-add serverId=host:port1:port2;port3[,…]] [-remove serverId[,…]*]
redo cmdno
removewatches path [-c|-d|-a] [-l]
set [-s] [-v version] path data
setAcl [-s] [-v version] [-R] path acl
setquota -n|-b|-N|-B val path
stat [-w] path
sync path
version
whoami
Command not found: Command not found rmr
09:32:47.584 [main] ERROR org.apache.zookeeper.util.ServiceUtils - Exiting JVM with code 127
查看具体生成日志发现如下报错：
Caused by: org.apache.dolphinscheduler.registry.api.RegistryException: Failed to put registry key: /nodes/master
239 at org.apache.dolphinscheduler.plugin.registry.zookeeper.ZookeeperRegistry.put(ZookeeperRegistry.java:186)
240 at org.apache.dolphinscheduler.service.registry.RegistryClient.initNodes(RegistryClient.java:247)
241 at org.apache.dolphinscheduler.service.registry.RegistryClient.afterConstruct(RegistryClient.java:77)
242 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
243 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
244 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
245 at java.lang.reflect.Method.invoke(Method.java:498)
246 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor L i f e c y c l e E l e m e n t . i n v o k e ( I n i t D e s t r o y A n n o t a t i o n B e a n P o s t P r o c e s s o r . j a v a : 389 ) 247 a t o r g . s p r i n g f r a m e w o r k . b e a n s . f a c t o r y . a n n o t a t i o n . I n i t D e s t r o y A n n o t a t i o n B e a n P o s t P r o c e s s o r LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) 247 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389)247atorg.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessorLifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333)
248 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157)
249 … 46 common frames omitted
250 Caused by: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /dolphinscheduler/nodes/master
251 at org.apache.zookeeper.KeeperException.create(KeeperException.java:106)
252 at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
253 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1450)
254 at org.apache.curator.framework.imps.CreateBuilderImpl$18.call(CreateBuilderImpl.java:1216)
255 at org.apache.curator.framework.imps.CreateBuilderImpl$18.call(CreateBuilderImpl.java:1193)
256 at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:93)
257 at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:1190)
258 at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:605)
259 at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:595)
260 at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:48)
261 at org.apache.dolphinscheduler.plugin.registry.zookeeper.ZookeeperRegistry.put(ZookeeperRegistry.java:184)
这里最新版本的ds与zk版本不兼容，最新版本的ds需要最新版本的zk，即3.8版本，当前环境安装的是3.4.6。通过查看ds依赖的zk的jarb包也能验证这一点
解决思路：下载3.8版本的zk并安装
下载地址:
https://www.apache.org/dyn/closer.lua/zookeeper/zookeeper-3.8.0/apache-zookeeper-3.8.0-bin.tar.gz
安装方法可以看这篇文章《zookeeper快速入门》
安装完成并启动zk，然后再重新启动ds成功。
访问master节点url连接：
http://<your_ip>:12345/dolphinscheduler/ui/login
默认账户密码：admin/dolphinscheduler123
3 [附加]ds源码修改与打包
当前ds-3.1.3版本数据源还不支持trino，可以修改源码，增加对trino数据源的支持，修改完成后使用如下命令重新编译并完成打包。
./mvnw clean install -Prelease -DskipTests=true
这里的-DskipTests=true指跳过测试，如果不加该参数，整个耗时会特别长。编译完成后可看到如下成功页面。
然后进入你的maven仓库即可发现已经打包的ds-3.1.3安装包文章来源：https://www.toymoban.com/news/detail-414393.html
cd /root/.m2/repository/org/apache/dolphinscheduler/dolphinscheduler-dist/3.1.3
ll -h
文章来源地址https://www.toymoban.com/news/detail-414393.html
到了这里，关于最新版海豚调度dolphinscheduler-3.1.3安装部署详细教程的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！
