Apache Hive安装部署-Toy模板网

这篇具有很好参考价值的文章主要介绍了Apache Hive安装部署。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

Apache Hive安装部署

🚃Hive元数据

描述数据的数据，主要描述数据属性信息，用来支持如指示存储位置，历史数据，资源查找，文件记录等功能。
存储在关系型数据库中。如hive内置Derby，或第三方MySql
Metastore元数据服务。作用管理元数据，对外暴露服务。

Apache Hive安装部署

metastore服务配置三种模式：

内嵌模式
本地模式
远程模式

区分配置方式方法：

metastore是否需要单独配置，单独启动
metadaa存储在内置derby中，还是第三方RDBMS

	内嵌	本地	远程
meta单独配置，启动	否	否	是
metadata存储介质	Derby	Mysql	Mysql

🚞部署实战

安装前准备：

Hive基于Hadoop，保证服务器基础环境正常
- 集群时间同步，防火墙关闭，Host映射，免密登录，JDK安装
Hadoop集群健康可用
- 等待HDFS安全模式关闭后再运行Hive

🏍️安装MySQL

MySQL只需一台机器安装并需要授权远程访问

卸载CentOS7自带mariadb

[root@node1 ~]# rpm -qa|grep mariadb
mariadb-libs-5.5.60-1.el7_5.x86_64
[root@node1 ~]# rpm -e mariadb-libs-5.5.60-1.el7_5.x86_64 --nodeps

[root@node1 ~]# rpm -qa|grep mariadb

安装mysql

[root@node1 ~]# mkdir /export/software/mysql
#上传mysql-5.7.29-1.el7.x86_64.rpm-bundle.tar 到上述文件夹下  解压
[root@node1 ~]# ls /export/software/mysql/
mysql-5.7.29-1.el7.x86_64.rpm-bundle.tar
[root@node1 ~]# cd /export/software/mysql/
[root@node1 mysql]# tar xvf mysql-5.7.29-1.el7.x86_64.rpm-bundle.tar
mysql-community-embedded-devel-5.7.29-1.el7.x86_64.rpm
mysql-community-test-5.7.29-1.el7.x86_64.rpm
mysql-community-embedded-5.7.29-1.el7.x86_64.rpm
mysql-community-embedded-compat-5.7.29-1.el7.x86_64.rpm
mysql-community-libs-5.7.29-1.el7.x86_64.rpm
mysql-community-client-5.7.29-1.el7.x86_64.rpm
mysql-community-server-5.7.29-1.el7.x86_64.rpm
mysql-community-devel-5.7.29-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.29-1.el7.x86_64.rpm
mysql-community-common-5.7.29-1.el7.x86_64.rpm
#安装依赖
[root@node1 mysql]# yum -y install libaio
[root@node1 mysql]# yum install -y net-tools

#开始安装
[root@node1 mysql]# rpm -ivh mysql-community-common-5.7.29-1.el7.x86_64.rpm mysql-community-libs-5.7.29-1.el7.x86_64.rpm mysql-community-client-5.7.29-1.el7.x86_64.rpm mysql-community-server-5.7.29-1.el7.x86_64.rpm

mysql初始化设置

#初始化
mysqld --initialize

#更改所属组
chown mysql:mysql /var/lib/mysql -R

#启动mysql
systemctl start mysqld.service

#查看生成的临时root密码
cat  /var/log/mysqld.log
#临时密码大概在这个位置
...
2023-06-13T12:23:52.391426Z 1 [Note] A temporary password is generated for root@localhost:
...

修改root密码授权远程访问设置开机自启动


mysql -u root -p#进入之后输入日志中生成的密码
#更新密码
mysql>  alter user user() identified by "hadoop";
Query OK, 0 rows affected (0.00 sec)

#权限赋予
mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'hadoop' WITH GRANT OPTION;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye


#mysql的启动和关闭 状态查看 
systemctl stop mysqld
systemctl status mysqld
systemctl start mysqld

#设置开机自启
systemctl enable  mysqld

如果需要卸载请执行

#关闭mysql服务
systemctl stop mysqld.service

#查找安装mysql的rpm包
[root@node1 ~]# rpm -qa | grep -i mysql      
mysql-community-libs-5.7.29-1.el7.x86_64
mysql-community-common-5.7.29-1.el7.x86_64
mysql-community-client-5.7.29-1.el7.x86_64
mysql-community-server-5.7.29-1.el7.x86_64

#卸载
[root@node1 ~]# yum remove mysql-community-libs-5.7.29-1.el7.x86_64 mysql-community-common-5.7.29-1.el7.x86_64 mysql-community-client-5.7.29-1.el7.x86_64 mysql-community-server-5.7.29-1.el7.x86_64

#查看是否卸载干净
rpm -qa | grep -i mysql

#查找mysql相关目录 删除
[root@node1 ~]# find / -name mysql
/var/lib/mysql
/var/lib/mysql/mysql
/usr/share/mysql

 rm -rf /var/lib/mysql
 rm -rf /var/lib/mysql/mysql
 rm -rf /usr/share/mysql

#删除默认配置 日志
rm -rf /etc/my.cnf 
rm -rf /var/log/mysqld.log

🛵安装Hive

上传Hive安装包（node1安装即可）


[root@node1 server]# pwd
/export/server

[root@node1 server]# ls
apache-hive-3.1.2-bin.tar.gz  hadoop-3.3.0  jdk1.8.0_241
#解压
tar zxvf apache-hive-3.1.2-bin.tar.gz

解决Hive和Hadoop之间guava版本差异

cd /export/server/apache-hive-3.1.2-bin/
rm -rf lib/guava-19.0.jar
cp /export/server/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar ./lib/

修改配置文件

hive.env.sh

cd /export/server/apache-hive-3.1.2-bin/conf
mv hive-env.sh.template hive-env.sh

vim hive-env.sh
export HADOOP_HOME=/export/server/hadoop-3.3.0
export HIVE_CONF_DIR=/export/server/apache-hive-3.1.2-bin/conf
export HIVE_AUX_JARS_PATH=/export/server/apache-hive-3.1.2-bin/lib

hive-site.xml

vim hive-site.xml

<configuration>
<!-- 存储元数据mysql相关配置 -->
<property>
	<name>javax.jdo.option.ConnectionURL</name>
	<value>jdbc:mysql://node1:3306/hive3?createDatabaseIfNotExist=true&amp;useSSL=false&amp;useUnicode=true&amp;characterEncoding=UTF-8</value>
</property>

<property>
	<name>javax.jdo.option.ConnectionDriverName</name>
	<value>com.mysql.jdbc.Driver</value>
</property>

<property>
	<name>javax.jdo.option.ConnectionUserName</name>
	<value>root</value>
</property>

<property>
	<name>javax.jdo.option.ConnectionPassword</name>
	<value>hadoop</value>
</property>

<!-- H2S运行绑定host -->
<property>
    <name>hive.server2.thrift.bind.host</name>
    <value>node1</value>
</property>

<!-- 远程模式部署metastore metastore地址 -->
<property>
    <name>hive.metastore.uris</name>
    <value>thrift://node1:9083</value>
</property>

<!-- 关闭元数据存储授权  --> 
<property>
    <name>hive.metastore.event.db.notification.api.auth</name>
    <value>false</value>
</property>
</configuration>

上传mysql jdbc驱动到hive安装包lib下


[root@node1 conf]# cd /export/server/apache-hive-3.1.2-bin/lib/
[root@node1 lib]# ls -aux | grep mysql-connector
mysql-connector-java-5.1.32.jar

初始化元数据

cd /export/server/apache-hive-3.1.2-bin/

bin/schematool -initSchema -dbType mysql -verbos

hdfs创建hive存储目录（存在不操作）

hadoop fs -mkdir /tmp
hadoop fs -mkdir -p /user/hive/warehouse
hadoop fs -chmod g+w /tmp
hadoop fs -chmod g+w /user/hive/warehouse

🛹启动

启动metastore服务

#前台启动  关闭ctrl+c
/export/server/apache-hive-3.1.2-bin/bin/hive --service metastore

#前台启动开启debug日志
/export/server/apache-hive-3.1.2-bin/bin/hive --service metastore --hiveconf hive.root.logger=DEBUG,console  

#后台启动 进程挂起  关闭使用jps+ kill -9
nohup /export/server/apache-hive-3.1.2-bin/bin/hive --service metastore &

🚂客户端启用

Hive自带客户端：

bin/hive
bin/beeline

在安装hive服务器上，首先启动metastore，然后启动hieserver2

  nohup /export/server/apache-hive-3.1.2-bin/bin/hive --service hiveserver2 &

beeline客户端连接
- 拷贝node1安装包到beeline客户端机器（node3）
```
scp -r /export/server/apache-hive-3.1.2-bin/ node3:/export/server/
```
- node3使用bin/hive连接服务
- node3使用bin/beeline连接服务