1. Toy模板网首页
  2. > Toy博客

mac上搭建hbase伪集群

9月前 作者：寂夜了无痕 分类：Toy博客 阅读(38) 违法举报

这篇具有很好参考价值的文章主要介绍了mac上搭建hbase伪集群。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

1. 前言

之前我们已经搭建过了 hbase单点环境，(单机版搭建参见： https://blog.csdn.net/a15835774652/article/details/135569456)
但是 为了模拟一把集群环境 我们还是尝试搭建一个伪集群版

2. 环境准备

  • jdk环境 1.8+
  • hdfs （hadoop环境 可选） 搭建参考 https://blog.csdn.net/a15835774652/article/details/135572760

3. 配置

搭建过 hadoop 才发现 hbase环境配置如此之少 可能hadoop东西比较多的缘故

  • 配置 jdk 修改 conf/hbase-env.sh
    # 修改为自己的jdk
export JAVA_HOME=/usr/local/develop/java/zulu-jdk17.0.7
#可选 jdk1.8 以上需要配置
export HBASE_OPTS="${HBASE_SHELL_OPTS} --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED"
  • 修改 conf/hbase-site.xml 新增属性配置
<property>
	<name>hbase.cluster.distributed</name>
	<value>true</value>
</property>
<!--配置hbase存储目录 这是用hdfs方式 -->
<property>
	<name>hbase.rootdir</name>
	<value>hdfs://localhost:8020/hbase</value>
</property>
<!--另外还可以选择本地目录方式 二选一即可 -->
<!--   <property>
<name>hbase.rootdir</name>
<value>file:///usr/local/develop/hbase</value>
</property> -->
  • 移除 hbase.tmp.dir 和 hbase.unsafe.stream.capability.enforce 配置

4. 启动

是的，你没有听错 配置到这就结束了。开始进入启动环节
直接输入命令进行启动

bin/start-hbase.sh

如果配置的没问题的话 使用jps 查看 可以看到 HMaster and HRegionServer 进程在运行

mac大家habase,大数据,macos,hbase

5. 检查hdfs中的 hbase目录

./bin/hadoop fs -ls /hbase

mac大家habase,大数据,macos,hbase

6. Start backup HBase Master

# 开启 3个 backup servers using ports 16002/16012, 16003/16013, and 16005/16015.  2 3 5每个参数是 相对默认端口16000的偏移量
bin/local-master-backup.sh start 2 3 5

然后我们可以访问本地的 hbase控制台试试效果
默认的是 localhost:16010
backup server localhost:16012 localhost:16013 localhost:16015

mac大家habase,大数据,macos,hbase
mac大家habase,大数据,macos,hbase
mac大家habase,大数据,macos,hbase
mac大家habase,大数据,macos,hbase

这样我们访问就有4个节点了 一个主节点 3个backup 节点 在通过一些组件例如keep alive 等 就实现了 多节点 高可用

7. 停止 backup HBase Master

要kill备用master而不杀死整个集群，需要找到其进程D（PID）。PID存储在文件中，名称为 /tmp/hbase-user-x-master.pid。文件的唯一内容是PID。您可以使用Kill -9命令杀死该PID。以下命令将使用端口偏移1杀死master，但使群集运行：
查看tmp目录下的 hbase backup master pid文件
mac大家habase,大数据,macos,hbase
选择对应的文件 并执行如下命令 即可kill 备用的 master

#注意hbase-leon-2-master.pid n替换为自己的pid文件
cat /tmp/hbase-leon-2-master.pid |xargs kill -9

8. 开启 备用的RegionServers

设置环境变量 /conf/hbase-env.sh

export HBASE_RS_BASE_PORT=16020
export HBASE_RS_INFO_BASE_PORT=16030

Hnegionserver按照HMaster指示管理其仓库中的数据。通常，一个hregionserver在集群中的每个节点运行。在同一系统上运行多个Hegionservers在伪分布模式下测试很有用。local-regionservers.sh命令允许您运行多个RegionServer。它以与local-master-backup.sh命令相似的方式工作，因为您提供的每个参数代表一个实例的端口偏移。每个区域服务器需要两个端口，默认端口为16020和16030。由于HBASE版本1.1.0，HMASTER不使用区域服务器端口，因此将10个端口（16020至16029和16029和16030至16039）用于ZEMIONIONSERVERS。为了支持其他区域服务器，设置环境变量HBASE_RS_BASE_PORT和HBASE_RS_INFO_BASE_PORT在运行脚本local-regionservers.sh之前。例如对于基本端口的值16200和16300，服务器上可以支持99个其他区域服务器。以下命令启动了另外三个区域服务器，从16022/16032（基本端口16020/16030加2）运行。

开启3个 RegionServers

# regionServers 默认端口为16020和16030
# 参数2 3 5 含义是相对默认端口的偏移量
./bin/local-regionservers.sh start 2 3 5

9. 停止 localRegionServers

./bin/local-regionservers.sh stop 3

以上就完成了 hbase的伪集群搭建 其中 master的 backup比较好配置 另外 多个regionServers 配置稍微复杂。

10. 问题记录

  • 问题一 ip绑定问题
    hbase 启动服务默认绑定ip到 自己电脑的IP地址上 而没有绑定到localhost 但是在hbase的shell工具中 默认连接的地址是localhost 然后报错提示 连接不到localhost的16000端口 报错如下
list 'test'
TABLE
2024-01-27 16:16:12,963 WARN  [RPCClient-NioEventLoopGroup-1-1] ipc.NettyRpcConnection (NettyRpcConnection.java:fail(318)) - Exception encountered while connecting to the server localhost:16000
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:16000
Caused by: java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
	at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337)
	at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
2024-01-27 16:16:12,978 INFO  [Registry-endpoints-refresh-end-points] client.RegistryEndpointsRefresher (RegistryEndpointsRefresher.java:mainLoop(78)) - Registry end points refresher loop exited.

ERROR: Failed contacting masters after 1 attempts.
Exceptions:
java.net.ConnectException: Call to address=localhost:16000 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:16000

For usage try 'help "list"'

Took 0.8411 seconds
hbase:002:0> exit

但是本机查看端口的占用 提示绑定到 自己的电脑的ip:16000上
但是当修改 hbase绑定ip到0.0.0.0时，又提示如下错误

hbase:002:0> create 'test', 'cf' 
2024-01-27 16:44:46,414 WARN [RPCClient-NioEventLoopGroup-1-2] client.AsyncRpcRetryingCaller (AsyncRpcRetryingCaller.java:onError(184)) - Call to master failed, tries = 6, maxAttempts = 7, timeout = 1200000 ms, time elapsed = 2217 ms org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet 
at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:3235) 
at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:3241) at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2380) 
at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:778) 
at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) 
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437) 
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) 
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) 
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)

Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ipc.ServerNotRunningYetException): org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet 
at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:3235) 
at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:3241) 
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2380) 
at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:778) 
at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) 
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437) 
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) 
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) 
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)  
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:388)

难道说 hbase不能绑定到0.0.0.0上
折中解决先把IP绑定到 localhost 上 或者127.0.0.1上 然后使用hbase shell测试

  • 问题2 hbase使用本地目录存储时 启动报错 如下错误 org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush
2024-01-27 14:59:46,330 WARN  [master/localhost:16000:becomeActiveMaster] wal.AbstractProtobufLogWriter (AbstractProtobufLogWriter.java:init(195)) - Init output failed, path=file:/usr/local/develop/hbase/MasterData/WALs/localhost,16000,1706338777398/localhost%2C16000%2C1706338777398.1706338786302
org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush
	at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:72)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:183)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:167)
	at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:114)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:749)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:755)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:127)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriterInternal(AbstractFSWAL.java:953)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.lambda$rollWriter$8(AbstractFSWAL.java:985)
	at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:985)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:595)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:536)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:163)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:65)
	at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:391)
	at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:211)
	at org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:271)
	at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:432)
	at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:135)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:981)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2478)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:602)
	at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155)
	at java.base/java.lang.Thread.run(Thread.java:833)
2024-01-27 14:59:46,330 ERROR [master/localhost:16000:becomeActiveMaster] wal.AsyncFSWALProvider (AsyncFSWALProvider.java:createAsyncWriter(118)) - The RegionServer async write ahead log provider relies on the ability to call hflush for proper operation during component failures, but the current FileSystem does not support doing so. Please check the config value of 'hbase.wal.dir' and ensure it points to a FileSystem mount that has suitable capabilities for output streams.
2024-01-27 14:59:46,335 ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active master
java.io.IOException: java.lang.NullPointerException
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1056)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1083)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:65)
	at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:391)
	at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:211)
	at org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:271)
	at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:432)
	at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:135)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:981)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2478)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:602)
	at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NullPointerException
	at java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
	at java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:778)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:835)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:1028)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:1023)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	... 1 more
2024-01-27 14:59:46,336 ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster (MarkerIgnoringBase.java:error(159)) - ***** ABORTING master localhost,16000,1706338777398: Unhandled exception. Starting shutdown. *****
java.io.IOException: java.lang.NullPointerException
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1056)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1083)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:65)
	at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:391)
	at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:211)
	at org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:271)
	at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:432)
	at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:135)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:981)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2478)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:602)
	at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NullPointerException
	at java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
	at java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:778)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:835)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:1028)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:1023)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	... 1 more
2024-01-27 14:59:46,337 INFO  [master/localhost:16000:becomeActiveMaster] master.HMaster (HMaster.java:stop(3222)) - ***** STOPPING master 'localhost,16000,1706338777398' *****
2024-01-27 14:59:46,337 INFO  [master/localhost:16000:becomeActiveMaster] master.HMaster (HMaster.java:stop(3224)) - STOPPED: Stopped by master/localhost:16000:becomeActiveMaster
2024-01-27 14:59:46,338 INFO  [master/localhost:16000] hbase.HBaseServerBase (HBaseServerBase.java:stopInfoServer(465)) - Stop info server
2024-01-27 14:59:46,359 INFO  [master/localhost:16000] handler.ContextHandler (ContextHandler.java:doStop(1159)) - Stopped o.a.h.t.o.e.j.w.WebAppContext@5a3cf878{master,/,null,STOPPED}{file:/usr/local/develop/hbase-3.0.0-alpha-4/hbase-webapps/master}
2024-01-27 14:59:46,365 INFO  [master/localhost:16000] server.AbstractConnector (AbstractConnector.java:doStop(383)) - Stopped ServerConnector@43d9f1a2{HTTP/1.1, (http/1.1)}{0.0.0.0:16010}
2024-01-27 14:59:46,366 INFO  [master/localhost:16000] server.session (HouseKeeper.java:stopScavenging(149)) - node0 Stopped scavenging
2024-01-27 14:59:46,366 INFO  [master/localhost:16000] handler.ContextHandler (ContextHandler.java:doStop(1159)) - Stopped o.a.h.t.o.e.j.s.ServletContextHandler@181b8c4b{static,/static,file:///usr/local/develop/hbase-3.0.0-alpha-4/hbase-webapps/static/,STOPPED}
2024-01-27 14:59:46,366 INFO  [master/localhost:16000] handler.ContextHandler (ContextHandler.java:doStop(1159)) - Stopped o.a.h.t.o.e.j.s.ServletContextHandler@4c1bdcc2{logs,/logs,file:///usr/local/develop/hbase-3.0.0-alpha-4/logs/,STOPPED}
2024-01-27 14:59:46,371 INFO  [master/localhost:16000] hbase.HBaseServerBase (HBaseServerBase.java:stopChoreService(435)) - Shutdown chores and chore service
2024-01-27 14:59:46,371 INFO  [master/localhost:16000] hbase.ChoreService (ChoreService.java:shutdown(370)) - Chore service for: master/localhost:16000 had [] on shutdown
2024-01-27 14:59:46,371 INFO  [master/localhost:16000] hbase.HBaseServerBase (HBaseServerBase.java:stopExecutorService(445)) - Shutdown executor service
2024-01-27 14:59:46,373 WARN  [master/localhost:16000] master.ActiveMasterManager (ActiveMasterManager.java:stop(344)) - Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null
2024-01-27 14:59:46,373 INFO  [master/localhost:16000] ipc.NettyRpcServer (NettyRpcServer.java:stop(202)) - Stopping server on /127.0.0.1:16000
2024-01-27 14:59:46,380 INFO  [master/localhost:16000] hbase.HBaseServerBase (HBaseServerBase.java:closeZooKeeper(476)) - Close zookeeper
2024-01-27 14:59:46,488 INFO  [master/localhost:16000] zookeeper.ZooKeeper (ZooKeeper.java:close(1422)) - Session: 0x1002ae462b30006 closed
2024-01-27 14:59:46,488 INFO  [main-EventThread] zookeeper.ClientCnxn (ClientCnxn.java:run(524)) - EventThread shut down for session: 0x1002ae462b30006
2024-01-27 14:59:46,489 INFO  [master/localhost:16000] hbase.HBaseServerBase (HBaseServerBase.java:closeTableDescriptors(483)) - Close table descriptors
2024-01-27 14:59:46,489 ERROR [main] master.HMasterCommandLine (HMasterCommandLine.java:startMaster(260)) - Master exiting
java.lang.RuntimeException: HMaster Aborted
	at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:255)
	at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:140)
	at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3363)
2024-01-27 14:59:46,492 INFO  [shutdown-hook-0] regionserver.ShutdownHook (ShutdownHook.java:run(105)) - Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@181e72d3
2024-01-27 14:59:46,493 INFO  [shutdown-hook-0] regionserver.ShutdownHook (ShutdownHook.java:run(114)) - Starting fs shutdown hook thread.
2024-01-27 14:59:46,493 INFO  [shutdown-hook-0] regionserver.ShutdownHook (ShutdownHook.java:run(129)) - Shutdown hook finished

这个问题导致hbase无法完成启动，只能被迫换了使用hdfs
hbase 默认不支持使用本地文件系统启动 需要hbase-site.xml 添加如下配置

 <property>
   <name>hbase.unsafe.stream.capability.enforce</name>
   <value>false</value>
 </property>

good day ！！！文章来源地址https://www.toymoban.com/news/detail-836318.html

到了这里，关于mac上搭建hbase伪集群的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处： 如若内容造成侵权/违法违规/事实不符，请点击违法举报进行投诉反馈，一经查实，立即删除！

分享到：
领支付宝红包 赞助服务器费用

相关文章

  • Hbase集群的搭建

    Hbase集群的搭建

    hadoop、zookeeper集群已正常安装 hadoop集群搭建 zookeeper集群搭建 hadoop-001 hadoop-002 hadoop-003 Hbase Hbase Hbase 解压安装包到指定目录/export/module下 修改名字 修改hbase/conf/hbase-env.sh export HBASE_MANAGES_ZK=false 修改hbase/conf/hbase-site.xml configuration   property     namehbase.rootdir/name     valuehdfs://

    2024年02月04日
    浏览(32)
  • Hbase 集群搭建

    Hbase 集群搭建

    Apache HBase – Apache HBase Downloads 选择版本 https://dlcdn.apache.org/hbase/2.4.17/hbase-2.4.17-bin.tar.gz Hbase是一个分布式系统 其中有一个管理角色： HMaster (一般2台  ,一台active、一台backup) 其它的数据节点角色： HRegionServer (很多台，看数据容量) 客户端要查数据， [root@node1 ~]# tar -zxvf hbase-

    2024年02月19日
    浏览(33)

  • 搭建HBase分布式集群

    0. Prerequisite There are 3 VMs - hadoop3/hadoop4/hadoop5 for fully-distributed HBase cluster, the setup plan looks like: hadoop3 hadoop4 hadoop5 Hadoop hdfs NameNode:8020 DateNode:50010 JobHistoryServer:19888 DataNode:50010 SecondaryNameNode:50090 DateNode:50010 Hadoop yarn NodeManger:8040 ResourceMananger:8088 NodeManger:8040 NodeManger:8040 Zookeeper Quor

    2024年02月08日
    浏览(34)
  • HBase集群环境搭建与测试

    HBase集群环境搭建与测试

    🥇🥇【大数据学习记录篇】-持续更新中~🥇🥇 个人主页：beixi@ 本文章收录于专栏（点击传送）：【大数据学习】 💓💓持续更新中，感谢各位前辈朋友们支持学习~💓💓 HBase集群由多个RegionServer节点和一个或多个HMaster节点组成。HMaster节点负责管理元数据（例如表的位置、

    2024年02月10日
    浏览(55)

  • 实操Hadoop大数据高可用集群搭建（hadoop3.1.3+zookeeper3.5.7+hbase3.1.3+kafka2.12）

    前言 纯实操，无理论，本文是给公司搭建测试环境时记录的，已经按照这一套搭了四五遍大数据集群了，目前使用还未发现问题。 有问题麻烦指出，万分感谢！ PS：Centos7.9、Rocky9.1可用 集群配置 ip hostname 系统 CPU 内存 系统盘 数据盘 备注 192.168.22.221 hadoop1 Centos7.9 4 16 250G 19

    2024年02月03日
    浏览(38)

  • 搭建HBase伪分布式集群

    1. Install zookeeper 1) download zookeeper from https://zookeeper.apache.org/releases.html#download 2) extract binary 3) configurate zoo.cfg 4) start zookeeper 5) try zookeeper Note: use following command to stop zookeeper 2. Install Hadoop 1) download Hadoop from https://hadoop.apache.org/releases.html 2) extract binary 3) configurate 4) format the filesyst

    2024年02月10日
    浏览(45)
  • ZooKeeper+HBase分布式集群环境搭建

    ZooKeeper+HBase分布式集群环境搭建

    安装版本：hadoop-2.10.1、zookeeper-3.4.12、hbase-2.3.1 一、zookeeper 集群搭建与配置 1. 下载zookeeper安装包 2. 解压移动zookeeper 3. 修改配置文件（创建文件夹） 4. 进入conf/ 5. 修改zoo.cfg文件 6. 进入/usr/local/zookeeper-3.4.12/zkdatas/这个路径下创建一个文件，文件名为myid ,文件内容为1 7. 拷贝到

    2024年02月08日
    浏览(44)
  • HBase详解（对hbase集群搭建、读写流程、hbase的javaApi等细致入微的讲解与保姆级的图解）

    HBase详解（对hbase集群搭建、读写流程、hbase的javaApi等细致入微的讲解与保姆级的图解）

    我本想用MySQL来与HBase作比较，但发现他们两者毫无可比性，因为两者运用领域不同，各自有各自的优点，就好比爬山穿登山鞋，潜水穿脚蹼一般。 一门技术的兴起，一个优秀的开源项目的存在肯定是有它所存在的意义，正如大数据一样，正是因为随着时间的发展，随着技术

    2024年02月11日
    浏览(39)
  • 云计算集群搭建记录[Hadoop|Zookeeper|Hbase|Spark | Docker |OpenStack]更新索引 |动态更新

    云计算集群搭建记录[Hadoop|Zookeeper|Hbase|Spark | Docker |OpenStack]更新索引 |动态更新

    为了能够更好的查看所更新的文章，讲该博文设为索引 为了解决在编辑文件等操作的过程中的权限问题，博主一律 默认采用 root 账户登录 对于初次安装的用户可以采用如下命令行： 另外推荐一款终端工具：Tabby，既能够连接自己的服务器，也能够连接自己本地的虚拟机，还

    2023年04月13日
    浏览(55)
  • 大数据NoSQL数据库HBase集群部署

    大数据NoSQL数据库HBase集群部署

    目录 1.  简介 2.  安装 1. HBase依赖Zookeeper、JDK、Hadoop（HDFS），请确保已经完成前面 2. 【node1执行】下载HBase安装包 3. 【node1执行】，修改配置文件，修改conf/hbase-env.sh文件 4. 【node1执行】，修改配置文件，修改conf/hbase-site.xml文件 5. 【node1执行】，修改配置文件，修改conf/regi

    2024年02月08日
    浏览(50)

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

博客赞助

微信扫一扫打赏

请作者喝杯咖啡吧~博客赞助

支付宝扫一扫领取红包，优惠每天领

二维码1

领取红包

二维码2

领红包