// 21/04/23 17:47:51 WARN AsyncProcess: #43, table=ns:table_test, attempt=1/1 failed=108ops, last exception: org.apache.hadoop.hbase.RegionTooBusyException: org.apache.hadoop.hbase.RegionTooBu
//syException: Above memstore limit, regionName=ns:table_test,08888888,1619170961003.a4abed7f398cd9b85062c800e95c6dac., server=ns:table_test, memstoreSize=542273360, block
//ingMemStoreSize=536870912
原因见其他博客说明,服务端不好更改,客户端的解决方式说到底就是加大和超时等待时间和重试次数,和提交间隔(文件分批个数)
原始参数
conf.addResource(new Path("/etc/hbase/conf/core-site.xml"))
conf.addResource(new Path("/etc/hbase/conf/hbase-site.xml"))
添加参数-解决
conf.set("hbase.hregion.memstore.block.multiplier","100")
conf.set("hbase.hstore.blockingWaitTime","600s")
conf.set("hbase.ipc.client.call.purge.timeout","300000")
conf.set("hbase.rpc.timeout","300000")
conf.set("hbase.hstore.compaction.max","1000")
conf.set("hbase.client.retries.number","30")
conf.set("hbase.client.pause","12000")
conf.set("zookeeper.recovery.retry","12")
该错误常见于大量的hbase写入场景,无法只通过客户端的调整避免,也受hbase状态影响
因此如果配置参数依然无法解决,看任务类型,可以把任务拆成不同的小粒度任务,分别执行