情况
很多时间遇到 "Caused by: java.lang.OutOfMemoryError: Java heap space"
Caused by: java.lang.OutOfMemoryError: Java heap space
可能会有下面错误 "Reducer tasks of hive job fails with Out Of Memory error during shuffle fetcher stage"
解决办法:
- 增加hive.tez.container.size .
- tez.runtime.shuffle.memory.limit.percent 改小一点
可选
- 减少tez.runtime.shuffle.fetch.buffer.percent 默认是0.25 , 尽管为0.15 if needed. (这个值要在区别 0.25 , 0.10)