NoSuchMethodError: ... addOnCompleteCallback

问题描述:

  使用ES 2.3.1和Spark streaming 2.1时,出现以上报错信息。

原因:

addOnCompleteCallback方法在spark2.0中移除了

The addOnCompleteCallback method was removed from the TaskContext in Spark 2.0. Spark 2.0 is not binary compatible with previous releases. Because of this we only support Spark 2.0 in es-hadoop version 5.0 and later. Please upgrade to fix the issue.

解决方法:

  

<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-spark_2.11</artifactId>
<version>2.4.4</version>
</dependency> 替换为
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-hadoop</artifactId>
<version>5.1.2</version>
</dependency>

参考:https://github.com/elastic/elasticsearch-hadoop/issues/905

上一篇:大数据 --> Spark和Hadoop作业之间的区别


下一篇:STM32的NVIC_PriorityGroupConfig使用及优先级分组方式理解