problems_spark

目录

1 执行sparksql报错

报错内容如下:
org.apache.spark.sql.AnalysisException: Unable to generate an encoder for inner class cn.itcast.spark.sql.Intro$Person without access to the scope that this class was defined in.Try moving this class out of its parent class.

解决方法:将case class Person(name:String,age:Int)这个样例类移动到父类的外面,就是移到最外面,不要让其包含在任何的类中。

2 wordcount程序报错

报错内容如下:
Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrder
解决步骤:

  1. 根据网上的方法,在maven中添加如下依赖:
<dependency>
          <groupId>com.fasterxml.jackson.core</groupId>
          <artifactId>jackson-core</artifactId>
          <version>2.2.4</version>
      </dependency>
      <dependency>
          <groupId>com.fasterxml.jackson.core</groupId>
          <artifactId>jackson-databind</artifactId>
          <version>2.2.4</version>
      </dependency>
  1. 再次执行,又报以下错误:
    Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;
    根据网上方法,maven中删除之前添加的依赖,改为添加以下依赖:
<dependency>
            <groupId>com.fasterxml.jackson.module</groupId>
            <artifactId>jackson-module-scala_2.11</artifactId>
            <version>2.7.4</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.module</groupId>
            <artifactId>jackson-module-jaxb-annotations</artifactId>
            <version>2.7.4</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.7.4</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <version>2.7.4</version>
        </dependency>

3

4

5

6

7

上一篇:springboot jackson


下一篇:json动态解析