如何将List对象转换为JavaDStream Spark?

我有List对象:

List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> rddQueue =
    Arrays.asList(tupleList);

如何将其转换为JavaDStream Spark?
谢谢提前!

解决方法:

你可以使用JavaStreamingContext和Queue< T>来做到这一点:

List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> 
  list = Arrays.asList();

// Random batch time, for the sake of the example.
JavaStreamingContext jssc = 
  new JavaStreamingContext("local[*]", "TestApp", new Duration(4000));

Queue<JavaRDD<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>>> rddQueue = 
  new LinkedList<>();

rddQueue.add(jssc.sparkContext().parallelize(list));

JavaDStream<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> dStream = 
  jssc.queueStream(rddQueue);
上一篇:SparkCore:History-Server/监控,查看程序历史运行日志


下一篇:Spark学习之路 (十五)SparkCore的源码解读(一)启动脚本