使用idea编写spark程序,想要查看源代码,点进去是compiled code
` private[sql] def this(sc : org.apache.spark.SparkContext) = { /* compiled code */ } def version : scala.Predef.String = { /* compiled code */ }`
网上很多方法都是下载java 反编译工具,如java bytecode compiler。
按照方法试了,倒是java类的class点进去能够看到反编译的代码,scala的class点进去依然没有反编译。可能是idea版本问题。
试了几次遂放弃。
因为即使是反编译成功了,依然不是我想要的结果。反统的代码毕竟不是源码,也没有注释。
所以为何不利用maven下载源码与注释呢。
可在全局配置里勾选source和doc两项,maven在引入依赖时自动下载源码和注释。
当然更好的方案是,不勾选此两项,仅在需要查看源码的时候,去下载。
好了,点击查看spark源码,这是我想要的效果:
`/**
-
Activate the stream and then wrap a callout to runActivatedStream, handling start and stop.
-
Note that this method ensures that [[QueryStartedEvent]] and [[QueryTerminatedEvent]] are
-
posted such that listeners are guaranteed to get a start event before a termination.
-
Furthermore, this method also ensures that [[QueryStartedEvent]] event is posted before the
-
start()
method returns.
*/
private def runStream(): Unit = {
try {
sparkSession.sparkContext.setJobGroup(runId.toString, getBatchDescriptionString,
interruptOnCancel = true)
sparkSession.sparkContext.setLocalProperty(StreamExecution.QUERY_ID_KEY, id.toString)
if (sparkSession.sessionState.conf.streamingMetricsEnabled) {
sparkSession.sparkContext.env.metricsSystem.registerSource(streamMetrics)
}//
postEvent
does not throw non fatal exception.
val startTimestamp = triggerClock.getTimeMillis()
postEvent(new QueryStartedEvent(id, runId, name, formatTimestamp(startTimestamp)))`