Created by Wang, Jerry, last modified on Aug 24, 2015

added by Jerry:…

/root/devExpert/spark-1.4.1/sbin/…/conf – ( Jerry: I haven’t copied out the template from my own confiugration file yet )

starting org.apache.spark.deploy.master.Master, logging to /root/devExpert/spark-1.4.1/sbin/…/logs/spark-root-org.apache.spark.deploy.master.Master-1-NKGV50849583FV1.out

Jerry- location of log file:

NKGV50849583FV1:~/devExpert/spark-1.4.1/logs # vi spark-root-org.apache.spark.deploy.master.Master-1-NKGV50849583FV1.out

1 added by Jerry: loading load-spark-env.sh !!!1

2 added by Jerry, number of Jars: 1

3 added by Jerry, launch_classpath: /root/devExpert/spark-1.4.1/assembly/target/scala-2.10/spark-assembly-1.4.1-hadoop2.4.0.jar

4 added by Jerry,RUNNER:/usr/jdk1.7.0_79/bin/java

5 added by Jerry, printf argument list: org.apache.spark.deploy.master.Master --ip NKGV50849583FV1 --port 7077 --webui-port 8080

Jerry: this is default value

6 Spark Command: /usr/jdk1.7.0_79/bin/java -cp /root/devExpert/spark-1.4.1/sbin/…/conf/:/root/devExpert/spark-1.4.1/assembly/target/scala-2.10/spark-assem bly-1.4.1-hadoop2.4.0.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucleus-rdbms-3.2.9.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucle us-core-3.2.10.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucleus-api-jdo-3.2.6.jar -Xms512m -Xmx512m -XX:MaxPermSize=256m org.apache.spark.dep loy.master.Master --ip NKGV50849583FV1 --port 7077 --webui-port 8080

7 ========================================

8 added by Jerry, I am in if-else branch: /usr/jdk1.7.0_79/bin/java -cp /root/devExpert/spark-1.4.1/sbin/…/conf/:/root/devExpert/spark-1.4.1/assembly/targ et/scala-2.10/spark-assembly-1.4.1-hadoop2.4.0.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucleus-rdbms-3.2.9.jar:/root/devExpert/spark-1.4.1/l ib_managed/jars/datanucleus-core-3.2.10.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucleus-api-jdo-3.2.6.jar -Xms512m -Xmx512m -XX:MaxPermSize= 256m org.apache.spark.deploy.master.Master --ip NKGV50849583FV1 --port 7077 --webui-port 8080

9 Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties

10 15/08/16 12:45:17 INFO Master: Registered signal handlers for [TERM, HUP, INT]

11 15/08/16 12:45:17 WARN Utils: Your hostname, NKGV50849583FV1 resolves to a loopback address: 127.0.0.1; using 10.128.184.131 instead (on interface eth0)

12 15/08/16 12:45:17 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address

Jerry: useful hint

13 15/08/16 12:45:18 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

14 15/08/16 12:45:18 INFO SecurityManager: Changing view acls to: root

15 15/08/16 12:45:18 INFO SecurityManager: Changing modify acls to: root

16 15/08/16 12:45:18 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with mo dify permissions: Set(root)

17 15/08/16 12:45:19 INFO Slf4jLogger: Slf4jLogger started

18 15/08/16 12:45:19 INFO Remoting: Starting remoting

19 15/08/16 12:45:19 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkMaster@NKGV50849583FV1:7077]

20 15/08/16 12:45:19 INFO Utils: Successfully started service ‘sparkMaster’ on port 7077.

21 15/08/16 12:45:19 INFO Utils: Successfully started service on port 6066.

22 15/08/16 12:45:19 INFO StandaloneRestServer: Started REST server for submitting applications on port 6066

23 15/08/16 12:45:19 INFO Master: Starting Spark master at spark://NKGV50849583FV1:7077

24 15/08/16 12:45:19 INFO Master: Running Spark version 1.4.1

25 15/08/16 12:45:19 INFO Utils: Successfully started service ‘MasterUI’ on port 8080.

26 15/08/16 12:45:19 INFO MasterWebUI: Started MasterWebUI at http://10.128.184.131:8080

27 15/08/16 12:45:20 INFO Master: I have been elected leader! New state: ALIVE - cool!