亚马逊aws emr hadoop集群 调度工具azkaban安装

  • 解压后重命名
    亚马逊aws emr hadoop集群 调度工具azkaban安装
    亚马逊aws emr hadoop集群 调度工具azkaban安装
    亚马逊aws emr hadoop集群 调度工具azkaban安装
  • keytool -keystore keystore -alias jetty -genkey -keyalg RSA在 azkaban目录下生成keystore文件,然后mv到/lib/azkaban/azkaban-web
    亚马逊aws emr hadoop集群 调度工具azkaban安装

亚马逊aws emr hadoop集群 调度工具azkaban安装

  • /lib/azkaban/azkaban-web/conf/azkaban.properties
#Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=/lib/azkaban/azkaban-web/web
default.timezone.id=Asia/Shanghai

#Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=/lib/azkaban/azkaban-web/conf/azkaban-users.xml
#Loader for projects
executor.global.properties=/lib/azkaban/azkaban-executor/conf/global.properties
azkaban.project.dir=projects

database.type=mysql
mysql.port=3306
mysql.host=xxx
mysql.database=azkaban
mysql.user=hadoop
mysql.password=123456
mysql.numconnections=100

# Velocity dev mode
velocity.dev.mode=false

# Azkaban Jetty server properties.
jetty.maxThreads=25
jetty.ssl.port=8443
jetty.port=8081
jetty.keystore=/lib/azkaban/azkaban-web/keystore
jetty.password=123456
jetty.keypassword=123456
jetty.truststore=/lib/azkaban/azkaban-web/keystore
jetty.trustpassword=123456

# Azkaban Executor settings
executor.port=12321

# mail settings
mail.sender=xxx@163.com
mail.host=smtp.163.com
mail.user=xxx@163.com
#授权码
mail.password=xxx
job.failure.email=xxx@163.com
job.success.email=xxx@163.com

lockdown.create.projects=false

cache.directory=cache
    • /lib/azkaban/azkaban-web/conf/azkaban-users.xml
<azkaban-users>
	<user username="azkaban" password="azkaban" roles="admin" groups="azkaban" />
	<user username="metrics" password="metrics" roles="metrics"/>
<user username="admin" password="admin" roles="admin,metrics"/>	
	<role name="admin" permissions="ADMIN" />
	<role name="metrics" permissions="METRICS"/>
</azkaban-users>

  • lib/azkaban/azkaban-executor/conf/azkaban.properties
#Azkaban
default.timezone.id=Asia/Shanghai

# Azkaban JobTypes Plugins
azkaban.jobtype.plugin.dir=plugins/jobtypes

#Loader for projects
executor.global.properties=/lib/azkaban/azkaban-executor/conf/global.properties
azkaban.project.dir=projects

database.type=mysql
mysql.port=3306
mysql.host=xxx
mysql.database=azkaban
mysql.user=hadoop
mysql.password=123456
mysql.numconnections=100

# Azkaban Executor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30


  • web https://xxxxx:8443/ u:admin p:admin
    亚马逊aws emr hadoop集群 调度工具azkaban安装
    亚马逊aws emr hadoop集群 调度工具azkaban安装
  • sqoop list-databases --connect jdbc:mysql://xxxx:3306 --username xx--password xxx
    亚马逊aws emr hadoop集群 调度工具azkaban安装
  • 在 有mysql的机器上 sudo mysql 进入cli
    亚马逊aws emr hadoop集群 调度工具azkaban安装
    select host,user,password,select_priv,insert_priv,drop_priv from mysql.user;
  • 添加远程账号hadoop
  • grant all privileges on . to hadoop@’%’ identified by ‘password’;
    亚马逊aws emr hadoop集群 调度工具azkaban安装
  • source /home/hadoop/azkaban-cli/create-all-sql-2.5.0.sql;在mysql中建azkaban需要用到的表
上一篇:BEGINNING SHAREPOINT® 2013 DEVELOPMENT 第5章节--Windows Azure概览 SharePoint和Windows Azure


下一篇:jdbc中Statement接口实现CRUD操作(了解)