Storm -- Distributed and fault-tolerant realtime computation
这是一个分布式的、容错的实时计算系统
把Storm依赖组件的版本贴出来供各位参考:
java(1.7.0_45)
python(2.6.6)
zookeeper (zookeeper-3.4.5)
zeromq(zeromq-2.1.7)
jzmq(github trunk)
storm(0.8.1)
java和Python的安装过程不赘述了,说下其他组件的安装过程
安装zookeeper
wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.5/zookeeper-3.4.5.tar.gz
tar xvf zookeeper-3.4..tar.gz
cd zookeeper-3.4.5
cp conf/zoo_sample.cfg conf/zoo.cfg
设置下环境变量
export ZOOKEEPER_HOME=/root/zookeeper/zookeeper-3.4.
export JAVA_HOME=/usr/lib/jvm/java-1.7.-openjdk-1.7.0.45.x86_64
export PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH
然后就可以启动zookeeper了
zkServer.sh start
可以运行下zkCli.sh,测试下zookeeper是否可用
安装zeromq(安装好后/usr/local/lib下会有相应的库文件)
wget http://download.zeromq.org/zeromq-2.1.7.tar.gz
tar xvf zeromq-2.1..tar.gz
cd zeromq-2.1.
yum install -y gcc gcc-c++.x86_64 uuid-devel.x86_64 libuuid-devel.x86_64
./configure
make
make install
安装jzmq
git clone https://github.com/nathanmarz/jzmq.git
./autogen.sh
./configure
make
make install
开始安装storm
wget https://github.com/downloads/nathanmarz/storm/storm-0.8.1.zip
unzip storm-0.8..zip
cd storm-0.8.
修改conf/storm.yaml如下
storm.zookeeper.servers:
- "127.0.0.1"
nimbus.host: "192.168.24.57"
storm.local.dir: "/root/storm/workdir"
java.library.path: "/usr/local/lib:/opt/local/lib:/usr/lib"
topology.debug: "true"
然后将storm/bin加入环境变量
export STORM_HOME=/root/storm/storm-0.8.
export PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin:$PATH
启动storm
storm nimbus &
storm supervisor &
storm ui &
jps可以看到如下进程,而且日志也没有报错。基本上就ok了,可以提交任务在storm环境中跑了
core
QuorumPeerMain
Jps
supervisor
nimbus
提交拓扑的代码
storm jar stormDemo.jar com.storm.example.SimpleTopology testhui3
杀死拓扑
storm kill testhui3
默认8080端口提供了一个ui dashboard
enjoy~