1.安装Flume
下载地址:
http://www.apache.org/dyn/closer.lua/flume/1.7.0/apache-flume-1.7.0-bin.tar.gz
(1)解压安装包
sudo tar -zxvf apache-flume-1.7.0-bin.tar.gz -C /usr/local
cd /usr/local
sudo mv ./apache-flume-1.7.0-bin ./flume
sudo chown -R hadoop:hadoop ./flume
(2)配置环境变量
sudo vim ~/.bashrc export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64; export FLUME_HOME=/usr/local/flume export FLUME_CONF_DIR=$FLUME_HOME/conf export PATH=$PATH:$FLUME_HOME/bin
修改 flume-env.sh 配置文件:
cd /usr/local/flume/conf sudo cp ./flume-env.sh.template ./flume-env.sh sudo vim ./flume-env.sh export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64;
(3)查看flume版本信息
cd /usr/local/flume ./bin/flume-ng version
2.使用 Avro 数据源测试 Flume
3.使用 netcat 数据源测试 Flume
4.使用 Flume 作为 Spark Streaming 数据源
------------------------------------------- Time: 1488029430000 ms ------------------------------------------- Received 0 flume events. #这里省略了其他屏幕信息 ------------------------------------------- Time: 1488029432000 ms ------------------------------------------- Received 8 flume events. #这里省略了其他屏幕信息 ------------------------------------------- Time: 1488029434000 ms ------------------------------------------- Received 21 flume events.