Hadoop之Flume 记录

Hadoop之Flume 记录

出现这个错误是自己的粗心大意,解决:

在配置flume-conf.properties文件时,source和channel的对应关系是:

myAgentName.sources.mySourceName.channels = myChannelName

myAgentName.sinks.mySinkName.channel = myChannelName

注意其中的后缀,带s和不带s后缀。

这也恰好说明

source可以“流向”多个channel,而sink只能接收一个channel的“流入”。

从channel的角度看:channel既可以接收多个source的“流入”,又可以“流向”多个sink

例如多对多关系:

Hadoop之Flume 记录

对应的配置如下:

 # example.conf: A single-node Flume configuration

 # Name the components on this agent
agent1.sources=sour1 s2 s3
agent1.sinks=sink1 sink12 sink21 sink3
agent1.channels=chan1 chan2 chan3 # Describe/configure the source
agent1.sources.sour1.type=netcat
agent1.sources.sour1.bind=localhost
agent1.sources.sour1.port=44444 agent1.sources.s2.type=netcat
agent1.sources.s2.bind=localhost
agent1.sources.s2.port=44445 agent1.sources.s3.type=netcat
agent1.sources.s3.bind=localhost
agent1.sources.s3.port=44446 # Describe the sink
agent1.sinks.sink1.type=logger
agent1.sinks.sink12.type=logger
agent1.sinks.sink21.type=logger
agent1.sinks.sink3.type=logger # Use a channel which buffers events in memory
agent1.channels.chan1.type=memory
agent1.channels.chan1.capacity=1000
#agent1.channels.chan1.transactionCapacity=100 agent1.channels.chan2.type=memory
agent1.channels.chan2.capacity=1000 agent1.channels.chan3.type=memory
agent1.channels.chan3.capacity=1000 # Bind the source and sink to the channel
agent1.sources.sour1.channels=chan1 chan2
agent1.sources.s2.channels=chan2
agent1.sources.s3.channels=chan2 chan3 agent1.sinks.sink1.channel=chan1
agent1.sinks.sink12.channel=chan1
agent1.sinks.sink21.channel=chan2
agent1.sinks.sink3.channel=chan3
上一篇:Windows编译OpenCV4Android解决undefined reference to std错误


下一篇:SikuliX简介及安装