安装sqoop 1.99.4

参考http://sqoop.apache.org/docs/1.99.4/Installation.html

1.简介
sqoop2分为server和client两部分。server作为maprde的客户端,必须安装在hadoop集群上。client无需安装在hadoop集群上,任何机器均可。

2.安装server

    server必须安装在hadoop结点上,namenode和datanode均可。测试是否安装交配置hadoop:
  1. [root@node2 ~]# $HADOOP_PREFIX/bin/hdfs dfs -ls /
  2. Found 4 items
  3. drwxr-xr-x - root supergroup 0 2016-02-25 18:44 /hbase
  4. drwxr-xr-x - root supergroup 0 2015-12-31 12:22 /test
  5. drwxrwxr-x - root supergroup 0 2015-12-31 21:30 /tmp
  6. drwxr-xr-x - root supergroup 0 2015-12-31 20:50 /user
实际上server需要读取namenode和resourcemanager的配置信息,并不依赖这些服务。

1)修改
server/conf/catalina.properties 中hadoop的lib路径:
common.loader中加入
  1. ,/usr/local/hadoop-2.6.3/share/hadoop/common/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/common/lib/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/hdfs/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/mapreduce/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/tools/lib/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/yarn/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/yarn/lib/*.jar,/usr/local/hadoop-2.6.3/share/hadoop/httpfs/tomcat/lib/*.jar
2)启动server
官网的文档已经过时,无法执行bin/sqoop2-server start这样的命令。执行:
bin/sqoop.sh server start
查看日志:
Caused by: org.apache.sqoop.common.SqoopException: MAPREDUCE_0002:Failure on submission engine initialization - Invalid Hadoop configuration 
directory (not a directory or permission issues): /etc/hadoop/conf/
不知道哪个地方配置了/etc/hadoop/conf,根本没有这目录。百度一下:http://www.myexception.cn/database/1600202.html
修改为/usr/local/hadoop-2.6.3/etc/hadoop/


3.client
    clinet无需要配置。把server所在机器的sqoop-1.99.3复制到其它机器上即可,然后执行
#bin/sqoop.sh client
sqoop:000> set server --host node1 --port 12000 --webApp sqoop即可连接server执行命令















































上一篇:图解TCP、IP笔记


下一篇:Slony-I的限制