安装sqoop

数据迁移工具（Hadoop 和关系数据库服务器之间迁移数据的工具）

sqoop只需要安装在一个节点上，这个节点要能够访问到hive、hbase，所以这里安装在hadoop01节点上。

下载地址，这里选择1.4.7版本

wget https://archive.apache.org/dist/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

解压tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz，改名

mv sqoop-1.4.7.bin__hadoop-2.6.0 sqoop

配置sqoop

配置环境变量

vim /etc/profile
export SQOOP_HOME=/usr/local/sqoop
export PATH=$PATH:$SQOOP_HOME/bin
export CLASSPATH=$CLASSPATH:$SQOOP_HOME/lib

source /etc/profile

查看环境变量是否配置成功sqoop-version

修改sqoop-env.sh

cd /usr/local/sqoop/conf
mv sqoop-env-template.sh sqoop-env.sh
vi sqoop-env.sh

增加如下内容:
export HADOOP_COMMON_HOME=/usr/local/hadoop
export HADOOP_MAPRED_HOME=/usr/local/hadoop
export HIVE_HOME=/usr/local/hive
export ZOOKEEPER_HOME=/usr/local/zookeeper
export ZOOCFGDIR=/usr/local/zookeeper/conf

修改configure-sqoop

cd /usr/local/sqoop/bin

vi configure-sqoop

注释掉HCatalog，Accumulo(除非你准备使用HCatalog，Accumulo等HADOOP上的组件)

# Moved to be a runtime check in sqoop.
#if [ ! -d "${HCAT_HOME}" ]; then
#  echo "Warning: $HCAT_HOME does not exist! HCatalog jobs will fail."
#  echo 'Please set $HCAT_HOME to the root of your HCatalog installation.'
#fi

#if [ ! -d "${ACCUMULO_HOME}" ]; then
#  echo "Warning: $ACCUMULO_HOME does not exist! Accumulo imports will fail."
#  echo 'Please set $ACCUMULO_HOME to the root of your Accumulo installation.'
#fi

# Add HCatalog to dependency list
#if [ -e "${HCAT_HOME}/bin/hcat" ]; then
#  TMP_SQOOP_CLASSPATH=${SQOOP_CLASSPATH}:`${HCAT_HOME}/bin/hcat -classpath`
#  if [ -z "${HIVE_CONF_DIR}" ]; then
#    TMP_SQOOP_CLASSPATH=${TMP_SQOOP_CLASSPATH}:${HIVE_CONF_DIR}
#  fi
#  SQOOP_CLASSPATH=${TMP_SQOOP_CLASSPATH}
#fi

# Add Accumulo to dependency list
#if [ -e "$ACCUMULO_HOME/bin/accumulo" ]; then
#  for jn in `$ACCUMULO_HOME/bin/accumulo classpath | grep file:.*accumulo.*jar | cut -d':' -f2`; do
#    SQOOP_CLASSPATH=$SQOOP_CLASSPATH:$jn
#  done
#  for jn in `$ACCUMULO_HOME/bin/accumulo classpath | grep file:.*zookeeper.*jar | cut -d':' -f2`; do
#    SQOOP_CLASSPATH=$SQOOP_CLASSPATH:$jn
#  done
#fi

进入到sqoop的lib目录，下载mysql和postgresql的JDBC驱动，

cd /usr/local/sqoop/lib/
wget https://jdbc.postgresql.org/download/postgresql-42.2.24.jar

测试sqoop

sqoop list-tables --connect jdbc:postgresql://hadoop01:5432/jdbcstudy --driver org.postgresql.Driver --username postgres --password 201314

完成。