Ubuntu16.04 Install Sqoop1.4.6

## 安装环境 OS: linux(ubuntu16.04) sqoop version: 1.4.6 hadoop version: 2.8.0 mysql version: 5.7.18

下载解压sqoop1.4.6

• 前往sqoop官网下载,默认下载目录为当前用户Downloads目录

    johnathon@ubuntu16:~$ cd Downloads/
    johnathon@ubuntu16:~/Downloads$ sudo tar -zxf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C /usr/local
    johnathon@ubuntu16:~/Downloads$ cd /usr/local
    johnathon@ubuntu16:/usr/local$ sudo mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop 
    johnathon@ubuntu16:/usr/local$ sudo chown -R hadoop:hadoop sqoop 
  

修改配置文件

1. 打开sqoop-env.sh

    hadoop@ubuntu16:~$ cd /usr/local/sqoop/conf/
    hadoop@ubuntu16:/usr/local/sqoop/conf$ cp sqoop-env-template.sh sqoop-env.sh
    hadoop@ubuntu16:/usr/local/sqoop/conf$ vi sqoop-env.sh
   

2. 添加以下信息

     export HADOOP_COMMON_HOME=/usr/local/hadoop
     export HADOOP_MAPRED_HOME=/usr/local/hadoop
     export HIVE_HOME=/usr/local/hive
   

配置环境变量

1. 打开~/.bashrc文件

    hadoop@ubuntu16:~$ vi ~/.bashrc
   

2. 添加以下信息

    export SQOOP_HOME=/usr/local/sqoop
    export PATH=$PATH:$SBT_HOME/bin:$SQOOP_HOME/bin
    export CLASSPATH=$CLASSPATH:$SQOOP_HOME/lib
   

3. 使修改生效

    hadoop@ubuntu16:~$ source ~/.bashrc
   

4. 添加mysql驱动到$SQOOP_HOME/lib下

    johnathon@ubuntu16:~/Downloads/mysql-connector-java-5.1.42$ sudo cp mysql-connector-java-5.1.42-bin.jar /usr/local/sqoop/lib/
    [sudo] password for johnathon:
    johnathon@ubuntu16:~/Downloads/mysql-connector-java-5.1.42$ cd /usr/local/sqoop/lib/
    johnathon@ubuntu16:/usr/local/sqoop/lib$ ll mysql-connector-java-5.1.42-bin.jar 
    -rw-r--r-- 1 root root 996444 Jul 16 17:25 mysql-connector-java-5.1.42-bin.jar
   

5. 测试连接mysql

• 测试命令

    sqoop list-databases --connect jdbc:mysql://127.0.0.1:3306/ --username root -P
  

• mysql数据库显示如下，则连接成功

    hadoop@ubuntu16:~$ sqoop list-databases --connect jdbc:mysql://127.0.0.1:3306/ --username root -P
    Warning: /usr/local/sqoop/../hbase does not exist! HBase imports will fail.
    Please set $HBASE_HOME to the root of your HBase installation.
    Warning: /usr/local/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
    Please set $HCAT_HOME to the root of your HCatalog installation.
    Warning: /usr/local/sqoop/../accumulo does not exist! Accumulo imports will fail.
    Please set $ACCUMULO_HOME to the root of your Accumulo installation.
    Warning: /usr/local/sqoop/../zookeeper does not exist! Accumulo imports will fail.
    Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
    17/07/16 18:04:06 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
    Enter password:
    17/07/16 18:04:12 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
    Sun Jul 16 18:04:12 CST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
    information_schema
    hive
    mysql
    performance_schema
    sys