Sqoop是一个数据迁移的工具,可以把数据在关系型数据库与hive/hdfs之间进行数据的迁移。一、下载
由于之前我们使用的hadoop版本是3.3.0,hive的版本是3.1.3,那么目前比较兼容的版本我们先择sqoop1.4.7下载地址:https://archive.apache.org/dist/sqoop/wget https://archive.apache.org/dist/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
二、配置
执行指令在哪个目录就下载到当前目录,下载完毕后目录下会出现sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /usr/local/software/sqoop-1.4.7
cd /usr/local/software/sqoop-1.4.7/conf
cp sqoop-env-template.sh sqoop-env.sh
#Set path to where bin/hadoop is availableexport HADOOP_COMMON_HOME=/usr/local/software/hadoop/hadoop-3.3.0#Set path to where hadoop-*-core.jar is availableexport HADOOP_MAPRED_HOME=/usr/local/software/hadoop/hadoop-3.3.0/#set the path to where bin/hbase is available#export HBASE_HOME=#Set the path to where bin/hive is availableexport HIVE_HOME=/usr/local/software/hive-3.1.3
另外需要在lib目录下添加MySQL驱动包和commons-lang包cp /usr/local/software/hive-3.1.3/lib/mysql-connector-java-8.0.28.jar /usr/local/software/sqoop-1.4.7/lib/
cp /usr/local/software/hive-3.1.3/lib/commons-lang-2.6.jar /usr/local/software/sqoop-1.4.7/lib/
#sqoopexport SQOOP_HOME=/usr/local/software/sqoop-1.4.7export PATH=$SQOOP_HOME/bin:$PATH
source /etc/profile使配置的环境变量生效。