准备工作
- 安装jdk 参考Ubuntu16.04 Install and Configure Oracle JDK
- 安装Hadoop 参考Ubuntu16.04 Install Hadoop 2.8.0
下载hive安装包
前往hive官方下载地址
解压安装hive
- 解压hive安装文件到相应的目录
hadoop@ubuntu16:~$ sudo tar -zxf apache-hive-2.1.1-bin.tar.gz -C /usr/local - 给hive目录重命名
hadoop@ubuntu16:~$ cd /usr/local hadoop@ubuntu16:/usr/local$ sudo mv apache-hive-2.1.1-bin/ hive - 将hive目录用户改为hadoop
hadoop@ubuntu16:/usr/local$ sudo chown -R hadoop hive/ - 查看hive目录
hadoop@ubuntu16:/usr/local$ cd hive/ hadoop@ubuntu16:/usr/local/hadoop$ ls -l hadoop@ubuntu16:/usr/local/hive$ ll total 112 drwxr-xr-x 10 hadoop root 4096 Jul 8 22:11 ./ drwxr-xr-x 12 root root 4096 Jul 8 20:11 ../ -rw-r--r-- 1 hadoop staff 29003 Nov 29 2016 LICENSE -rw-r--r-- 1 hadoop staff 578 Nov 29 2016 NOTICE -rw-r--r-- 1 hadoop staff 4122 Nov 29 2016 README.txt -rw-r--r-- 1 hadoop staff 18501 Nov 30 2016 RELEASE_NOTES.txt drwxr-xr-x 3 hadoop root 4096 Jul 8 20:11 bin/ drwxr-xr-x 3 hadoop root 4096 Jul 8 22:15 conf/ drwxr-xr-x 4 hadoop root 4096 Jul 8 20:11 examples/ drwxr-xr-x 7 hadoop root 4096 Jul 8 20:11 hcatalog/ drwxr-xr-x 2 hadoop root 4096 Jul 8 20:11 jdbc/ drwxr-xr-x 4 hadoop root 12288 Jul 8 21:10 lib/ drwxr-xr-x 4 hadoop root 4096 Jul 8 20:11 scripts/ drwxrwxr-x 3 hadoop hadoop 4096 Jul 8 22:19 tmp/
设置hive环境变量
打开~/.bashrc文件，并添加如下
# set hive env start export HIVE_HOME=/usr/local/hive export PATH=$PATH:$HIVE_HOME/bin export PATH=$PATH:$HIVE_HOME/hcatalog/bin:$HIVE_HOME/hcatalog/sbin export CLASSPATH=$CLASSPATH:$HIVE_HOME/lib export HIVE_CONF_DIR=$HIVE_HOME/conf # set hive env end使配置文件生效
hadoop@ubuntu16:~$ source ~/.bashrc
配置hive
修改默认配置文件名使配置文件生效
hadoop@ubuntu16:~$ cd /usr/local/hive/conf/ hadoop@ubuntu16:/usr/local/hive/conf$ cp hive-env.sh.template hive-env.sh hadoop@ubuntu16:/usr/local/hive/conf$ cp hive-default.xml.template hive-site.xml hadoop@ubuntu16:/usr/local/hive/conf$ cp hive-log4j2.properties.template hive-log4j2.properties hadoop@ubuntu16:/usr/local/hive/conf$ cp hive-exec-log4j2.properties.template hive-exec-log4j2.properties修改hive-env.sh,添加如下
export JAVA_HOME=/home/johnathon/Java/jdk1.8.0_131 export HADOOP_HOME=/usr/local/hadoop export HIVE_HOME=/usr/local/hive export HIVE_CONF_DIR=/usr/local/hive/conf创建hdfs目录
hadoop@ubuntu16:~$ hdfs dfs -mkdir -p /user/hive/warehouse hadoop@ubuntu16:~$ hdfs dfs -mkdir -p /user/hive/tmp hadoop@ubuntu16:~$ hdfs dfs -mkdir -p /user/hive/log hadoop@ubuntu16:~$ hdfs dfs -chmod -R 777 /user/hive/warehouse hadoop@ubuntu16:~$ hdfs DFS -chmod -R 777 /user/hive/tmp hadoop@ubuntu16:~$ hdfs dfs -chmod -R 777 /user/hive/log hadoop@ubuntu16:~$ hdfs dfs -ls /user/hive Found 3 items drwxrwxrwx - hadoop supergroup 0 2017-07-08 20:50 /user/hive/log drwxrwxrwx - hadoop supergroup 0 2017-07-08 21:54 /user/hive/tmp drwxrwxrwx - hadoop supergroup 0 2017-07-08 20:50 /user/hive/warehouse本地建立tmp目录
hadoop@ubuntu16:/usr/local/hive$ mkdir tmp安装mysql数据库，并作相关配置
获取最近的软件包的列表hadoop@ubuntu16:~$ sudo apt-get update安装mysql服务和客户端，中间会要求输入root密码
hadoop@ubuntu16:~$ sudo apt-get install mysql-server mysql-clientroot登陆mysql
hadoop@ubuntu16:~$ mysql -u root -p创建hive用户
mysql> create user 'hive' identified by 'hive';查看数据库
mysql> show databases;创建数据库命名为hive
mysql> create database hive;为hive用户授权
mysql> grant all privileges on *.* to 'hive'@'localhost' identified by 'hive'; mysql> flush privileges退出root登陆
mysql> exit;hive用户登录
hadoop@ubuntu16:~$ mysql -u hive -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 67 Server version: 5.7.18-0ubuntu0.16.04.1 (Ubuntu) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>修改hive-site.xml文件
1 相关目录信息
<property> <name>hive.exec.scratchdir</name> <value>/user/hive/tmp</value> <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/<username> is created, with ${hive.scratch.dir.permission}.</description> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value> <description>location of default database for the warehouse</description> </property> <property> <name>hive.querylog.location</name> <value>/user/hive/log</value> <description>Location of Hive run time structured log file</description> </property>2 ${system:java.io.tmpdir} 和 ${system:user.name} 分别替换成 /user/local/hive/tmp 和 ${user.name}
3 mysql数据库连接信息,需要将mysql的jar包放入hive/lib目录下
<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&characterEncoding=UTF-8&useSSL=false</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>hive</value> </property>
启动hive
hadoop@ubuntu16:~$ hive SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in file:/usr/local/hive/conf/hive-log4j2.properties Async: true Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. hive>
