hive 结合mysql快速安装

系统使用centos7.1
官方下载hadoop,hive,jdk的安装包,解压在新建用户hive的目录下,安装mariadb-server(mysql-server)
[hive@localhost ~]$ ll
total 544900
drwxrwxr-x 9 hive hive      4096 Sep 10 07:31 apache-hive-2.1.0-bin
-rw-rw-r-- 1 hive hive 149599799 Jun 21 01:26 apache-hive-2.1.0-bin.tar.gz
drwxrwxr-x 9 hive hive       149 Sep  9 10:45 apache-tomcat-7.0.70
-rw-r--r-- 1 hive hive   8924465 Sep  9 10:29 apache-tomcat-7.0.70.tar.gz
drwxr-xr-x 9 root root       139 Aug 18 01:49 hadoop-2.7.3
-rw-r--r-- 1 root root 214092195 Aug 25 19:25 hadoop-2.7.3.tar.gz
lrwxrwxrwx 1 hive hive        12 Sep  9 08:28 hadoop-last -> hadoop-2.7.3
lrwxrwxrwx 1 hive hive        21 Sep  9 09:08 hive-last -> apache-hive-2.1.0-bin
drwxr-xr-x 8 hive hive      4096 Jun 23 01:56 jdk1.8.0_102
-rw-r--r-- 1 root root 181435897 Sep  9 06:57 jdk-8u102-linux-x64.tar.gz
drwxr-xr-x 4 hive hive       143 May  4 11:11 mysql-connector-java-5.1.39
-rw-r--r-- 1 hive hive   3899019 Sep 10 07:18 mysql-connector-java-5.1.39.tar.gz
-rw-rw-r-- 1 hive hive     11183 Sep  9 10:18 wc.txt

修改用户的环境
[hive@localhost ~]$ cat .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs
export JAVA_HOME=/home/hive/jdk1.8.0_102
export HADOOP_HOME=/home/hive/hadoop-last
export HIVE_HOME=/home/hive/hive-last

PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HIVE_HOME/bin:$HIVE_HOME/conf

export PATH

启动mysql数据库,为hadoop/hive添加一个用户
[root@localhost ~]# systemctl start mariadb
[root@localhost ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 5.5.50-MariaDB MariaDB Server

Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> CREATE USER 'hadoop'@'localhost' IDENTIFIED BY 'hadoop';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'hadoop'@'localhost' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> EXIT
Bye

MariaDB [(none)]> CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> exit
Bye

进入$HIVE_HOME/conf,去掉以下三个文件的后辍.template
-rw-r--r-- 1 hive hive   2378 Sep 10 06:24 hive-env.sh
-rw-r--r-- 1 hive hive   2299 Jun  3 10:43 hive-exec-log4j2.properties
-rw-r--r-- 1 hive hive   2950 Sep 10 06:25 hive-log4j2.properties
重命令hive-default.xml.template为hive-site.xml
-rw-r--r-- 1 hive hive 225729 Sep 10 03:21 hive-site.xml
添加hive-env.sh两行
export HADOOP_HOME=/home/hive/hadoop-last
export HIVE_CONF_DIR=/home/hive/hive-last/conf
修改hive-site.xml
默认hive.metastore.warehouse.dir是/user/hive/warehouse
hive.exec.scratchdir是/tmp/hive
创建这个目录

如果是使用内置的derby存储metadata
[hive@localhost hive-last]$ schematool -dbType derby -initSchema
which: no hbase in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hive/.local/bin:/home/hive/bin:/home/hive/jdk1.8.0_102/bin:/home/hive/hadoop-last/bin:/home/hive/hive-last/bin:/home/hive/hive-last/conf)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hive/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hive/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :    org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:       APP
Starting metastore schema initialization to 2.1.0
Initialization script hive-schema-2.1.0.derby.sql
Initialization script completed
schemaTool completed
修改
chmod a+rw /tmp/hive/
下面修改为使用mysql

hive-site.xml
480:    javax.jdo.option.ConnectionPassword
481-    hive
498:    javax.jdo.option.ConnectionURL
499-    jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true
932:    javax.jdo.option.ConnectionDriverName
933-    com.mysql.jdbc.Driver
960:    javax.jdo.option.ConnectionUserName
961-    hive

下载mysql-connector
[hive@localhost mysql-connector-java-5.1.39]$ cp mysql-connector-java-5.1.39-bin.jar ~/hive-last/lib/ -v
‘mysql-connector-java-5.1.39-bin.jar’ -> ‘/home/hive/hive-last/lib/mysql-connector-java-5.1.39-bin.jar’


[hive@localhost hive-last]$ schematool -dbType mysql -initSchema
which: no hbase in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hive/.local/bin:/home/hive/bin:/home/hive/jdk1.8.0_102/bin:/home/hive/hadoop-last/bin:/home/hive/hive-last/bin:/home/hive/hive-last/conf)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hive/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hive/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       hive
Starting metastore schema initialization to 2.1.0
Initialization script hive-schema-2.1.0.mysql.sql
Initialization script completed
schemaTool completed


修改hive-site.xml的system变量,解决变量无法识别的问题,使用绝对路径
:%s#${system:java.io.tmpdir}#/tmp/javaiotmp#
:%s#${system:user.name}#hive#
优秀的个人博客,低调大师

微信关注我们

原文链接:https://yq.aliyun.com/articles/683734

转载内容版权归作者及来源网站所有!

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

相关文章

发表评论

资源下载

更多资源
Mario,低调大师唯一一个Java游戏作品

Mario,低调大师唯一一个Java游戏作品

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长,特征是大鼻子、头戴帽子、身穿背带裤,还留着胡子。与他的双胞胎兄弟路易基一起,长年担任任天堂的招牌角色。

Apache Tomcat7、8、9(Java Web服务器)

Apache Tomcat7、8、9(Java Web服务器)

Tomcat是Apache 软件基金会(Apache Software Foundation)的Jakarta 项目中的一个核心项目,由Apache、Sun 和其他一些公司及个人共同开发而成。因为Tomcat 技术先进、性能稳定,而且免费,因而深受Java 爱好者的喜爱并得到了部分软件开发商的认可,成为目前比较流行的Web 应用服务器。

Eclipse(集成开发环境)

Eclipse(集成开发环境)

Eclipse 是一个开放源代码的、基于Java的可扩展开发平台。就其本身而言,它只是一个框架和一组服务,用于通过插件组件构建开发环境。幸运的是,Eclipse 附带了一个标准的插件集,包括Java开发工具(Java Development Kit,JDK)。

Sublime Text 一个代码编辑器

Sublime Text 一个代码编辑器

Sublime Text具有漂亮的用户界面和强大的功能,例如代码缩略图,Python的插件,代码段等。还可自定义键绑定,菜单和工具栏。Sublime Text 的主要功能包括:拼写检查,书签,完整的 Python API , Goto 功能,即时项目切换,多选择,多窗口等等。Sublime Text 是一个跨平台的编辑器,同时支持Windows、Linux、Mac OS X等操作系统。