hive 结合mysql快速安装
系统使用centos7.1 官方下载hadoop,hive,jdk的安装包,解压在新建用户hive的目录下,安装mariadb-server(mysql-server) [hive@localhost ~]$ ll total 544900 drwxrwxr-x 9 hive hive 4096 Sep 10 07:31 apache-hive-2.1.0-bin -rw-rw-r-- 1 hive hive 149599799 Jun 21 01:26 apache-hive-2.1.0-bin.tar.gz drwxrwxr-x 9 hive hive 149 Sep 9 10:45 apache-tomcat-7.0.70 -rw-r--r-- 1 hive hive 8924465 Sep 9 10:29 apache-tomcat-7.0.70.tar.gz drwxr-xr-x 9 root root 139 Aug 18 01:49 hadoop-2.7.3 -rw-r--r-- 1 root root 214092195 Aug 25 19:25 hadoop-2.7.3.tar.gz lrwxrwxrwx 1 hive hive 12 Sep 9 08:28 hadoop-last -> hadoop-2.7.3 lrwxrwxrwx 1 hive hive 21 Sep 9 09:08 hive-last -> apache-hive-2.1.0-bin drwxr-xr-x 8 hive hive 4096 Jun 23 01:56 jdk1.8.0_102 -rw-r--r-- 1 root root 181435897 Sep 9 06:57 jdk-8u102-linux-x64.tar.gz drwxr-xr-x 4 hive hive 143 May 4 11:11 mysql-connector-java-5.1.39 -rw-r--r-- 1 hive hive 3899019 Sep 10 07:18 mysql-connector-java-5.1.39.tar.gz -rw-rw-r-- 1 hive hive 11183 Sep 9 10:18 wc.txt 修改用户的环境 [hive@localhost ~]$ cat .bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs export JAVA_HOME=/home/hive/jdk1.8.0_102 export HADOOP_HOME=/home/hive/hadoop-last export HIVE_HOME=/home/hive/hive-last PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HIVE_HOME/bin:$HIVE_HOME/conf export PATH 启动mysql数据库,为hadoop/hive添加一个用户 [root@localhost ~]# systemctl start mariadb [root@localhost ~]# mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 3 Server version: 5.5.50-MariaDB MariaDB Server Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> CREATE USER 'hadoop'@'localhost' IDENTIFIED BY 'hadoop'; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'hadoop'@'localhost' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> EXIT Bye MariaDB [(none)]> CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive'; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> exit Bye 进入$HIVE_HOME/conf,去掉以下三个文件的后辍.template -rw-r--r-- 1 hive hive 2378 Sep 10 06:24 hive-env.sh -rw-r--r-- 1 hive hive 2299 Jun 3 10:43 hive-exec-log4j2.properties -rw-r--r-- 1 hive hive 2950 Sep 10 06:25 hive-log4j2.properties 重命令hive-default.xml.template为hive-site.xml -rw-r--r-- 1 hive hive 225729 Sep 10 03:21 hive-site.xml 添加hive-env.sh两行 export HADOOP_HOME=/home/hive/hadoop-last export HIVE_CONF_DIR=/home/hive/hive-last/conf 修改hive-site.xml 默认hive.metastore.warehouse.dir是/user/hive/warehouse hive.exec.scratchdir是/tmp/hive 创建这个目录 如果是使用内置的derby存储metadata [hive@localhost hive-last]$ schematool -dbType derby -initSchema which: no hbase in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hive/.local/bin:/home/hive/bin:/home/hive/jdk1.8.0_102/bin:/home/hive/hadoop-last/bin:/home/hive/hive-last/bin:/home/hive/hive-last/conf) SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/hive/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hive/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:derby:;databaseName=metastore_db;create=true Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver Metastore connection User: APP Starting metastore schema initialization to 2.1.0 Initialization script hive-schema-2.1.0.derby.sql Initialization script completed schemaTool completed 修改 chmod a+rw /tmp/hive/ 下面修改为使用mysql hive-site.xml 480: javax.jdo.option.ConnectionPassword 481- hive 498: javax.jdo.option.ConnectionURL 499- jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true 932: javax.jdo.option.ConnectionDriverName 933- com.mysql.jdbc.Driver 960: javax.jdo.option.ConnectionUserName 961- hive 下载mysql-connector [hive@localhost mysql-connector-java-5.1.39]$ cp mysql-connector-java-5.1.39-bin.jar ~/hive-last/lib/ -v ‘mysql-connector-java-5.1.39-bin.jar’ -> ‘/home/hive/hive-last/lib/mysql-connector-java-5.1.39-bin.jar’ [hive@localhost hive-last]$ schematool -dbType mysql -initSchema which: no hbase in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hive/.local/bin:/home/hive/bin:/home/hive/jdk1.8.0_102/bin:/home/hive/hadoop-last/bin:/home/hive/hive-last/bin:/home/hive/hive-last/conf) SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/hive/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hive/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: hive Starting metastore schema initialization to 2.1.0 Initialization script hive-schema-2.1.0.mysql.sql Initialization script completed schemaTool completed 修改hive-site.xml的system变量,解决变量无法识别的问题,使用绝对路径 :%s#${system:java.io.tmpdir}#/tmp/javaiotmp# :%s#${system:user.name}#hive#
)
