Cloudera-manager(CDH6.3.0)大数据平台搭建一指禅指南(impala,kudu,hdfs,hive,kafka,yarn,spark,hbase,hue)
CHD6，大量hadoop生态的重大更新升级，果断把现有系统升级到CHD6上。
准备：
192.168.88.31 master--》机器名修改为:manager 还是重复的master把
192.168.88.32 utility
192.168.88.33 gateway
192.168.88.34 worker1
每台虚拟机8g内存（不想给自己找麻烦，就搞8G内存，否则会卡主各种莫名其妙的问题的）,建议最低4台测试。3台虚拟机也是可以安装环境的。
[root@manager ~]# vi /etc/sysconfig/network
HOSTNAME=master
# [root@manager ~]# vi /etc/hosts
127.0.0.1 localhost manager localhost4 localhost4.localdomain4 （需要删除）
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 （需要删除）
//永久性的修改主机名称，重启后能保持修改后的。
[root@manager ~]# hostnamectl set-hostname master
[root@manager ~]# cat /etc/hostname
[root@manager ~]# reboot
[root@manager ~]# hostname
manager
设置静态IP、DNS、网关
[hadoop@utility ~]$ sudo vi /etc/sysconfig/network-scripts/ifcfg-ens33
#BOOTPROTO=dhcp
BOOTPROTO=static #设置网卡获得ip地址的方式，可能的选项为static，dhcp或bootp，分别对应静态指定的 ip地址，通过dhcp协议获得的ip地址，通过bootp协议获得的ip地址
IPADDR=192.168.88.31 #如果设置网卡获得 ip地址的方式为静态指定，此字段就指定了网卡对应的ip地址
NETMASK=255.255.255.0 #网卡对应的网络掩码
NETWORK=192.168.88.0 #网卡对应的网络地址
GATEWAY=192.168.88.2 #有时候叫做default route address
DNS1=192.168.88.2
[hadoop@utility ~]$ sudo service network restart
[hadoop@utility ~]$ sudo ifconfig -a
[hadoop@utility ~]$ sudo nslookup www.baidu.com
[hadoop@utility ~]$ sudo nmcli con mod ens33 ipv4.dns "192.168.88.2"
[hadoop@utility ~]$ sudo nmcli con up ens33
[hadoop@utility ~]$ cat /etc/resolv.conf
给每台机器的hadoop用户添加sudo免密码的功能
[root@localhost ~]# vi /etc/sudoers
添加如下的声明:
hadoop ALL=(ALL) NOPASSWD: ALL
[hadoop@manager ~]$ sudo yum install -y vim wget ntp net-tools
给每台机器都增加hosts文件的配置,修改完成后，相互之间可以ping通
[hadoop@manager ~]$ sudo vi /etc/hosts
192.168.88.31 master
192.168.88.32 utility
192.168.88.33 gateway
192.168.88.34 worker1
在这里有个坑需要注意的，以前初始的hosts文件中是有
127.0.0.1 localhost manager localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
的，这个是需要删除的，否则后面安装报错。
[hadoop@manager ~]$ ping master
[hadoop@manager ~]$ ping www.baidu.com
每台主机关闭防火墙（折行写的命令最后不能有空格，否则报错）
[hadoop@manager ~]$
sudo systemctl stop firewalld \
&& sudo systemctl disable firewalld \
&& sudo systemctl status firewalld
每台机器关闭selinux
[hadoop@manager ~]$ sudo setenforce 0
[hadoop@manager ~]$ sudo vim /etc/selinux/config
SELINUX=permissive
[hadoop@manager ~]$ getenforce
Permissive
每台机器配置NTP服务
[hadoop@manager ~]$ sudo vim /etc/ntp.conf
#将时钟服务器更改为如下4个
server 0.cn.pool.ntp.org
server 1.cn.pool.ntp.org
server 2.cn.pool.ntp.org
server 3.cn.pool.ntp.org
[hadoop@manager ~]$
sudo systemctl start ntpd \
&& sudo systemctl enable ntpd \
&& sudo ntpdate -u 0.cn.pool.ntp.org \
&& sudo hwclock --systohc \
&& sudo date
在tools主机上安装（生产环境，减轻master的负担，不过在我本机开发，都是用的master，所以这里我用master进行安装）
上传下载的文件到
[hadoop@manager cloudera]$ pwd
/home/hadoop/Downloads/cloudera
cloudera/
├── cloudera-manager
│ ├── allkeys.asc
│ ├── cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm
│ ├── cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm
│ ├── cloudera-manager-server-6.3.0-1281944.el7.x86_64.rpm
│ ├── cloudera-manager-server-db-2-6.3.0-1281944.el7.x86_64.rpm
│ ├── enterprise-debuginfo-6.3.0-1281944.el7.x86_64.rpm
│ └── oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
├── mysql
│ └── mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar
├── mysql-jdbc
│ └── mysql-connector-java-5.1.47.tar.gz
└── parcel
├── CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel
└── manifest.json
安装httpd服务，以及为本地创建yum安装的库做准备
[hadoop@manager cloudera]$
sudo yum -y install httpd createrepo \
&& sudo systemctl start httpd \
&& sudo systemctl enable httpd \
&& sudo cd /home/hadoop/Downloads/cloudera/cloudera-manager/ \
&& sudo createrepo . \
&& sudo mv /home/hadoop/Downloads/cloudera/cloudera-manager/ /var/www/html/
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.njupt.edu.cn
* extras: mirrors.njupt.edu.cn
* updates: mirrors.njupt.edu.cn
Package createrepo-0.9.9-28.el7.noarch already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 0:2.4.6-90.el7.centos will be installed
--> Processing Dependency: httpd-tools = 2.4.6-90.el7.centos for package: httpd-2.4.6-90.el7.centos.x86_64
--> Running transaction check
---> Package httpd-tools.x86_64 0:2.4.6-90.el7.centos will be installed
--> Finished Dependency Resolution
Dependencies Resolved
===================================================================================================================================================================================================================
Package Arch Version Repository Size
===================================================================================================================================================================================================================
Installing:
httpd x86_64 2.4.6-90.el7.centos base 2.7 M
Installing for dependencies:
httpd-tools x86_64 2.4.6-90.el7.centos base 91 k
Transaction Summary
===================================================================================================================================================================================================================
Install 1 Package (+1 Dependent package)
Total download size: 2.8 M
Installed size: 9.5 M
Downloading packages:
(1/2): httpd-2.4.6-90.el7.centos.x86_64.rpm | 2.7 MB 00:00:01
httpd-tools-2.4.6-90.el7.cento FAILED
http://mirrors.cqu.edu.cn/CentOS/7.7.1908/os/x86_64/Packages/httpd-tools-2.4.6-90.el7.centos.x86_64.rpm: [Errno 14] curl#6 - "Could not resolve host: mirrors.cqu.edu.cn; Unknown error"B/s | 2.2 MB 00:00:00 ETA
Trying other mirror.
(2/2): httpd-tools-2.4.6-90.el7.centos.x86_64.rpm | 91 kB 00:00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 271 kB/s | 2.8 MB 00:00:10
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : httpd-tools-2.4.6-90.el7.centos.x86_64 1/2
Installing : httpd-2.4.6-90.el7.centos.x86_64 2/2
Verifying : httpd-tools-2.4.6-90.el7.centos.x86_64 1/2
Verifying : httpd-2.4.6-90.el7.centos.x86_64 2/2
Installed:
httpd.x86_64 0:2.4.6-90.el7.centos
Dependency Installed:
httpd-tools.x86_64 0:2.4.6-90.el7.centos
Complete!
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
Spawning worker 0 with 2 pkgs
Spawning worker 1 with 2 pkgs
Spawning worker 2 with 1 pkgs
Spawning worker 3 with 1 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[hadoop@manager ~]$
[root@manager cloudera-manager]# pwd
/var/www/html/cloudera-manager
[root@manager cloudera-manager]# createrepo .
Spawning worker 0 with 2 pkgs
Spawning worker 1 with 2 pkgs
Spawning worker 2 with 1 pkgs
Spawning worker 3 with 1 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@manager cloudera-manager]# ls -al
total 1378008
drwxr-xr-x. 3 hadoop hadoop 4096 Oct 31 00:17 .
drwxr-xr-x. 3 root root 30 Oct 30 23:03 ..
-rw-r--r--. 1 hadoop hadoop 14041 Oct 30 20:25 allkeys.asc
-rw-r--r--. 1 hadoop hadoop 10479136 Oct 30 20:25 cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm
-rw-r--r--. 1 hadoop hadoop 1201341068 Oct 30 20:28 cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm
-rw-r--r--. 1 hadoop hadoop 11464 Oct 30 20:25 cloudera-manager-server-6.3.0-1281944.el7.x86_64.rpm
-rw-r--r--. 1 hadoop hadoop 10996 Oct 30 20:25 cloudera-manager-server-db-2-6.3.0-1281944.el7.x86_64.rpm
-rw-r--r--. 1 hadoop hadoop 14209884 Oct 30 20:25 enterprise-debuginfo-6.3.0-1281944.el7.x86_64.rpm
-rw-r--r--. 1 hadoop hadoop 184988341 Oct 30 20:26 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
drwxr-xr-x. 2 root root 4096 Oct 31 00:17 repodata -----》这个目录要检查一下，否则将来的本地yum是没法通过的
[root@manager cloudera-manager]#
完成后，我们使用firefox打开--》http://192.168.88.31/cloudera-manager/ ，之后我们要能看到一个网页的文件列表
Index of /cloudera-manager
[ICO] Name Last modified Size Description
[PARENTDIR] Parent Directory -
[TXT] allkeys.asc 2019-10-30 20:25 14K
[ ] cloudera-manager-age..> 2019-10-30 20:25 10M
[ ] cloudera-manager-dae..> 2019-10-30 20:28 1.1G
[ ] cloudera-manager-ser..> 2019-10-30 20:25 11K
[ ] cloudera-manager-ser..> 2019-10-30 20:25 11K
[ ] enterprise-debuginfo..> 2019-10-30 20:25 14M
[ ] oracle-j2sdk1.8-1.8...> 2019-10-30 20:26 176M
[DIR] repodata/ ----》这个是需要检查的，如果没有，将来是会报错的，必须检查
安装JDK
[hadoop@manager cloudera-manager]$ java -version
openjdk version "1.8.0_222-ea"
OpenJDK Runtime Environment (build 1.8.0_222-ea-b03)
OpenJDK 64-Bit Server VM (build 25.222-b03, mixed mode)
[hadoop@manager cloudera-manager]$ rpm -qa | grep java
java-1.7.0-openjdk-headless-1.7.0.221-2.6.18.1.el7.x86_64
javamail-1.4.6-8.el7.noarch
tzdata-java-2019b-1.el7.noarch
java-1.8.0-openjdk-1.8.0.222.b03-1.el7.x86_64
java-1.8.0-openjdk-headless-1.8.0.222.b03-1.el7.x86_64
python-javapackages-3.4.1-11.el7.noarch
javapackages-tools-3.4.1-11.el7.noarch
java-1.7.0-openjdk-1.7.0.221-2.6.18.1.el7.x86_64
javassist-3.16.1-10.el7.noarch
原来安装的统统都删除掉，但是noarch文件可以不用删除
[hadoop@manager cloudera-manager]$
sudo rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.221-2.6.18.1.el7.x86_64 \
&& sudo rpm -e --nodeps java-1.8.0-openjdk-1.8.0.222.b03-1.el7.x86_64 \
&& sudo rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.222.b03-1.el7.x86_64 \
&& sudo rpm -e --nodeps java-1.7.0-openjdk-1.7.0.221-2.6.18.1.el7.x86_64
[hadoop@manager cloudera-manager]$ sudo rpm -ivh /var/www/html/cloudera-manager/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
warning: /var/www/html/cloudera-manager/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:oracle-j2sdk1.8-1.8.0+update181-1################################# [100%]
[hadoop@manager jdk1.8.0_181-cloudera]$ pwd
/usr/java/jdk1.8.0_181-cloudera
[hadoop@manager jdk1.8.0_181-cloudera]$ sudo vi /etc/profile
# set java environment
export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:${PATH}
[hadoop@manager jdk1.8.0_181-cloudera]$ source /etc/profile
[hadoop@manager jdk1.8.0_181-cloudera]$ java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
安装mysql
[hadoop@manager ~]$ sudo yum -y remove mariadb-libs
Loaded plugins: fastestmirror, langpacks
Resolving Dependencies
--> Running transaction check
---> Package mariadb-libs.x86_64 1:5.5.64-1.el7 will be erased
--> Processing Dependency: libmysqlclient.so.18()(64bit) for package: perl-DBD-MySQL-4.023-6.el7.x86_64
--> Processing Dependency: libmysqlclient.so.18()(64bit) for package: 1:net-snmp-agent-libs-5.7.2-43.el7.x86_64
--> Processing Dependency: libmysqlclient.so.18()(64bit) for package: 2:postfix-2.10.1-7.el7.x86_64
--> Processing Dependency: libmysqlclient.so.18()(64bit) for package: 1:net-snmp-5.7.2-43.el7.x86_64
--> Processing Dependency: libmysqlclient.so.18()(64bit) for package: 1:qt-mysql-4.8.7-3.el7_6.x86_64
--> Processing Dependency: libmysqlclient.so.18(libmysqlclient_18)(64bit) for package: perl-DBD-MySQL-4.023-6.el7.x86_64
--> Processing Dependency: libmysqlclient.so.18(libmysqlclient_18)(64bit) for package: 1:net-snmp-agent-libs-5.7.2-43.el7.x86_64
--> Processing Dependency: libmysqlclient.so.18(libmysqlclient_18)(64bit) for package: 2:postfix-2.10.1-7.el7.x86_64
--> Processing Dependency: libmysqlclient.so.18(libmysqlclient_18)(64bit) for package: 1:qt-mysql-4.8.7-3.el7_6.x86_64
--> Processing Dependency: mariadb-libs(x86-64) = 1:5.5.64-1.el7 for package: 1:mariadb-server-5.5.64-1.el7.x86_64
--> Processing Dependency: mariadb-libs(x86-64) = 1:5.5.64-1.el7 for package: 1:mariadb-5.5.64-1.el7.x86_64
--> Processing Dependency: mysql-libs for package: 1:net-snmp-5.7.2-43.el7.x86_64
--> Running transaction check
---> Package mariadb.x86_64 1:5.5.64-1.el7 will be erased
---> Package mariadb-server.x86_64 1:5.5.64-1.el7 will be erased
--> Processing Dependency: mariadb-server for package: akonadi-mysql-1.9.2-4.el7.x86_64
---> Package net-snmp.x86_64 1:5.7.2-43.el7 will be erased
---> Package net-snmp-agent-libs.x86_64 1:5.7.2-43.el7 will be erased
--> Processing Dependency: libnetsnmpagent.so.31()(64bit) for package: keepalived-1.3.5-16.el7.x86_64
--> Processing Dependency: libnetsnmpmibs.so.31()(64bit) for package: keepalived-1.3.5-16.el7.x86_64
---> Package perl-DBD-MySQL.x86_64 0:4.023-6.el7 will be erased
---> Package postfix.x86_64 2:2.10.1-7.el7 will be erased
---> Package qt-mysql.x86_64 1:4.8.7-3.el7_6 will be erased
--> Running transaction check
---> Package akonadi-mysql.x86_64 0:1.9.2-4.el7 will be erased
---> Package keepalived.x86_64 0:1.3.5-16.el7 will be erased
--> Finished Dependency Resolution
Dependencies Resolved
===================================================================================================================================================================================================================
Package Arch Version Repository Size
===================================================================================================================================================================================================================
Removing:
mariadb-libs x86_64 1:5.5.64-1.el7 @anaconda 4.4 M
Removing for dependencies:
akonadi-mysql x86_64 1.9.2-4.el7 @anaconda 9.6 k
keepalived x86_64 1.3.5-16.el7 @anaconda 1.0 M
mariadb x86_64 1:5.5.64-1.el7 @anaconda 49 M
mariadb-server x86_64 1:5.5.64-1.el7 @anaconda 58 M
net-snmp x86_64 1:5.7.2-43.el7 @anaconda 882 k
net-snmp-agent-libs x86_64 1:5.7.2-43.el7 @anaconda 2.1 M
perl-DBD-MySQL x86_64 4.023-6.el7 @anaconda 323 k
postfix x86_64 2:2.10.1-7.el7 @anaconda 12 M
qt-mysql x86_64 1:4.8.7-3.el7_6 @anaconda 74 k
Transaction Summary
===================================================================================================================================================================================================================
Remove 1 Package (+9 Dependent packages)
Installed size: 128 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
** Found 1 pre-existing rpmdb problem(s), 'yum check' output follows:
icedtea-web-1.7.1-1.el7.x86_64 has missing requires of java-1.8.0-openjdk
Erasing : akonadi-mysql-1.9.2-4.el7.x86_64 1/10
Erasing : 1:net-snmp-5.7.2-43.el7.x86_64 2/10
Erasing : 1:mariadb-server-5.5.64-1.el7.x86_64 3/10
Erasing : 1:mariadb-5.5.64-1.el7.x86_64 4/10
Erasing : perl-DBD-MySQL-4.023-6.el7.x86_64 5/10
Erasing : 1:qt-mysql-4.8.7-3.el7_6.x86_64 6/10
Erasing : 2:postfix-2.10.1-7.el7.x86_64 7/10
Erasing : keepalived-1.3.5-16.el7.x86_64 8/10
Erasing : 1:net-snmp-agent-libs-5.7.2-43.el7.x86_64 9/10
Erasing : 1:mariadb-libs-5.5.64-1.el7.x86_64 10/10
Verifying : 1:mariadb-libs-5.5.64-1.el7.x86_64 1/10
Verifying : 1:mariadb-5.5.64-1.el7.x86_64 2/10
Verifying : 1:qt-mysql-4.8.7-3.el7_6.x86_64 3/10
Verifying : keepalived-1.3.5-16.el7.x86_64 4/10
Verifying : akonadi-mysql-1.9.2-4.el7.x86_64 5/10
Verifying : 2:postfix-2.10.1-7.el7.x86_64 6/10
Verifying : perl-DBD-MySQL-4.023-6.el7.x86_64 7/10
Verifying : 1:net-snmp-agent-libs-5.7.2-43.el7.x86_64 8/10
Verifying : 1:mariadb-server-5.5.64-1.el7.x86_64 9/10
Verifying : 1:net-snmp-5.7.2-43.el7.x86_64 10/10
Removed:
mariadb-libs.x86_64 1:5.5.64-1.el7
Dependency Removed:
akonadi-mysql.x86_64 0:1.9.2-4.el7 keepalived.x86_64 0:1.3.5-16.el7 mariadb.x86_64 1:5.5.64-1.el7 mariadb-server.x86_64 1:5.5.64-1.el7 net-snmp.x86_64 1:5.7.2-43.el7
net-snmp-agent-libs.x86_64 1:5.7.2-43.el7 perl-DBD-MySQL.x86_64 0:4.023-6.el7 postfix.x86_64 2:2.10.1-7.el7 qt-mysql.x86_64 1:4.8.7-3.el7_6
Complete!
[hadoop@manager ~]$ cd /home/hadoop/Downloads/cloudera/mysql/;tar -xvf mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar
mysql-community-libs-5.7.27-1.el7.x86_64.rpm
mysql-community-embedded-devel-5.7.27-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.27-1.el7.x86_64.rpm
mysql-community-devel-5.7.27-1.el7.x86_64.rpm
mysql-community-embedded-compat-5.7.27-1.el7.x86_64.rpm
mysql-community-common-5.7.27-1.el7.x86_64.rpm
mysql-community-client-5.7.27-1.el7.x86_64.rpm
mysql-community-server-5.7.27-1.el7.x86_64.rpm
mysql-community-test-5.7.27-1.el7.x86_64.rpm
mysql-community-embedded-5.7.27-1.el7.x86_64.rpm
[hadoop@manager mysql]$ pwd
/home/hadoop/Downloads/cloudera/mysql
[hadoop@manager ~]$
sudo yum install -y libaio \
&& sudo rpm -ivh mysql-community-common-5.7.27-1.el7.x86_64.rpm \
&& sudo rpm -ivh mysql-community-libs-5.7.27-1.el7.x86_64.rpm \
&& sudo rpm -ivh mysql-community-client-5.7.27-1.el7.x86_64.rpm \
&& sudo rpm -ivh mysql-community-server-5.7.27-1.el7.x86_64.rpm \
&& sudo rpm -ivh mysql-community-libs-compat-5.7.27-1.el7.x86_64.rpm
[root@manager ~]# echo character-set-server=utf8 >> /etc/my.cnf
这里的数据库密码【123456Aa.】,比较重要，后面都会用到这个密码的
数据库名称也是比较重要的，后面也会要求填写这些数据库名称的（也就是在安装CDH之前，就要想好要安装哪些hadoop组件，一个组件，一个mysql库）
[hadoop@manager mysql]$
(
cat <<EOF
set password for root@localhost = password('123456Aa.');
grant all privileges on *.* to 'root'@'%' identified by '123456Aa.';
flush privileges;
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456Aa.';
SHOW DATABASES;
EOF
) >> /home/hadoop/Downloads/cloudera/c.sql
[root@manager ~]# systemctl enable mysqld && systemctl start mysqld && grep password /var/log/mysqld.log | sed 's/.*\(............\)$/\1/'
dqfwl;rFO4>o --》首次登录的密码
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY '123xxxxxx';
Query OK, 0 rows affected (0.01 sec)
mysql> exit;
mysql> source /home/hadoop/Downloads/cloudera/c.sql
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 1 row affected (0.01 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
+--------------------+
| Database |
+--------------------+
| information_schema |
| amon |
| hue |
| metastore |
| mysql |
| nav |
| navms |
| oozie |
| performance_schema |
| rman |
| scm |
| sentry |
| sys |
+--------------------+
13 rows in set (0.00 sec)
mysql>
准备jdbc的驱动程序
[hadoop@manager mysql-jdbc]$
sudo mkdir -p /usr/share/java/ \
&& cd /home/hadoop/Downloads/cloudera/mysql-jdbc/;tar -zxvf mysql-connector-java-5.1.47.tar.gz \
&& sudo cp /home/hadoop/Downloads/cloudera/mysql-jdbc/mysql-connector-java-5.1.47/mysql-connector-java-5.1.47-bin.jar /usr/share/java/mysql-connector-java.jar
所有机器更换国内的阿里yum镜像（这里有坑，不是master机器做就可以了，是所有机器都要做）
[root@manager ~]# mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bk
[root@manager ~]# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
[root@manager ~]#
(
cat <<EOF
[cloudera-manager]
name=Cloudera Manager 6.3.0
baseurl=http://192.168.88.31/cloudera-manager/
gpgcheck=0
enabled=1
EOF
) >> /etc/yum.repos.d/cloudera-manager.repo \
&& yum clean all \
&& yum makecache
[root@manager ~]# vi /etc/yum.repos.d/cloudera-manager.repo
[root@manager ~]# cat /etc/yum.repos.d/cloudera-manager.repo
安装Cloudera Manager
[hadoop@manager ~]$ sudo yum install -y cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package cloudera-manager-agent.x86_64 0:6.3.0-1281944.el7 will be installed
--> Processing Dependency: python-psycopg2 for package: cloudera-manager-agent-6.3.0-1281944.el7.x86_64
--> Processing Dependency: openssl-devel for package: cloudera-manager-agent-6.3.0-1281944.el7.x86_64
--> Processing Dependency: mod_ssl for package: cloudera-manager-agent-6.3.0-1281944.el7.x86_64
--> Processing Dependency: MySQL-python for package: cloudera-manager-agent-6.3.0-1281944.el7.x86_64
--> Processing Dependency: /lib/lsb/init-functions for package: cloudera-manager-agent-6.3.0-1281944.el7.x86_64
--> Processing Dependency: libpq.so.5()(64bit) for package: cloudera-manager-agent-6.3.0-1281944.el7.x86_64
---> Package cloudera-manager-daemons.x86_64 0:6.3.0-1281944.el7 will be installed
---> Package cloudera-manager-server.x86_64 0:6.3.0-1281944.el7 will be installed
--> Running transaction check
---> Package MySQL-python.x86_64 0:1.2.5-1.el7 will be installed
---> Package mod_ssl.x86_64 1:2.4.6-90.el7.centos will be installed
---> Package openssl-devel.x86_64 1:1.0.2k-19.el7 will be installed
--> Processing Dependency: zlib-devel(x86-64) for package: 1:openssl-devel-1.0.2k-19.el7.x86_64
--> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-19.el7.x86_64
---> Package postgresql-libs.x86_64 0:9.2.24-1.el7_5 will be installed
---> Package python-psycopg2.x86_64 0:2.5.1-4.el7 will be installed
---> Package redhat-lsb-core.x86_64 0:4.1-27.el7.centos.1 will be installed
--> Processing Dependency: redhat-lsb-submod-security(x86-64) = 4.1-27.el7.centos.1 for package: redhat-lsb-core-4.1-27.el7.centos.1.x86_64
--> Processing Dependency: spax for package: redhat-lsb-core-4.1-27.el7.centos.1.x86_64
--> Processing Dependency: /usr/sbin/sendmail for package: redhat-lsb-core-4.1-27.el7.centos.1.x86_64
--> Running transaction check
---> Package krb5-devel.x86_64 0:1.15.1-37.el7_7.2 will be installed
--> Processing Dependency: libkadm5(x86-64) = 1.15.1-37.el7_7.2 for package: krb5-devel-1.15.1-37.el7_7.2.x86_64
--> Processing Dependency: krb5-libs(x86-64) = 1.15.1-37.el7_7.2 for package: krb5-devel-1.15.1-37.el7_7.2.x86_64
--> Processing Dependency: libverto-devel for package: krb5-devel-1.15.1-37.el7_7.2.x86_64
--> Processing Dependency: libselinux-devel for package: krb5-devel-1.15.1-37.el7_7.2.x86_64
--> Processing Dependency: libcom_err-devel for package: krb5-devel-1.15.1-37.el7_7.2.x86_64
--> Processing Dependency: keyutils-libs-devel for package: krb5-devel-1.15.1-37.el7_7.2.x86_64
---> Package postfix.x86_64 2:2.10.1-7.el7 will be installed
---> Package redhat-lsb-submod-security.x86_64 0:4.1-27.el7.centos.1 will be installed
---> Package spax.x86_64 0:1.5.2-13.el7 will be installed
---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed
--> Running transaction check
---> Package keyutils-libs-devel.x86_64 0:1.5.8-3.el7 will be installed
---> Package krb5-libs.x86_64 0:1.15.1-37.el7_6 will be updated
---> Package krb5-libs.x86_64 0:1.15.1-37.el7_7.2 will be an update
---> Package libcom_err-devel.x86_64 0:1.42.9-16.el7 will be installed
---> Package libkadm5.x86_64 0:1.15.1-37.el7_7.2 will be installed
---> Package libselinux-devel.x86_64 0:2.5-14.1.el7 will be installed
--> Processing Dependency: libsepol-devel(x86-64) >= 2.5-10 for package: libselinux-devel-2.5-14.1.el7.x86_64
--> Processing Dependency: pkgconfig(libsepol) for package: libselinux-devel-2.5-14.1.el7.x86_64
--> Processing Dependency: pkgconfig(libpcre) for package: libselinux-devel-2.5-14.1.el7.x86_64
---> Package libverto-devel.x86_64 0:0.2.5-4.el7 will be installed
--> Running transaction check
---> Package libsepol-devel.x86_64 0:2.5-10.el7 will be installed
---> Package pcre-devel.x86_64 0:8.32-17.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
===================================================================================================================================================================================================================
Package Arch Version Repository Size
===================================================================================================================================================================================================================
Installing:
cloudera-manager-agent x86_64 6.3.0-1281944.el7 cloudera-manager 10 M
cloudera-manager-daemons x86_64 6.3.0-1281944.el7 cloudera-manager 1.1 G
cloudera-manager-server x86_64 6.3.0-1281944.el7 cloudera-manager 11 k
Installing for dependencies:
MySQL-python x86_64 1.2.5-1.el7 base 90 k
keyutils-libs-devel x86_64 1.5.8-3.el7 base 37 k
krb5-devel x86_64 1.15.1-37.el7_7.2 updates 272 k
libcom_err-devel x86_64 1.42.9-16.el7 base 32 k
libkadm5 x86_64 1.15.1-37.el7_7.2 updates 178 k
libselinux-devel x86_64 2.5-14.1.el7 base 187 k
libsepol-devel x86_64 2.5-10.el7 base 77 k
libverto-devel x86_64 0.2.5-4.el7 base 12 k
mod_ssl x86_64 1:2.4.6-90.el7.centos base 112 k
openssl-devel x86_64 1:1.0.2k-19.el7 base 1.5 M
pcre-devel x86_64 8.32-17.el7 base 480 k
postfix x86_64 2:2.10.1-7.el7 base 2.4 M
postgresql-libs x86_64 9.2.24-1.el7_5 base 234 k
python-psycopg2 x86_64 2.5.1-4.el7 base 132 k
redhat-lsb-core x86_64 4.1-27.el7.centos.1 base 38 k
redhat-lsb-submod-security x86_64 4.1-27.el7.centos.1 base 15 k
spax x86_64 1.5.2-13.el7 base 260 k
zlib-devel x86_64 1.2.7-18.el7 base 50 k
Updating for dependencies:
krb5-libs x86_64 1.15.1-37.el7_7.2 updates 805 k
Transaction Summary
===================================================================================================================================================================================================================
Install 3 Packages (+18 Dependent packages)
Upgrade ( 1 Dependent package)
Total download size: 1.1 G
Downloading packages:
No Presto metadata available for updates
(1/22): cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm | 10 MB 00:00:00
(2/22): cloudera-manager-server-6.3.0-1281944.el7.x86_64.rpm | 11 kB 00:00:00
(3/22): MySQL-python-1.2.5-1.el7.x86_64.rpm | 90 kB 00:00:00
(4/22): keyutils-libs-devel-1.5.8-3.el7.x86_64.rpm | 37 kB 00:00:00
(5/22): libcom_err-devel-1.42.9-16.el7.x86_64.rpm | 32 kB 00:00:00
(6/22): krb5-devel-1.15.1-37.el7_7.2.x86_64.rpm | 272 kB 00:00:00
(7/22): krb5-libs-1.15.1-37.el7_7.2.x86_64.rpm | 805 kB 00:00:00
(8/22): libkadm5-1.15.1-37.el7_7.2.x86_64.rpm | 178 kB 00:00:00
(9/22): libsepol-devel-2.5-10.el7.x86_64.rpm | 77 kB 00:00:00
(10/22): libselinux-devel-2.5-14.1.el7.x86_64.rpm | 187 kB 00:00:00
(11/22): libverto-devel-0.2.5-4.el7.x86_64.rpm | 12 kB 00:00:00
(12/22): mod_ssl-2.4.6-90.el7.centos.x86_64.rpm | 112 kB 00:00:00
(13/22): pcre-devel-8.32-17.el7.x86_64.rpm | 480 kB 00:00:00
(14/22): openssl-devel-1.0.2k-19.el7.x86_64.rpm | 1.5 MB 00:00:00
(15/22): postgresql-libs-9.2.24-1.el7_5.x86_64.rpm | 234 kB 00:00:00
(16/22): python-psycopg2-2.5.1-4.el7.x86_64.rpm | 132 kB 00:00:00
(17/22): postfix-2.10.1-7.el7.x86_64.rpm | 2.4 MB 00:00:00
(18/22): redhat-lsb-core-4.1-27.el7.centos.1.x86_64.rpm | 38 kB 00:00:00
(19/22): redhat-lsb-submod-security-4.1-27.el7.centos.1.x86_64.rpm | 15 kB 00:00:00
(20/22): zlib-devel-1.2.7-18.el7.x86_64.rpm | 50 kB 00:00:00
(21/22): spax-1.5.2-13.el7.x86_64.rpm | 260 kB 00:00:00
(22/22): cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm | 1.1 GB 00:00:31
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 37 MB/s | 1.1 GB 00:00:31
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
** Found 1 pre-existing rpmdb problem(s), 'yum check' output follows:
icedtea-web-1.7.1-1.el7.x86_64 has missing requires of java-1.8.0-openjdk
Updating : krb5-libs-1.15.1-37.el7_7.2.x86_64 1/23
Installing : postgresql-libs-9.2.24-1.el7_5.x86_64 2/23
Installing : cloudera-manager-daemons-6.3.0-1281944.el7.x86_64 3/23
Installing : python-psycopg2-2.5.1-4.el7.x86_64 4/23
Installing : libkadm5-1.15.1-37.el7_7.2.x86_64 5/23
Installing : spax-1.5.2-13.el7.x86_64 6/23
Installing : libsepol-devel-2.5-10.el7.x86_64 7/23
Installing : 1:mod_ssl-2.4.6-90.el7.centos.x86_64 8/23
Installing : 2:postfix-2.10.1-7.el7.x86_64 9/23
Installing : zlib-devel-1.2.7-18.el7.x86_64 10/23
Installing : libverto-devel-0.2.5-4.el7.x86_64 11/23
Installing : pcre-devel-8.32-17.el7.x86_64 12/23
Installing : libselinux-devel-2.5-14.1.el7.x86_64 13/23
Installing : MySQL-python-1.2.5-1.el7.x86_64 14/23
Installing : libcom_err-devel-1.42.9-16.el7.x86_64 15/23
Installing : keyutils-libs-devel-1.5.8-3.el7.x86_64 16/23
Installing : krb5-devel-1.15.1-37.el7_7.2.x86_64 17/23
Installing : 1:openssl-devel-1.0.2k-19.el7.x86_64 18/23
Installing : redhat-lsb-submod-security-4.1-27.el7.centos.1.x86_64 19/23
Installing : redhat-lsb-core-4.1-27.el7.centos.1.x86_64 20/23
Installing : cloudera-manager-agent-6.3.0-1281944.el7.x86_64 21/23
Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-agent.service to /usr/lib/systemd/system/cloudera-scm-agent.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/supervisord.service to /usr/lib/systemd/system/supervisord.service.
Installing : cloudera-manager-server-6.3.0-1281944.el7.x86_64 22/23
Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-server.service to /usr/lib/systemd/system/cloudera-scm-server.service.
Cleanup : krb5-libs-1.15.1-37.el7_6.x86_64 23/23
Verifying : redhat-lsb-submod-security-4.1-27.el7.centos.1.x86_64 1/23
Verifying : keyutils-libs-devel-1.5.8-3.el7.x86_64 2/23
Verifying : krb5-devel-1.15.1-37.el7_7.2.x86_64 3/23
Verifying : libcom_err-devel-1.42.9-16.el7.x86_64 4/23
Verifying : MySQL-python-1.2.5-1.el7.x86_64 5/23
Verifying : pcre-devel-8.32-17.el7.x86_64 6/23
Verifying : libverto-devel-0.2.5-4.el7.x86_64 7/23
Verifying : zlib-devel-1.2.7-18.el7.x86_64 8/23
Verifying : 2:postfix-2.10.1-7.el7.x86_64 9/23
Verifying : cloudera-manager-daemons-6.3.0-1281944.el7.x86_64 10/23
Verifying : cloudera-manager-server-6.3.0-1281944.el7.x86_64 11/23
Verifying : cloudera-manager-agent-6.3.0-1281944.el7.x86_64 12/23
Verifying : krb5-libs-1.15.1-37.el7_7.2.x86_64 13/23
Verifying : python-psycopg2-2.5.1-4.el7.x86_64 14/23
Verifying : 1:mod_ssl-2.4.6-90.el7.centos.x86_64 15/23
Verifying : 1:openssl-devel-1.0.2k-19.el7.x86_64 16/23
Verifying : libsepol-devel-2.5-10.el7.x86_64 17/23
Verifying : spax-1.5.2-13.el7.x86_64 18/23
Verifying : libkadm5-1.15.1-37.el7_7.2.x86_64 19/23
Verifying : libselinux-devel-2.5-14.1.el7.x86_64 20/23
Verifying : redhat-lsb-core-4.1-27.el7.centos.1.x86_64 21/23
Verifying : postgresql-libs-9.2.24-1.el7_5.x86_64 22/23
Verifying : krb5-libs-1.15.1-37.el7_6.x86_64 23/23
Installed:
cloudera-manager-agent.x86_64 0:6.3.0-1281944.el7 cloudera-manager-daemons.x86_64 0:6.3.0-1281944.el7 cloudera-manager-server.x86_64 0:6.3.0-1281944.el7
Dependency Installed:
MySQL-python.x86_64 0:1.2.5-1.el7 keyutils-libs-devel.x86_64 0:1.5.8-3.el7 krb5-devel.x86_64 0:1.15.1-37.el7_7.2 libcom_err-devel.x86_64 0:1.42.9-16.el7
libkadm5.x86_64 0:1.15.1-37.el7_7.2 libselinux-devel.x86_64 0:2.5-14.1.el7 libsepol-devel.x86_64 0:2.5-10.el7 libverto-devel.x86_64 0:0.2.5-4.el7
mod_ssl.x86_64 1:2.4.6-90.el7.centos openssl-devel.x86_64 1:1.0.2k-19.el7 pcre-devel.x86_64 0:8.32-17.el7 postfix.x86_64 2:2.10.1-7.el7
postgresql-libs.x86_64 0:9.2.24-1.el7_5 python-psycopg2.x86_64 0:2.5.1-4.el7 redhat-lsb-core.x86_64 0:4.1-27.el7.centos.1 redhat-lsb-submod-security.x86_64 0:4.1-27.el7.centos.1
spax.x86_64 0:1.5.2-13.el7 zlib-devel.x86_64 0:1.2.7-18.el7
Dependency Updated:
krb5-libs.x86_64 0:1.15.1-37.el7_7.2
Complete!
[hadoop@manager ~]$
配置本地Parcel存储库
[root@manager ~]# mv /home/hadoop/Downloads/cloudera/parcel/* /opt/cloudera/parcel-repo
[root@manager ~]# cd /opt/cloudera/parcel-repo
[root@manager parcel-repo]# pwd
/opt/cloudera/parcel-repo
[root@manager ~]# sha1sum CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel | awk '{ print $1 }' > CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel.sha
[root@manager ~]# chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/*
初始化scm库
[root@manager parcel-repo]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm 123456Aa.
JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing: /usr/java/jdk1.8.0_181-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Thu Oct 31 00:33:50 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
[ main] DbCommandExecutor INFO Successfully connected to database.
All done, your SCM database is configured correctly!
启动CM
[root@manager ~]#
systemctl stop cloudera-scm-server \
&& systemctl start cloudera-scm-server \
&& sleep 2 \
&& tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log | grep "INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server"
1分钟等待后，如果出现：
2019-10-31 00:39:00,113 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
则代表CM启动成功了
访问---》【http://192.168.88.31:7180】
账号密码为admin/admin
在整个启动过程中，可以新开一个窗口，然后tail -f看看日志有啥错误
[root@manager ~]# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
如果在中间出现问题，要先
[root@manager ~]# systemctl stop cloudera-scm-server
再运行上面的启动命令。数据库密码【123456Aa.】
初始化CM
如果在中间报异常了，那么基本上就需要做回滚和清理动作了（还不如重新来个新的操作系统来得简单）
【一】、Cluster Basics
给个人看得懂的集群名字即可
【二】、Accept JDK License
接受，不接受走不下去
【三】、Enter Login Credentials
root用户可以安装（sudo用户？？？？）安装 Cloudera 包需要有主机的 root 访问权限。此安装程序将通过 SSH 连接到您的主机，然后直接以 root 用户身份登录，或者以另一个具有变为 root 用户的无密码 sudo/pbrun 权限的用户身份登录。
【四】、Specify Hosts
192.168.88.[31-34]，可以通过这样的表达式来搜索多台机器
【五】、Select Repository
填入本地的yum库---》http://192.168.88.31/cloudera-manager/
CDH-6.3.0-1.cdh6.3.0.p0.1279813，这里我们需要明确我们自己安装的版本，因为系统会把比较新的版本都显示出来，如果选择错了，就是从网络下载了，那个时间就超长了
【六】、Install Agents
耗时10、20分钟的样子，由于页面是js写的，因此不要刷新浏览器。如果出错了，根据错误修正接节点的问题，然后retry。如果我们在这里放弃了，基本上这个操作系统也就废掉了（删除清理，估计也是非常麻烦的）
【七】、Install Parcels
耗时几个小时，distubuted哪里会刷新传递分发的速度 xxM/s。这个阶段磁盘占用率非常高。页面上是有简单的进度条显示整体进度的。没有到100%，continue的按钮是点击不了的。
【八】、Inspect Cluster
点击Inspect Network Performance按钮
点击Inspect Hosts按钮，可以参考这里：https://blog.csdn.net/qq_39680564/article/details/100013275，相关的报错汇总
主机健康检查、I understand the risks, let me continue with cluster setup.，点选这个走下去。
各个组件的验证
【选择安装的服务】
impala、hdfs、hbase等组件
【自行分配角色至主机】
这里官方文档中，有按照规模建议指定机器的分配方式
【测试数据库链接】
这里的数据库实例名，用户名、密码都是上面的脚本创建的，这里是直接使用已经创建好的库，不能乱填的，要按照上面的创建mysql库的脚本来进行填写。
常见报错1：JDBC driver cannot be found. Unable to find the JDBC database jar on host
[hadoop@manager mysql-jdbc]$
sudo mkdir -p /usr/share/java/ \
&& cd /home/hadoop/Downloads/cloudera/mysql-jdbc/;tar -zxvf mysql-connector-java-5.1.47.tar.gz \
&& sudo cp /home/hadoop/Downloads/cloudera/mysql-jdbc/mysql-connector-java-5.1.47/mysql-connector-java-5.1.47-bin.jar /usr/share/java/mysql-connector-java.jar
常见报错2:Cloudera 建议将 /proc/sys/vm/swappiness 设置为最大值 10
临时生效：
sysctl vm.swappiness=10 && cat /proc/sys/vm/swappiness
echo 'vm.swappiness=10'>> /etc/sysctl.conf
永久生效：
echo 'vm.swappiness=10'>> /etc/sysctl.conf
常见报错3:已启用透明大页面压缩，可能会导致重大性能问题。
临时生效：
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local
echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local
永久生效：
echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local
echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local
【配置实例的安装位置】
kudu的配置参数
Kudu Master WAL Directory:/kudumaster/wal
Kudu Master Data Directories:/kudumaster/data
Kudu Tablet Server WAL Directory:/kudutablet/wal
Kudu Tablet Server Data Directories:/kudutablet/data
HDFS 块大小 128
接受的 DataNode 失败的卷:0
DataNode 数据目录 /dfs/dn
NameNode 数据目录 /dfs/nn
HDFS 检查点目录 /dfs/snn
Host Monitor 存储目录 /var/lib/cloudera-host-monitor
Service Monitor 存储目录 /var/lib/cloudera-service-monitor
NodeManager 本地目录 /yarn/nm
数据目录 /var/lib/zookeeper
事务日志目录 /var/lib/zookeeper
【安装服务】
各种错误，可以参照：
https://www.cnblogs.com/felixzh/p/9082344.html
https://blog.csdn.net/sardtass/article/details/93723610
来进行问题解决。解决后，不停重试安装即可。
【安装完成成功提示】
关闭所有服务：
在页面Cluster集群的cluster1后面有个下拉三角，点击出下拉列表后，点击stop，这个stop也就是把集群中的所有服务，全部定制
在Cloudera Management Service后面有个下拉三角，点击出下拉列表后，点击stop，这个stop也就是把集群中的所有服务，全部定制
仅仅在主机上运行
[root@master cloudera]# service cloudera-scm-server stop
Redirecting to /bin/systemctl stop cloudera-scm-server.service
每台机器都要运行这个指令
[root@master cloudera]# service cloudera-scm-agent stop
Redirecting to /bin/systemctl stop cloudera-scm-agent.service
[root@manager ~]#
service cloudera-scm-server stop \
&& service cloudera-scm-agent stop
重新启动整个集群:
service cloudera-scm-server start
所有都需要运行service cloudera-scm-agent start
先需要启动CM
[root@manager ~]#
systemctl stop cloudera-scm-server \
&& systemctl start cloudera-scm-server \
&& sleep 2 \
&& tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log | grep "INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server"
[root@manager ~]#
service cloudera-scm-server restart \
&& service cloudera-scm-agent restart
在Cloudera Management Service后面有个下拉三角，点击出下拉列表后，点击stop，这个start也就是把集群中的所有服务，全部定制
在页面Cluster集群的cluster1后面有个下拉三角，点击出下拉列表后，点击stop，这个start也就是把集群中的所有服务，全部定制
service cloudera-scm-server status
service cloudera-scm-agent status
[root@manager ~]#
service cloudera-scm-server status \
&& service cloudera-scm-agent status
