Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

             Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

                                        作者:尹正杰

版权声明:原创作品,谢绝转载!否则将追究法律责任。

  我之前分享过关于离线方式部署Cloudera Manager5.15.1的笔记。本篇博客给大家分享基于yum源的方式部署Cloudera Manager。由于我的台式机内存有限,本次实验就采用三台虚拟机进行实验。

一.集群大数据生态圈集群环境准备(温馨提示:最好保持集群中的每台机器的硬件配置和软件配置都一致。即每台机器最好都配置一致!)

1>.3台主机配置以及主机名与IP对应关系

[root@node101 ~]# cat /etc/redhat-release
CentOS Linux release 7.2. (Core)
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# uname -r
3.10.-.el7.x86_64
[root@node101 ~]#
[root@node101 ~]# uname -m
x86_64
[root@node101 ~]#
[root@node101 ~]# hostname -i
172.30.1.101
[root@node101 ~]#
[root@node101 ~]# free -m
total used free shared buff/cache available
Mem:
Swap:
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# more /etc/hosts | grep yinzhengjie
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# hostname
node101.yinzhengjie.org.cn
[root@node101 ~]#

Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

2>.配置Windows的主机名与IP地址的映射关系

  Windows的主机名配置文件路径在:“C:\Windows\System32\drivers\etc”,在该目录下有一个hosts的配置文件,我们对其进行编辑如下:Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

3>.创建部署集群的管理用户并配置sudo权限为无秘钥登录(温馨提示:这个步骤可以不做,咱们可以直接使用root用户来部署即可,我推荐使用root用户,用普通用户的话对某些敏感文件都需要加sudo关键字,尽管我这里教大家配置无需密码验证我觉得麻烦,因此这里只是告诉大家一个配置管理用户的方法!而接下来的实验我都是以root用户来配置的!如果你非要用普通用户的话,需要按照我下面的步骤配置即可,如果遇到权限问题,直接使用sudo提权即可!)

[root@node101 ~]# useradd yinzhengjie                                              #创建一个普通用户
[root@node101 ~]#
[root@node101 ~]# echo "yinzhengjie" | passwd yinzhengjie --stdin #为用户创建密码
Changing password for user yinzhengjie.
passwd: all authentication tokens updated successfully.
[root@node101 ~]#
[root@node101 ~]# chmod u+w /etc/sudoers #添加些的权限
[root@node101 ~]#
[root@node101 ~]# vi /etc/sudoers #具体修改的内容参考下图
[root@node101 ~]#
[root@node101 ~]# gpasswd -a yinzhengjie wheel #将你的用户(普通用户)调整至wheel用户组里面,这样就可以以后用户执行sudo就不用输入密码了.
Adding user yinzhengjie to group wheel
[root@node101 ~]#
[root@node101 ~]# chmod /etc/sudoers #将对文件写的权限关闭掉
[root@node101 ~]#
[root@node101 ~]# ll /etc/sudoers
-r--r----- root root Oct : /etc/sudoers
[root@node101 ~]#
[root@node101 ~]# id yinzhengjie         #查看“yinzhengjie”用户的id信息
uid=(yinzhengjie) gid=(yinzhengjie) groups=(yinzhengjie),(wheel)
[root@node101 ~]#
[root@node101 ~]# su yinzhengjie #接下来,我们进行测试
[yinzhengjie@node101 root]$
[yinzhengjie@node101 root]$ sudo useradd test #执行你想要执行的任何命令吧,只需要前面加个sudo,并且不需要输入root密码!
[yinzhengjie@node101 root]$ sudo userdel -r test
[yinzhengjie@node101 root]$

  创建“yinzhengjie”用户后,想要给这个用户配置管理员权限,就得使用vi编辑“/etc/sudoers”配置文件,并修改以下两处。

Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

4>.配置ssh无秘钥登录

[root@node101 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
eb:ee:5b:4b::d6::d4:2c:2d:a5:a3:b9:::4f root@node101.yinzhengjie.org.cn
The key's randomart image is:
+--[ RSA ]----+
| .=. |
| .o.+ |
| . Eo. |
| * = |
| + S o |
| . o o . |
| + . o |
| . o o . |
| o=.. |
+-----------------+
[root@node101 ~]#

使用root用户生成秘钥对([root@node101 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa)

[root@node101 ~]# ssh-copy-id root@node101.yinzhengjie.org.cn
The authenticity of host 'node101.yinzhengjie.org.cn (172.30.1.101)' can't be established.
ECDSA key fingerprint is b6::e8:e7::d4:c2:4c:e0::7e:9c:d8::d8:.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@node101.yinzhengjie.org.cn's password: Number of key(s) added: Now try logging into the machine, with: "ssh 'root@node101.yinzhengjie.org.cn'"
and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#

配置root用户与本机的无秘钥登录([root@node101 ~]# ssh-copy-id root@node101.yinzhengjie.org.cn)

[root@node101 ~]# ssh-copy-id root@node102.yinzhengjie.org.cn
The authenticity of host 'node102.yinzhengjie.org.cn (172.30.1.102)' can't be established.
ECDSA key fingerprint is b6::e8:e7::d4:c2:4c:e0::7e:9c:d8::d8:.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@node102.yinzhengjie.org.cn's password: Number of key(s) added: Now try logging into the machine, with: "ssh 'root@node102.yinzhengjie.org.cn'"
and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#

配置root用户与node102.yinzhengjie.org.cn的无秘钥登录([root@node101 ~]# ssh-copy-id root@node102.yinzhengjie.org.cn)

[root@node101 ~]# ssh-copy-id root@node103.yinzhengjie.org.cn
The authenticity of host 'node103.yinzhengjie.org.cn (172.30.1.103)' can't be established.
ECDSA key fingerprint is b6::e8:e7::d4:c2:4c:e0::7e:9c:d8::d8:.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@node103.yinzhengjie.org.cn's password: Number of key(s) added: Now try logging into the machine, with: "ssh 'root@node103.yinzhengjie.org.cn'"
and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#

配置root用户与node103.yinzhengjie.org.cn的无秘钥登录([root@node101 ~]# ssh-copy-id root@node103.yinzhengjie.org.cn)

5>.编写集群的管理脚本

[root@node101 ~]# vi /usr/local/bin/xcall.sh
[root@node101 ~]#
[root@node101 ~]# chmod +x /usr/local/bin/xcall.sh
[root@node101 ~]#
[root@node101 ~]# more `which xcall.sh`
#!/bin/bash
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie
#EMAIL:y1053419035@qq.com #判断用户是否传参
if [ $# -lt ];then
echo "请输入参数"
exit
fi #获取用户输入的命令
cmd=$@ for (( i=;i<=;i++ ))
do
#使终端变绿色
tput setaf
echo ============= node${i}.yinzhengjie.org.cn : $cmd ============
#使终端变回原来的颜色,即白灰色
tput setaf
#远程执行命令
ssh node${i}.yinzhengjie.org.cn $cmd
#判断命令是否执行成功
if [ $? == ];then
echo "命令执行成功"
fi
done
[root@node101 ~]#
[root@node101 ~]# xcall.sh ls -d /home/yinzhengjie/
============= node101.yinzhengjie.org.cn : ls -d /home/yinzhengjie/ ============
/home/yinzhengjie/
命令执行成功
============= node102.yinzhengjie.org.cn : ls -d /home/yinzhengjie/ ============
/home/yinzhengjie/
命令执行成功
============= node103.yinzhengjie.org.cn : ls -d /home/yinzhengjie/ ============
/home/yinzhengjie/
命令执行成功
[root@node101 ~]#

编辑批量执行命令的脚本([root@node101 ~]# vi /usr/local/bin/xcall.sh)

[root@node101 ~]# xcall.sh "yum -y install rsync"                                      #很显然,我已经安装成功啦!如果初次安装的话不会提示你已经安装成功哟!
============= node101.yinzhengjie.org.cn : yum -y install rsync ============
Loaded plugins: fastestmirror
Repodata is over weeks old. Install yum-cron? Or run: yum makecache fast
Determining fastest mirrors
* base: mirrors.huaweicloud.com
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
Package rsync-3.1.-.el7.x86_64 already installed and latest version
Nothing to do
命令执行成功
============= node102.yinzhengjie.org.cn : yum -y install rsync ============
Loaded plugins: fastestmirror
Repodata is over weeks old. Install yum-cron? Or run: yum makecache fast
Determining fastest mirrors
* base: mirror.bit.edu.cn
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
Package rsync-3.1.-.el7.x86_64 already installed and latest version
Nothing to do
命令执行成功
============= node103.yinzhengjie.org.cn : yum -y install rsync ============
Loaded plugins: fastestmirror
Repodata is over weeks old. Install yum-cron? Or run: yum makecache fast
Determining fastest mirrors
* base: mirrors.huaweicloud.com
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
Package rsync-3.1.-.el7.x86_64 already installed and latest version
Nothing to do
命令执行成功
[root@node101 ~]#

使用我们上面自定义脚本批量安装rsync服务([root@node1 ~]# xcall.sh "yum -y install rsync"

[root@node101 ~]# vi /usr/local/bin/xrsync.sh
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# chmod +x /usr/local/bin/xrsync.sh
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# more `which xrsync.sh`
#!/bin/bash
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie
#EMAIL:y1053419035@qq.com #判断用户是否传参
if [ $# -lt ];then
echo "请输入参数";
exit
fi #获取文件路径
file=$@ #获取子路径
filename=`basename $file` #获取父路径
dirpath=`dirname $file` #获取完整路径
cd $dirpath
fullpath=`pwd -P` #同步文件到DataNode
for (( i=;i<=;i++ ))
do
#使终端变绿色
tput setaf
echo =========== node${i}.yinzhengjie.org.cn : $file ===========
#使终端变回原来的颜色,即白灰色
tput setaf
#远程执行命令
rsync -lr $filename `whoami`@node${i}.yinzhengjie.org.cn:$fullpath
#判断命令是否执行成功
if [ $? == ];then
echo "命令执行成功"
fi
done
[root@node101 ~]#

编写同步集群配置文件的脚本([root@node101 ~]# more `which xrsync.sh`)

[root@node101 ~]# xrsync.sh /etc/hosts
=========== node102.yinzhengjie.org.cn : /etc/hosts ===========
命令执行成功
=========== node103.yinzhengjie.org.cn : /etc/hosts ===========
命令执行成功
[root@node101 ~]#

使用我们上面自定义的xrsync.sh同步数据([root@node101 ~]# xrsync.sh /etc/hosts)

6>.关闭集群的防火墙

[root@node101 ~]# xcall.sh systemctl stop firewalld
============= node101.yinzhengjie.org.cn : systemctl stop firewalld ============
命令执行成功
============= node102.yinzhengjie.org.cn : systemctl stop firewalld ============
命令执行成功
============= node103.yinzhengjie.org.cn : systemctl stop firewalld ============
命令执行成功
[root@node101 ~]#

关闭服务器的firewalld服务,立即生效无需重启操作系统([root@node101 ~]# xcall.sh systemctl stop firewalld)

[root@node101 ~]# xcall.sh systemctl disable firewalld
============= node101.yinzhengjie.org.cn : systemctl disable firewalld ============
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
命令执行成功
============= node102.yinzhengjie.org.cn : systemctl disable firewalld ============
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
命令执行成功
============= node103.yinzhengjie.org.cn : systemctl disable firewalld ============
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
命令执行成功
[root@node101 ~]#

禁用firewalld的开机自启([root@node101 ~]# xcall.sh systemctl disable firewalld)

7>.禁用集群的selinux

[root@node101 ~]# xcall.sh setenforce                                #很显然,我已经禁用了selinux,如果你没有禁用的话,selinux设置为0,也是临时的关闭哟!
============= node101.yinzhengjie.org.cn : setenforce ============
setenforce: SELinux is disabled
============= node102.yinzhengjie.org.cn : setenforce ============
setenforce: SELinux is disabled
============= node103.yinzhengjie.org.cn : setenforce ============
setenforce: SELinux is disabled
[root@node101 ~]#

关闭selinux服务的权限管理,无需重启操纵系统,立即生效([root@node101 ~]# xcall.sh setenforce 0 )

[root@node101 ~]# xcall.sh sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config
============= node101.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============
命令执行成功
============= node102.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============
命令执行成功
============= node103.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============
命令执行成功
[root@node101 ~]#

禁用selinux开机自启,需重启生效([root@node101 ~]# xcall.sh sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config)

8>.禁用IPV6(每台服务器都需要禁用ipv6,我下面就举例了一台的案例

>.编辑/etc/sysctl.conf配置,增加net.ipv6.conf.all.disable_ipv6=
[root@node101 ~]# cat /etc/sysctl.conf | grep -v ^#
[root@node101 ~]#
[root@node101 ~]# echo "#Add by yinzhengjie" >> /etc/sysctl.conf
[root@node101 ~]#
[root@node101 ~]# echo "net.ipv6.conf.all.disable_ipv6=1" >> /etc/sysctl.conf
You have new mail in /var/spool/mail/root
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# cat /etc/sysctl.conf | grep -v ^#
net.ipv6.conf.all.disable_ipv6=
[root@node101 ~]#
[root@node101 ~]# >.编辑/etc/sysconfig/network配置,增加 NETWORKING_IPV6=no
[root@node101 ~]# cat /etc/sysconfig/network
# Created by anaconda
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# cat /etc/sysconfig/network | grep -v ^#
[root@node101 ~]#
[root@node101 ~]# echo "#Add by yinzhengjie" >> /etc/sysconfig/network
You have new mail in /var/spool/mail/root
[root@node101 ~]#
[root@node101 ~]# echo "NETWORKING_IPV6=no" >> /etc/sysconfig/network
[root@node101 ~]#
[root@node101 ~]# cat /etc/sysconfig/network | grep -v ^#
NETWORKING_IPV6=no
[root@node101 ~]#
[root@node101 ~]# >.在配置网卡信息上再次禁用ipv6
[root@node101 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
BOOTPROTO=static
ONBOOT=yes
BONDING_OPTS="miimon=100 mode=4"
IPADDR=172.30.1.101
NETMASK=255.255.255.0
GATEWAY=172.30.1.254
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# echo NETWORKING_IPV6=no >> /etc/sysconfig/network-scripts/ifcfg-bond0
[root@node101 ~]#
[root@node101 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0 (我之所以修改这个配置文件是因为我的服务器双网卡做的事bound)
DEVICE=bond0
BOOTPROTO=static
ONBOOT=yes
BONDING_OPTS="miimon=100 mode=4"
IPADDR=172.30.1.101
NETMASK=255.255.255.0
GATEWAY=172.30.1.254
NETWORKING_IPV6=no
[root@node101 ~]# >.执行sysctl -p或者reboot重启命令
[root@node101 ~]# sysctl -p
net.ipv6.conf.all.disable_ipv6 =
[root@node101 ~]#

9>.卸载jdk

[root@node102 ~]# rpm -aq | grep java                  #查看操作系统下载的安装包
tzdata-java-2018e-.el7.noarch
[root@node102 ~]#
[root@node102 ~]# rpm -e --nodeps tzdata-java-2018e-.el7.noarch    #使用rpm工具卸载上面安装的软件
[root@node102 ~]#
[root@node102 ~]# rpm -aq | grep java                  #再次验证,是否卸载成功啦!
[root@node102 ~]#
[root@node102 ~]#

10>.设置文件打开数据和用户最大进程数(针对每个版本Cloudera版本参数设置可能会有一些小的的偏差,我这里给的是相对来说统统的配置。)

[root@node101 ~]# ulimit -n                                          #查看默认指定同一时间最多可开启的文件数,这个值显然是不合理的,我们必须得修改它!

[root@node103 ~]#
[root@node101 ~]# ulimit -u       #查看默认用户最多可开启的程序数目,这个貌似是根据你的机器配置会有相应的变化 [root@node101 ~]#
[root@node101 ~]# ulimit -a       #显示目前资源限制的设定
core file size (blocks, -c)          #core文件的最大值为100 blocks。
data seg size (kbytes, -d) unlimited      #进程的数据段可以任意大。
scheduling priority (-e)        #指定调度优先级
file size (blocks, -f) unlimited        #文件可以任意大。
pending signals (-i)          #最多有31152个待处理的信号。
max locked memory (kbytes, -l)      #一个任务锁住的物理内存的最大值为64KB。
max memory size (kbytes, -m) unlimited      #一个任务的常驻物理内存的最大值。
open files (-n)          #一个任务最多可以同时打开1024的文件。
pipe size ( bytes, -p)        #管道的最大空间为4096字节。
POSIX message queues (bytes, -q)    #POSIX的消息队列的最大值为819200字节。
real-time priority (-r)          #指定实时优先级
stack size (kbytes, -s)         #进程的栈的最大值为8192字节。
cpu time (seconds, -t) unlimited       #进程使用的CPU时间。
max user processes (-u)       #当前用户同时打开的进程(包括线程)的最大个数为31152。
virtual memory (kbytes, -v) unlimited      #没有限制进程的最大地址空间。
file locks (-x) unlimited         #所能锁住的文件的最大个数没有限制。
[root@node101 ~]#
[root@node101 ~]# vi /etc/security/limits.conf   #往里面配置相应的参数,我这里就做了6项优化
[root@node101 ~]# cat /etc/security/limits.conf | grep -v ^# | grep -v ^$
* soft nofile
* hard nofile
* soft nproc
* hard nproc unlimited
* soft memlock unlimited
* hard memlock unlimited
[root@node101 ~]#
[root@node101 ~]# reboot    #按照上面的配置完成后,需要重启操作系统 Connection closed by foreign host. Disconnected from remote host(node101.yinzhengjie.org.cn) at ::. Type `help' to learn how to use Xshell prompt.
[d:\~]$ Connecting to 172.30.1.101:...
Connection established.
To escape to local shell, press 'Ctrl+Alt+]'. Last login: Mon Oct :: from 172.30.1.1
[root@node101 ~]#
[root@node101 ~]# ulimit -a      #重启操作系统后,再次查看目前资源限制的设定,来我们一起找一下不同吧
core file size (blocks, -c)
data seg size (kbytes, -d) unlimited
scheduling priority (-e)
file size (blocks, -f) unlimited
pending signals (-i)
max locked memory (kbytes, -l) unlimited     #这个值发生变化了,之前是64,现在是没有限制啦!
max memory size (kbytes, -m) unlimited
open files (-n)         #这个值发生了变化,之前是1024!
pipe size ( bytes, -p)
POSIX message queues (bytes, -q)
real-time priority (-r)
stack size (kbytes, -s)
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited      #这个值也发生了变化,之前是31152,现在是没有限制
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[root@node101 ~]#

  想要了解更多ulimit的小技巧,可参考:https://www.cnblogs.com/yinzhengjie/p/9734489.html

11>.时间同步

  详情请参考:https://www.cnblogs.com/yinzhengjie/p/9480665.html

二.构建平台并搭建配置本地yum源

1>.Cloudera manager安装方式

  1.1>.基于bin文件方式安装

    下载地址:http://archive.cloudera.com/cm5/installer/

  1.2>.基于rpm包方式安装

    下载地址:http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/

  1.3>.基于yum源的方式安装(本篇博客采取的安装方式,因为这种方式安装很简单)

    下载地址:http://archive.cloudera.com/cm5/repo-as-tarball/

2>.启动Http服务(当然,你也可以选取nginx来提点http服务)

[root@node101 ~]# yum -y install httpd
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.huaweicloud.com
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 :2.4.-.el7.centos. will be installed
--> Processing Dependency: httpd-tools = 2.4.-.el7.centos. for package: httpd-2.4.-.el7.centos..x86_64
--> Processing Dependency: /etc/mime.types for package: httpd-2.4.-.el7.centos..x86_64
--> Processing Dependency: libaprutil-.so.()(64bit) for package: httpd-2.4.-.el7.centos..x86_64
--> Processing Dependency: libapr-.so.()(64bit) for package: httpd-2.4.-.el7.centos..x86_64
--> Running transaction check
---> Package apr.x86_64 :1.4.-.el7_4. will be installed
---> Package apr-util.x86_64 :1.5.-.el7 will be installed
---> Package httpd-tools.x86_64 :2.4.-.el7.centos. will be installed
---> Package mailcap.noarch :2.1.-.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved =========================================================================================================================================================
Package Arch Version Repository Size
=========================================================================================================================================================
Installing:
httpd x86_64 2.4.-.el7.centos. updates 2.7 M
Installing for dependencies:
apr x86_64 1.4.-.el7_4. base k
apr-util x86_64 1.5.-.el7 base k
httpd-tools x86_64 2.4.-.el7.centos. updates k
mailcap noarch 2.1.-.el7 base k Transaction Summary
=========================================================================================================================================================
Install Package (+ Dependent packages) Total download size: 3.0 M
Installed size: M
Downloading packages:
(/): apr-1.4.-.el7_4..x86_64.rpm | kB ::
(/): httpd-tools-2.4.-.el7.centos..x86_64.rpm | kB ::
(/): apr-util-1.5.-.el7.x86_64.rpm | kB ::
(/): mailcap-2.1.-.el7.noarch.rpm | kB ::
(/): httpd-2.4.-.el7.centos..x86_64.rpm | 2.7 MB ::
---------------------------------------------------------------------------------------------------------------------------------------------------------
Total 6.5 MB/s | 3.0 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : apr-1.4.-.el7_4..x86_64 /
Installing : apr-util-1.5.-.el7.x86_64 /
Installing : httpd-tools-2.4.-.el7.centos..x86_64 /
Installing : mailcap-2.1.-.el7.noarch /
Installing : httpd-2.4.-.el7.centos..x86_64 /
Verifying : mailcap-2.1.-.el7.noarch /
Verifying : httpd-tools-2.4.-.el7.centos..x86_64 /
Verifying : apr-util-1.5.-.el7.x86_64 /
Verifying : apr-1.4.-.el7_4..x86_64 /
Verifying : httpd-2.4.-.el7.centos..x86_64 / Installed:
httpd.x86_64 :2.4.-.el7.centos. Dependency Installed:
apr.x86_64 :1.4.-.el7_4. apr-util.x86_64 :1.5.-.el7 httpd-tools.x86_64 :2.4.-.el7.centos. mailcap.noarch :2.1.-.el7 Complete!
[root@node101 ~]#

安装web服务器([root@node101 ~]# yum -y install httpd)

[root@node101 ~]# systemctl start httpd                      #启动Apache服务
[root@node101 ~]#
[root@node101 ~]# systemctl enable httpd #设置Apache的开启自启
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@node101 ~]#

启动Apache服务([root@node101 ~]# systemctl start httpd)

  启动成功后,我们可以访问webUI来查看是否真实启动,如下图:

Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

3>.下载CM资源文件并安装到Apache的webUI路径中

[root@node101 ~]# mkdir -pv /yinzhengjie/download/ && cd /yinzhengjie/download
mkdir: created directory ‘/yinzhengjie’
mkdir: created directory ‘/yinzhengjie/download/’
[root@node101 download]#
[root@node101 download]# yum -y install wget #安装下载工具wget,如果你已经装了这一步骤可以省略
[root@node101 download]#
[root@node101 download]# wget http://archive.cloudera.com/cm5/repo-as-tarball/5.15.1/cm5.15.1-centos7.tar.gz #下载cm的安装包
---- ::-- http://archive.cloudera.com/cm5/repo-as-tarball/5.15.1/cm5.15.1-centos7.tar.gz
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.108.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.108.167|:... connected.
HTTP request sent, awaiting response... OK
Length: (990M) [application/x-tar]
Saving to: ‘cm5.15.1-centos7.tar.gz’ %[=============================================================================================================>] ,,, 902KB/s in 30m 30s -- :: ( KB/s) - ‘cm5.15.1-centos7.tar.gz’ saved [/] [root@node101 download]# ll
total
-rw-r--r-- root root Aug : cm5.15.1-centos7.tar.gz
[root@node101 download]#
[root@node101 download]# mkdir -pv /yinzhengjie/softwares
mkdir: created directory ‘/yinzhengjie/softwares’
[root@node101 download]#
[root@node101 download]# tar -zxf cm5.15.1-centos7.tar.gz -C /yinzhengjie/softwares/ #将下载后的文件进行解压
[root@node101 download]#
[root@node101 download]# cd /yinzhengjie/softwares/cm/5.15./RPMS/x86_64/ #进入到解压后的CM的rpm工具包中
[root@node101 x86_64]# ll
total
-rw-rw-r-- Jul : cloudera-manager-agent-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-daemons-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-server-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-server-db--5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : enterprise-debuginfo-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : generated_index.html
-rw-rw-r-- Jul : jdk-6u31-linux-amd64.rpm
-rw-rw-r-- Jul : oracle-j2sdk1.-1.7.+update67-.x86_64.rpm
[root@node101 x86_64]#
[root@node101 x86_64]# mv ./* /var/www/html/cm-5.15.1/ #将资源安装到Apache的webUI目录中
[root@node101 x86_64]#
[root@node101 x86_64]# cd /var/www/html/cm-5.15.1/
[root@node101 cm-5.15.1]#
[root@node101 cm-5.15.1]# ll
total 1016664
-rw-rw-r-- 1 1106 592 9857532 Jul 31 18:05 cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm
-rw-rw-r-- 1 1106 592 787321332 Jul 31 18:05 cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm
-rw-rw-r-- 1 1106 592 8712 Jul 31 18:05 cloudera-manager-server-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm
-rw-rw-r-- 1 1106 592 10616 Jul 31 18:05 cloudera-manager-server-db-2-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm
-rw-rw-r-- 1 1106 592 30603956 Jul 31 18:05 enterprise-debuginfo-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm
-rw-rw-r-- 1 1106 592 1066 Jul 31 18:12 generated_index.html
-rw-rw-r-- 1 1106 592 71204325 Jul 31 18:05 jdk-6u31-linux-amd64.rpm
-rw-rw-r-- 1 1106 592 142039186 Jul 31 18:05 oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
[root@node101 cm-5.15.1]#

  配置成功后,我们可以在wen目录上看到相应的信息。

Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

4>.创建repodata数据文件

[root@node101 cm-5.15.]# yum -y install createrepo
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.huaweicloud.com
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package createrepo.noarch :0.9.-.el7 will be installed
--> Processing Dependency: python-deltarpm for package: createrepo-0.9.-.el7.noarch
--> Processing Dependency: libxml2-python for package: createrepo-0.9.-.el7.noarch
--> Processing Dependency: deltarpm for package: createrepo-0.9.-.el7.noarch
--> Running transaction check
---> Package deltarpm.x86_64 :3.6-.el7 will be installed
---> Package libxml2-python.x86_64 :2.9.-.el7_2. will be installed
---> Package python-deltarpm.x86_64 :3.6-.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved =========================================================================================================================================================
Package Arch Version Repository Size
=========================================================================================================================================================
Installing:
createrepo noarch 0.9.-.el7 base k
Installing for dependencies:
deltarpm x86_64 3.6-.el7 base k
libxml2-python x86_64 2.9.-.el7_2. base k
python-deltarpm x86_64 3.6-.el7 base k Transaction Summary
=========================================================================================================================================================
Install Package (+ Dependent packages) Total download size: k
Installed size: 2.0 M
Downloading packages:
(/): deltarpm-3.6-.el7.x86_64.rpm | kB ::
(/): python-deltarpm-3.6-.el7.x86_64.rpm | kB ::
(/): createrepo-0.9.-.el7.noarch.rpm | kB ::
(/): libxml2-python-2.9.-.el7_2..x86_64.rpm | kB ::
---------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | kB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : deltarpm-3.6-.el7.x86_64 /
Installing : python-deltarpm-3.6-.el7.x86_64 /
Installing : libxml2-python-2.9.-.el7_2..x86_64 /
Installing : createrepo-0.9.-.el7.noarch /
Verifying : createrepo-0.9.-.el7.noarch /
Verifying : libxml2-python-2.9.-.el7_2..x86_64 /
Verifying : deltarpm-3.6-.el7.x86_64 /
Verifying : python-deltarpm-3.6-.el7.x86_64 / Installed:
createrepo.noarch :0.9.-.el7 Dependency Installed:
deltarpm.x86_64 :3.6-.el7 libxml2-python.x86_64 :2.9.-.el7_2. python-deltarpm.x86_64 :3.6-.el7 Complete!
[root@node101 cm-5.15.]#

安装创建repodata的工具([root@node101 cm-5.15.1]# yum -y install createrepo)

[root@node101 cm-5.15.]# pwd
/var/www/html/cm-5.15.
[root@node101 cm-5.15.]# ll
total
-rw-rw-r-- Jul : cloudera-manager-agent-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-daemons-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-server-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-server-db--5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : enterprise-debuginfo-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : generated_index.html
-rw-rw-r-- Jul : jdk-6u31-linux-amd64.rpm
-rw-rw-r-- Jul : oracle-j2sdk1.-1.7.+update67-.x86_64.rpm
[root@node101 cm-5.15.]#
[root@node101 cm-5.15.]# createrepo .            #生成数据目录,我们看在webUI中查看相应的数据目录
Spawning worker with pkgs
Spawning worker with pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@node101 cm-5.15.]#
[root@node101 cm-5.15.]# ll
total
-rw-rw-r-- Jul : cloudera-manager-agent-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-daemons-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-server-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : cloudera-manager-server-db--5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : enterprise-debuginfo-5.15.-.cm5151.p0..el7.x86_64.rpm
-rw-rw-r-- Jul : generated_index.html
-rw-rw-r-- Jul : jdk-6u31-linux-amd64.rpm
-rw-rw-r-- Jul : oracle-j2sdk1.-1.7.+update67-.x86_64.rpm
drwxr-xr-x root root Oct : repodata                            #温馨提示:注意看这里,发现没有?这个目录是新生成的!
[root@node101 cm-5.15.]#

  再次查看webUI是否有repodata目录生成:

Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

5>.配置集群各机器本地yum源

[root@node101 ~]# cd /etc/yum.repos.d/
[root@node101 yum.repos.d]#
[root@node101 yum.repos.d]# ll #下面是默认的repo源
total
-rw-r--r--. root root Dec CentOS-Base.repo
-rw-r--r--. root root Dec CentOS-CR.repo
-rw-r--r--. root root Dec CentOS-Debuginfo.repo
-rw-r--r--. root root Dec CentOS-fasttrack.repo
-rw-r--r--. root root Dec CentOS-Media.repo
-rw-r--r--. root root Dec CentOS-Sources.repo
-rw-r--r--. root root Dec CentOS-Vault.repo
[root@node101 yum.repos.d]#
[root@node101 yum.repos.d]# mkdir repo-backup
[root@node101 yum.repos.d]# mv ./*.repo repo-backup/ #备份默认的repo源,为了防止你操作失败可以还原之前的状态!
[root@node101 yum.repos.d]# ll
total 4
drwxr-xr-x 2 root root 4096 Oct 1 15:50 repo-backup
[root@node101 yum.repos.d]#
[root@node101 yum.repos.d]# vi yinzhengjie-cm.repo
[root@node101 yum.repos.d]# ll
total 8
drwxr-xr-x 2 root root 4096 Oct 1 15:50 repo-backup
-rw-r--r-- 1 root root 204 Oct 1 15:58 yinzhengjie-cm.repo
[root@node101 yum.repos.d]#
[root@node101 yum.repos.d]# more yinzhengjie-cm.repo #这个配置参考自:可参考:http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/cloudera-manager.repo
[yinzhengjie-cm]
# Packages for Cloudera Manager, Version 5, on RedHat or CentOS 7 x86_64
name=Cloudera Manager
baseurl=https://node101.yinzhengjie.org.cn/cm-5.15.1
enabled = 1
gpgcheck = 0
[root@node101 yum.repos.d]#
[root@node101 yum.repos.d]# xrsync.sh /etc/yum.repos.d/yinzhengjie-cm.repo #将配置同步到其它两个节点中
=========== node102.yinzhengjie.org.cn : /etc/yum.repos.d/yinzhengjie-cm.repo ===========
命令执行成功
=========== node103.yinzhengjie.org.cn : /etc/yum.repos.d/yinzhengjie-cm.repo ===========
命令执行成功
[root@node101 yum.repos.d]#

三.Cloudera Manager服务安装详解

1>.Cloudera Manager服务架构设计

Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

  1.1>.Agent 1 .. .Agent n:

    所有的Agent相当于集群。Agent部署在每一台服务器上,Server的控制命令最终由Agent来完成,而Agent通过Python调用shell脚本来控制具体的启动或者停止的脚本执行。

  1.2>.Server

    相当于Cloudera Manager的主节点,它可以管理整个集群。它主要负责以下几件事情:

        第一:处理软件和服务器的逻辑关系;

        第二:处理服务的安装配置,启动,停止等;

        第三:控制对应的角色的启动和停止;

        第四:管理集群(每一台Agent)

  1.3>.Database

    需要关系型数据库存储数据。

  1.4>.Management Server

    对整个集群运行的硬件和软件做一些预警,监控,报告等。当你的图标没有出来的时候,你就得考虑它是否正常启动了。

  1.5>.Cloudera Repository

    Cloudera官网的支持,可以下载软件包。

  1.6>.Clients

    通过webUI或者调用对应的API来控制Server。如果你是通过webUI界面点击了某个操作,比如让某个集群停止或启动,首先这个指令会发送给Server端,然后Server端根据自己的记录去找相应的Agent执行相应的脚本来停止或者启动应用。默认Server端会想Agent发送五次指令,如果这五次指令Agent都没有相应,则Server端判定Agent端存在故障。

2>.安装PostgreSQL数据库

  PostgreSQL是Cloudera Manager默认使用的数据库。它负责存储信息内容包括:

    2.1>.保存平台所有的配置信息(比如对各个组件的参数配置等等),便于各个节点获取数据库的配置信息;

    2.2>.所有节点的监控状态(各个节点将自己的状态信息存储到数据库中,以便主服务从数据库中获取数据信息,从而达到监控的目的);

  数据库主要有以上两个功能,具体安装步骤如下:

[root@node101 packages]# yum -y install postgresql
Loaded plugins: fastestmirror
base | 3.6 kB ::
extras | 3.4 kB ::
updates | 3.4 kB ::
Loading mirror speeds from cached hostfile
* base: mirrors.tuna.tsinghua.edu.cn
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package postgresql.x86_64 :9.2.-.el7_5 will be installed
--> Processing Dependency: postgresql-libs(x86-) = 9.2.-.el7_5 for package: postgresql-9.2.-.el7_5.x86_64
--> Processing Dependency: libpq.so.()(64bit) for package: postgresql-9.2.-.el7_5.x86_64
--> Running transaction check
---> Package postgresql-libs.x86_64 :9.2.-.el7_5 will be installed
--> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================================================================
Package Arch Version Repository Size
======================================================================================================================================================================================================================
Installing:
postgresql x86_64 9.2.-.el7_5 updates 3.0 M
Installing for dependencies:
postgresql-libs x86_64 9.2.-.el7_5 updates k Transaction Summary
======================================================================================================================================================================================================================
Install Package (+ Dependent package) Total download size: 3.3 M
Installed size: M
Downloading packages:
(/): postgresql-libs-9.2.-.el7_5.x86_64.rpm | kB ::
(/): postgresql-9.2.-.el7_5.x86_64.rpm | 3.0 MB ::
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | 3.3 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : postgresql-libs-9.2.-.el7_5.x86_64 /
Installing : postgresql-9.2.-.el7_5.x86_64 /
Verifying : postgresql-libs-9.2.-.el7_5.x86_64 /
Verifying : postgresql-9.2.-.el7_5.x86_64 / Installed:
postgresql.x86_64 :9.2.-.el7_5 Dependency Installed:
postgresql-libs.x86_64 :9.2.-.el7_5 Complete!
[root@node101 packages]#

安装postgresql([root@node101 packages]# yum -y install postgresql)

3>.安装Cloudera manager Server

[root@node101 ~]# cd /yinzhengjie/download/
[root@node101 download]# wget http://archive.cloudera.com/cm5/installer/5.15.1/cloudera-manager-installer.bin
---- ::-- http://archive.cloudera.com/cm5/installer/5.15.1/cloudera-manager-installer.bin
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.72.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.72.167|:... connected.
HTTP request sent, awaiting response... OK
Length: (509K) [application/octet-stream]
Saving to: ‘cloudera-manager-installer.bin’ %[============================================================================================================================================================================>] , 599KB/s in .8s -- :: ( KB/s) - ‘cloudera-manager-installer.bin’ saved [/] [root@node101 download]# ll
total
-rw-r--r-- root root Aug : cloudera-manager-installer.bin
-rw-r--r-- root root Aug : cm5.15.1-centos7.tar.gz
[root@node101 download]#

下载bin文件([root@node101 download]# wget http://archive.cloudera.com/cm5/installer/5.15.1/cloudera-manager-installer.bin)

[root@node101 download]# pwd
/yinzhengjie/download
[root@node101 download]#
[root@node101 download]# ll
total
-rwxr-xr-x root root Aug : cloudera-manager-installer.bin
-rw-r--r-- root root Aug : cm5.15.1-centos7.tar.gz
[root@node101 download]#
[root@node101 download]# chmod +x cloudera-manager-installer.bin
[root@node101 download]#
[root@node101 download]# ./cloudera-manager-installer.bin --skip_repo_package=      #注意,后面的参数必须要写,目的是为了防止cm去官网下载repo,我们之前的操作依据将repo下载到本地了,这里跳过验证即可
[root@node101 download]#

  关于安装的详细步骤请参考:https://www.cnblogs.com/yinzhengjie/p/9735095.html。安装成功后,可以访问webUI界面,如下图:Hadoop生态圈-基于yum源的方式部署Cloudera Manager5.15.1

[root@node101 ~]# systemctl cloudera-scm-server status
Unknown operation 'cloudera-scm-server'.
[root@node101 ~]#
[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: active (exited) since Mon -- :: CST; 45min ago
Docs: man:systemd-sysv-generator()
Process: ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=/SUCCESS) Oct :: node101.yinzhengjie.org.cn systemd[]: Starting LSB: Cloudera SCM Server...
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: /etc/rc.d/init.d/cloudera-scm-server: line : pstree: command not found
Oct :: node101.yinzhengjie.org.cn su[]: (to cloudera-scm) root on none
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Starting cloudera-scm-server: [ OK ]
Oct :: node101.yinzhengjie.org.cn systemd[]: Started LSB: Cloudera SCM Server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@node101 ~]#

查看Cloudera manager Server端的运行状态([root@node101 ~]# systemctl cloudera-scm-server status)

[root@node101 ~]# systemctl cloudera-scm-server status
Unknown operation 'cloudera-scm-server'.
[root@node101 ~]#
[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: active (exited) since Mon -- :: CST; 45min ago
Docs: man:systemd-sysv-generator()
Process: ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=/SUCCESS) Oct :: node101.yinzhengjie.org.cn systemd[]: Starting LSB: Cloudera SCM Server...
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: /etc/rc.d/init.d/cloudera-scm-server: line : pstree: command not found
Oct :: node101.yinzhengjie.org.cn su[]: (to cloudera-scm) root on none
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Starting cloudera-scm-server: [ OK ]
Oct :: node101.yinzhengjie.org.cn systemd[]: Started LSB: Cloudera SCM Server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@node101 ~]#
[root@node101 ~]# systemctl stop cloudera-scm-server
[root@node101 ~]#
[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: inactive (dead) since Mon -- :: CST; 2s ago
Docs: man:systemd-sysv-generator()
Process: ExecStop=/etc/rc.d/init.d/cloudera-scm-server stop (code=exited, status=/SUCCESS)
Process: ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=/SUCCESS) Oct :: node101.yinzhengjie.org.cn systemd[]: Starting LSB: Cloudera SCM Server...
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: /etc/rc.d/init.d/cloudera-scm-server: line : pstree: command not found
Oct :: node101.yinzhengjie.org.cn su[]: (to cloudera-scm) root on none
Oct :: node101.yinzhengjie.org.cn systemd[]: Started LSB: Cloudera SCM Server.
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Starting cloudera-scm-server: [ OK ]
Oct :: node101.yinzhengjie.org.cn systemd[]: Stopping LSB: Cloudera SCM Server...
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: /etc/rc.d/init.d/cloudera-scm-server: line : pstree: command not found
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Stopping cloudera-scm-server: [ OK ]
Oct :: node101.yinzhengjie.org.cn systemd[]: Stopped LSB: Cloudera SCM Server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@node101 ~]#

停止Cloudera manager Server服务([root@node101 ~]# systemctl stop cloudera-scm-server)

[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: inactive (dead) since Mon -- :: CST; 2s ago
Docs: man:systemd-sysv-generator()
Process: ExecStop=/etc/rc.d/init.d/cloudera-scm-server stop (code=exited, status=/SUCCESS)
Process: ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=/SUCCESS) Oct :: node101.yinzhengjie.org.cn systemd[]: Starting LSB: Cloudera SCM Server...
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: /etc/rc.d/init.d/cloudera-scm-server: line : pstree: command not found
Oct :: node101.yinzhengjie.org.cn su[]: (to cloudera-scm) root on none
Oct :: node101.yinzhengjie.org.cn systemd[]: Started LSB: Cloudera SCM Server.
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Starting cloudera-scm-server: [ OK ]
Oct :: node101.yinzhengjie.org.cn systemd[]: Stopping LSB: Cloudera SCM Server...
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: /etc/rc.d/init.d/cloudera-scm-server: line : pstree: command not found
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Stopping cloudera-scm-server: [ OK ]
Oct :: node101.yinzhengjie.org.cn systemd[]: Stopped LSB: Cloudera SCM Server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@node101 ~]#
[root@node101 ~]# systemctl start cloudera-scm-server
[root@node101 ~]#
[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: active (exited) since Mon -- :: CST; 53s ago
Docs: man:systemd-sysv-generator()
Process: ExecStop=/etc/rc.d/init.d/cloudera-scm-server stop (code=exited, status=/SUCCESS)
Process: ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=/SUCCESS) Oct :: node101.yinzhengjie.org.cn systemd[]: Starting LSB: Cloudera SCM Server...
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: /etc/rc.d/init.d/cloudera-scm-server: line : pstree: command not found
Oct :: node101.yinzhengjie.org.cn su[]: (to cloudera-scm) root on none
Oct :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Starting cloudera-scm-server: [ OK ]
Oct :: node101.yinzhengjie.org.cn systemd[]: Started LSB: Cloudera SCM Server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@node101 ~]#

启动Cloudera manager Server服务([root@node101 ~]# systemctl start cloudera-scm-server)

4>.下载parcels包并配置到CM主服务器目录下(“/opt/cloudera/parcel-repo/”)

[root@node101 ~]# mkdir -pv /yinzhengjie/softwares/parcels
mkdir: created directory ‘/yinzhengjie/softwares/parcels’
[root@node101 ~]#
[root@node101 ~]# cd /yinzhengjie/softwares/parcels/
[root@node101 parcels]#
[root@node101 parcels]# ll
total
[root@node101 parcels]#
[root@node101 parcels]# wget http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 #下载CDH校验包
---- ::-- http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.108.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.108.167|:... connected.
HTTP request sent, awaiting response... OK
Length: [binary/octet-stream]
Saving to: ‘CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1’ %[=====================================================================================================>] --.-K/s in 0s -- :: (2.72 MB/s) - ‘CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1’ saved [/] [root@node101 parcels]#

下载CDH校验包([root@node101 parcels]# wget http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1)

[root@node101 parcels]#
[root@node101 parcels]# wget http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel #下载CDH安装包
---- ::-- http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.228.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.228.167|:... connected.
HTTP request sent, awaiting response... OK
Length: (.0G) [binary/octet-stream]
Saving to: ‘CDH-5.15.-.cdh5.15.1.p0.-el7.parcel’ %[=====================================================================================================>] 9.17 MB/s in 0s -- :: (3.07 MB/s) - ‘CDH-5.15.-.cdh5.15.1.p0.-el7.parcel’ saved [/] [root@node101 parcels]#
[root@node101 parcels]# ll
total
-rw-r--r-- root root Aug : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Sep : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1
[root@node101 parcels]#

下载CDH安装包([root@node101 parcels]# wget http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel )

[root@node101 parcels]# ll
total
-rw-r--r-- root root Aug : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Sep : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1
[root@node101 parcels]#
[root@node101 parcels]# mv CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1 CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
[root@node101 parcels]#
[root@node101 parcels]# ll
total
-rw-r--r-- root root Aug : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Sep : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
[root@node101 parcels]#

将校验包进行重命名操作([root@node101 parcels]# mv CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha)

[root@node101 parcels]# pwd
/yinzhengjie/softwares/parcels
[root@node101 parcels]# ll
total
-rw-r--r-- root root Aug : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Sep : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
[root@node101 parcels]#
[root@node101 parcels]# mv * /opt/cloudera/parcel-repo/ #将资源拷贝到parcel-repo目录下
[root@node101 parcels]#
[root@node101 parcels]# ll
total
[root@node101 parcels]#
[root@node101 parcels]#
[root@node101 parcels]# ll /opt/cloudera/parcel-repo/
total
-rw-r--r-- root root Aug : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Sep : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
[root@node101 parcels]#

5>.检查本地源并安装Cloudera manager agent端

  详情强参考:https://www.cnblogs.com/yinzhengjie/p/9735451.html

6>.

上一篇:thinkphp 常用的查询


下一篇:3.C#基础篇-->堆和栈