溫馨提示×
您好，登錄后才能下訂單哦！
點擊 登錄注冊 即表示同意《億速云用戶服務條款》
您好，登錄后才能下訂單哦！
這期內容當中小編將會給大家帶來有關如何實現Ambari大數據平臺集群利器的探索與實踐，文章內容豐富且以專業的角度為大家分析和敘述，閱讀完這篇文章希望大家可以有所收獲。
Ambari是 Apache Software Foundation 中的一個頂級項目，它是用來創建、管理、監視Hadoop整個生態圈的工具。 Ambari是分布式架構，主要由Ambari Server和Ambari Agent組成。
HDP是hortonworks的軟件棧，里面包含了hadoop生態系統的所有軟件項目，其實就是軟件包合集。
HDP-UTILS是工具類庫。各軟件版本對照表：
https://supportmatrix.hortonworks.com/
hadoop生態圈示圖： 
[root@ambari-server01 ~]# wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.1.0/ambari-2.7.1.0-centos7.tar.gz [root@ambari-server01 ~]# wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.1.0/HDP-3.0.1.0-centos7-rpm.tar.gz [root@ambari-server01 ~]# wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
#!/bin/bash # #服務器初始化腳本 # 關閉防火墻 systemctl stop firewalld systemctl disable firewalld # 關閉 SeLinux setenforce 0 sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config # 關閉 swap swapoff -a sed -i 's/.*swap.*/#&/' /etc/fstab # echo "1" >/proc/sys/net/bridge/bridge-nf-call-iptables # 安裝必要的調試程序 yum install -y vim lrzsz telnet net-tools tar wget yum install -y java-1.8.0-openjdk* # 安裝配置時間同步服務 yum install -y ntp systemctl start ntpd.service systemctl enable ntpd.service # 設置進程文件最大打開數量 echo "* soft nofile 65535" >> /etc/security/limits.conf echo "* hard nofile 65535" >> /etc/security/limits.conf # 設置系統總限制文件最大打開數量 echo 6553560 > /proc/sys/fs/file-max echo "fs.file-max = 6553560" >> /etc/sysctl.conf
# 三臺主機分別設置hostname [root@ambari-server01 ~]# hostnamectl set-hostname ambari-server01.test.com [root@ambari-node01 ~]# hostnamectl set-hostname ambari-node01.test.com [root@ambari-node02 ~]# hostnamectl set-hostname ambari-node02.test.com # 添加hosts解析（所有節點） [root@ambari-server01 ~]# cat >> /etc/hosts << EOF 192.168.2.111 ambari-server01.test.com 192.168.2.112 ambari-node01.test.com 192.168.2.113 ambari-node02.test.com EOF
# 在192.168.2.111節點執行： [root@ambari-server01 ~]# ssh-keygen [root@ambari-server01 ~]# ssh-copy-id ambari-server01.test.com [root@ambari-server01 ~]# ssh-copy-id ambari-node01.test.com [root@ambari-server01 ~]# ssh-copy-id ambari-node02.test.com [root@ambari-server01 ~]# scp -r .ssh/ ambari-node01.test.com:/root [root@ambari-server01 ~]# scp -r .ssh/ ambari-node02.test.com:/root
# 下載并安裝mysql
[root@ambari-server01 ~]# wget -i -c http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
[root@ambari-server01 ~]# yum -y install mysql-community-release-el7-5.noarch.rpm
[root@ambari-server01 ~]# yum -y install mysql-community-server
# 啟動mysql
[root@ambari-server01 ~]# systemctl start mysqld.service # 啟動mysql
[root@ambari-server01 ~]# systemctl status mysqld.service # 查看mysql狀態
[root@ambari-server01 ~]# systemctl enable mysqld.service # 開機自啟
# 設置mysql密碼
> set password for root@localhost = password('root123'); # 方法一，需要登陸mysql執行
# 創建ambari數據庫及用戶名和密碼
mysql> create database ambari character set utf8;
mysql> CREATE USER 'ambari'@'%'IDENTIFIED BY 'Ambari123';
mysql> GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'%';
mysql> GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'ambari-server01.test.com'IDENTIFIED BY 'Ambari123';
mysql> FLUSH PRIVILEGES;
# 創建hive數據庫及用戶名和密碼（如果不需要安裝hive服務，則不需要配置以下數據庫）
mysql> create database hive character set utf8;
mysql> CREATE USER 'hive'@'%'IDENTIFIED BY 'Hive123';
mysql> GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'%';
mysql> GRANT ALL PRIVILEGES ON hive.* TO 'ambari'@'ambari-server01.test.com'IDENTIFIED BY 'Hive123';
mysql> FLUSH PRIVILEGES;
# 如果需要安裝oozie、ranger服務，創建數據庫方法同上。
# 下載 mysql-connector-java-5.1.46-bin.jar包
# 復制到/var/lib/ambari-server/resources/mysql-connector-java-5.1.46-bin.jar目錄；
# 再復制到/usr/share/java/mysql-connector-java-5.1.46-bin.jar到這個目錄一份。# 安裝軟件 [root@ambari-server01 ~]# yum -y install yum-utils createrepo httpd # 創建目錄 [root@ambari-server01 ~]# mkdir -p /var/www/html/ambari/ [root@ambari-server01 ~]# mkdir -p /var/www/html/hdp/ [root@ambari-server01 ~]# mkdir -p /var/www/html/hdp/HDP-UTILS/ # 解壓tar包 [root@ambari-server01 ~]# tar -zxvf ambari-2.7.1.0-centos7.tar.gz -C /var/www/html/ambari/ [root@ambari-server01 ~]# tar -zxvf HDP-3.0.1.0-centos7-rpm.tar.gz -C /var/www/html/hdp/ [root@ambari-server01 ~]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp/HDP-UTILS/ # 啟動httpd [root@ambari-server01 ~]# systemctl start httpd [root@ambari-server01 ~]# systemctl enable httpd
**瀏覽器查看，如下即為正常**  # 配置本地repo [root@ambari-server01 ~]# wget -O /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.1.0/ambari.repo # 將下載的repo修改內容為如下 [root@ambari-server01 ~]# vim /etc/yum.repos.d/ambari.repo #VERSION_NUMBER=2.7.1.0-139 [ambari-2.7.1.0] #json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json name=ambari Version - ambari-2.7.1.0 baseurl=http://192.168.2.111/ambari/ambari/centos7/2.7.1.0-169 gpgcheck=1 gpgkey=http://192.168.2.111/ambari/ambari/centos7/2.7.1.0-169/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 # 配置HDP和HDP-UTILS的repo [root@ambari-server01 ~]# vim /etc/yum.repos.d/HDP.repo #VERSION_NUMBER=3.0.1.0-187 [HDP-3.0.1.0] name=HDP Version - HDP-3.0.1.0 baseurl=http://192.168.2.111/hdp/HDP/centos7 gpgcheck=1 gpgkey=http://192.168.2.111/hdp/HDP/centos7/3.0.1.0-187/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [HDP-UTILS-1.1.0.22] name=HDP-UTILS Version - HDP-UTILS-1.1.0.22 baseurl=http://192.168.2.111/hdp/HDP-UTILS/centos7/1.1.0.22 gpgcheck=1 gpgkey=http://192.168.2.111/hdp/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 # 將repo拷貝到其他節點 [root@ambari-server01 ~]# cd /etc/yum.repos.d/ [root@ambari-server01 ~]# scp ambari.repo HDP.repo root@192.168.2.112:/etc/yum.repos.d/ [root@ambari-server01 ~]# scp ambari.repo HDP.repo root@192.168.2.113:/etc/yum.repos.d/ # 生成本地源 [root@ambari-server01 ~]# createrepo /var/www/html/hdp/HDP/centos7/ [root@ambari-server01 ~]# createrepo /var/www/html/hdp/HDP-UTILS/
[root@ambari-server01 ~]# yum -y install ambari-server [root@ambari-server01 ~]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? y Enter user account for ambari-server daemon (root):root # 用戶 Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 2 # 選擇自定義jdk WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/ #jdk安裝路徑 Validating JDK on Ambari Server...done. Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... ============================================================================== Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB ============================================================================== Enter choice (1): 3 # 選擇安裝的mysql Hostname (localhost): ambari-server01.test.com # 配置hostname Port (3306): # 默認 Database name (ambari): # 默認 Username (ambari): # 默認 Enter Database Password (bigdata): # 輸入密碼 Re-enter password: Configuring ambari database... Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? y Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql # 此處需注意，啟動ambari之前需要執行此句 Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ambari-admin-2.7.3.0.139.jar .... Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it... Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. # 安裝成功
[root@ambari-server01 ~]# mysql -u ambari -pAmbari123 -h ambari-server01.test.com mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql; mysql> show tables;
[root@ambari-server01 ~]# ambari-server start
[root@ambari-node01 ~]# yum -y install ambari-agent
訪問地址：http://192.168.2.111:8080/ 默認賬號：admin 默認密碼：admin
進行web頁面配置時沒有截圖，以下截圖為網上下載，僅供參考
# 執行以下命令（ambari-server01節點） [root@ambari-server01 ~]# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java-5.1.46-bin.jar
等待啟動完畢，不用擔心警告，后期可以調整，搭建完成，可以在展示頁面進行查看集群狀態 
可以查看監控界面，可以看到大數據組件中出現錯誤，單個組件點開處理 
至此，搭建完成！
上述就是小編為大家分享的如何實現Ambari大數據平臺集群利器的探索與實踐了，如果剛好有類似的疑惑，不妨參照上述分析進行理解。如果想知道更多相關知識，歡迎關注億速云行業資訊頻道。
免責聲明：本站發布的內容（圖片、視頻和文字）以原創、轉載和分享為主，文章觀點不代表本網站立場，如果涉及侵權請聯系站長郵箱：is@yisu.com進行舉報，并提供相關證據，一經查實，將立刻刪除涉嫌侵權內容。
