共计 9906 个字符，预计需要花费 25 分钟才能阅读完成。
RHEL6.5 上使用 Ambari 快速搭建 Hadoop 集群
ip hostname
10.7.1.1 master.coke
10.7.1.2 dn1.coke
10.7.1.3 dn2.coke
10.7.1.4 dn3.coke
10.7.1.5 dn4.coke
1,ssh
master.coke 生成 pub key 作为中控机，添加到每台机器包括 master.coke 的 /root/.ssh/authorized_keys
2,hostname
对照表格更改 hadoop 集群的每台机器的 hostname 和 /etc/sysconfig/network HOSTNAME=XXXX
3,ntp
启动 ntp 服务保证各台机器时钟同步
4,repo
https://cwiki.apache.org/confluence/display/AMBARI/Install+Ambari+1.7.0+from+Public+Repositories
cd /etc/yum.repos.d/
wget http://public-repo-1.hortonworks.com/ambari/CentOS6/1.x/updates/1.7.0/ambari.repo
yum install ambari-server
yum install MySQL-server mysql
提前建立库和授权
mysql>grant all on ambari.* to‘ambari’@’localhost‘identified by“175an.com”;
mysql> flush privileges;
[@master.coke yum.repos.d]# yum install ambari-server
Loaded plugins: product-id, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
6ASU5-updates | 1.1 kB 00:00
HighAvailability | 3.9 kB 00:00
LoadBalancer | 3.9 kB 00:00
ResilientStorage | 3.9 kB 00:00
ScalableFileSystem | 3.9 kB 00:00
Server | 3.9 kB 00:00
Updates-ambari-1.7.0 | 2.9 kB 00:00
ambari-1.x | 1.3 kB 00:00
epel | 1.1 kB 00:00
Setting up Install Process
Resolving Dependencies
–> Running transaction check
—> Package ambari-server.noarch 0:1.7.0-169 will be installed
–> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.7.0-169.noarch
–> Running transaction check
—> Package postgresql-server.x86_64 0:8.4.20-1.el6_5 will be installed
–> Processing Dependency: postgresql(x86-64) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Processing Dependency: postgresql-libs(x86-64) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Running transaction check
—> Package postgresql.x86_64 0:8.4.20-1.el6_5 will be installed
—> Package postgresql-libs.x86_64 0:8.4.20-1.el6_5 will be installed
–> Finished Dependency ResolutionDependencies Resolved
=================================================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================================================
Installing:
ambari-server noarch 1.7.0-169 Updates-ambari-1.7.0 96 M
Installing for dependencies:
postgresql x86_64 8.4.20-1.el6_5 6ASU5-updates 2.6 M
postgresql-libs x86_64 8.4.20-1.el6_5 6ASU5-updates 201 k
postgresql-server x86_64 8.4.20-1.el6_5 6ASU5-updates 3.4 M
Transaction Summary
=================================================================================================================================================================================
Install 4 Package(s)
Total download size: 102 M
Installed size: 152 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): ambari-server-1.7.0-169.noarch.rpm | 96 MB 00:14
(2/4): postgresql-8.4.20-1.el6_5.x86_64.rpm | 2.6 MB 00:00
(3/4): postgresql-libs-8.4.20-1.el6_5.x86_64.rpm | 201 kB 00:00
(4/4): postgresql-server-8.4.20-1.el6_5.x86_64.rpm | 3.4 MB 00:00
———————————————————————————————————————————————————————————
Total 6.7 MB/s | 102 MB 00:15
warning: rpmts_HdrFromFdno: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY
Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Importing GPG key 0x07513CAD:
Userid:“Jenkins (HDP Builds) <jenkin@hortonworks.com>”
From : http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : postgresql-libs-8.4.20-1.el6_5.x86_64 1/4
Installing : postgresql-8.4.20-1.el6_5.x86_64 2/4
Installing : postgresql-server-8.4.20-1.el6_5.x86_64 3/4
Installing : ambari-server-1.7.0-169.noarch 4/4
Verifying : postgresql-server-8.4.20-1.el6_5.x86_64 1/4
Verifying : postgresql-libs-8.4.20-1.el6_5.x86_64 2/4
Verifying : postgresql-8.4.20-1.el6_5.x86_64 3/4
Verifying : ambari-server-1.7.0-169.noarch 4/4
Installed:
ambari-server.noarch 0:1.7.0-169
Dependency Installed:
postgresql.x86_64 0:8.4.20-1.el6_5 postgresql-libs.x86_64 0:8.4.20-1.el6_5 postgresql-server.x86_64 0:8.4.20-1.el6_5
Complete!
ambari-server setup
Using Python /usr/bin/python2.6
Setup ambari-server
Checking SELinux…
SELinux status is‘disabled’
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership…
Checking firewall…
Checking JDK…
[1] – Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[2] – Oracle JDK 1.6 + Java Cryptography Extension (JCE) Policy Files 6
[3] – Custom JDK
==============================================================================
Enter choice (1):
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.htmland not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
JDK distribution size is 142376665 bytes
jdk-7u67-linux-x64.tar.gz… 100% (135.8 MB of 135.8 MB)
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
Installing JDK to /usr/jdk64
Successfully installed JDK to /usr/jdk64/jdk1.7.0_67
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/UnlimitedJCEPolicyJDK7.zip to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
Completing setup…
Configuring database…
Enter advanced database configuration [y/n] (n)? y
==============================================================================
Choose one of the following options:
[1] – PostgreSQL (Embedded)
[2] – Oracle
[3] – MySQL
[4] – PostgreSQL
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database Name (ambari):
Username (ambari):
Enter Database Password (bigdata):
WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
Press <enter> to continue.
ERROR: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
ERROR: Exiting with exit code -1.
REASON: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
yum install -y mysql-connector-java # resolve the error
重新来过
[@master.coke 6Server]# ambari-server setup
Using python /usr/bin/python2.6
Setup ambari-server
Checking SELinux…
SELinux status is‘disabled’
Ambari-server daemon is configured to run under user‘root’. Change this setting [y/n] (n)?
Adjusting ambari-server permissions and ownership…
Checking firewall…
Checking JDK…
Do you want to change Oracle JDK [y/n] (n)?
Completing setup…
Configuring database…
Enter advanced database configuration [y/n] (n)? y
==============================================================================
Choose one of the following options:
[1] – PostgreSQL (Embedded)
[2] – Oracle
[3] – MySQL
[4] – PostgreSQL
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database Name (ambari):
Username (ambari): amb
Enter Database Password (bigdata):
Re-enter password:
Copying JDBC drivers to server resources…
Configuring remote database connection properties…
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)?
Copying JDBC drivers to server resources…
Extracting system views…
…
Adjusting ambari-server permissions and ownership…
Ambari Server‘setup’completed successfully.
导入数据库
mysql ambari -u175an -p < /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
grant all on ambari.* to“amb@localhost”identified by“amb123″;
Admin Name : admin
Cluster Name : coke
Total Hosts : 6 (6 new)
Repositories:
RedHat5 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/centos5/2.x/GA/2.2.0.0
redhat5 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos5
redhat6 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/centos6/2.x/GA/2.2.0.0
redhat6 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6
SUSE11 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/GA/2.2.0.0
suse11 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/suse11sp3
Ubuntu12 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/GA/2.2.0.0
ubuntu12 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12
Services:
HDFS DataNode : 6 hosts
NameNode : master.coke
SNameNode : dn1.coke
YARN + MapReduce2 App Timeline Server : master.coke
NodeManager : 6 hosts
ResourceManager : master.coke
Tez Clients : 6 hosts
Nagios Server : master.coke
Administrator : nagiosadmin / (qilinli@sohu-inc.com)
Ganglia Server : master.coke
Hive Metastore : master.coke
HiveServer2 : master.coke
WebHCat Server : master.coke
Database : MySQL (Existing MySQL Database)
HBase Master : dn1.coke
RegionServer : 6 hosts
Pig Clients : 6 hosts
Sqoop Clients : 6 hosts
Oozie Server : dn1.coke
Database : MySQL (Existing MySQL Database)
ZooKeeper Server : 3 hosts
Falcon Server : dn3.coke
Storm DRPC Server : dn1.coke
Nimbus : dn1.coke
UI Server : dn1.coke
Supervisor : 6 hosts
Flume Flume : 2 hosts
Slider Clients : 6 hosts
Knox Gateway : dn1.coke
Kafka Broker : 3 hosts
CentOS 安装和配置 Hadoop2.2.0 http://www.linuxidc.com/Linux/2014-01/94685.htm
Ubuntu 13.04 上搭建 Hadoop 环境 http://www.linuxidc.com/Linux/2013-06/86106.htm
Ubuntu 12.10 +Hadoop 1.2.1 版本集群配置 http://www.linuxidc.com/Linux/2013-09/90600.htm
Ubuntu 上搭建 Hadoop 环境（单机模式 + 伪分布模式）http://www.linuxidc.com/Linux/2013-01/77681.htm
Ubuntu 下 Hadoop 环境的配置 http://www.linuxidc.com/Linux/2012-11/74539.htm
单机版搭建 Hadoop 环境图文教程详解 http://www.linuxidc.com/Linux/2012-02/53927.htm
搭建 Hadoop 环境（在 Winodws 环境下用虚拟机虚拟两个 Ubuntu 系统进行搭建）http://www.linuxidc.com/Linux/2011-12/48894.htm
更多 Hadoop 相关信息见 Hadoop 专题页面 http://www.linuxidc.com/topicnews.aspx?tid=13
