阿里云-云小站(无限量代金券发放中)
【腾讯云】云服务器、云数据库、COS、CDN、短信等热卖云产品特惠抢购

RHEL6.5上使用Ambari快速搭建Hadoop集群

158次阅读
没有评论

共计 9906 个字符,预计需要花费 25 分钟才能阅读完成。

RHEL6.5 上使用 Ambari 快速搭建 Hadoop 集群

ip hostname
10.7.1.1 master.coke
10.7.1.2 dn1.coke
10.7.1.3 dn2.coke
10.7.1.4 dn3.coke
10.7.1.5 dn4.coke

1,ssh

master.coke 生成 pub key  作为中控机,添加到每台机器包括 master.coke 的  /root/.ssh/authorized_keys

2,hostname

对照表格更改 hadoop 集群的每台机器的 hostname 和 /etc/sysconfig/network HOSTNAME=XXXX

3,ntp

启动 ntp 服务保证各台机器时钟同步

4,repo

https://cwiki.apache.org/confluence/display/AMBARI/Install+Ambari+1.7.0+from+Public+Repositories

cd /etc/yum.repos.d/

wget http://public-repo-1.hortonworks.com/ambari/CentOS6/1.x/updates/1.7.0/ambari.repo

yum install ambari-server

yum install MySQL-server mysql

提前建立库和授权

mysql>grant all on ambari.* to‘ambari’@’localhost‘identified by“175an.com”;

 mysql> flush privileges;
 

[@master.coke yum.repos.d]# yum install ambari-server
 Loaded plugins: product-id, security, subscription-manager
 This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
 6ASU5-updates | 1.1 kB 00:00
 HighAvailability | 3.9 kB 00:00
 LoadBalancer | 3.9 kB 00:00
 ResilientStorage | 3.9 kB 00:00
 ScalableFileSystem | 3.9 kB 00:00
 Server | 3.9 kB 00:00
 Updates-ambari-1.7.0 | 2.9 kB 00:00
 ambari-1.x | 1.3 kB 00:00
 epel | 1.1 kB 00:00
 Setting up Install Process
 Resolving Dependencies
–> Running transaction check
—> Package ambari-server.noarch 0:1.7.0-169 will be installed
–> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.7.0-169.noarch
–> Running transaction check
—> Package postgresql-server.x86_64 0:8.4.20-1.el6_5 will be installed
–> Processing Dependency: postgresql(x86-64) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Processing Dependency: postgresql-libs(x86-64) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Running transaction check
—> Package postgresql.x86_64 0:8.4.20-1.el6_5 will be installed
—> Package postgresql-libs.x86_64 0:8.4.20-1.el6_5 will be installed
–> Finished Dependency ResolutionDependencies Resolved

=================================================================================================================================================================================
 Package Arch Version Repository Size
 =================================================================================================================================================================================
 Installing:
 ambari-server noarch 1.7.0-169 Updates-ambari-1.7.0 96 M
 Installing for dependencies:
 postgresql x86_64 8.4.20-1.el6_5 6ASU5-updates 2.6 M
 postgresql-libs x86_64 8.4.20-1.el6_5 6ASU5-updates 201 k
 postgresql-server x86_64 8.4.20-1.el6_5 6ASU5-updates 3.4 M

Transaction Summary
 =================================================================================================================================================================================
 Install 4 Package(s)

Total download size: 102 M
 Installed size: 152 M
 Is this ok [y/N]: y
 Downloading Packages:
 (1/4): ambari-server-1.7.0-169.noarch.rpm | 96 MB 00:14
 (2/4): postgresql-8.4.20-1.el6_5.x86_64.rpm | 2.6 MB 00:00
 (3/4): postgresql-libs-8.4.20-1.el6_5.x86_64.rpm | 201 kB 00:00
 (4/4): postgresql-server-8.4.20-1.el6_5.x86_64.rpm | 3.4 MB 00:00
———————————————————————————————————————————————————————————
Total 6.7 MB/s | 102 MB 00:15
 warning: rpmts_HdrFromFdno: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY
 Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
 Importing GPG key 0x07513CAD:
 Userid:“Jenkins (HDP Builds) <jenkin@hortonworks.com>”
From : http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
 Is this ok [y/N]: y
 Running rpm_check_debug
 Running Transaction Test
 Transaction Test Succeeded
 Running Transaction
 Installing : postgresql-libs-8.4.20-1.el6_5.x86_64 1/4
 Installing : postgresql-8.4.20-1.el6_5.x86_64 2/4
 Installing : postgresql-server-8.4.20-1.el6_5.x86_64 3/4
 Installing : ambari-server-1.7.0-169.noarch 4/4
 Verifying : postgresql-server-8.4.20-1.el6_5.x86_64 1/4
 Verifying : postgresql-libs-8.4.20-1.el6_5.x86_64 2/4
 Verifying : postgresql-8.4.20-1.el6_5.x86_64 3/4
 Verifying : ambari-server-1.7.0-169.noarch 4/4

Installed:
 ambari-server.noarch 0:1.7.0-169

Dependency Installed:
 postgresql.x86_64 0:8.4.20-1.el6_5 postgresql-libs.x86_64 0:8.4.20-1.el6_5 postgresql-server.x86_64 0:8.4.20-1.el6_5

Complete!
 

ambari-server setup

 

Using Python /usr/bin/python2.6
 Setup ambari-server
 Checking SELinux…
SELinux status is‘disabled’
Customize user account for ambari-server daemon [y/n] (n)? y
 Enter user account for ambari-server daemon (root):
 Adjusting ambari-server permissions and ownership…
Checking firewall…
Checking JDK…
[1] – Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
 [2] – Oracle JDK 1.6 + Java Cryptography Extension (JCE) Policy Files 6
 [3] – Custom JDK
 ==============================================================================
 Enter choice (1):
 To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.htmland not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
 Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y
 Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
 JDK distribution size is 142376665 bytes
 jdk-7u67-linux-x64.tar.gz… 100% (135.8 MB of 135.8 MB)
 Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
 Installing JDK to /usr/jdk64
 Successfully installed JDK to /usr/jdk64/jdk1.7.0_67
 Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/UnlimitedJCEPolicyJDK7.zip to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
 Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
 Completing setup…
Configuring database…
Enter advanced database configuration [y/n] (n)? y
 ==============================================================================
 Choose one of the following options:
 [1] – PostgreSQL (Embedded)
 [2] – Oracle
 [3] – MySQL
 [4] – PostgreSQL
 ==============================================================================
 Enter choice (1): 3
 Hostname (localhost):
 Port (3306):
 Database Name (ambari):
 Username (ambari):
 Enter Database Password (bigdata):
 WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
 Press <enter> to continue.
 ERROR: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
 ERROR: Exiting with exit code -1.
 REASON: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
 

yum install -y mysql-connector-java  # resolve the  error

重新来过

 

[@master.coke 6Server]# ambari-server setup

Using python /usr/bin/python2.6

Setup ambari-server

Checking SELinux…

SELinux status is‘disabled’

Ambari-server daemon is configured to run under user‘root’. Change this setting [y/n] (n)?

Adjusting ambari-server permissions and ownership…

Checking firewall…

Checking JDK…

Do you want to change Oracle JDK [y/n] (n)?

Completing setup…

Configuring database…

Enter advanced database configuration [y/n] (n)? y

==============================================================================

Choose one of the following options:

[1] – PostgreSQL (Embedded)

[2] – Oracle

[3] – MySQL

[4] – PostgreSQL

==============================================================================

Enter choice (1): 3

Hostname (localhost):

Port (3306):

Database Name (ambari):

Username (ambari): amb

Enter Database Password (bigdata):

Re-enter password:

Copying JDBC drivers to server resources…

Configuring remote database connection properties…

WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

Proceed with configuring remote database connection properties [y/n] (y)?

Copying JDBC drivers to server resources…

Extracting system views…

Adjusting ambari-server permissions and ownership…

Ambari Server‘setup’completed successfully.
 

导入数据库

 

mysql ambari -u175an -p < /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
 

grant all on ambari.* to“amb@localhost”identified by“amb123″;

 

 

Admin Name : admin

Cluster Name : coke

Total Hosts : 6 (6 new)

Repositories:

RedHat5 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/centos5/2.x/GA/2.2.0.0

redhat5 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos5

redhat6 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/centos6/2.x/GA/2.2.0.0

redhat6 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6

SUSE11 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/GA/2.2.0.0

suse11 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/suse11sp3

Ubuntu12 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/GA/2.2.0.0

ubuntu12 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12

 

Services:
HDFS DataNode : 6 hosts
NameNode : master.coke
SNameNode : dn1.coke

YARN + MapReduce2 App Timeline Server : master.coke
NodeManager : 6 hosts
ResourceManager : master.coke

Tez Clients : 6 hosts

Nagios Server : master.coke
Administrator : nagiosadmin / (qilinli@sohu-inc.com)

Ganglia Server : master.coke

Hive Metastore : master.coke
HiveServer2 : master.coke
WebHCat Server : master.coke
Database : MySQL (Existing MySQL Database)

HBase Master : dn1.coke
RegionServer : 6 hosts

Pig Clients : 6 hosts

Sqoop Clients : 6 hosts

Oozie Server : dn1.coke
Database : MySQL (Existing MySQL Database)

ZooKeeper Server : 3 hosts

Falcon Server : dn3.coke

Storm DRPC Server : dn1.coke
Nimbus : dn1.coke
UI Server : dn1.coke
Supervisor : 6 hosts

Flume Flume : 2 hosts

Slider Clients : 6 hosts

Knox Gateway : dn1.coke

Kafka Broker : 3 hosts

CentOS 安装和配置 Hadoop2.2.0  http://www.linuxidc.com/Linux/2014-01/94685.htm

Ubuntu 13.04 上搭建 Hadoop 环境 http://www.linuxidc.com/Linux/2013-06/86106.htm

Ubuntu 12.10 +Hadoop 1.2.1 版本集群配置 http://www.linuxidc.com/Linux/2013-09/90600.htm

Ubuntu 上搭建 Hadoop 环境(单机模式 + 伪分布模式)http://www.linuxidc.com/Linux/2013-01/77681.htm

Ubuntu 下 Hadoop 环境的配置 http://www.linuxidc.com/Linux/2012-11/74539.htm

单机版搭建 Hadoop 环境图文教程详解 http://www.linuxidc.com/Linux/2012-02/53927.htm

搭建 Hadoop 环境(在 Winodws 环境下用虚拟机虚拟两个 Ubuntu 系统进行搭建)http://www.linuxidc.com/Linux/2011-12/48894.htm

更多 Hadoop 相关信息见 Hadoop 专题页面 http://www.linuxidc.com/topicnews.aspx?tid=13

正文完
星哥说事-微信公众号
post-qrcode
 0
星锅
版权声明:本站原创文章,由 星锅 于2022-01-20发表,共计9906字。
转载说明:除特殊说明外本站文章皆由CC-4.0协议发布,转载请注明出处。
【腾讯云】推广者专属福利,新客户无门槛领取总价值高达2860元代金券,每种代金券限量500张,先到先得。
阿里云-最新活动爆款每日限量供应
评论(没有评论)
验证码
【腾讯云】云服务器、云数据库、COS、CDN、短信等云产品特惠热卖中