阿里云-云小站(无限量代金券发放中)
【腾讯云】云服务器、云数据库、COS、CDN、短信等热卖云产品特惠抢购

CDH的安装和设置

160次阅读
没有评论

共计 3570 个字符,预计需要花费 9 分钟才能阅读完成。

采用伪分布模式安装和设置 CDH,前提是已经安装了 Java 和 SSH。

1. 下载 Hadoop-2.6.0-cdh5.9.0,复制到 /opt/ 下,再解压;

2. 进入 /opt/hadoop-2.6.0-cdh5.9.0/etc/hadoop/,在 hadoop-env.sh 中添加:

export JAVA_HOME=/opt/jdk1.8.0_121
export HADOOP_HOME=/opt/hadoop-2.6.0-cdh5.9.0

修改配置文件 core-tite.xml:

<?xml version=”1.0″ encoding=”UTF-8″?>
<?xml-stylesheet type=”text/xsl” href=”https://www.linuxidc.com/Linux/2017-02/configuration.xsl”?>
<!–
  Licensed under the Apache License, Version 2.0 (the “License”);
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an “AS IS” BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
–>

<!– Put site-specific property overrides in this file. –>

<configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/home/hadoop/tmp</value>
    </property>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://192.168.1.104:9000</value>
    </property>
</configuration>

其中 hadoop.tmp.dir 最好自己设置,不要采用默认的设置,因为默认的设置是在 /tmp/ 下面,机器重启以后会被删除掉,造成 Hadoop 不能运行,要再次格式化 NameNode 才能运行。

hdfs-site.xml:

<?xml version=”1.0″ encoding=”UTF-8″?>
<?xml-stylesheet type=”text/xsl” href=”https://www.linuxidc.com/Linux/2017-02/configuration.xsl”?>
<!–
  Licensed under the Apache License, Version 2.0 (the “License”);
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an “AS IS” BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
–>

<!– Put site-specific property overrides in this file. –>

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.name.dir</name>
        <value>/opt/hdfs/name</value>
    </property>
    <property>
        <name>dfs.data.dir</name>
        <value>/opt/hdfs/data</value>
    </property>
    <property>
            <name>dfs.tmp.dir</name>
            <value>/opt/hdfs/tmp</value>
    </property>
</configuration>

mapred-site.xml:

<?xml version=”1.0″?>
<?xml-stylesheet type=”text/xsl” href=”https://www.linuxidc.com/Linux/2017-02/configuration.xsl”?>
<!–
  Licensed under the Apache License, Version 2.0 (the “License”);
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an “AS IS” BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
–>

<!– Put site-specific property overrides in this file. –>

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapred.job.tracker</name>
        <value>hdfs://192.168.1.104:9001</value>
    </property>
</configuration>

3. 在 /etc/profile 后面加上:

export HADOOP_HOME=/opt/hadoop-2.6.0-cdh5.9.0
export PATH=$PATH:$HADOOP_HOME/bin

并且输入命令:

source /etc/profile

使设置生效。

4. 输入命令:

hadoop namenode -format

格式化 NameNode,如果结果提示 Successful 表明格式化成功。

5. 进入 /opt/hadoop-2.6.0-cdh5.9.0/etc/hadoop/sbin,输入命令:

./start-all.sh

启动 Hadoop。为了检验是否启动成功,输入命令:

jps

如果结果包含了以下几个进程,则表明启动成功:

CDH 的安装和设置

也可以在浏览器里面输入地址 http://localhost:50070,检验是否启动成功:

CDH 的安装和设置

本文永久更新链接地址 :http://www.linuxidc.com/Linux/2017-02/140707.htm

正文完
星哥说事-微信公众号
post-qrcode
 0
星锅
版权声明:本站原创文章,由 星锅 于2022-01-21发表,共计3570字。
转载说明:除特殊说明外本站文章皆由CC-4.0协议发布,转载请注明出处。
【腾讯云】推广者专属福利,新客户无门槛领取总价值高达2860元代金券,每种代金券限量500张,先到先得。
阿里云-最新活动爆款每日限量供应
评论(没有评论)
验证码
【腾讯云】云服务器、云数据库、COS、CDN、短信等云产品特惠热卖中