Hadoop環境如何配置

這篇文章主要介紹了Hadoop環境如何配置，具有一定借鑒價值，感興趣的朋友可以參考下，希望大家閱讀完這篇文章之后大有收獲，下面讓小編帶著大家一起了解一下。

    Hadoop環境安裝完之后需要進行相應的配置

    Hadoop組件通過XML來配置，core-site.xml用于配置Common組件的屬性，hdfs-site.xml文件用于配置HDFS屬性，mapred-site.xml文件用于配置MapReduce屬性，hdoop支持獨立模式、偽分布模式、全分布模式，在不同模式下，各屬性的配置：

    hadoop日志配置：在hadoop-env.sh中添加export HADOOP_LOG_DIR=/usr/lib/hadoop/logs

    我hadoop2的配置如下：

    core-site.xml

<configuration>
        <property>
                <name>fs.default.name</name>
                <value>hdfs://localhost/</value>
        </property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>/usr/lib/hadoop/tmp/</value>
        </property>
</configuration>

    hdfs-site.xml:

<configuration>
        <property>
                <name>dfs.default.name</name>
                <value>/usr/lib/hadoop/logs/</value>
        </property>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
        <property>
                <name>dfs.permissions</name>
                <value>false</value>
        </property>
</configuration>

    mapred-site.xml:

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>Yarn</value>
        </property>
</configuration>

    yarn-site.xml:

<configuration>
        <property>
                <name>yarn.resourcemanager.resource-tracker.address</name>
                <value>localhost:8990</value>
        </property>
        <property>
                <name>yarn.resourcemanager.scheduler.address</name>
                <value>localhost:8991</value>
        </property>
        <property>
                <name>yarn.resourcemanager.scheduler.class</name>
                <value>org.apache.hadoop.yarn.server.resourcemanage.scheduler.capacity.CapacityScheduler</value>
        </property>
        <property>
                <name>yarn.resourcemanager.address</name>
                <value>localhost:8993</value>
        </property>
        <property>
                <name>yarn.nodemanager.local-dirs</name>
                <value>/usr/lib/hadoop/tmp/node</value>
        </property>
        <property>
                <name>yarn.nodemanager.address</name>
                <value>localhost:8994</value>
        </property>
        <property>
                <name>yarn.nodemanager.resource.memory-mb</name>
                <value>102040</value>
        </property>
        <property>
                <name>yarn.nodemanager.remote-app-log-dir</name>
                <value>/usr/lib/hadoop/tmp/app-logs</value>
        </property>
        <property>
                <name>yarn.nodemanager.log-dirs</name>
                <value>/usr/lib/hadoop/tmp/node</value>
        </property>
        <property>
                <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
        </property>
</configuration>

    slaves:

localhost

    完事之后執行hadoop namenode -fromat 格式化hdfs文件系統。

感謝你能夠認真閱讀完這篇文章，希望小編分享的“Hadoop環境如何配置”這篇文章對大家有幫助，同時也希望大家多多支持億速云，關注億速云行業資訊頻道，更多相關知識等著你來學習!