分享

Hadoop_2.x_mapred-site.xml详细参数

 IT技术武馆 2014-07-16

<!-- jobhistory properties -->

<property>
  <name>mapreduce.jobhistory.address</name>
  <value>0.0.0.0:10020</value>
  <description>MapReduce JobHistory Server IPC host:port</description>
</property>

注释:历史作业ipc的ip端口。

<property>
  <name>mapreduce.jobhistory.webapp.address</name>
  <value>0.0.0.0:19888</value>
  <description>MapReduce JobHistory Server Web UI host:port</description>
</property>

 注释:历史作业的web端访问ip地址

<property>
  <name>mapreduce.jobhistory.keytab</name>
  <description>
    Location of the kerberos keytab file for the MapReduce
    JobHistory Server.
  </description>
  <value>/etc/security/keytab/jhs.service.keytab</value>
</property>
<property>
  <name>mapreduce.jobhistory.principal</name>
  <description>
    Kerberos principal name for the MapReduce JobHistory Server.
  </description>
  <value>jhs/_HOST@REALM.TLD</value>
</property>

<!-- Job Notification Configuration -->

<property>
 <name>mapreduce.job.end-notification.url</name>
 <!--<value>http://localhost:8080/jobstatus.php?jobId=$jobId&jobStatus=$jobStatus</value>-->
 <description>Indicates url which will be called on completion of job to inform
              end status of job.
              User can give at most 2 variables with URI : $jobId and $jobStatus.
              If they are present in URI, then they will be replaced by their
              respective values.
</description>
</property>
<property>
  <name>mapreduce.job.end-notification.retry.attempts</name>
  <value>0</value>
  <description>The number of times the submitter of the job wants to retry job
    end notification if it fails. This is capped by
    mapreduce.job.end-notification.max.attempts</description>
</property>
<property>
  <name>mapreduce.job.end-notification.retry.interval</name>
  <value>1000</value>
  <description>The number of milliseconds the submitter of the job wants to
    wait before job end notification is retried if it fails. This is capped by
    mapreduce.job.end-notification.max.retry.interval</description>
</property>
<property>
  <name>mapreduce.job.end-notification.max.attempts</name>
  <value>5</value>
  <final>true</final>
  <description>The maximum number of times a URL will be read for providing job
    end notification. Cluster administrators can set this to limit how long
    after end of a job, the Application Master waits before exiting. Must be
    marked as final to prevent users from overriding this.
  </description>
</property>
<property>
  <name>mapreduce.job.end-notification.max.retry.interval</name>
  <value>5000</value>
  <final>true</final>
  <description>The maximum amount of time (in milliseconds) to wait before
     retrying job end notification. Cluster administrators can set this to
     limit how long the Application Master waits before exiting. Must be marked
     as final to prevent users from overriding this.</description>
</property>
<property>
  <name>yarn.app.mapreduce.am.env</name>
  <value></value>
  <description>User added environment variables for the MR App Master 
  processes. Example :
  1) A=foo  This will set the env variable A to foo
  2) B=$B:c This is inherit tasktracker's B env variable.  
  </description>
</property>
<property>
  <name>yarn.app.mapreduce.am.command-opts</name>
  <value>-Xmx1024m</value>
  <description>Java opts for the MR App Master processes.  
  The following symbol, if present, will be interpolated: @taskid@ is replaced 
  by current TaskID. Any other occurrences of '@' will go unchanged.
  For example, to enable verbose gc logging to a file named for the taskid in
  /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of:
        -Xmx1024m -verbose:gc -Xloggc:/tmp/@taskid@.gc
  
  Usage of -Djava.library.path can cause programs to no longer function if
  hadoop native libraries are used. These values should instead be set as part 
  of LD_LIBRARY_PATH in the map / reduce JVM env using the mapreduce.map.env and 
  mapreduce.reduce.env config settings. 
  </description>
</property>
<property>
  <name>yarn.app.mapreduce.am.admin-command-opts</name>
  <value></value>
  <description>Java opts for the MR App Master processes for admin purposes.
  It will appears before the opts set by yarn.app.mapreduce.am.command-opts and
  thus its options can be overridden user. 
  
  Usage of -Djava.library.path can cause programs to no longer function if
  hadoop native libraries are used. These values should instead be set as part 
  of LD_LIBRARY_PATH in the map / reduce JVM env using the mapreduce.map.env and 
  mapreduce.reduce.env config settings. 
  </description>
</property>
<property>
  <name>yarn.app.mapreduce.am.job.task.listener.thread-count</name>
  <value>30</value>
  <description>The number of threads used to handle RPC calls in the 
    MR AppMaster from remote tasks</description>
</property>
<property>
  <name>yarn.app.mapreduce.am.job.client.port-range</name>
  <value></value>
  <description>Range of ports that the MapReduce AM can use when binding.
    Leave blank if you want all possible ports.  
    For example 50000-50050,50100-50200</description>
</property>
<property>
  <name>yarn.app.mapreduce.am.job.committer.cancel-timeout</name>
  <value>60000</value>
  <description>The amount of time in milliseconds to wait for the output
    committer to cancel an operation if the job is killed</description>
</property>
<property>
  <name>yarn.app.mapreduce.am.job.committer.commit-window</name>
  <value>10000</value>
  <description>Defines a time window in milliseconds for output commit
  operations.  If contact with the RM has occurred within this window then
  commits are allowed, otherwise the AM will not allow output commits until
  contact with the RM has been re-established.</description>
</property>
<property>
  <name>yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms</name>
  <value>1000</value>
  <description>The interval in ms at which the MR AppMaster should send
    heartbeats to the ResourceManager</description>
</property>
<property>
  <name>yarn.app.mapreduce.client-am.ipc.max-retries</name>
  <value>1</value>
  <description>The number of client retries to the AM - before reconnecting
    to the RM to fetch Application Status.</description>
</property>
<property>
  <name>yarn.app.mapreduce.client.max-retries</name>
  <value>3</value>
  <description>The number of client retries to the RM/HS/AM before
    throwing exception. This is a layer above the ipc.</description>
</property>
<property>
  <name>yarn.app.mapreduce.am.resource.mb</name>
  <value>1536</value>
  <description>The amount of memory the MR AppMaster needs.</description>
</property>

    本站是提供个人知识管理的网络存储空间,所有内容均由用户发布,不代表本站观点。请注意甄别内容中的联系方式、诱导购买等信息,谨防诈骗。如发现有害或侵权内容,请点击一键举报。
    转藏 分享 献花(0

    0条评论

    发表

    请遵守用户 评论公约

    类似文章 更多