:original_name: mrs_01_0858.html

.. _mrs_01_0858:

Configuring Container Log Aggregation
=====================================

Scenario
--------

Yarn provides the container log aggregation function to collect logs generated by containers on each node to HDFS to release local disk space. You can collect logs in either of the following ways:

-  After the application is complete, collect container logs to HDFS at a time.
-  During application running, periodically collect log segments generated by containers and save them to HDFS.

Configuration Description
-------------------------

**Navigation path for setting parameters:**

Go to the **All Configurations** page of Yarn and enter a parameter name list in :ref:`Table 1 <mrs_01_0858__en-us_topic_0000001219029153_te1c6a71343c044a9a63faeb4a644fb5a>` in the search box by referring to :ref:`Modifying Cluster Service Configuration Parameters <mrs_01_1293>`.

The **yarn.nodemanager.remote-app-log-dir-suffix** parameter must be configured on the Yarn client. The configurations on the ResourceManager, NodeManager, and JobHistory nodes must be the same as those on the Yarn client.

The periodic log collection function applies only to MapReduce applications, for which rolling output of log files must be configured. :ref:`Table 3 <mrs_01_0858__en-us_topic_0000001219029153_t969e9ceee95b43149610e8f72cf9d91f>` describes the configurations in the **mapred-site.xml** configuration file on the MapReduce client node.

.. _mrs_01_0858__en-us_topic_0000001219029153_te1c6a71343c044a9a63faeb4a644fb5a:

.. table:: **Table 1** Parameter description

    
   | Parameter                                                                              | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | Default Value         |
    
   | yarn.log-aggregation-enable                                                            | Whether to enable container log aggregation                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | true                  |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | -  If this parameter is set to **true**, logs are collected to the HDFS directory.                                                                                                                                                                                                                                                                                                                                                                                                                                           |                       |
   |                                                                                        | -  If this parameter is set to **false**, the function is disabled, and logs are not collected to HDFS.                                                                                                                                                                                                                                                                                                                                                                                                                      |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | After changing the parameter value, restart the Yarn service for the setting to take effect.                                                                                                                                                                                                                                                                                                                                                                                                                                 |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | .. note::                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        |    -  The container logs that are generated before the parameter is set to **false** and the setting takes effect cannot be obtained from the web UI.                                                                                                                                                                                                                                                                                                                                                                        |                       |
   |                                                                                        |    -  If you want to view the logs generated before on the web UI, you are advised to set this parameter to **true**.                                                                                                                                                                                                                                                                                                                                                                                                        |                       |
    
   | yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds                      | Interval for NodeManager to periodically collect logs                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | -1                    |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | -  If this parameter is set to **-1** or **0**, periodic log collection is disabled. Logs are collected at a time after application running is complete.                                                                                                                                                                                                                                                                                                                                                                     |                       |
   |                                                                                        | -  The minimum collection interval can be set to 3,600 seconds. If this parameter is set to a value greater than 0 and less than 3,600, the collection interval is 3,600 seconds.                                                                                                                                                                                                                                                                                                                                            |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | Interval for NodeManager to wake up and upload logs. If this parameter is set to **-1** or **0**, rolling monitoring is disabled and logs are aggregated when the application task is complete. The value must be greater than or equal to **-1**.                                                                                                                                                                                                                                                                           |                       |
    
   | yarn.nodemanager.disk-health-checker.log-dirs.max-disk-utilization-per-disk-percentage | Maximum percentage of the Yarn disk quota that can be occupied by the container log directory on each disk. When the space occupied by the log directory exceeds the value of this parameter, the periodic log collection service is triggered to start a log collection activity beyond the period to release the local disk space. Maximum space for container logs that can be provided on each disk. If the disk space occupied by container logs exceeds this threshold, data aggregation in rolling mode is triggered. | 25                    |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | The valid value range of the maximum disk quota percentage is -1 to 100. If the value is less than or equal to -1, it is forcibly reset to **25**. If the value is greater than 100, the value is forcibly reset to **25**. If you set the value to a negative number, the disk capacity detection function for Container log directory is disabled.                                                                                                                                                                         |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | .. note::                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        |    -  Percentage of the available disk space of the container log directory = Percentage of the available disk space of Yarn (**yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage**) x Percentage of the available disk space of the container log directory (**yarn.nodemanager.disk-health-checker.log-dirs.max-disk-utilization-per-disk-percentage**)                                                                                                                                        |                       |
   |                                                                                        |    -  Only applications with the periodic log collection function enabled can trigger log collection when the disk quota of the log directory exceeds the threshold.                                                                                                                                                                                                                                                                                                                                                         |                       |
    
   | yarn.nodemanager.remote-app-log-dir-suffix                                             | Name of the HDFS folder in which container logs are to be stored. This parameter and **yarn.nodemanager.remote-app-log-dir** form the full path for storing container logs. That is, **{yarn.nodemanager.remote-app-log-dir}/${user}/{yarn.nodemanager.remote-app-log-dir-suffix}**.                                                                                                                                                                                                                                         | logs                  |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | .. note::                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        |    *{user}* indicates the username for running the task.                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |                       |
    
   | yarn.nodemanager.log-aggregator.on-fail.remain-log-in-sec                              | Duration for retaining container logs on the local host after the logs fail to be collected, in second                                                                                                                                                                                                                                                                                                                                                                                                                       | 604800                |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        | -  If this parameter is set to a negative number or **0**, local logs are deleted immediately.                                                                                                                                                                                                                                                                                                                                                                                                                               |                       |
   |                                                                                        | -  If this parameter is set to a positive number, local logs are retained for this period.                                                                                                                                                                                                                                                                                                                                                                                                                                   |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        |    .. note::                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |                       |
   |                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                       |
   |                                                                                        |       This parameter takes effect only when **yarn.nodemanager.delete.debug-delay-sec** is not set to **-1**. The default value is **0**. **yarn.nodemanager.delete.debug-delay-sec** also controls the file deletion function. If this parameter is set to **-1**, other functions will be changed. Exercise caution when setting this parameter.                                                                                                                                                                           |                       |
    

Go to the **All Configurations** page of MapReduce and enter a parameter name in :ref:`Table 2 <mrs_01_0858__en-us_topic_0000001219029153_table11598435102812>` in the search box by referring to :ref:`Modifying Cluster Service Configuration Parameters <mrs_01_1293>`.

.. _mrs_01_0858__en-us_topic_0000001219029153_table11598435102812:

.. table:: **Table 2** Parameter description

   +----------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | Parameter                                          | Description                                                                                                                                        | Default Value         |
   +====================================================+====================================================================================================================================================+=======================+
   | yarn.log-aggregation.retain-seconds                | Duration for retaining aggregated logs, in second                                                                                                  | 1296000               |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    | -  If this parameter is set to a negative integer, the container logs will be retained permanently in the HDFS.                                    |                       |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    | -  If this parameter is set to **0** or a positive integer, container logs will be stored for such a period and deleted after the period expires.  |                       |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    |    .. note::                                                                                                                                       |                       |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    |       A short period may increase load of the NameNode. Therefore, you are advised to set this parameter to a proper value.                        |                       |
   +----------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | yarn.log-aggregation.retain-check-interval-seconds | Interval for storing container logs in HDFS, in second                                                                                             | 86400                 |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    | -  If this parameter is set to **-1** or **0**, the interval will be one tenth of the period specified by **yarn.log-aggregation.retain-seconds**. |                       |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    |    .. note::                                                                                                                                       |                       |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    |       If this parameter is set to **-1** or **0**, **yarn.log-aggregation.retain-seconds** cannot be set to **0**.                                 |                       |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    | -  If this parameter is set to a positive number, container logs in HDFS will be scanned at such an interval.                                      |                       |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    |    .. note::                                                                                                                                       |                       |
   |                                                    |                                                                                                                                                    |                       |
   |                                                    |       A short interval may increase load of the NameNode. Therefore, you are advised to set this parameter to a proper value.                      |                       |
   +----------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+

Go to the **All Configurations** page of Yarn and enter a parameter name list in :ref:`Table 3 <mrs_01_0858__en-us_topic_0000001219029153_t969e9ceee95b43149610e8f72cf9d91f>` in the search box by referring to :ref:`Modifying Cluster Service Configuration Parameters <mrs_01_1293>`.

.. _mrs_01_0858__en-us_topic_0000001219029153_t969e9ceee95b43149610e8f72cf9d91f:

.. table:: **Table 3** Configuring rolling output of MapReduce application log files

   +-----------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | Parameter                                     | Description                                                                                                                                                                                                                                                                                                    | Default Value         |
   +===============================================+================================================================================================================================================================================================================================================================================================================+=======================+
   | mapreduce.task.userlog.limit.kb               | Maximum size of a single task log file of the MapReduce application. When the maximum size of the log file has been reached, a new log file is generated. The value **0** indicates that the size of the log file is not limited.                                                                              | 51200                 |
   +-----------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | yarn.app.mapreduce.task.container.log.backups | Maximum number of task logs that can be retained for the MapReduce application.                                                                                                                                                                                                                                | 10                    |
   |                                               |                                                                                                                                                                                                                                                                                                                |                       |
   |                                               | If this parameter is set to **0**, rolling output is disabled.                                                                                                                                                                                                                                                 |                       |
   |                                               |                                                                                                                                                                                                                                                                                                                |                       |
   |                                               | Number of task log backup files when ContainerRollingLogAppender (CRLA) is used. By default, ContainerLogAppender (CLA) is used and container logs are not rolled back.                                                                                                                                        |                       |
   |                                               |                                                                                                                                                                                                                                                                                                                |                       |
   |                                               | When both **mapreduce.task.userlog.limit.kb** and **yarn.app.mapreduce.task.container.log.backups** are greater than 0, CRLA is enabled. The value ranges from 0 to 999.                                                                                                                                       |                       |
   +-----------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | yarn.app.mapreduce.am.container.log.limit.kb  | Maximum size of a single ApplicationMaster log file of the MapReduce application, in KB. When the maximum size of the log file has been reached, a new log file is generated. The value **0** indicates that the size of a single ApplicationMaster log file is not limited.                                   | 51200                 |
   +-----------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | yarn.app.mapreduce.am.container.log.backups   | Maximum number of ApplicationMaster logs that can be retained for the MapReduce application. If this parameter is set to **0**, rolling output is disabled. Number of ApplicationMaster log backup files when CRLA is used. By default, CLA is used and container logs are not rolled back.                    | 20                    |
   |                                               |                                                                                                                                                                                                                                                                                                                |                       |
   |                                               | When both **yarn.app.mapreduce.am.container.log.limit.kb** and **yarn.app.mapreduce.am.container.log.backups** are greater than 0, CRLA is enabled for the ApplicationMaster. The value ranges from 0 to 999.                                                                                                  |                       |
   +-----------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | yarn.app.mapreduce.shuffle.log.backups        | Maximum number of shuffle logs that can be retained for the MapReduce application. If this parameter is set to **0**, rolling output is disabled.                                                                                                                                                              | 10                    |
   |                                               |                                                                                                                                                                                                                                                                                                                |                       |
   |                                               | When both **yarn.app.mapreduce.shuffle.log.limit.kb** and **yarn.app.mapreduce.shuffle.log.backups** are greater than 0, **syslog.shuffle** uses CRLA. The value ranges from 0 to 999.                                                                                                                         |                       |
   +-----------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | yarn.app.mapreduce.shuffle.log.limit.kb       | Maximum size of a single shuffle log file of the MapReduce application, in KB. When the maximum size of the log file has been reached, a new log file is generated. If this parameter is set to **0**, the size of a single shuffle log file is not limited. The value must be greater than or equal to **0**. | 51200                 |
   +-----------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
