doc-exports/docs/cce/umn/cce_10_0555.html

Collecting Container Logs Using the Cloud Native Log Collection Add-on

The Cloud Native Log Collection add-on (Cloud Native Log Collection) is developed based on Fluent Bit and OpenTelemetry for collecting logs and Kubernetes events. This add-on supports CRD-based log collection policies. It collects and forwards stdout logs, container file logs, node logs, and Kubernetes events in a cluster based on configured policies.

Notes and Constraints

• Up to 100 log rules can be created for each cluster.
• The Cloud Native Log Collection add-on cannot collect .gz, .tar, and .zip logs and cannot access symbolic links of logs.
• If the node storage driver is Device Mapper, container file logs must be collected from the path where the data disk is attached to the node.
• If the container runtime is containerd, each stdout log cannot be in multiple lines. (This does not apply to the Cloud Native Log Collection add-on of version 1.3.0 or later.)
• If a volume is mounted to the directory of a service container, this add-on cannot collect data from the parent directory. In this case, you need to configure a complete data directory.
• If the lifetime of a container is less than 1 minute, logs cannot be collected in a timely manner. As a result, logs may be lost.
• When you create a log stream using a collection policy, ensure that the log stream name is unique because LTS does not allow log streams with the same name (both before and after renaming), even in different log groups.

Enabling Logging on the Console

1. Enable logging.

   Enabling logging during cluster creation

   1. Log in to the CCE console.
   2. Click Create Cluster.
   3. Configure the parameters by referring to Creating a CCE Standard/Turbo Cluster. Then, click Next: Select Add-on in the lower right corner.
   4. On the Select Add-on page, select Cloud Native Log Collection.
   5. Click Next: Configure Add-on in the lower right corner and select the required logs.
      • Container logs: A log collection policy named default-stdout will be created, and stdout logs in all namespaces will be reported to LTS.
      • Kubernetes events: A log collection policy named default-event will be created, and Kubernetes events in all namespaces will be reported to LTS.
   6. Click Next: Confirm Settings in the lower right corner. On the displayed page, click Submit.

2. View and configure log collection policies.
   1. In the navigation pane, choose Logging.

   2. Click View Log Collection Policies in the upper right corner.

      All log collection policies reported to LTS are displayed.

   3. Click Create Log Collection Policy.
      

      • To keep the logs organized, you are advised to select different log streams for different types of logs when configuring the log collection policies.
      • The following are requirements for configuring the container and node file log paths:
        • Log directory: Enter an absolute path, for example, /log. The path must start with a slash (/) and contain a maximum of 512 characters. Only uppercase letters, lowercase letters, digits, hyphens (-), underscores (_), slashes (/), asterisks (*), and question marks (?) are allowed.
        • Log file name: It can contain only uppercase letters, lowercase letters, digits, hyphens (-), underscores (_), asterisks (*), question marks (?), and periods (.). Logs in the format of .gz, .tar, and .zip are not supported.

        The directory and file names must be complete and support asterisks (*) and question marks (?) as wildcards. A maximum of three levels of directories can be matched using wildcards. The level-1 directory does not support wildcards. An asterisk (*) can match multiple characters. A question mark (?) can match only one character. For example:

        • If the directory is /var/logs/* and the file name is *.log, the match expression is /var/logs/*/*.log, indicating that any files with the extension .log in all level-1 directories in the /var/logs directory are matched. Note that this expression cannot match any files with the extension .log in the /var/logs directory and multi-level directories in the /var/logs directory.
        • If the directory is /var/logs/app_* and the file name is *.log, any log files with the extension .log in all directories that match app_* in the /var/logs directory will be reported.

      Table 1 Parameters of a custom policy

      Parameter	Description
      Log Type	Container standard output: used to collect container stdout logs. You can create a log collection policy by namespace, workload name, or instance label.	Container file log: used to collect text logs. You can specify a workload or instance label to create a log collection policy.	Node file log: used to collect logs from a node. Only one file path can be configured for a log collection policy.	Kubernetes Events: used to collect Kubernetes events. You can configure collection policies by namespace.
      Log Source	All containers: You can specify all containers in a namespace. If this parameter is not specified, logs of containers in all namespaces will be collected. Workload: You can specify a workload and its containers. If this parameter is not specified, logs of all containers running the workload will be collected. Workload with target label: You can specify a workload by label and its containers. If this parameter is not specified, logs of all containers running the workload will be collected.	Workload: You can specify a workload and its containers. If this parameter is not specified, logs of all containers running the workload will be collected. Workload with target label: You can specify a workload by label and its containers. If this parameter is not specified, logs of all containers running the workload will be collected. You also need to specify the log collection path. For details, see the log path configuration requirements.	Collection Path: used to configure the log collection path. For details, see the log path configuration requirements.	You can specify a namespace whose Kubernetes events are to be collected. If no namespace is specified, Kubernetes events in all namespaces are collected.
      Log Format	Single-line Each log contains only one line of text. The newline character \n denotes the start of a new log. Multi-line Some programs (for example, Java program) print a log that occupies multiple lines. By default, logs are collected by line. If you want to display logs as a single message, you can enable multi-line logging and use the regular pattern. When you select Multi-line, configure Log Matching Format. For example, if logs need to be collected by line and each log starts with a date and occupies three lines, you can set Log Matching Format to the regular expression of the date, for example, \d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}.*. The three lines starting with the date are regarded as a log. 2022-01-01 00:00:00 Exception in thread "main" java.lang.RuntimeException: Something has gone wrong, aborting! at com.myproject.module.MyProject.badMethod(MyProject.java:22) at com.myproject.module.MyProject.oneMoreMethod(MyProject.java:18)
      LTS Collection	This parameter is used to configure the log group and log stream for log reporting. Centralized: The default log group (k8s-log-{Cluster ID}) and default log stream (stdout-{Cluster ID}) are automatically selected. Custom: Select a log group and log stream from the drop-down list. Log Group: A log group is the basic unit for LTS to manage logs. If you do not have a log group, CCE prompts you to create one. The default name is k8s-log-{Cluster ID}, for example, k8s-log-bb7eaa87-07dd-11ed-ab6c-0255ac1001b3. Log Stream: A log stream is the basic unit for reading and writing logs. You can put different types of logs into different streams to ease management. When you install the add-on or create a log collection policy based on the policy template, the following log streams are automatically created: - stdout-{Cluster ID} for container logs, for example, stdout-bb7eaa87-07dd-11ed-ab6c-0255ac1001b3 - event-{Cluster ID} for Kubernetes events, for example, event-bb7eaa87-07dd-11ed-ab6c-0255ac1001b3

3. View the logs.
   1. In the navigation pane, choose Logging.
   2. View different types of logs:

Parent topic: Collecting Container Logs