mikacur/MRS-operation-guide-lts
1
0
Fork 0
forked from docs/mapreduce-service
Code Pull Requests Activity

Update content

Browse Source
This commit is contained in:
OpenTelekomCloud Proposal Bot 2022-11-30 08:58:56 +00:00
parent 621bf787fe
commit 654864ec9e
15 changed files with 106 additions and 106 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/component-operation-guide/source/using_clickhouse/clickhouse_table_engine_overview.rst
View File

@ -235,7 +235,7 @@ When data is written, a table with this type of engine divides data into differe
ORDER BY (name1,name2)
PRIMARY KEY name1;
When data of the AggregateFunction type is written or queried, the **\*state** and **\*merge** functions need to be called. The asterisk (*) indicates the aggregate functions used for defining the field type. For example, the **uniq** and **sum** functions are specified for the **name3** and **name4** fields defined in the **test_table**, respectively. Therefore, you need to call the **uniqState** and **sumState** functions and run the **INSERT** and **SELECT** statements when writing data into the table.
When data of the AggregateFunction type is written or queried, the **\*state** and **\*merge** functions need to be called. The asterisk (``*``) indicates the aggregate functions used for defining the field type. For example, the **uniq** and **sum** functions are specified for the **name3** and **name4** fields defined in the **test_table**, respectively. Therefore, you need to call the **uniqState** and **sumState** functions and run the **INSERT** and **SELECT** statements when writing data into the table.
.. code-block::

2
doc/component-operation-guide/source/using_hive/hive_performance_tuning/optimizing_join.rst
View File

@ -23,7 +23,7 @@ Hive Map Join applies to small tables (the table size is less than 25 MB) that c
Map Join has two methods:
- Use /*+ MAPJOIN(join_table) \*/.
- Use ``/*+ MAPJOIN(join_table) */``.
- Set the following parameter before running the statement. The default value is true in the current version.

2
doc/component-operation-guide/source/using_kafka/synchronizing_binlog-based_mysql_data_to_the_mrs_cluster.rst
View File

@ -149,7 +149,7 @@ If the **conf** directory exists in the **maxwell-XXX** folder, configure the **
| | | | |
| | | - If only the **mytable**, **mydate_123**, and **mydate_456** tables in the **mydatabase** database need to be collected, set this parameter to the following: | |
| | | | |
| | | exclude: \*.*,include: mydatabase.mytable, include: mydatabase./mydate_\\\\d*/ | |
| | | exclude: \*.*,include: mydatabase.mytable, include: mydatabase./mydate\ ``_``\\\\d*/ | |
+-------------------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
.. _mrs_01_0441__section182543817211:

32
doc/component-operation-guide/source/using_loader/loader_log_overview.rst
View File

@ -21,37 +21,37 @@ The automatic compression and archiving function are enabled for Loader run logs
.. table:: **Table 1** Loader log list
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Log Type | Log File Name | Description |
+=======================+=========================================+=============================================================================================================================================================================================================================================================================+
+=======================+============================================+=============================================================================================================================================================================================================================================================================+
| Run log | loader.log | Loader system log file that records most of the logs generated when the TelcoFS system is running. |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | loader-omm-***-pid***-gc.log.*.current | Loader process GC log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | loader-omm-``***``-pid***-gc.log.*.current | Loader process GC log file |
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | sqoopInstanceCheck.log | Loader instance health check log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Audit log | default.audit | Loader operation audit log file that records operations such as adding, deleting, modifying, and querying jobs and user login |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Tomcat log | catalina.out | Tomcat run log file. |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | catalina. <yyyy-mm-dd >.log | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | host-manager. <yyyy-mm-dd >.log | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | localhost_access_log. <yyyy-mm-dd >.txt | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | manager <yyyy-mm-dd >.log | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | localhost. <yyyy-mm-dd >.log | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Script log | postInstall.log | Loader installation script log file |
| | | |
| | | Log file generated during the execution of the Loader installation script (**postInstall.sh**) |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | preStart.log | Pre-startup script log file of the Loader service During startup of the Loader service, a series of preparation operations are first performed (by executing **preStart.sh**), such as generating the keytab file. This log file records information about these operations |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | loader_ctl.log | Log file generated when Loader executes the service start and stop script (**sqoop.sh**) |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Log Level
---------

4
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_hbase.rst
View File

@ -34,13 +34,13 @@ Procedure
+===================================+==============================================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| HBase Table | Name of a table to which the policy applies. |
| | |
| | The value can contain wildcard (*). For example, **table1:\*** indicates all tables in **table1**. |
| | The value can contain wildcard (``*``). For example, **table1:\*** indicates all tables in **table1**. |
| | |
| | The **Include** policy applies to the current input object, and the **Exclude** policy applies to objects other than the current input object. |
| | |

4
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_hdfs.rst
View File

@ -34,11 +34,11 @@ Procedure
+===================================+===================================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Resource Path | Resource path, which is the HDFS path folder or file to which the current policy applies. You can enter multiple values and use the wildcard (*), for example, **/test/\***. |
| Resource Path | Resource path, which is the HDFS path folder or file to which the current policy applies. You can enter multiple values and use the wildcard (``*``), for example, **/test/\***. |
| | |
| | To enable a subdirectory to inherit the permission of its upper-level directory, enable the recursion function. |
| | |

6
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_hive.rst
View File

@ -35,7 +35,7 @@ Procedure
+===================================+==========================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -205,7 +205,7 @@ Ranger supports data masking for Hive data. It can process the returned result o
+===================================+===========================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -263,7 +263,7 @@ Ranger allows you to filter data at the row level when you perform the **select*
+===================================+===========================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

2
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_kafka.rst
View File

@ -34,7 +34,7 @@ Procedure
+===================================+===================================================================================================================================================================================================================================================================================================================+
| Policy Type | Access type. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

6
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_spark2x.rst
View File

@ -52,7 +52,7 @@ Procedure
+===================================+==========================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -206,7 +206,7 @@ Ranger supports data masking for Spark2x data. It can process the returned resul
+===================================+=====================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -254,7 +254,7 @@ Ranger allows you to filter data at the row level when you perform the **select*
+===================================+===========================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

2
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_storm.rst
View File

@ -31,7 +31,7 @@ Procedure
+-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Description |
+===================================+===========================================================================================================================================================================================================================================================================================================+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
| | |

4
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_yarn.rst
View File

@ -34,11 +34,11 @@ Procedure
+===================================+==========================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Queue | Queue name. The wildcard (*) is supported. |
| Queue | Queue name. The wildcard (``*``) is supported. |
| | |
| | To enable a sub-queue to inherit the permission of its upper-level queue, enable the recursion function. |
| | |

2
doc/component-operation-guide/source/using_spark2x/common_issues_about_spark2x/spark_sql_and_dataframe/why_no_result_is_found_when_2016-6-30_is_set_in_the_date_field_as_the_filter_condition.rst
View File

@ -10,7 +10,7 @@ Question
Why no result is found when 2016-6-30 is set in the date field as the filter condition?
As shown in the following figure, trx_dte_par in the select count (*) from trxfintrx2012 a where trx_dte_par='2016-6-30' statement is a date field. However, no search result is found when the filter condition is where trx_dte_par='2016-6-30'. Search results are found only when the filter condition is where trx_dte_par='2016-06-30'.
As shown in the following figure, trx_dte_par in the select count (``*``) from trxfintrx2012 a where trx_dte_par='2016-6-30' statement is a date field. However, no search result is found when the filter condition is where trx_dte_par='2016-6-30'. Search results are found only when the filter condition is where trx_dte_par='2016-06-30'.
.. figure:: /_static/images/en-us_image_0000001348771241.jpg

56
doc/component-operation-guide/source/using_spark2x/spark2x_logs.rst
View File

@ -29,57 +29,57 @@ Log Description
.. table:: **Table 1** Spark2x log list
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| Log Type | Name | Description |
+=======================+===================================================================================+=================================================================================================+
+=======================+=======================================================================================+=================================================================================================+
| SparkResource2x logs | spark.log | Spark2x service initialization log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | prestart.log | Prestart script log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | cleanup.log | Cleanup log file for instance installation and uninstallation |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-availability-check.log | Spark2x service health check log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-service-check.log | Spark2x service check log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| JDBCServer2x logs | JDBCServer-start.log | JDBCServer2x startup log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | JDBCServer-stop.log | JDBCServer2x stop log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | JDBCServer.log | JDBCServer2x run log on the server |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | jdbc-state-check.log | JDBCServer2x health check log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | jdbcserver-omm-pid***-gc.log.*.current | IJDBCServer2x process GC log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.sql.hive.thriftserver.HiveThriftProxyServer2-***.out\* | JDBCServer2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.sql.hive.thriftserver.HiveThriftProxyServer2-``***``.out\* | JDBCServer2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| JobHistory2x logs | jobHistory-start.log | JobHistory2x startup log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | jobHistory-stop.log | JobHistory2x stop log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | JobHistory.log | JobHistory2x running process log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | jobhistory-omm-pid***-gc.log.*.current | JobHistory2x process GC log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.deploy.history.HistoryServer-***.out\* | JobHistory2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.deploy.history.HistoryServer-``***``.out\* | JobHistory2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| IndexServer2x logs | IndexServer-start.log | IndexServer2x startup log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | IndexServer-stop.log | IndexServer2x stop log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | IndexServer.log | IndexServer2x run log on the server |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | indexserver-state-check.log | IndexServer2x health check log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | indexserver-omm-pid***-gc.log.*.current | IndexServer2x process GC log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.sql.hive.thriftserver.IndexServerProxy-***.out\* | IndexServer2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.sql.hive.thriftserver.IndexServerProxy-``***``.out\* | IndexServer2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| Audit Log | jdbcserver-audit.log | JDBCServer2x audit log |
| | | |
| | ranger-audit.log | |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
+-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
Log levels
----------

4
doc/component-operation-guide/source/using_spark2x/using_carbondata_for_first_query.rst
View File

@ -30,7 +30,7 @@ Configure **prequeryParams.properties** by referring to :ref:`Table 1 <mrs_01_23
+==================================+=======================================================================================================================================================================================================================+==================================================================================+
| spark.prequery.period.max.minute | Maximum preheating duration, in minutes. | 60 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.prequery.tables | Table name configuration, *database.table:int*. The table name supports the wildcard (*). **int** indicates the duration (unit: day) within which the table is updated before it is preheated. | default.test*:10 |
| spark.prequery.tables | Table name configuration, *database.table:int*. The table name supports the wildcard (``*``). **int** indicates the duration (unit: day) within which the table is updated before it is preheated. | default.test*:10 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.prequery.maxThreads | Maximum number of concurrent threads during preheating | 50 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
@ -38,7 +38,7 @@ Configure **prequeryParams.properties** by referring to :ref:`Table 1 <mrs_01_23
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.prequery.driver | IP address and port number of JDBCServer. The format is *IP address:Port number*. If multiple servers need to be preheated, enter multiple *IP address:Port number* of the servers and separate them with commas (,). | 192.168.0.2:22550 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.prequery.sql | SQL statement for preheating. Different statements are separated by colons (:). | SELECT COUNT(*) FROM %s;SELECT \* FROM %s LIMIT 1 |
| spark.prequery.sql | SQL statement for preheating. Different statements are separated by colons (:). | SELECT COUNT(``*``) FROM %s;SELECT \* FROM %s LIMIT 1 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.security.url | URL required by JDBC in security mode | ;saslQop=auth-conf;auth=KERBEROS;principal=spark2x/hadoop.hadoop.com@HADOOP.COM; |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+