mikacur/MRS-operation-guide-lts
1
0
Fork 0
forked from docs/mapreduce-service
Code Pull Requests Activity

Update content

Browse Source
This commit is contained in:
OpenTelekomCloud Proposal Bot 2022-11-30 08:58:56 +00:00
parent 621bf787fe
commit 654864ec9e
15 changed files with 106 additions and 106 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/component-operation-guide/source/using_clickhouse/clickhouse_table_engine_overview.rst
View File

@ -235,7 +235,7 @@ When data is written, a table with this type of engine divides data into differe
ORDER BY (name1,name2) ORDER BY (name1,name2)
PRIMARY KEY name1; PRIMARY KEY name1;
When data of the AggregateFunction type is written or queried, the **\*state** and **\*merge** functions need to be called. The asterisk (*) indicates the aggregate functions used for defining the field type. For example, the **uniq** and **sum** functions are specified for the **name3** and **name4** fields defined in the **test_table**, respectively. Therefore, you need to call the **uniqState** and **sumState** functions and run the **INSERT** and **SELECT** statements when writing data into the table. When data of the AggregateFunction type is written or queried, the **\*state** and **\*merge** functions need to be called. The asterisk (``*``) indicates the aggregate functions used for defining the field type. For example, the **uniq** and **sum** functions are specified for the **name3** and **name4** fields defined in the **test_table**, respectively. Therefore, you need to call the **uniqState** and **sumState** functions and run the **INSERT** and **SELECT** statements when writing data into the table.
.. code-block:: .. code-block::

2
doc/component-operation-guide/source/using_hive/hive_performance_tuning/optimizing_join.rst
View File

@ -23,7 +23,7 @@ Hive Map Join applies to small tables (the table size is less than 25 MB) that c
Map Join has two methods: Map Join has two methods:
- Use /*+ MAPJOIN(join_table) \*/. - Use ``/*+ MAPJOIN(join_table) */``.
- Set the following parameter before running the statement. The default value is true in the current version. - Set the following parameter before running the statement. The default value is true in the current version.

2
doc/component-operation-guide/source/using_kafka/synchronizing_binlog-based_mysql_data_to_the_mrs_cluster.rst
View File

@ -149,7 +149,7 @@ If the **conf** directory exists in the **maxwell-XXX** folder, configure the **
| | | | | | | | | |
| | | - If only the **mytable**, **mydate_123**, and **mydate_456** tables in the **mydatabase** database need to be collected, set this parameter to the following: | | | | | - If only the **mytable**, **mydate_123**, and **mydate_456** tables in the **mydatabase** database need to be collected, set this parameter to the following: | |
| | | | | | | | | |
| | | exclude: \*.*,include: mydatabase.mytable, include: mydatabase./mydate_\\\\d*/ | | | | | exclude: \*.*,include: mydatabase.mytable, include: mydatabase./mydate\ ``_``\\\\d*/ | |
+-------------------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+ +-------------------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
.. _mrs_01_0441__section182543817211: .. _mrs_01_0441__section182543817211:

32
doc/component-operation-guide/source/using_loader/loader_log_overview.rst
View File

@ -21,37 +21,37 @@ The automatic compression and archiving function are enabled for Loader run logs
.. table:: **Table 1** Loader log list .. table:: **Table 1** Loader log list
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Log Type | Log File Name | Description | | Log Type | Log File Name | Description |
+=======================+=========================================+=============================================================================================================================================================================================================================================================================+ +=======================+============================================+=============================================================================================================================================================================================================================================================================+
| Run log | loader.log | Loader system log file that records most of the logs generated when the TelcoFS system is running. | | Run log | loader.log | Loader system log file that records most of the logs generated when the TelcoFS system is running. |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | loader-omm-***-pid***-gc.log.*.current | Loader process GC log file | | | loader-omm-``***``-pid***-gc.log.*.current | Loader process GC log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | sqoopInstanceCheck.log | Loader instance health check log file | | | sqoopInstanceCheck.log | Loader instance health check log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Audit log | default.audit | Loader operation audit log file that records operations such as adding, deleting, modifying, and querying jobs and user login | | Audit log | default.audit | Loader operation audit log file that records operations such as adding, deleting, modifying, and querying jobs and user login |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Tomcat log | catalina.out | Tomcat run log file. | | Tomcat log | catalina.out | Tomcat run log file. |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | catalina. <yyyy-mm-dd >.log | Tomcat run log file | | | catalina. <yyyy-mm-dd >.log | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | host-manager. <yyyy-mm-dd >.log | Tomcat run log file | | | host-manager. <yyyy-mm-dd >.log | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | localhost_access_log. <yyyy-mm-dd >.txt | Tomcat run log file | | | localhost_access_log. <yyyy-mm-dd >.txt | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | manager <yyyy-mm-dd >.log | Tomcat run log file | | | manager <yyyy-mm-dd >.log | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | localhost. <yyyy-mm-dd >.log | Tomcat run log file | | | localhost. <yyyy-mm-dd >.log | Tomcat run log file |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Script log | postInstall.log | Loader installation script log file | | Script log | postInstall.log | Loader installation script log file |
| | | | | | | |
| | | Log file generated during the execution of the Loader installation script (**postInstall.sh**) | | | | Log file generated during the execution of the Loader installation script (**postInstall.sh**) |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | preStart.log | Pre-startup script log file of the Loader service During startup of the Loader service, a series of preparation operations are first performed (by executing **preStart.sh**), such as generating the keytab file. This log file records information about these operations | | | preStart.log | Pre-startup script log file of the Loader service During startup of the Loader service, a series of preparation operations are first performed (by executing **preStart.sh**), such as generating the keytab file. This log file records information about these operations |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| | loader_ctl.log | Log file generated when Loader executes the service start and stop script (**sqoop.sh**) | | | loader_ctl.log | Log file generated when Loader executes the service start and stop script (**sqoop.sh**) |
+-----------------------+-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------+--------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Log Level Log Level
--------- ---------

4
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_hbase.rst
View File

@ -34,13 +34,13 @@ Procedure
+===================================+==============================================================================================================================================================================================================================================================================================================================+ +===================================+==============================================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| HBase Table | Name of a table to which the policy applies. | | HBase Table | Name of a table to which the policy applies. |
| | | | | |
| | The value can contain wildcard (*). For example, **table1:\*** indicates all tables in **table1**. | | | The value can contain wildcard (``*``). For example, **table1:\*** indicates all tables in **table1**. |
| | | | | |
| | The **Include** policy applies to the current input object, and the **Exclude** policy applies to objects other than the current input object. | | | The **Include** policy applies to the current input object, and the **Exclude** policy applies to objects other than the current input object. |
| | | | | |

4
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_hdfs.rst
View File

@ -34,11 +34,11 @@ Procedure
+===================================+===================================================================================================================================================================================================================================================================================================================+ +===================================+===================================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Resource Path | Resource path, which is the HDFS path folder or file to which the current policy applies. You can enter multiple values and use the wildcard (*), for example, **/test/\***. | | Resource Path | Resource path, which is the HDFS path folder or file to which the current policy applies. You can enter multiple values and use the wildcard (``*``), for example, **/test/\***. |
| | | | | |
| | To enable a subdirectory to inherit the permission of its upper-level directory, enable the recursion function. | | | To enable a subdirectory to inherit the permission of its upper-level directory, enable the recursion function. |
| | | | | |

6
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_hive.rst
View File

@ -35,7 +35,7 @@ Procedure
+===================================+==========================================================================================================================================================================================================================================================================================================+ +===================================+==========================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -205,7 +205,7 @@ Ranger supports data masking for Hive data. It can process the returned result o
+===================================+===========================================================================================================================================================================================================================================================================+ +===================================+===========================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -263,7 +263,7 @@ Ranger allows you to filter data at the row level when you perform the **select*
+===================================+===========================================================================================================================================================================================================================================================================+ +===================================+===========================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**, **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

2
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_kafka.rst
View File

@ -34,7 +34,7 @@ Procedure
+===================================+===================================================================================================================================================================================================================================================================================================================+ +===================================+===================================================================================================================================================================================================================================================================================================================+
| Policy Type | Access type. | | Policy Type | Access type. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

6
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_spark2x.rst
View File

@ -52,7 +52,7 @@ Procedure
+===================================+==========================================================================================================================================================================================================================================================================================================+ +===================================+==========================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -206,7 +206,7 @@ Ranger supports data masking for Spark2x data. It can process the returned resul
+===================================+=====================================================================================================================================================================================================================================================================+ +===================================+=====================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -254,7 +254,7 @@ Ranger allows you to filter data at the row level when you perform the **select*
+===================================+===========================================================================================================================================================================================================================================================================+ +===================================+===========================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

2
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_storm.rst
View File

@ -31,7 +31,7 @@ Procedure
+-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Description | | Parameter | Description |
+===================================+===========================================================================================================================================================================================================================================================================================================+ +===================================+===========================================================================================================================================================================================================================================================================================================+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
| | | | | |

4
doc/component-operation-guide/source/using_ranger_mrs_3.x/adding_a_ranger_access_permission_policy_for_yarn.rst
View File

@ -34,11 +34,11 @@ Procedure
+===================================+==========================================================================================================================================================================================================================================================================================================+ +===================================+==========================================================================================================================================================================================================================================================================================================+
| Policy Name | Policy name, which can be customized and must be unique in the service. | | Policy Name | Policy name, which can be customized and must be unique in the service. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (*), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. | | Policy Conditions | IP address filtering policy, which can be customized. You can enter one or more IP addresses or IP address segments. The IP address can contain the wildcard character (``*``), for example, **192.168.1.10**,\ **192.168.1.20**, or **192.168.1.\***. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. | | Policy Label | A label specified for the current policy. You can search for reports and filter policies based on labels. |
+-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Queue | Queue name. The wildcard (*) is supported. | | Queue | Queue name. The wildcard (``*``) is supported. |
| | | | | |
| | To enable a sub-queue to inherit the permission of its upper-level queue, enable the recursion function. | | | To enable a sub-queue to inherit the permission of its upper-level queue, enable the recursion function. |
| | | | | |

2
doc/component-operation-guide/source/using_spark2x/common_issues_about_spark2x/spark_sql_and_dataframe/why_no_result_is_found_when_2016-6-30_is_set_in_the_date_field_as_the_filter_condition.rst
View File

@ -10,7 +10,7 @@ Question
Why no result is found when 2016-6-30 is set in the date field as the filter condition? Why no result is found when 2016-6-30 is set in the date field as the filter condition?
As shown in the following figure, trx_dte_par in the select count (*) from trxfintrx2012 a where trx_dte_par='2016-6-30' statement is a date field. However, no search result is found when the filter condition is where trx_dte_par='2016-6-30'. Search results are found only when the filter condition is where trx_dte_par='2016-06-30'. As shown in the following figure, trx_dte_par in the select count (``*``) from trxfintrx2012 a where trx_dte_par='2016-6-30' statement is a date field. However, no search result is found when the filter condition is where trx_dte_par='2016-6-30'. Search results are found only when the filter condition is where trx_dte_par='2016-06-30'.
.. figure:: /_static/images/en-us_image_0000001348771241.jpg .. figure:: /_static/images/en-us_image_0000001348771241.jpg

56
doc/component-operation-guide/source/using_spark2x/spark2x_logs.rst
View File

@ -29,57 +29,57 @@ Log Description
.. table:: **Table 1** Spark2x log list .. table:: **Table 1** Spark2x log list
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| Log Type | Name | Description | | Log Type | Name | Description |
+=======================+===================================================================================+=================================================================================================+ +=======================+=======================================================================================+=================================================================================================+
| SparkResource2x logs | spark.log | Spark2x service initialization log | | SparkResource2x logs | spark.log | Spark2x service initialization log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | prestart.log | Prestart script log | | | prestart.log | Prestart script log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | cleanup.log | Cleanup log file for instance installation and uninstallation | | | cleanup.log | Cleanup log file for instance installation and uninstallation |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-availability-check.log | Spark2x service health check log | | | spark-availability-check.log | Spark2x service health check log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-service-check.log | Spark2x service check log | | | spark-service-check.log | Spark2x service check log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| JDBCServer2x logs | JDBCServer-start.log | JDBCServer2x startup log | | JDBCServer2x logs | JDBCServer-start.log | JDBCServer2x startup log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | JDBCServer-stop.log | JDBCServer2x stop log | | | JDBCServer-stop.log | JDBCServer2x stop log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | JDBCServer.log | JDBCServer2x run log on the server | | | JDBCServer.log | JDBCServer2x run log on the server |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | jdbc-state-check.log | JDBCServer2x health check log | | | jdbc-state-check.log | JDBCServer2x health check log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | jdbcserver-omm-pid***-gc.log.*.current | IJDBCServer2x process GC log | | | jdbcserver-omm-pid***-gc.log.*.current | IJDBCServer2x process GC log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.sql.hive.thriftserver.HiveThriftProxyServer2-***.out\* | JDBCServer2x process startup log. If the process stops, the **jstack** information is printed. | | | spark-omm-org.apache.spark.sql.hive.thriftserver.HiveThriftProxyServer2-``***``.out\* | JDBCServer2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| JobHistory2x logs | jobHistory-start.log | JobHistory2x startup log | | JobHistory2x logs | jobHistory-start.log | JobHistory2x startup log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | jobHistory-stop.log | JobHistory2x stop log | | | jobHistory-stop.log | JobHistory2x stop log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | JobHistory.log | JobHistory2x running process log | | | JobHistory.log | JobHistory2x running process log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | jobhistory-omm-pid***-gc.log.*.current | JobHistory2x process GC log | | | jobhistory-omm-pid***-gc.log.*.current | JobHistory2x process GC log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.deploy.history.HistoryServer-***.out\* | JobHistory2x process startup log. If the process stops, the **jstack** information is printed. | | | spark-omm-org.apache.spark.deploy.history.HistoryServer-``***``.out\* | JobHistory2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| IndexServer2x logs | IndexServer-start.log | IndexServer2x startup log | | IndexServer2x logs | IndexServer-start.log | IndexServer2x startup log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | IndexServer-stop.log | IndexServer2x stop log | | | IndexServer-stop.log | IndexServer2x stop log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | IndexServer.log | IndexServer2x run log on the server | | | IndexServer.log | IndexServer2x run log on the server |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | indexserver-state-check.log | IndexServer2x health check log | | | indexserver-state-check.log | IndexServer2x health check log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | indexserver-omm-pid***-gc.log.*.current | IndexServer2x process GC log | | | indexserver-omm-pid***-gc.log.*.current | IndexServer2x process GC log |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| | spark-omm-org.apache.spark.sql.hive.thriftserver.IndexServerProxy-***.out\* | IndexServer2x process startup log. If the process stops, the **jstack** information is printed. | | | spark-omm-org.apache.spark.sql.hive.thriftserver.IndexServerProxy-``***``.out\* | IndexServer2x process startup log. If the process stops, the **jstack** information is printed. |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
| Audit Log | jdbcserver-audit.log | JDBCServer2x audit log | | Audit Log | jdbcserver-audit.log | JDBCServer2x audit log |
| | | | | | | |
| | ranger-audit.log | | | | ranger-audit.log | |
+-----------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+ +-----------------------+---------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+
Log levels Log levels
---------- ----------

4
doc/component-operation-guide/source/using_spark2x/using_carbondata_for_first_query.rst
View File

@ -30,7 +30,7 @@ Configure **prequeryParams.properties** by referring to :ref:`Table 1 <mrs_01_23
+==================================+=======================================================================================================================================================================================================================+==================================================================================+ +==================================+=======================================================================================================================================================================================================================+==================================================================================+
| spark.prequery.period.max.minute | Maximum preheating duration, in minutes. | 60 | | spark.prequery.period.max.minute | Maximum preheating duration, in minutes. | 60 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.prequery.tables | Table name configuration, *database.table:int*. The table name supports the wildcard (*). **int** indicates the duration (unit: day) within which the table is updated before it is preheated. | default.test*:10 | | spark.prequery.tables | Table name configuration, *database.table:int*. The table name supports the wildcard (``*``). **int** indicates the duration (unit: day) within which the table is updated before it is preheated. | default.test*:10 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.prequery.maxThreads | Maximum number of concurrent threads during preheating | 50 | | spark.prequery.maxThreads | Maximum number of concurrent threads during preheating | 50 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
@ -38,7 +38,7 @@ Configure **prequeryParams.properties** by referring to :ref:`Table 1 <mrs_01_23
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.prequery.driver | IP address and port number of JDBCServer. The format is *IP address:Port number*. If multiple servers need to be preheated, enter multiple *IP address:Port number* of the servers and separate them with commas (,). | 192.168.0.2:22550 | | spark.prequery.driver | IP address and port number of JDBCServer. The format is *IP address:Port number*. If multiple servers need to be preheated, enter multiple *IP address:Port number* of the servers and separate them with commas (,). | 192.168.0.2:22550 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.prequery.sql | SQL statement for preheating. Different statements are separated by colons (:). | SELECT COUNT(*) FROM %s;SELECT \* FROM %s LIMIT 1 | | spark.prequery.sql | SQL statement for preheating. Different statements are separated by colons (:). | SELECT COUNT(``*``) FROM %s;SELECT \* FROM %s LIMIT 1 |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| spark.security.url | URL required by JDBC in security mode | ;saslQop=auth-conf;auth=KERBEROS;principal=spark2x/hadoop.hadoop.com@HADOOP.COM; | | spark.security.url | URL required by JDBC in security mode | ;saslQop=auth-conf;auth=KERBEROS;principal=spark2x/hadoop.hadoop.com@HADOOP.COM; |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+