forked from docs/mapreduce-service
propose-mrs_operation-guide-lts-748-RST repair #1
@ -1,13 +1,18 @@
|
||||
:original_name: en-us_topic_0000001298722056.html
|
||||
:original_name: mrs_01_17512.html
|
||||
|
||||
.. _en-us_topic_0000001298722056:
|
||||
.. _mrs_01_17512:
|
||||
|
||||
Change History
|
||||
==============
|
||||
|
||||
+-----------------------------------+-----------------------------------------------------------------------------------------------------+
|
||||
+-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Released On | What's New |
|
||||
+===================================+=====================================================================================================+
|
||||
+===================================+=====================================================================================================================================================================================================================================================================================================================================================+
|
||||
| 2023-11-01 | - Deleted the description of some parameters that are not applicable to Flink. For details, see :ref:`JobManager & TaskManager <mrs_01_1566>`. |
|
||||
| | - Modified the description of the digital display identifier in the Ranger anonymization policy. For details, see :ref:`Adding a Ranger Access Permission Policy for Hive <mrs_01_1858>`, :ref:`Adding a Ranger Access Permission Policy for Spark2x <mrs_01_1860>`, :ref:`Adding a Ranger Access Permission Policy for HetuEngine <mrs_01_1862>`. |
|
||||
| | - Deleted related content because Hive materialized views are no longer recommended. |
|
||||
| | - The titles of some sections are optimized to avoid repeated content in the sections where components are not connected. |
|
||||
+-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| 2023-05-04 | Modified the feature documentation of MRS 3.2.0-LTS.1. |
|
||||
| | |
|
||||
| | - Added the following content: |
|
||||
@ -44,7 +49,7 @@ Change History
|
||||
| | |
|
||||
| | - Modified the following content: |
|
||||
| | |
|
||||
| | - :ref:`Configuring the Password of the Default Account of a ClickHouse Cluster <mrs_01_2395>` |
|
||||
| | - :ref:`Configuring the Password of the Default Account of a ClickHouse Cluster(for MRS 3.1.2) <mrs_01_2395>` |
|
||||
| | - :ref:`Interconnecting FlinkServer with ClickHouse <mrs_01_24148>` |
|
||||
| | - :ref:`Interconnecting FlinkServer with Hudi <mrs_01_24180>` |
|
||||
| | - :ref:`Introduction to Flume Logs <mrs_01_1081>` |
|
||||
@ -53,6 +58,6 @@ Change History
|
||||
| | - :ref:`Hive Supporting Transactions <mrs_01_0975>` |
|
||||
| | - :ref:`Stream Write <mrs_01_24036>` |
|
||||
| | - :ref:`Introduction to Kafka Logs <mrs_01_1042>` |
|
||||
+-----------------------------------+-----------------------------------------------------------------------------------------------------+
|
||||
+-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| 2022-11-01 | This issue is the first official release. |
|
||||
+-----------------------------------+-----------------------------------------------------------------------------------------------------+
|
||||
+-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
Using CarbonData
|
||||
================
|
||||
|
||||
- :ref:`Overview <mrs_01_1401>`
|
||||
- :ref:`Spark CarbonData Overview <mrs_01_1401>`
|
||||
- :ref:`Configuration Reference <mrs_01_1404>`
|
||||
- :ref:`CarbonData Operation Guide <mrs_01_1405>`
|
||||
- :ref:`CarbonData Performance Tuning <mrs_01_1417>`
|
||||
@ -18,7 +18,7 @@ Using CarbonData
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
overview/index
|
||||
spark_carbondata_overview/index
|
||||
configuration_reference
|
||||
carbondata_operation_guide/index
|
||||
carbondata_performance_tuning/index
|
||||
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_1401:
|
||||
|
||||
Overview
|
||||
========
|
||||
Spark CarbonData Overview
|
||||
=========================
|
||||
|
||||
- :ref:`CarbonData Overview <mrs_01_1402>`
|
||||
- :ref:`Main Specifications of CarbonData <mrs_01_1403>`
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_24790:
|
||||
|
||||
Overview
|
||||
========
|
||||
ClickHouse Multi-Tenancy Overview
|
||||
=================================
|
||||
|
||||
.. note::
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
ClickHouse Multi-Tenancy
|
||||
========================
|
||||
|
||||
- :ref:`Overview <mrs_01_24790>`
|
||||
- :ref:`ClickHouse Multi-Tenancy Overview <mrs_01_24790>`
|
||||
- :ref:`Enabling the CPU Priority Feature <mrs_01_24789>`
|
||||
- :ref:`Managing ClickHouse Tenants <mrs_01_24791>`
|
||||
- :ref:`Modifying the Memory Limit of ClickHouse on a ClickHouseServer Node <mrs_01_24786>`
|
||||
@ -14,7 +14,7 @@ ClickHouse Multi-Tenancy
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
overview
|
||||
clickhouse_multi-tenancy_overview
|
||||
enabling_the_cpu_priority_feature
|
||||
managing_clickhouse_tenants
|
||||
modifying_the_memory_limit_of_clickhouse_on_a_clickhouseserver_node
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,45 @@
|
||||
:original_name: mrs_01_2395.html
|
||||
|
||||
.. _mrs_01_2395:
|
||||
|
||||
Configuring the Password of the Default Account of a ClickHouse Cluster(for MRS 3.1.2)
|
||||
======================================================================================
|
||||
|
||||
After a ClickHouse cluster is created, you can use the ClickHouse client to connect to the ClickHouse server. The default username is **default**.
|
||||
|
||||
This section describes how to set ClickHouse username and password after a ClickHouse cluster is successfully created.
|
||||
|
||||
.. note::
|
||||
|
||||
- This section applies to MRS 3.1.2.
|
||||
- **default** is the default internal user of ClickHouse. It is an administrator user available only in normal mode (kerberos authentication disabled).
|
||||
|
||||
Setting the ClickHouse Username and Password
|
||||
--------------------------------------------
|
||||
|
||||
#. Log in to Manager and choose **Cluster** > **Services** > **ClickHouse**. Click the **Configurations** tab and then **All Configurations**.
|
||||
|
||||
#. Search for the **users.default.password** parameter in the search box and change its password, as shown in :ref:`Figure 1 <mrs_01_2395__fig42238424419>`.
|
||||
|
||||
.. _mrs_01_2395__fig42238424419:
|
||||
|
||||
.. figure:: /_static/images/en-us_image_0000001296059672.png
|
||||
:alt: **Figure 1** Changing the default user password
|
||||
|
||||
**Figure 1** Changing the default user password
|
||||
|
||||
#. Log in to the node where the client is installed and run the following command to switch to the client installation directory.
|
||||
|
||||
**cd**\ *Cluster client installation directory*
|
||||
|
||||
#. Run the following command to configure environment variables:
|
||||
|
||||
**source bigdata_env**
|
||||
|
||||
#. Log in to ClickHouse using the new password.
|
||||
|
||||
**clickhouse client --host** *IP address of the ClickHouse instance* **--user** *default* **--password** *xxx*
|
||||
|
||||
.. note::
|
||||
|
||||
To obtain the service IP address of the ClickHouse instance, choose **Components** > **ClickHouse** > **Instances** on the cluster details page.
|
||||
@ -6,7 +6,7 @@ User Management and Authentication
|
||||
==================================
|
||||
|
||||
- :ref:`ClickHouse User and Permission Management <mrs_01_24057>`
|
||||
- :ref:`Configuring the Password of the Default Account of a ClickHouse Cluster <mrs_01_2395>`
|
||||
- :ref:`Configuring the Password of the Default Account of a ClickHouse Cluster(for MRS 3.1.2) <mrs_01_2395>`
|
||||
- :ref:`Configuring the Password of the Default Account of a ClickHouse Cluster <mrs_01_24575>`
|
||||
|
||||
.. toctree::
|
||||
@ -14,5 +14,5 @@ User Management and Authentication
|
||||
:hidden:
|
||||
|
||||
clickhouse_user_and_permission_management
|
||||
configuring_the_password_of_the_default_account_of_a_clickhouse_cluster
|
||||
configuring_the_password_of_the_default_account_of_a_clickhouse_clusterfor_mrs_3.1.2
|
||||
configuring_the_password_of_the_default_account_of_a_clickhouse_cluster
|
||||
|
||||
@ -38,14 +38,8 @@ Main configuration items include communication port, memory management, connecti
|
||||
+------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
|
||||
| taskmanager.network.numberOfBuffers | Number of TaskManager network transmission buffer stacks. If an error indicates insufficient system buffer, increase the parameter value. | 2048 | No |
|
||||
+------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
|
||||
| taskmanager.memory.fraction | Ratio of JVM heap memory that TaskManager reserves for sorting, hash tables, and caching of intermediate results. | 0.7 | No |
|
||||
+------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
|
||||
| taskmanager.memory.off-heap | Whether TaskManager uses off-heap memory for sorting, hash tables and intermediate status. You are advised to enable this item for large memory needs to improve memory operation efficiency. | false | Yes |
|
||||
+------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
|
||||
| taskmanager.memory.segment-size | Size of the memory buffer used by the memory manager and network stack The unit is bytes. | 32768 | No |
|
||||
+------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
|
||||
| taskmanager.memory.preallocate | Whether TaskManager allocates reserved memory space upon startup. You are advised to enable this item when off-heap memory is used. | false | No |
|
||||
+------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
|
||||
| taskmanager.debug.memory.startLogThread | Enable this item for debugging Flink memory and garbage collection (GC)-related problems. TaskManager periodically collects memory and GC statistics, including the current utilization of heap and off-heap memory pools and GC time. | false | No |
|
||||
+------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
|
||||
| taskmanager.debug.memory.logIntervalMs | Interval at which TaskManager periodically collects memory and GC statistics. | 0 | No |
|
||||
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_1590:
|
||||
|
||||
Configuring Process Parameters
|
||||
==============================
|
||||
Configuring Flink Process Parameters
|
||||
====================================
|
||||
|
||||
Scenario
|
||||
--------
|
||||
@ -7,7 +7,7 @@ Optimization DataStream
|
||||
|
||||
- :ref:`Memory Configuration Optimization <mrs_01_1588>`
|
||||
- :ref:`Configuring DOP <mrs_01_1589>`
|
||||
- :ref:`Configuring Process Parameters <mrs_01_1590>`
|
||||
- :ref:`Configuring Flink Process Parameters <mrs_01_1590>`
|
||||
- :ref:`Optimizing the Design of Partitioning Method <mrs_01_1591>`
|
||||
- :ref:`Configuring the Netty Network Communication <mrs_01_1592>`
|
||||
- :ref:`Summarization <mrs_01_1593>`
|
||||
@ -18,7 +18,7 @@ Optimization DataStream
|
||||
|
||||
memory_configuration_optimization
|
||||
configuring_dop
|
||||
configuring_process_parameters
|
||||
configuring_flink_process_parameters
|
||||
optimizing_the_design_of_partitioning_method
|
||||
configuring_the_netty_network_communication
|
||||
summarization
|
||||
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_24015:
|
||||
|
||||
Overview
|
||||
========
|
||||
Flink Web UI Overview
|
||||
=====================
|
||||
|
||||
- :ref:`Introduction to Flink Web UI <mrs_01_24016>`
|
||||
- :ref:`Flink Web UI Application Process <mrs_01_24017>`
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_24048:
|
||||
|
||||
Overview
|
||||
========
|
||||
FlinkServer Permissions Overview
|
||||
================================
|
||||
|
||||
User **admin** of Manager does not have the FlinkServer service operation permission. To perform FlinkServer service operations, you need to grant related permission to the user.
|
||||
|
||||
@ -5,12 +5,12 @@
|
||||
FlinkServer Permissions Management
|
||||
==================================
|
||||
|
||||
- :ref:`Overview <mrs_01_24048>`
|
||||
- :ref:`FlinkServer Permissions Overview <mrs_01_24048>`
|
||||
- :ref:`Authentication Based on Users and Roles <mrs_01_24049>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
overview
|
||||
flinkserver_permissions_overview
|
||||
authentication_based_on_users_and_roles
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
Using the Flink Web UI
|
||||
======================
|
||||
|
||||
- :ref:`Overview <mrs_01_24015>`
|
||||
- :ref:`Flink Web UI Overview <mrs_01_24015>`
|
||||
- :ref:`FlinkServer Permissions Management <mrs_01_24047>`
|
||||
- :ref:`Accessing the Flink Web UI <mrs_01_24019>`
|
||||
- :ref:`Creating an Application on the Flink Web UI <mrs_01_24020>`
|
||||
@ -14,14 +14,14 @@ Using the Flink Web UI
|
||||
- :ref:`Managing Tables on the Flink Web UI <mrs_01_24023>`
|
||||
- :ref:`Managing Jobs on the Flink Web UI <mrs_01_24024>`
|
||||
- :ref:`Importing and Exporting Jobs <mrs_01_24481>`
|
||||
- :ref:`Managing UDFs on the Flink Web UI <mrs_01_24223>`
|
||||
- :ref:`Managing UDFs <mrs_01_24223>`
|
||||
- :ref:`Interconnecting FlinkServer with External Components <mrs_01_24226>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
overview/index
|
||||
flink_web_ui_overview/index
|
||||
flinkserver_permissions_management/index
|
||||
accessing_the_flink_web_ui
|
||||
creating_an_application_on_the_flink_web_ui
|
||||
@ -30,5 +30,5 @@ Using the Flink Web UI
|
||||
managing_tables_on_the_flink_web_ui
|
||||
managing_jobs_on_the_flink_web_ui
|
||||
importing_and_exporting_jobs
|
||||
managing_udfs_on_the_flink_web_ui/index
|
||||
managing_udfs/index
|
||||
interconnecting_flinkserver_with_external_components/index
|
||||
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_24223:
|
||||
|
||||
Managing UDFs on the Flink Web UI
|
||||
=================================
|
||||
Managing UDFs
|
||||
=============
|
||||
|
||||
- :ref:`Managing UDFs on the Flink Web UI <mrs_01_24211>`
|
||||
- :ref:`UDF Java and SQL Examples <mrs_01_24224>`
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_1074:
|
||||
|
||||
Overview
|
||||
========
|
||||
Flume Service Model Overview
|
||||
============================
|
||||
|
||||
Guide a reasonable Flume service configuration by providing performance differences between Flume common modules, to avoid a nonstandard overall service performance caused when a frontend Source and a backend Sink do not match in performance.
|
||||
|
||||
@ -5,12 +5,12 @@
|
||||
Configuring the Flume Service Model
|
||||
===================================
|
||||
|
||||
- :ref:`Overview <mrs_01_1074>`
|
||||
- :ref:`Flume Service Model Overview <mrs_01_1074>`
|
||||
- :ref:`Service Model Configuration Guide <mrs_01_1075>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
overview
|
||||
flume_service_model_overview
|
||||
service_model_configuration_guide
|
||||
|
||||
@ -6,11 +6,11 @@ Encrypted Transmission
|
||||
======================
|
||||
|
||||
- :ref:`Configuring the Encrypted Transmission <mrs_01_1069>`
|
||||
- :ref:`Typical Scenario: Collecting Local Static Logs and Uploading Them to HDFS <mrs_01_1070>`
|
||||
- :ref:`Typical Scenario: Collecting Local Static Logs and Uploading Them to HDFS Encrypted Transmission <mrs_01_1070>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
configuring_the_encrypted_transmission
|
||||
typical_scenario_collecting_local_static_logs_and_uploading_them_to_hdfs
|
||||
typical_scenario_collecting_local_static_logs_and_uploading_them_to_hdfs_encrypted_transmission
|
||||
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_1070:
|
||||
|
||||
Typical Scenario: Collecting Local Static Logs and Uploading Them to HDFS
|
||||
=========================================================================
|
||||
Typical Scenario: Collecting Local Static Logs and Uploading Them to HDFS Encrypted Transmission
|
||||
================================================================================================
|
||||
|
||||
Scenario
|
||||
--------
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_0391:
|
||||
|
||||
Overview
|
||||
========
|
||||
Flume Overview
|
||||
==============
|
||||
|
||||
Flume is a distributed, reliable, and highly available system for aggregating massive logs, which can efficiently collect, aggregate, and move massive log data from different data sources and store the data in a centralized data storage system. Various data senders can be customized in the system to collect data. Additionally, Flume provides simple data processes capabilities and writes data to data receivers (which is customizable).
|
||||
|
||||
@ -6,7 +6,7 @@ Using Flume
|
||||
===========
|
||||
|
||||
- :ref:`Using Flume from Scratch <mrs_01_0397>`
|
||||
- :ref:`Overview <mrs_01_0391>`
|
||||
- :ref:`Flume Overview <mrs_01_0391>`
|
||||
- :ref:`Installing the Flume Client on Clusters <mrs_01_1595>`
|
||||
- :ref:`Viewing Flume Client Logs <mrs_01_0393>`
|
||||
- :ref:`Stopping or Uninstalling the Flume Client <mrs_01_0394>`
|
||||
@ -30,7 +30,7 @@ Using Flume
|
||||
:hidden:
|
||||
|
||||
using_flume_from_scratch
|
||||
overview
|
||||
flume_overview
|
||||
installing_the_flume_client_on_clusters
|
||||
viewing_flume_client_logs
|
||||
stopping_or_uninstalling_the_flume_client
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
:original_name: en-us_topic_0000001295898904.html
|
||||
:original_name: mrs_01_24112.html
|
||||
|
||||
.. _en-us_topic_0000001295898904:
|
||||
.. _mrs_01_24112:
|
||||
|
||||
Configuring HBase Data Compression and Encoding
|
||||
===============================================
|
||||
|
||||
@ -10,7 +10,7 @@ HBase Performance Tuning
|
||||
- :ref:`Optimizing Put and Scan Performance <mrs_01_1016>`
|
||||
- :ref:`Improving Real-time Data Write Performance <mrs_01_1017>`
|
||||
- :ref:`Improving Real-time Data Read Performance <mrs_01_1018>`
|
||||
- :ref:`Optimizing JVM Parameters <mrs_01_1019>`
|
||||
- :ref:`Optimizing HBase JVM Parameters <mrs_01_1019>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
@ -21,4 +21,4 @@ HBase Performance Tuning
|
||||
optimizing_put_and_scan_performance
|
||||
improving_real-time_data_write_performance
|
||||
improving_real-time_data_read_performance
|
||||
optimizing_jvm_parameters
|
||||
optimizing_hbase_jvm_parameters
|
||||
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_1019:
|
||||
|
||||
Optimizing JVM Parameters
|
||||
=========================
|
||||
Optimizing HBase JVM Parameters
|
||||
===============================
|
||||
|
||||
Scenario
|
||||
--------
|
||||
@ -15,7 +15,7 @@ Using HBase
|
||||
- :ref:`In-House Enhanced Phoenix <mrs_01_24579>`
|
||||
- :ref:`Configuring HBase DR <mrs_01_1609>`
|
||||
- :ref:`Performing an HBase DR Service Switchover <mrs_01_1610>`
|
||||
- :ref:`Configuring HBase Data Compression and Encoding <en-us_topic_0000001295898904>`
|
||||
- :ref:`Configuring HBase Data Compression and Encoding <mrs_01_24112>`
|
||||
- :ref:`Performing an HBase DR Active/Standby Cluster Switchover <mrs_01_1611>`
|
||||
- :ref:`Community BulkLoad Tool <mrs_01_1612>`
|
||||
- :ref:`Configuring the MOB <mrs_01_1631>`
|
||||
|
||||
@ -26,7 +26,7 @@ HetuEngine supports quick joint query of multiple data sources and GUI-based dat
|
||||
+-----------------+---------------+------------------+---------------------------------------------------------------+
|
||||
| | GaussDB | | GaussDB 200 and GaussDB A 8.0.0 |
|
||||
+-----------------+---------------+------------------+---------------------------------------------------------------+
|
||||
| | Hudi | | MRS 3.1.1 or later |
|
||||
| | Hudi | | MRS 3.1.2 or later |
|
||||
+-----------------+---------------+------------------+---------------------------------------------------------------+
|
||||
| | ClickHouse | | MRS 3.1.1 or later |
|
||||
+-----------------+---------------+------------------+---------------------------------------------------------------+
|
||||
@ -36,7 +36,7 @@ HetuEngine supports quick joint query of multiple data sources and GUI-based dat
|
||||
+-----------------+---------------+------------------+---------------------------------------------------------------+
|
||||
| | Elasticsearch | | Elasticsearch of the current cluster |
|
||||
+-----------------+---------------+------------------+---------------------------------------------------------------+
|
||||
| | Hudi | | MRS 3.1.1 or later |
|
||||
| | Hudi | | MRS 3.1.2 or later |
|
||||
+-----------------+---------------+------------------+---------------------------------------------------------------+
|
||||
| | ClickHouse | | MRS 3.1.1 or later |
|
||||
+-----------------+---------------+------------------+---------------------------------------------------------------+
|
||||
|
||||
@ -27,38 +27,34 @@ Concepts
|
||||
|
||||
.. table:: **Table 1** Using HetuEngine tables, columns, or data
|
||||
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| Scenario | Required Permission |
|
||||
+===================================+=============================================================================================================================================+
|
||||
+============================+===========================================================================================+
|
||||
| DESCRIBE TABLE | Select |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| ANALYZE TABLE | Select and Insert |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| SHOW COLUMNS | Select |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| SHOW TABLE STATUS | Select |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| SHOW TABLE PROPERTIES | Select |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| SELECT | Select |
|
||||
| | |
|
||||
| | .. note:: |
|
||||
| | |
|
||||
| | To perform the SELECT operation on a view, you must have the **Select** permission on the view and the tables corresponding to the view. |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| EXPLAIN | Select |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| CREATE VIEW | Select, Grant Of Select, and Create |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| CREATE TABLE | Create |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| ALTER TABLE ADD PARTITION | Insert |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| INSERT | Insert |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| INSERT OVERWRITE | Insert and Delete |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| ALTER TABLE DROP PARTITION | The table-level Alter and Delete, and column-level Select permissions need to be granted. |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
| ALTER DATABASE | Hive Admin Privilege |
|
||||
+-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
+----------------------------+-------------------------------------------------------------------------------------------+
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
HetuEngine MetaStore-based Permission Control
|
||||
=============================================
|
||||
|
||||
- :ref:`Overview <mrs_01_1725>`
|
||||
- :ref:`MetaStore Permission Overview <mrs_01_1725>`
|
||||
- :ref:`Creating a HetuEngine Role <mrs_01_2350>`
|
||||
- :ref:`Configuring Permissions for Tables, Columns, and Databases <mrs_01_2352>`
|
||||
|
||||
@ -13,6 +13,6 @@ HetuEngine MetaStore-based Permission Control
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
overview
|
||||
metastore_permission_overview
|
||||
creating_a_hetuengine_role
|
||||
configuring_permissions_for_tables_columns_and_databases
|
||||
|
||||
@ -2,8 +2,8 @@
|
||||
|
||||
.. _mrs_01_1725:
|
||||
|
||||
Overview
|
||||
========
|
||||
MetaStore Permission Overview
|
||||
=============================
|
||||
|
||||
Constraints: This parameter applies only to the Hive data source.
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
Using HetuEngine Cross-Domain Function
|
||||
======================================
|
||||
|
||||
- :ref:`Introduction to HetuEngine Cross-Source Function <mrs_01_2334>`
|
||||
- :ref:`Introduction to HetuEngine Cross-Domain Function <mrs_01_2334>`
|
||||
- :ref:`HetuEngine Cross-Domain Function Usage <mrs_01_2335>`
|
||||
- :ref:`HetuEngine Cross-Domain Rate Limit Function <mrs_01_24284>`
|
||||
|
||||
@ -13,6 +13,6 @@ Using HetuEngine Cross-Domain Function
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
introduction_to_hetuengine_cross-source_function
|
||||
introduction_to_hetuengine_cross-domain_function
|
||||
hetuengine_cross-domain_function_usage
|
||||
hetuengine_cross-domain_rate_limit_function
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
.. _mrs_01_2334:
|
||||
|
||||
Introduction to HetuEngine Cross-Source Function
|
||||
Introduction to HetuEngine Cross-Domain Function
|
||||
================================================
|
||||
|
||||
HetuEngine provide unified standard SQL to implement efficient access to multiple data sources distributed in multiple regions (or data centers), shields data differences in the structure, storage, and region, and decouples data and applications.
|
||||
@ -53,9 +53,9 @@ For details about the SQL statements for materialized views, see :ref:`Table 1 <
|
||||
+-----------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Verifying whether SQL statements can be optimized by rewriting a query to a materialized view | Verify whether the SELECT statement can be rewritten and optimized by **mv.default.mv1**. | verify materialized view mvname(mv.default.mv1) originalsql select id from hive.mvschema.t1; | ``-`` |
|
||||
+-----------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Enabling the specified materialized view at the SQL level to optimize the SQL statements | Forcibly use **mv.default.mv1** for SQL statement optimization in queries. | ``/*+ REWRITE(mv.default.mv1) */`` select id from hive.mvschema.t1; | ``-`` |
|
||||
| Enabling the specified materialized view at the SQL level to optimize the SQL statements | Forcibly use **mv.default.mv1** for SQL statement optimization in queries. | /``*+`` REWRITE(mv.default.mv1) \*/ select id from hive.mvschema.t1; | ``-`` |
|
||||
+-----------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Disabling materialized views at the SQL level to optimize the SQL statements | Do not use materialized views for SQL statement optimization in queries. | ``/*+ NOREWRITE */`` select id from hive.mvschema.t1; | ``-`` |
|
||||
| Disabling materialized views at the SQL level to optimize the SQL statements | Do not use materialized views for SQL statement optimization in queries. | /``*+`` NOREWRITE \*/ select id from hive.mvschema.t1; | ``-`` |
|
||||
+-----------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Refreshing the metadata cache of materialized views | Synchronize the metadata cache of materialized views between tenants. | refresh catalog mv; | ``-`` |
|
||||
+-----------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
:original_name: en-us_topic_0000001295738648.html
|
||||
:original_name: mrs_01_17511.html
|
||||
|
||||
.. _en-us_topic_0000001295738648:
|
||||
.. _mrs_01_17511:
|
||||
|
||||
Connecting Hive with External RDS
|
||||
=================================
|
||||
@ -105,7 +105,7 @@ Connecting Hive with External RDS
|
||||
|
||||
#. Log in to each MetaStore background node and check whether the local directory **/opt/Bigdata/tmp** exists.
|
||||
|
||||
- If yes, go to :ref:`8 <en-us_topic_0000001295738648__en-us_topic_0000001219350615_li24241321154318>`.
|
||||
- If yes, go to :ref:`8 <mrs_01_17511__en-us_topic_0000001219350615_li24241321154318>`.
|
||||
|
||||
- If no, run the following commands to create one:
|
||||
|
||||
@ -113,6 +113,6 @@ Connecting Hive with External RDS
|
||||
|
||||
**chmod 755 /opt/Bigdata/tmp**
|
||||
|
||||
#. .. _en-us_topic_0000001295738648__en-us_topic_0000001219350615_li24241321154318:
|
||||
#. .. _mrs_01_17511__en-us_topic_0000001219350615_li24241321154318:
|
||||
|
||||
Save the configuration. Choose **Dashboard** > **More** > **Restart Service**, and enter the password to restart the Hive service.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -34,10 +34,9 @@ Using Hive
|
||||
- :ref:`Hive Supports Isolation of Metastore instances Based on Components <mrs_01_24467>`
|
||||
- :ref:`Hive Supporting Transactions <mrs_01_0975>`
|
||||
- :ref:`Switching the Hive Execution Engine to Tez <mrs_01_1750>`
|
||||
- :ref:`Connecting Hive with External RDS <en-us_topic_0000001295738648>`
|
||||
- :ref:`Connecting Hive with External RDS <mrs_01_17511>`
|
||||
- :ref:`Interconnecting Hive with External Self-Built Relational Databases <mrs_01_1751>`
|
||||
- :ref:`Redis-based CacheStore of HiveMetaStore <mrs_01_2302>`
|
||||
- :ref:`Hive Materialized View <mrs_01_2311>`
|
||||
- :ref:`Hive Supporting Reading Hudi Tables <mrs_01_24040>`
|
||||
- :ref:`Hive Supporting Cold and Hot Storage of Partitioned Metadata <mrs_01_24118>`
|
||||
- :ref:`Hive Supporting ZSTD Compression Formats <mrs_01_24121>`
|
||||
@ -85,7 +84,6 @@ Using Hive
|
||||
connecting_hive_with_external_rds
|
||||
interconnecting_hive_with_external_self-built_relational_databases
|
||||
redis-based_cachestore_of_hivemetastore
|
||||
hive_materialized_view
|
||||
hive_supporting_reading_hudi_tables
|
||||
hive_supporting_cold_and_hot_storage_of_partitioned_metadata
|
||||
hive_supporting_zstd_compression_formats
|
||||
|
||||
@ -19,4 +19,4 @@ You can use either of the following methods to perform cleaning:
|
||||
|
||||
**spark-submit --master yarn --jars /opt/client/Hudi/hudi/lib/hudi-client-common-**\ *xxx*\ **.jar --class org.apache.hudi.utilities.HoodieCleaner /opt/client/Hudi/hudi/lib/hudi-utilities\_**\ *xxx*\ **.jar --target-base-path /tmp/default/tb_test_mor**
|
||||
|
||||
For details about more cleaning parameters, see :ref:`Configuration Reference <mrs_01_24032>`.
|
||||
For details about more cleaning parameters, see :ref:`Hudi Configuration Reference <mrs_01_24032>`.
|
||||
|
||||
@ -82,7 +82,7 @@ How to Execute Clustering
|
||||
|
||||
hoodie.clustering.inline.max.commits=4
|
||||
|
||||
For details, see :ref:`Configuration Reference <mrs_01_24032>`.
|
||||
For details, see :ref:`Hudi Configuration Reference <mrs_01_24032>`.
|
||||
|
||||
.. caution::
|
||||
|
||||
|
||||
@ -31,7 +31,7 @@ Metadata Table
|
||||
|
||||
When using Spark to write data, set **hoodie.metadata.enable** in the **option** parameter to **true**.
|
||||
|
||||
For details about more parameters, see :ref:`Configuration Reference <mrs_01_24032>` or visit Hudi official website http://hudi.apache.org/docs/configurations.html#metadata-config.
|
||||
For details about more parameters, see :ref:`Hudi Configuration Reference <mrs_01_24032>` or visit Hudi official website http://hudi.apache.org/docs/configurations.html#metadata-config.
|
||||
|
||||
- **Performance improvement**
|
||||
|
||||
|
||||
@ -54,7 +54,7 @@ How to Use the Concurrency Mechanism
|
||||
|
||||
**hoodie.write.lock.zookeeper.base_path**\ =\ *<table_path>*
|
||||
|
||||
For details about more parameters, see :ref:`Configuration Reference <mrs_01_24032>`.
|
||||
For details about more parameters, see :ref:`Hudi Configuration Reference <mrs_01_24032>`.
|
||||
|
||||
.. caution::
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user