mikacur/data-ingestion-service_aligns_with_the_OTC_platform_07_23
1
0
Fork 0
forked from docs/data-ingestion-service
Code Pull Requests Activity

Changes to dis_api-ref from docs/doc-exports#574 (DIS API 20230116 version

Browse Source 
Alig

Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com>
Co-authored-by: proposalbot <proposalbot@otc-service.com>
Co-committed-by: proposalbot <proposalbot@otc-service.com>
This commit is contained in:
proposalbot 2023-06-27 15:25:12 +00:00 committed by zuul
parent 94e75fbba4
commit f8c0bfcf4c
47 changed files with 1232 additions and 3601 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

0
api-ref/source/_static/images/en-us_image_0000001238312593.jpg → api-ref/source/_static/images/en-us_image_0191919629.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 110 KiB

After

Width:  |  Height:  |  Size: 110 KiB

38
api-ref/source/api_description/app_management/creating_consumption_apps.rst
View File

@ -15,7 +15,7 @@ URI
POST /v2/{project_id}/apps
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ===========
Parameter Mandatory Type Description
@ -28,27 +28,27 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
.. table:: **Table 3** Request body parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=================================================================================================================+
| app_name | Yes | String | Unique identifier of the consumer application to be created. |
| | | | |
| | | | The application name contains 1 to 200 characters, including letters, digits, underscores (_), and hyphens (-). |
| | | | |
| | | | Minimum: **1** |
| | | | |
| | | | Maximum: **200** |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+========================================================================================================================+
| app_name | Yes | String | Name of the consumer application to be created |
| | | | |
| | | | The application name contains 1 to 200 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| | | | |
| | | | Minimum: **1** |
| | | | |
| | | | Maximum: **200** |
+-----------------+-----------------+-----------------+------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

16
api-ref/source/api_description/app_management/deleting_apps.rst
View File

@ -15,7 +15,7 @@ URI
DELETE /v2/{project_id}/apps/{app_name}
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ==============================
Parameter Mandatory Type Description
@ -29,13 +29,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

83
api-ref/source/api_description/app_management/querying_app_consumption_status.rst
View File

File diff suppressed because it is too large Load Diff

16
api-ref/source/api_description/app_management/querying_app_details.rst
View File

@ -15,7 +15,7 @@ URI
GET /v2/{project_id}/apps/{app_name}
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ==============================
Parameter Mandatory Type Description
@ -29,13 +29,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

67
api-ref/source/api_description/app_management/querying_apps.rst
View File

File diff suppressed because it is too large Load Diff

66
api-ref/source/api_description/checkpoint_management/deleting_checkpoints.rst
View File

@ -15,7 +15,7 @@ URI
DELETE /v2/{project_id}/checkpoints
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ===========
Parameter Mandatory Type Description
@ -23,46 +23,44 @@ DELETE /v2/{project_id}/checkpoints
project_id Yes String Project ID.
========== ========= ====== ===========
.. table:: **Table 2** Query parameters
.. table:: **Table 2** Query Parameters
+-----------------+-----------------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=========================================================================================================================================================================+
| stream_name | Yes | String | Name of the stream to which the checkpoint belongs. |
+-----------------+-----------------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| app_name | Yes | String | Name of the application associated with the checkpoint. |
| | | | |
| | | | Minimum: **1** |
| | | | |
| | | | Maximum: **50** |
+-----------------+-----------------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checkpoint_type | Yes | String | Type of the checkpoint. LAST_READ: Only sequence numbers are recorded in databases. |
| | | | |
| | | | Enumeration values: |
| | | | |
| | | | - **LAST_READ** |
+-----------------+-----------------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| partition_id | No | String | Identifier of the stream partition to which the checkpoint belongs. The value can be in either of the following formats: |
| | | | |
| | | | - shardId-0000000000 |
| | | | |
| | | | - 0 |
| | | | |
| | | | For example, if a stream has three partitions, the partition identifiers are 0, 1, and 2, and shardId-0000000000, shardId-0000000001, shardId-0000000002, respectively. |
+-----------------+-----------------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+============================================================================================================================================================================+
| stream_name | Yes | String | Name of the stream to which the checkpoint belongs. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| app_name | Yes | String | Name of the application associated with the checkpoint. |
| | | | |
| | | | Minimum: **1** |
| | | | |
| | | | Maximum: **50** |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checkpoint_type | Yes | String | Type of the checkpoint |
| | | | |
| | | | - **LAST_READ**: Only sequence numbers are recorded in databases. |
| | | | |
| | | | Enumeration values: |
| | | | |
| | | | - **LAST_READ** |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| partition_id | No | String | Identifier of the stream partition to which the checkpoint belongs. The value can be in either of the following formats:- shardId-0000000000- 0 |
| | | | |
| | | | For example, if a stream has three partitions, the partition identifiers are 0, 1, and 2, or shardId-0000000000, shardId-0000000001, and shardId-0000000002, respectively. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Request Parameters
------------------
.. table:: **Table 3** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

28
api-ref/source/api_description/checkpoint_management/querying_checkpoint_details.rst
View File

@ -15,7 +15,7 @@ URI
GET /v2/{project_id}/checkpoints
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ===========
Parameter Mandatory Type Description
@ -23,26 +23,22 @@ GET /v2/{project_id}/checkpoints
project_id Yes String Project ID.
========== ========= ====== ===========
.. table:: **Table 2** Query parameters
.. table:: **Table 2** Query Parameters
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+============================================================================================================================================================================+
| stream_name | Yes | String | Name of the stream to which the checkpoint belongs. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| partition_id | Yes | String | Identifier of the stream partition to which the checkpoint belongs. The value can be in either of the following formats: |
| | | | |
| | | | - shardId-0000000000 |
| | | | |
| | | | - 0 |
| partition_id | Yes | String | Identifier of the stream partition to which the checkpoint belongs. The value can be in either of the following formats:- shardId-0000000000- 0 |
| | | | |
| | | | For example, if a stream has three partitions, the partition identifiers are 0, 1, and 2, or shardId-0000000000, shardId-0000000001, and shardId-0000000002, respectively. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| app_name | Yes | String | Name of the app associated with the checkpoint. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checkpoint_type | Yes | String | Type of the checkpoint. |
| checkpoint_type | Yes | String | Type of the checkpoint |
| | | | |
| | | | - LAST_READ: Only sequence numbers are recorded in databases. |
| | | | - **LAST_READ**: Only sequence numbers are recorded in databases. |
| | | | |
| | | | Enumeration values: |
| | | | |
@ -54,13 +50,13 @@ Request Parameters
.. table:: **Table 3** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

28
api-ref/source/api_description/checkpoint_management/submitting_checkpoints.rst
View File

@ -15,7 +15,7 @@ URI
POST /v2/{project_id}/checkpoints
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ===========
Parameter Mandatory Type Description
@ -28,13 +28,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
.. table:: **Table 3** Request body parameters
@ -43,9 +43,9 @@ Request Parameters
+=================+=================+=================+============================================================================================================================================================================+
| app_name | Yes | String | Name of the app, which is the unique identifier of a user data consumption program. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checkpoint_type | Yes | String | Type of the checkpoint. |
| checkpoint_type | Yes | String | Type of the checkpoint |
| | | | |
| | | | - LAST_READ: Only sequence numbers are recorded in databases. |
| | | | - **LAST_READ**: Only sequence numbers are recorded in databases. |
| | | | |
| | | | Enumeration values: |
| | | | |
@ -53,17 +53,13 @@ Request Parameters
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| stream_name | Yes | String | Name of the stream. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| partition_id | Yes | String | Partition identifier of the stream. The value can be in either of the following formats: |
| | | | |
| | | | - shardId-0000000000 |
| | | | |
| | | | - 0 |
| partition_id | Yes | String | Partition ID of the stream The value can be in either of the following formats:- shardId-0000000000- 0 |
| | | | |
| | | | For example, if a stream has three partitions, the partition identifiers are 0, 1, and 2, or shardId-0000000000, shardId-0000000001, and shardId-0000000002, respectively. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| sequence_number | Yes | String | Sequence number to be submitted, which is used to record the consumption checkpoint of the stream. Ensure that the sequence number is within the valid range. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| metadata | No | String | Metadata information of the consumer application. |
| metadata | No | String | Metadata information of the consumer application |
| | | | |
| | | | The metadata information can contain a maximum of 1,000 characters. |
| | | | |

70
api-ref/source/api_description/data_management/downloading_data.rst
View File

File diff suppressed because it is too large Load Diff

126
api-ref/source/api_description/data_management/obtaining_data_cursors.rst
View File

File diff suppressed because it is too large Load Diff

86
api-ref/source/api_description/data_management/uploading_data.rst
View File

File diff suppressed because it is too large Load Diff

447
api-ref/source/api_description/dump_task_management/adding_cloudtable_dump_tasks.rst
View File

File diff suppressed because it is too large Load Diff

163
api-ref/source/api_description/dump_task_management/adding_dli_dump_tasks.rst
View File

File diff suppressed because it is too large Load Diff

282
api-ref/source/api_description/dump_task_management/adding_dws_dump_tasks.rst
View File

File diff suppressed because it is too large Load Diff

197
api-ref/source/api_description/dump_task_management/adding_mrs_dump_tasks.rst
View File

File diff suppressed because it is too large Load Diff

16
api-ref/source/api_description/dump_task_management/adding_obs_dump_tasks.rst
View File

@ -48,10 +48,6 @@ Request Parameters
| destination_type | Yes | String | Dump destination. Possible values: |
| | | | |
| | | | - OBS: Data is dumped to OBS. |
| | | | - MRS: Data is dumped to MRS. |
| | | | - DLI: Data is dumped to DLI. |
| | | | - CLOUDTABLE: Data is dumped to CloudTable. |
| | | | - DWS: Data is dumped to DWS. |
| | | | |
| | | | Default: **NOWHERE** |
| | | | |
@ -152,20 +148,12 @@ Request Parameters
| destination_file_type | No | String | Dump file format. Possible values: |
| | | | |
| | | | - Text (default) |
| | | | - Parquet |
| | | | - CarbonData |
| | | | |
| | | | Note: |
| | | | |
| | | | You can select Parquet or CarbonData only when Source Data Type is set to JSON and Dump Destination is set to OBS. |
| | | | |
| | | | Default: **text** |
| | | | |
| | | | Enumeration values: |
| | | | |
| | | | - **text** |
| | | | - **parquet** |
| | | | - **carbon** |
+-----------------------+-----------------+------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| processing_schema | No | :ref:`ProcessingSchema <dis_02_0410__request_processingschema>` object | Dump time directory generated based on the timestamp of the source data and the configured partition_format. Directory structure of the object file written into OBS. The directory structure is in the format of yyyy/MM/dd/HH/mm. |
+-----------------------+-----------------+------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@ -245,7 +233,7 @@ Example Requests
}
}
- Adding OBS Dump Tasks (The dump file format is Parquet.)
- Adding OBS Dump Tasks (The dump file format is Text.)
.. code-block:: text
@ -257,7 +245,7 @@ Example Requests
"task_name" : "newtask",
"consumer_strategy" : "LATEST",
"agency_name" : "dis_admin_agency",
"destination_file_type" : "parquet",
"destination_file_type" : "text",
"obs_bucket_path" : "obsbucket",
"file_prefix" : "",
"partition_format" : "yyyy/MM/dd/HH/mm",

16
api-ref/source/api_description/dump_task_management/deleting_dump_tasks.rst
View File

@ -15,7 +15,7 @@ URI
DELETE /v2/{project_id}/streams/{stream_name}/transfer-tasks/{task_name}
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
=========== ========= ====== ====================================
Parameter Mandatory Type Description
@ -30,13 +30,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

12
api-ref/source/api_description/dump_task_management/index.rst
View File

@ -6,27 +6,19 @@ Dump Task Management
====================
- :ref:`Adding OBS Dump Tasks <dis_02_0410>`
- :ref:`Querying Dump Tasks <listtransfertasks>`
- :ref:`Querying Dump Task <listtransfertasks>`
- :ref:`Deleting Dump Tasks <deletetransfertask>`
- :ref:`Querying Dump Task Details <showtransfertask>`
- :ref:`Starting Dump Tasks in Batches <batchstarttransfertask>`
- :ref:`Pausing Dump Tasks in Batches <batchstoptransfertask>`
- :ref:`Adding DWS Dump Tasks <createdwstransfertask>`
- :ref:`Adding MRS Dump Tasks <createmrstransfertask>`
- :ref:`Adding DLI Dump Tasks <createdlitransfertask>`
- :ref:`Adding CloudTable Dump Tasks <createcloudtabletransfertask>`
.. toctree::
:maxdepth: 1
:hidden:
adding_obs_dump_tasks
querying_dump_tasks
querying_dump_task
deleting_dump_tasks
querying_dump_task_details
starting_dump_tasks_in_batches
pausing_dump_tasks_in_batches
adding_dws_dump_tasks
adding_mrs_dump_tasks
adding_dli_dump_tasks
adding_cloudtable_dump_tasks

26
api-ref/source/api_description/dump_task_management/pausing_dump_tasks_in_batches.rst
View File

@ -15,7 +15,7 @@ URI
POST /v2/{project_id}/streams/{stream_name}/transfer-tasks/action
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
+-----------------+-----------------+-----------------+-----------------------------------+
| Parameter | Mandatory | Type | Description |
@ -32,19 +32,17 @@ Request Parameters
.. table:: **Table 2** Request body parameters
+-----------------+-----------------+----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+==============================================================================================+============================================================================+
| action | Yes | String | Dump task operation. Currently, only the following operation is supported: |
| | | | |
| | | | - stop: The dump task is stopped. |
| | | | |
| | | | Enumeration values: |
| | | | |
| | | | - **stop** |
+-----------------+-----------------+----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
| tasks | Yes | Array of :ref:`BatchTransferTask <batchstoptransfertask__request_batchtransfertask>` objects | List of dump tasks to be paused. |
+-----------------+-----------------+----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
+-----------------+-----------------+----------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+==============================================================================================+============================================================================================+
| action | Yes | String | Dump task operation. The value can only be **stop**, which indicates stopping a dump task. |
| | | | |
| | | | Enumeration values: |
| | | | |
| | | | - **stop** |
+-----------------+-----------------+----------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
| tasks | Yes | Array of :ref:`BatchTransferTask <batchstoptransfertask__request_batchtransfertask>` objects | List of dump tasks to be paused. |
+-----------------+-----------------+----------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
.. _batchstoptransfertask__request_batchtransfertask:

30
api-ref/source/api_description/dump_task_management/querying_dump_tasks.rst → api-ref/source/api_description/dump_task_management/querying_dump_task.rst
View File

@ -2,8 +2,8 @@
.. _ListTransferTasks:
Querying Dump Tasks
===================
Querying Dump Task
==================
Function
--------
@ -69,54 +69,28 @@ Response Parameters
| state | String | Dump task status. Possible values: |
| | | |
| | | - ERROR: An error occurs. |
| | | |
| | | - STARTING: The dump task is being started. |
| | | |
| | | - PAUSED: The dump task has been stopped. |
| | | |
| | | - RUNNING: The dump task is running. |
| | | |
| | | - DELETE: The dump task has been deleted. |
| | | |
| | | - ABNORMAL: The dump task is abnormal. |
| | | |
| | | Enumeration values: |
| | | |
| | | - **ERROR** |
| | | |
| | | - **STARTING** |
| | | |
| | | - **PAUSED** |
| | | |
| | | - **RUNNING** |
| | | |
| | | - **DELETE** |
| | | |
| | | - **ABNORMAL** |
+-------------------------+-----------------------+----------------------------------------------+
| destination_type | String | Dump destination. Possible values: |
| | | |
| | | - OBS: Data is dumped to OBS. |
| | | |
| | | - MRS: Data is dumped to MRS. |
| | | |
| | | - DLI: Data is dumped to DLI. |
| | | |
| | | - CLOUDTABLE: Data is dumped to CloudTable. |
| | | |
| | | - DWS: Data is dumped to DWS. |
| | | |
| | | Enumeration values: |
| | | |
| | | - **OBS** |
| | | |
| | | - **MRS** |
| | | |
| | | - **DLI** |
| | | |
| | | - **CLOUDTABLE** |
| | | |
| | | - **DWS** |
+-------------------------+-----------------------+----------------------------------------------+
| create_time | Long | Time when the dump task is created. |
+-------------------------+-----------------------+----------------------------------------------+

696
api-ref/source/api_description/dump_task_management/querying_dump_task_details.rst
View File

File diff suppressed because it is too large Load Diff

26
api-ref/source/api_description/dump_task_management/starting_dump_tasks_in_batches.rst
View File

@ -15,7 +15,7 @@ URI
POST /v2/{project_id}/streams/{stream_name}/transfer-tasks/action
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
+-----------------+-----------------+-----------------+-----------------------------------+
| Parameter | Mandatory | Type | Description |
@ -32,19 +32,17 @@ Request Parameters
.. table:: **Table 2** Request body parameters
+-----------------+-----------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+===============================================================================================+============================================================================+
| action | Yes | String | Dump task operation. Currently, only the following operation is supported: |
| | | | |
| | | | - start: The dump task is started. |
| | | | |
| | | | Enumeration values: |
| | | | |
| | | | - **start** |
+-----------------+-----------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
| tasks | Yes | Array of :ref:`BatchTransferTask <batchstarttransfertask__request_batchtransfertask>` objects | List of dump tasks to be operated. |
+-----------------+-----------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
+-----------------+-----------------+-----------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+===============================================================================================+=============================================================================================+
| action | Yes | String | Dump task operation. The value can only be **start**, which indicates starting a dump task. |
| | | | |
| | | | Enumeration values: |
| | | | |
| | | | - **start** |
+-----------------+-----------------+-----------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------+
| tasks | Yes | Array of :ref:`BatchTransferTask <batchstarttransfertask__request_batchtransfertask>` objects | List of dump tasks to be operated. |
+-----------------+-----------------+-----------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------+
.. _batchstarttransfertask__request_batchtransfertask:

54
api-ref/source/api_description/monitoring_management/querying_partition_monitoring_data.rst
View File

@ -15,40 +15,34 @@ URI
GET /v2/{project_id}/streams/{stream_name}/partitions/{partition_id}/metrics
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+============================================================================================================================================================================+
| project_id | Yes | String | Project ID. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| stream_name | Yes | String | Name of the stream. |
| | | | |
| | | | Maximum: **60** |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| partition_id | Yes | String | Partition No. The value can be in either of the following formats: |
| | | | |
| | | | - shardId-0000000000 |
| | | | |
| | | | - 0 |
| | | | |
| | | | For example, if a stream has three partitions, the partition identifiers are 0, 1, and 2, or shardId-0000000000, shardId-0000000001, and shardId-0000000002, respectively. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+======================================================================================================================================================================================================================================================================+
| project_id | Yes | String | Project ID. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| stream_name | Yes | String | Name of the stream. |
| | | | |
| | | | Maximum: **60** |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| partition_id | Yes | String | Partition ID The value can be in either of the following formats: - shardId-0000000000- 0 For example, if a stream has three partitions, the partition identifiers are 0, 1, and 2, or shardId-0000000000, shardId-0000000001, and shardId-0000000002, respectively. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
.. table:: **Table 2** Query parameters
.. table:: **Table 2** Query Parameters
+-----------------+-----------------+-----------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+======================================================================================================================================================================================+
| label | No | String | Partition monitoring metric. (Either label or label_list must be specified. If both label_list and label are specified, label_list prevails.) |
| label | No | String | Partition monitoring metric. (Either **label** or **label_list** must be specified. If they are both specified, **label_list** prevails.) |
| | | | |
| | | | - total_put_bytes_per_partition: total input traffic of the partition (byte) |
| | | | - **total_put_bytes_per_partition**: total input traffic in a partition (byte) |
| | | | |
| | | | - total_get_bytes_per_partition: total output traffic of the partition (byte) |
| | | | - **total_get_bytes_per_partition**: total output traffic in a partition (byte) |
| | | | |
| | | | - total_put_records_per_partition: total number of input records of the partition |
| | | | - **total_put_records_per_partition**: total number of input records in a partition |
| | | | |
| | | | - total_get_records_per_partition: total number of output records of the partition |
| | | | - **total_get_records_per_partition**: total number of output records in a partition |
| | | | |
| | | | Enumeration values: |
| | | | |
@ -72,13 +66,11 @@ Request Parameters
.. table:: **Table 3** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+--------------+-----------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+==============+===========+========+=====================================================================================================================================================================+
| X-Auth-Token | Yes | String | User token The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+--------------+-----------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

40
api-ref/source/api_description/monitoring_management/querying_stream_monitoring_data.rst
View File

@ -15,7 +15,7 @@ URI
GET /v2/{project_id}/streams/{stream_name}/metrics
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
+-----------------+-----------------+-----------------+---------------------+
| Parameter | Mandatory | Type | Description |
@ -27,32 +27,32 @@ GET /v2/{project_id}/streams/{stream_name}/metrics
| | | | Maximum: **60** |
+-----------------+-----------------+-----------------+---------------------+
.. table:: **Table 2** Query parameters
.. table:: **Table 2** Query Parameters
+-----------------+-----------------+-----------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+================================================================================================================================================================================+
| label | No | String | Stream monitoring metric. (Either label or label_list must be specified. If both label_list and label are specified, label_list prevails.) |
| label | No | String | Stream monitoring metric (Either **label** or **label_list** must be specified. If they are both specified, **label_list** prevails.) |
| | | | |
| | | | - total_put_bytes_per_stream: total input traffic (byte) |
| | | | - **total_put_bytes_per_stream**: total input traffic (byte) |
| | | | |
| | | | - total_get_bytes_per_stream: total output traffic (byte) |
| | | | - **total_get_bytes_per_stream**: total output traffic (byte) |
| | | | |
| | | | - total_put_records_per_stream: total number of input records |
| | | | - **total_put_records_per_stream**: total number of input records |
| | | | |
| | | | - total_get_records_per_stream: total number of output records |
| | | | - **total_get_records_per_stream**: total number of output records |
| | | | |
| | | | - total_put_req_latency: average processing time of upload requests (millisecond) |
| | | | - **total_put_req_latency**: average processing time of upload requests (millisecond) |
| | | | |
| | | | - total_get_req_latency: average processing time of download requests (millisecond) |
| | | | - **total_get_req_latency**: average processing time of download requests (millisecond) |
| | | | |
| | | | - total_put_req_suc_per_stream: number of successful upload requests |
| | | | - **total_put_req_suc_per_stream**: number of successful upload requests |
| | | | |
| | | | - total_get_req_suc_per_stream: number of successful download requests |
| | | | - **total_get_req_suc_per_stream**: number of successful download requests |
| | | | |
| | | | - traffic_control_put: number of rejected upload requests due to flow control |
| | | | - **traffic_control_put**: number of rejected upload requests due to flow control |
| | | | |
| | | | - traffic_control_get: number of rejected download requests due to flow control |
| | | | - **traffic_control_get**: number of rejected download requests due to flow control |
| | | | |
| | | | Enumeration values: |
| | | | |
@ -88,13 +88,13 @@ Request Parameters
.. table:: **Table 3** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

128
api-ref/source/api_description/stream_management/adding_permission_policies.rst
View File

File diff suppressed because it is too large Load Diff

52
api-ref/source/api_description/stream_management/changing_partition_quantity.rst
View File

@ -15,7 +15,7 @@ URI
PUT /v2/{project_id}/streams/{stream_name}
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
+-----------------+-----------------+-----------------+------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
@ -32,33 +32,35 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
.. table:: **Table 3** Request body parameters
+------------------------+-----------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+========================+=================+=================+==========================================================================================================================================================================================================================================+
| stream_name | Yes | String | Name of the stream whose partition quantity needs to be changed. |
| | | | |
| | | | Maximum: **64** |
+------------------------+-----------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| target_partition_count | Yes | Integer | Number of the target partitions. |
| | | | |
| | | | The value is an integer greater than 0. |
| | | | |
| | | | If the value is greater than the number of current partitions, scaling-up is required. If the value is less than the number of current partitions, scale-down is required. |
| | | | |
| | | | Note: A maximum of five scale-up/down operations can be performed for each stream within one hour. If a scale-up/down operation is successfully performed, you cannot perform one more scale-up/down operation within the next one hour. |
| | | | |
| | | | Minimum: **0** |
+------------------------+-----------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+------------------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+========================+=================+=================+============================================================================================================================================================================+
| stream_name | Yes | String | Name of the stream whose partition quantity needs to be changed. |
| | | | |
| | | | Maximum: **64** |
+------------------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| target_partition_count | Yes | Integer | Number of the target partitions |
| | | | |
| | | | The value is an integer greater than 0. |
| | | | |
| | | | If the value is greater than the number of current partitions, scaling-up is required. If the value is less than the number of current partitions, scale-down is required. |
| | | | |
| | | | Note: |
| | | | |
| | | | Each stream can be scaled up and down for five times within one hour. After a stream is scaled up or down, it cannot be scaled up or down again in the next one hour. |
| | | | |
| | | | Minimum: **0** |
+------------------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

261
api-ref/source/api_description/stream_management/creating_streams.rst
View File

File diff suppressed because it is too large Load Diff

16
api-ref/source/api_description/stream_management/deleting_specified_streams.rst
View File

@ -15,7 +15,7 @@ URI
DELETE /v2/{project_id}/streams/{stream_name}
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
+-----------------+-----------------+-----------------+-----------------------------------+
| Parameter | Mandatory | Type | Description |
@ -32,13 +32,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

6
api-ref/source/api_description/stream_management/index.rst
View File

@ -10,9 +10,6 @@ Stream Management
- :ref:`Deleting Specified Streams <deletestream>`
- :ref:`Querying Stream Details <dis_02_0025>`
- :ref:`Changing Partition Quantity <updatepartitioncount>`
- :ref:`Updating Stream Information <updatestream>`
- :ref:`Adding Permission Policies <createpolicies>`
- :ref:`Querying Permission Policies <listpolicies>`
.. toctree::
:maxdepth: 1
@ -23,6 +20,3 @@ Stream Management
deleting_specified_streams
querying_stream_details
changing_partition_quantity
updating_stream_information
adding_permission_policies
querying_permission_policies

132
api-ref/source/api_description/stream_management/querying_permission_policies.rst
View File

File diff suppressed because it is too large Load Diff

328
api-ref/source/api_description/stream_management/querying_stream_details.rst
View File

File diff suppressed because it is too large Load Diff

295
api-ref/source/api_description/stream_management/querying_streams.rst
View File

File diff suppressed because it is too large Load Diff

134
api-ref/source/api_description/stream_management/updating_stream_information.rst
View File

File diff suppressed because it is too large Load Diff

94
api-ref/source/api_description/tag_management/adding_resource_tags_in_batches.rst
View File

File diff suppressed because it is too large Load Diff

70
api-ref/source/api_description/tag_management/adding_tags_for_specified_streams.rst
View File

@ -15,7 +15,7 @@ URI
POST /v2/{project_id}/stream/{stream_id}/tags
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
+-----------------+-----------------+-----------------+-----------------+
| Parameter | Mandatory | Type | Description |
@ -32,13 +32,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
.. table:: **Table 3** Request body parameters
@ -52,33 +52,33 @@ Request Parameters
.. table:: **Table 4** Tag
+-----------------+-----------------+-----------------+--------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+============================================================================================+
| key | No | String | Key. |
| | | | |
| | | | - This field cannot be left blank. |
| | | | |
| | | | - The key value of a resource must be unique. |
| | | | |
| | | | - Character set: A-Z, a-z, 0-9, '-', '_', and Unicode characters (\\u4E00-\\u9FFF). |
| | | | |
| | | | Minimum: **1** |
| | | | |
| | | | Maximum: **36** |
+-----------------+-----------------+-----------------+--------------------------------------------------------------------------------------------+
| value | No | String | Value. |
| | | | |
| | | | - The value contains a maximum of 43 characters. |
| | | | |
| | | | - Character set: A-Z, a-z, 0-9, '. ', '-', '_', and Unicode characters (\\u4E00-\\u9FFF). |
| | | | |
| | | | - The value can contain only digits, letters, hyphens (-), and underscores (_). |
| | | | |
| | | | Minimum: **0** |
| | | | |
| | | | Maximum: **43** |
+-----------------+-----------------+-----------------+--------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=================================================================================================================================================+
| key | No | String | Key |
| | | | |
| | | | - It cannot be left blank. |
| | | | |
| | | | - It must be unique for each resource. |
| | | | |
| | | | - It can contain uppercase and lowercase letters, digits, hyphens (-), underscores (_), and Unicode characters (\\u4E00-\\u9FFF). |
| | | | |
| | | | Minimum: **1** |
| | | | |
| | | | Maximum: **36** |
+-----------------+-----------------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| value | No | String | Value |
| | | | |
| | | | - It can contain a maximum of 43 characters. |
| | | | |
| | | | - It can contain uppercase and lowercase letters, digits, periods (.), hyphens (-), underscores (_), and Unicode characters (\\u4E00-\\u9FFF). |
| | | | |
| | | | - It can only contain digits, letters, hyphens (-), and underscores (_). |
| | | | |
| | | | Minimum: **0** |
| | | | |
| | | | Maximum: **43** |
+-----------------+-----------------+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

92
api-ref/source/api_description/tag_management/deleting_resource_tags_in_batches.rst
View File

File diff suppressed because it is too large Load Diff

16
api-ref/source/api_description/tag_management/deleting_tags_of_specified_streams.rst
View File

@ -15,7 +15,7 @@ URI
DELETE /v2/{project_id}/stream/{stream_id}/tags/{key}
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ===========
Parameter Mandatory Type Description
@ -30,13 +30,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------

50
api-ref/source/api_description/tag_management/querying_tags_of_specified_regions.rst
View File

@ -15,7 +15,7 @@ URI
GET /v2/{project_id}/stream/tags
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ===========
Parameter Mandatory Type Description
@ -28,13 +28,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------
@ -53,23 +53,23 @@ Response Parameters
.. table:: **Table 4** Tags
+-----------------------+-----------------------+--------------------------------------------------------------------------------------+
| Parameter | Type | Description |
+=======================+=======================+======================================================================================+
| key | String | Key. |
| | | |
| | | - This field cannot be left blank. |
| | | |
| | | - The key value of a resource must be unique. |
| | | |
| | | - Character set: A-Z, a-z, 0-9, '-', '_', and Unicode characters (\\u4E00-\\u9FFF). |
| | | |
| | | Maximum: **36** |
+-----------------------+-----------------------+--------------------------------------------------------------------------------------+
| values | Array of strings | Tag value list. |
| | | |
| | | If values are null, it indicates any_value. The relationship between values is OR. |
+-----------------------+-----------------------+--------------------------------------------------------------------------------------+
+-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Type | Description |
+=======================+=======================+====================================================================================================================================+
| key | String | Key |
| | | |
| | | - It cannot be left blank. |
| | | |
| | | - It must be unique for each resource. |
| | | |
| | | - It can contain uppercase and lowercase letters, digits, hyphens (-), underscores (_), and Unicode characters (\\u4E00-\\u9FFF). |
| | | |
| | | Maximum: **36** |
+-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+
| values | Array of strings | List of tag values |
| | | |
| | | If **values** is an empty list, it indicates **any_value**. The values are in the OR relationship. |
+-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+
Example Requests
----------------

70
api-ref/source/api_description/tag_management/querying_tags_of_specified_streams.rst
View File

@ -15,7 +15,7 @@ URI
GET /v2/{project_id}/stream/{stream_id}/tags
.. table:: **Table 1** Path parameters
.. table:: **Table 1** Path Parameters
========== ========= ====== ===========
Parameter Mandatory Type Description
@ -29,13 +29,13 @@ Request Parameters
.. table:: **Table 2** Request header parameters
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+=====================================================================================================+
| X-Auth-Token | Yes | String | User token. |
| | | | |
| | | | The token can be obtained by calling the IAM API (value of X-Subject-Token in the response header). |
+-----------------+-----------------+-----------------+-----------------------------------------------------------------------------------------------------+
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Mandatory | Type | Description |
+=================+=================+=================+==========================================================================================================================================================+
| X-Auth-Token | Yes | String | User token |
| | | | |
| | | | The token can be obtained by calling the IAM API used to obtain a user token. The value of **X-Subject-Token** in the response header is the user token. |
+-----------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
Response Parameters
-------------------
@ -54,33 +54,33 @@ Response Parameters
.. table:: **Table 4** Tag
+-----------------------+-----------------------+--------------------------------------------------------------------------------------------+
| Parameter | Type | Description |
+=======================+=======================+============================================================================================+
| key | String | Key. |
| | | |
| | | - This field cannot be left blank. |
| | | |
| | | - The key value of a resource must be unique. |
| | | |
| | | - Character set: A-Z, a-z, 0-9, '-', '_', and Unicode characters (\\u4E00-\\u9FFF). |
| | | |
| | | Minimum: **1** |
| | | |
| | | Maximum: **36** |
+-----------------------+-----------------------+--------------------------------------------------------------------------------------------+
| value | String | Value. |
| | | |
| | | - The value contains a maximum of 43 characters. |
| | | |
| | | - Character set: A-Z, a-z, 0-9, '. ', '-', '_', and Unicode characters (\\u4E00-\\u9FFF). |
| | | |
| | | - The value can contain only digits, letters, hyphens (-), and underscores (_). |
| | | |
| | | Minimum: **0** |
| | | |
| | | Maximum: **43** |
+-----------------------+-----------------------+--------------------------------------------------------------------------------------------+
+-----------------------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | Type | Description |
+=======================+=======================+=================================================================================================================================================+
| key | String | Key |
| | | |
| | | - It cannot be left blank. |
| | | |
| | | - It must be unique for each resource. |
| | | |
| | | - It can contain uppercase and lowercase letters, digits, hyphens (-), underscores (_), and Unicode characters (\\u4E00-\\u9FFF). |
| | | |
| | | Minimum: **1** |
| | | |
| | | Maximum: **36** |
+-----------------------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| value | String | Value |
| | | |
| | | - It can contain a maximum of 43 characters. |
| | | |
| | | - It can contain uppercase and lowercase letters, digits, periods (.), hyphens (-), underscores (_), and Unicode characters (\\u4E00-\\u9FFF). |
| | | |
| | | - It can only contain digits, letters, hyphens (-), and underscores (_). |
| | | |
| | | Minimum: **0** |
| | | |
| | | Maximum: **43** |
+-----------------------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
Example Requests
----------------

178
api-ref/source/api_description/tag_management/using_tags_to_filter_resources_streams.rst
View File

File diff suppressed because it is too large Load Diff

20
api-ref/source/appendix/error_codes.rst
View File

@ -72,26 +72,8 @@ Error Codes
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4336 | Invalid HDFS path. | The MRS HDFS path entered during MRS dump task creation is invalid. | Ensure that the MRS HDFS path exists. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4337 | The DLI database does not exist. | The DLI database entered during DLI dump task creation does not exist. | Ensure that the DLI database exists. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4338 | The DLI table does not exist. | The DLI table entered during DLI dump task creation does not exist. | Ensure that the DLI table exists and is an internal table. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4339 | Consumer quota exceeded. | The consumer quota of the consumer group is insufficient. | Allocate consumers properly or create a consumer group to meet the requirement. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4341 | The CloudTable cluster does not exist. | The CloudTable cluster entered during CloudTable dump task creation does not exist. | Ensure that the CloudTable cluster exists and is running properly. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4342 | The CloudTable table does not exist | The CloudTable table entered during CloudTable dump task creation does not exist. | Ensure that the CloudTable table exists. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4343 | The CloudTable table family does not exist. | The CloudTable column family entered during CloudTable dump task creation does not exist. | Ensure that the CloudTable column family exists. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4345 | Invalid CloudTable schema. | The schema entered during CloudTable dump task creation is invalid. | Check the schema based on the returned details to ensure that the configured JSON attribute name exists and the parameters are valid. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4348 | Invalid CloudTable openTSDB schema. | The schema entered during CloudTable OpenTSDB dump task creation is invalid. | Check the schema based on the returned details to ensure that the configured JSON attribute name exists and the parameters are valid. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4350 | Invalid DWS cluster. | The DWS cluster entered during DWS dump task creation does not exist. | Ensure that the DWS cluster exists and is running properly. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4351 | Invalid KMS userKey. | The KMS key entered during DWS dump task creation is invalid. | Ensure that the KMS key exists. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4354 | The transfer task does not exist. | The dump task to be deleted or updated does not exist. | Ensure that the dump task exists. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4355 | The transfer task already exists. | A dump task with the same name already exists. | Enter a new dump task name. |
@ -112,7 +94,7 @@ Error Codes
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 400 | DIS.4605 | The action is not supported. | The current tag operation is not supported. | Ensure that the current tag operation is valid. Currently, only the create and delete operations are supported. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 403 | DIS.4116 | Invalid RBAC. %s | User operations are restricted. | Ensure that the account has passed real-name authentication, is not in arrears, or has permissions to operate DIS. |
| 403 | DIS.4116 | Invalid RBAC. %s | User operations are restricted. | Ensure that the account is not in arrears, or has permissions to operate DIS. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| 500 | DIS.5000 | System error. | System error. | Contact customer service or technical support to handle system errors. |
+-------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+

6
api-ref/source/appendix/obtaining_a_project_id.rst
View File

@ -8,9 +8,9 @@ Obtaining a Project ID
Obtaining a Project ID by Calling an API
----------------------------------------
You can obtain a project ID by calling the API for `Querying Project Information Based on the Specified Criteria <https://docs.otc.t-systems.com/api/iam/en-us_topic_0057845625.html>`__.
You can obtain the project ID by calling the API `used to query project information <https://docs.otc.t-systems.com/api/iam/en-us_topic_0057845625.html>`__.
The API for obtaining the project ID is **GET https://{Endpoint}/v3/projects/**, where *Endpoint* indicates the endpoint of IAM. You can obtain the endpoint from `Regions and Endpoints <https://docs.otc.t-systems.com/en-us/endpoint/index.html>`__. For details about API authentication, see :ref:`Authentication <dis_02_0517>`.
The API used to obtain a project ID is GET **https://{Endpoint}/v3/projects**. **{Endpoint}** is the IAM endpoint and can be obtained from `Regions and Endpoints <https://docs.otc.t-systems.com/en-us/endpoint/index.html>`__. For details about API authentication, see :ref:`Authentication <dis_02_0517>`.
The following is an example response. The value of **id** is the project ID. If multiple IDs are returned, obtain the desired project ID based on the actual region (name).
@ -52,7 +52,7 @@ A project ID is required for some URLs when an API is called. To obtain a projec
On the **My Credentials** page, view project IDs in the project list.
.. figure:: /_static/images/en-us_image_0000001238312593.jpg
.. figure:: /_static/images/en-us_image_0191919629.jpg
:alt: **Figure 1** Viewing project IDs
**Figure 1** Viewing project IDs

13
api-ref/source/application_example.rst
View File

@ -10,7 +10,7 @@ Scenarios
DIS provides efficient collection, transmission, and distribution capabilities for real-time data and provides a variety of APIs to help you quickly build real-time data applications.
The following describes how to create a DIS stream by calling the :ref:`Before You Start <dis_02_0016_01>` API. For details, see :ref:`Calling APIs <dis_02_0516>`.
The following describes how to create a DIS stream by calling the :ref:`Before You Start <dis_02_0511>` API. For details, see :ref:`Calling APIs <dis_02_0516>`.
.. note::
@ -29,7 +29,7 @@ Prerequisites
You have planned the region where DIS is located and determined the endpoint for calling an API based on the region.
An endpoint is the **request address** for calling an API. Endpoints vary depending on services and regions. You can obtain endpoints from `Regions and Endpoints <https://docs.otc.t-systems.com/en-us/endpoint/index.html>`__.
An endpoint is the **request address** for calling an API. Endpoints vary depending on services and regions. You can obtain endpoints of the service from `Regions and Endpoints <https://docs.otc.t-systems.com/en-us/endpoint/index.html>`__.
Creating a Stream
-----------------
@ -120,16 +120,9 @@ You can also configure a schema for the stream. When using DIS to dump data to o
"auto_scale_enabled": true,
"auto_scale_min_partition_count": 1,
"auto_scale_max_partition_count": 10
"data_type": "JSON",
"data_schema":
"{\"type\":\"record\",\"name\":\"RecordName\",\"fields\":[{\"name\":\"key1\",\"type\":\"string\"},{\"name\":\"key2\",\"type\":\"string\"}]}"
"data_type": "BLOG",
}
In this example, a stream whose source data type is JSON and that contains the key1 and key2 attributes is created.
- **data_type** indicates the type of source data. The value **JSON** indicates that the data format in the partition is JSON.
- **data_schema** indicates the source data schema, which describes the source data structures in JSON and CSV formats using the Avro Schema syntax.
If the request is successful, 201 Created is returned.
If the request fails, an error code and error information are returned. For details, see :ref:`Error Codes <errorcode>`.

7
api-ref/source/calling_apis/authentication.rst
View File

@ -84,7 +84,12 @@ To obtain an access key, perform the following steps:
#. Click **OK** and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the **credentials.csv** file to view **Access Key Id** and **Secret Access Key**.
In AK/SK-based authentication, you can use an AK/SK to sign requests based on the signature algorithm or use the signing SDK to sign requests.
.. note::
- Only two access keys can be added for each user.
- To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.
In AK/SK-based authentication, you can use an AK/SK to sign requests based on the signature algorithm or use the signing SDK to sign requests. For details about how to sign requests and use the signature SDK, see API Request Signing Guide.
.. important::

48
api-ref/source/change_history.rst
View File

@ -5,18 +5,36 @@
Change History
==============
+-----------------------------------+---------------------------------------------------------------+
| Released On | Description |
+===================================+===============================================================+
| 2018-10-25 | Accepted in OTC 3.2. |
+-----------------------------------+---------------------------------------------------------------+
| 2018-09-29 | Modified the following contents: |
| | |
| | Added tag management APIs |
+-----------------------------------+---------------------------------------------------------------+
| 2018-09-26 | Modified the following contents: |
| | |
| | A status code was added in :ref:`Status Codes <dis_02_0022>`. |
+-----------------------------------+---------------------------------------------------------------+
| 2018-03-16 | This issue is the first official release. |
+-----------------------------------+---------------------------------------------------------------+
+-----------------------------------+-------------------------------------------------------------------------------------+
| Released On | Description |
+===================================+=====================================================================================+
| 2023-04-03 | Modified the following sections: |
| | |
| | :ref:`Creating Streams <dis_02_0016_01>` |
+-----------------------------------+-------------------------------------------------------------------------------------+
| 2022-09-30 | Modified the following sections: |
| | |
| | - :ref:`Application Example <dis_02_0003>` |
| | - :ref:`Stream Management <topic_300000000>` |
| | - :ref:`App Management <topic_300000001>` |
| | - :ref:`Checkpoint Management <topic_300000002>` |
| | - :ref:`Permissions Policies and Supported Actions <en-us_topic_0000001079240698>` |
| | |
| | Offline the following sections: |
| | |
| | - Updating Stream Information |
| | - Adding Permission Policies |
| | - Querying Permission Policies |
+-----------------------------------+-------------------------------------------------------------------------------------+
| 2018-10-25 | Accepted in OTC 3.2. |
+-----------------------------------+-------------------------------------------------------------------------------------+
| 2018-09-29 | Modified the following contents: |
| | |
| | Added tag management APIs |
+-----------------------------------+-------------------------------------------------------------------------------------+
| 2018-09-26 | Modified the following contents: |
| | |
| | A status code was added in :ref:`Status Codes <dis_02_0022>`. |
+-----------------------------------+-------------------------------------------------------------------------------------+
| 2018-03-16 | This issue is the first official release. |
+-----------------------------------+-------------------------------------------------------------------------------------+

188
api-ref/source/permissions_policies_and_supported_actions.rst
View File

File diff suppressed because it is too large Load Diff