mikacur/data-ingestion-service_aligns_with_the_OTC_platform_07_23
1
0
Fork 0
forked from docs/data-ingestion-service
Code Pull Requests Activity

Update content

Browse Source
This commit is contained in:
OpenTelekomCloud Proposal Bot 2022-10-27 13:50:01 +00:00 committed by Artem Goncharov
parent dfaf57a24a
commit a4d6cc9f6b
17 changed files with 51 additions and 70 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
umn/source/dis_sdk_usage_guide/configuring_sample_projects.rst
View File

@ -5,7 +5,7 @@
Configuring Sample Projects
===========================
The **dis-sdk-**\ *1.2.3*\ **.zip** package of the latest version in :ref:`Downloading SDKs <dis_06_0005>` provides a sample project. You can use a development tool (such as Eclipse) on the local server to compile and run the sample project. You can also develop your applications based on the sample project. The sample project code is available in the **\\dis-sdk-demo\src\com\bigdata\dis\sdk\demo** directory.
The **dis-sdk-**\ *1.2.3*\ **.zip** package of the latest version in :ref:`Downloading SDKs <dis_06_0005>` provides a sample project. You can use a development tool (such as Eclipse) on the local server to compile and run the sample project. You can also develop your applications based on the sample project. The sample project code is available in the **\\dis-sdk-demo\\src\\com\\bigdata\\dis\\sdk\\demo** directory.
================= ====================
Sample Code Description
@ -31,7 +31,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001222097302.png
:alt: **Figure 1** Import Projects
**Figure 1** Import Projects
d. Click **Finish** to import the project.
@ -49,7 +48,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001222417262.png
:alt: **Figure 2** Resource
**Figure 2** Resource
b. Import a JAR dependency package.
@ -64,7 +62,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001266977433.png
:alt: **Figure 3** Java Build Path
**Figure 3** Java Build Path
c. Add the JDK.

2
umn/source/dis_sdk_usage_guide/downloading_streaming_data.rst
View File

@ -8,7 +8,7 @@ Downloading Streaming Data
Sample Code
-----------
Use the initialized client instance to obtain data through the DIS stream. The sample code is available in the **ConsumerDemo.java** file under the **dis-sdk-demo\src\main\java\com\bigdata\dis\sdk\demo** directory.
Use the initialized client instance to obtain data through the DIS stream. The sample code is available in the **ConsumerDemo.java** file under the **dis-sdk-demo\\src\\main\\java\\com\\bigdata\\dis\\sdk\\demo** directory.
The value of **streamName** must be the same as that of **Stream Name** configured in :ref:`Step 1: Creating a DIS Stream <dis_01_0601>`.

2
umn/source/dis_sdk_usage_guide/initializing_a_dis_client.rst
View File

@ -22,7 +22,7 @@ You can use either of the following methods to initialize a DIS SDK client insta
- Use the configuration file to initialize a DIS SDK client instance.
Add the following configuration items to the **dis.propertites** file in the **dis-sdk-demo\resources** directory:
Add the following configuration items to the **dis.propertites** file in the **dis-sdk-demo\\resources** directory:
- ak/sk: AK/SK created on the IAM
- region: region of the stream

2
umn/source/dis_sdk_usage_guide/uploading_real-time_files/uploading_files.rst
View File

@ -30,7 +30,7 @@ Enter the absolute path of the file to be uploaded.
Specifying the Name of the File for Storing the Uploaded Real-Time Data
-----------------------------------------------------------------------
Set the file name to be uploaded. If the file name contains the backslash (\), use the escape character to escape the backslash.
Set the file name to be uploaded. If the file name contains the backslash (\\), use the escape character to escape the backslash.
.. code-block::

2
umn/source/dis_sdk_usage_guide/uploading_streaming_data.rst
View File

@ -8,7 +8,7 @@ Uploading Streaming Data
Sample Code
-----------
Use the initialized client instance to upload your streaming data to DIS. The sample code is available in the **ProducerDemo.java** file in the **dis-sdk-demo\src\com\bigdata\dis\sdk\demo** directory.
Use the initialized client instance to upload your streaming data to DIS. The sample code is available in the **ProducerDemo.java** file in the **dis-sdk-demo\\src\\com\\bigdata\\dis\\sdk\\demo** directory.
The value of **streamName** must be the same as that of **Stream Name** configured in :ref:`Step 1: Creating a DIS Stream <dis_01_0601>`.

2
umn/source/getting_started/creating_an_iam_agency.rst
View File

@ -24,7 +24,6 @@ Creating an IAM Agency
.. figure:: /_static/images/en-us_image_0000001266897473.jpg
:alt: **Figure 1** Creating an IAM agency
**Figure 1** Creating an IAM agency
#. Configure agency parameters and click **OK**.
@ -33,7 +32,6 @@ Creating an IAM Agency
.. figure:: /_static/images/en-us_image_0000001266777501.jpg
:alt: **Figure 2** Configuring agency parameters
**Figure 2** Configuring agency parameters
.. table:: **Table 1** Agency parameters

1
umn/source/getting_started/step_1_creating_a_dis_stream.rst
View File

@ -16,7 +16,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001266897449.jpg
:alt: **Figure 1** DIS management console
**Figure 1** DIS management console
#. Click |image1| in the upper left corner of the page and select a region and project.

6
umn/source/getting_started/step_2_preparing_a_dis_application_development_environment.rst
View File

@ -26,7 +26,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001266697405.png
:alt: **Figure 1** Preferences
**Figure 1** Preferences
c. In the navigation tree, choose **Java** > **Installed JREs**.
@ -38,7 +37,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001266777405.png
:alt: **Figure 2** Installed JREs
**Figure 2** Installed JREs
#. .. _dis_01_0602__li12377149194529:
@ -53,7 +51,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001222257286.png
:alt: **Figure 3** JRE Type
**Figure 3** JRE Type
#. Select a JRE type and click **Next**.
@ -62,7 +59,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001266897385.png
:alt: **Figure 4** JRE Definition
**Figure 4** JRE Definition
#. .. _dis_01_0602__li6851708153416:
@ -88,7 +84,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001222417242.png
:alt: **Figure 5** Importing a project
**Figure 5** Importing a project
d. Click **Finish**.
@ -114,7 +109,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001222097282.png
:alt: **Figure 6** Java Build Path
**Figure 6** Java Build Path
c. Add the JDK.

3
umn/source/getting_started/step_3_sending_data_to_dis.rst
View File

@ -19,7 +19,7 @@ Local data is continuously uploaded to DIS.
Sample Code
-----------
The example code file is the **ProducerDemo.java** file in the **\\dis-sdk-demo\src\main\java\com\bigdata\dis\sdk\demo** directory decompressed from the **dis-sdk-1.2.3.zip** package. The compression package is downloaded in :ref:`Step 2: Preparing a DIS Application Development Environment <dis_01_0602>`.
The example code file is the **ProducerDemo.java** file in the **\\dis-sdk-demo\\src\\main\\java\\com\\bigdata\\dis\\sdk\\demo** directory decompressed from the **dis-sdk-1.2.3.zip** package. The compression package is downloaded in :ref:`Step 2: Preparing a DIS Application Development Environment <dis_01_0602>`.
Running the Producer Program
----------------------------
@ -30,7 +30,6 @@ Right-click the producer application and choose **Run As** > **1 Java Applicatio
.. figure:: /_static/images/en-us_image_0000001266977421.png
:alt: **Figure 1** Running a producer application
**Figure 1** Running a producer application
While data is being sent to DIS, the DIS console displays DIS stream information. If information similar to the following is displayed, the data has been successfully sent to DIS:

2
umn/source/getting_started/step_4_obtaining_data_from_dis.rst
View File

@ -13,7 +13,7 @@ You can retrieve data from DIS when needed.
Sample Code
-----------
The example code file is the **ConsumerDemo.java** file in the **\\dis-sdk-demo\src\main\java\com\bigdata\dis\sdk\demo** directory decompressed from the **dis-sdk-XXX.zip** package. The compression package is downloaded in :ref:`Step 2: Preparing a DIS Application Development Environment <dis_01_0602>`.
The example code file is the **ConsumerDemo.java** file in the **\\dis-sdk-demo\\src\\main\\java\\com\\bigdata\\dis\\sdk\\demo** directory decompressed from the **dis-sdk-XXX.zip** package. The compression package is downloaded in :ref:`Step 2: Preparing a DIS Application Development Environment <dis_01_0602>`.
Running the Consumer Application
--------------------------------

2
umn/source/managing_a_dump_task/creating_a_dump_task.rst
View File

@ -72,7 +72,7 @@ If a dump task is created for a DIS stream, data sent to the DIS stream can be a
| | - Comma (,) | |
| | - Semicolon (;) | |
| | - Vertical bar (|) | |
| | - Newline (\n) | |
| | - Newline (\\n) | |
| | - NULL | |
| | | |
| | You can only select but not enter a value in this field. | |

2
umn/source/managing_dis_streams/deleting_a_stream.rst
View File

@ -19,7 +19,6 @@ Deleting a Stream
.. figure:: /_static/images/en-us_image_0000001222417238.jpg
:alt: **Figure 1** Managing DIS streams
**Figure 1** Managing DIS streams
#. Click **Delete** in the **Operation** column of the stream that you want to delete. The **Delete Stream** dialog box is displayed.
@ -28,7 +27,6 @@ Deleting a Stream
.. figure:: /_static/images/en-us_image_0000001222257278.jpg
:alt: **Figure 2** Deleting a stream
**Figure 2** Deleting a stream
#. Click **OK** to delete the selected stream.

1
umn/source/managing_dis_streams/listing_dis_streams.rst
View File

@ -11,7 +11,6 @@ The **Stream Management** page displays all DIS streams created. After clicking
.. figure:: /_static/images/en-us_image_0000001222097286.jpg
:alt: **Figure 1** Stream Management
**Figure 1** Stream Management
- **Name/ID**: Unique name of the DIS stream to be created. A stream name is 1 to 64 characters long. Only letters, digits, hyphens (-), and underscores (_) are allowed.

1
umn/source/managing_dis_streams/viewing_stream_monitoring_metrics.rst
View File

@ -19,7 +19,6 @@ You can view stream monitoring information on the console and monitor the data c
.. figure:: /_static/images/en-us_image_0000001266897429.jpg
:alt: **Figure 1** Monitoring page
**Figure 1** Monitoring page
#. On the **Monitoring** page, click the **Streams** or **Partitions** tab to view stream or partition monitoring metrics. :ref:`Table 1 <dis_01_0012__table2942144318834>` describes the monitoring parameters. For details about basic stream information, see :ref:`3 <dis_01_0601__li23032735111458>`.

1
umn/source/monitoring/querying_metrics.rst
View File

@ -42,7 +42,6 @@ Procedure
.. figure:: /_static/images/en-us_image_0000001222417338.png
:alt: **Figure 1** Viewing DIS monitoring metrics
**Figure 1** Viewing DIS monitoring metrics
#. Click |image2| in the upper right corner of the graphs to zoom out the graphs.

86
umn/source/monitoring/supported_metrics.rst
View File

@ -24,49 +24,49 @@ Metrics
.. table:: **Table 1** DIS metrics
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Metric Name | Meaning | Value Range | Monitored Object | Monitoring Period (Raw Data) |
+==============================================+===================================================================================================+=============+==================+==============================+
| Total Input Traffic | The amount of data uploaded through a stream during a specific period. | ≥ 0 bytes/s | Stream | 1 minute |
| | | | | |
| | Unit: byte/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Total Output Traffic | The amount of data downloaded through a stream during a specific period. | ≥ 0 bytes/s | Stream | 1 minute |
| | | | | |
| | Unit: byte/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Total Input Records | The number of records uploaded through a stream during a specific period. | ≥ 0 Count/s | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Total Output Records | The number of records downloaded through a stream during a specific period. | ≥ 0 Count/s | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Successful Upload Requests | The number of successful requests for uploading data through a stream during a specific period. | ≥ 0 Count/s | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Successful Download Requests | The number of successful requests for downloading data through a stream during a specific period. | ≥ 0 Count/s | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Average Processing Time of Upload Requests | Average upload request delay of a stream during a specific period. | 0 to 50 ms | Stream | 1 minute |
| | | | | |
| | Unit: ms | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Average Processing Time of Download Requests | Average download request delay of a stream during a specific period. | 0 to 50 ms | Stream | 1 minute |
| | | | | |
| | Unit: ms | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Throttled Upload Requests | The number of rejected upload requests due to flow control. | 0 to 1 | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
| Throttled Download Requests | The number of rejected download requests due to flow control. | 0 to 1 | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+-------------+------------------+------------------------------+
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Metric Name | Meaning | Value Range | Monitored Object | Monitoring Period (Raw Data) |
+==============================================+===================================================================================================+==============+==================+==============================+
| Total Input Traffic | The amount of data uploaded through a stream during a specific period. | >= 0 bytes/s | Stream | 1 minute |
| | | | | |
| | Unit: byte/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Total Output Traffic | The amount of data downloaded through a stream during a specific period. | >= 0 bytes/s | Stream | 1 minute |
| | | | | |
| | Unit: byte/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Total Input Records | The number of records uploaded through a stream during a specific period. | >= 0 Count/s | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Total Output Records | The number of records downloaded through a stream during a specific period. | >= 0 Count/s | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Successful Upload Requests | The number of successful requests for uploading data through a stream during a specific period. | >= 0 Count/s | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Successful Download Requests | The number of successful requests for downloading data through a stream during a specific period. | >= 0 Count/s | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Average Processing Time of Upload Requests | Average upload request delay of a stream during a specific period. | 0 to 50 ms | Stream | 1 minute |
| | | | | |
| | Unit: ms | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Average Processing Time of Download Requests | Average download request delay of a stream during a specific period. | 0 to 50 ms | Stream | 1 minute |
| | | | | |
| | Unit: ms | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Throttled Upload Requests | The number of rejected upload requests due to flow control. | 0 to 1 | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
| Throttled Download Requests | The number of rejected download requests due to flow control. | 0 to 1 | Stream | 1 minute |
| | | | | |
| | Unit: count/s | | | |
+----------------------------------------------+---------------------------------------------------------------------------------------------------+--------------+------------------+------------------------------+
Dimension
---------

1
umn/source/overview/what_is_dis.rst
View File

@ -16,7 +16,6 @@ If you set the dump destination to Object Storage Service (OBS), the streaming d
.. figure:: /_static/images/en-us_image_0000001222097314.png
:alt: **Figure 1** Data flow-1
**Figure 1** Data flow-1
Basic Concepts