mikacur/data-ingestion-service_aligns_with_the_OTC_platform_07_23
1
0
Fork 0
forked from docs/data-ingestion-service
Code Pull Requests Activity

Update content

Browse Source
This commit is contained in:
OpenTelekomCloud Proposal Bot 2023-03-28 14:24:02 +00:00
parent d85c3e7a69
commit a1b4bc5386
16 changed files with 26 additions and 106 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
umn/source/_static/images/en-us_image_0000001499513972.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

BIN
umn/source/_static/images/en-us_image_0124306703.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 43 KiB

7
umn/source/change_history.rst
View File

@ -8,6 +8,13 @@ Change History
+-----------------------------------+----------------------------------------------------------------------------------+
| Release Date | What's New |
+===================================+==================================================================================+
| 2023-03-30 | Modified the following sections: |
| | |
| | - :ref:`General Procedure <dis_01_0009>` |
| | - :ref:`Step 1: Creating a DIS Stream <dis_01_0601>` |
| | - :ref:`Viewing Stream Monitoring Metrics <dis_01_0012>` |
| | - :ref:`Configuring Sample Projects <dis_06_0013>` |
+-----------------------------------+----------------------------------------------------------------------------------+
| 2022-09-30 | Removed the following section: |
| | |
| | CDR Specification |

2
umn/source/dis_sdk_usage_guide/configuring_sample_projects.rst
View File

@ -45,7 +45,7 @@ Procedure
#. Click **Apply and Close**.
.. figure:: /_static/images/en-us_image_0124306703.png
.. figure:: /_static/images/en-us_image_0000001499513972.png
:alt: **Figure 2** Resource
**Figure 2** Resource

8
umn/source/faqs.rst Normal file
View File

@ -0,0 +1,8 @@
:original_name: en-us_topic_0124374673.html
.. _en-us_topic_0124374673:
FAQs
====
This chapter describes some FAQs.

37
umn/source/faqs/how_do_i_check_software_package_integrity.rst
View File

@ -1,37 +0,0 @@
:original_name: dis_faq_0009.html
.. _dis_faq_0009:
How Do I Check Software Package Integrity?
==========================================
This section describes how to verify integrity of the DIS SDK software package on a Linux system by using a verification file.
Prerequisites
-------------
- The PuTTY tool is available.
- The WinSCP tool is available.
Procedure
---------
#. Upload the DIS SDK software package **dis-sdk-1.2.3.zip** to any directory on the Linux system by using WinSCP.
#. .. _dis_faq_0009__l8c35f745b7e945219234467020564558:
Log in to the Linux system by using PuTTY. In the directory in which **dis-sdk-1.2.3.zip** is stored, run the following command to obtain the verification code of the DIS SDK software package:
**sha256sum dis-sdk-1.2.3.zip**
Example verification code:
.. code-block::
# sha256sum dis-sdk-1.2.3.zip
8be2c937e8d78b1a9b99777cee4e7131f8bf231de3f839cf214e7c5b5ba3c088 dis-sdk-1.2.3.zip
#. Open the DIS SDK verification file **dis-sdk-1.2.3.zip.sha256sum** and compare it with the verification code obtained in :ref:`Step 2 <dis_faq_0009__l8c35f745b7e945219234467020564558>`.
- If they are consistent, it indicates that the DIS SDK software package is not tampered with.
- If they are inconsistent, the DIS SDK software package is tampered with and you need to obtain an untampered package.

13
umn/source/faqs/how_does_dis_send_and_receive_data.rst
View File

@ -1,13 +0,0 @@
:original_name: dis_faq_0003.html
.. _dis_faq_0003:
How Does DIS Send and Receive Data?
===================================
#. Create a DIS stream, and obtain your Access Key ID/Secret Access Key (AK/SK) from the Identity and Access Management (IAM) service.
#. Download the package from https://dis.obs.eu-de.otc.t-systems.com/dis/download/dis-sdk-1.2.3.zip.
#. Create a project and configure the user AK/SK, endpoint, project ID, region, stream name, and partition quantity.
#. After the configuration is completed, run the application to send data.
#. Create a project and configure the user AK/SK, endpoint, project, region, stream name, partition ID, and startingSequenceNumber.
#. After the configuration is completed, run the application to receive data.

18
umn/source/faqs/index.rst
View File

@ -1,18 +0,0 @@
:original_name: dis_faq_0000.html
.. _dis_faq_0000:
FAQs
====
- :ref:`How Does DIS Send and Receive Data? <dis_faq_0003>`
- :ref:`What Is the Difference Between Storing Data into DIS and Dumping Data Elsewhere? <dis_faq_0007>`
- :ref:`How Do I Check Software Package Integrity? <dis_faq_0009>`
.. toctree::
:maxdepth: 1
:hidden:
how_does_dis_send_and_receive_data
what_is_the_difference_between_storing_data_into_dis_and_dumping_data_elsewhere
how_do_i_check_software_package_integrity

27
umn/source/faqs/what_is_the_difference_between_storing_data_into_dis_and_dumping_data_elsewhere.rst
View File

@ -1,27 +0,0 @@
:original_name: dis_faq_0007.html
.. _dis_faq_0007:
What Is the Difference Between Storing Data into DIS and Dumping Data Elsewhere?
================================================================================
After DIS is enabled, data is stored in DIS by default. After a dump task is added, the data can be dumped to other resources. :ref:`Table 1 <dis_faq_0007__tdc4c4ec0b2454cfeb8421d92412355e6>` describes the specific differences.
- Data is stored in DIS by default.
- If **Dump Destination** is set to **OBS**, data is stored in DIS and periodically imported to Object Storage Service (OBS).
.. _dis_faq_0007__tdc4c4ec0b2454cfeb8421d92412355e6:
.. table:: **Table 1** Difference between storing data into DIS and dumping data elsewhere
+---------------------------------------------------------------------+-----------------------------------------------------------------------------+
| DIS Storage | OBS Storage |
+=====================================================================+=============================================================================+
| You can store data into DIS without applying for storage resources. | You must apply for OBS resources before dumping data to OBS. |
+---------------------------------------------------------------------+-----------------------------------------------------------------------------+
| No additional payment is required. | Additional cost for the use of OBS. For details, see the OBS price details. |
+---------------------------------------------------------------------+-----------------------------------------------------------------------------+
| Data is temporarily stored in DIS for up to 168 hours. | Data is stored in OBS until your OBS bucket expires. |
+---------------------------------------------------------------------+-----------------------------------------------------------------------------+
| Data is stored only in DIS. | Data is stored in DIS and periodically dumped to OBS. |
+---------------------------------------------------------------------+-----------------------------------------------------------------------------+

2
umn/source/getting_started/general_procedure.rst
View File

@ -7,7 +7,7 @@ General Procedure
The following is the general procedure for using DIS:
:ref:`4.2-Step 1: Creating a DIS Stream <dis_01_0601>`
:ref:`Step 1: Creating a DIS Stream <dis_01_0601>`
You need to create a stream before using DIS.

8
umn/source/getting_started/step_1_creating_a_dis_stream.rst
View File

@ -20,9 +20,9 @@ Procedure
#. Click |image1| in the upper left corner of the page and select a region and project.
#. .. _dis_01_0601__li23032735111458:
#. Click Create Stream and set related parameters.
Click Create Stream and set related parameters.
.. _dis_01_0601__table1518745742610:
.. table:: **Table 1** Stream parameters
@ -42,7 +42,7 @@ Procedure
| | | |
| | - For a common DIS stream, the value of **Partitions** is an integer from 1 to 50. A tenant can create a maximum of 50 partitions. | |
| | - For an advanced DIS stream, the value of **Partitions** is an integer from 1 to 10. A tenant can create a maximum of 10 partitions. | |
| | - Both common DIS sreams and advanced DIS streams can automatically scale up. You can also scale them up manually. **Significantly**, a maximum of 10 automatic scale-up, 10 manual scale-up, and 1 scale-down operations can be performed for each stream **within one hour.** | |
| | - Both common DIS streams and advanced DIS streams can automatically scale up. You can also scale them up manually. **Significantly**, a maximum of 10 automatic scale-up, 10 manual scale-up, and 1 scale-down operations can be performed for each stream **within one hour.** | |
+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
| Partition Calculator | Calculator used to calculate the estimated number of partitions based on the information you entered. | ``-`` |
| | | |
@ -80,7 +80,7 @@ Procedure
| Tag | Identifier of the stream. Adding tags to streams can help you identify and manage your stream resources. | ``-`` |
+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
#. Click **Submit**. The **Details** page is displayed.
#. Click **Next**. The **Details** page is displayed.
#. Click **Submit**.

2
umn/source/getting_started/step_3_sending_data_to_dis.rst
View File

@ -19,7 +19,7 @@ Local data is continuously uploaded to DIS.
Sample Code
-----------
The example code file is the **ProducerDemo.java** file in the **\\dis-sdk-demo\\src\\main\\java\\com\\bigdata\\dis\\sdk\\demo** directory decompressed from the **dis-sdk-1.2.3.zip** package.
The example code file is the **ProducerDemo.java** file in the **\\dis-sdk-demo\\src\\main\\java\\com\\bigdata\\dis\\sdk\\demo** directory decompressed from the **dis-sdk-1.2.3.zip** package. The compression package is downloaded from the `DIS SDK <https://dis-publish.obs-website.cn-north-1.myhuaweicloud.com/>`__.
Running the Producer Program
----------------------------

2
umn/source/getting_started/step_4_obtaining_data_from_dis.rst
View File

@ -13,7 +13,7 @@ You can retrieve data from DIS when needed.
Sample Code
-----------
The example code file is the **ConsumerDemo.java** file in the **\\dis-sdk-demo\\src\\main\\java\\com\\bigdata\\dis\\sdk\\demo** directory decompressed from the **dis-sdk-XXX.zip** package.
The example code file is the **ConsumerDemo.java** file in the **\\dis-sdk-demo\\src\\main\\java\\com\\bigdata\\dis\\sdk\\demo** directory decompressed from the **dis-sdk-XXX.zip** package. The compression package is downloaded from the `DIS SDK <https://dis-publish.obs-website.cn-north-1.myhuaweicloud.com/>`__.
Running the Consumer Application
--------------------------------

2
umn/source/index.rst
View File

@ -11,5 +11,5 @@ Data Ingestion Service - User Guide
managing_a_dump_task/index
monitoring/index
dis_sdk_usage_guide/index
faqs/index
faqs
change_history

2
umn/source/managing_dis_streams/viewing_stream_monitoring_metrics.rst
View File

@ -21,7 +21,7 @@ You can view stream monitoring information on the console and monitor the data c
**Figure 1** Monitoring page
#. On the **Monitoring** page, click the **Streams** or **Partitions** tab to view stream or partition monitoring metrics. :ref:`Table 1 <dis_01_0012__table2942144318834>` describes the monitoring parameters. For details about basic stream information, see :ref:`Step 3 <dis_01_0601__li23032735111458>`.
#. On the **Monitoring** page, click the **Streams** or **Partitions** tab to view stream or partition monitoring metrics. :ref:`Table 1 <dis_01_0012__table2942144318834>` describes the monitoring parameters. For details about basic stream information, see :ref:`Table 1 <dis_01_0601__table1518745742610>`.
.. _dis_01_0012__table2942144318834:

2
umn/source/overview/functions.rst
View File

@ -41,5 +41,5 @@ Key Capabilities
- Scalable: A DIS stream can seamlessly scale its data throughput from megabytes to terabytes per hour, and from thousands to millions of PUT records per second.
- Easy to use: You can create a DIS stream within seconds. It is easy to put data into a stream, and build a data processing application.
- Low cost: DIS has no upfront cost and you only pay for the resources you use.
- Parallel processing: DIS allows you to have multiple DIS applications processing the same stream concurrently. For example, you can have one application running real-time analytics and another sending data to OBS from the same stream.
- Parallel processing: DIS allows you to have multiple DIS applications processing the same stream concurrently.
- Secure and reliable: DIS can retain data for 24 to 72 hours to prevent data loss in case of application faults, individual machine faults, or facility faults.