forked from docs/data-ingestion-service
Compare commits
2 Commits
main
...
propose-di
| Author | SHA1 | Date | |
|---|---|---|---|
| a1b4bc5386 | |||
| d85c3e7a69 |
|
Before Width: | Height: | Size: 84 KiB After Width: | Height: | Size: 84 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 120 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 120 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 43 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 101 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 120 KiB |
BIN
umn/source/_static/images/en-us_image_0000001402769822.gif
Normal file
BIN
umn/source/_static/images/en-us_image_0000001402769822.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
BIN
umn/source/_static/images/en-us_image_0000001403080926.gif
Normal file
BIN
umn/source/_static/images/en-us_image_0000001403080926.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 14 KiB |
BIN
umn/source/_static/images/en-us_image_0000001403086902.gif
Normal file
BIN
umn/source/_static/images/en-us_image_0000001403086902.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
BIN
umn/source/_static/images/en-us_image_0000001453010069.gif
Normal file
BIN
umn/source/_static/images/en-us_image_0000001453010069.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 12 KiB |
BIN
umn/source/_static/images/en-us_image_0000001453327325.gif
Normal file
BIN
umn/source/_static/images/en-us_image_0000001453327325.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
BIN
umn/source/_static/images/en-us_image_0000001499513972.png
Normal file
BIN
umn/source/_static/images/en-us_image_0000001499513972.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 23 KiB |
|
Before Width: | Height: | Size: 75 KiB After Width: | Height: | Size: 75 KiB |
|
Before Width: | Height: | Size: 15 KiB After Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 23 KiB |
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 25 KiB |
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
|
Before Width: | Height: | Size: 949 B After Width: | Height: | Size: 949 B |
|
Before Width: | Height: | Size: 949 B After Width: | Height: | Size: 949 B |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 36 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 36 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 34 KiB |
|
Before Width: | Height: | Size: 47 KiB After Width: | Height: | Size: 47 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 36 KiB |
|
Before Width: | Height: | Size: 6.2 KiB After Width: | Height: | Size: 6.2 KiB |
|
Before Width: | Height: | Size: 339 B After Width: | Height: | Size: 339 B |
|
Before Width: | Height: | Size: 949 B After Width: | Height: | Size: 949 B |
@ -1,52 +0,0 @@
|
||||
:original_name: dis_01_0223.html
|
||||
|
||||
.. _dis_01_0223:
|
||||
|
||||
Billing Model
|
||||
=============
|
||||
|
||||
Data Ingestion Service (DIS) is charged for resource usage, there are three main billing factors: the duration of Partition and PUT Payload Unit number.
|
||||
|
||||
- General partition contains duration, PUT Payload Unit number, Data storage size billing factor. Advanced partition contains duration billing factor.
|
||||
- The duration of Partition: Partition is the base capacity unit of DIS. There are two types of partition: General partition and advanced partition. The general partition provides a capacity of 1MB/sec data input , 2MB/sec data outputand support up to 1000 records put per second. The advanced partition provides a capacity of 5MB/sec data input ,10MB/sec data output and support up to 2000 records put per second .You specify the number of partitions when create the stream based on the throughput requirements. You are charged for each partition at an hourly rate.
|
||||
- PUT Payload Unit number: A record is the data that your data producer adds to DIS stream. A PUT Payload Unit is counted in 25KB payload "chunks" that comprise a record. For example, a 5KB record contains one PUT Payload Unit, a 45KB record contains two PUT Payload Units, and a 1000KB record contains 40 PUT Payload Units. PUT Payload Unit is charged with a per million PUT Payload Units rate.
|
||||
- Data storage size: the data storage size in DIS input from the partition. The Customer can specify stream data storage time: from One day to Seven days when create the stream. If the data storage time exceeds the specified time, the data will be automatically aged. In general, the data storage one day is free, it means that 84GB of storage space for each partition is free to use. Of course, the final free space size can be specified according to the actual situation.
|
||||
|
||||
DIS is charged for resource usage as follow:
|
||||
|
||||
- One Partition Costs = partition time \* partition rate + Payload Unit Num \* Payload Unit Rate + Data storage size \* storage size rate
|
||||
- Total Costs per stream = Partition 1 Cost + Partition 2 Cost + … + Partition n Cost
|
||||
|
||||
Both resources will be charged at the same time. Each charged will be listed separately. There will be 3 CDR records for each stream in the billing interval.
|
||||
|
||||
The data will be cached for some time in the service, and default cache time is 24 hours. If the user want dump the data to OBS, the operation in OBS will be charged separately.
|
||||
|
||||
Example:
|
||||
|
||||
Let us assume that our data producers put 50 records per second in aggregate, and each record is 35KB. In this case, the total data input rate is 1.7MB/sec (50 records/sec*35KB/record). For simplicity, we assume that the throughput and data size of each record are stable and constant throughout the day.
|
||||
|
||||
We first calculate the number of partitions needed for our stream to achieve the required throughput. As one partition provides a capacity of 1MB/sec data input and supports 1000 records/sec, two partitions provide a capacity of 2MB/sec data input and support 2000 records/sec. So a stream with two partitions satisfies our required throughput of 1.7MB/sec at 50 records/sec.
|
||||
|
||||
We then calculate our costs using DIS pricing:
|
||||
|
||||
- Partition Hour: One Partition costs $X per hour. Our stream has two partitions so that it costs $2X per hour.
|
||||
- PUT Payload Unit (25KB): The 1 million Payload Unit costs $Y. As our record is 35KB, each record contains two PUT Payload Units. Our data producers put 50 records or 100 PUT Payload Units per second in aggregate. That is 180000 records or 360000 PUT Payload Units per hour.so it costs 0.36*Y per hour.
|
||||
|
||||
If the data in the DIS stored 1 day, the Data storage size is always less than 84GB, so the total cost per hour is :$(2X + Y)
|
||||
|
||||
Optionally, we can choose to increase the data retention period of our stream from 24 hours to up to 7 days. In this example, if the throughput is stable, the data storage size is 514.08GB per partition(0.85MB/sec \* 3600sec \* 24hour \* 7day).We assume that the data storage rate is $Z per GB*Hour, the cost is $860.16*Z( the detail is (514.08-84)GB \* 2 partitions \* Z). The total cost should $(2*X + Y + 860.16*Z).
|
||||
|
||||
**Shard Hour**
|
||||
|
||||
Shard is the base throughput unit of an Amazon Kinesis stream. One shard provides a capacity of 1MB/sec data input and 2MB/sec data output. One shard can support up to 1000 records per second. You specify the number of shards needed within your stream based on your throughput requirements. You are charged for each shard at an hourly rate.
|
||||
|
||||
**PUT Payload Unit (25KB)**
|
||||
|
||||
A record is the data that your data producer adds to your Amazon Kinesis stream. A PUT Payload Unit is counted in 25KB payload "chunks" that comprise a record. For example, a 5KB record contains one PUT Payload Unit, a 45KB record contains two PUT Payload Units, and a 1MB record contains 40 PUT Payload Units. PUT Payload Unit is charged with a per million PUT Payload Units rate.
|
||||
|
||||
**Extended Data Retention (Up to 7 days)**
|
||||
|
||||
Amazon Kinesis stores your data for 24 hours by default. You can choose to increase the data retention period of your stream to up to 7 days. You are charged for an additional rate on each shard hour incurred by your stream once you enable extended data retention.
|
||||
|
||||
- Getting records from Amazon Kinesis stream is free.
|
||||
- Data transfer is free. AWS does not charge for data transfer from your data producers to Amazon Kinesis Streams, or from Amazon Kinesis Streams to your Amazon Kinesis Applications.
|
||||
@ -1,16 +0,0 @@
|
||||
:original_name: dis_01_0226.html
|
||||
|
||||
.. _dis_01_0226:
|
||||
|
||||
CDR Folder
|
||||
==========
|
||||
|
||||
+-------------------------------------------+------------------------------------------+
|
||||
| Export Directory | Backup Directory |
|
||||
+===========================================+==========================================+
|
||||
| - /opt/meterfiles/uploads/dispartition | - /opt/meterfiles/backup/dispartition |
|
||||
+-------------------------------------------+------------------------------------------+
|
||||
| - /opt/meterfiles/uploads/dispayloadunit | - /opt/meterfiles/backup/dispayloadunit |
|
||||
+-------------------------------------------+------------------------------------------+
|
||||
| - /opt/meterfiles/uploads/disdatasize | - /opt/meterfiles/backup/disdatasize |
|
||||
+-------------------------------------------+------------------------------------------+
|
||||
@ -1,18 +0,0 @@
|
||||
:original_name: dis_01_0224.html
|
||||
|
||||
.. _dis_01_0224:
|
||||
|
||||
CDR Folder and File Name
|
||||
========================
|
||||
|
||||
- :ref:`Overview <dis_01_0225>`
|
||||
- :ref:`CDR Folder <dis_01_0226>`
|
||||
- :ref:`Service CDR File Name <dis_01_0227>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
overview
|
||||
cdr_folder
|
||||
service_cdr_file_name
|
||||
@ -1,10 +0,0 @@
|
||||
:original_name: dis_01_0225.html
|
||||
|
||||
.. _dis_01_0225:
|
||||
|
||||
Overview
|
||||
========
|
||||
|
||||
Similar to other cloud services , CDR files for DIS will be generated, containing DIS usage information, OTC billing system will fetch these files from certain directory on Common CDR SFTP via public key to complete the charging of DIS.
|
||||
|
||||
The file compression agonization, MD5 verification, error file handling are totally same as the CDR files of other cloud services.for details, please refer to document of"OTC CDR File Specifications".
|
||||
@ -1,22 +0,0 @@
|
||||
:original_name: dis_01_0227.html
|
||||
|
||||
.. _dis_01_0227:
|
||||
|
||||
Service CDR File Name
|
||||
=====================
|
||||
|
||||
OTC_{region_code}_dispart_YYYYMMDDHHMMSS_YYYYMMDDHHMMSS_{az_code}_{serial_number}.csv/zip
|
||||
|
||||
OTC_{region_code}_dispayunit_YYYYMMDDHHMMSS_YYYYMMDDHHMMSS_{az_code}_{serial_number}.csv/zip
|
||||
|
||||
OTC_{region_code}_dissize_YYYYMMDDHHMMSS_YYYYMMDDHHMMSS_{az_code}_{serial_number}.csv/zip
|
||||
|
||||
The {az_code} is not used by dis, the value is set the AZ01 number by default.
|
||||
|
||||
**For example:**
|
||||
|
||||
OTC_eu-de_dispart_20170427100000_20170427105959_eu-de-01_0001.csv/zip
|
||||
|
||||
OTC_eu-de_dispayunit_20170427100000_20170427105959_eu-de-01_0001.csv/zip
|
||||
|
||||
OTC_eu-de_dissize\ **\_20170427100000_20170427105959\_ eu-de-01 \_0001.csv/zip**
|
||||
@ -1,20 +0,0 @@
|
||||
:original_name: dis_01_0228.html
|
||||
|
||||
.. _dis_01_0228:
|
||||
|
||||
CDR Specification
|
||||
=================
|
||||
|
||||
- :ref:`The CDR Specification <dis_01_0229>`
|
||||
- :ref:`The Partition CDR Content <dis_01_0230>`
|
||||
- :ref:`The PUT Payload Unit CDR Content <dis_01_0231>`
|
||||
- :ref:`The Data Store Size CDR Content <dis_01_0232>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
the_cdr_specification
|
||||
the_partition_cdr_content
|
||||
the_put_payload_unit_cdr_content
|
||||
the_data_store_size_cdr_content
|
||||
@ -1,30 +0,0 @@
|
||||
:original_name: dis_01_0229.html
|
||||
|
||||
.. _dis_01_0229:
|
||||
|
||||
The CDR Specification
|
||||
=====================
|
||||
|
||||
Charging requirement: Charging starts from the time when the DIS instance is created and stops when the DIS instance is deleted.
|
||||
|
||||
CDR record cycle: The minimum granularity is minute, and the current cycle is 60 minutes. The interval for uploading offline CDR files is one hour.
|
||||
|
||||
There are three types of Product ID corresponding to three billing factors in DIS. The detail information of the product ID looks like below:
|
||||
|
||||
.. table:: **Table 1** DIS Product ID
|
||||
|
||||
+--------------------------------+-----------------------+-------------------------------------------------------------+-----------------+
|
||||
| ResourceTypeCode | ID Commercial Product | Describe of Commercial Product EN | Unit of Measure |
|
||||
+================================+=======================+=============================================================+=================+
|
||||
| *otc*.resource.type.dispart | OTC_DIS_GEN_TIME | The length of time the user is using the general partition | second |
|
||||
+--------------------------------+-----------------------+-------------------------------------------------------------+-----------------+
|
||||
| | OTC_DIS_ADV_TIME | The length of time the user is using the advanced partition | second |
|
||||
+--------------------------------+-----------------------+-------------------------------------------------------------+-----------------+
|
||||
| *otc*.resource.type.dispayunit | OTC_DIS_GEN_UNIT | The number of the PUT Payload Unit for general partition | number |
|
||||
+--------------------------------+-----------------------+-------------------------------------------------------------+-----------------+
|
||||
| | OTC_DIS_ADV_UNIT | The number of the PUT Payload Unit for advanced partition | number |
|
||||
+--------------------------------+-----------------------+-------------------------------------------------------------+-----------------+
|
||||
| *otc*.resource.type.dissize | OTC_DIS_GEN_STORE | The data storage size in general partition. | byte |
|
||||
+--------------------------------+-----------------------+-------------------------------------------------------------+-----------------+
|
||||
| | OTC_DIS_ADV_STORE | The data storage size in advanced partition. | byte |
|
||||
+--------------------------------+-----------------------+-------------------------------------------------------------+-----------------+
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,74 +0,0 @@
|
||||
:original_name: dis_01_0230.html
|
||||
|
||||
.. _dis_01_0230:
|
||||
|
||||
The Partition CDR Content
|
||||
=========================
|
||||
|
||||
.. table:: **Table 1** DIS Partition CDR Record Content
|
||||
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Field | Location | Type | Description | Remarks |
|
||||
+======================+=============+=============+==================================================+===============================================================================================================================================================================================================================================================================================+
|
||||
| RecordType | 1 | String | Type of a record in a file. | Number of normal offline CDR records: 20 |
|
||||
| | | | | |
|
||||
| | | | | Number of corrected error offline CDR records: 30 |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| TimeStamp | 2 | String | Time when an offline CDR record is generated. | UTC Format: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| UserID | 3 | String | Unique ID of a subscriber. | The value is the same as the value of **ProjectID**. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| RegionCode | 4 | String | ID of a region. | ID of a region. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| AZCode | 5 | String | Code of an available zone. | The DIS is Region Service, The AZCode is set AZ01 by default. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| CloudServiceTypeCode | 6 | String | Type of a cloud service. | Cloud service type corresponding to **ResourceID**. For example, cloudservicetype for VM in **EC2** is recorded as **otc.service.type.ec2**, and cloudservicetype for VM in **VDC** is recorded as **otc\ .\ service.type.vdc**. |
|
||||
| | | | | |
|
||||
| | | | | For DIS: the string is **otc.service.type.dis** |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ResourceTypeCode | 7 | String | Type of a cloud resource. | Cloud resource type corresponding to **ResourceID**, such as VM. |
|
||||
| | | | | |
|
||||
| | | | | **ResourceType** is a resource type specified by **CloudServiceType**, and **CloudServiceType** contains multiple sets of **ResourceType**. For example, EC2 is provided by VM, Volume, IP, and Image, and EC2 is a **CloudServiceType**, and VM, Volume, IP, and Image are **ResourceType**. |
|
||||
| | | | | |
|
||||
| | | | | For DIS: the string is **otc.resource.type.dis** |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ResourceSpecCode | 8 | String | Code of specified resource specifications. | There are two type resourcespeccode for DIS: |
|
||||
| | | | | |
|
||||
| | | | | dis.general.partition |
|
||||
| | | | | |
|
||||
| | | | | dis.advanced.partition |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ResourceID | 9 | String | ID of a resource instance. | ID of the partition in the DIS stream instance. |
|
||||
| | | | | |
|
||||
| | | | | The ID is different from partitions. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| BSSParams | 10 | String | Extended operation parameter. | The contract ID of one tenant. The contract ID is the xdomain_id of user token. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| BeginTime | 11 | String | Start time of the statistics period of a record. | UTC Format: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| EndTime | 12 | String | End time of the statistics period of a record. | UTC Format: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Duration | 13 | String | Name of a specific accumulative factor. | Accumulative factor: Duration |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Time | 14 | Double | Accumulative value of **AccumulateFactorName**. | Accumulative value of duration in the period ranging from BeginTime to EndTime multiply the number of partitions. The value is 0 or a positive number. |
|
||||
| | | | | |
|
||||
| | | | | The time unit is hour. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ExtendParams | 15 | String | Extended field. | This field is not used in this CDR record, the value is empty. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ProductID | 16 | String | Product ID of this service | Product ID : |
|
||||
| | | | | |
|
||||
| | | | | OTC_DIS_GEN_TIME |
|
||||
| | | | | |
|
||||
| | | | | OTC_DIS_ADV_TIME |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| CET/CEST BeginTime | 17 | String | Start time of the statistics period of a record. | Server Local Time: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| CET/CEST EndTime | 18 | String | End time of the statistics period of a record. | Server Local Time: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Tag | 19 | String | Extended field. | This field is not used in DIS service, the value is empty. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
Example of Partition Duration CDR content:
|
||||
|
||||
20|20161013150423 \| a21be352dfa7682dfa768dfa7682dfdf \| eu-west-0 *\|* \| *otc.service.type.dis* \| *otc.resource.type.dispartition* \| dis.general.partition \| 2cegee33-2173-4982-881c-7w8f27d23h54 \|\ *00000000001000003344* \| 20161013130000 \| 20161013135959 \| Duration \| 3600 \| \| OTC_DIS_GEN_TIME \| 20161013140000 \| 20161013145959\|
|
||||
@ -1,78 +0,0 @@
|
||||
:original_name: dis_01_0231.html
|
||||
|
||||
.. _dis_01_0231:
|
||||
|
||||
The PUT Payload Unit CDR Content
|
||||
================================
|
||||
|
||||
.. table:: **Table 1** DIS PUT Payload Unit CDR Record Content
|
||||
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Field | Location | Type | Description | Remarks |
|
||||
+======================+=============+=============+==================================================+===============================================================================================================================================================================================================================================================================================+
|
||||
| RecordType | 1 | String | Type of a record in a file. | Number of normal offline CDR records: 20 |
|
||||
| | | | | |
|
||||
| | | | | Number of corrected error offline CDR records: 30 |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| TimeStamp | 2 | String | Time when an offline CDR record is generated. | UTC Format: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| UserID | 3 | String | Unique ID of a subscriber. | The value is the same as the value of **ProjectID**. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| RegionCode | 4 | String | ID of a region. | ID of a region. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| AZCode | 5 | String | Code of an available zone. | Code of an available zone. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| CloudServiceTypeCode | 6 | String | Type of a cloud service. | Cloud service type corresponding to **ResourceID**. For example, cloudservicetype for VM in **EC2** is recorded as **otc.service.type.ec2**, and cloudservicetype for VM in **VDC** is recorded as **otc.service.type.vdc**. |
|
||||
| | | | | |
|
||||
| | | | | For DIS: the string is **otc.service.type.dis** |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ResourceTypeCode | 7 | String | Type of a cloud resource. | Cloud resource type corresponding to **ResourceID**, such as VM. |
|
||||
| | | | | |
|
||||
| | | | | **ResourceType** is a resource type specified by **CloudServiceType**, and **CloudServiceType** contains multiple sets of **ResourceType**. For example, EC2 is provided by VM, Volume, IP, and Image, and EC2 is a **CloudServiceType**, and VM, Volume, IP, and Image are **ResourceType**. |
|
||||
| | | | | |
|
||||
| | | | | For DIS: the string is **otc.resource.type.dis** |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ResourceSpecCode | 8 | String | Code of specified resource specifications. | There are two type resourcespeccode for DIS: |
|
||||
| | | | | |
|
||||
| | | | | dis.general.partition |
|
||||
| | | | | |
|
||||
| | | | | dis.advanced.partition |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ResourceID | 9 | String | ID of a resource instance. | ID of the partition in the DIS stream instance. |
|
||||
| | | | | |
|
||||
| | | | | The ID is different from partitions. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| BSSParams | 10 | String | Extended operation parameter. | The contract ID of one tenant. The contract ID is the xdomain_id of user token. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| BeginTime | 11 | String | Start time of the statistics period of a record. | UTC Format: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| EndTime | 12 | String | End time of the statistics period of a record. | UTC Format: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| InputTraffic | 13 | String | Name of a specific accumulative factor. | Accumulative factor: InputUnitNum |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| InputTrafficValue | 14 | Double | Accumulative value of **AccumulateFactorName**. | Accumulative value of InputUnitNum. |
|
||||
| | | | | |
|
||||
| | | | | This is the total Num of input record(sum/million) in the partition during this time period. |
|
||||
| | | | | |
|
||||
| | | | | The value unit is sum/million |
|
||||
| | | | | |
|
||||
| | | | | . |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ExtendParams | 15 | String | Extended field. | This field indicates the amount of data the user is actually putting. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| ProductID | 16 | String | Product ID of this service | Product ID: |
|
||||
| | | | | |
|
||||
| | | | | OTC_DIS_GEN_UNIT |
|
||||
| | | | | |
|
||||
| | | | | OTC_DIS_ADV_UNIT |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| CET/CEST BeginTime | 17 | String | Start time of the statistics period of a record. | Server Local Time: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| CET/CEST EndTime | 18 | String | End time of the statistics period of a record. | Server Local Time: YYYYMMDDHH24MISS |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Tag | 19 | String | Extended field. | This field is not used in DIS service, the value is empty. |
|
||||
+----------------------+-------------+-------------+--------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
Example of PUT Payload Unit CDR content:
|
||||
|
||||
20|20161013150423 \| a21be352dfa7682dfa768dfa7682dfdf \| eu-west-0 *\|* \| *otc.service.type.dis* \| *otc.resource.type.dispayloadunit* \| dis.general.partition \| 1dbcee33-7837-4316-852d-5d9f27f11e28 \|\ *00000000001000003344* \| 20161013130000 \| 20161013135959 \| InputUnitNum \| 12000 \| 3201284 \| OTC_DIS_GEN_UNIT \|20161013140000 \| 20161013145959\|
|
||||
@ -1,18 +0,0 @@
|
||||
:original_name: dis_01_0222.html
|
||||
|
||||
.. _dis_01_0222:
|
||||
|
||||
CDR Specification
|
||||
=================
|
||||
|
||||
- :ref:`Billing Model <dis_01_0223>`
|
||||
- :ref:`CDR Folder and File Name <dis_01_0224>`
|
||||
- :ref:`CDR Specification <dis_01_0228>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
billing_model
|
||||
cdr_folder_and_file_name/index
|
||||
cdr_specification/index
|
||||
@ -8,17 +8,16 @@ Change History
|
||||
+-----------------------------------+----------------------------------------------------------------------------------+
|
||||
| Release Date | What's New |
|
||||
+===================================+==================================================================================+
|
||||
| 2022-06-15 | Modified the following section: |
|
||||
| 2023-03-30 | Modified the following sections: |
|
||||
| | |
|
||||
| | :ref:`What Is DIS? <dis_01_0001>` |
|
||||
| | |
|
||||
| | Added the following sections: |
|
||||
| | |
|
||||
| | :ref:`CDR Specification <dis_01_0222>` |
|
||||
| | - :ref:`General Procedure <dis_01_0009>` |
|
||||
| | - :ref:`Step 1: Creating a DIS Stream <dis_01_0601>` |
|
||||
| | - :ref:`Viewing Stream Monitoring Metrics <dis_01_0012>` |
|
||||
| | - :ref:`Configuring Sample Projects <dis_06_0013>` |
|
||||
+-----------------------------------+----------------------------------------------------------------------------------+
|
||||
| 2020-01-03 | Modified the following section: |
|
||||
| 2022-09-30 | Removed the following section: |
|
||||
| | |
|
||||
| | :ref:`Viewing Stream Monitoring Metrics <dis_01_0012>` |
|
||||
| | CDR Specification |
|
||||
+-----------------------------------+----------------------------------------------------------------------------------+
|
||||
| 2019-06-12 | Modified the following section: |
|
||||
| | |
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
Configuring Sample Projects
|
||||
===========================
|
||||
|
||||
The **dis-sdk-**\ *1.2.3*\ **.zip** package of the latest version in :ref:`Downloading SDKs <dis_06_0005>` provides a sample project. You can use a development tool (such as Eclipse) on the local server to compile and run the sample project. You can also develop your applications based on the sample project. The sample project code is available in the **\\dis-sdk-demo\\src\\com\\bigdata\\dis\\sdk\\demo** directory.
|
||||
Download the **dis-sdk-**\ *1.2.3*\ **.zip** package from the URL noted in section :ref:`Downloading SDKs <dis_06_0005>`. The package provides a sample project. You can use a development tool (such as Eclipse) on the local server to compile and run the sample project. You can also develop your applications based on the sample project. The sample project code is available in the **\\dis-sdk-demo\\src\\com\\bigdata\\dis\\sdk\\demo** directory.
|
||||
|
||||
================= ====================
|
||||
Sample Code Description
|
||||
@ -28,7 +28,7 @@ Procedure
|
||||
|
||||
.. _dis_06_0013__f23cd224614d04888bfa7c759107662f3:
|
||||
|
||||
.. figure:: /_static/images/en-us_image_0000001222097302.png
|
||||
.. figure:: /_static/images/en-us_image_0124306701.png
|
||||
:alt: **Figure 1** Import Projects
|
||||
|
||||
**Figure 1** Import Projects
|
||||
@ -45,7 +45,7 @@ Procedure
|
||||
#. Click **Apply and Close**.
|
||||
|
||||
|
||||
.. figure:: /_static/images/en-us_image_0000001222417262.png
|
||||
.. figure:: /_static/images/en-us_image_0000001499513972.png
|
||||
:alt: **Figure 2** Resource
|
||||
|
||||
**Figure 2** Resource
|
||||
@ -59,7 +59,7 @@ Procedure
|
||||
#. On the **Properties for dis-sdk-demo** page, click **Apply and Close** to import all the **.jar** files in the current path and the **third_lib** folder to the project.
|
||||
|
||||
|
||||
.. figure:: /_static/images/en-us_image_0000001266977433.png
|
||||
.. figure:: /_static/images/en-us_image_0124306705.png
|
||||
:alt: **Figure 3** Java Build Path
|
||||
|
||||
**Figure 3** Java Build Path
|
||||
|
||||
@ -7,7 +7,7 @@ Creating a Stream
|
||||
|
||||
Initialize a DIS client instance named **dic**. For details, see :ref:`Initializing a DIS Client <dis_06_0015>`.
|
||||
|
||||
When you use the DIS SDK to create a DIS stream, specify the stream name, number of partitions in the stream, and stream type.
|
||||
When you use the DIS SDK to create a DIS stream, specify the stream name, number of partitions in the stream and stream type.
|
||||
|
||||
**STREAM_TYPE_COMMON** indicates a common stream, and **STREAM_TYPE_ADVANCED** indicates an advanced stream.
|
||||
|
||||
|
||||
@ -6,4 +6,4 @@ Importing Third-party Dependency Packages
|
||||
=========================================
|
||||
|
||||
#. In the navigation tree, click **Project Explorer**, right-click a project and choose **Properties > Java Build Path > Add External JARs**.
|
||||
#. Locate the directory containing the decompressed **dis-sdk** folder, and import all the files with the extension **JAR** in the current directory and the **third_party** folder to the project.
|
||||
#. Locate the directory containing the decompressed **dis-sdk** folder and import all the files with the extension **JAR** in the current directory and the **third_party** folder to the project.
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
Initializing a DIS Client
|
||||
=========================
|
||||
|
||||
You can use either of the following methods to initialize a DIS SDK client instance: For details about **endpoint**, **ak**, **sk**, **region**, and **projectId**, see :ref:`Obtaining Authentication Information <dis_01_0043>`.
|
||||
You can use one of the following methods to initialize a DIS SDK client instance: For details about **endpoint**, **ak**, **sk**, **region**, and **projectId**, see :ref:`Obtaining Authentication Information <dis_01_0043>`.
|
||||
|
||||
- Use the code to initialize the DIS SDK client instance. For details about the code example, see the **ProducerDemo.java** file.
|
||||
|
||||
|
||||
@ -5,6 +5,6 @@
|
||||
Preparing the Environment
|
||||
=========================
|
||||
|
||||
- Download JDK1.8 or a later version from the `Oracle official website <http://www.oracle.com/technetwork/java/archive-139210.html>`__ and install it, and configure Java environment variables.
|
||||
- Download Eclipse IDE for Java Developers of the latest version from the `Eclipse's official website <http://www.eclipse.org/downloads/eclipse-packages/>`__, and install it.
|
||||
- Download JDK1.8 or a later version from the `Oracle official website <http://www.oracle.com/technetwork/java/archive-139210.html>`__, install it and configure Java environment variables.
|
||||
- Download the latest version of Eclipse IDE for Java Developers from the `Eclipse's official website <http://www.eclipse.org/downloads/eclipse-packages/>`__ and install it.
|
||||
- Configure the JDK in Eclipse.
|
||||
|
||||
@ -8,7 +8,7 @@ Uploading Files
|
||||
Configuring a Stream for Uploading Real-Time Files
|
||||
--------------------------------------------------
|
||||
|
||||
You must use the stream enabled to transmit real-time files. Here, **fileUploadStream** is used as an example.
|
||||
You must use the enabled stream to transmit real-time files. Here, **fileUploadStream** is used as an example.
|
||||
|
||||
The value of **streamName** must be the same as that of **Stream Name** configured in :ref:`Step 1: Creating a DIS Stream <dis_01_0601>`.
|
||||
|
||||
|
||||
8
umn/source/faqs.rst
Normal file
8
umn/source/faqs.rst
Normal file
@ -0,0 +1,8 @@
|
||||
:original_name: en-us_topic_0124374673.html
|
||||
|
||||
.. _en-us_topic_0124374673:
|
||||
|
||||
FAQs
|
||||
====
|
||||
|
||||
This chapter describes some FAQs.
|
||||
@ -1,37 +0,0 @@
|
||||
:original_name: dis_faq_0009.html
|
||||
|
||||
.. _dis_faq_0009:
|
||||
|
||||
How Do I Check Software Package Integrity?
|
||||
==========================================
|
||||
|
||||
This section describes how to verify integrity of the DIS SDK software package on a Linux system by using a verification file.
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
|
||||
- The PuTTY tool is available.
|
||||
- The WinSCP tool is available.
|
||||
|
||||
Procedure
|
||||
---------
|
||||
|
||||
#. Upload the DIS SDK software package **dis-sdk-1.2.3.zip** to any directory on the Linux system by using WinSCP.
|
||||
|
||||
#. .. _dis_faq_0009__l8c35f745b7e945219234467020564558:
|
||||
|
||||
Log in to the Linux system by using PuTTY. In the directory in which **dis-sdk-1.2.3.zip** is stored, run the following command to obtain the verification code of the DIS SDK software package:
|
||||
|
||||
**sha256sum dis-sdk-1.2.3.zip**
|
||||
|
||||
Example verification code:
|
||||
|
||||
.. code-block::
|
||||
|
||||
# sha256sum dis-sdk-1.2.3.zip
|
||||
8be2c937e8d78b1a9b99777cee4e7131f8bf231de3f839cf214e7c5b5ba3c088 dis-sdk-1.2.3.zip
|
||||
|
||||
#. Open the DIS SDK verification file **dis-sdk-1.2.3.zip.sha256sum** and compare it with the verification code obtained in :ref:`2 <dis_faq_0009__l8c35f745b7e945219234467020564558>`.
|
||||
|
||||
- If they are consistent, it indicates that the DIS SDK software package is not tampered with.
|
||||
- If they are inconsistent, the DIS SDK software package is tampered with and you need to obtain an untampered package.
|
||||
@ -1,13 +0,0 @@
|
||||
:original_name: dis_faq_0003.html
|
||||
|
||||
.. _dis_faq_0003:
|
||||
|
||||
How Does DIS Send and Receive Data?
|
||||
===================================
|
||||
|
||||
#. Create a DIS stream, and obtain your Access Key ID/Secret Access Key (AK/SK) from the Identity and Access Management (IAM) service.
|
||||
#. Download the package from https://dis.obs.eu-de.otc.t-systems.com/dis/download/dis-sdk-1.2.3.zip.
|
||||
#. Create a project and configure the user AK/SK, endpoint, project ID, region, stream name, and partition quantity.
|
||||
#. After the configuration is completed, run the application to send data.
|
||||
#. Create a project and configure the user AK/SK, endpoint, project, region, stream name, partition ID, and startingSequenceNumber.
|
||||
#. After the configuration is completed, run the application to receive data.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user