yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
adding_new_docs
doc-exports/docs/dis/umn/CLASS.TXT.json
Xiong, Chen Xiao 1a4b9e91b7 DIS UMN test2 version 
Reviewed-by: gtema <artem.goncharov@gmail.com>
Co-authored-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com>
Co-committed-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com>
2022-11-14 16:38:55 +00:00

560 lines
21 KiB
JSON
Raw Permalink Blame History

[
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"Overview",
"uri":"dis_01_0600.html",
"doc_type":"usermanual",
"p_code":"",
"code":"1"
},
{
"desc":"Data Ingestion Service (DIS) addresses the challenge of transmitting data from outside the cloud to inside the cloud. DIS builds data intake streams for custom applicatio",
"product_code":"dis",
"title":"What Is DIS?",
"uri":"dis_01_0001.html",
"doc_type":"usermanual",
"p_code":"1",
"code":"2"
},
{
"desc":"You can use DIS for rapid data intake from producers and continuous data processing. The following are typical scenarios for using DIS:Accelerated log and data feed intak",
"product_code":"dis",
"title":"What Can I Do with DIS?",
"uri":"dis_01_0002.html",
"doc_type":"usermanual",
"p_code":"1",
"code":"3"
},
{
"desc":"DIS manages the infrastructure, storage, networking, and configuration needed to stream your data. You do not have to worry about provisioning, deployment, and constant m",
"product_code":"dis",
"title":"Functions",
"uri":"dis_01_0003.html",
"doc_type":"usermanual",
"p_code":"1",
"code":"4"
},
{
"desc":"DIS works with the following services:Object Storage Service (OBS)If you set Dump Destination to OBS, the uploaded data will be stored to OBS.Identity and Access Manageme",
"product_code":"dis",
"title":"Related Services",
"uri":"dis_01_0004.html",
"doc_type":"usermanual",
"p_code":"1",
"code":"5"
},
{
"desc":"You need an account on the management console to create, query, and delete DIS streams, and need an Access Key ID/Secret Access Key (AK/SK) file to push, pull, and dump d",
"product_code":"dis",
"title":"Permissions",
"uri":"dis_01_0005.html",
"doc_type":"usermanual",
"p_code":"1",
"code":"6"
},
{
"desc":"Read the following precautions before using DIS:Recommended browser to access the DIS consoleGoogle Chrome: 48.0 or laterMozilla FireFox: 43.0 or laterInternet Explorer: ",
"product_code":"dis",
"title":"Precautions",
"uri":"dis_01_0006.html",
"doc_type":"usermanual",
"p_code":"1",
"code":"7"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"Getting Started",
"uri":"dis_01_0008.html",
"doc_type":"usermanual",
"p_code":"",
"code":"8"
},
{
"desc":"The following is the general procedure for using DIS:Step 1: Creating a DIS StreamYou need to create a stream before using DIS.Step 2: Preparing a DIS Application Develop",
"product_code":"dis",
"title":"General Procedure",
"uri":"dis_01_0009.html",
"doc_type":"usermanual",
"p_code":"8",
"code":"9"
},
{
"desc":"You can create a DIS stream on the DIS management console.",
"product_code":"dis",
"title":"Step 1: Creating a DIS Stream",
"uri":"dis_01_0601.html",
"doc_type":"usermanual",
"p_code":"8",
"code":"10"
},
{
"desc":"Before developing DIS applications, prepare an application development environment, and then obtain a software development kit (SDK) and sample project and import them to",
"product_code":"dis",
"title":"Step 2: Preparing a DIS Application Development Environment",
"uri":"dis_01_0602.html",
"doc_type":"usermanual",
"p_code":"8",
"code":"11"
},
{
"desc":"Local data is continuously uploaded to DIS.Data can be stored in DIS, OBS, and DLI. For details about how to configure a storage location, see Creating a Dump Task.The ma",
"product_code":"dis",
"title":"Step 3: Sending Data to DIS",
"uri":"dis_01_0603.html",
"doc_type":"usermanual",
"p_code":"8",
"code":"12"
},
{
"desc":"You can retrieve data from DIS when needed.The example code file is the ConsumerDemo.java file in the \\dis-sdk-demo\\src\\main\\java\\com\\bigdata\\dis\\sdk\\demo directory decom",
"product_code":"dis",
"title":"Step 4: Obtaining Data from DIS",
"uri":"dis_01_0604.html",
"doc_type":"usermanual",
"p_code":"8",
"code":"13"
},
{
"desc":"Access Key ID/Secret Access Key (AK/SK) is created on Identity and Access Management (IAM) to authenticate calls to application programming interfaces (APIs) on the publi",
"product_code":"dis",
"title":"Obtaining Authentication Information",
"uri":"dis_01_0043.html",
"doc_type":"usermanual",
"p_code":"8",
"code":"14"
},
{
"desc":"DIS can upload data to Object Storage Service (OBS).An IAM agency has been created by following the procedure in Creating an IAM Agency. This IAM agency entrusts DIS to a",
"product_code":"dis",
"title":"Interconnecting with OBS",
"uri":"dis_01_0045.html",
"doc_type":"usermanual",
"p_code":"8",
"code":"15"
},
{
"desc":"If you choose to dump data from DIS to OBS, create an IAM agency that grants DIS permissions to access OBS.",
"product_code":"dis",
"title":"Creating an IAM Agency",
"uri":"dis_01_0605.html",
"doc_type":"usermanual",
"p_code":"8",
"code":"16"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"Managing DIS Streams",
"uri":"dis_01_0010.html",
"doc_type":"usermanual",
"p_code":"",
"code":"17"
},
{
"desc":"The Stream Management page displays all DIS streams created. After clicking a stream, you can view the following information about this stream:Name/ID: Unique name of the",
"product_code":"dis",
"title":"Listing DIS Streams",
"uri":"dis_01_0011.html",
"doc_type":"usermanual",
"p_code":"17",
"code":"18"
},
{
"desc":"You can view stream monitoring information on the console and monitor the data consumed by applications in the stream.",
"product_code":"dis",
"title":"Viewing Stream Monitoring Metrics",
"uri":"dis_01_0012.html",
"doc_type":"usermanual",
"p_code":"17",
"code":"19"
},
{
"desc":"A tag is an identifier of a stream. Adding tags to streams can help you identify and manage your stream resources.You can add a maximum of 10 tags to a stream when creati",
"product_code":"dis",
"title":"Managing Stream Tags",
"uri":"dis_01_0050.html",
"doc_type":"usermanual",
"p_code":"17",
"code":"20"
},
{
"desc":"A deleted stream will not be charged and cannot be restored. Exercise caution when performing this operation.",
"product_code":"dis",
"title":"Deleting a Stream",
"uri":"dis_01_0013.html",
"doc_type":"usermanual",
"p_code":"17",
"code":"21"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"Managing a Dump Task",
"uri":"dis_01_0115.html",
"doc_type":"usermanual",
"p_code":"",
"code":"22"
},
{
"desc":"If a dump task is created for a DIS stream, data sent to the DIS stream can be automatically dumped to the selected target specified in the dump task.A maximum of five du",
"product_code":"dis",
"title":"Creating a Dump Task",
"uri":"dis_01_0047.html",
"doc_type":"usermanual",
"p_code":"22",
"code":"23"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"Monitoring",
"uri":"dis_01_0130.html",
"doc_type":"usermanual",
"p_code":"",
"code":"24"
},
{
"desc":"This section describes metrics reported by DIS to Cloud Eye as well as their namespaces and dimensions. You can use Cloud Eye to query metric information generated for DI",
"product_code":"dis",
"title":"Supported Metrics",
"uri":"dis_01_0131.html",
"doc_type":"usermanual",
"p_code":"24",
"code":"25"
},
{
"desc":"You can customize the monitored objects and notification policies by setting alarm rules to learn DIS running status in a timely manner.A DIS alarm rule includes the alar",
"product_code":"dis",
"title":"Setting Alarm Rules",
"uri":"dis_01_0132.html",
"doc_type":"usermanual",
"p_code":"24",
"code":"26"
},
{
"desc":"Cloud Eye monitors the running status of the DIS stream. You can obtain the monitoring metrics of CDM on the Cloud Eye management console.Monitored data requires a period",
"product_code":"dis",
"title":"Querying Metrics",
"uri":"dis_01_0134.html",
"doc_type":"usermanual",
"p_code":"24",
"code":"27"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"CDR Specification",
"uri":"dis_01_0222.html",
"doc_type":"usermanual",
"p_code":"",
"code":"28"
},
{
"desc":"Data Ingestion Service (DIS) is charged for resource usage, there are three main billing factors: the duration of Partition and PUT Payload Unit number.General partition ",
"product_code":"dis",
"title":"Billing Model",
"uri":"dis_01_0223.html",
"doc_type":"usermanual",
"p_code":"28",
"code":"29"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"CDR Folder and File Name",
"uri":"dis_01_0224.html",
"doc_type":"usermanual",
"p_code":"28",
"code":"30"
},
{
"desc":"Similar to other cloud services , CDR files for DIS will be generated, containing DIS usage information, OTC billing system will fetch these files from certain directory ",
"product_code":"dis",
"title":"Overview",
"uri":"dis_01_0225.html",
"doc_type":"usermanual",
"p_code":"30",
"code":"31"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"CDR Folder",
"uri":"dis_01_0226.html",
"doc_type":"usermanual",
"p_code":"30",
"code":"32"
},
{
"desc":"OTC_{region_code}_dispart_YYYYMMDDHHMMSS_YYYYMMDDHHMMSS_{az_code}_{serial_number}.csv/zipOTC_{region_code}_dispayunit_YYYYMMDDHHMMSS_YYYYMMDDHHMMSS_{az_code}_{serial_numb",
"product_code":"dis",
"title":"Service CDR File Name",
"uri":"dis_01_0227.html",
"doc_type":"usermanual",
"p_code":"30",
"code":"33"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"CDR Specification",
"uri":"dis_01_0228.html",
"doc_type":"usermanual",
"p_code":"28",
"code":"34"
},
{
"desc":"Charging requirement: Charging starts from the time when the DIS instance is created and stops when the DIS instance is deleted.CDR record cycle: The minimum granularity ",
"product_code":"dis",
"title":"The CDR Specification",
"uri":"dis_01_0229.html",
"doc_type":"usermanual",
"p_code":"34",
"code":"35"
},
{
"desc":"Example of Partition Duration CDR content:20|20161013150423 | a21be352dfa7682dfa768dfa7682dfdf | eu-west-0|| otc.service.type.dis | otc.resource.type.dispartition | dis.g",
"product_code":"dis",
"title":"The Partition CDR Content",
"uri":"dis_01_0230.html",
"doc_type":"usermanual",
"p_code":"34",
"code":"36"
},
{
"desc":"Example of PUT Payload Unit CDR content:20|20161013150423 | a21be352dfa7682dfa768dfa7682dfdf | eu-west-0|| otc.service.type.dis | otc.resource.type.dispayloadunit | dis.g",
"product_code":"dis",
"title":"The PUT Payload Unit CDR Content",
"uri":"dis_01_0231.html",
"doc_type":"usermanual",
"p_code":"34",
"code":"37"
},
{
"desc":"Sample scenario for the user:Starting at 11:15(UTC time), a user created a stream containing 2 partitions, the data store time is 1 day. The user put 50 records per secon",
"product_code":"dis",
"title":"The Data Store Size CDR Content",
"uri":"dis_01_0232.html",
"doc_type":"usermanual",
"p_code":"34",
"code":"38"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"DIS SDK Usage Guide",
"uri":"dis_06_0011.html",
"doc_type":"usermanual",
"p_code":"",
"code":"39"
},
{
"desc":"Data Ingestion Service Software Development Kit (DIS SDK) is the encapsulation of RESTful APIs provided by DIS to simplify development. You can use DIS by calling API fun",
"product_code":"dis",
"title":"Downloading SDKs",
"uri":"dis_06_0005.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"40"
},
{
"desc":"Download JDK1.8 or a later version from the Oracle official website and install it, and configure Java environment variables.Download Eclipse IDE for Java Developers of t",
"product_code":"dis",
"title":"Preparing the Environment",
"uri":"dis_06_0012.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"41"
},
{
"desc":"The DIS SDK provides compatibility with native kafka client APIs.",
"product_code":"dis",
"title":"Connecting to kafka consumer",
"uri":"dis_06_0020.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"42"
},
{
"desc":"The dis-sdk-1.2.3.zip package of the latest version in Downloading SDKs provides a sample project. You can use a development tool (such as Eclipse) on the local server to",
"product_code":"dis",
"title":"Configuring Sample Projects",
"uri":"dis_06_0013.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"43"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"Importing Third-party Dependency Packages",
"uri":"dis_06_0014.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"44"
},
{
"desc":"You can use either of the following methods to initialize a DIS SDK client instance: For details about endpoint, ak, sk, region, and projectId, see Obtaining Authenticati",
"product_code":"dis",
"title":"Initializing a DIS Client",
"uri":"dis_06_0015.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"45"
},
{
"desc":"Initialize a DIS client instance named dic. For details, see Initializing a DIS Client.When you use the DIS SDK to create a DIS stream, specify the stream name, number of",
"product_code":"dis",
"title":"Creating a Stream",
"uri":"dis_06_0501.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"46"
},
{
"desc":"Initialize a DIS client instance named dic. For details, see Initializing a DIS Client.Use the DIS SDK to delete a specified DIS stream.After configuring DeleteStreamRequ",
"product_code":"dis",
"title":"Deleting a Stream",
"uri":"dis_06_0502.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"47"
},
{
"desc":"Initialize a DIS client instance named dic. For details, see Initializing a DIS Client.Use the DIS SDK to list active streams.Use the setLimit method to set the number of",
"product_code":"dis",
"title":"Querying a Stream List",
"uri":"dis_06_0503.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"48"
},
{
"desc":"Initialize a DIS client instance named dic. For details, see Initializing a DIS Client.Use the DIS SDK to query the details about a specified stream.The returned stream d",
"product_code":"dis",
"title":"Querying Stream Details",
"uri":"dis_06_0504.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"49"
},
{
"desc":"Use the initialized client instance to upload your streaming data to DIS. The sample code is available in the ProducerDemo.java file in the dis-sdk-demo\\src\\com\\bigdata\\d",
"product_code":"dis",
"title":"Uploading Streaming Data",
"uri":"dis_06_0016.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"50"
},
{
"desc":"Use the initialized client instance to obtain data through the DIS stream. The sample code is available in the ConsumerDemo.java file under the dis-sdk-demo\\src\\main\\java",
"product_code":"dis",
"title":"Downloading Streaming Data",
"uri":"dis_06_0017.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"51"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"Uploading Real-Time Files",
"uri":"dis_06_0022.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"52"
},
{
"desc":"In the Project Explorer pane, right-click com.bigdata.dis.sdk.demo, and choose New > Class from the shortcut menu. The New Java Class dialog box is displayed. Set the cat",
"product_code":"dis",
"title":"Creating a Demo Class",
"uri":"dis_06_0506.html",
"doc_type":"usermanual",
"p_code":"52",
"code":"53"
},
{
"desc":"Initialize a DIS SDK asynchronous client instance. For details about endpoint, ak, sk, region, and projectId, see Obtaining Authentication Information.//Create a DIS clie",
"product_code":"dis",
"title":"Initializing a DIS SDK Asynchronous Client Instance",
"uri":"dis_06_0023.html",
"doc_type":"usermanual",
"p_code":"52",
"code":"54"
},
{
"desc":"You must use the stream enabled to transmit real-time files. Here, fileUploadStream is used as an example.The value of streamName must be the same as that of Stream Name ",
"product_code":"dis",
"title":"Uploading Files",
"uri":"dis_06_0507.html",
"doc_type":"usermanual",
"p_code":"52",
"code":"55"
},
{
"desc":"Initialize a DIS client instance named dic. For details, see Initializing a DIS Client.Use the DIS SDK to obtain the information about the data cursor.",
"product_code":"dis",
"title":"Obtaining the Data Cursor",
"uri":"dis_06_0505.html",
"doc_type":"usermanual",
"p_code":"39",
"code":"56"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"FAQs",
"uri":"dis_faq_0000.html",
"doc_type":"usermanual",
"p_code":"",
"code":"57"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"How Does DIS Send and Receive Data?",
"uri":"dis_faq_0003.html",
"doc_type":"usermanual",
"p_code":"57",
"code":"58"
},
{
"desc":"After DIS is enabled, data is stored to DIS by default. After a dump task is added, the data can be dumped to other resources. Table 1 describes the specific differences.",
"product_code":"dis",
"title":"What Is the Difference Between Storing Data into DIS and Dumping Data Elsewhere?",
"uri":"dis_faq_0007.html",
"doc_type":"usermanual",
"p_code":"57",
"code":"59"
},
{
"desc":"This section describes how to verify integrity of the DIS SDK software package on a Linux system by using a verification file.The PuTTY tool is available.The WinSCP tool ",
"product_code":"dis",
"title":"How Do I Check Software Package Integrity?",
"uri":"dis_faq_0009.html",
"doc_type":"usermanual",
"p_code":"57",
"code":"60"
},
{
"desc":"HUAWEI CLOUD Help Center presents technical documents to help you quickly get started with HUAWEI CLOUD services. The technical documents include Service Overview, Price Details, Purchase Guide, User Guide, API Reference, Best Practices, FAQs, and Videos.",
"product_code":"dis",
"title":"Change History",
"uri":"dis_01_0014.html",
"doc_type":"usermanual",
"p_code":"",
"code":"61"
},
{
"desc":"For details, see Glossary.",
"product_code":"dis",
"title":"Glossary",
"uri":"dis_01_0000.html",
"doc_type":"usermanual",
"p_code":"",
"code":"62"
}
]
View Git Blame Copy Permalink