yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity

dli_umn_20250212

Browse Source 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
This commit is contained in:
Su, Xiaomeng
2025-02-24 08:11:08 +00:00
committed by zuul
parent 6878ea6e04
commit 3a07c40729
368 changed files with 13610 additions and 11149 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9352
docs/dli/umn/ALL_META.TXT.json
View File

File diff suppressed because it is too large Load Diff

3656
docs/dli/umn/CLASS.TXT.json
View File

File diff suppressed because it is too large Load Diff

27
docs/dli/umn/dli_01_00006.html
View File

@ -8,22 +8,29 @@
</th>
</tr>
</thead>
<tbody><tr id="dli_01_00006__row15952172464713"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p738312864717">2024-07-11</p>
<tbody><tr id="dli_01_00006__row189735272308"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p679643320304">2025-02-08</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p1796113323010">Modified the following sections:</p>
<ul id="dli_01_00006__ul16796833163014"><li id="dli_01_00006__li1979613316302">Added descriptions about the specifications and use cases of elastic resource pools to <a href="dli_07_0027.html">Specifications</a>.</li><li id="dli_01_00006__li888314338315">Added instructions on how to change a DLI agency to <strong id="dli_01_00006__b36793109118">dli_management_agency</strong> to <a href="dli_01_0419.html">DLI Agency Overview</a>.</li><li id="dli_01_00006__li15566455193317">Added instructions on how to view a SQL execution plan on the DLI management console to <a href="dli_01_0638.html">Viewing a SQL Execution Plan</a>.</li><li id="dli_01_00006__li550215152409">Added instructions on how to connect DLI to AOM Prometheus for monitoring to <a href="dli_01_0665.html">Connecting DLI to AOM's Prometheus Monitoring</a>.</li><li id="dli_01_00006__li660522419010">Taken the following section offline:<p id="dli_01_00006__p8105155919011"><a name="dli_01_00006__li660522419010"></a><a name="li660522419010"></a>Creating a Flink SQL Job</p>
</li></ul>
</td>
</tr>
<tr id="dli_01_00006__row15952172464713"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p738312864717">2024-07-11</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p15383828204713">Modified the following section:</p>
<ul id="dli_01_00006__ul13383162834712"><li id="dli_01_00006__li18383142874716">Added the description on how to import OBS data to DLI to <a href="dli_01_0420.html">Importing Data to a DLI Table</a>.</li></ul>
<ul id="dli_01_00006__ul13383162834712"><li id="dli_01_00006__li18383142874716">Added the description on how to import OBS data to DLI to <a href="dli_01_0509.html">Managing Enhanced Datasource Connections</a>.</li></ul>
</td>
</tr>
<tr id="dli_01_00006__row1340918386143"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p10168134117141">2024-04-28</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p181681241101413">Added the following section:</p>
<ul id="dli_01_00006__ul416812413143"><li id="dli_01_00006__li11168134151414"><a href="dli_01_0508.html">Elastic Resource Pool</a></li></ul>
<ul id="dli_01_00006__ul416812413143"><li id="dli_01_00006__li11168134151414"><a href="dli_01_0508.html">Creating an Elastic Resource Pool and Queues Within It</a></li></ul>
</td>
</tr>
<tr id="dli_01_00006__row4248101042"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p14248407419">2024-02-27</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p752534712233">Added the following content:</p>
<ul id="dli_01_00006__ul1052694717232"><li id="dli_01_00006__li45262474232">Added Spark 3.1.1 dependencies to <a href="dli_01_0397.html">Built-in Dependencies</a>.</li></ul>
<ul id="dli_01_00006__ul1052694717232"><li id="dli_01_00006__li45262474232">Added Spark 3.1.1 dependencies to <a href="dli_01_0397.html">DLI Built-in Dependencies</a>.</li></ul>
<p id="dli_01_00006__p593013411147">Modified the following section:</p>
<p id="dli_01_00006__p132481401644">Changed <strong id="dli_01_00006__b129917574313">DDS MongoDB</strong> to <strong id="dli_01_00006__b22823511410">DDS</strong> in <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</p>
</td>
@ -31,31 +38,31 @@
<tr id="dli_01_00006__row6758122510313"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p462652810312">2023-11-01</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p462614287310">Modified the following content:</p>
<ul id="dli_01_00006__ul13626728339"><li id="dli_01_00006__li186261028531">Modified the link for obtaining service support during quota application in <a href="dli_07_0009.html">Quotas</a>.</li><li id="dli_01_00006__li26261928732">Added the link to <em id="dli_01_00006__i1661311132615">Data Lake Insight API Reference</em> to <a href="dli_01_0378.html">What Is Data Lake Insight?</a>.</li><li id="dli_01_00006__li14959163416518">Modified the method of obtaining host information in <a href="dli_01_0013.html">Modifying Host Information in an Elastic Resource Pool</a>.</li></ul>
<ul id="dli_01_00006__ul13626728339"><li id="dli_01_00006__li186261028531">Modified the link for obtaining service support during quota application in <a href="dli_07_0009.html">Quotas</a>.</li><li id="dli_01_00006__li26261928732">Added the link to <em id="dli_01_00006__i1661311132615">Data Lake Insight API Reference</em> to <a href="dli_01_0378.html">What Is Data Lake Insight</a>.</li><li id="dli_01_00006__li14959163416518">Modified the method of obtaining host information in <a href="dli_01_0013.html">Modifying Host Information in an Elastic Resource Pool</a>.</li></ul>
</td>
</tr>
<tr id="dli_01_00006__row1569106191216"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p36566919122">2023-10-08</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p465618951211">Modified the following content:</p>
<ul id="dli_01_00006__ul82951936181419"><li id="dli_01_00006__li129511362149">Modified the example format in <a href="dli_01_0384.html#dli_01_0384__table17326112613420">Table 2</a>.</li><li id="dli_01_00006__li3295163691419">Added notes and constraints on tag keys and tag values to <a href="dli_01_0455.html">Creating a Flink SQL Job</a> and <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</li></ul>
<ul id="dli_01_00006__ul82951936181419"><li id="dli_01_00006__li129511362149">Modified the example format in <a href="dli_01_0384.html#dli_01_0384__table17326112613420">Table 2</a>.</li><li id="dli_01_00006__li3295163691419">Added notes and constraints on tag keys and tag values for creating Flink SQL jobs and <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</li></ul>
</td>
</tr>
<tr id="dli_01_00006__row12145240104716"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p193441641970">2023-09-07</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p63854280713">Added the following sections:</p>
<ul id="dli_01_00006__ul895111424710"><li id="dli_01_00006__li5328124733814"><a href="dli_01_0552.html">Managing Tags</a></li><li id="dli_01_00006__li11801555203817"><a href="dli_01_0384.html#dli_01_0384__table17326112613420">Table 2</a></li><li id="dli_01_00006__li109276713912"><a href="dli_01_0534.html">Enabling Dynamic Scaling for Flink Jobs</a></li><li id="dli_01_00006__li1356101513915"><a href="dli_01_0535.html">Setting the Priority for a Job</a></li></ul>
<ul id="dli_01_00006__ul895111424710"><li id="dli_01_00006__li5328124733814"><a href="dli_01_0552.html">Managing Tags</a></li><li id="dli_01_00006__li11801555203817"><a href="dli_01_0384.html#dli_01_0384__table17326112613420">Table 2</a></li><li id="dli_01_00006__li109276713912"><a href="dli_01_0534.html">Enabling Dynamic Scaling for Flink Jobs</a></li><li id="dli_01_00006__li1356101513915"><a href="dli_01_0535.html">Setting the Priority for a SQL Job</a></li></ul>
<p id="dli_01_00006__p523145072419">Deleted the following section:</p>
<ul id="dli_01_00006__ul204315504240"><li id="dli_01_00006__li64316503244">Deleted the content related to Flink job debugging.</li></ul>
<p id="dli_01_00006__p19615231153914">Modified the following section:</p>
<ul id="dli_01_00006__ul561713318395"><li id="dli_01_00006__li10617203123911"><a href="dli_01_0461.html">Performing Operations on a Flink Job</a>: Modified the description of the <strong id="dli_01_00006__b85796421240">Auto Restart upon Exception</strong> parameter in the runtime configuration. Flink SQL jobs and Flink OpenSource SQL jobs cannot be restored from checkpoints.</li></ul>
<div class="p" id="dli_01_00006__p10632114012391">Optimized the following sections:<ul id="dli_01_00006__ul1350219391399"><li id="dli_01_00006__li18502193953911"><a href="dli_01_0426.html">Enhanced Datasource Connections</a></li><li id="dli_01_00006__li12502839173915"><a href="dli_01_0422.html">Datasource Authentication</a></li></ul>
<ul id="dli_01_00006__ul561713318395"><li id="dli_01_00006__li10617203123911"><a href="dli_01_0461.html">Common Operations of Flink Jobs</a>: Modified the description of the <strong id="dli_01_00006__b85796421240">Auto Restart upon Exception</strong> parameter in the runtime configuration. Flink SQL jobs and Flink OpenSource SQL jobs cannot be restored from checkpoints.</li></ul>
<div class="p" id="dli_01_00006__p10632114012391">Optimized the following sections:<ul id="dli_01_00006__ul1350219391399"><li id="dli_01_00006__li18502193953911"><a href="dli_01_0426.html">Configuring the Network Connection Between DLI and Data Sources (Enhanced Datasource Connection)</a></li><li id="dli_01_00006__li12502839173915"><a href="dli_01_0422.html">Using DLI Datasource Authentication to Manage Access Credentials for Data Sources</a></li></ul>
</div>
</td>
</tr>
<tr id="dli_01_00006__row33702917592"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p1528713120599">2023-05-11</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p16287151205919">Modified the following content:</p>
<ul id="dli_01_00006__ul209491872110"><li id="dli_01_00006__li99491719111">Added the dependency package information of Spark 2.4.5 and Flink 1.12 to <a href="dli_01_0397.html">Built-in Dependencies</a>.</li><li id="dli_01_00006__li119492716114">Modified the description of log buckets in <a href="dli_01_0320.html">SQL Editor</a>.</li></ul>
<ul id="dli_01_00006__ul209491872110"><li id="dli_01_00006__li99491719111">Added the dependency package information of Spark 2.4.5 and Flink 1.12 to <a href="dli_01_0397.html">DLI Built-in Dependencies</a>.</li><li id="dli_01_00006__li119492716114">Modified the description of log buckets in <a href="dli_01_0320.html">Creating and Submitting a SQL Job</a>.</li></ul>
</td>
</tr>
<tr id="dli_01_00006__row01821691115"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p1518161613115">2023-01-30</p>

83
docs/dli/umn/dli_01_0001.html
View File

@ -1,19 +1,72 @@
<a name="dli_01_0001"></a><a name="dli_01_0001"></a>
<h1 class="topictitle1">Job Management</h1>
<div id="body1532683323732"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0567.html">Overview</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0017.html">SQL Job Management</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0389.html">Flink Job Management</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0465.html">Spark Job Management</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0535.html">Setting the Priority for a Job</a></strong><br>
</li>
</ul>
<h1 class="topictitle1">DLI Job Development Process</h1>
<div id="body1532683323732"><p id="dli_01_0001__p634719445401">This chapter walks you through on how to develop a DLI job.</p>
<div class="section" id="dli_01_0001__section20352194884317"><h4 class="sectiontitle">Creating an IAM User and Granting Permissions</h4><ul id="dli_01_0001__ul135691231713"><li id="dli_01_0001__li18569113115114">To manage fine-grained permissions for your DLI resources using IAM, create an IAM user and grant them permissions to DLI if you are an enterprise user. For details, see <a href="dli_01_0418.html">Creating an IAM User and Granting Permissions</a>.</li><li id="dli_01_0001__li1984631912407">When using DLI for the first time, you need to update the DLI agency according to the console's guidance so that DLI can use other cloud services and perform resource O&amp;M operations on your behalf. The agency includes permissions to obtain IAM user information, access and use VPCs, CIDR blocks, routes, and peering connections, and send notifications via SMN in case of job execution failure.<p id="dli_01_0001__p31302219401"><a name="dli_01_0001__li1984631912407"></a><a name="li1984631912407"></a>For more information on the specific permissions included in the agency, refer to <a href="dli_01_0618.html">Configuring DLI Agency Permissions</a>.</p>
</li></ul>
</div>
<div class="section" id="dli_01_0001__section14601204916117"><h4 class="sectiontitle">Creating Compute Resources and Metadata Required for Running Jobs</h4><ul id="dli_01_0001__ul63253525168"><li id="dli_01_0001__li4325052101616">Before submitting a job using DLI, you need to create an elastic resource pool and create queues within it. This will provide the necessary compute resources for running the job. For how to create an elastic resource pool and create queues within it, see <a href="dli_01_0504.html">Overview of DLI Elastic Resource Pools and Queues</a>.<p id="dli_01_0001__p76301243103911">Alternatively, you can enhance DLI's computing environment by creating custom images. Specifically, to enhance the functions and performance of Spark and Flink jobs, you can create custom images by downloading the base images provided by DLI and adding dependencies (files, JAR files, or software) and private capabilities required for job execution. This changes the container runtime environment for the jobs.</p>
<p id="dli_01_0001__p1558916387396">For example, you can add a Python package or C library related to machine learning to a custom image to help you extend functions. For how to create a custom image, see <a href="dli_01_0494.html">Using a Custom Image to Enhance the Job Running Environment</a>.</p>
</li><li id="dli_01_0001__li20816155172111">DLI metadata is the basis for developing SQL and Spark jobs. Before executing a job, you need to define databases and tables based on your business scenario.<div class="note" id="dli_01_0001__note9930169132116"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0001__p1793029152116">Flink allows for dynamic data types, enabling the definition of data structures at runtime without the need for predefined metadata.</p>
</div></div>
<ul id="dli_01_0001__ul199267513211"><li id="dli_01_0001__li11495849102120">Define your data structures, including data catalogs, databases, and tables. For details, see <a href="dli_01_0390.html">Creating Databases and Tables</a>.</li><li id="dli_01_0001__li892912291714">Create a bucket to store temporary data generated during job running, such as job logs and job results. For details, see <a href="dli_01_0536.html">Configuring a DLI Job Bucket</a>.</li><li id="dli_01_0001__li39297261711">Configure the permission to access metadata. For details, see <a href="dli_01_0447.html">Configuring Database Permissions on the DLI Console</a> and <a href="dli_01_0448.html">Configuring Table Permissions on the DLI Console</a>.</li></ul>
</li></ul>
</div>
<div class="section" id="dli_01_0001__section1542818345419"><h4 class="sectiontitle">Importing Data to DLI</h4><ul id="dli_01_0001__ul9974162293510"><li id="dli_01_0001__li17844113819217">DLI allows you to analyze and query data stored in OBS without the need to migrate it. Simply upload your data to OBS and use DLI for data analysis.</li><li id="dli_01_0001__li56411849203516">Cross-source access can reduce data duplication and latency when real-time access and processing of data from different sources is required for service needs.<p id="dli_01_0001__p1837516595371"><a name="dli_01_0001__li56411849203516"></a><a name="li56411849203516"></a>The prerequisites for cross-source access are that DLI can communicate with the data source network and DLI can obtain the access credentials to the data source.</p>
<ul id="dli_01_0001__ul1717315116388"><li id="dli_01_0001__li1764741614373">Configure network connection between DLI and the data source by referring to <a href="dli_01_0426.html">Configuring the Network Connection Between DLI and Data Sources (Enhanced Datasource Connection)</a>.</li><li id="dli_01_0001__li1062417276376">Manage data source credentials.<ul id="dli_01_0001__ul207641255531"><li id="dli_01_0001__li77501728185218">You can use DLI's datasource authentication to manage the authentication information for accessing a specified datasource.<p id="dli_01_0001__p36802397523"><a name="dli_01_0001__li77501728185218"></a><a name="li77501728185218"></a>This applies to SQL jobs and Flink 1.12 jobs. For details, see <a href="dli_01_0422.html">Using DLI Datasource Authentication to Manage Access Credentials for Data Sources</a>.</p>
</li><li id="dli_01_0001__li161116426526">You can also use DEW to manage access credentials for data sources and use a custom agency to authorize DLI to access DEW.<p id="dli_01_0001__p124621343145214"><a name="dli_01_0001__li161116426526"></a><a name="li161116426526"></a>This applies to Spark 3.3.1 or later and Flink 1.15 or later. For details, see <a href="dli_01_0636.html">Using DEW to Manage Access Credentials for Data Sources</a> and <a href="dli_01_0486.html">Configuring an Agency to Allow DLI to Access Other Cloud Services</a>.</p>
</li></ul>
</li></ul>
</li></ul>
</div>
<div class="section" id="dli_01_0001__section4783695410"><h4 class="sectiontitle">Submitting a Job Using DLI</h4><ul id="dli_01_0001__ul11108618173715"><li id="dli_01_0001__li5108191853716">DLI offers a serverless service that integrates stream processing, batch processing, and interactive analytics. It supports various job types to meet different data processing needs.
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0001__table74751136151810" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Job types supported by DLI</caption><thead align="left"><tr id="dli_01_0001__row1847593651818"><th align="left" class="cellrowborder" valign="top" width="14.85148514851485%" id="mcps1.3.5.2.1.1.2.4.1.1"><p id="dli_01_0001__p1747510367189">Job Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="51.81518151815182%" id="mcps1.3.5.2.1.1.2.4.1.2"><p id="dli_01_0001__p18475136181820">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.5.2.1.1.2.4.1.3"><p id="dli_01_0001__p1447503612184">Use Case</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0001__row124751436191819"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.2.1.1.2.4.1.1 "><p id="dli_01_0001__p10475173613181">SQL job</p>
</td>
<td class="cellrowborder" valign="top" width="51.81518151815182%" headers="mcps1.3.5.2.1.1.2.4.1.2 "><p id="dli_01_0001__p191717173329">This type is suitable for scenarios where standard SQL statements are used for querying. It is typically used for querying and analyzing structured data.</p>
<p id="dli_01_0001__p44613412135">For details, see <a href="dli_01_0320.html">Creating and Submitting a SQL Job</a>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.5.2.1.1.2.4.1.3 "><p id="dli_01_0001__p14101929162512">It applies to scenarios such as data warehouse query, report generation, and online analytical processing (OLAP).</p>
</td>
</tr>
<tr id="dli_01_0001__row347523611187"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.2.1.1.2.4.1.1 "><p id="dli_01_0001__p147517368184">Flink job</p>
</td>
<td class="cellrowborder" valign="top" width="51.81518151815182%" headers="mcps1.3.5.2.1.1.2.4.1.2 "><p id="dli_01_0001__p153579315325">This type is specifically designed for real-time data stream processing, making it ideal for scenarios that require low latency and quick response. It is well-suited for real-time monitoring and online analysis.</p>
<ul id="dli_01_0001__ul4984146132314"><li id="dli_01_0001__li15836281243">Flink OpenSource job: DLI provides standard connectors and various APIs to facilitate quick integration with other data systems. For details, see <a href="dli_01_0498.html">Creating a Flink OpenSource SQL Job</a>.</li><li id="dli_01_0001__li4498152019247">Flink Jar job: allows you to submit Flink jobs compiled into JAR files, providing greater flexibility and customization capabilities.<p id="dli_01_0001__p71471445173319"><a name="dli_01_0001__li4498152019247"></a><a name="li4498152019247"></a>It is suitable for complex data processing scenarios that require user-defined functions (UDFs) or specific library integration. The Flink ecosystem can be utilized to implement advanced stream processing logic and status management. For details, see <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</p>
</li></ul>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.5.2.1.1.2.4.1.3 "><p id="dli_01_0001__p1147533613181">It applies to scenarios that require quick response, such as real-time data monitoring and real-time recommender systems.</p>
<p id="dli_01_0001__p49171742132511">Flink Jar jobs are suitable for data analysis scenarios that require custom stream processing logic, complex state management, or integration with specific libraries.</p>
</td>
</tr>
<tr id="dli_01_0001__row347516361180"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.2.1.1.2.4.1.1 "><p id="dli_01_0001__p14475203651818">Spark job</p>
</td>
<td class="cellrowborder" valign="top" width="51.81518151815182%" headers="mcps1.3.5.2.1.1.2.4.1.2 "><p id="dli_01_0001__p139869502349">Compute jobs can be submitted through interactive sessions or batch processing. Jobs are submitted to queues created within an elastic resource pool, simplifying resource management and job scheduling.</p>
<p id="dli_01_0001__p18795137193514">It supports multiple data sources and formats, providing rich data processing capabilities, including but not limited to SQL queries and machine learning. For details, see <a href="dli_01_0384.html">Creating a Spark Job</a>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.5.2.1.1.2.4.1.3 "><p id="dli_01_0001__p15365640113613">It is suitable for large-scale data processing and analysis, such as machine learning training, log analysis, and large-scale data mining.</p>
</td>
</tr>
</tbody>
</table>
</div>
</li><li id="dli_01_0001__li13696820133818">Manage program packages of Jar jobs.<p id="dli_01_0001__p1832341621419"><a name="dli_01_0001__li13696820133818"></a><a name="li13696820133818"></a>DLI allows you to submit Flink or Spark jobs compiled as JAR files, which contain the necessary code and dependency information for executing the job. These files are used for specific data processing tasks such as data query, analysis, and machine learning. You can manage program packages required for jobs on the DLI console.</p>
<p id="dli_01_0001__p206967221948">To submit a Spark Jar or Flink Jar job, you must first upload the program package to OBS, create a program package in DLI, and then submit the program package, data, and job parameters to run the job. For details, see <a href="dli_01_0366.html">Managing Program Packages of Jar Jobs</a>.</p>
<div class="note" id="dli_01_0001__note767611358420"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0001__p1167612351346">For Spark 3.3.1 or later and Flink 1.15 or later, when creating a Jar job, you can directly configure the program package in OBS. Program packages cannot be read from DLI.</p>
</div></div>
</li></ul>
</div>
<div class="section" id="dli_01_0001__section6191614351"><h4 class="sectiontitle">Using Cloud Eye to Monitor DLI</h4><p id="dli_01_0001__p18521249179">You can query DLI monitoring metrics and alarms through Cloud Eye management console or APIs.</p>
<p id="dli_01_0001__p4109426491">For example, you can monitor the resource usage and job status of a DLI queue. For details about DLI metrics, see <a href="dli_01_0445.html">Using Cloud Eye to Monitor DLI</a>.</p>
</div>
<div class="section" id="dli_01_0001__section1252619542612"><h4 class="sectiontitle">Using CTS to Audit DLI</h4><p id="dli_01_0001__p133537411474">With CTS, you can log operations related to DLI, making it easier to search, audit, and trace in the future. For the supported operations, see <a href="dli_01_0318.html">Using CTS to Audit DLI</a>.</p>
</div>
</div>

215
docs/dli/umn/dli_01_0002.html
View File

File diff suppressed because it is too large Load Diff

43
docs/dli/umn/dli_01_0003.html
View File

@ -1,18 +1,47 @@
<a name="dli_01_0003"></a><a name="dli_01_0003"></a>
<h1 class="topictitle1">Overview</h1>
<div id="body0000001105195630"><div class="section" id="dli_01_0003__section96073483278"><h4 class="sectiontitle">What Is Enhanced Datasource Connection?</h4><p id="dli_01_0003__p522817397547">In cross-source data analysis scenarios, DLI needs to connect to external data sources. However, due to the different VPCs between the data source and DLI, the network cannot be connected, which results in DLI being unable to read data from the data source. DLI's enhanced datasource connection feature enables network connectivity between DLI and the data source.</p>
<h1 class="topictitle1">Overview of Enhanced Datasource Connections</h1>
<div id="body0000001105195630"><div class="section" id="dli_01_0003__section96073483278"><h4 class="sectiontitle">Why Create Enhanced Datasource Connections?</h4><p id="dli_01_0003__p522817397547">In cross-source data analysis scenarios, DLI needs to connect to external data sources. However, due to the different VPCs between the data source and DLI, the network cannot be connected, which results in DLI being unable to read data from the data source. DLI's enhanced datasource connection feature enables network connectivity between DLI and the data source.</p>
<p id="dli_01_0003__p712043744118">This section will introduce a solution for cross-VPC data source network connectivity:</p>
<ul id="dli_01_0003__ul080613266425"><li id="dli_01_0003__li4806192604220">Creating an enhanced datasource connection: Establish a VPC peering connection to connect DLI and the data source's VPC network.</li><li id="dli_01_0003__li8806226134218">Testing network connectivity: Verify the connectivity between the queue and the data source's network.</li></ul>
<p id="dli_01_0003__p5494121814320">For details about the data sources that support cross-source access, see <a href="dli_01_0410.html">Cross-Source Analysis Development Methods</a>.</p>
<div class="caution" id="dli_01_0003__note132261042151315"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="dli_01_0003__p1822614217130">In cross-source development scenarios, there is a risk of password leakage if datasource authentication information is directly configured. You are advised to use the datasource authentication provided by DLI. For details, see <a href="dli_01_0561.html">Overview</a>.</p>
<p id="dli_01_0003__p5494121814320">For details about the data sources that support cross-source access, see <a href="dli_01_0410.html">Common Development Methods for DLI Cross-Source Analysis</a>.</p>
<div class="caution" id="dli_01_0003__note132261042151315"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="dli_01_0003__p1822614217130">In cross-source development scenarios, there is a risk of password leakage if datasource authentication information is directly configured. You are advised to use Data Encryption Workshop (DEW) to store authentication information of data sources when Spark 3.3.1 or later and Flink 1.15 or later jobs access data sources using datasource connections. This will help you address issues related to data security, key security, and complex key management. For details, see <a href="dli_01_0636.html">Using DEW to Manage Access Credentials for Data Sources</a>.</p>
</div></div>
</div>
<div class="section" id="dli_01_0003__section12469945134416"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0003__ul920917130139"><li id="dli_01_0003__dli_07_0005_li82094136131">Datasource connections cannot be created for the <strong id="dli_01_0003__dli_07_0005_b275994812408">default</strong> queue.</li><li id="dli_01_0003__dli_07_0005_li2599248131320">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li><li id="dli_01_0003__dli_07_0005_li3320184019107"><strong id="dli_01_0003__dli_07_0005_b15858611104116">VPC Administrator</strong> permissions are required for enhanced connections to use VPCs, subnets, routes, VPC peering connections.</li><li id="dli_01_0003__dli_07_0005_li196339467176">If you use an enhanced datasource connection, the CIDR block of the elastic resource pool or queue cannot overlap with that of the data source.</li><li id="dli_01_0003__dli_07_0005_li3209111310134">Only queues bound with datasource connections can access datasource tables.</li><li id="dli_01_0003__dli_07_0005_li1120931310136">Datasource tables do not support the preview function.</li><li id="dli_01_0003__dli_07_0005_li11700174614348">When checking the connectivity of datasource connections, the notes and constraints on IP addresses are:<ul id="dli_01_0003__dli_07_0005_ul1254904218358"><li id="dli_01_0003__dli_07_0005_li15478548367">The IP address must be valid, which consists of four decimal numbers separated by periods (.). The value ranges from 0 to 255.</li><li id="dli_01_0003__dli_07_0005_li151317527373">During the test, you can add a port after the IP address and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0003__dli_07_0005_p26801956133713"><a name="dli_01_0003__dli_07_0005_li151317527373"></a><a name="dli_07_0005_li151317527373"></a>For example, <strong id="dli_01_0003__dli_07_0005_b11869171511417">192.168.</strong><em id="dli_01_0003__dli_07_0005_i1917651964115">xx</em><strong id="dli_01_0003__dli_07_0005_b37501722194120">.</strong><em id="dli_01_0003__dli_07_0005_i151751326134116">xx</em> or <strong id="dli_01_0003__dli_07_0005_b4220103184115">192.168.</strong><em id="dli_01_0003__dli_07_0005_i7940153620413">xx</em><strong id="dli_01_0003__dli_07_0005_b11270124004111">.</strong><em id="dli_01_0003__dli_07_0005_i197261843124114">xx</em><strong id="dli_01_0003__dli_07_0005_b14999194774118">:8181</strong>.</p>
<div class="section" id="dli_01_0003__section12469945134416"><h4 class="sectiontitle">Notes and Constraints</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0003__table1235203484811" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Notes and constraints on enhanced datasource connections</caption><thead align="left"><tr id="dli_01_0003__dli_07_0005_row16110394335"><th align="left" class="cellrowborder" valign="top" width="24.36%" id="mcps1.3.2.2.2.3.1.1"><p id="dli_01_0003__dli_07_0005_p171539103311">Item</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="75.64%" id="mcps1.3.2.2.2.3.1.2"><p id="dli_01_0003__dli_07_0005_p191163973315">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0003__dli_07_0005_row131133933316"><td class="cellrowborder" valign="top" width="24.36%" headers="mcps1.3.2.2.2.3.1.1 "><p id="dli_01_0003__dli_07_0005_p10133917337">Use case</p>
</td>
<td class="cellrowborder" valign="top" width="75.64%" headers="mcps1.3.2.2.2.3.1.2 "><ul id="dli_01_0003__dli_07_0005_ul920917130139"><li id="dli_01_0003__dli_07_0005_li82094136131">Datasource connections cannot be created for the <strong id="dli_01_0003__dli_07_0005_b275994812408">default</strong> queue.</li><li id="dli_01_0003__dli_07_0005_li2599248131320">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li></ul>
</td>
</tr>
<tr id="dli_01_0003__dli_07_0005_row17163973311"><td class="cellrowborder" valign="top" width="24.36%" headers="mcps1.3.2.2.2.3.1.1 "><p id="dli_01_0003__dli_07_0005_p1610391330">Permission</p>
</td>
<td class="cellrowborder" valign="top" width="75.64%" headers="mcps1.3.2.2.2.3.1.2 "><ul id="dli_01_0003__dli_07_0005_ul152971649355"><li id="dli_01_0003__dli_07_0005_li3320184019107"><strong id="dli_01_0003__dli_07_0005_b15858611104116">VPC Administrator</strong> permissions are required for enhanced connections to use VPCs, subnets, routes, VPC peering connections.</li></ul>
</td>
</tr>
<tr id="dli_01_0003__dli_07_0005_row131639133318"><td class="cellrowborder" valign="top" width="24.36%" headers="mcps1.3.2.2.2.3.1.1 "><p id="dli_01_0003__dli_07_0005_p2183903314">Usage</p>
</td>
<td class="cellrowborder" valign="top" width="75.64%" headers="mcps1.3.2.2.2.3.1.2 "><ul id="dli_01_0003__dli_07_0005_ul129393016367"><li id="dli_01_0003__dli_07_0005_li196339467176">If you use an enhanced datasource connection, the CIDR block of the elastic resource pool or queue cannot overlap with that of the data source.</li><li id="dli_01_0003__dli_07_0005_li3209111310134">Only queues bound with datasource connections can access datasource tables.</li><li id="dli_01_0003__dli_07_0005_li1120931310136">Datasource tables do not support the preview function.</li></ul>
</td>
</tr>
<tr id="dli_01_0003__dli_07_0005_row2021939113318"><td class="cellrowborder" valign="top" width="24.36%" headers="mcps1.3.2.2.2.3.1.1 "><p id="dli_01_0003__dli_07_0005_p621639153312">Connectivity check</p>
</td>
<td class="cellrowborder" valign="top" width="75.64%" headers="mcps1.3.2.2.2.3.1.2 "><ul id="dli_01_0003__dli_07_0005_ul827071003617"><li id="dli_01_0003__dli_07_0005_li11700174614348">When checking the connectivity of datasource connections, the notes and constraints on IP addresses are:<ul id="dli_01_0003__dli_07_0005_ul1254904218358"><li id="dli_01_0003__dli_07_0005_li15478548367">The IP address must be valid, which consists of four decimal numbers separated by periods (.). The value ranges from 0 to 255.</li><li id="dli_01_0003__dli_07_0005_li151317527373">During the test, you can add a port after the IP address and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0003__dli_07_0005_p26801956133713"><a name="dli_01_0003__dli_07_0005_li151317527373"></a><a name="dli_07_0005_li151317527373"></a>For example, <strong id="dli_01_0003__dli_07_0005_b11869171511417">192.168.</strong><em id="dli_01_0003__dli_07_0005_i1917651964115">xx</em><strong id="dli_01_0003__dli_07_0005_b37501722194120">.</strong><em id="dli_01_0003__dli_07_0005_i151751326134116">xx</em> or <strong id="dli_01_0003__dli_07_0005_b4220103184115">192.168.</strong><em id="dli_01_0003__dli_07_0005_i7940153620413">xx</em><strong id="dli_01_0003__dli_07_0005_b11270124004111">.</strong><em id="dli_01_0003__dli_07_0005_i197261843124114">xx</em><strong id="dli_01_0003__dli_07_0005_b14999194774118">:8181</strong>.</p>
</li></ul>
</li><li id="dli_01_0003__dli_07_0005_li1152733115719">When checking the connectivity of datasource connections, the notes and constraints on domain names are:<ul id="dli_01_0003__dli_07_0005_ul166789287574"><li id="dli_01_0003__dli_07_0005_li46789287572">The domain name can contain 1 to 255 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed.</li><li id="dli_01_0003__dli_07_0005_li667882815577">The top-level domain name must contain at least two letters, for example, <strong id="dli_01_0003__dli_07_0005_b11901337154319">.com</strong>, <strong id="dli_01_0003__dli_07_0005_b1028814124319">.net</strong>, and <strong id="dli_01_0003__dli_07_0005_b19839144614436">.cn</strong>.</li><li id="dli_01_0003__dli_07_0005_li1867852835715">During the test, you can add a port after the domain name and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0003__dli_07_0005_p1667882814575"><a name="dli_01_0003__dli_07_0005_li1867852835715"></a><a name="dli_07_0005_li1867852835715"></a>For example, <strong id="dli_01_0003__dli_07_0005_b177998236451">example.com:8080</strong>.</p>
</li><li id="dli_01_0003__dli_07_0005_li1327071015364">When checking the connectivity of datasource connections, the notes and constraints on domain names are:<ul id="dli_01_0003__dli_07_0005_ul7270201020363"><li id="dli_01_0003__dli_07_0005_li46789287572">The domain name can contain 1 to 255 characters. Only letters, numbers, underscores (_), and hyphens (-) are allowed.</li><li id="dli_01_0003__dli_07_0005_li667882815577">The top-level domain name must contain at least two letters, for example, <strong id="dli_01_0003__dli_07_0005_b11901337154319">.com</strong>, <strong id="dli_01_0003__dli_07_0005_b1028814124319">.net</strong>, and <strong id="dli_01_0003__dli_07_0005_b19839144614436">.cn</strong>.</li><li id="dli_01_0003__dli_07_0005_li627041023619">During the test, you can add a port after the domain name and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0003__dli_07_0005_p172706106361"><a name="dli_01_0003__dli_07_0005_li627041023619"></a><a name="dli_07_0005_li627041023619"></a>Example: <strong id="dli_01_0003__dli_07_0005_b10341174121215">example.com:8080</strong></p>
</li></ul>
</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_01_0003__section12566111415470"><h4 class="sectiontitle">Cross-Source Analysis Process</h4><p id="dli_01_0003__p239162984817">To use DLI for cross-source analysis, you need to create a datasource connection to connect DLI to the data source, and then develop jobs to access the data source.</p>
<div class="fignone" id="dli_01_0003__fig1175617348416"><span class="figcap"><b>Figure 1 </b>Cross-source analysis flowchart</span><br><span><img id="dli_01_0003__image20990341165118" src="en-us_image_0000001570712116.png"></span></div>
@ -20,7 +49,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Enhanced Datasource Connections</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Configuring the Network Connection Between DLI and Data Sources (Enhanced Datasource Connection)</a></div>
</div>
</div>

14
docs/dli/umn/dli_01_0004.html
View File

@ -1,13 +1,21 @@
<a name="dli_01_0004"></a><a name="dli_01_0004"></a>
<h1 class="topictitle1">Data Management</h1>
<h1 class="topictitle1">Managing Database Resources on the DLI Console</h1>
<div id="body1501038213757"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0390.html">Databases and Tables</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0447.html">Configuring Database Permissions on the DLI Console</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0366.html">Package Management</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0011.html">Deleting a Database on the DLI Console</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0376.html">Changing the Database Owner on the DLI Console</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0552.html">Managing Tags</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0390.html">Creating Databases and Tables</a></div>
</div>
</div>

147
docs/dli/umn/dli_01_0005.html
View File

File diff suppressed because it is too large Load Diff

49
docs/dli/umn/dli_01_0006.html
View File

@ -5,19 +5,48 @@
<p id="dli_01_0006__p25621621205213">For example, to connect DLI to the MRS, RDS, CSS, Kafka, or GaussDB(DWS) data source, you need to enable the network between DLI and the VPC of the data source.</p>
<p id="dli_01_0006__p163611244144318">Create an enhanced datasource connection on the console.</p>
</div>
<div class="section" id="dli_01_0006__section1497781674410"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0006__dli_01_0003_ul920917130139"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li82094136131">Datasource connections cannot be created for the <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b275994812408">default</strong> queue.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li2599248131320">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li3320184019107"><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b15858611104116">VPC Administrator</strong> permissions are required for enhanced connections to use VPCs, subnets, routes, VPC peering connections.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li196339467176">If you use an enhanced datasource connection, the CIDR block of the elastic resource pool or queue cannot overlap with that of the data source.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li3209111310134">Only queues bound with datasource connections can access datasource tables.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1120931310136">Datasource tables do not support the preview function.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li11700174614348">When checking the connectivity of datasource connections, the notes and constraints on IP addresses are:<ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul1254904218358"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li15478548367">The IP address must be valid, which consists of four decimal numbers separated by periods (.). The value ranges from 0 to 255.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li151317527373">During the test, you can add a port after the IP address and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0006__dli_01_0003_dli_07_0005_p26801956133713"><a name="dli_01_0006__dli_01_0003_dli_07_0005_li151317527373"></a><a name="dli_01_0003_dli_07_0005_li151317527373"></a>For example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11869171511417">192.168.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i1917651964115">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b37501722194120">.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i151751326134116">xx</em> or <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b4220103184115">192.168.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i7940153620413">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11270124004111">.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i197261843124114">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b14999194774118">:8181</strong>.</p>
<div class="section" id="dli_01_0006__section1497781674410"><h4 class="sectiontitle">Notes and Constraints</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0006__dli_01_0003_table1235203484811" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Notes and constraints on enhanced datasource connections</caption><thead align="left"><tr id="dli_01_0006__dli_01_0003_dli_07_0005_row16110394335"><th align="left" class="cellrowborder" valign="top" width="24.36%" id="mcps1.3.2.2.2.3.1.1"><p id="dli_01_0006__dli_01_0003_dli_07_0005_p171539103311">Item</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="75.64%" id="mcps1.3.2.2.2.3.1.2"><p id="dli_01_0006__dli_01_0003_dli_07_0005_p191163973315">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0006__dli_01_0003_dli_07_0005_row131133933316"><td class="cellrowborder" valign="top" width="24.36%" headers="mcps1.3.2.2.2.3.1.1 "><p id="dli_01_0006__dli_01_0003_dli_07_0005_p10133917337">Use case</p>
</td>
<td class="cellrowborder" valign="top" width="75.64%" headers="mcps1.3.2.2.2.3.1.2 "><ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul920917130139"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li82094136131">Datasource connections cannot be created for the <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b275994812408">default</strong> queue.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li2599248131320">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li></ul>
</td>
</tr>
<tr id="dli_01_0006__dli_01_0003_dli_07_0005_row17163973311"><td class="cellrowborder" valign="top" width="24.36%" headers="mcps1.3.2.2.2.3.1.1 "><p id="dli_01_0006__dli_01_0003_dli_07_0005_p1610391330">Permission</p>
</td>
<td class="cellrowborder" valign="top" width="75.64%" headers="mcps1.3.2.2.2.3.1.2 "><ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul152971649355"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li3320184019107"><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b15858611104116">VPC Administrator</strong> permissions are required for enhanced connections to use VPCs, subnets, routes, VPC peering connections.</li></ul>
</td>
</tr>
<tr id="dli_01_0006__dli_01_0003_dli_07_0005_row131639133318"><td class="cellrowborder" valign="top" width="24.36%" headers="mcps1.3.2.2.2.3.1.1 "><p id="dli_01_0006__dli_01_0003_dli_07_0005_p2183903314">Usage</p>
</td>
<td class="cellrowborder" valign="top" width="75.64%" headers="mcps1.3.2.2.2.3.1.2 "><ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul129393016367"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li196339467176">If you use an enhanced datasource connection, the CIDR block of the elastic resource pool or queue cannot overlap with that of the data source.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li3209111310134">Only queues bound with datasource connections can access datasource tables.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1120931310136">Datasource tables do not support the preview function.</li></ul>
</td>
</tr>
<tr id="dli_01_0006__dli_01_0003_dli_07_0005_row2021939113318"><td class="cellrowborder" valign="top" width="24.36%" headers="mcps1.3.2.2.2.3.1.1 "><p id="dli_01_0006__dli_01_0003_dli_07_0005_p621639153312">Connectivity check</p>
</td>
<td class="cellrowborder" valign="top" width="75.64%" headers="mcps1.3.2.2.2.3.1.2 "><ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul827071003617"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li11700174614348">When checking the connectivity of datasource connections, the notes and constraints on IP addresses are:<ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul1254904218358"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li15478548367">The IP address must be valid, which consists of four decimal numbers separated by periods (.). The value ranges from 0 to 255.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li151317527373">During the test, you can add a port after the IP address and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0006__dli_01_0003_dli_07_0005_p26801956133713"><a name="dli_01_0006__dli_01_0003_dli_07_0005_li151317527373"></a><a name="dli_01_0003_dli_07_0005_li151317527373"></a>For example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11869171511417">192.168.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i1917651964115">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b37501722194120">.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i151751326134116">xx</em> or <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b4220103184115">192.168.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i7940153620413">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11270124004111">.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i197261843124114">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b14999194774118">:8181</strong>.</p>
</li></ul>
</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1152733115719">When checking the connectivity of datasource connections, the notes and constraints on domain names are:<ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul166789287574"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li46789287572">The domain name can contain 1 to 255 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li667882815577">The top-level domain name must contain at least two letters, for example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11901337154319">.com</strong>, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b1028814124319">.net</strong>, and <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b19839144614436">.cn</strong>.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1867852835715">During the test, you can add a port after the domain name and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0006__dli_01_0003_dli_07_0005_p1667882814575"><a name="dli_01_0006__dli_01_0003_dli_07_0005_li1867852835715"></a><a name="dli_01_0003_dli_07_0005_li1867852835715"></a>For example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b177998236451">example.com:8080</strong>.</p>
</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1327071015364">When checking the connectivity of datasource connections, the notes and constraints on domain names are:<ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul7270201020363"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li46789287572">The domain name can contain 1 to 255 characters. Only letters, numbers, underscores (_), and hyphens (-) are allowed.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li667882815577">The top-level domain name must contain at least two letters, for example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11901337154319">.com</strong>, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b1028814124319">.net</strong>, and <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b19839144614436">.cn</strong>.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li627041023619">During the test, you can add a port after the domain name and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0006__dli_01_0003_dli_07_0005_p172706106361"><a name="dli_01_0006__dli_01_0003_dli_07_0005_li627041023619"></a><a name="dli_01_0003_dli_07_0005_li627041023619"></a>Example: <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b10341174121215">example.com:8080</strong></p>
</li></ul>
</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_01_0006__section192487124322"><h4 class="sectiontitle">Process</h4><div class="fignone" id="dli_01_0006__fig5997444134911"><span class="figcap"><b>Figure 1 </b>Enhanced datasource connection creation flowchart</span><br><span><img id="dli_01_0006__image479310122019" src="en-us_image_0000001620789569.png"></span></div>
</div>
<div class="section" id="dli_01_0006__section2087512244812"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_01_0006__ul13774121195114"><li id="dli_01_0006__li18667173210567">An elastic resource pool or queue has been created.</li><li id="dli_01_0006__li17253192520517">You have obtained the VPC, subnet, private IP address, port, and security group information of the external data source.</li><li id="dli_01_0006__li15340358125415">The security group of the external data source has allowed access from the CIDR block of the elastic resource pool or queue.</li></ul>
</div>
<div class="section" id="dli_01_0006__section55221617125115"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0006__ol145341443135715"><li id="dli_01_0006__li125341743145718"><span><strong id="dli_01_0006__b9922185765711">Create an Enhanced Datasource Connection</strong></span><p><ol type="a" id="dli_01_0006__ol194951321175112"><li id="dli_01_0006__li56433424917">Log in to the DLI management console.</li><li id="dli_01_0006__li16495421185119">In the left navigation pane, choose <strong id="dli_01_0006__b97721728103818">Datasource Connections</strong>.</li><li id="dli_01_0006__li2495142110511">On the <strong id="dli_01_0006__b372845314391">Enhanced</strong> tab page displayed, click <strong id="dli_01_0006__b855155893910">Create</strong>.<p id="dli_01_0006__p948710436532">Configure parameters according to <a href="#dli_01_0006__table9495521165111">Table 1</a>.</p>
<div class="section" id="dli_01_0006__section55221617125115"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0006__ol145341443135715"><li id="dli_01_0006__li125341743145718"><span><strong id="dli_01_0006__b9922185765711">Create an Enhanced Datasource Connection</strong></span><p><ol type="a" id="dli_01_0006__ol194951321175112"><li id="dli_01_0006__li56433424917">Log in to the DLI management console.</li><li id="dli_01_0006__li16495421185119">In the navigation pane on the left, choose <strong id="dli_01_0006__b17637134113407">Datasource Connections</strong>.</li><li id="dli_01_0006__li2495142110511">On the displayed <strong id="dli_01_0006__b5334135871419">Enhanced</strong> tab, click <strong id="dli_01_0006__b11335155831417">Create</strong>.<p id="dli_01_0006__p948710436532">Set parameters based on <a href="#dli_01_0006__table9495521165111">Table 2</a>.</p>
<div class="tablenoborder"><a name="dli_01_0006__table9495521165111"></a><a name="table9495521165111"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0006__table9495521165111" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="dli_01_0006__row649562118511"><th align="left" class="cellrowborder" valign="top" width="21.22%" id="mcps1.3.5.2.1.2.1.3.4.2.3.1.1"><p id="dli_01_0006__p249592118517">Parameter</p>
<div class="tablenoborder"><a name="dli_01_0006__table9495521165111"></a><a name="table9495521165111"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0006__table9495521165111" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Parameters</caption><thead align="left"><tr id="dli_01_0006__row649562118511"><th align="left" class="cellrowborder" valign="top" width="21.22%" id="mcps1.3.5.2.1.2.1.3.4.2.3.1.1"><p id="dli_01_0006__p249592118517">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="78.78%" id="mcps1.3.5.2.1.2.1.3.4.2.3.1.2"><p id="dli_01_0006__p2495122112514">Description</p>
</th>
@ -26,7 +55,7 @@
<tbody><tr id="dli_01_0006__row14496152114519"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.5.2.1.2.1.3.4.2.3.1.1 "><p id="dli_01_0006__p3496102175119">Connection Name</p>
</td>
<td class="cellrowborder" valign="top" width="78.78%" headers="mcps1.3.5.2.1.2.1.3.4.2.3.1.2 "><p id="dli_01_0006__p1249620215510">Name of the created datasource connection.</p>
<ul id="dli_01_0006__ul94960217518"><li id="dli_01_0006__li104961821135115">The name can contain only letters, digits, and underscores (_). The parameter must be specified.</li><li id="dli_01_0006__li3496421165117">A maximum of 64 characters are allowed.</li></ul>
<ul id="dli_01_0006__ul94960217518"><li id="dli_01_0006__li104961821135115">Only letters, numbers, and underscores (_) are allowed. The parameter must be specified.</li><li id="dli_01_0006__li3496421165117">A maximum of 64 characters are allowed.</li></ul>
</td>
</tr>
<tr id="dli_01_0006__row137713913469"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.5.2.1.2.1.3.4.2.3.1.1 "><p id="dli_01_0006__p113462159464">Resource Pool</p>
@ -62,10 +91,10 @@
<td class="cellrowborder" valign="top" width="78.78%" headers="mcps1.3.5.2.1.2.1.3.4.2.3.1.2 "><p id="dli_01_0006__p52841736202320">Tags used to identify cloud resources. A tag includes the tag key and tag value. If you want to use the same tag to identify multiple cloud resources, that is, to select the same tag from the drop-down list box for all services, you are advised to create predefined tags on the Tag Management Service (TMS).</p>
<div class="note" id="dli_01_0006__note647119469162"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0006__dli_01_0363_ul1830018521911"><li id="dli_01_0006__dli_01_0363_li186815313365">A maximum of 20 tags can be added.</li><li id="dli_01_0006__dli_01_0363_li183008521591">Only one tag value can be added to a tag key.</li><li id="dli_01_0006__dli_01_0363_li14484164661614">The key name in each resource must be unique.</li></ul>
</div></div>
<ul id="dli_01_0006__ul6617123554815"><li id="dli_01_0006__li9617123518481">Tag key: Enter a tag key name in the text box.<div class="note" id="dli_01_0006__note18617135124816"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0006__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0006__dli_01_0363_b544594615499">_sys_</strong>.</p>
<ul id="dli_01_0006__ul6617123554815"><li id="dli_01_0006__li9617123518481">Tag key: Enter a tag key name in the text box.<div class="note" id="dli_01_0006__note18617135124816"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0006__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0006__dli_01_0363_b544594615499">_sys_</strong>.</p>
</div></div>
</li></ul>
<ul id="dli_01_0006__ul16911451899"><li id="dli_01_0006__li26911851193">Tag value: Enter a tag value in the text box.<div class="note" id="dli_01_0006__note86179355480"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0006__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed. The value cannot start or end with a space.</p>
<ul id="dli_01_0006__ul16911451899"><li id="dli_01_0006__li26911851193">Tag value: Enter a tag value in the text box.<div class="note" id="dli_01_0006__note86179355480"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0006__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed.</p>
</div></div>
</li></ul>
</td>
@ -76,9 +105,9 @@
</li><li id="dli_01_0006__li849810219511">Click <span class="uicontrol" id="dli_01_0006__uicontrol1549872119517"><b>OK</b></span>.<p id="dli_01_0006__p166041546111114">After the creation is complete, the enhanced datasource connection is in the <strong id="dli_01_0006__b862271723011">Active</strong> state, indicating that the connection is successfully created.</p>
</li></ol>
</p></li><li id="dli_01_0006__li1079044885818"><span><strong id="dli_01_0006__b164443125590">Security Group Where the Data Source Belongs Allows Access from the CIDR Block of the Elastic Resource Pool</strong></span><p><ol type="a" id="dli_01_0006__ol6775618125918"><li id="dli_01_0006__li10521202411010">On the DLI management console, obtain the network segment of the elastic resource pool or queue.<p id="dli_01_0006__p12851630504"><a name="dli_01_0006__li10521202411010"></a><a name="li10521202411010"></a>Choose <strong id="dli_01_0006__b46924239339">Resources</strong> &gt; <strong id="dli_01_0006__b9185162413344">Queue Management</strong> from the left navigation pane. On the page displayed, locate the queue on which jobs are running, and click the button next to the queue name to obtain the CIDR block of the queue.</p>
</li><li id="dli_01_0006__li53111030115020">Log in to the VPC console and find the VPC the data source belongs to.</li><li id="dli_01_0006__li28840256810">On the network console, choose <strong id="dli_01_0006__b19300141712614">Virtual Private Cloud</strong> &gt; <strong id="dli_01_0006__b157649201769">Network Interfaces</strong>. On the <strong id="dli_01_0006__b1779719461369">Network Interfaces</strong> tab page displayed, search for the security group name, click <strong id="dli_01_0006__b149016389713">More</strong> in the <strong id="dli_01_0006__b73843356719">Operation</strong> column, and select <strong id="dli_01_0006__b7445155516718">Change Security Group</strong>.</li><li id="dli_01_0006__li08589549503">In the navigation pane on the left, choose <strong id="dli_01_0006__b129421258105016">Access Control</strong> &gt; <strong id="dli_01_0006__b189441358145018">Security Groups</strong>.</li><li id="dli_01_0006__li0824167115110">Click the name of the security group to which the external data source belongs.</li><li id="dli_01_0006__li1859194042211">Click the <strong id="dli_01_0006__b76315488519">Inbound Rules</strong> tab and add a rule to allow access from the CIDR block of the queue.<p id="dli_01_0006__p515115815710">Configure the inbound rule parameters according to <a href="#dli_01_0006__table4276105765618">Table 2</a>.</p>
</li><li id="dli_01_0006__li53111030115020">Log in to the VPC console and find the VPC the data source belongs to.</li><li id="dli_01_0006__li28840256810">On the network console, choose <strong id="dli_01_0006__b19300141712614">Virtual Private Cloud</strong> &gt; <strong id="dli_01_0006__b157649201769">Network Interfaces</strong>. On the <strong id="dli_01_0006__b1779719461369">Network Interfaces</strong> tab page displayed, search for the security group name, click <strong id="dli_01_0006__b149016389713">More</strong> in the <strong id="dli_01_0006__b73843356719">Operation</strong> column, and select <strong id="dli_01_0006__b7445155516718">Change Security Group</strong>.</li><li id="dli_01_0006__li08589549503">In the navigation pane on the left, choose <strong id="dli_01_0006__b129421258105016">Access Control</strong> &gt; <strong id="dli_01_0006__b189441358145018">Security Groups</strong>.</li><li id="dli_01_0006__li0824167115110">Click the name of the security group to which the external data source belongs.</li><li id="dli_01_0006__li1859194042211">Click the <strong id="dli_01_0006__b76315488519">Inbound Rules</strong> tab and add a rule to allow access from the CIDR block of the queue.<p id="dli_01_0006__p515115815710">Configure the inbound rule parameters according to <a href="#dli_01_0006__table4276105765618">Table 3</a>.</p>
<div class="tablenoborder"><a name="dli_01_0006__table4276105765618"></a><a name="table4276105765618"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0006__table4276105765618" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Inbound rule parameters</caption><thead align="left"><tr id="dli_01_0006__row172755576567"><th align="left" class="cellrowborder" valign="top" width="19.881988198819883%" id="mcps1.3.5.2.2.2.1.6.3.2.4.1.1"><p id="dli_01_0006__p142751957115617">Parameter</p>
<div class="tablenoborder"><a name="dli_01_0006__table4276105765618"></a><a name="table4276105765618"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0006__table4276105765618" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Inbound rule parameters</caption><thead align="left"><tr id="dli_01_0006__row172755576567"><th align="left" class="cellrowborder" valign="top" width="19.881988198819883%" id="mcps1.3.5.2.2.2.1.6.3.2.4.1.1"><p id="dli_01_0006__p142751957115617">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="36.42364236423642%" id="mcps1.3.5.2.2.2.1.6.3.2.4.1.2"><p id="dli_01_0006__p162751957115613">Description</p>
</th>
@ -143,7 +172,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Enhanced Datasource Connections</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Configuring the Network Connection Between DLI and Data Sources (Enhanced Datasource Connection)</a></div>
</div>
</div>

4
docs/dli/umn/dli_01_0007.html
View File

@ -1,6 +1,6 @@
<a name="dli_01_0007"></a><a name="dli_01_0007"></a>
<h1 class="topictitle1">Previewing Data</h1>
<h1 class="topictitle1">Previewing Table Data on the DLI Console</h1>
<div id="body1501038213757"><p id="dli_01_0007__p569436192936">The <span class="wintitle" id="dli_01_0007__wintitle159646911143"><b>Preview</b></span> page displays the first 10 records in the table.</p>
<div class="section" id="dli_01_0007__section220516431119"><h4 class="sectiontitle">Procedure</h4><div class="p" id="dli_01_0007__p39191032115617">You can preview data on either the <span class="wintitle" id="dli_01_0007__wintitle202728967716401"><b>Data Management</b></span> page or the <span class="wintitle" id="dli_01_0007__wintitle1059923363164013"><b>SQL Editor</b></span> page.<ul id="dli_01_0007__ul57312886162410"><li id="dli_01_0007__li46053929162410">To preview data on the <span class="wintitle" id="dli_01_0007__wintitle4564762181553"><b>Data Management</b></span> page:<ol id="dli_01_0007__ol48633399145054"><li id="dli_01_0007__li35417823145054">On the left of the management console, choose <span class="uicontrol" id="dli_01_0007__uicontrol18482320394"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0007__uicontrol748212014913"><b>Databases and Tables</b></span>.</li><li id="dli_01_0007__li22126743145247">On the displayed <span class="wintitle" id="dli_01_0007__wintitle108175242144"><b>Data Management</b></span> page, click the name of the database where the target table whose data you want to export resides to switch to the <strong id="dli_01_0007__b38223243141">Manage Tables</strong> page.</li><li id="dli_01_0007__li36387870145557">Click <span class="uicontrol" id="dli_01_0007__uicontrol52771577102"><b>More</b></span> in the <strong id="dli_01_0007__b3277187171011">Operation</strong> column of the target table and select <span class="wintitle" id="dli_01_0007__wintitle1827707101011"><b>View Properties</b></span>.</li><li id="dli_01_0007__li86216531552">Click the <span class="wintitle" id="dli_01_0007__wintitle3454141011716"><b>Preview</b></span> tab to preview the table data.</li></ol>
</li><li id="dli_01_0007__li11072851162410">To preview data on the <span class="wintitle" id="dli_01_0007__wintitle1606590135165610"><b>SQL Editor</b></span> page:<ol id="dli_01_0007__ol24485723162410"><li id="dli_01_0007__li19044916162410">On the left of the management console, click <span class="uicontrol" id="dli_01_0007__uicontrol666915283918"><b>SQL Editor</b></span>.</li><li id="dli_01_0007__li16991452612">In the navigation pane of the displayed <span class="wintitle" id="dli_01_0007__wintitle146852941019"><b>SQL Editor</b></span> page, click <strong id="dli_01_0007__b0685209161016">Databases</strong>.</li><li id="dli_01_0007__li66243256162410">Click the corresponding database name to view the tables in the database.</li><li id="dli_01_0007__li64103520162410">Click <span><img id="dli_01_0007__image14366125210269" src="en-us_image_0237994908.png"></span> on the right of the corresponding table, choose <strong id="dli_01_0007__b5366852132610">View Properties</strong> from the list menu, and click the <strong id="dli_01_0007__b73671852102615">Preview</strong> tab to preview the data of the table.</li></ol>
@ -10,7 +10,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0390.html">Databases and Tables</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0625.html">Managing Table Resources on the DLI Console</a></div>
</div>
</div>

4
docs/dli/umn/dli_01_0008.html
View File

@ -1,6 +1,6 @@
<a name="dli_01_0008"></a><a name="dli_01_0008"></a>
<h1 class="topictitle1">Viewing Metadata</h1>
<h1 class="topictitle1">Viewing Table Metadata</h1>
<div id="body1501038213757"><div class="section" id="dli_01_0008__section35142993104733"><h4 class="sectiontitle">Metadata Description</h4><ul id="dli_01_0008__ul3266683161032"><li id="dli_01_0008__li54899781161032">Metadata is used to define data types. It describes information about the data, including the source, size, format, and other data features. In database fields, metadata interprets data content in the data warehouse.</li><li id="dli_01_0008__li57091439161034">When you create a table, metadata is defined, consisting of the column name, type, and description.</li><li id="dli_01_0008__li194120268212">The <span class="wintitle" id="dli_01_0008__wintitle4795312310"><b>Metadata</b></span> page displays information about the target table, including <strong id="dli_01_0008__b98013311133">Column Name</strong>, <strong id="dli_01_0008__b184461231103011">Column Type</strong>, <strong id="dli_01_0008__b6809313317">Data Type</strong>, and <strong id="dli_01_0008__b1801231632">Description</strong>.</li></ul>
</div>
<div class="section" id="dli_01_0008__section38089543314"><h4 class="sectiontitle">Procedure</h4><div class="p" id="dli_01_0008__p11776437345">You can view metadata on either the <span class="wintitle" id="dli_01_0008__wintitle202728967716401"><b>Data Management</b></span> page or the <span class="wintitle" id="dli_01_0008__wintitle1059923363164013"><b>SQL Editor</b></span> page.<ul id="dli_01_0008__ul41981589145011"><li id="dli_01_0008__li30083560145011">To view metadata on the <span class="wintitle" id="dli_01_0008__wintitle4564762181553"><b>Data Management</b></span> page:<ol id="dli_01_0008__ol48633399145054"><li id="dli_01_0008__li35417823145054">On the left of the management console, choose <span class="uicontrol" id="dli_01_0008__uicontrol165882511882"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0008__uicontrol1058910511686"><b>Databases and Tables</b></span>.</li><li id="dli_01_0008__li22126743145247">On the displayed <span class="wintitle" id="dli_01_0008__wintitle6486209315148"><b>Data Management</b></span> page, click the name of the database where the target table whose data you want to export resides to switch to the <strong id="dli_01_0008__b116953258919">Manage Tables</strong> page.</li><li id="dli_01_0008__li36387870145557">Click <span class="uicontrol" id="dli_01_0008__uicontrol516817179511"><b>More</b></span> in the <strong id="dli_01_0008__b1176621554">Operation</strong> column of the target table and select <strong id="dli_01_0008__b12853104513520">View Properties</strong>. In the <strong id="dli_01_0008__b134534533517">Metadata</strong> tab, view the metadata of the table.</li></ol>
@ -11,7 +11,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0390.html">Databases and Tables</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0390.html">Creating Databases and Tables</a></div>
</div>
</div>

4
docs/dli/umn/dli_01_0010.html
View File

@ -1,6 +1,6 @@
<a name="dli_01_0010"></a><a name="dli_01_0010"></a>
<h1 class="topictitle1">Exporting Data from DLI to OBS</h1>
<h1 class="topictitle1">Exporting DLI Table Data to OBS</h1>
<div id="body1501291906720"><p id="dli_01_0010__p12368142105715">You can export data from a DLI table to OBS. During the export, a folder is created in OBS or the content in the existing folder is overwritten.</p>
<div class="section" id="dli_01_0010__section181111466568"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0010__ul12225587299"><li id="dli_01_0010__li0222105872918">The exported file can be in JSON format, and the text format can only be UTF-8.</li><li id="dli_01_0010__li6222205822916">Only the data in the DLI table (the table type is <strong id="dli_01_0010__b17552458537">Managed</strong>) can be exported to the OBS bucket, and the export path must be specified to the folder level.</li><li id="dli_01_0010__li675153163611">Data can be exported across accounts. That is, after account B authorizes account A, account A can export data to the OBS path of account B if account A has the permission to read the metadata and permission information about the OBS bucket of account B and read and write the path.</li></ul>
</div>
@ -66,7 +66,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0390.html">Databases and Tables</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0625.html">Managing Table Resources on the DLI Console</a></div>
</div>
</div>

13
docs/dli/umn/dli_01_0011.html
View File

@ -1,16 +1,11 @@
<a name="dli_01_0011"></a><a name="dli_01_0011"></a>
<h1 class="topictitle1">Deleting a Database or a Table</h1>
<div id="body1501038213757"><p id="dli_01_0011__p72900011460">You can delete unnecessary databases and tables based on actual conditions.</p>
<div class="section" id="dli_01_0011__section591721184417"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0011__ul9368914114615"><li id="dli_01_0011__li8641339162958">You are not allowed to delete databases or tables that are being used for running jobs.</li><li id="dli_01_0011__li55947051162958">The administrator, database owner, and users with the database deletion permission can delete the database. The administrator, database owner, and users with the table deletion permission can delete the table. <div class="note" id="dli_01_0011__note2036861411465"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0011__p236921415463">If a database or table is deleted, it cannot be recovered. Exercise caution when performing this operation.</p>
<h1 class="topictitle1">Deleting a Database on the DLI Console</h1>
<div id="body1501038213757"><p id="dli_01_0011__p72900011460">You can delete databases as needed.</p>
<div class="section" id="dli_01_0011__section591721184417"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0011__ul9368914114615"><li id="dli_01_0011__li8641339162958">You are not allowed to delete databases or tables that are being used for running jobs.</li><li id="dli_01_0011__li55947051162958">The administrator, database owner, and users with the database deletion permission can delete the database.<div class="note" id="dli_01_0011__note2036861411465"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0011__p236921415463">If a database or table is deleted, it cannot be recovered. Exercise caution when performing this operation.</p>
</div></div>
</li></ul>
</div>
<div class="section" id="dli_01_0011__section4685189174137"><h4 class="sectiontitle">Deleting a Table</h4><div class="p" id="dli_01_0011__p17554192016499">You can delete a table on either the <span class="wintitle" id="dli_01_0011__wintitle1351164893315"><b>Data Management</b></span> page or the <span class="wintitle" id="dli_01_0011__wintitle035294843313"><b>SQL Editor</b></span> page.<ul id="dli_01_0011__ul10405142364818"><li id="dli_01_0011__li2040592311485">Delete the table on the <span class="wintitle" id="dli_01_0011__wintitle73872501337"><b>Data Management</b></span> page.<ol id="dli_01_0011__ol1735193915485"><li id="dli_01_0011__li2073593924816">On the left of the management console, choose <span class="uicontrol" id="dli_01_0011__uicontrol348919541445"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0011__uicontrol14490145410416"><b>Databases and Tables</b></span>.</li><li id="dli_01_0011__li33015647174137">Locate the row where the database whose tables you want to delete, click the database name to switch to the <strong id="dli_01_0011__b9318135618332">Table Management</strong> page.</li><li id="dli_01_0011__li57021755174137">Locate the row where the target table locates and click <strong id="dli_01_0011__b1751314565019">More</strong> &gt; <span class="uicontrol" id="dli_01_0011__uicontrol43433751174137"><b>Delete</b></span> in the <strong id="dli_01_0011__b1080714149149">Operation</strong> column.</li><li id="dli_01_0011__li55359439174137">In the displayed dialog box, click <span class="uicontrol" id="dli_01_0011__uicontrol27840389155213"><b>Yes</b></span>.</li></ol>
</li><li id="dli_01_0011__li10384371151118">Delete a table on the <span class="wintitle" id="dli_01_0011__wintitle743355813315"><b>SQL Editor</b></span> page.<ol id="dli_01_0011__ol39382464151459"><li id="dli_01_0011__li26745450151459">On the top menu bar of the DLI management console, click <span class="uicontrol" id="dli_01_0011__uicontrol136221153121411"><b>SQL Editor</b></span>.</li><li id="dli_01_0011__li1773513617043">In the navigation tree on the left, click <strong id="dli_01_0011__b1777018312279">Databases</strong>. Click the name of a database where the table you want belongs. The tables of the selected database are displayed.</li><li id="dli_01_0011__li2614201531">Click <span><img id="dli_01_0011__image87961316122313" src="en-us_image_0237983632.png"></span> on the right of the table and choose <strong id="dli_01_0011__b379617166234">Delete</strong> from the shortcut menu.</li><li id="dli_01_0011__li0998173818546">In the dialog box that is displayed, click <span class="uicontrol" id="dli_01_0011__en-us_topic_0122016948_en-us_topic_0093946752_uicontrol44850627112429"><b>OK</b></span>.</li></ol>
</li></ul>
</div>
</div>
<div class="section" id="dli_01_0011__section168648524611"><h4 class="sectiontitle">Deleting a Database</h4><ol id="dli_01_0011__ol547114061117"><li id="dli_01_0011__li447124091115">On the left of the management console, choose <span class="uicontrol" id="dli_01_0011__uicontrol146221132504"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0011__uicontrol1162281395018"><b>Databases and Tables</b></span>.</li><li id="dli_01_0011__li1148194015113">Locate the row where the target database locates and click <strong id="dli_01_0011__b7893516155013">More</strong> &gt; <span class="uicontrol" id="dli_01_0011__uicontrol189361635014"><b>Drop Database</b></span> in the <strong id="dli_01_0011__b189419168503">Operation</strong> column.<div class="note" id="dli_01_0011__note16416124311218"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0011__p24172043172113">You cannot delete databases that contain tables. To delete a database containing tables, delete the tables first.</p>
</div></div>
</li><li id="dli_01_0011__li13481408117">In the displayed dialog box, click <span class="uicontrol" id="dli_01_0011__uicontrol153561931162917"><b>Yes</b></span>.</li></ol>
@ -18,7 +13,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0390.html">Databases and Tables</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0004.html">Managing Database Resources on the DLI Console</a></div>
</div>
</div>

33
docs/dli/umn/dli_01_0012.html
View File

@ -1,33 +0,0 @@
<a name="dli_01_0012"></a><a name="dli_01_0012"></a>
<h1 class="topictitle1">Queue Management</h1>
<div id="body1501038213757"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0402.html">Overview</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0015.html">Queue Permission Management</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0363.html">Creating a Queue</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0016.html">Deleting a Queue</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0565.html">Allocating a Queue to an Enterprise Project</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0443.html">Modifying the CIDR Block</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0487.html">Elastic Scaling of Queues</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0488.html">Scheduling CU Changes</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0489.html">Testing Address Connectivity</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0421.html">Creating an SMN Topic</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0022.html">Managing Queue Tags</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0563.html">Setting Queue Properties</a></strong><br>
</li>
</ul>
</div>

2
docs/dli/umn/dli_01_0013.html
View File

@ -25,7 +25,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Enhanced Datasource Connections</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0509.html">Managing Enhanced Datasource Connections</a></div>
</div>
</div>

4
docs/dli/umn/dli_01_0014.html
View File

@ -1,6 +1,6 @@
<a name="dli_01_0014"></a><a name="dli_01_0014"></a>
<h1 class="topictitle1">Adding a Route</h1>
<h1 class="topictitle1">Adding a Route for an Enhanced Datasource Connection</h1>
<div id="body0000001151955397"><div class="section" id="dli_01_0014__section2071923931014"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0014__p197659534560">A route is configured with the destination, next hop type, and next hop to determine where the network traffic is directed. Routes are classified into system routes and custom routes.</p>
<p id="dli_01_0014__p477315405482">After an enhanced connection is created, the subnet is automatically associated with the default route. You can add custom routes as needed to forward traffic destined for the destination to the specified next hop.</p>
<div class="note" id="dli_01_0014__note111421815105414"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0014__ul17762151184110"><li id="dli_01_0014__li22368248550">When an enhanced connection is created, the associated route table is the one associated with the subnet of the data source.</li><li id="dli_01_0014__li18611123515516">The route to be added in the <strong id="dli_01_0014__b16657053314">Add Route</strong> dialog box must be one in the route table associated with the subnet of the resource pool.</li><li id="dli_01_0014__li1037929184116">The subnet of the data source must be different from that used by the resource pool. Otherwise, a network segment conflict occurs.</li></ul>
@ -35,7 +35,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Enhanced Datasource Connections</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0509.html">Managing Enhanced Datasource Connections</a></div>
</div>
</div>

28
docs/dli/umn/dli_01_0015.html
View File

@ -1,27 +1,31 @@
<a name="dli_01_0015"></a><a name="dli_01_0015"></a>
<h1 class="topictitle1">Queue Permission Management</h1>
<div id="body1501038213757"><div class="section" id="dli_01_0015__section11499577152559"><h4 class="sectiontitle">Scenario</h4><ul id="dli_01_0015__ul45004672142627"><li id="dli_01_0015__li27643373142627">You can isolate queues allocated to different users by setting permissions to ensure data query performance.</li><li id="dli_01_0015__li24342715142316">The administrator and queue owner have all permissions, which cannot be set or modified by other users.</li></ul>
<div id="body1501038213757"><p id="dli_01_0015__p199871958124613">Administrators and queue owners have full operation permissions on queues. They can grant operation permissions to other users based on service needs. This ensures that users can execute their jobs independently without any impact on the performance of other users' job execution. This section describes how to manage queue permissions.</p>
<div class="section" id="dli_01_0015__section1833812393482"><h4 class="sectiontitle">Operation Precautions</h4><ul id="dli_01_0015__ul45004672142627"><li id="dli_01_0015__li24342715142316">The administrator and queue owner have all permissions, which cannot be set or modified by other users.</li><li id="dli_01_0015__li1032816519482">When setting queue permissions for a new user, ensure that the region of the user group to which the user belongs has the <strong id="dli_01_0015__b13258638171312">Tenant Guest</strong> permission.<p id="dli_01_0015__p1426035612483"></p>
<p id="dli_01_0015__p48121057144815"></p>
<p id="dli_01_0015__p102341554184819"></p>
</li></ul>
</div>
<div class="section" id="dli_01_0015__section20128473551"><h4 class="sectiontitle">Operations</h4><ol id="dli_01_0015__ol31809540103928"><li id="dli_01_0015__li10639209101950"><span>On the top menu bar of the DLI management console, click <strong id="dli_01_0015__b761084715252">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0015__uicontrol4872154310282"><b>Queue Management</b></span>.</span></li><li id="dli_01_0015__li5676629917517"><span>Select the queue to be configured and choose <strong id="dli_01_0015__b1355011215377">Manage Permissions</strong> in the <strong id="dli_01_0015__b165505213716">Operation</strong> column. The <strong id="dli_01_0015__b842352706171520">User Permission Info</strong> area displays the list of users who have permissions on the queue.</span><p><p id="dli_01_0015__p123538119117">You can assign queue permissions to new users, modify permissions for users who have some permissions of a queue, and revoke all permissions of a user on a queue.</p>
<ul id="dli_01_0015__ul60761295191140"><li id="dli_01_0015__li188984072720">Assign permissions to a new user.<div class="p" id="dli_01_0015__p3651154182719"><a name="dli_01_0015__li188984072720"></a><a name="li188984072720"></a>A new user does not have permissions on the queue.<ol type="a" id="dli_01_0015__ol38619773141515"><li id="dli_01_0015__li1545661718056">Click <strong id="dli_01_0015__b16897820560">Set Permission</strong> on the right of <strong id="dli_01_0015__b1908875614">User Permissions</strong> page. The <span class="wintitle" id="dli_01_0015__wintitle1990108175620"><b>Set Permission</b></span> dialog box is displayed.</li><li id="dli_01_0015__li12033638141515">Specify <span class="parmname" id="dli_01_0015__parmname1572165209191914"><b>Username</b></span> and select corresponding permissions.</li><li id="dli_01_0015__li136261439132">Click <span class="uicontrol" id="dli_01_0015__uicontrol131405151191935"><b>OK</b></span>.<p id="dli_01_0015__p97671211131410"><a href="#dli_01_0015__table15710625151416">Table 1</a> describes the related parameters.</p>
<div class="section" id="dli_01_0015__section20128473551"><h4 class="sectiontitle">Operations</h4><ol id="dli_01_0015__ol31809540103928"><li id="dli_01_0015__li10639209101950"><span>On the top menu bar of the DLI management console, click <strong id="dli_01_0015__b761084715252">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0015__uicontrol4872154310282"><b>Queue Management</b></span>.</span></li><li id="dli_01_0015__li5676629917517"><span>Select the queue to be configured and choose <strong id="dli_01_0015__b1355011215377">Manage Permissions</strong> in the <strong id="dli_01_0015__b165505213716">Operation</strong> column. The <strong id="dli_01_0015__b842352706171520">User Permission Info</strong> area displays the list of users who have permissions on the queue.</span><p><p id="dli_01_0015__p123538119117">You can grant queue permissions to new users, modify permissions for users who have some permissions on a queue, and revoke all permissions for a user on a queue.</p>
<ul id="dli_01_0015__ul60761295191140"><li id="dli_01_0015__li188984072720"><strong id="dli_01_0015__b6310839111817">Grant permissions to a new user.</strong><div class="p" id="dli_01_0015__p3651154182719">A new user does not have permissions on the queue.<ol type="a" id="dli_01_0015__ol38619773141515"><li id="dli_01_0015__li1545661718056">Click <strong id="dli_01_0015__b16897820560">Set Permission</strong> on the right of <strong id="dli_01_0015__b1908875614">User Permissions</strong> page. The <span class="wintitle" id="dli_01_0015__wintitle1990108175620"><b>Set Permission</b></span> dialog box is displayed.</li><li id="dli_01_0015__li12033638141515">Specify <span class="parmname" id="dli_01_0015__parmname1572165209191914"><b>Username</b></span> and select corresponding permissions.</li><li id="dli_01_0015__li136261439132">Click <span class="uicontrol" id="dli_01_0015__uicontrol131405151191935"><b>OK</b></span>.<p id="dli_01_0015__p97671211131410"><a href="#dli_01_0015__table15710625151416">Table 1</a> describes the related parameters.</p>
<div class="tablenoborder"><a name="dli_01_0015__table15710625151416"></a><a name="table15710625151416"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0015__table15710625151416" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_01_0015__row1771212551410"><th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.2.2.2.2.2.1.1.1.3.3.2.3.1.1"><p id="dli_01_0015__p4713132520147">Parameter</p>
<div class="tablenoborder"><a name="dli_01_0015__table15710625151416"></a><a name="table15710625151416"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0015__table15710625151416" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_01_0015__row1771212551410"><th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.3.2.2.2.2.1.2.1.3.3.2.3.1.1"><p id="dli_01_0015__p4713132520147">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="80%" id="mcps1.3.2.2.2.2.2.1.1.1.3.3.2.3.1.2"><p id="dli_01_0015__p1971410256146">Description</p>
<th align="left" class="cellrowborder" valign="top" width="80%" id="mcps1.3.3.2.2.2.2.1.2.1.3.3.2.3.1.2"><p id="dli_01_0015__p1971410256146">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0015__row207191525121418"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.2.2.2.1.1.1.3.3.2.3.1.1 "><p id="dli_01_0015__p5721152541411">Username</p>
<tbody><tr id="dli_01_0015__row207191525121418"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.2.2.2.2.1.2.1.3.3.2.3.1.1 "><p id="dli_01_0015__p5721152541411">Username</p>
</td>
<td class="cellrowborder" valign="top" width="80%" headers="mcps1.3.2.2.2.2.2.1.1.1.3.3.2.3.1.2 "><p id="dli_01_0015__p77232254145">Name of the authorized user.</p>
<td class="cellrowborder" valign="top" width="80%" headers="mcps1.3.3.2.2.2.2.1.2.1.3.3.2.3.1.2 "><p id="dli_01_0015__p77232254145">Name of the authorized user.</p>
<div class="note" id="dli_01_0015__note18961324477"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0015__p13896623472">The username is an existing IAM user name and has logged in to the DLI management console.</p>
</div></div>
</td>
</tr>
<tr id="dli_01_0015__row1273232517141"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.2.2.2.1.1.1.3.3.2.3.1.1 "><p id="dli_01_0015__p1873352541412">Permission Settings</p>
<tr id="dli_01_0015__row1273232517141"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.2.2.2.2.1.2.1.3.3.2.3.1.1 "><p id="dli_01_0015__p1873352541412">Permission Settings</p>
</td>
<td class="cellrowborder" valign="top" width="80%" headers="mcps1.3.2.2.2.2.2.1.1.1.3.3.2.3.1.2 "><ul id="dli_01_0015__ul19734132517140"><li id="dli_01_0015__li1273552518147"><strong id="dli_01_0015__b19616182775213">Delete Queues</strong>: This permission allows you to delete the queue.</li><li id="dli_01_0015__li57405251141"><strong id="dli_01_0015__b842352706165610">Submit Jobs</strong>: This permission allows you to submit jobs using this queue.</li><li id="dli_01_0015__li6741725151419"><strong id="dli_01_0015__b842352706165628">Terminate Jobs</strong>: This permission allows you to terminate jobs submitted using this queue.</li><li id="dli_01_0015__li187441525101419"><strong id="dli_01_0015__b842352706165645">Grant Permission</strong>: This permission allows you to grant queue permissions to other users.</li><li id="dli_01_0015__li1574612513141"><strong id="dli_01_0015__b84235270616572">Revoke Permission</strong>: This permission allows you to revoke the queue permissions that other users have but cannot revoke the queue owner's permissions.</li><li id="dli_01_0015__li55847342101527"><strong id="dli_01_0015__b842352706162234">View Other User's Permissions</strong>: This permission allows you to view the queue permissions of other users.</li><li id="dli_01_0015__li17629232181312"><strong id="dli_01_0015__b204581124191616">Restart Queues</strong>: This permission allows you to restart queues.</li><li id="dli_01_0015__li590916091412"><strong id="dli_01_0015__b350110393199">Modify Queue Specifications</strong>: This permission allows you to modify queue specifications.</li></ul>
<td class="cellrowborder" valign="top" width="80%" headers="mcps1.3.3.2.2.2.2.1.2.1.3.3.2.3.1.2 "><ul id="dli_01_0015__ul19734132517140"><li id="dli_01_0015__li1273552518147"><strong id="dli_01_0015__b19616182775213">Delete Queues</strong>: This permission allows you to delete the queue.</li><li id="dli_01_0015__li57405251141"><strong id="dli_01_0015__b842352706165610">Submit Jobs</strong>: This permission allows you to submit jobs using this queue.</li><li id="dli_01_0015__li6741725151419"><strong id="dli_01_0015__b842352706165628">Terminate Jobs</strong>: This permission allows you to terminate jobs submitted using this queue.</li><li id="dli_01_0015__li187441525101419"><strong id="dli_01_0015__b842352706165645">Grant Permission</strong>: This permission allows you to grant queue permissions to other users.</li><li id="dli_01_0015__li1574612513141"><strong id="dli_01_0015__b84235270616572">Revoke Permission</strong>: This permission allows you to revoke the queue permissions that other users have but cannot revoke the queue owner's permissions.</li><li id="dli_01_0015__li55847342101527"><strong id="dli_01_0015__b842352706162234">View Other User's Permissions</strong>: This permission allows you to view the queue permissions of other users.</li><li id="dli_01_0015__li17629232181312"><strong id="dli_01_0015__b204581124191616">Restart Queues</strong>: This permission allows you to restart queues.</li><li id="dli_01_0015__li590916091412"><strong id="dli_01_0015__b350110393199">Modify Queue Specifications</strong>: This permission allows you to modify queue specifications.</li></ul>
</td>
</tr>
</tbody>
@ -29,16 +33,16 @@
</div>
</li></ol>
</div>
</li><li id="dli_01_0015__li5373564191140">To assign or revoke permissions of a user who has some permissions on the queue, perform the following steps:<ol type="a" id="dli_01_0015__ol64022357191649"><li id="dli_01_0015__li11136112815177">In the list under <span class="wintitle" id="dli_01_0015__wintitle2724849191220"><b>User Permission Info</b></span> for a queue, select the user whose permissions need to be modified and click <span class="uicontrol" id="dli_01_0015__uicontrol197263497126"><b>Set Permission</b></span> in the <span class="uicontrol" id="dli_01_0015__uicontrol156991298917161"><b>Operation</b></span> column.</li><li id="dli_01_0015__li3483530142353">In the displayed <span class="wintitle" id="dli_01_0015__wintitle1283331883119"><b>Set Permission</b></span> dialog box, modify the permissions of the current user. <a href="#dli_01_0015__table15710625151416">Table 1</a> lists the detailed permission descriptions.<p id="dli_01_0015__p3341848294412">If all options under <span class="parmname" id="dli_01_0015__parmname191181136193316"><b>Set Permission</b></span> are gray, you are not allowed to change permissions on this queue. You can apply to the administrator, queue owner, or other authorized users for queue permission granting and revoking.</p>
</li><li id="dli_01_0015__li5373564191140"><strong id="dli_01_0015__b1647918533213">To grant or revoke permissions for a user who already has certain permissions on a queue, perform the following steps:</strong><ol type="a" id="dli_01_0015__ol64022357191649"><li id="dli_01_0015__li11136112815177">In the list under <span class="wintitle" id="dli_01_0015__wintitle2724849191220"><b>User Permission Info</b></span> for a queue, select the user whose permissions need to be modified and click <span class="uicontrol" id="dli_01_0015__uicontrol197263497126"><b>Set Permission</b></span> in the <span class="uicontrol" id="dli_01_0015__uicontrol156991298917161"><b>Operation</b></span> column.</li><li id="dli_01_0015__li3483530142353">In the displayed <span class="wintitle" id="dli_01_0015__wintitle1283331883119"><b>Set Permission</b></span> dialog box, modify the permissions of the current user. <a href="#dli_01_0015__table15710625151416">Table 1</a> lists the detailed permission descriptions.<p id="dli_01_0015__p3341848294412">If all options under <span class="parmname" id="dli_01_0015__parmname191181136193316"><b>Set Permission</b></span> are gray, you are not allowed to change permissions on this queue. You can apply to the administrator, queue owner, or other authorized users for queue permission granting and revoking.</p>
</li><li id="dli_01_0015__li10389193018234">Click <span class="uicontrol" id="dli_01_0015__uicontrol1516336014162242"><b>OK</b></span>.</li></ol>
</li><li id="dli_01_0015__li2536165618122">To revoke all permission of a user on the queue, perform the following steps:<p id="dli_01_0015__p41775579123"><a name="dli_01_0015__li2536165618122"></a><a name="li2536165618122"></a>In the user list under <span class="wintitle" id="dli_01_0015__wintitle652036544"><b>Permission Info</b></span>, select the user whose permission needs to be revoked and click <span class="uicontrol" id="dli_01_0015__uicontrol1439083445"><b>Revoke Permission</b></span> under <span class="uicontrol" id="dli_01_0015__uicontrol18942202419342"><b>Operation</b></span>. In the <strong id="dli_01_0015__b19765113823218">Revoke Permission</strong> dialog box, click <span class="uicontrol" id="dli_01_0015__uicontrol35891343193211"><b>OK</b></span>. All permissions on this queue are revoked.</p>
</li><li id="dli_01_0015__li2536165618122"><strong id="dli_01_0015__b3926205032220">To revoke all permissions for a user on a queue, perform the following steps:</strong><p id="dli_01_0015__p41775579123">In the user list under <span class="wintitle" id="dli_01_0015__wintitle780039627"><b>Permission Info</b></span>, select the user whose permission needs to be revoked and click <span class="uicontrol" id="dli_01_0015__uicontrol398062711"><b>Revoke Permission</b></span> under <span class="uicontrol" id="dli_01_0015__uicontrol18942202419342"><b>Operation</b></span>. In the <strong id="dli_01_0015__b19765113823218">Revoke Permission</strong> dialog box, click <span class="uicontrol" id="dli_01_0015__uicontrol35891343193211"><b>OK</b></span>. All permissions on this queue are revoked.</p>
</li></ul>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0012.html">Queue Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0528.html">Managing Queues</a></div>
</div>
</div>

4
docs/dli/umn/dli_01_0016.html
View File

@ -4,14 +4,14 @@
<div id="body1501038213757"><p id="dli_01_0016__p16461459191012">You can delete a queue based on actual conditions.</p>
<div class="note" id="dli_01_0016__note9413185151111"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0016__ul45795743311"><li id="dli_01_0016__li1358011733310">This operation will fail if there are jobs in the <strong id="dli_01_0016__b27141763611">Submitting</strong> or <strong id="dli_01_0016__b125820153610">Running</strong> state on this queue.</li><li id="dli_01_0016__li1063413282129">Deleting a queue does not cause table data loss in your database.</li></ul>
</div></div>
<div class="section" id="dli_01_0016__section1015535913555"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0016__ol17321838112226"><li id="dli_01_0016__li10639209101950"><span>On the left of the DLI management console, click <strong id="dli_01_0016__b178272619304">Resources</strong> &gt;<span class="uicontrol" id="dli_01_0016__uicontrol89261914012"><b>Queue Management</b></span>.</span></li><li id="dli_01_0016__li172323754417"><span>Locate the row where the target queue locates and click <span class="uicontrol" id="dli_01_0016__uicontrol115498314509"><b>Delete</b></span> in the <span class="parmname" id="dli_01_0016__parmname11555193125011"><b>Operation</b></span> column.</span><p><div class="note" id="dli_01_0016__note2123195011570"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0016__p1477434110573">If <span class="uicontrol" id="dli_01_0016__uicontrol18933114220509"><b>Delete</b></span> in the <span class="parmname" id="dli_01_0016__parmname29395425502"><b>Operation</b></span> column is gray, the current user does not have the permission of deleting the queue. You can apply to the administrator for the permission.</p>
<div class="section" id="dli_01_0016__section1015535913555"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0016__ol17321838112226"><li id="dli_01_0016__li10639209101950"><span>In the navigation pane on the left of the DLI management console, choose <strong id="dli_01_0016__b685115615140">Resources</strong> &gt; <strong id="dli_01_0016__b193417241512">Queue Management</strong>.</span></li><li id="dli_01_0016__li172323754417"><span>Locate the row where the target queue locates and click <span class="uicontrol" id="dli_01_0016__uicontrol115498314509"><b>Delete</b></span> in the <span class="parmname" id="dli_01_0016__parmname11555193125011"><b>Operation</b></span> column.</span><p><div class="note" id="dli_01_0016__note2123195011570"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0016__p1477434110573">If <span class="uicontrol" id="dli_01_0016__uicontrol18933114220509"><b>Delete</b></span> in the <span class="parmname" id="dli_01_0016__parmname29395425502"><b>Operation</b></span> column is gray, the current user does not have the permission of deleting the queue. You can apply to the administrator for the permission.</p>
</div></div>
</p></li><li id="dli_01_0016__li2295562111246"><span>In the displayed dialog box, click <span class="uicontrol" id="dli_01_0016__uicontrol24383318512"><b>OK</b></span>.</span></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0012.html">Queue Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0528.html">Managing Queues</a></div>
</div>
</div>

111
docs/dli/umn/dli_01_0017.html
View File

File diff suppressed because it is too large Load Diff

2
docs/dli/umn/dli_01_0018.html
View File

@ -20,7 +20,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Enhanced Datasource Connections</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0509.html">Managing Enhanced Datasource Connections</a></div>
</div>
</div>

6
docs/dli/umn/dli_01_0019.html
View File

@ -22,7 +22,7 @@
<ul id="dli_01_0019__ul6423144611162"><li id="dli_01_0019__li14251946151610">Click the text box and select a predefined tag key from the drop-down list.<p id="dli_01_0019__p12169132323510"><a name="dli_01_0019__li14251946151610"></a><a name="li14251946151610"></a>To add a predefined tag, you need to create one on TMS and then select it from the <span class="parmname" id="dli_01_0019__parmname17849229544175"><b>Tag key</b></span> drop-down list. You can click <strong id="dli_01_0019__b39408883441711">View predefined tags</strong> to go to the <strong id="dli_01_0019__b119048434541711">Predefined Tags</strong> page of the TMS console. Then, click <strong id="dli_01_0019__b141119490641711">Create Tag</strong> in the upper corner of the page to create a predefined tag.</p>
<p id="dli_01_0019__p1926211187356"></p>
</li></ul>
<ul id="dli_01_0019__ul643911462165"><li id="dli_01_0019__li7441146181612">Enter a tag key in the text box.<div class="note" id="dli_01_0019__note206961611105313"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0019__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0019__dli_01_0363_b544594615499">_sys_</strong>.</p>
<ul id="dli_01_0019__ul643911462165"><li id="dli_01_0019__li7441146181612">Enter a tag key in the text box.<div class="note" id="dli_01_0019__note206961611105313"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0019__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0019__dli_01_0363_b544594615499">_sys_</strong>.</p>
</div></div>
</li></ul>
</td>
@ -30,7 +30,7 @@
<tr id="dli_01_0019__row645064691617"><td class="cellrowborder" valign="top" width="12.94%" headers="mcps1.3.2.2.4.4.2.3.1.1 "><p id="dli_01_0019__p345210468163">Tag value</p>
</td>
<td class="cellrowborder" valign="top" width="87.06%" headers="mcps1.3.2.2.4.4.2.3.1.2 "><p id="dli_01_0019__p1545619468165">You can perform the following operations:</p>
<ul id="dli_01_0019__ul1045720469162"><li id="dli_01_0019__li545914661612">Click the text box and select a predefined tag value from the drop-down list.</li><li id="dli_01_0019__li184611546101610">Enter a tag value in the text box.<div class="note" id="dli_01_0019__note11572532152719"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0019__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed. The value cannot start or end with a space.</p>
<ul id="dli_01_0019__ul1045720469162"><li id="dli_01_0019__li545914661612">Click the text box and select a predefined tag value from the drop-down list.</li><li id="dli_01_0019__li184611546101610">Enter a tag value in the text box.<div class="note" id="dli_01_0019__note11572532152719"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0019__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed.</p>
</div></div>
</li></ul>
</td>
@ -43,7 +43,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Enhanced Datasource Connections</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0509.html">Managing Enhanced Datasource Connections</a></div>
</div>
</div>

441
docs/dli/umn/dli_01_0020.html Normal file
View File

File diff suppressed because it is too large Load Diff

121
docs/dli/umn/dli_01_0021.html
View File

File diff suppressed because it is too large Load Diff

6
docs/dli/umn/dli_01_0022.html
View File

@ -21,7 +21,7 @@
<ul id="dli_01_0022__ul6423144611162"><li id="dli_01_0022__li14251946151610">Click the text box and select a predefined tag key from the drop-down list.<p id="dli_01_0022__p12169132323510"><a name="dli_01_0022__li14251946151610"></a><a name="li14251946151610"></a>To add a predefined tag, you need to create one on TMS and then select it from the <span class="parmname" id="dli_01_0022__parmname672234855114"><b>Tag key</b></span> drop-down list. You can click <strong id="dli_01_0022__b1270845085112">View predefined tags</strong> to go to the <strong id="dli_01_0022__b170955055115">Predefined Tags</strong> page of the TMS console. Then, click <strong id="dli_01_0022__b157091850115120">Create Tag</strong> in the upper corner of the page to create a predefined tag.</p>
<p id="dli_01_0022__p1926211187356"></p>
</li></ul>
<ul id="dli_01_0022__ul643911462165"><li id="dli_01_0022__li7441146181612">Enter a tag key in the text box.<div class="note" id="dli_01_0022__note206961611105313"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0022__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0022__dli_01_0363_b544594615499">_sys_</strong>.</p>
<ul id="dli_01_0022__ul643911462165"><li id="dli_01_0022__li7441146181612">Enter a tag key in the text box.<div class="note" id="dli_01_0022__note206961611105313"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0022__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0022__dli_01_0363_b544594615499">_sys_</strong>.</p>
</div></div>
</li></ul>
</td>
@ -29,7 +29,7 @@
<tr id="dli_01_0022__row645064691617"><td class="cellrowborder" valign="top" width="12.94%" headers="mcps1.3.1.8.4.2.1.2.3.1.1 "><p id="dli_01_0022__p345210468163">Tag value</p>
</td>
<td class="cellrowborder" valign="top" width="87.06%" headers="mcps1.3.1.8.4.2.1.2.3.1.2 "><p id="dli_01_0022__p1545619468165">You can specify the tag value in either of the following ways:</p>
<ul id="dli_01_0022__ul1045720469162"><li id="dli_01_0022__li545914661612">Click the text box and select a predefined tag value from the drop-down list.</li><li id="dli_01_0022__li184611546101610">Enter a tag value in the text box.<div class="note" id="dli_01_0022__note11572532152719"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0022__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed. The value cannot start or end with a space.</p>
<ul id="dli_01_0022__ul1045720469162"><li id="dli_01_0022__li545914661612">Click the text box and select a predefined tag value from the drop-down list.</li><li id="dli_01_0022__li184611546101610">Enter a tag value in the text box.<div class="note" id="dli_01_0022__note11572532152719"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0022__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed.</p>
</div></div>
</li></ul>
</td>
@ -44,7 +44,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0012.html">Queue Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0528.html">Managing Queues</a></div>
</div>
</div>

10
docs/dli/umn/dli_01_0220.html
View File

@ -4,13 +4,15 @@
<div id="body1510283588485"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0002.html">Creating and Submitting a Spark SQL Job</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0002.html">Using DLI to Submit a SQL Job to Query OBS Data</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0512.html">Developing and Submitting a Spark SQL Job Using the TPC-H Sample Template</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0020.html">Using DLI to Submit a SQL Job to Query RDS for MySQL Data</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0375.html">Creating and Submitting a Spark Jar Job</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0531.html">Using DLI to Submit a Flink OpenSource SQL Job to Query RDS for MySQL Data</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0531.html">Creating and Submitting a Flink OpenSource SQL Job</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0512.html">Using DLI to Submit a Flink Jar Job</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0375.html">Using DLI to Submit a Spark Jar Job</a></strong><br>
</li>
</ul>
</div>

154
docs/dli/umn/dli_01_0228.html
View File

File diff suppressed because it is too large Load Diff

8
docs/dli/umn/dli_01_0253.html
View File

@ -1,7 +1,7 @@
<a name="dli_01_0253"></a><a name="dli_01_0253"></a>
<h1 class="topictitle1">Importing Data to the Table</h1>
<div id="body1519716643081"><p id="dli_01_0253__p1793214255012">You can import data from OBS to a table created in DLI.</p>
<h1 class="topictitle1">Importing OBS Data to DLI</h1>
<div id="body1519716643081"><p id="dli_01_0253__p1793214255012">This section describes how to import data stored in OBS to a table on the DLI console.</p>
<div class="section" id="dli_01_0253__section321713588229"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0253__ul7225153265013"><li id="dli_01_0253__li10225103235019">Only one path can be specified during data import. The path cannot contain commas (,).</li><li id="dli_01_0253__li119556312517">To import data in CSV format to a partitioned table, place the column to be partitioned in the last column of the data source.</li><li id="dli_01_0253__li1962615137530">You are advised not to concurrently import data in to a table. If you concurrently import data into a table, there is a possibility that conflicts occur, leading to failed data import.</li><li id="dli_01_0253__li5626191395311">The imported file can be in CSV, Parquet, ORC, JSON, and Avro format. The encoding format must be UTF-8.</li></ul>
</div>
<div class="section" id="dli_01_0253__section46923850144935"><h4 class="sectiontitle">Prerequisites</h4><p id="dli_01_0253__p2990805493216">The data to be imported has been stored on OBS.</p>
@ -60,7 +60,7 @@
<tr id="dli_01_0253__row17846481431"><td class="cellrowborder" valign="top" width="20.3%" headers="mcps1.3.4.2.2.2.1.2.4.1.1 "><p id="dli_01_0253__p16846128931">Table Header: No/Yes</p>
</td>
<td class="cellrowborder" valign="top" width="64.2%" headers="mcps1.3.4.2.2.2.1.2.4.1.2 "><p id="dli_01_0253__p9543131714501">This parameter is valid only when <span class="parmname" id="dli_01_0253__parmname17355153517345"><b>File Format</b></span> is set to <span class="parmvalue" id="dli_01_0253__parmvalue1435693563411"><b>CSV</b></span>. Whether the data source to be imported contains the table header.</p>
<p id="dli_01_0253__p784648833">Click <span class="parmname" id="dli_01_0253__parmname321352818815"><b>Advanced Settings</b></span> and select the check box next to <span class="parmname" id="dli_01_0253__parmname7213328389"><b>Table Header: No</b></span>. If the check box is selected, the table header is displayed. If the check box is deselected, no table header is displayed.</p>
<p id="dli_01_0253__p784648833">Click <strong id="dli_01_0253__b1284662819310">Advanced Settings</strong> and select the checkbox next to <strong id="dli_01_0253__b12847162843118">Table Header: No</strong>. If the checkbox is selected, the table header is displayed. If the checkbox is deselected, no table header is displayed.</p>
</td>
<td class="cellrowborder" valign="top" width="15.5%" headers="mcps1.3.4.2.2.2.1.2.4.1.3 "><p id="dli_01_0253__p1846128435">-</p>
</td>
@ -127,7 +127,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0390.html">Databases and Tables</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0625.html">Managing Table Resources on the DLI Console</a></div>
</div>
</div>

485
docs/dli/umn/dli_01_0318.html
View File

File diff suppressed because it is too large Load Diff

345
docs/dli/umn/dli_01_0320.html
View File

File diff suppressed because it is too large Load Diff

79
docs/dli/umn/dli_01_0363.html
View File

File diff suppressed because it is too large Load Diff

16
docs/dli/umn/dli_01_0366.html
View File

@ -1,25 +1,25 @@
<a name="dli_01_0366"></a><a name="dli_01_0366"></a>
<h1 class="topictitle1">Package Management</h1>
<h1 class="topictitle1">Managing Program Packages of Jar Jobs</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0407.html">Overview</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0407.html">Package Management Overview</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0477.html">Managing Permissions on Packages and Package Groups</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0367.html">Creating a DLI Package</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0367.html">Creating a Package</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0477.html">Configuring DLI Package Permissions</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0369.html">Deleting a Package</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0478.html">Changing the DLI Package Owner</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0478.html">Modifying the Owner</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0369.html">Managing DLI Package Tags</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0397.html">Built-in Dependencies</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0397.html">DLI Built-in Dependencies</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0004.html">Data Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0567.html">Common DLI Management Operations</a></div>
</div>
</div>

9
docs/dli/umn/dli_01_0367.html
View File

@ -1,6 +1,6 @@
<a name="dli_01_0367"></a><a name="dli_01_0367"></a>
<h1 class="topictitle1">Creating a Package</h1>
<h1 class="topictitle1">Creating a DLI Package</h1>
<div id="body8662426"><p id="dli_01_0367__en-us_topic_0122016946_p93081221124114">DLI allows you to submit program packages in batches to the general-use queue for running.</p>
<div class="note" id="dli_01_0367__note7234131133213"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0367__p92352011173213">If you need to update a package, you can use the same package or file to upload it to the same location (in the same group) on DLI to overwrite the original package or file.</p>
</div></div>
@ -44,10 +44,10 @@
<td class="cellrowborder" valign="top" width="83.67999999999999%" headers="mcps1.3.4.2.3.3.2.3.1.2 "><p id="dli_01_0367__p52841736202320">Tags used to identify cloud resources. A tag includes the tag key and tag value. If you want to use the same tag to identify multiple cloud resources, that is, to select the same tag from the drop-down list box for all services, you are advised to create predefined tags on the Tag Management Service (TMS).</p>
<div class="note" id="dli_01_0367__note647119469162"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0367__dli_01_0363_ul1830018521911"><li id="dli_01_0367__dli_01_0363_li186815313365">A maximum of 20 tags can be added.</li><li id="dli_01_0367__dli_01_0363_li183008521591">Only one tag value can be added to a tag key.</li><li id="dli_01_0367__dli_01_0363_li14484164661614">The key name in each resource must be unique.</li></ul>
</div></div>
<ul id="dli_01_0367__ul6617123554815"><li id="dli_01_0367__li9617123518481">Tag key: Enter a tag key name in the text box.<div class="note" id="dli_01_0367__note18617135124816"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0367__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0367__dli_01_0363_b544594615499">_sys_</strong>.</p>
<ul id="dli_01_0367__ul6617123554815"><li id="dli_01_0367__li9617123518481">Tag key: Enter a tag key name in the text box.<div class="note" id="dli_01_0367__note18617135124816"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0367__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0367__dli_01_0363_b544594615499">_sys_</strong>.</p>
</div></div>
</li></ul>
<ul id="dli_01_0367__ul16911451899"><li id="dli_01_0367__li26911851193">Tag value: Enter a tag value in the text box.<div class="note" id="dli_01_0367__note86179355480"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0367__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed. The value cannot start or end with a space.</p>
<ul id="dli_01_0367__ul16911451899"><li id="dli_01_0367__li26911851193">Tag value: Enter a tag value in the text box.<div class="note" id="dli_01_0367__note86179355480"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0367__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed.</p>
</div></div>
</li></ul>
</td>
@ -56,12 +56,13 @@
</table>
</div>
</li><li id="dli_01_0367__li472825420248">Click <span class="uicontrol" id="dli_01_0367__uicontrol4864202813586"><b>OK</b></span>.<p id="dli_01_0367__en-us_topic_0122016946_en-us_topic_0093946917_p1543112413517">After a package is created, you can view and select the package for use on the <span class="wintitle" id="dli_01_0367__wintitle1066123765811"><b>Package Management</b></span> page.</p>
<p id="dli_01_0367__p12899334125211">Once a job is executed, you can free up DLI storage space by promptly deleting the job package from the package management page if it is no longer needed.</p>
</li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0366.html">Package Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0366.html">Managing Program Packages of Jar Jobs</a></div>
</div>
</div>

39
docs/dli/umn/dli_01_0369.html
View File

@ -1,13 +1,44 @@
<a name="dli_01_0369"></a><a name="dli_01_0369"></a>
<h1 class="topictitle1">Deleting a Package</h1>
<div id="body8662426"><p id="dli_01_0369__en-us_topic_0122016948_en-us_topic_0093946752_p19218322171945">You can delete a package based on actual conditions.</p>
<div class="section" id="dli_01_0369__en-us_topic_0122016948_en-us_topic_0093946752_section1015535913555"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0369__ol92311048152411"><li id="dli_01_0369__li82319482246">On the left of the management console, choose <span class="uicontrol" id="dli_01_0369__uicontrol0527755124512"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0369__uicontrol12527185517458"><b>Package Management</b></span>.</li><li id="dli_01_0369__li2231144811249">Click <span class="uicontrol" id="dli_01_0369__uicontrol192517519462"><b>Delete</b></span> in the <span class="parmname" id="dli_01_0369__parmname17366101213467"><b>Operation</b></span> column of the package to be deleted.</li><li id="dli_01_0369__li2232104852413">In the dialog box that is displayed, click <span class="uicontrol" id="dli_01_0369__uicontrol177411619462"><b>Yes</b></span>.</li></ol>
<h1 class="topictitle1">Managing DLI Package Tags</h1>
<div id="body8662426"><p id="dli_01_0369__p985684415815">Tags are key-value pairs that you can define to identify cloud resources. They assist you in categorizing and searching for cloud resources. A tag consists of a key and a value.</p>
<p id="dli_01_0369__p1650242964010">DLI allows you to add tags to package groups or packages.</p>
<ol id="dli_01_0369__ol73523543494"><li id="dli_01_0369__li203521549491">Log in to the DLI management console and choose <strong id="dli_01_0369__b181811147342">Data Management</strong> &gt; <strong id="dli_01_0369__b17186148341">Package Management</strong>.</li><li id="dli_01_0369__li14352195434919">On the <strong id="dli_01_0369__b71852506343">Package Management</strong> page, locate the desired package, click <strong id="dli_01_0369__b2777172817355">More</strong> in the <strong id="dli_01_0369__b16121203416351">Operation</strong> column, and select <strong id="dli_01_0369__b31351142173517">Tags</strong>.</li><li id="dli_01_0369__li18352115419495">On the page that appears, click <strong id="dli_01_0369__b818652610362">Add/Edit Tag</strong> in the upper left corner.</li><li id="dli_01_0369__li235285444912">In the <strong id="dli_01_0369__b1663031173713">Add/Edit Tag</strong> dialog box, set parameters.
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0369__table139874691610" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Tag parameters</caption><thead align="left"><tr id="dli_01_0369__row1640724610160"><th align="left" class="cellrowborder" valign="top" width="12.94%" id="mcps1.3.3.4.2.2.3.1.1"><p id="dli_01_0369__p17410246121616">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="87.06%" id="mcps1.3.3.4.2.2.3.1.2"><p id="dli_01_0369__p154131746131615">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0369__row13417154613164"><td class="cellrowborder" valign="top" width="12.94%" headers="mcps1.3.3.4.2.2.3.1.1 "><p id="dli_01_0369__p74201346111616">Tag key</p>
</td>
<td class="cellrowborder" valign="top" width="87.06%" headers="mcps1.3.3.4.2.2.3.1.2 "><p id="dli_01_0369__p642018464167">You can specify the tag key in either of the following ways:</p>
<ul id="dli_01_0369__ul6423144611162"><li id="dli_01_0369__li14251946151610">Click the text box and select a predefined tag key from the drop-down list.<p id="dli_01_0369__p12169132323510"><a name="dli_01_0369__li14251946151610"></a><a name="li14251946151610"></a>To add a predefined tag, you need to create one on TMS and then select it from the tag key drop-down list. You can click <strong id="dli_01_0369__b493974583811">View predefined tags</strong> to go to the <strong id="dli_01_0369__b19939145203820">Predefined Tags</strong> page of the TMS console. Then, click <strong id="dli_01_0369__b1093911456383">Create Tag</strong> in the upper corner of the page to create a predefined tag.</p>
<p id="dli_01_0369__p1926211187356"></p>
</li></ul>
<ul id="dli_01_0369__ul643911462165"><li id="dli_01_0369__li7441146181612">Enter a tag key in the text box.<div class="note" id="dli_01_0369__note206961611105313"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0369__dli_01_0363_p1448010306508">A tag key can contain a maximum of 128 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0369__dli_01_0363_b544594615499">_sys_</strong>.</p>
</div></div>
</li></ul>
</td>
</tr>
<tr id="dli_01_0369__row645064691617"><td class="cellrowborder" valign="top" width="12.94%" headers="mcps1.3.3.4.2.2.3.1.1 "><p id="dli_01_0369__p345210468163">Tag value</p>
</td>
<td class="cellrowborder" valign="top" width="87.06%" headers="mcps1.3.3.4.2.2.3.1.2 "><p id="dli_01_0369__p1545619468165">You can specify the tag value in either of the following ways:</p>
<ul id="dli_01_0369__ul1045720469162"><li id="dli_01_0369__li545914661612">Click the text box and select a predefined tag value from the drop-down list.</li><li id="dli_01_0369__li184611546101610">Enter a tag value in the text box.<div class="note" id="dli_01_0369__note11572532152719"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0369__dli_01_0363_p4702194215507">A tag value can contain a maximum of 255 characters. Only letters, numbers, spaces, and special characters (_.:+-@) are allowed.</p>
</div></div>
</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
<div class="note" id="dli_01_0369__note647119469162"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0369__dli_01_0363_ul1830018521911"><li id="dli_01_0369__dli_01_0363_li186815313365">A maximum of 20 tags can be added.</li><li id="dli_01_0369__dli_01_0363_li183008521591">Only one tag value can be added to a tag key.</li><li id="dli_01_0369__dli_01_0363_li14484164661614">The key name in each resource must be unique.</li></ul>
</div></div>
</li><li id="dli_01_0369__li335225418495">Click <strong id="dli_01_0369__b197301723113811">OK</strong>.</li><li id="dli_01_0369__li435215410493">(Optional) To delete a tag, locate the tag in the tag list and click <strong id="dli_01_0369__b6978349154316">Delete</strong> in its <strong id="dli_01_0369__b14103121164415">Operation</strong> column.</li></ol>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0366.html">Package Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0366.html">Managing Program Packages of Jar Jobs</a></div>
</div>
</div>

201
docs/dli/umn/dli_01_0375.html
View File

File diff suppressed because it is too large Load Diff

10
docs/dli/umn/dli_01_0376.html
View File

@ -1,18 +1,16 @@
<a name="dli_01_0376"></a><a name="dli_01_0376"></a>
<h1 class="topictitle1">Modifying the Owners of Databases and Tables</h1>
<div id="body1569491496985"><p id="dli_01_0376__p7729122613405">During actual use, developers create databases and tables and submit them to test personnel for testing. After the test is complete, the databases and tables are transferred to O&amp;M personnel for user experience. In this case, you can change the owner of the databases and tables to transfer data to other owners.</p>
<div class="section" id="dli_01_0376__section48746710711"><h4 class="sectiontitle">Modifying the Database Owner</h4><div class="p" id="dli_01_0376__p17554192016499">You can change the owner of a database on either the <span class="wintitle" id="dli_01_0376__wintitle112031233219"><b>Data Management</b></span> page or the <span class="wintitle" id="dli_01_0376__wintitle112037333118"><b>SQL Editor</b></span> page.<ul id="dli_01_0376__ul10405142364818"><li id="dli_01_0376__li2040592311485">On the <span class="wintitle" id="dli_01_0376__wintitle1588611347119"><b>Data Management</b></span> page, change the database owner.<ol id="dli_01_0376__ol153397745813"><li id="dli_01_0376__li182443471248">On the left of the management console, choose <span class="uicontrol" id="dli_01_0376__uicontrol14267241951"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0376__uicontrol226424355"><b>Databases and Tables</b></span>.</li><li id="dli_01_0376__li1924517478247">On the <span class="wintitle" id="dli_01_0376__wintitle25814432053"><b>Databases and Tables</b></span> page, locate the database you want and click <strong id="dli_01_0376__b029313718312">More</strong> &gt; <span class="uicontrol" id="dli_01_0376__uicontrol119711541751"><b>Modify Database</b></span> in the <strong id="dli_01_0376__b829315585512">Operation</strong> column.</li><li id="dli_01_0376__li1979150165815">In the displayed dialog box, enter a new owner name (an existing username) and click <span class="uicontrol" id="dli_01_0376__uicontrol3175153914115"><b>OK</b></span>.</li></ol>
<h1 class="topictitle1">Changing the Database Owner on the DLI Console</h1>
<div id="body1569491496985"><p id="dli_01_0376__p7729122613405">In practical use, developers create databases and tables, which are then handed over to testers for testing. Once testing is complete, the databases and tables are handed over to O&amp;M personnel for experience. In this scenario, ownership of the data can be transferred to another owner by changing the database owner.</p>
<div class="section" id="dli_01_0376__section48746710711"><h4 class="sectiontitle">Changing the Database Owner</h4><div class="p" id="dli_01_0376__p17554192016499">You can change the owner of a database on either the <span class="wintitle" id="dli_01_0376__wintitle112031233219"><b>Data Management</b></span> page or the <span class="wintitle" id="dli_01_0376__wintitle112037333118"><b>SQL Editor</b></span> page.<ul id="dli_01_0376__ul10405142364818"><li id="dli_01_0376__li2040592311485">On the <span class="wintitle" id="dli_01_0376__wintitle1588611347119"><b>Data Management</b></span> page, change the database owner.<ol id="dli_01_0376__ol153397745813"><li id="dli_01_0376__li182443471248">On the left of the management console, choose <span class="uicontrol" id="dli_01_0376__uicontrol14267241951"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0376__uicontrol226424355"><b>Databases and Tables</b></span>.</li><li id="dli_01_0376__li1924517478247">On the <span class="wintitle" id="dli_01_0376__wintitle25814432053"><b>Databases and Tables</b></span> page, locate the database you want and click <strong id="dli_01_0376__b029313718312">More</strong> &gt; <span class="uicontrol" id="dli_01_0376__uicontrol119711541751"><b>Modify Database</b></span> in the <strong id="dli_01_0376__b829315585512">Operation</strong> column.</li><li id="dli_01_0376__li1979150165815">In the displayed dialog box, enter a new owner name (an existing username) and click <span class="uicontrol" id="dli_01_0376__uicontrol3175153914115"><b>OK</b></span>.</li></ol>
</li><li id="dli_01_0376__li10384371151118">Change the database owner on the <span class="wintitle" id="dli_01_0376__wintitle1853761213618"><b>SQL Editor</b></span> page.<ol id="dli_01_0376__ol39382464151459"><li id="dli_01_0376__li169951648123920">On the left of the management console, click <strong id="dli_01_0376__b198591277618">SQL Editor</strong>.</li><li id="dli_01_0376__li1773513617043">In the navigation tree on the left, click <strong id="dli_01_0376__b71971655102315">Databases</strong>, click <span><img id="dli_01_0376__image819875518239" src="en-us_image_0237984360.png"></span> on the right of the database you want to modify, and choose <strong id="dli_01_0376__b21982055122314">Modify Database</strong> from the shortcut menu.</li><li id="dli_01_0376__li0998173818546">In the displayed dialog box, enter a new owner name (an existing username) and click <span class="uicontrol" id="dli_01_0376__uicontrol1952613461213"><b>OK</b></span>.</li></ol>
</li></ul>
</div>
</div>
<div class="section" id="dli_01_0376__section587311710717"><h4 class="sectiontitle">Modifying the Table Owner</h4><ol id="dli_01_0376__ol1098894162614"><li id="dli_01_0376__li5988141102620">On the left of the management console, choose <span class="uicontrol" id="dli_01_0376__uicontrol07043303612"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0376__uicontrol57057309615"><b>Databases and Tables</b></span>.</li><li id="dli_01_0376__li11988184117268">Click the name of the database corresponding to the table to be modified. The <strong id="dli_01_0376__b4573203212613">Manage Tables</strong> page of the database is displayed.</li><li id="dli_01_0376__li14988134192617">In the <strong id="dli_01_0376__b5541114516512">Operation</strong> column of the target table, choose <span class="uicontrol" id="dli_01_0376__uicontrol7358125715115"><b>More</b></span> &gt; <span class="uicontrol" id="dli_01_0376__uicontrol13196251527"><b>Modify Owner</b></span>.</li><li id="dli_01_0376__li698814192614">In the displayed dialog box, enter a new owner name (an existing username) and click <span class="uicontrol" id="dli_01_0376__uicontrol66025616110"><b>OK</b></span>.</li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0390.html">Databases and Tables</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0004.html">Managing Database Resources on the DLI Console</a></div>
</div>
</div>

32
docs/dli/umn/dli_01_0377.html
View File

@ -1,17 +1,27 @@
<a name="dli_01_0377"></a><a name="dli_01_0377"></a>
<h1 class="topictitle1">DLI Console Overview</h1>
<div id="body1532683323732"><p id="dli_01_0377__p19682162013016">The <strong id="dli_01_0377__b535816326619">Overview</strong> page of the DLI console provides you with the DLI workflow and resource usage.</p>
<div class="section" id="dli_01_0377__section7541203715336"><h4 class="sectiontitle">How to Use DLI</h4><p id="dli_01_0377__p1797340143412">The process of using DLI is as follows:</p>
<ol id="dli_01_0377__ol1121119317351"><li id="dli_01_0377__li221114373520"><strong id="dli_01_0377__b17561621973">Create a queue.</strong><p id="dli_01_0377__p664319264367">Queues are DLI's compute resources. There are SQL queues and general-purpose queues. For a SQL queue, you can only submit Spark SQL jobs. For a general-purpose queue, you can submit Spark programs, Flink OpenSource SQL jobs, and Flink Jar jobs.</p>
</li><li id="dli_01_0377__li20271118193517"><strong id="dli_01_0377__b183815436312">Prepare data.</strong><p id="dli_01_0377__p5643825193617">Create databases and tables before you run a Spark SQL job. Upload a program package before you run a Spark job or a Flink Jar job.</p>
</li><li id="dli_01_0377__li18383152593512"><strong id="dli_01_0377__b13693154719103">Edit and submit a job.</strong><p id="dli_01_0377__p372844153816">After you set the job parameters, you can submit the job.</p>
</li><li id="dli_01_0377__li1015653416359"><strong id="dli_01_0377__b166172496311">View job status.</strong><p id="dli_01_0377__p372012313368">Go to the <strong id="dli_01_0377__b9247924181213">Job Management</strong> page to view the job execution status.</p>
<p id="dli_01_0377__p2711644164815"></p>
</li></ol>
<h1 class="topictitle1">Managing Flink Jobs</h1>
<div id="body0000002056966232"><p id="dli_01_0377__p8060118"></p>
</div>
<div class="section" id="dli_01_0377__section613011388443"><h4 class="sectiontitle">Queue Usage (hours)</h4><p id="dli_01_0377__p16401944124415">The overview page provides usage durations of all queues or a specific queue.</p>
<ul id="dli_01_0377__ul1656115212459"><li id="dli_01_0377__li356152114512">Usage Usage (hours): an overview of the usage of all resources.</li><li id="dli_01_0377__li8927630134517">Selected Queue Usage (hours): recent usage of a single queue.</li></ul>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0462.html">Viewing Flink Job Details</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0650.html">Setting the Priority for a Flink Job</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0534.html">Enabling Dynamic Scaling for Flink Jobs</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0651.html">Querying Logs for Flink Jobs</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0461.html">Common Operations of Flink Jobs</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0389.html">Submitting a Flink Job Using DLI</a></div>
</div>
</div>

41
docs/dli/umn/dli_01_0378.html
View File

@ -1,18 +1,19 @@
<a name="dli_01_0378"></a><a name="dli_01_0378"></a>
<h1 class="topictitle1">What Is Data Lake Insight?</h1>
<h1 class="topictitle1">What Is Data Lake Insight</h1>
<div id="body39451090"><div class="section" id="dli_01_0378__section118410576246"><h4 class="sectiontitle">DLI Introduction</h4><p id="dli_01_0378__p104391318592">Data Lake Insight (DLI) is a serverless data processing and analysis service fully compatible with <a href="https://spark.apache.org/" target="_blank" rel="noopener noreferrer">Apache Spark</a> and <a href="https://flink.apache.org/" target="_blank" rel="noopener noreferrer">Apache Flink</a> ecosystems. It frees you from managing any servers.</p>
<p id="dli_01_0378__p35691490142221">DLI supports standard SQL and is compatible with Spark SQL and Flink SQL. It also supports multiple access modes, and is compatible with mainstream data formats. DLI supports SQL statements and Spark applications for heterogeneous data sources, including CloudTable, RDS, GaussDB(DWS), CSS, OBS, custom databases on ECSs, and offline databases.</p>
</div>
<div class="section" id="dli_01_0378__section189215415182"><h4 class="sectiontitle">Functions</h4><p id="dli_01_0378__p3872545124817">You can query and analyze heterogeneous data sources such as RDS, and GaussDB(DWS) on the cloud using access methods, such as visualized interface, RESTful API, JDBC, and Beeline. The data format is compatible with five mainstream data formats: CSV, JSON, Parquet, and ORC.</p>
<ul id="dli_01_0378__ul95252494910"><li id="dli_01_0378__li37017520555">Basic functions<ul id="dli_01_0378__ul33508814556"><li id="dli_01_0378__li4884112014444">You can use standard SQL statements to query in SQL jobs. </li><li id="dli_01_0378__li583064563216">Flink jobs support Flink SQL online analysis. Aggregation functions such as Window and Join, geographic functions, and CEP functions are supported. SQL is used to express service logic, facilitating service implementation. </li><li id="dli_01_0378__li652124194912">For spark jobs, fully-managed Spark computing can be performed. You can submit computing tasks through interactive sessions or in batch to analyze data in the fully managed Spark queues. </li></ul>
</li><li id="dli_01_0378__li13705141165713">Federated analysis of heterogeneous data sources<ul id="dli_01_0378__ul14754404579"><li id="dli_01_0378__li68911658193117">Spark datasource connection: Data sources such as GaussDB(DWS), RDS, and CSS can be accessed through DLI. </li><li id="dli_01_0378__li9919175613471">Interconnection with multiple cloud services is supported in Flink jobs to form a rich stream ecosystem. The DLI stream ecosystem consists of cloud service ecosystems and open source ecosystems.<ul id="dli_01_0378__ul4978181016481"><li id="dli_01_0378__li10978151011487">Cloud service ecosystem: DLI can interconnect with other services in Flink SQL. You can directly use SQL to read and write data from cloud services.</li><li id="dli_01_0378__li4978610154816">Open-source ecosystems: After connections to other VPCs are established through datasource connections, you can access all data sources and output targets (such as Kafka, HBase, and Elasticsearch) supported by Flink and Spark in your dedicated DLI queue.</li></ul>
<ul id="dli_01_0378__ul95252494910"><li id="dli_01_0378__li37017520555">Basic functions<ul id="dli_01_0378__ul33508814556"><li id="dli_01_0378__li4884112014444">You can use standard SQL statements to query in SQL jobs. </li><li id="dli_01_0378__li583064563216">Flink jobs support Flink SQL online analysis capabilities: supporting aggregation functions such as Window and Join, using SQL to express service logic, and achieving service implementation conveniently and quickly. </li><li id="dli_01_0378__li652124194912">For spark jobs, fully-managed Spark computing can be performed. You can submit computing tasks through interactive sessions or in batch to analyze data in the fully managed Spark queues. </li></ul>
</li><li id="dli_01_0378__li13705141165713">Federated analysis of heterogeneous data sources<ul id="dli_01_0378__ul14754404579"><li id="dli_01_0378__li68911658193117">Spark datasource connection: Data sources such as GaussDB(DWS), RDS, and CSS can be accessed through DLI. </li><li id="dli_01_0378__li9919175613471">Interconnection with multiple cloud services is supported in Flink jobs to form a rich stream ecosystem. The DLI stream ecosystem consists of cloud service ecosystems and open source ecosystems.<ul id="dli_01_0378__ul4978181016481"><li id="dli_01_0378__li10978151011487">Cloud service ecosystem: DLI can interconnect with other services in Flink SQL. You can directly use SQL to read and write data from cloud services.</li><li id="dli_01_0378__li4978610154816">Open-source ecosystem: By establishing network connections with other VPCs through enhanced datasource connections, you can access all Flink and Spark-supported data sources and output sources, such as Kafka, Hbase, Elasticsearch, in the tenant-authorized DLI queues.</li></ul>
<p id="dli_01_0378__p11978131024820"></p>
</li></ul>
</li><li id="dli_01_0378__li620462561618">Storage-compute decoupling<p id="dli_01_0378__p12236173712168"><a name="dli_01_0378__li620462561618"></a><a name="li620462561618"></a>DLI is interconnected with OBS for data analysis. In this architecture where storage and compute are decoupled, resources of these two types are charged separately, helping you reduce costs and improving resource utilization.</p>
<p id="dli_01_0378__p656615493515">You can choose single-AZ or multi-AZ storage when you create an OBS bucket for storing redundant data on the DLI console. The differences between the two storage policies are as follows:</p>
<ul id="dli_01_0378__ul1139013284314"><li id="dli_01_0378__li1244173017434">Multi-AZ storage means data is stored in multiple AZs, improving data reliability. If the multi-AZ storage is enabled for a bucket, data is stored in multiple AZs in the same region. If one AZ becomes unavailable, data can still be properly accessed from the other AZs. The multi-AZ storage is ideal for scenarios that demand high reliability. You are advised to use this policy.</li><li id="dli_01_0378__li9190134234418">Single-AZ storage means that data is stored in a single AZ, with lower costs. </li></ul>
</li><li id="dli_01_0378__li145513501110">Elastic resource pool<p id="dli_01_0378__p1330171916542"><a name="dli_01_0378__li145513501110"></a><a name="li145513501110"></a>Elastic resource pools support the CCE cluster architecture for heterogeneous resources so you can centrally manage and allocate them. For details, see <a href="dli_01_0508.html">Elastic Resource Pool</a>.</p>
</li><li id="dli_01_0378__li145513501110">Elastic resource pool<p id="dli_01_0378__p98976332116"><a name="dli_01_0378__li145513501110"></a><a name="li145513501110"></a>The backend of elastic resource pools adopts a CCE cluster architecture, supporting heterogeneous resources, so you can manage and schedule resources in a unified manner.</p>
<p id="dli_01_0378__p1330171916542">For details, see <a href="dli_01_0508.html">Creating an Elastic Resource Pool and Queues Within It</a>.</p>
<p id="dli_01_0378__p3728784154">Elastic resource pools have the following advantages:</p>
<ul id="dli_01_0378__ul12164112181519"><li id="dli_01_0378__li016415219151"><strong id="dli_01_0378__b5154155822516">Unified management</strong><ul id="dli_01_0378__ul1218693571714"><li id="dli_01_0378__li2029283011175">You can manage multiple internal clusters and schedule jobs. You can manage millions of cores for compute resources.</li><li id="dli_01_0378__li498114313170">Elastic resource pools can be deployed across multiple AZs to support high availability.</li></ul>
</li><li id="dli_01_0378__li1150419264155"><strong id="dli_01_0378__b1368615252613">Tenant resource isolation</strong><p id="dli_01_0378__p290771761819">Resources of different queues are isolated to reduce the impact on each other.</p>
@ -22,40 +23,40 @@
</li></ul>
<p id="dli_01_0378__p116431932124919">Using elastic resource pools has the following advantages.</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0378__table194674374565" frame="border" border="1" rules="all"><thead align="left"><tr id="dli_01_0378__row1846743785614"><th align="left" class="cellrowborder" valign="top" width="7.51%" id="mcps1.3.2.3.4.5.1.4.1.1"><p id="dli_01_0378__p1731485951215">Advantage</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0378__table194674374565" frame="border" border="1" rules="all"><thead align="left"><tr id="dli_01_0378__row1846743785614"><th align="left" class="cellrowborder" valign="top" width="7.51%" id="mcps1.3.2.3.4.6.1.4.1.1"><p id="dli_01_0378__p1731485951215">Advantage</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="39.92%" id="mcps1.3.2.3.4.5.1.4.1.2"><p id="dli_01_0378__p19467163745615">No Elastic Resource Pool</p>
<th align="left" class="cellrowborder" valign="top" width="39.92%" id="mcps1.3.2.3.4.6.1.4.1.2"><p id="dli_01_0378__p19467163745615">No Elastic Resource Pool</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="52.56999999999999%" id="mcps1.3.2.3.4.5.1.4.1.3"><p id="dli_01_0378__p12467737115610">Use Elastic Resource Pool</p>
<th align="left" class="cellrowborder" valign="top" width="52.56999999999999%" id="mcps1.3.2.3.4.6.1.4.1.3"><p id="dli_01_0378__p12467737115610">Use Elastic Resource Pool</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0378__row546773765617"><td class="cellrowborder" valign="top" width="7.51%" headers="mcps1.3.2.3.4.5.1.4.1.1 "><p id="dli_01_0378__p931417598126">Efficiency</p>
<tbody><tr id="dli_01_0378__row546773765617"><td class="cellrowborder" valign="top" width="7.51%" headers="mcps1.3.2.3.4.6.1.4.1.1 "><p id="dli_01_0378__p931417598126">Efficiency</p>
</td>
<td class="cellrowborder" valign="top" width="39.92%" headers="mcps1.3.2.3.4.5.1.4.1.2 "><p id="dli_01_0378__p14467193710561">You need to set scaling tasks repeatedly to improve the resource utilization.</p>
<td class="cellrowborder" valign="top" width="39.92%" headers="mcps1.3.2.3.4.6.1.4.1.2 "><p id="dli_01_0378__p14467193710561">You need to set scaling tasks repeatedly to improve the resource utilization.</p>
</td>
<td class="cellrowborder" valign="top" width="52.56999999999999%" headers="mcps1.3.2.3.4.5.1.4.1.3 "><p id="dli_01_0378__p74679377564">Dynamic scaling can be done in seconds.</p>
<td class="cellrowborder" valign="top" width="52.56999999999999%" headers="mcps1.3.2.3.4.6.1.4.1.3 "><p id="dli_01_0378__p74679377564">Dynamic scaling can be done in seconds.</p>
</td>
</tr>
<tr id="dli_01_0378__row10467637105611"><td class="cellrowborder" rowspan="2" valign="top" width="7.51%" headers="mcps1.3.2.3.4.5.1.4.1.1 "><p id="dli_01_0378__p631415913126">Resource utilization</p>
<tr id="dli_01_0378__row10467637105611"><td class="cellrowborder" rowspan="2" valign="top" width="7.51%" headers="mcps1.3.2.3.4.6.1.4.1.1 "><p id="dli_01_0378__p631415913126">Resource utilization</p>
<p id="dli_01_0378__p231913871418"></p>
</td>
<td class="cellrowborder" valign="top" width="39.92%" headers="mcps1.3.2.3.4.5.1.4.1.2 "><p id="dli_01_0378__p5467037195618">Resources cannot be shared among different queues.</p>
<td class="cellrowborder" valign="top" width="39.92%" headers="mcps1.3.2.3.4.6.1.4.1.2 "><p id="dli_01_0378__p5467037195618">Resources cannot be shared among different queues.</p>
<p id="dli_01_0378__p1127911542114">For example, a queue has idle CUs and another queue is heavily loaded. Resources cannot be shared. You can only scale up the second queue.</p>
</td>
<td class="cellrowborder" valign="top" width="52.56999999999999%" headers="mcps1.3.2.3.4.5.1.4.1.3 "><p id="dli_01_0378__p184671437135615">Queues added to the same elastic resource pool can share compute resources.</p>
<td class="cellrowborder" valign="top" width="52.56999999999999%" headers="mcps1.3.2.3.4.6.1.4.1.3 "><p id="dli_01_0378__p184671437135615">Queues added to the same elastic resource pool can share compute resources.</p>
</td>
</tr>
<tr id="dli_01_0378__row1031911831417"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.4.5.1.4.1.1 "><p id="dli_01_0378__p5312181119518">When you set a data source, you must allocate different network segments to each queue, which requires a large number of VPC network segments.</p>
<tr id="dli_01_0378__row1031911831417"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.4.6.1.4.1.1 "><p id="dli_01_0378__p5312181119518">When you set a data source, you must allocate different network segments to each queue, which requires a large number of VPC network segments.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.4.5.1.4.1.2 "><p id="dli_01_0378__p769213815134">You can add multiple general-purpose queues in the same elastic resource pool to one network segment, simplifying the data source configuration.</p>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.4.6.1.4.1.2 "><p id="dli_01_0378__p769213815134">You can add multiple general-purpose queues in the same elastic resource pool to one network segment, simplifying the data source configuration.</p>
</td>
</tr>
<tr id="dli_01_0378__row529116935714"><td class="cellrowborder" valign="top" width="7.51%" headers="mcps1.3.2.3.4.5.1.4.1.1 "><p id="dli_01_0378__p19814163512137">Resource allocation</p>
<tr id="dli_01_0378__row529116935714"><td class="cellrowborder" valign="top" width="7.51%" headers="mcps1.3.2.3.4.6.1.4.1.1 "><p id="dli_01_0378__p19814163512137">Resource allocation</p>
</td>
<td class="cellrowborder" valign="top" width="39.92%" headers="mcps1.3.2.3.4.5.1.4.1.2 "><p id="dli_01_0378__p122925915716">If resources are insufficient for scale-out tasks of multiple queues, some queues will fail to be scaled out.</p>
<td class="cellrowborder" valign="top" width="39.92%" headers="mcps1.3.2.3.4.6.1.4.1.2 "><p id="dli_01_0378__p122925915716">If resources are insufficient for scale-out tasks of multiple queues, some queues will fail to be scaled out.</p>
</td>
<td class="cellrowborder" valign="top" width="52.56999999999999%" headers="mcps1.3.2.3.4.5.1.4.1.3 "><p id="dli_01_0378__p029279145715">You can set the priority for each queue in the elastic resource pool based on the peak hours to ensure proper resource allocation.</p>
<td class="cellrowborder" valign="top" width="52.56999999999999%" headers="mcps1.3.2.3.4.6.1.4.1.3 "><p id="dli_01_0378__p029279145715">You can set the priority for each queue in the elastic resource pool based on the peak hours to ensure proper resource allocation.</p>
</td>
</tr>
</tbody>
@ -63,7 +64,7 @@
</div>
</li></ul>
</div>
<div class="section" id="dli_01_0378__section19222675412"><h4 class="sectiontitle">DLI Core Engine: Spark+Flink</h4><ul id="dli_01_0378__ul14792133255715"><li id="dli_01_0378__li13792103275712">Spark is a unified analysis engine that is ideal for large-scale data processing. It focuses on query, compute, and analysis. DLI optimizes performance and reconstructs services based on open-source Spark. It is compatible with the Apache Spark ecosystem and interfaces, and improves performance by 2.5x when compared with open-source Spark. In this way, DLI enables you to perform query and analysis of EB's of data within hours.</li><li id="dli_01_0378__li14792133295711">Flink is a distributed compute engine that is ideal for batch processing, that is, for processing static data sets and historical data sets. You can also use it for stream processing, that is, processing real-time data streams and generating data results in real time. DLI enhances features and security based on the open-source Flink and provides the Stream SQL feature required for data processing.</li></ul>
<div class="section" id="dli_01_0378__section19222675412"><h4 class="sectiontitle">DLI Core Engine: Spark+Flink+Trino</h4><ul id="dli_01_0378__ul14792133255715"><li id="dli_01_0378__li13792103275712">Spark is a unified analysis engine that is ideal for large-scale data processing. It focuses on query, compute, and analysis. DLI optimizes performance and reconstructs services based on open-source Spark. It is compatible with the Apache Spark ecosystem and interfaces, and improves performance by 2.5x when compared with open-source Spark. In this way, DLI enables you to perform query and analysis of EB's of data within hours.</li><li id="dli_01_0378__li14792133295711">Flink is a distributed compute engine that is ideal for batch processing, that is, for processing static data sets and historical data sets. You can also use it for stream processing, that is, processing real-time data streams and generating data results in real time. DLI enhances features and security based on the open-source Flink and provides the Stream SQL feature required for data processing.</li></ul>
</div>
<div class="section" id="dli_01_0378__section128371951391"><h4 class="sectiontitle">Serverless Architecture</h4><p id="dli_01_0378__p439010110106">DLI is a serverless big data query and analysis service. It has the following advantages:</p>
<ul id="dli_01_0378__ul156203189145"><li id="dli_01_0378__li166201818121418">Auto scaling: DLI ensures you always have enough capacity on hand to deal with any traffic spikes.</li></ul>
@ -74,6 +75,8 @@
</li></ul>
<ul id="dli_01_0378__ul6602184155118"><li id="dli_01_0378__li17602645513">Using APIs<p id="dli_01_0378__p886223541010"><a name="dli_01_0378__li17602645513"></a><a name="li17602645513"></a>If you need to integrate DLI into a third-party system for secondary development, you can call DLI APIs to use the service.</p>
<p id="dli_01_0378__p129419363381">For details, see <a href="https://docs.otc.t-systems.com/data-lake-insight/api-ref/" target="_blank" rel="noopener noreferrer">Data Lake Insight API Reference</a>.</p>
</li><li id="dli_01_0378__li1578124910569">DataArts Studio<p id="dli_01_0378__p168043441434"><a name="dli_01_0378__li1578124910569"></a><a name="li1578124910569"></a>DataArts Studio is a one-stop data operations platform that provides intelligent data lifecycle management. It supports intelligent construction of industrial knowledge libraries and incorporates data foundations such as big data storage, computing, and analysis engines. With DataArts Studio, your company can easily construct end-to-end intelligent data systems. These systems can help eliminate data silos, unify data standards, accelerate data monetization, and promote digital transformation.</p>
<p id="dli_01_0378__p126512525268">Create a data connection on the DataArts Studio management console to access DLI for data analysis.</p>
</li></ul>
</div>
</div>

31
docs/dli/umn/dli_01_0379.html
View File

@ -1,17 +1,24 @@
<a name="dli_01_0379"></a><a name="dli_01_0379"></a>
<h1 class="topictitle1">Job Templates</h1>
<div id="body1532683323732"></div>
<h1 class="topictitle1">Querying Logs for Spark Jobs</h1>
<div id="body1532683323732"><div class="section" id="dli_01_0379__section39941136154214"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0379__p1134604114501">DLI job buckets are used to store temporary data generated during DLI job running, such as job logs and results.</p>
<p id="dli_01_0379__p394165795113">This section describes how to configure a bucket for DLI jobs on the DLI console and obtain Spark job logs.</p>
</div>
<div class="section" id="dli_01_0379__section1599484610524"><h4 class="sectiontitle">Notes</h4><ul id="dli_01_0379__ul16665810152313"><li id="dli_01_0379__li136652107233">To avoid disordered job results, do not use the OBS bucket configured for DLI jobs for any other purposes.</li><li id="dli_01_0379__li9665410172319">DLI jobs must be set and modified by the main account as IAM users do not have required permissions.</li><li id="dli_01_0379__li2665161013239">You cannot view the logs for DLI jobs before configuring a bucket.</li><li id="dli_01_0379__li027416136238">You can configure lifecycle rules to periodically delete objects from buckets or change storage classes of objects.</li><li id="dli_01_0379__li196651510192312">Exercise caution when modifying the job bucket, as it may result in the inability to retrieve historical data.</li><li id="dli_01_0379__li735318419151">Spark log splitting rules:<ul id="dli_01_0379__ul997081719162"><li id="dli_01_0379__li129414813153">Split by size: By default, each log file has a maximum size of 128 MB. </li><li id="dli_01_0379__li612815471611">Split by time: A new log file is automatically created every hour.</li></ul>
</li></ul>
</div>
<div class="section" id="dli_01_0379__section733182634316"><h4 class="sectiontitle">Prerequisites</h4><p id="dli_01_0379__p16611122532011">Before the configuration, create an OBS bucket or parallel file system (PFS). In big data scenarios, you are advised to create a PFS. PFS is a high-performance file system provided by OBS, with access latency in milliseconds. PFS can achieve a bandwidth performance of up to TB/s and millions of IOPS, which makes it ideal for processing high-performance computing (HPC) workloads.</p>
</div>
<div class="section" id="dli_01_0379__section26141553165620"><h4 class="sectiontitle">Configuring a Bucket for DLI Jobs</h4><ol id="dli_01_0379__ol3466216175716"><li id="dli_01_0379__li1216717496572">In the navigation pane of the DLI console, choose <strong id="dli_01_0379__b97932310341">Global Configuration</strong> &gt; <strong id="dli_01_0379__b1979373163416">Project</strong>.</li><li id="dli_01_0379__li11674105415712">On the <strong id="dli_01_0379__b822073012210">Project</strong> page, click <span><img id="dli_01_0379__image1722163014213" src="en-us_image_0000001995796834.png"></span> next to <strong id="dli_01_0379__b132213301820">Job Bucket</strong> to configure bucket information.</li><li id="dli_01_0379__li10719540416">Click <span><img id="dli_01_0379__image744954619597" src="en-us_image_0000002032236253.png"></span> to view available buckets.</li><li id="dli_01_0379__li1058412263820">In the displayed <strong id="dli_01_0379__b1461112401012">OBS</strong> dialog box, click the name of a bucket or search for and click a bucket name and then click <strong id="dli_01_0379__b1536218110318">OK</strong>. In the <strong id="dli_01_0379__b966982915110">Set Job Bucket</strong> dialog box, click <strong id="dli_01_0379__b1087118171117">OK</strong>.<p id="dli_01_0379__p11790738382">Temporary data generated during DLI job running will be stored in the OBS bucket.</p>
</li></ol>
</div>
<div class="section" id="dli_01_0379__section1218555864210"><h4 class="sectiontitle">Querying Logs for Spark Jobs</h4><ol id="dli_01_0379__ol11681726194811"><li id="dli_01_0379__li41681726154813">Log in to the DLI console. In the navigation pane on the left, choose <strong id="dli_01_0379__b15924157129">Job Management</strong> &gt; <strong id="dli_01_0379__b192418591216">Spark Jobs</strong>.</li><li id="dli_01_0379__li141685261489">Select the Spark job whose jobs you want to query, click <strong id="dli_01_0379__b15948215144">More</strong> in the <strong id="dli_01_0379__b1379512511144">Operation</strong> column, and select <strong id="dli_01_0379__b97541146121510">View Log</strong>.<p id="dli_01_0379__p47726278482">The system automatically switches to the log path of the DLI job bucket.</p>
</li><li id="dli_01_0379__li216832612483">On the <strong id="dli_01_0379__b52211803170">Files</strong> tab, select the log file of the desired date and time and click <strong id="dli_01_0379__b82211503176">Download</strong> in the <strong id="dli_01_0379__b1222120171714">Operation</strong> column to download the file to your local host.</li></ol>
</div>
</div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0021.html">Managing SQL Templates</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0464.html">Managing Flink Templates</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0551.html">Managing Spark SQL Templates</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_05110.html">Appendix</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0465.html">Submitting a Spark Job Using DLI</a></div>
</div>
</div>

146
docs/dli/umn/dli_01_0384.html
View File

File diff suppressed because it is too large Load Diff

53
docs/dli/umn/dli_01_0385.html
View File

@ -1,58 +1,53 @@
<a name="dli_01_0385"></a><a name="dli_01_0385"></a>
<h1 class="topictitle1">Spark Job Management</h1>
<div id="body8662426"><p id="dli_01_0385__li13792103275712p0">Based on the open-source Spark, DLI optimizes performance and reconstructs services to be compatible with the Apache Spark ecosystem and interfaces, and executes batch processing tasks.</p>
<p id="dli_01_0385__p1543415146258">DLI also allows you to use Spark jobs to access DLI metadata.</p>
<p id="dli_01_0385__p66161828392">Spark job management provides the following functions:</p>
<ul id="dli_01_0385__ul8657743184112"><li id="dli_01_0385__li135810130187"><a href="dli_01_0384.html">Creating a Spark Job</a></li><li id="dli_01_0385__li19657124354118"><a href="#dli_01_0385__section168728364416">Re-executing a Job</a></li><li id="dli_01_0385__li9471154134119"><a href="#dli_01_0385__section9242154518244">Searching for a Job</a></li><li id="dli_01_0385__li68120361581"><a href="#dli_01_0385__en-us_topic_0122090417_section3753111385816">Terminating a Job</a></li></ul>
<p id="dli_01_0385__p1435019551239">In addition, you can click <strong id="dli_01_0385__b313811421683">Quick Links</strong> to switch to the details on <em id="dli_01_0385__i1713814210814">User Guide</em>.</p>
<div class="section" id="dli_01_0385__section12526165519235"><h4 class="sectiontitle">Spark Jobs Page</h4><p id="dli_01_0385__en-us_topic_0122090417_p66482014175614">On the <strong id="dli_01_0385__b12378211295">Overview</strong> page, click <strong id="dli_01_0385__b33842119911">Spark Jobs</strong> to go to the SQL job management page. Alternatively, you can click <strong id="dli_01_0385__b14384815912">Job Management</strong> &gt; <strong id="dli_01_0385__b203841011191">Spark Jobs</strong>. The page displays all Spark jobs. If there are a large number of jobs, they will be displayed on multiple pages. DLI allows you to view jobs in all statuses.</p>
<h1 class="topictitle1">Managing Spark Jobs</h1>
<div id="body8662426"><div class="section" id="dli_01_0385__section12526165519235"><h4 class="sectiontitle">Viewing Basic Information</h4><p id="dli_01_0385__en-us_topic_0122090417_p66482014175614">On the <strong id="dli_01_0385__b12378211295">Overview</strong> page, click <strong id="dli_01_0385__b33842119911">Spark Jobs</strong> to go to the SQL job management page. Alternatively, you can click <strong id="dli_01_0385__b14384815912">Job Management</strong> &gt; <strong id="dli_01_0385__b203841011191">Spark Jobs</strong>. The page displays all Spark jobs. If there are a large number of jobs, they will be displayed on multiple pages. DLI allows you to view jobs in all statuses.</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0385__en-us_topic_0122090417_table3950169215120" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Job management parameters</caption><thead align="left"><tr id="dli_01_0385__en-us_topic_0122090417_row2555468715120"><th align="left" class="cellrowborder" valign="top" width="16.07%" id="mcps1.3.6.3.2.3.1.1"><p id="dli_01_0385__en-us_topic_0122090417_p4021197415120">Parameter</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0385__en-us_topic_0122090417_table3950169215120" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Job management parameters</caption><thead align="left"><tr id="dli_01_0385__en-us_topic_0122090417_row2555468715120"><th align="left" class="cellrowborder" valign="top" width="16.07%" id="mcps1.3.1.3.2.3.1.1"><p id="dli_01_0385__en-us_topic_0122090417_p4021197415120">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="83.93%" id="mcps1.3.6.3.2.3.1.2"><p id="dli_01_0385__en-us_topic_0122090417_p3594448915120">Description</p>
<th align="left" class="cellrowborder" valign="top" width="83.93%" id="mcps1.3.1.3.2.3.1.2"><p id="dli_01_0385__en-us_topic_0122090417_p3594448915120">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0385__en-us_topic_0122090417_row46758327132"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0385__en-us_topic_0122090417_p16413434141957">Job ID</p>
<tbody><tr id="dli_01_0385__en-us_topic_0122090417_row46758327132"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.1.3.2.3.1.1 "><p id="dli_01_0385__en-us_topic_0122090417_p16413434141957">Job ID</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0385__en-us_topic_0122090417_p54419740141957">ID of a submitted Spark job, which is generated by the system by default.</p>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.1.3.2.3.1.2 "><p id="dli_01_0385__en-us_topic_0122090417_p54419740141957">ID of a submitted Spark job, which is generated by the system by default.</p>
</td>
</tr>
<tr id="dli_01_0385__en-us_topic_0122090417_row32873162171713"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0385__en-us_topic_0122090417_p45480448171713">Name</p>
<tr id="dli_01_0385__en-us_topic_0122090417_row32873162171713"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.1.3.2.3.1.1 "><p id="dli_01_0385__en-us_topic_0122090417_p45480448171713">Name</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0385__en-us_topic_0122090417_p18579134217227">Name of a submitted Spark job.</p>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.1.3.2.3.1.2 "><p id="dli_01_0385__en-us_topic_0122090417_p18579134217227">Name of a submitted Spark job.</p>
</td>
</tr>
<tr id="dli_01_0385__row861981775614"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0385__p126191217125614">Queues</p>
<tr id="dli_01_0385__row861981775614"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.1.3.2.3.1.1 "><p id="dli_01_0385__p126191217125614">Queues</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0385__p448202510569">Queue where the submitted Spark job runs</p>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.1.3.2.3.1.2 "><p id="dli_01_0385__p448202510569">Queue where the submitted Spark job runs</p>
</td>
</tr>
<tr id="dli_01_0385__row1140933010565"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0385__p74098303564">Username</p>
<tr id="dli_01_0385__row1140933010565"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.1.3.2.3.1.1 "><p id="dli_01_0385__p74098303564">Username</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0385__p15101132685817">Name of the user who executed the Spark job</p>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.1.3.2.3.1.2 "><p id="dli_01_0385__p15101132685817">Name of the user who executed the Spark job</p>
</td>
</tr>
<tr id="dli_01_0385__row3110429195719"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0385__p111109294578">Status</p>
<tr id="dli_01_0385__row3110429195719"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.1.3.2.3.1.1 "><p id="dli_01_0385__p111109294578">Status</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0385__p476421517585">Job status. The following values are available:</p>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.1.3.2.3.1.2 "><p id="dli_01_0385__p476421517585">Job status. The following values are available:</p>
<ul id="dli_01_0385__ul14764191515583"><li id="dli_01_0385__li8764121513588"><strong id="dli_01_0385__b135641011868">Starting</strong>: The job is being started.</li><li id="dli_01_0385__li207641415185817"><strong id="dli_01_0385__b1955918121260">Running</strong>: The job is being executed.</li><li id="dli_01_0385__li7764315175820"><strong id="dli_01_0385__b203530131567">Failed</strong>: The session has exited.</li><li id="dli_01_0385__li476471519584"><strong id="dli_01_0385__b8011419617">Finished</strong>: The session is successfully executed.</li><li id="dli_01_0385__li7764191535817"><strong id="dli_01_0385__b1571694014810">Restoring</strong>: The job is being restored.</li></ul>
</td>
</tr>
<tr id="dli_01_0385__en-us_topic_0122090417_row6424839516213"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0385__en-us_topic_0122090417_p50569641162134">Created</p>
<tr id="dli_01_0385__en-us_topic_0122090417_row6424839516213"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.1.3.2.3.1.1 "><p id="dli_01_0385__en-us_topic_0122090417_p50569641162134">Created</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0385__en-us_topic_0122090417_p18910361162145">Time when a job is created. Jobs can be displayed in ascending or descending order of the job creation time.</p>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.1.3.2.3.1.2 "><p id="dli_01_0385__en-us_topic_0122090417_p18910361162145">Time when a job is created. Jobs can be displayed in ascending or descending order of the job creation time.</p>
</td>
</tr>
<tr id="dli_01_0385__row1536633125019"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0385__p145363334505">Last Modified</p>
<tr id="dli_01_0385__row1536633125019"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.1.3.2.3.1.1 "><p id="dli_01_0385__p145363334505">Last Modified</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0385__p153603315013">Time when a job is completed.</p>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.1.3.2.3.1.2 "><p id="dli_01_0385__p153603315013">Time when a job is completed.</p>
</td>
</tr>
<tr id="dli_01_0385__en-us_topic_0122090417_row1662880815250"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0385__en-us_topic_0122090417_p475621615250">Operation</p>
<tr id="dli_01_0385__en-us_topic_0122090417_row1662880815250"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.1.3.2.3.1.1 "><p id="dli_01_0385__en-us_topic_0122090417_p475621615250">Operation</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><ul id="dli_01_0385__en-us_topic_0122090417_ul181927155164"><li id="dli_01_0385__li31578516594"><strong id="dli_01_0385__b105273101889">Edit</strong>: You can modify the current job configuration and re-execute the job.</li><li id="dli_01_0385__li156921658183219"><strong id="dli_01_0385__b1971814139819">SparkUI</strong>: After you click this button, the Spark job execution page is displayed.<div class="note" id="dli_01_0385__note1149352315379"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0385__ul17774418206"><li id="dli_01_0385__li47774118203">The SparkUI page cannot be viewed for jobs in the <strong id="dli_01_0385__b10335047121315">Starting</strong> state.</li><li id="dli_01_0385__li1375744472013">Currently, only the latest 100 job information records are displayed on the SparkUI of DLI.</li></ul>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.1.3.2.3.1.2 "><ul id="dli_01_0385__en-us_topic_0122090417_ul181927155164"><li id="dli_01_0385__li31578516594"><strong id="dli_01_0385__b105273101889">Edit</strong>: You can modify the current job configuration and re-execute the job.</li><li id="dli_01_0385__li156921658183219"><strong id="dli_01_0385__b1971814139819">SparkUI</strong>: After you click this button, the Spark job execution page is displayed.<div class="note" id="dli_01_0385__note1149352315379"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0385__ul17774418206"><li id="dli_01_0385__li47774118203">The SparkUI page cannot be viewed for jobs in the <strong id="dli_01_0385__b10335047121315">Starting</strong> state.</li><li id="dli_01_0385__li1375744472013">Currently, only the latest 100 job information records are displayed on the SparkUI of DLI.</li></ul>
</div></div>
</li><li id="dli_01_0385__li1370757905"><strong id="dli_01_0385__b6248346869">Terminate Job</strong>: Cancel a job that is being started or running.</li><li id="dli_01_0385__li99661755165919"><strong id="dli_01_0385__b109467691584043">Re-execute</strong>: Run the job again.</li><li id="dli_01_0385__li10304115213253"><strong id="dli_01_0385__b10525747115219">Archive Log</strong>: Save job logs to the temporary bucket created by DLI.</li><li id="dli_01_0385__en-us_topic_0122090417_li11155202552811"><strong id="dli_01_0385__b25716541817">Commit Log</strong>: View the logs of submitted jobs.</li><li id="dli_01_0385__li245513563418"><strong id="dli_01_0385__b188631512145">Driver Log</strong>: View the logs of running jobs.</li></ul>
</td>
@ -61,16 +56,16 @@
</table>
</div>
</div>
<div class="section" id="dli_01_0385__section168728364416"><a name="dli_01_0385__section168728364416"></a><a name="section168728364416"></a><h4 class="sectiontitle">Re-executing a Job</h4><p id="dli_01_0385__p1654818201253">On the <span class="wintitle" id="dli_01_0385__wintitle71721024113012"><b>Spark Jobs</b></span> page, click <span class="uicontrol" id="dli_01_0385__uicontrol142071616153014"><b>Edit</b></span> in the <strong id="dli_01_0385__b34941574308">Operation</strong> column of the job. On the Spark job creation page that is displayed, modify parameters as required and execute the job.</p>
<div class="section" id="dli_01_0385__section168728364416"><h4 class="sectiontitle">Re-executing a Job</h4><p id="dli_01_0385__p1654818201253">On the <span class="wintitle" id="dli_01_0385__wintitle71721024113012"><b>Spark Jobs</b></span> page, click <span class="uicontrol" id="dli_01_0385__uicontrol142071616153014"><b>Edit</b></span> in the <strong id="dli_01_0385__b34941574308">Operation</strong> column of the job. On the Spark job creation page that is displayed, modify parameters as required and execute the job.</p>
</div>
<div class="section" id="dli_01_0385__section9242154518244"><a name="dli_01_0385__section9242154518244"></a><a name="section9242154518244"></a><h4 class="sectiontitle">Searching for a Job</h4><p id="dli_01_0385__p8421143910714">On the <span class="wintitle" id="dli_01_0385__wintitle136462051812"><b>Spark Jobs</b></span> page, select <strong id="dli_01_0385__b733422871812">Status</strong> or <strong id="dli_01_0385__b1616814325181">Queues</strong>. The system displays the jobs that meet the filter condition in the job list.</p>
<div class="section" id="dli_01_0385__section9242154518244"><h4 class="sectiontitle">Searching for a Job</h4><p id="dli_01_0385__p8421143910714">On the <span class="wintitle" id="dli_01_0385__wintitle136462051812"><b>Spark Jobs</b></span> page, select <strong id="dli_01_0385__b733422871812">Status</strong> or <strong id="dli_01_0385__b1616814325181">Queues</strong>. The system displays the jobs that meet the filter condition in the job list.</p>
</div>
<div class="section" id="dli_01_0385__en-us_topic_0122090417_section3753111385816"><a name="dli_01_0385__en-us_topic_0122090417_section3753111385816"></a><a name="en-us_topic_0122090417_section3753111385816"></a><h4 class="sectiontitle">Terminating a Job</h4><p id="dli_01_0385__en-us_topic_0122090417_p209202331547">On the <span class="wintitle" id="dli_01_0385__wintitle18382471933"><b>Spark Jobs</b></span> page, choose <strong id="dli_01_0385__b143351153134">More</strong> &gt; <span class="uicontrol" id="dli_01_0385__uicontrol688113571241"><b>Terminate Job</b></span> in the <strong id="dli_01_0385__b7732018513">Operation</strong> column of the job that you want to stop.</p>
<div class="section" id="dli_01_0385__en-us_topic_0122090417_section3753111385816"><h4 class="sectiontitle">Terminating a Job</h4><p id="dli_01_0385__en-us_topic_0122090417_p209202331547">On the <span class="wintitle" id="dli_01_0385__wintitle18382471933"><b>Spark Jobs</b></span> page, choose <strong id="dli_01_0385__b143351153134">More</strong> &gt; <span class="uicontrol" id="dli_01_0385__uicontrol688113571241"><b>Terminate Job</b></span> in the <strong id="dli_01_0385__b7732018513">Operation</strong> column of the job that you want to stop.</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0465.html">Spark Job Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0465.html">Submitting a Spark Job Using DLI</a></div>
</div>
</div>

24
docs/dli/umn/dli_01_0389.html
View File

@ -1,33 +1,23 @@
<a name="dli_01_0389"></a><a name="dli_01_0389"></a>
<h1 class="topictitle1">Flink Job Management</h1>
<h1 class="topictitle1">Submitting a Flink Job Using DLI</h1>
<div id="body1536287138300"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0403.html">Overview</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0403.html">Flink Job Overview</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0479.html">Managing Flink Job Permissions</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0454.html">Preparing Flink Job Data</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0498.html">(Recommended) Creating a Flink OpenSource SQL Job</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0455.html">Creating a Flink SQL Job</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0498.html">Creating a Flink OpenSource SQL Job</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0457.html">Creating a Flink Jar Job</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0461.html">Performing Operations on a Flink Job</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0479.html">Configuring Flink Job Permissions</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0462.html">Flink Job Details</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0377.html">Managing Flink Jobs</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0463.html">Tag Management</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0464.html">Managing Flink Job Templates</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0534.html">Enabling Dynamic Scaling for Flink Jobs</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0463.html">Adding Tags to a Flink Job</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0001.html">Job Management</a></div>
</div>
</div>

28
docs/dli/umn/dli_01_0390.html
View File

@ -1,35 +1,19 @@
<a name="dli_01_0390"></a><a name="dli_01_0390"></a>
<h1 class="topictitle1">Databases and Tables</h1>
<h1 class="topictitle1">Creating Databases and Tables</h1>
<div id="body1536287138300"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0228.html">Overview</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0228.html">Understanding Data Catalogs, Databases, and Tables</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0447.html">Managing Database Permissions</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0005.html">Creating a Database and Table on the DLI Console</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0448.html">Managing Table Permissions</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0008.html">Viewing Table Metadata</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0005.html">Creating a Database or a Table</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0004.html">Managing Database Resources on the DLI Console</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0011.html">Deleting a Database or a Table</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0376.html">Modifying the Owners of Databases and Tables</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0253.html">Importing Data to the Table</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0010.html">Exporting Data from DLI to OBS</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0008.html">Viewing Metadata</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0007.html">Previewing Data</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0552.html">Managing Tags</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0625.html">Managing Table Resources on the DLI Console</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0004.html">Data Management</a></div>
</div>
</div>

4
docs/dli/umn/dli_01_0397.html
View File

@ -1,6 +1,6 @@
<a name="dli_01_0397"></a><a name="dli_01_0397"></a>
<h1 class="topictitle1">Built-in Dependencies</h1>
<h1 class="topictitle1">DLI Built-in Dependencies</h1>
<div id="body1542680042498"><p id="dli_01_0397__p154301714104319">DLI built-in dependencies are provided by the platform by default. In case of conflicts, you do not need to upload them when packaging JAR packages of Spark or Flink Jar jobs.</p>
<div class="section" id="dli_01_0397__section33121419143116"><h4 class="sectiontitle">Spark 3.1.1 Dependencies</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0397__table1965817161359" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Spark 3.1.1 dependencies</caption><thead align="left"><tr id="dli_01_0397__row161951930183510"><th align="left" class="cellrowborder" colspan="3" valign="top" id="mcps1.3.2.2.2.4.1.1"><p id="dli_01_0397__p966119374355">Dependency</p>
@ -2963,7 +2963,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0366.html">Package Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0366.html">Managing Program Packages of Jar Jobs</a></div>
</div>
</div>

141
docs/dli/umn/dli_01_0402.html
View File

File diff suppressed because it is too large Load Diff

83
docs/dli/umn/dli_01_0403.html
View File

@ -1,85 +1,16 @@
<a name="dli_01_0403"></a><a name="dli_01_0403"></a>
<h1 class="topictitle1">Overview</h1>
<div id="body1543376454662"><p id="dli_01_0403__p1866444210407">On the Job Management page of Flink jobs, you can submit a Flink job. Currently, the following job types are supported:</p>
<ul id="dli_01_0403__ul3319024204014"><li id="dli_01_0403__li153191024124015"><strong id="dli_01_0403__b582202818343">Flink SQL</strong> uses SQL statements to define jobs and can be submitted to any general purpose queue.</li><li id="dli_01_0403__li069317403210"><strong id="dli_01_0403__b208655222421">Flink Jar</strong> customizes a JAR package job based on Flink APIs. It runs on dedicated queues.</li></ul>
<p id="dli_01_0403__p66161828392">Flink job management provides the following functions:</p>
<ul id="dli_01_0403__ul8657743184112"><li id="dli_01_0403__li81824019583"><a href="dli_01_0479.html">Managing Flink Job Permissions</a></li><li id="dli_01_0403__li19657124354118"><a href="dli_01_0455.html">Creating a Flink SQL Job</a></li><li id="dli_01_0403__li49002025195814"><a href="dli_01_0457.html">Creating a Flink Jar Job</a></li><li id="dli_01_0403__li321633381117"><a href="dli_01_0461.html#dli_01_0461__section1950210297542">Editing a job</a></li><li id="dli_01_0403__li149164314117"><a href="dli_01_0461.html#dli_01_0461__section20957159163012">Starting a Job</a></li><li id="dli_01_0403__li1918757145613"><a href="dli_01_0461.html#dli_01_0461__section8678193324114">Stopping a Job</a></li><li id="dli_01_0403__li14179161612574"><a href="dli_01_0461.html#dli_01_0461__section1691624195713">Deleting a Job</a></li><li id="dli_01_0403__li529715414492"><a href="dli_01_0461.html#dli_01_0461__section135831511323">Exporting a Job</a></li><li id="dli_01_0403__li12456172025013"><a href="dli_01_0461.html#dli_01_0461__section75781665389">Importing a Job</a></li><li id="dli_01_0403__li7887305574"><a href="dli_01_0461.html#dli_01_0461__section15861321183619">Modifying Name and Description</a></li><li id="dli_01_0403__li131144243198"><a href="dli_01_0461.html#dli_01_0461__section83412445175">Importing to a Savepoint</a></li><li id="dli_01_0403__li5693174816504"><a href="dli_01_0461.html#dli_01_0461__section11401152191015">Triggering a Savepoint</a></li><li id="dli_01_0403__li255018247317"><a href="dli_01_0461.html#dli_01_0461__section48961252113110">Runtime Configuration</a></li><li id="dli_01_0403__li153181731145913"><a href="dli_01_0462.html">Job Details</a></li></ul>
<div class="section" id="dli_01_0403__section12526165519235"><h4 class="sectiontitle">Flink Jobs Page</h4><p id="dli_01_0403__en-us_topic_0122090417_p66482014175614">On the <strong id="dli_01_0403__b5725174712329">Overview</strong> page, click <strong id="dli_01_0403__b16730204714323">Flink Jobs</strong> to go to the Flink job management page. Alternatively, you can choose <strong id="dli_01_0403__b177319476328">Job Management</strong> &gt; <strong id="dli_01_0403__b173184718320">Flink Jobs</strong> from the navigation pane on the left. The page displays all Flink jobs. If there are a large number of jobs, they will be displayed on multiple pages. DLI allows you to view jobs in all statuses.</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0403__en-us_topic_0122090417_table3950169215120" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Job management parameters</caption><thead align="left"><tr id="dli_01_0403__en-us_topic_0122090417_row2555468715120"><th align="left" class="cellrowborder" valign="top" width="16.07%" id="mcps1.3.6.3.2.3.1.1"><p id="dli_01_0403__en-us_topic_0122090417_p4021197415120">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="83.93%" id="mcps1.3.6.3.2.3.1.2"><p id="dli_01_0403__en-us_topic_0122090417_p3594448915120">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0403__en-us_topic_0122090417_row46758327132"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__en-us_topic_0122090417_p16413434141957">ID</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__en-us_topic_0122090417_p54419740141957">ID of a submitted Flink job, which is generated by the system by default.</p>
</td>
</tr>
<tr id="dli_01_0403__en-us_topic_0122090417_row32873162171713"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__en-us_topic_0122090417_p45480448171713">Name</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__en-us_topic_0122090417_p18579134217227">Name of the submitted Flink job.</p>
</td>
</tr>
<tr id="dli_01_0403__row3289513151315"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__p102901113161318">Type</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__p1029001315135">Type of the submitted Flink job. Including:</p>
<ul id="dli_01_0403__ul864114454138"><li id="dli_01_0403__li464144518136">Flink SQL: Flink SQL jobs</li><li id="dli_01_0403__li1896411561765">Flink Jar: Flink Jar jobs</li></ul>
</td>
</tr>
<tr id="dli_01_0403__en-us_topic_0122090417_row31011923151038"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__en-us_topic_0122090417_p10671857151038">Status</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__en-us_topic_0122090417_p59114099151038">Job statuses, including:</p>
<ul id="dli_01_0403__en-us_topic_0122090417_ul32930526154023"><li id="dli_01_0403__li5304438104719"><strong id="dli_01_0403__b1218612714459">Draft</strong></li><li id="dli_01_0403__li230510384473"><strong id="dli_01_0403__b312364144518">Submitting</strong></li><li id="dli_01_0403__li1530543815477"><strong id="dli_01_0403__b10386111119456">Submission failed</strong></li><li id="dli_01_0403__li530483810474"><strong id="dli_01_0403__b54881649194919">Running</strong>: After the job is submitted, a normal result is returned.</li><li id="dli_01_0403__li27807399719"><strong id="dli_01_0403__b343275416497">Running exception</strong>: The job stops running due to an exception.</li><li id="dli_01_0403__li1196810358454"><strong id="dli_01_0403__b194672233472">Downloading</strong></li><li id="dli_01_0403__li194778183469"><strong id="dli_01_0403__b3168162564710">Idle</strong></li><li id="dli_01_0403__li19305838104720"><strong id="dli_01_0403__b5819182654717">Stopping</strong></li><li id="dli_01_0403__li130510384477"><strong id="dli_01_0403__b10747633104717">Stopped</strong></li><li id="dli_01_0403__li1430520388475"><strong id="dli_01_0403__b202671406472">Stopping failed</strong></li><li id="dli_01_0403__li73970453461"><strong id="dli_01_0403__b12394545124713">Creating the savepoint</strong></li><li id="dli_01_0403__li1330553812474"><strong id="dli_01_0403__b158604563497">Completed</strong></li></ul>
</td>
</tr>
<tr id="dli_01_0403__en-us_topic_0122090417_row36301606171658"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__en-us_topic_0122090417_p14394959151048">Description</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__en-us_topic_0122090417_p51238775151048">Description of the submitted Flink job.</p>
</td>
</tr>
<tr id="dli_01_0403__row4736911141810"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__p1973721141811">Username</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__p373781161815">Name of the user who submits a job.</p>
</td>
</tr>
<tr id="dli_01_0403__en-us_topic_0122090417_row6424839516213"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__en-us_topic_0122090417_p50569641162134">Created</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__en-us_topic_0122090417_p18910361162145">Time when a job is created.</p>
</td>
</tr>
<tr id="dli_01_0403__row15840729143612"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__p13841152911367">Started</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__p20841122983612">Time when a Flink job starts to run.</p>
</td>
</tr>
<tr id="dli_01_0403__row1536633125019"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__p145363334505">Duration</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><p id="dli_01_0403__p153603315013">Time consumed by job running.</p>
</td>
</tr>
<tr id="dli_01_0403__en-us_topic_0122090417_row1662880815250"><td class="cellrowborder" valign="top" width="16.07%" headers="mcps1.3.6.3.2.3.1.1 "><p id="dli_01_0403__en-us_topic_0122090417_p475621615250">Operation</p>
</td>
<td class="cellrowborder" valign="top" width="83.93%" headers="mcps1.3.6.3.2.3.1.2 "><ul id="dli_01_0403__en-us_topic_0122090417_ul181927155164"><li id="dli_01_0403__en-us_topic_0122090417_li18623798286"><strong id="dli_01_0403__b692115307259">Edit</strong>: Edit a created job. For details, see <a href="dli_01_0461.html#dli_01_0461__section1950210297542">Editing a Job</a>.</li><li id="dli_01_0403__li016442515152"><strong id="dli_01_0403__b4887113372511">Start</strong>: Start and run a job. For details, see <a href="dli_01_0461.html#dli_01_0461__section20957159163012">Starting a Job</a>.</li><li id="dli_01_0403__li1228521944015"><strong id="dli_01_0403__b1893782782517">More</strong><ul id="dli_01_0403__ul2162826144010"><li id="dli_01_0403__en-us_topic_0122090417_li11155202552811"><strong id="dli_01_0403__b857792212255">FlinkUI</strong>: After you click this button, the Flink job execution page is displayed.<div class="note" id="dli_01_0403__note151024818191"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0403__p5501165371013">When you execute a job on a created queue, the cluster is restarted. It takes about 10 minutes. If you click <strong id="dli_01_0403__b25538517556">FlinkUI</strong> before the cluster is created, an empty <strong id="dli_01_0403__b668185785511">projectID</strong> will be cached. The <strong id="dli_01_0403__b541324185618">FlinkUI</strong> page cannot be displayed.</p>
<p id="dli_01_0403__p181031586197">You are advised to use a dedicated queue so that the cluster will not be released. Alternatively, wait for a while after the job is submitted (the cluster is created), and then check <strong id="dli_01_0403__b890617447568">FlinkUI</strong>.</p>
</div></div>
</li><li id="dli_01_0403__li2385531539"><strong id="dli_01_0403__b1631116258182">Stop</strong>: Stop a Flink job. If this function is unavailable, jobs in the current status cannot be stopped.</li><li id="dli_01_0403__li156921658183219"><strong id="dli_01_0403__b12871121811253">Delete</strong>: Delete a job.<div class="note" id="dli_01_0403__note386711433506"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0403__p16867104316506">A deleted job cannot be restored.</p>
</div></div>
</li><li id="dli_01_0403__li206103185417"><strong id="dli_01_0403__b16895181212519">Modify Name and Description</strong>: You can modify the name and description of a job. For details, see <a href="dli_01_0461.html#dli_01_0461__section15861321183619">Modifying Name and Description</a>.</li><li id="dli_01_0403__li263854454016"><strong id="dli_01_0403__b19473912254">Import Savepoint</strong>: Import the data exported from the original CS job. For details, see <a href="dli_01_0461.html#dli_01_0461__section83412445175">Importing to a Savepoint</a>.</li><li id="dli_01_0403__li108001859451"><strong id="dli_01_0403__b14100464255">Trigger Savepoint</strong>: You can click this button for jobs in the <strong id="dli_01_0403__b1979513473815">Running</strong> status to save the job status. For details, see <a href="dli_01_0461.html#dli_01_0461__section11401152191015">Triggering a Savepoint</a>.</li><li id="dli_01_0403__li104707813616"><strong id="dli_01_0403__b107351826254">Permissions</strong>: You can view the user permissions corresponding to the job and grant permissions to other users. For details, see <a href="dli_01_0479.html">Managing Flink Job Permissions</a>.</li><li id="dli_01_0403__li048918565301"><strong id="dli_01_0403__b1813663111189">Runtime Configuration</strong>: You can enable <strong id="dli_01_0403__b7850337197">Alarm Generation upon Job Exception</strong> and <strong id="dli_01_0403__b13219115315918">Auto Restart upon Exception</strong>. For details, see <a href="dli_01_0461.html#dli_01_0461__section48961252113110">Runtime Configuration</a>.</li></ul>
<h1 class="topictitle1">Flink Job Overview</h1>
<div id="body1543376454662"><p id="dli_01_0403__p197159242011">DLI supports two types of Flink jobs:</p>
<ul id="dli_01_0403__ul880217371911"><li id="dli_01_0403__li580214371818"><strong id="dli_01_0403__b86497450118">Flink OpenSource SQL job:</strong><ul id="dli_01_0403__ul128519571213"><li id="dli_01_0403__li3294854816">It is fully compatible with Flink of the community edition, ensuring that jobs can run smoothly on these Flink versions.</li><li id="dli_01_0403__li1029545413117">DLI Flink has expanded the support for connectors based on Flink of the community edition, supporting Redis and GaussDB(DWS) as new data source types. With this expansion, you can now utilize a wider range of data source types, providing greater flexibility and convenience when working with datasets.</li><li id="dli_01_0403__li82952541118">Flink OpenSource SQL jobs are ideal for scenarios where stream processing logic can be defined and executed through SQL statements. This simplifies stream processing, allowing developers to focus more on implementing service logic.</li></ul>
<p id="dli_01_0403__p1266313112610">For how to create a Flink OpenSource SQL job, see <a href="dli_01_0498.html">Creating a Flink OpenSource SQL Job</a>.</p>
</li><li id="dli_01_0403__li16803183716114"><strong id="dli_01_0403__b204806364211">Flink Jar job:</strong><ul id="dli_01_0403__ul4725424217"><li id="dli_01_0403__li1896195913114">DLI allows you to submit Flink jobs compiled into JAR files, providing higher flexibility and customization capabilities. It is applicable to scenarios where complex data processing is required.</li><li id="dli_01_0403__li1689612598118">If the connectors provided by Flink of the community edition cannot meet specific needs, you can use Jar jobs to implement custom connectors or data processing logic.</li><li id="dli_01_0403__li188962595120">It is ideal for scenarios where user-defined functions (UDFs) or specific library integration are required. You can use the Flink ecosystem to implement advanced stream processing logic and status management.</li></ul>
<p id="dli_01_0403__p1941167142616">For how to create a Flink Jar job, see <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</p>
</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0389.html">Flink Job Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0389.html">Submitting a Flink Job Using DLI</a></div>
</div>
</div>

70
docs/dli/umn/dli_01_0407.html
View File

File diff suppressed because it is too large Load Diff

2
docs/dli/umn/dli_01_0408.html
View File

@ -6,8 +6,6 @@
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0440.html">Overview</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0418.html">Creating an IAM User and Granting Permissions</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0451.html">Creating a Custom Policy</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0417.html">DLI Resources</a></strong><br>

8
docs/dli/umn/dli_01_0410.html
View File

@ -1,16 +1,16 @@
<a name="dli_01_0410"></a><a name="dli_01_0410"></a>
<h1 class="topictitle1">Cross-Source Analysis Development Methods</h1>
<h1 class="topictitle1">Common Development Methods for DLI Cross-Source Analysis</h1>
<div id="body1568974021769"><div class="section" id="dli_01_0410__section168291914161312"><h4 class="sectiontitle">Cross-Source Analysis</h4><p id="dli_01_0410__p5554135617130">If DLI needs to access external data sources, you need to establish enhanced datasource connections to enable the network between DLI and the data sources, and then develop different types of jobs to access the data sources. This is the process of DLI cross-source analysis.</p>
<p id="dli_01_0410__p19151036162116">This section describes how to develop data sources supported by DLI for cross-source analysis.</p>
</div>
<div class="section" id="dli_01_0410__section17130115116195"><h4 class="sectiontitle">Notes</h4><ul id="dli_01_0410__ul571971214216"><li id="dli_01_0410__li1371901292119">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li><li id="dli_01_0410__li167197122213">You are advised to use enhanced datasource connections to connect DLI to data sources.</li></ul>
</div>
<div class="section" id="dli_01_0410__section3279182415532"><h4 class="sectiontitle">DLI Supported Data Sources</h4><p id="dli_01_0410__p19826240125720"><a href="#dli_01_0410__table1771918377534">Table 1</a> lists the data sources supported by DLI. For details about how to use the data sources, see <em id="dli_01_0410__i6759525164318">Data Lake Insight SQL Syntax Reference</em>.</p>
<div class="section" id="dli_01_0410__section3279182415532"><h4 class="sectiontitle">DLI Supported Data Sources</h4><p id="dli_01_0410__p19826240125720"><a href="#dli_01_0410__table1771918377534">Table 1</a> lists the data sources supported by DLI. For how to use the data sources, see the <em id="dli_01_0410__i6759525164318">Data Lake Insight SQL Syntax Reference</em>.</p>
<div class="tablenoborder"><a name="dli_01_0410__table1771918377534"></a><a name="table1771918377534"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0410__table1771918377534" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Supported data sources</caption><thead align="left"><tr id="dli_01_0410__row1671953725316"><th align="left" class="cellrowborder" valign="top" width="22.75%" id="mcps1.3.3.3.2.6.1.1"><p id="dli_01_0410__p1010013484111">Service</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="19.11%" id="mcps1.3.3.3.2.6.1.2"><p id="dli_01_0410__p11719437125318">Spark SQL Job</p>
<th align="left" class="cellrowborder" valign="top" width="19.11%" id="mcps1.3.3.3.2.6.1.2"><p id="dli_01_0410__p11719437125318">SQL Job</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="19.99%" id="mcps1.3.3.3.2.6.1.3"><p id="dli_01_0410__p671983713538">Spark Jar Job</p>
</th>
@ -148,7 +148,7 @@
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Enhanced Datasource Connections</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Configuring the Network Connection Between DLI and Data Sources (Enhanced Datasource Connection)</a></div>
</div>
</div>

7
docs/dli/umn/dli_01_0418.html
View File

@ -9,17 +9,16 @@
<div class="section" id="dli_01_0418__section63665495717"><h4 class="sectiontitle">Process Flow</h4><div class="fignone" id="dli_01_0418__fig4118155455715"><a name="dli_01_0418__fig4118155455715"></a><a name="fig4118155455715"></a><span class="figcap"><b>Figure 1 </b>Process for granting DLI permissions</span><br><span><img id="dli_01_0418__image03801418915" src="en-us_image_0206789726.jpg"></span></div>
<p id="dli_01_0418__p582413330719"></p>
<p id="dli_01_0418__p626243075914"></p>
<ol id="dli_01_0418__ol139492810017"><li id="dli_01_0418__li895020818018"><a name="dli_01_0418__li895020818018"></a><a name="li895020818018"></a>Create a user group and grant the permission to it.<p id="dli_01_0418__p99501985019"><a name="dli_01_0418__li895020818018"></a><a name="li895020818018"></a>Create a user group on the IAM console, and assign the <strong id="dli_01_0418__b163744195013">DLI ReadOnlyAccess</strong> permission to the group.</p>
<ol id="dli_01_0418__ol139492810017"><li id="dli_01_0418__li895020818018"><a name="dli_01_0418__li895020818018"></a><a name="li895020818018"></a>Create a user group and grant the permission to it.<p id="dli_01_0418__p99501985019"><a name="dli_01_0418__li895020818018"></a><a name="li895020818018"></a>Create a user group on the IAM console and grant the <strong id="dli_01_0418__b163744195013">DLI ReadOnlyAccess</strong> permission to it.</p>
</li><li id="dli_01_0418__li59503817014">Create a user and add the user to the user group.<p id="dli_01_0418__p4950882013"><a name="dli_01_0418__li59503817014"></a><a name="li59503817014"></a>Create a user on the IAM console and add the user to the group created in <a href="#dli_01_0418__li895020818018">1</a>.</p>
</li><li id="dli_01_0418__li159506814017">Log in and verify the permission.<p id="dli_01_0418__p29911201511"><a name="dli_01_0418__li159506814017"></a><a name="li159506814017"></a>Log in to the management console using the newly created user, and verify that the user's permissions.</p>
<ul id="dli_01_0418__ul9992304111"><li id="dli_01_0418__li149921301015">Choose <strong id="dli_01_0418__b651616084">Service List</strong> &gt; <strong id="dli_01_0418__b1573796139">Data Lake Insight</strong>. The DLI management console is displayed. If you can view the queue list on the <strong id="dli_01_0418__b86423619315">Queue Management</strong> page but cannot buy DLI queues by clicking <strong id="dli_01_0418__b1701367313">Buy Queue</strong> in the upper right corner (assume that the current permission contains only <strong id="dli_01_0418__b1070193615314">DLI ReadOnlyAccess</strong>), the <strong id="dli_01_0418__b7710363316">DLI ReadOnlyAccess</strong> permission has taken effect.</li><li id="dli_01_0418__li29921500115">Choose any other service in <strong id="dli_01_0418__b688152518442">Service List</strong>. If a message appears indicating that you have insufficient permissions to access the service, the <strong id="dli_01_0418__b17881525164411">DLI ReadOnlyAccess</strong> permission has already taken effect.</li></ul>
<ul id="dli_01_0418__ul9992304111"><li id="dli_01_0418__li149921301015">Choose <strong id="dli_01_0418__b837346639">Service List</strong> &gt; <strong id="dli_01_0418__b1501447806">Data Lake Insight</strong>. The DLI management console is displayed. If you can view the queue list on the <strong id="dli_01_0418__b86423619315">Queue Management</strong> page but cannot buy DLI queues by clicking <strong id="dli_01_0418__b1701367313">Buy Queue</strong> in the upper right corner (assume that the current permission contains only <strong id="dli_01_0418__b1070193615314">DLI ReadOnlyAccess</strong>), the <strong id="dli_01_0418__b7710363316">DLI ReadOnlyAccess</strong> permission has taken effect.</li><li id="dli_01_0418__li29921500115">Choose any other service in <strong id="dli_01_0418__b688152518442">Service List</strong>. If a message appears indicating that you have insufficient permissions to access the service, the <strong id="dli_01_0418__b17881525164411">DLI ReadOnlyAccess</strong> permission has already taken effect.</li></ul>
</li></ol>
<p id="dli_01_0418__p05981420125912"></p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0408.html">Permissions Management</a></div>
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0611.html">Preparations</a></div>
</div>
</div>

93
docs/dli/umn/dli_01_0419.html Normal file
View File

@ -0,0 +1,93 @@
<a name="dli_01_0419"></a><a name="dli_01_0419"></a>
<h1 class="topictitle1">DLI Agency Overview</h1>
<div id="body1557992073881"><div class="section" id="dli_01_0419__section766995911819"><h4 class="sectiontitle">What Is an Agency?</h4><p id="dli_01_0419__p732919591212">Cloud services often interact with each other, with some of which dependent on other services. You can create an agency to delegate DLI to use other cloud services and perform resource O&amp;M on your behalf.</p>
<p id="dli_01_0419__p33476205477">For example, the AK/SK required by DLI Flink jobs is stored in DEW. To allow DLI to access DEW data during job execution, you need to provide an IAM agency to delegate the permissions to perform operations on DEW data to DLI.</p>
<div class="fignone" id="dli_01_0419__fig1966812813338"><span class="figcap"><b>Figure 1 </b>DLI service agency</span><br><span><img id="dli_01_0419__image1767061717334" src="en-us_image_0000001742695104.png"></span></div>
</div>
<div class="section" id="dli_01_0419__section20471205014460"><h4 class="sectiontitle">DLI Agencies</h4><p id="dli_01_0419__p14101142142817">Before using DLI, you are advised to set up DLI agency permissions to ensure the proper functioning of DLI.</p>
<ul id="dli_01_0419__ul15735918104"><li id="dli_01_0419__li47351418100">By default, DLI provides the following agencies: <strong id="dli_01_0419__b2030184531011">dli_admin_agency</strong>, <strong id="dli_01_0419__b13505204714109">dli_management_agency</strong>, and <strong id="dli_01_0419__b152271457121012">dli_data_clean_agency</strong>. The names of these agencies are fixed, but the permissions contained in them can be customized. In other scenarios, you need to create custom agencies. For details about the agencies, see <a href="#dli_01_0419__table966993514116">Table 1</a>.</li><li id="dli_01_0419__li12329172119283">DLI upgrades <strong id="dli_01_0419__b24782383013">dli_admin_agency</strong> to <strong id="dli_01_0419__b74862312308">dli_management_agency</strong> to meet the demand for fine-grained agency permissions management. The new agency has the necessary permissions for datasource operations, notifications, and user authorization operations. For details, see <a href="dli_01_0618.html">Configuring DLI Agency Permissions</a>.</li><li id="dli_01_0419__li37481611899">To use Flink 1.15, Spark 3.3.1 (Spark general queue scenario), or a later version to execute jobs, perform the following operations:<div class="p" id="dli_01_0419__p1858212277911"><a name="dli_01_0419__li37481611899"></a><a name="li37481611899"></a>Create an agency on the IAM console and add the agency information to the job configuration. For details, see <a href="dli_01_0616.html">Creating a Custom DLI Agency</a>.<ul id="dli_01_0419__ul1886232712110"><li id="dli_01_0419__li14355142519111">Common scenarios for creating an agency: DLI is allowed to read and write data from and to OBS, dump logs, and read and write Flink checkpoints. DLI is allowed to access DEW to obtain data access credentials and access catalogs to obtain metadata.</li><li id="dli_01_0419__li13659251418">You cannot use the default agency names <strong id="dli_01_0419__b9396134972910">dli_admin_agency</strong>, <strong id="dli_01_0419__b1632315542293">dli_management_agency</strong>, or <strong id="dli_01_0419__b11632218302">dli_data_clean_agency</strong>. It must be unique.</li></ul>
</div>
</li><li id="dli_01_0419__li19890124102819">If the engine version is earlier than Flink 1.15, <strong id="dli_01_0419__b14468141594018">dli_admin_agency</strong> is used by default during job execution. If the engine version is earlier than Spark 3.3.1, user authentication information (AK/SK and security token) is used during job execution.<p id="dli_01_0419__p098126132812">This means that jobs whose engine versions are earlier than Flink 1.15 or Spark 3.3.1 are not affected by the update of agency permissions and do not require custom agencies.</p>
</li><li id="dli_01_0419__li1773518112105">To maintain compatibility with existing job agency permission requirements, <strong id="dli_01_0419__b16380184273218">dli_admin_agency</strong> will still be listed in the IAM agency list even after the update.</li></ul>
<div class="note" id="dli_01_0419__note876811312109"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0419__ul19959752163312"><li id="dli_01_0419__li119590526334">Only the tenant account or a member account of user group <strong id="dli_01_0419__b1328117482327">admin</strong> can authorize the service.</li><li id="dli_01_0419__li199605524332">Do not delete the agency created by the system by default.</li></ul>
</div></div>
<div class="tablenoborder"><a name="dli_01_0419__table966993514116"></a><a name="table966993514116"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0419__table966993514116" frame="border" border="1" rules="all"><caption><b>Table 1 </b>DLI agencies</caption><thead align="left"><tr id="dli_01_0419__row16693352117"><th align="left" class="cellrowborder" valign="top" width="26.93%" id="mcps1.3.2.5.2.4.1.1"><p id="dli_01_0419__p1766917354117">Permission</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="19.39%" id="mcps1.3.2.5.2.4.1.2"><p id="dli_01_0419__p1553011215712">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="53.68000000000001%" id="mcps1.3.2.5.2.4.1.3"><p id="dli_01_0419__p1866923513114">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0419__row1966943519110"><td class="cellrowborder" valign="top" width="26.93%" headers="mcps1.3.2.5.2.4.1.1 "><p id="dli_01_0419__p66691359112">dli_admin_agency</p>
</td>
<td class="cellrowborder" valign="top" width="19.39%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dli_01_0419__p9530152076">Default agency</p>
</td>
<td class="cellrowborder" valign="top" width="53.68000000000001%" headers="mcps1.3.2.5.2.4.1.3 "><p id="dli_01_0419__p1456314513121">This agency has been discarded and is not recommended. Upgrade the agency to <strong id="dli_01_0419__b2548319389">dli_management_agency</strong> as soon as possible.</p>
<p id="dli_01_0419__p136695356116">For details about how to update an agency, see <a href="dli_01_0618.html">Configuring DLI Agency Permissions</a>.</p>
</td>
</tr>
<tr id="dli_01_0419__row7669835913"><td class="cellrowborder" valign="top" width="26.93%" headers="mcps1.3.2.5.2.4.1.1 "><p id="dli_01_0419__p1266983516117">dli_management_agency</p>
</td>
<td class="cellrowborder" valign="top" width="19.39%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dli_01_0419__p185301421876">Default agency</p>
</td>
<td class="cellrowborder" valign="top" width="53.68000000000001%" headers="mcps1.3.2.5.2.4.1.3 "><p id="dli_01_0419__p156691335711">DLI system agency, which is used to delegate operation permissions to DLI so that DLI can use other cloud services and perform resource O&amp;M operations on your behalf. This agency grants permissions for datasource operations, message notifications, and user authorization operations. For details about the permissions of an agency, see <a href="#dli_01_0419__table1165153511519">Table 2</a>.</p>
</td>
</tr>
<tr id="dli_01_0419__row1966983515110"><td class="cellrowborder" valign="top" width="26.93%" headers="mcps1.3.2.5.2.4.1.1 "><p id="dli_01_0419__p19669103515117">dli_data_clean_agency</p>
</td>
<td class="cellrowborder" valign="top" width="19.39%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dli_01_0419__p553042579">Default agency, which needs to be authorized by users</p>
</td>
<td class="cellrowborder" valign="top" width="53.68000000000001%" headers="mcps1.3.2.5.2.4.1.3 "><p id="dli_01_0419__p51191495504">Data cleanup agency, which is used to clean up data according to the lifecycle of a table and clean up lakehouse table data. You need to create a DLI agency named <strong id="dli_01_0419__b153921625131515">dli_data_clean_agency</strong> on IAM and grant permissions to it.</p>
<p id="dli_01_0419__p176698355112">You need to create an agency and customize permissions for it. However, the agency name is fixed to <strong id="dli_01_0419__b13710613169">dli_data_clean_agency</strong>.</p>
<p id="dli_01_0419__p59345537123">For details about the permission policies of an agency, see <a href="dli_01_0617.html">Agency Permission Policies in Common Scenarios</a>.</p>
</td>
</tr>
<tr id="dli_01_0419__row79411341575"><td class="cellrowborder" valign="top" width="26.93%" headers="mcps1.3.2.5.2.4.1.1 "><p id="dli_01_0419__p694134478">Other custom agencies</p>
</td>
<td class="cellrowborder" valign="top" width="19.39%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dli_01_0419__p79410341178">Custom agency</p>
</td>
<td class="cellrowborder" valign="top" width="53.68000000000001%" headers="mcps1.3.2.5.2.4.1.3 "><p id="dli_01_0419__p568018319811">When using Flink 1.15, Spark 3.3, or a later version to execute jobs, create an agency on the IAM console and add new agency information to the job configuration. For details, see <a href="dli_01_0616.html">Creating a Custom DLI Agency</a>.</p>
<p id="dli_01_0419__p106801534815">Common scenarios for creating an agency: DLI is allowed to read and write data from and to OBS to transfer logs. DLI is allowed to access DEW to obtain data access credentials and access catalogs to obtain metadata.</p>
<p id="dli_01_0419__p36806320815">You cannot use the default agency names <strong id="dli_01_0419__b15582191313303">dli_admin_agency</strong>, <strong id="dli_01_0419__b7582151363013">dli_management_agency</strong>, or <strong id="dli_01_0419__b558261353015">dli_data_clean_agency</strong>. It must be unique.</p>
<p id="dli_01_0419__p10141122461314">For details about the permission policies of an agency, see <a href="dli_01_0617.html">Agency Permission Policies in Common Scenarios</a>.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><a name="dli_01_0419__table1165153511519"></a><a name="table1165153511519"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0419__table1165153511519" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Permissions contained in the dli_management_agency agency</caption><thead align="left"><tr id="dli_01_0419__row15661435115115"><th align="left" class="cellrowborder" valign="top" width="41.949999999999996%" id="mcps1.3.2.6.2.3.1.1"><p id="dli_01_0419__p7735181295714">Policy</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="58.050000000000004%" id="mcps1.3.2.6.2.3.1.2"><p id="dli_01_0419__p11735111217573">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0419__row566435135110"><td class="cellrowborder" valign="top" width="41.949999999999996%" headers="mcps1.3.2.6.2.3.1.1 "><p id="dli_01_0419__p1460918155715">IAM ReadOnlyAccess</p>
</td>
<td class="cellrowborder" valign="top" width="58.050000000000004%" headers="mcps1.3.2.6.2.3.1.2 "><p id="dli_01_0419__p1860988115718">To authorize IAM users who have not logged in to DLI, you need to obtain their information. So, the permissions contained in the <strong id="dli_01_0419__b592212192913">IAM ReadOnlyAccess</strong> policy are required.</p>
</td>
</tr>
<tr id="dli_01_0419__row866535175116"><td class="cellrowborder" valign="top" width="41.949999999999996%" headers="mcps1.3.2.6.2.3.1.1 "><p id="dli_01_0419__p56673516510">DLI Datasource Connections Agency Access</p>
</td>
<td class="cellrowborder" valign="top" width="58.050000000000004%" headers="mcps1.3.2.6.2.3.1.2 "><p id="dli_01_0419__p66613516515">Permissions to access and use VPCs, subnets, routes, and VPC peering connections</p>
</td>
</tr>
<tr id="dli_01_0419__row16623517519"><td class="cellrowborder" valign="top" width="41.949999999999996%" headers="mcps1.3.2.6.2.3.1.1 "><p id="dli_01_0419__p766113525119">DLI Notification Agency Access</p>
</td>
<td class="cellrowborder" valign="top" width="58.050000000000004%" headers="mcps1.3.2.6.2.3.1.2 "><p id="dli_01_0419__p966435155113">Permissions to send notifications through SMN when a job fails to be executed</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0486.html">Configuring an Agency to Allow DLI to Access Other Cloud Services</a></div>
</div>
</div>

Some files were not shown because too many files have changed in this diff Show More