forked from docs/doc-exports
Compare commits
12 Commits
hcdc8
...
update-met
| Author | SHA1 | Date | |
|---|---|---|---|
| c59b149892 | |||
| 9a6a028567 | |||
| 6f818add66 | |||
| 53bc92e1f5 | |||
| 48a9cbde3c | |||
| 904c1aad8b | |||
| 190f09501c | |||
| 06c8ac5e62 | |||
| 60127db659 | |||
| 66d959f7d2 | |||
| 043e229a46 | |||
| 0bfc29fc2c |
@ -1,4 +1,4 @@
|
||||
name: Run HCDC
|
||||
name: Check for illegal characters
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -252,7 +252,7 @@
|
||||
"code":"28"
|
||||
},
|
||||
{
|
||||
"desc":"After adding a risky operation, you can view the risk, enable, edit, disable, or delete the risky operation, or set its priority.The risky operation has been added.Before",
|
||||
"desc":"More than 100 risk types are supported. After adding a risky operation, you can view the risk, enable, edit, disable, or delete the risky operation, or set its priority.T",
|
||||
"product_code":"dbss",
|
||||
"title":"Managing Risky Operations",
|
||||
"uri":"dbss_01_0201.html",
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<div class="section" id="dbss_01_0046__section135491616118"><h4 class="sectiontitle">Constraints and Limitations</h4><p id="dbss_01_0046__p1734941819118">The risky SQL statements can be added to the whitelist in data reports.</p>
|
||||
</div>
|
||||
<div class="section" id="dbss_01_0046__section169851556119"><h4 class="sectiontitle">Adding Scanned SQL Statements</h4><ol id="dbss_01_0046__ol61521721102215"><li id="dbss_01_0046__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0046__li199221335798"><span>Select a region, click <span><img id="dbss_01_0046__dbss_01_0204_dbss_01_0186_image12421104763114" src="en-us_image_0000001074398929.png"></span>, and choose <span class="menucascade" id="dbss_01_0046__dbss_01_0204_dbss_01_0186_menucascade1215612612394"><b><span class="uicontrol" id="dbss_01_0046__dbss_01_0204_dbss_01_0186_uicontrol415682653911">Security</span></b> > <b><span class="uicontrol" id="dbss_01_0046__dbss_01_0204_dbss_01_0186_uicontrol2156182620396">Database Security Service</span></b></span>. The <strong id="dbss_01_0046__dbss_01_0204_dbss_01_0186_b6156926113915">Dashboard</strong> page is displayed.</span></li><li id="dbss_01_0046__li517404379"><span>In the navigation tree on the left, choose <span class="uicontrol" id="dbss_01_0046__uicontrol34301640181319"><b>Data Reports</b></span>. The <span class="wintitle" id="dbss_01_0046__wintitle13430124071314"><b>Data Reports</b></span> page is displayed.</span></li><li id="dbss_01_0046__li168521066525"><span>In the <strong id="dbss_01_0046__b1766594619135">Instance</strong> drop-down list, select the instance whose session information you want to view.</span></li><li id="dbss_01_0046__li843695702418"><span>Click the <span class="wintitle" id="dbss_01_0046__wintitle67612030132515"><b>Statements</b></span> tab to view risky SQL statements.</span></li><li id="dbss_01_0046__li2390182182813"><span>Add SQL statements to the whitelist.</span><p><ul id="dbss_01_0046__ul11601325162815"><li id="dbss_01_0046__li26244718283">Add a single SQL statement.<ol type="a" id="dbss_01_0046__ol1918810318280"><li id="dbss_01_0046__li41899318283">Click <strong id="dbss_01_0046__b9698131532">Add to Whitelist</strong> in the <strong id="dbss_01_0046__b136988315310">Operation</strong> column of the target SQL statement.</li><li id="dbss_01_0046__li17328418182916">In the displayed dialog box, select the database and description of the target SQL statement.</li><li id="dbss_01_0046__li09411942163012">Click <strong id="dbss_01_0046__b1429111015397">OK</strong>.</li></ol>
|
||||
</li><li id="dbss_01_0046__li889182713281">Add SQL statements in batches.<ol type="a" id="dbss_01_0046__ol34111821143716"><li id="dbss_01_0046__li5411172193716">Select the target SQL statement and click <span class="uicontrol" id="dbss_01_0046__uicontrol933126103810"><b>One-Clink Whitelisting</b></span>.</li><li id="dbss_01_0046__li1651864223820">In the displayed dialog box, select the database and description of the target SQL statement.</li><li id="dbss_01_0046__li11895102773919">Click <strong id="dbss_01_0046__dbss_01_0046_b1429111015397">OK</strong>.</li></ol>
|
||||
</li><li id="dbss_01_0046__li889182713281">Add SQL statements in batches.<ol type="a" id="dbss_01_0046__ol34111821143716"><li id="dbss_01_0046__li5411172193716">Select the target SQL statement and click <span class="uicontrol" id="dbss_01_0046__uicontrol933126103810"><b>One-Click Whitelisting</b></span>.</li><li id="dbss_01_0046__li1651864223820">In the displayed dialog box, select the database and description of the target SQL statement.</li><li id="dbss_01_0046__li11895102773919">Click <strong id="dbss_01_0046__dbss_01_0046_b1429111015397">OK</strong>.</li></ol>
|
||||
</li></ul>
|
||||
</p></li></ol>
|
||||
</div>
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
<h1 class="topictitle1">Viewing Tracing Logs</h1>
|
||||
<div id="body1524537597090"><p id="dbss_01_0144__p132501813413">After you enable CTS, the system starts recording operations on DBSS. Operation records for the last seven days can be viewed on the CTS console.</p>
|
||||
<div class="section" id="dbss_01_0144__section159610420302"><h4 class="sectiontitle">Viewing a DBSS Trace on the CTS Console</h4><ol id="dbss_01_0144__ol1194546193110"><li id="dbss_01_0144__li394596113118"><span>Log in to the management console.</span></li><li id="dbss_01_0144__li42331433151114"><span>In the navigation pane on the left, click <span><img id="dbss_01_0144__image7233233121114" src="en-us_image_0000001570429773.png"></span> and choose <strong id="dbss_01_0144__b14615159104620">Management & Deployment</strong> > <strong id="dbss_01_0144__b1361559204612">Cloud Trace Service</strong>.</span></li><li id="dbss_01_0144__li8240924153816"><span>Choose <strong id="dbss_01_0144__b42162954165740">Trace List</strong> in the navigation pane.</span></li><li id="dbss_01_0144__li1199125415539"><span>Click <strong id="dbss_01_0144__b13683746114710">Region</strong> at the top of the <strong id="dbss_01_0144__b1468415465470">Trace List</strong> page to set the corresponding conditions.</span><p><div class="p" id="dbss_01_0144__p941145514539">The following four filters are available:<ul class="subitemlist" id="dbss_01_0144__ul4173195016221"><li id="dbss_01_0144__li25120207165721"><span class="parmname" id="dbss_01_0144__parmname1622918592477"><b>Trace Type</b></span>, <span class="parmname" id="dbss_01_0144__parmname18230859124713"><b>Trace Source</b></span>, <span class="parmname" id="dbss_01_0144__parmname1923185944716"><b>Resource Type</b></span>, and <span class="parmname" id="dbss_01_0144__parmname92316592470"><b>Search By</b></span><ul id="dbss_01_0144__ul138358421566"><li id="dbss_01_0144__li422110403562">Select the filter from the drop-down list. Set <strong id="dbss_01_0144__b842352706163855">Trace Source</strong> to <strong id="dbss_01_0144__b842352706163859">DBSS</strong>.</li><li id="dbss_01_0144__li5224174025618">When you select <strong id="dbss_01_0144__b842352706153249">Trace name</strong> for <strong id="dbss_01_0144__b1803301537153246">Search By</strong>, you also need to select a specific trace name.</li><li id="dbss_01_0144__li222614017560">When you select <strong id="dbss_01_0144__b1369790384153349">Resource ID</strong> for <strong id="dbss_01_0144__b1590770393153349">Search By</strong>, you also need to select or enter a specific resource ID.</li><li id="dbss_01_0144__li1822754014568">When you select <strong id="dbss_01_0144__b565110228153447">Resource name</strong> for <strong id="dbss_01_0144__b1329536783153447">Search By</strong>, you also need to select or enter a specific resource name.</li></ul>
|
||||
<div class="section" id="dbss_01_0144__section159610420302"><h4 class="sectiontitle">Viewing a DBSS Trace on the CTS Console</h4><ol id="dbss_01_0144__ol1194546193110"><li id="dbss_01_0144__li394596113118"><span>Log in to the management console.</span></li><li id="dbss_01_0144__li42331433151114"><span>In the navigation pane on the left, click <span><img id="dbss_01_0144__image7233233121114" src="en-us_image_0000001570429773.png"></span> and choose <strong id="dbss_01_0144__b14615159104620">Management & Governance</strong> > <strong id="dbss_01_0144__b1361559204612">Cloud Trace Service</strong>.</span></li><li id="dbss_01_0144__li8240924153816"><span>Choose <strong id="dbss_01_0144__b42162954165740">Trace List</strong> in the navigation pane.</span></li><li id="dbss_01_0144__li1199125415539"><span>Click <strong id="dbss_01_0144__b13683746114710">Region</strong> at the top of the <strong id="dbss_01_0144__b1468415465470">Trace List</strong> page to set the corresponding conditions.</span><p><div class="p" id="dbss_01_0144__p941145514539">The following four filters are available:<ul class="subitemlist" id="dbss_01_0144__ul4173195016221"><li id="dbss_01_0144__li25120207165721"><span class="parmname" id="dbss_01_0144__parmname1622918592477"><b>Trace Type</b></span>, <span class="parmname" id="dbss_01_0144__parmname18230859124713"><b>Trace Source</b></span>, <span class="parmname" id="dbss_01_0144__parmname1923185944716"><b>Resource Type</b></span>, and <span class="parmname" id="dbss_01_0144__parmname92316592470"><b>Search By</b></span><ul id="dbss_01_0144__ul138358421566"><li id="dbss_01_0144__li422110403562">Select the filter from the drop-down list. Set <strong id="dbss_01_0144__b842352706163855">Trace Source</strong> to <strong id="dbss_01_0144__b842352706163859">DBSS</strong>.</li><li id="dbss_01_0144__li5224174025618">When you select <strong id="dbss_01_0144__b842352706153249">Trace name</strong> for <strong id="dbss_01_0144__b1803301537153246">Search By</strong>, you also need to select a specific trace name.</li><li id="dbss_01_0144__li222614017560">When you select <strong id="dbss_01_0144__b1369790384153349">Resource ID</strong> for <strong id="dbss_01_0144__b1590770393153349">Search By</strong>, you also need to select or enter a specific resource ID.</li><li id="dbss_01_0144__li1822754014568">When you select <strong id="dbss_01_0144__b565110228153447">Resource name</strong> for <strong id="dbss_01_0144__b1329536783153447">Search By</strong>, you also need to select or enter a specific resource name.</li></ul>
|
||||
</li><li id="dbss_01_0144__li16990144143538"><strong id="dbss_01_0144__b842352706153633">Operator</strong>: Select a specific operator (a user other than tenant).</li><li id="dbss_01_0144__li2227630716221"><strong id="dbss_01_0144__b842352706153531">Trace Rating</strong>: Available options include <strong id="dbss_01_0144__b1447794024144642">All trace status</strong>, <strong id="dbss_01_0144__b842352706153558">normal</strong>, <strong id="dbss_01_0144__b84235270615364">warning</strong>, and <strong id="dbss_01_0144__b842352706153613">incident</strong>. You can only select one of them.</li><li id="dbss_01_0144__li2484476616221">In the upper right corner of the page, you can query traces in the last 1 hour, last 1 day, last 1 week, or within a customized period.</li></ul>
|
||||
</div>
|
||||
</p></li><li id="dbss_01_0144__li1326512181411"><span>Click <strong id="dbss_01_0144__b842352706164122">Query</strong>.</span></li><li id="dbss_01_0144__li11445413104011"><span>Click <span><img id="dbss_01_0144__image134427196485" src="en-us_image_0210925109.png"></span> on the left of a trace to expand its details.</span><p><div class="fignone" id="dbss_01_0144__fig17669149112013"><span class="figcap"><b>Figure 1 </b>Expanding trace details</span><br><span><img id="dbss_01_0144__image277616258384" src="en-us_image_0217009691.png"></span></div>
|
||||
|
||||
@ -50,7 +50,15 @@
|
||||
<td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dbss_01_0192__p198613573313">-</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dbss_01_0192__row1319658616304"><td class="cellrowborder" valign="top" width="21%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dbss_01_0192__p6229055916304">Client IP Address or IP Range</p>
|
||||
<tr id="dbss_01_0192__row147581436763"><td class="cellrowborder" valign="top" width="21%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dbss_01_0192__p2075863610615">Exception Client IP Address or IP Range (Optional)</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dbss_01_0192__p18758136465">IP addresses that do not need to be audited.</p>
|
||||
<p id="dbss_01_0192__p561032410711">The IP address can be an IPv4 address (for example, 192.168.1.1) or an IPv6 address (for example, fe80:0000:0000:0000:0000:0000:0000:0000).</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dbss_01_0192__p1316118319715">192.168.0.0</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dbss_01_0192__row1319658616304"><td class="cellrowborder" valign="top" width="21%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dbss_01_0192__p6229055916304">Client IP Address/IP Address Segment (Optional)</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dbss_01_0192__p1237050416304">IP address or IP address range of the client</p>
|
||||
<p id="dbss_01_0192__p52991354131711">The IP address can be an IPv4 address (for example, 192.168.1.1) or an IPv6 address (for example, fe80:0000:0000:0000:0000:0000:0000:0000).</p>
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
<a name="dbss_01_0201"></a><a name="dbss_01_0201"></a>
|
||||
|
||||
<h1 class="topictitle1">Managing Risky Operations</h1>
|
||||
<div id="body1545227594709"><p id="dbss_01_0201__p1268218531877">After adding a risky operation, you can view the risk, enable, edit, disable, or delete the risky operation, or set its priority.</p>
|
||||
<div id="body1545227594709"><p id="dbss_01_0201__p480184717920">More than 100 risk types are supported. After adding a risky operation, you can view the risk, enable, edit, disable, or delete the risky operation, or set its priority.</p>
|
||||
<div class="section" id="dbss_01_0201__section070891116319"><h4 class="sectiontitle">Prerequisites</h4><ul id="dbss_01_0201__ul13013312410"><li id="dbss_01_0201__li113021731940">The risky operation has been added.</li><li id="dbss_01_0201__li19732134261016">Before enabling the risky operation, ensure that its status is <strong id="dbss_01_0201__b11420827113618">Disabled</strong>.</li><li id="dbss_01_0201__li125026531295">Before disabling the risky operation, ensure that its status is <strong id="dbss_01_0201__b159185468365">Enabled</strong>.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dbss_01_0201__section348118488534"><h4 class="sectiontitle">Sets the Priority of the Risky Operation</h4><ol id="dbss_01_0201__ol14485848145320"><li id="dbss_01_0201__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0201__li1723651814598"><span>In the navigation tree on the left, choose <strong id="dbss_01_0201__b5439144625812">Audit Rules</strong>.</span></li><li id="dbss_01_0201__li59251104112"><span>In the <strong id="dbss_01_0201__b1128316104817">Instance</strong> drop-down list, select an instance to set risky operation priority. Click the <strong id="dbss_01_0201__b118081524144811">Risky Operations</strong> tab.</span></li><li id="dbss_01_0201__li1452124825315"><span>In the <strong id="dbss_01_0201__b8916161215516">Operation</strong> column of the desired risky operation, click <strong id="dbss_01_0201__b11917412125114">Set Priority</strong>.</span></li><li id="dbss_01_0201__li452910482536"><span>In the displayed dialog box, select a priority and click <strong id="dbss_01_0201__b16379918134612">OK</strong>.</span></li></ol>
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
<div id="body1544587094894"><p id="dbss_01_0204__p6573544910">After connecting the database to the database audit instance, view the audit statistics, including the database audit information, instance information, and data analysis information.</p>
|
||||
<div class="section" id="dbss_01_0204__section441811405410"><h4 class="sectiontitle">Prerequisites</h4><ul id="dbss_01_0204__ul1416718161018"><li id="dbss_01_0204__li89051957124915">You have applied for a database audit instance and the <strong id="dbss_01_0204__en-us_topic_0144723368_b1155217115215">Status</strong> is <strong id="dbss_01_0204__en-us_topic_0144723368_b185532015214">Running</strong>.</li><li id="dbss_01_0204__li1610855617422">Database audit has been enabled.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dbss_01_0204__section16337113512514"><h4 class="sectiontitle">Procedure</h4><ol id="dbss_01_0204__ol8561155420380"><li id="dbss_01_0204__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0204__li10800202815715"><span>Select a region, click <span><img id="dbss_01_0204__dbss_01_0186_image12421104763114" src="en-us_image_0000001074398929.png"></span>, and choose <span class="menucascade" id="dbss_01_0204__dbss_01_0186_menucascade1215612612394"><b><span class="uicontrol" id="dbss_01_0204__dbss_01_0186_uicontrol415682653911">Security</span></b> > <b><span class="uicontrol" id="dbss_01_0204__dbss_01_0186_uicontrol2156182620396">Database Security Service</span></b></span>. The <strong id="dbss_01_0204__dbss_01_0186_b6156926113915">Dashboard</strong> page is displayed.</span></li><li id="dbss_01_0204__li2020718217812"><span>In the <strong id="dbss_01_0204__b6578132315344">Instance</strong> drop-down list, select the instance whose audit information you want to view.</span></li><li id="dbss_01_0204__li56501633204913"><span>View the overall audit statistics, risk distribution, session statistics, and SQL distribution.</span><p><ul id="dbss_01_0204__ul9293658163018"><li id="dbss_01_0204__li1028161893016">Select <strong id="dbss_01_0204__b14585505710">All databases</strong> or a specified database from the <strong id="dbss_01_0204__b1311513157720">Database</strong> drop-down list to view the statistics about all databases in the instance or a specified database.</li><li id="dbss_01_0204__li17214152618304">Select <strong id="dbss_01_0204__b153781741594">Last 30 minutes</strong>, <strong id="dbss_01_0204__b5811693910">1 hour</strong>, <strong id="dbss_01_0204__b242617121093">Today</strong>, <strong id="dbss_01_0204__b102491717913">7 days</strong>, or <strong id="dbss_01_0204__b13372920891">30 days</strong>, or click <span><img id="dbss_01_0204__image623063410010" src="en-us_image_0000001148018506.png"></span> to customize start time and end time to view the statistics of the specified time range.</li></ul>
|
||||
<div class="section" id="dbss_01_0204__section16337113512514"><h4 class="sectiontitle">Procedure</h4><ol id="dbss_01_0204__ol8561155420380"><li id="dbss_01_0204__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0204__li10800202815715"><span>Select a region, click <span><img id="dbss_01_0204__dbss_01_0186_image12421104763114" src="en-us_image_0000001074398929.png"></span>, and choose <span class="menucascade" id="dbss_01_0204__dbss_01_0186_menucascade1215612612394"><b><span class="uicontrol" id="dbss_01_0204__dbss_01_0186_uicontrol415682653911">Security</span></b> > <b><span class="uicontrol" id="dbss_01_0204__dbss_01_0186_uicontrol2156182620396">Database Security Service</span></b></span>. The <strong id="dbss_01_0204__dbss_01_0186_b6156926113915">Dashboard</strong> page is displayed.</span></li><li id="dbss_01_0204__li2020718217812"><span>In the <strong id="dbss_01_0204__b6578132315344">Instance</strong> drop-down list, select the instance whose audit information you want to view.</span></li><li id="dbss_01_0204__li56501633204913"><span>View the overall audit statistics, risk distribution, session statistics, and SQL distribution.</span><p><ul id="dbss_01_0204__ul9293658163018"><li id="dbss_01_0204__li1028161893016">Select <strong id="dbss_01_0204__b14585505710">All databases</strong> or a specified database from the <strong id="dbss_01_0204__b1311513157720">Database</strong> drop-down list to view the statistics about all databases in the instance or a specified database.</li><li id="dbss_01_0204__li17214152618304">Select <strong id="dbss_01_0204__b1821495112129">Last 30 minutes</strong>, <strong id="dbss_01_0204__b132141651151212">1 hour</strong>, <strong id="dbss_01_0204__b2215105113120">Today</strong>, <strong id="dbss_01_0204__b17215551151216">7 days</strong>, or <strong id="dbss_01_0204__b4215195161212">30 days</strong>, or click <span><img id="dbss_01_0204__image22151051131214" src="en-us_image_0000001148018506.png"></span> to customize start time and end time to view the statistics of the specified time range.</li></ul>
|
||||
<div class="fignone" id="dbss_01_0204__fig077713432352"><span class="figcap"><b>Figure 1 </b>Viewing the audit statistics</span><br><span><img id="dbss_01_0204__image1935219111117" src="en-us_image_0000001293796118.png"></span></div>
|
||||
<div class="fignone" id="dbss_01_0204__fig11444449413"><span class="figcap"><b>Figure 2 </b>Risk distribution</span><br><span><img id="dbss_01_0204__image207866489584" src="en-us_image_0000001173095271.png"></span></div>
|
||||
<div class="fignone" id="dbss_01_0204__fig842781211482"><span class="figcap"><b>Figure 3 </b>Session statistics</span><br><span><img id="dbss_01_0204__image1596162919592" src="en-us_image_0000001127059132.png"></span></div>
|
||||
|
||||
@ -6,8 +6,7 @@
|
||||
|
||||
</div>
|
||||
<div class="section" id="dbss_01_0205__section16337113512514"><h4 class="sectiontitle">Procedure</h4><ol id="dbss_01_0205__ol8561155420380"><li id="dbss_01_0205__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0205__li199221335798"><span>Select a region, click <span><img id="dbss_01_0205__dbss_01_0204_dbss_01_0186_image12421104763114" src="en-us_image_0000001074398929.png"></span>, and choose <span class="menucascade" id="dbss_01_0205__dbss_01_0204_dbss_01_0186_menucascade1215612612394"><b><span class="uicontrol" id="dbss_01_0205__dbss_01_0204_dbss_01_0186_uicontrol415682653911">Security</span></b> > <b><span class="uicontrol" id="dbss_01_0205__dbss_01_0204_dbss_01_0186_uicontrol2156182620396">Database Security Service</span></b></span>. The <strong id="dbss_01_0205__dbss_01_0204_dbss_01_0186_b6156926113915">Dashboard</strong> page is displayed.</span></li><li id="dbss_01_0205__li517404379"><span>In the navigation tree on the left, choose <span class="uicontrol" id="dbss_01_0205__uicontrol58079427133858"><b>Data Reports</b></span>. The <span class="wintitle" id="dbss_01_0205__wintitle99615821433858"><b>Data Reports</b></span> page is displayed.</span></li><li id="dbss_01_0205__li168521066525"><span>In the <strong id="dbss_01_0205__b993520346407">Instance</strong> drop-down list, select the instance whose SQL statement information you want to view.</span></li><li id="dbss_01_0205__li4174122125815"><span>Click the <strong id="dbss_01_0205__b1922342184017">Statements</strong> tab.</span></li><li id="dbss_01_0205__li4880132612226"><span>View SQL statement information.</span><p><div class="fignone" id="dbss_01_0205__fig78811126122218"><span class="figcap"><b>Figure 1 </b>Querying SQL statements</span><br><span><img id="dbss_01_0205__image93064119142" src="en-us_image_0000001294276970.png"></span></div>
|
||||
<p id="dbss_01_0205__p1588120267221">To query a specified SQL statement, perform the following steps:</p>
|
||||
<ul id="dbss_01_0205__ul288192602216"><li id="dbss_01_0205__li17881152616226">Select <strong id="dbss_01_0205__b655917154292">All</strong>, <strong id="dbss_01_0205__b35591015112916">Last 30 minutes</strong>, <strong id="dbss_01_0205__b1555931592919">1 hour</strong>, <strong id="dbss_01_0205__b5559181562915">24 hours</strong>, <strong id="dbss_01_0205__b95591815102915">7 days</strong>, or <strong id="dbss_01_0205__b75597155294">30 days</strong> for <strong id="dbss_01_0205__b195606151290">Time</strong> and click <span><img id="dbss_01_0205__image456081518293" src="en-us_image_0000001625668645.png"></span> to view SQL statements of the specified time range.</li><li id="dbss_01_0205__li1988232622210">Select <strong id="dbss_01_0205__b273084591413">All</strong>, <strong id="dbss_01_0205__b13256184815146">High</strong>, <strong id="dbss_01_0205__b14784758141417">Moderate</strong>, <strong id="dbss_01_0205__b142831821151">Low</strong>, or <strong id="dbss_01_0205__b498519920159">Trusted</strong> for <strong id="dbss_01_0205__b1410613010157">Risk Severity</strong> and click <span><img id="dbss_01_0205__image14453822579" src="en-us_image_0000001626494753.png"></span>. SQL statements of specified severity are displayed in the list.<div class="p" id="dbss_01_0205__p953010369711"><div class="note" id="dbss_01_0205__note1111242193316"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dbss_01_0205__p212144223319">A maximum of 10,000 records can be retrieved in a query.</p>
|
||||
<ul id="dbss_01_0205__ul288192602216"><li id="dbss_01_0205__li1988232622210"><div class="p" id="dbss_01_0205__p953010369711"><a name="dbss_01_0205__li1988232622210"></a><a name="li1988232622210"></a><div class="note" id="dbss_01_0205__note1111242193316"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dbss_01_0205__p212144223319">A maximum of 10,000 records can be retrieved in a query.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
</li></ul>
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
<h1 class="topictitle1">How Do I Back Up the Database Audit Logs?</h1>
|
||||
<div id="body1544666186153"><p id="dbss_01_0226__p18174105916405">Database audit supports manual backup and automatic backup. Audit logs are backed up to OBS. Buckets will be automatically created and will incur a separate bill.</p>
|
||||
<p id="dbss_01_0226__p19970016192810">Perform the following operations to automatically back up audit logs.</p>
|
||||
<div class="section" id="dbss_01_0226__section117835520294"><h4 class="sectiontitle">Automatically Backing Up Database Audit Logs</h4><ol id="dbss_01_0226__en-us_topic_0145057228_ol18449141653"><li id="dbss_01_0226__en-us_topic_0145057228_li1829112494372"><span>Log in to the management console.</span></li><li id="dbss_01_0226__en-us_topic_0145057228_li1723651814598"><span>In the navigation tree on the left, choose <strong id="dbss_01_0226__en-us_topic_0145057228_b1432475172820">Settings</strong>.</span></li><li id="dbss_01_0226__en-us_topic_0145057228_li6202174705919"><span>In the <strong id="dbss_01_0226__en-us_topic_0145057228_b2218329102913">Instance</strong> drop-down list, select the required instance and click the <strong id="dbss_01_0226__en-us_topic_0145057228_b1098142173015">Backup and Restoration</strong> tab.</span></li><li id="dbss_01_0226__en-us_topic_0145057228_li1859172013308"><span>Click <strong id="dbss_01_0226__en-us_topic_0145057228_b203402020155415">Set Automated Backup</strong>. In the displayed dialog box, set the auto backup parameters. <a href="#dbss_01_0226__en-us_topic_0145057228_table18602206309">Table 1</a> describes the parameters. </span><p><div class="fignone" id="dbss_01_0226__en-us_topic_0145057228_fig559320113012"><span class="figcap"><b>Figure 1 </b>Configure Automatic Backup dialog box</span><br><span><img id="dbss_01_0226__en-us_topic_0145057228_image18589115017461" src="en-us_image_0000001294284718.png"></span></div>
|
||||
<div class="section" id="dbss_01_0226__section117835520294"><h4 class="sectiontitle">Automatically Backing Up Database Audit Logs</h4><ol id="dbss_01_0226__en-us_topic_0145057228_ol18449141653"><li id="dbss_01_0226__en-us_topic_0145057228_li1829112494372"><span>Log in to the management console.</span></li><li id="dbss_01_0226__en-us_topic_0145057228_li1723651814598"><span>In the navigation tree on the left, choose <strong id="dbss_01_0226__en-us_topic_0145057228_b1432475172820">Settings</strong>.</span></li><li id="dbss_01_0226__en-us_topic_0145057228_li6202174705919"><span>In the <strong id="dbss_01_0226__en-us_topic_0145057228_b2218329102913">Instance</strong> drop-down list, select the required instance and click the <strong id="dbss_01_0226__en-us_topic_0145057228_b1098142173015">Backup and Restoration</strong> tab.</span></li><li id="dbss_01_0226__en-us_topic_0145057228_li1859172013308"><span>Click <strong id="dbss_01_0226__en-us_topic_0145057228_b089617522219">Set Automated Backup</strong>. In the displayed dialog box, set the auto backup parameters. <a href="#dbss_01_0226__en-us_topic_0145057228_table18602206309">Table 1</a> describes the parameters. </span><p><div class="fignone" id="dbss_01_0226__en-us_topic_0145057228_fig559320113012"><span class="figcap"><b>Figure 1 </b>Configure Automatic Backup dialog box</span><br><span><img id="dbss_01_0226__en-us_topic_0145057228_image18589115017461" src="en-us_image_0000001294284718.png"></span></div>
|
||||
|
||||
<div class="tablenoborder"><a name="dbss_01_0226__en-us_topic_0145057228_table18602206309"></a><a name="en-us_topic_0145057228_table18602206309"></a><table cellpadding="4" cellspacing="0" summary="" id="dbss_01_0226__en-us_topic_0145057228_table18602206309" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="dbss_01_0226__en-us_topic_0145057228_row66082019309"><th align="left" class="cellrowborder" valign="top" width="21%" id="mcps1.3.3.2.4.2.2.2.4.1.1"><p id="dbss_01_0226__en-us_topic_0145057228_p1660112023010">Parameter</p>
|
||||
</th>
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
<h1 class="topictitle1">How Do I View All Alarms in Database Audit?</h1>
|
||||
<div id="body1545297742259"><p id="dbss_01_0234__p8060118">To check the alarms of database audit, perform the following steps:</p>
|
||||
<ol id="dbss_01_0234__ol3976165912213"><li id="dbss_01_0234__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0234__li4990487358"><span id="dbss_01_0234__p1983613271494">To query specified alarms, perform the following steps:</span><p><ul id="dbss_01_0234__dbss_01_0252_ul1149562962616"><li id="dbss_01_0234__dbss_01_0252_li174951829102614">Select <strong id="dbss_01_0234__dbss_01_0252_b9987155142420">Last 30 minutes</strong>, <strong id="dbss_01_0234__dbss_01_0252_b1679416565247">1 hour</strong>, <strong id="dbss_01_0234__dbss_01_0252_b6251152252">24 hours</strong>, <strong id="dbss_01_0234__dbss_01_0252_b119871276255">7 days</strong>, or <strong id="dbss_01_0234__dbss_01_0252_b121311118257">30 days</strong> from the drop-down list, and click <span><img id="dbss_01_0234__dbss_01_0252_image167968241613" src="en-us_image_0000001575677796.png"></span> to view alarms of the specified time range.</li><li id="dbss_01_0234__dbss_01_0252_li204961629132617">Select <strong id="dbss_01_0234__dbss_01_0252_b1075116454468">All</strong>, <strong id="dbss_01_0234__dbss_01_0252_b675224510461">High</strong>, <strong id="dbss_01_0234__dbss_01_0252_b1175214456463">Moderate</strong>, or <strong id="dbss_01_0234__dbss_01_0252_b167532454463">Low</strong> for <strong id="dbss_01_0234__dbss_01_0252_b8753144514461">Risk Severity</strong>. Alarms of specified severity are displayed in the list.</li><li id="dbss_01_0234__dbss_01_0252_li13496729152615">Select an alarm type, and alarms of specified alarm type is displayed in the list.</li></ul>
|
||||
<ol id="dbss_01_0234__ol3976165912213"><li id="dbss_01_0234__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0234__li4990487358"><span id="dbss_01_0234__p1983613271494">To query specified alarms, perform the following steps:</span><p><ul id="dbss_01_0234__dbss_01_0252_ul1149562962616"><li id="dbss_01_0234__dbss_01_0252_li174951829102614">Select <strong id="dbss_01_0234__b1179424335014">Last 30 minutes</strong>, <strong id="dbss_01_0234__b15794154311507">1 hour</strong>, <strong id="dbss_01_0234__b1779494319500">24 hours</strong>, <strong id="dbss_01_0234__b10794144311504">7 days</strong>, or <strong id="dbss_01_0234__b4794184395018">30 days</strong> from the drop-down list, and click <span><img id="dbss_01_0234__dbss_01_0252_image167968241613" src="en-us_image_0000002291173653.png"></span> to view alarms of the specified time range.</li><li id="dbss_01_0234__dbss_01_0252_li204961629132617">Select <strong id="dbss_01_0234__b162671719142717">All</strong>, <strong id="dbss_01_0234__b18267519132713">High</strong>, <strong id="dbss_01_0234__b1026791915277">Moderate</strong>, or <strong id="dbss_01_0234__b19267111910273">Low</strong> for <strong id="dbss_01_0234__b112679197271">Risk Severity</strong>. Alarms of specified severity are displayed in the list.</li><li id="dbss_01_0234__dbss_01_0252_li13496729152615">Select an alarm type, and alarms of specified alarm type is displayed in the list.</li></ul>
|
||||
</p></li></ol>
|
||||
</div>
|
||||
<div>
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<p id="dbss_01_0235__p133740292293">In this case, perform the following operations to troubleshoot the problem:</p>
|
||||
<ul id="dbss_01_0235__ul125631048143312"><li id="dbss_01_0235__li9563144816332"><a href="#dbss_01_0235__section8232320142510">Checking Database Information and Audit Function Settings</a></li><li id="dbss_01_0235__li12299591336"><a href="#dbss_01_0235__section1815818210314">Checking Audited Database Settings</a></li><li id="dbss_01_0235__li177281613413"><a href="#dbss_01_0235__section439162819315">Checking Database Agent Status</a></li><li id="dbss_01_0235__li538318241347"><a href="#dbss_01_0235__section20936182118324">Checking the Security Group Rules of the Database Audit Instance</a></li></ul>
|
||||
</div>
|
||||
<div class="section" id="dbss_01_0235__section8232320142510"><a name="dbss_01_0235__section8232320142510"></a><a name="section8232320142510"></a><h4 class="sectiontitle">Checking Database Information and Audit Function Settings</h4><ol id="dbss_01_0235__ol19428205443415"><li id="dbss_01_0235__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0235__li62917254391"><span>Check whether the database information is correct.</span><p><ul id="dbss_01_0235__ul57626357389"><li id="dbss_01_0235__li20762135143813">If the database information is correct, go to <a href="#dbss_01_0235__li941038153911">3</a>.</li><li id="dbss_01_0235__li1762335143810">If the database information is incorrect, click <strong id="dbss_01_0235__b1289185919195">Delete</strong> to delete the database, and then click <strong id="dbss_01_0235__b108928596194">Add Database</strong> to add the database again.<ul id="dbss_01_0235__ul976223513810"><li id="dbss_01_0235__li77621359383">If the fault is rectified, no further operation is required.</li><li id="dbss_01_0235__li7762203513810">If the problem persists, go to <a href="#dbss_01_0235__li941038153911">3</a>.</li></ul>
|
||||
<div class="section" id="dbss_01_0235__section8232320142510"><a name="dbss_01_0235__section8232320142510"></a><a name="section8232320142510"></a><h4 class="sectiontitle">Checking Database Information and Audit Function Settings</h4><ol id="dbss_01_0235__ol19428205443415"><li id="dbss_01_0235__li1896224713913"><span>Log in to the management console.</span></li><li id="dbss_01_0235__li10761161110533"><span>Select a region, click <span><img id="dbss_01_0235__dbss_01_0186_image12421104763114" src="en-us_image_0000001074398929.png"></span>, and choose <span class="menucascade" id="dbss_01_0235__dbss_01_0186_menucascade1215612612394"><b><span class="uicontrol" id="dbss_01_0235__dbss_01_0186_uicontrol415682653911">Security</span></b> > <b><span class="uicontrol" id="dbss_01_0235__dbss_01_0186_uicontrol2156182620396">Database Security Service</span></b></span>. The <strong id="dbss_01_0235__dbss_01_0186_b6156926113915">Dashboard</strong> page is displayed.</span></li><li id="dbss_01_0235__li174304541364"><span>In the navigation tree on the left, choose <strong id="dbss_01_0235__b10912153281313">Databases</strong>.</span></li><li id="dbss_01_0235__li147246523913"><span>View the database information.</span></li><li id="dbss_01_0235__li62917254391"><span>Check whether the database information is correct.</span><p><ul id="dbss_01_0235__ul57626357389"><li id="dbss_01_0235__li20762135143813">If the database information is correct, go to <a href="#dbss_01_0235__li941038153911">6</a>.</li><li id="dbss_01_0235__li1762335143810">If the database information is incorrect, click <strong id="dbss_01_0235__b1289185919195">Delete</strong> to delete the database, and then click <strong id="dbss_01_0235__b108928596194">Add Database</strong> to add the database again.<ul id="dbss_01_0235__ul976223513810"><li id="dbss_01_0235__li77621359383">If the fault is rectified, no further operation is required.</li><li id="dbss_01_0235__li7762203513810">If the problem persists, go to <a href="#dbss_01_0235__li941038153911">6</a>.</li></ul>
|
||||
</li></ul>
|
||||
</p></li><li id="dbss_01_0235__li941038153911"><a name="dbss_01_0235__li941038153911"></a><a name="li941038153911"></a><span>Check whether the database audit function is enabled.</span><p><ul id="dbss_01_0235__ul5763435133814"><li id="dbss_01_0235__li276314359382">If <strong id="dbss_01_0235__b382782911225">Audit Status</strong> is <strong id="dbss_01_0235__b1832329142217">Enabled</strong>, go to <a href="#dbss_01_0235__section1815818210314">Checking Audited Database Settings</a>.</li><li id="dbss_01_0235__li97631835123819">If <strong id="dbss_01_0235__b149091720238">Audit Status</strong> is <strong id="dbss_01_0235__b149097792314">Disabled</strong>, click <strong id="dbss_01_0235__b6910187122315">Enable</strong> to enable the database audit function.<ul id="dbss_01_0235__ul17763143533816"><li id="dbss_01_0235__li27631535183811">If the fault is rectified, no further operation is required.</li><li id="dbss_01_0235__li13763183503814">If the problem persists, go to <a href="#dbss_01_0235__section1815818210314">Checking Audited Database Settings</a>.</li></ul>
|
||||
</li></ul>
|
||||
|
||||
@ -36,7 +36,7 @@
|
||||
</tr>
|
||||
<tr id="dbss_01_0252__row1666214161896"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.3.2.3.2.2.2.3.1.1 "><p id="dbss_01_0252__p9663171613916">Confirmed Or Not</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="83%" headers="mcps1.3.3.2.3.2.2.2.3.1.2 "><p id="dbss_01_0252__p1666371620915">Confirmation status of an alarm. Click <span><img id="dbss_01_0252__image733125633810" src="en-us_image_0214600463.png"></span> to filter alarms in <strong id="dbss_01_0252__b45531415422">Unconfirmed</strong> or <strong id="dbss_01_0252__b756353194215">Confirmed</strong> state.</p>
|
||||
<td class="cellrowborder" valign="top" width="83%" headers="mcps1.3.3.2.3.2.2.2.3.1.2 "><p id="dbss_01_0252__p1666371620915">Confirmation status of an alarm. </p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dbss_01_0252__row1839683313910"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.3.2.3.2.2.2.3.1.1 "><p id="dbss_01_0252__p639612334919">Description</p>
|
||||
@ -47,8 +47,6 @@
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<div class="p" id="dbss_01_0252__p1553584417558">To query specified alarms, perform the following steps:<ul id="dbss_01_0252__ul1149562962616"><li id="dbss_01_0252__li174951829102614">Select <strong id="dbss_01_0252__b9987155142420">Last 30 minutes</strong>, <strong id="dbss_01_0252__b1679416565247">1 hour</strong>, <strong id="dbss_01_0252__b6251152252">24 hours</strong>, <strong id="dbss_01_0252__b119871276255">7 days</strong>, or <strong id="dbss_01_0252__b121311118257">30 days</strong> from the drop-down list, and click <span><img id="dbss_01_0252__image167968241613" src="en-us_image_0000001575677796.png"></span> to view alarms of the specified time range.</li><li id="dbss_01_0252__li204961629132617">Select <strong id="dbss_01_0252__b1075116454468">All</strong>, <strong id="dbss_01_0252__b675224510461">High</strong>, <strong id="dbss_01_0252__b1175214456463">Moderate</strong>, or <strong id="dbss_01_0252__b167532454463">Low</strong> for <strong id="dbss_01_0252__b8753144514461">Risk Severity</strong>. Alarms of specified severity are displayed in the list.</li><li id="dbss_01_0252__li13496729152615">Select an alarm type, and alarms of specified alarm type is displayed in the list.</li></ul>
|
||||
</div>
|
||||
</p></li></ol>
|
||||
</div>
|
||||
<div class="section" id="dbss_01_0252__section030712235511"><h4 class="sectiontitle">Follow-Up Procedure</h4><div class="p" id="dbss_01_0252__p58451054135516"><div class="note" id="dbss_01_0252__note726013364559"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dbss_01_0252__p162601036125513">You can select multiple alarms to be confirmed and click <strong id="dbss_01_0252__b265185591015">Batch Confirm</strong> to batch confirm alarms.</p>
|
||||
|
||||
@ -2,10 +2,7 @@
|
||||
|
||||
<h1 class="topictitle1">Auditing an RDS DB Instance (Without Agents)</h1>
|
||||
<div id="body8662426"><div class="section" id="dbss_06_0020__en-us_topic_0000001438668561_section921711138525"><h4 class="sectiontitle">Overview</h4><p id="dbss_06_0020__en-us_topic_0000001438668561_p1761614631710">This section describes how to audit the security of a relational database instance. (Applications connected to this DB instance are deployed on ECS.) DBSS can audit certain types of relational databases without installing agents.</p>
|
||||
<ul id="dbss_06_0020__en-us_topic_0000001438668561_ul18625144274212"><li id="dbss_06_0020__en-us_topic_0000001438668561_li562534274216"> </li></ul>
|
||||
<div class="note" id="dbss_06_0020__en-us_topic_0000001438668561_note37158149344"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dbss_06_0020__en-us_topic_0000001438668561_ul4313105418360"><li id="dbss_06_0020__en-us_topic_0000001438668561_li1887763545">DBSS without agents is easy to configure and use, but the following functions are not supported:<ul id="dbss_06_0020__en-us_topic_0000001438668561_ul1344181055413"><li id="dbss_06_0020__en-us_topic_0000001438668561_li16313754133614">Successful and failed login sessions cannot be counted.</li><li id="dbss_06_0020__en-us_topic_0000001438668561_li115681539203914">The port number of the client for accessing the database cannot be obtained.</li></ul>
|
||||
</li><li id="dbss_06_0020__en-us_topic_0000001438668561_li526881565413">GaussDB(DWS) has the permission control policy for the log audit function. Only accounts and users with the <strong id="dbss_06_0020__en-us_topic_0000001438668561_b7976103894318">Security Administrator</strong> permission can enable or disable the DWS database audit function.</li><li id="dbss_06_0020__en-us_topic_0000001438668561_li1141810349569">By default, DDL is disabled in GaussDB. You need to enable it by referring to the GaussDB user manual (audit_system_object = 130023423). In addition, ensure that the date format is yyyy-MM-dd HH:mm: ss+Z (datastyle=ISO,YMD) .</li></ul>
|
||||
</div></div>
|
||||
|
||||
</div>
|
||||
<div class="section" id="dbss_06_0020__en-us_topic_0000001438668561_section687975712555"><h4 class="sectiontitle">Solution Architecture</h4><p id="dbss_06_0020__en-us_topic_0000001438668561_p15908312198">The DBSS instance receives the logs sent from databases, such as certain GaussDB(for MySQL) or RDS for MySQL versions, and saves the logs to its log library for security analysis, aggregation statistics, and compliance analysis.</p>
|
||||
<div class="fignone" id="dbss_06_0020__en-us_topic_0000001438668561_fig181911021314"><span class="figcap"><b>Figure 1 </b>Auditing an RDS DB instance (without agents)</span><br><span><img id="dbss_06_0020__en-us_topic_0000001438668561_image81910213312" src="en-us_image_0000001438417909.png"></span></div>
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 500 B |
Binary file not shown.
|
Before Width: | Height: | Size: 500 B |
|
Before Width: | Height: | Size: 443 B After Width: | Height: | Size: 443 B |
Binary file not shown.
|
Before Width: | Height: | Size: 406 B |
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -28,7 +28,7 @@
|
||||
<tr id="ddm_02_0000__row19792241185513"><td class="cellrowborder" valign="top" width="21%" headers="mcps1.3.1.2.3.2.1.2.3.1.1 "><p id="ddm_02_0000__p684143418318">Schema</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="79%" headers="mcps1.3.1.2.3.2.1.2.3.1.2 "><p id="ddm_02_0000__p78418343314">Schema to be associated with the DDM account. You can select an existing schema from the drop-down list.</p>
|
||||
<p id="ddm_02_0000__p08411134123118">Only the associated schemas can be accessed using the account.</p>
|
||||
<p id="ddm_02_0000__p08411134123118">The account can be used to access only the associated schemas.</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="ddm_02_0000__row0533163934910"><td class="cellrowborder" valign="top" width="21%" headers="mcps1.3.1.2.3.2.1.2.3.1.1 "><p id="ddm_02_0000__p15352390493">Permissions</p>
|
||||
|
||||
@ -193,6 +193,19 @@
|
||||
<td class="cellrowborder" valign="top" width="17.398260173982603%" headers="mcps1.3.3.2.2.7.1.6 "><p id="ddm_03_0051__en-us_topic_0242898940_p8811191331919">1 minute</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="ddm_03_0051__en-us_topic_0242898940_row208978557516"><td class="cellrowborder" valign="top" width="14.72852714728527%" headers="mcps1.3.3.2.2.7.1.1 "><p id="ddm_03_0051__en-us_topic_0242898940_p889755511514">ddm_kernel_fgc_cnt</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="11.738826117388262%" headers="mcps1.3.3.2.2.7.1.2 "><p id="ddm_03_0051__en-us_topic_0242898940_p17897115595111">Kernel Full GC Times</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="27.657234276572346%" headers="mcps1.3.3.2.2.7.1.3 "><p id="ddm_03_0051__en-us_topic_0242898940_p989745595120">Full GC times of each DDM node per minute</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="12.548745125487452%" headers="mcps1.3.3.2.2.7.1.4 "><p id="ddm_03_0051__en-us_topic_0242898940_p9572155316557">≥ 0 counts</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="15.928407159284072%" headers="mcps1.3.3.2.2.7.1.5 "><p id="ddm_03_0051__en-us_topic_0242898940_p7141812568">DDM nodes</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="17.398260173982603%" headers="mcps1.3.3.2.2.7.1.6 "><p id="ddm_03_0051__en-us_topic_0242898940_p1089885555120">1 minute</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="ddm_03_0051__en-us_topic_0242898940_row16956164018564"><td class="cellrowborder" valign="top" width="14.72852714728527%" headers="mcps1.3.3.2.2.7.1.1 "><p id="ddm_03_0051__en-us_topic_0242898940_p149561940185611">ddm_node_status_alarm_code</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="11.738826117388262%" headers="mcps1.3.3.2.2.7.1.2 "><p id="ddm_03_0051__en-us_topic_0242898940_p6956134065611">DDM Node Connectivity</p>
|
||||
@ -206,6 +219,19 @@
|
||||
<td class="cellrowborder" valign="top" width="17.398260173982603%" headers="mcps1.3.3.2.2.7.1.6 "><p id="ddm_03_0051__en-us_topic_0242898940_p159573407563">1 minute</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="ddm_03_0051__en-us_topic_0242898940_row14475135152215"><td class="cellrowborder" valign="top" width="14.72852714728527%" headers="mcps1.3.3.2.2.7.1.1 "><p id="ddm_03_0051__en-us_topic_0242898940_p144110178102">ddm_global_sequence_threshold_exceeded_count</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="11.738826117388262%" headers="mcps1.3.3.2.2.7.1.2 "><p id="ddm_03_0051__en-us_topic_0242898940_p1244141711019">Global Secondary Sequences Beyond Threshold</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="27.657234276572346%" headers="mcps1.3.3.2.2.7.1.3 "><p id="ddm_03_0051__en-us_topic_0242898940_p24411417101018">Number of global sequences whose usage exceeds 75%. Usage = Current value/Maximum value. The default threshold is 75%. The maximum value of a global sequence depends on the global sequence type. For example, the maximum value of BIGINT is 2^63-1.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="12.548745125487452%" headers="mcps1.3.3.2.2.7.1.4 "><p id="ddm_03_0051__en-us_topic_0242898940_p5441161711010">≥ 0 counts</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="15.928407159284072%" headers="mcps1.3.3.2.2.7.1.5 "><p id="ddm_03_0051__en-us_topic_0242898940_p761019376118">DDM instances</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="17.398260173982603%" headers="mcps1.3.3.2.2.7.1.6 "><p id="ddm_03_0051__en-us_topic_0242898940_p104411417101016">10 minutes</p>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
@ -77,7 +77,7 @@
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.4.1.2 "><p id="ddm_04_0025__p11321172519219">After a physical table is deleted, data stored on DDM will be lost. The corresponding logical table becomes unavailable on DDM.</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="ddm_04_0025__row124737268816"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.4.1.1 "><p id="ddm_04_0025__p114738261289">Modifying the name of a physical table created on DDM</p>
|
||||
<tr id="ddm_04_0025__row124737268816"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.4.1.1 "><p id="ddm_04_0025__p114738261289">Changing the name of a physical table created on DDM</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.4.1.2 "><p id="ddm_04_0025__p1147318261083">DDM cannot obtain data of the corresponding logical table, and the logical table becomes unavailable on DDM.</p>
|
||||
</td>
|
||||
|
||||
@ -12,6 +12,8 @@
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="ddm_05_0008.html">Resetting the Password of an Account</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="ddm_05_0020.html">Managing Permissions</a></strong><br>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
<h1 class="topictitle1">Creating an Account</h1>
|
||||
<div id="body1585554409299"><div class="section" id="ddm_05_0002__section096911171979"><h4 class="sectiontitle">Prerequisites</h4><ul id="ddm_05_0002__ul17317123816495"><li id="ddm_05_0002__li14317133816491">You have logged in to the DDM console.</li><li id="ddm_05_0002__li16981210165518">There are schemas available in the DDM instance that you want to create an account for.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="ddm_05_0002__section18952118124916"><h4 class="sectiontitle">Procedure</h4><ol id="ddm_05_0002__ol172601822134919"><li id="ddm_05_0002__li18845223192514"><span>In the instance list, locate the required DDM instance and click its name.</span></li><li id="ddm_05_0002__li16917753172513"><span>In the navigation pane, choose <strong id="ddm_05_0002__b1293162314445">Accounts</strong>.</span></li><li id="ddm_05_0002__li12604227495"><span>On the displayed page, click <strong id="ddm_05_0002__b139231941201012">Create Account</strong> and configure the required parameters.</span><p><p id="ddm_05_0002__p029754414612"></p>
|
||||
<div class="section" id="ddm_05_0002__section18952118124916"><h4 class="sectiontitle">Procedure</h4><ol id="ddm_05_0002__ol172601822134919"><li id="ddm_05_0002__li18845223192514"><span>In the instance list, locate the required DDM instance and click its name.</span></li><li id="ddm_05_0002__li16917753172513"><span>In the navigation pane, choose <strong id="ddm_05_0002__b1293162314445">Accounts</strong>.</span></li><li id="ddm_05_0002__li12604227495"><span>On the displayed page, click <strong id="ddm_05_0002__b765035713517">Create Account</strong> and configure the required parameters.</span><p><p id="ddm_05_0002__p029754414612"></p>
|
||||
|
||||
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ddm_05_0002__table12803338145112" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Required parameters</caption><thead align="left"><tr id="ddm_05_0002__row28007385516"><th align="left" class="cellrowborder" valign="top" width="21%" id="mcps1.3.2.2.3.2.2.2.3.1.1"><p id="ddm_05_0002__p9800133815110"><strong id="ddm_05_0002__b4303142234619">Parameter</strong></p>
|
||||
</th>
|
||||
@ -20,7 +20,7 @@
|
||||
<tr id="ddm_05_0002__row1480113811519"><td class="cellrowborder" valign="top" width="21%" headers="mcps1.3.2.2.3.2.2.2.3.1.1 "><p id="ddm_05_0002__p10801638175118">Password</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="79%" headers="mcps1.3.2.2.3.2.2.2.3.1.2 "><p id="ddm_05_0002__p78011038175112">Password of the account. The password:</p>
|
||||
<ul id="ddm_05_0002__ul6801153819514"><li id="ddm_05_0002__li1380123835110">Can include 8 to 32 characters.</li><li id="ddm_05_0002__li568620181711">Must contain at least three of the following character types: lowercase letters, uppercase letters, digits, and special characters ~!@#%^*-_=+?</li><li id="ddm_05_0002__li211322910598">Cannot be a weak password. It cannot be overly simple and easily guessed.</li><li id="ddm_05_0002__li4388184703919">Cannot be the username or the username spelled backwards.</li></ul>
|
||||
<ul id="ddm_05_0002__ul6801153819514"><li id="ddm_05_0002__li1380123835110">Can include 8 to 32 characters.</li><li id="ddm_05_0002__li568620181711">Must contain at least three of the following character types: lowercase letters, uppercase letters, digits, and special characters ~!@#%^*-_+?</li><li id="ddm_05_0002__li211322910598">Cannot be a weak password. It cannot be overly simple and easily guessed.</li><li id="ddm_05_0002__li4388184703919">Cannot be the username or the username spelled backwards.</li></ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="ddm_05_0002__row158011385516"><td class="cellrowborder" valign="top" width="21%" headers="mcps1.3.2.2.3.2.2.2.3.1.1 "><p id="ddm_05_0002__p780153835111">Confirm Password</p>
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
<h1 class="topictitle1">Resetting the Password of an Account</h1>
|
||||
<div id="body0000001086698804"><div class="section" id="ddm_05_0008__section1539512419299"><h4 class="sectiontitle">Prerequisites</h4><ul id="ddm_05_0008__ul02984652012"><li id="ddm_05_0008__li192997652019">You have logged in to the DDM console.</li><li id="ddm_05_0008__li5299196112012">Resetting the DDM account password is a high-risk operation. Ensure that you have the IAM permission to modify DDM accounts.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="ddm_05_0008__section766111524611"><h4 class="sectiontitle">Procedure</h4><ol id="ddm_05_0008__ol172601822134919"><li id="ddm_05_0008__li18845223192514"><span>In the instance list, locate the DDM instance with the account whose password you want to reset and click its name.</span></li><li id="ddm_05_0008__li16917753172513"><span>In the navigation pane, choose <strong id="ddm_05_0008__b13301310183215">Accounts</strong>.</span></li><li id="ddm_05_0008__li877881015353"><span>In the account list, locate the required account and choose <strong id="ddm_05_0008__b197356131329">More</strong> > <strong id="ddm_05_0008__b473711311327">Reset Password</strong> in the <strong id="ddm_05_0008__b1173781373219">Operation</strong> column.</span></li><li id="ddm_05_0008__li31613483616"><span>In the displayed dialog box, enter the new password, confirm the new password, and click <strong id="ddm_05_0008__b981514919337">OK</strong>.</span></li></ol>
|
||||
<div class="section" id="ddm_05_0008__section766111524611"><h4 class="sectiontitle">Procedure</h4><ol id="ddm_05_0008__ol172601822134919"><li id="ddm_05_0008__li18845223192514"><span>In the instance list, locate the DDM instance with the account whose password you want to reset and click its name.</span></li><li id="ddm_05_0008__li16917753172513"><span>In the navigation pane, choose <strong id="ddm_05_0008__b13301310183215">Accounts</strong>.</span></li><li id="ddm_05_0008__li877881015353"><span>In the account list, locate the required account and choose <strong id="ddm_05_0008__b1588295217526">More</strong> > <strong id="ddm_05_0008__b188822523522">Reset Password</strong> in the <strong id="ddm_05_0008__b888255210523">Operation</strong> column.</span></li><li id="ddm_05_0008__li31613483616"><span>In the displayed dialog box, enter the new password, confirm the new password, and click <strong id="ddm_05_0008__b981514919337">OK</strong>.</span></li></ol>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
|
||||
19
docs/ddm/umn/ddm_05_0020.html
Normal file
19
docs/ddm/umn/ddm_05_0020.html
Normal file
@ -0,0 +1,19 @@
|
||||
<a name="ddm_05_0020"></a><a name="ddm_05_0020"></a>
|
||||
|
||||
<h1 class="topictitle1">Managing Permissions</h1>
|
||||
<div id="body0000001175237459"></div>
|
||||
<div>
|
||||
<ul class="ullinks">
|
||||
<li class="ulchildlink"><strong><a href="ddm_12_0002.html">Account Permissions</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="ddm_12_0003.html">Account Requirements</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="ddm_12_0004.html">Managing Permissions</a></strong><br>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="ddm_05_0001.html">Account Management</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
<div class="note" id="ddm_05_0134__note1442122410492"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="ddm_05_0134__p144315241499">Deleted accounts cannot be recovered. Exercise caution when performing this operation.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
<div class="section" id="ddm_05_0134__section1024831334916"><h4 class="sectiontitle">Procedure</h4><ol id="ddm_05_0134__ol172601822134919"><li id="ddm_05_0134__li18845223192514"><span>In the instance list, locate the DDM instance with the account that you want to delete and click its name.</span></li><li id="ddm_05_0134__li16917753172513"><span>In the navigation pane, choose <strong id="ddm_05_0134__b1343101215226">Accounts</strong>.</span></li><li id="ddm_05_0134__li877881015353"><span>In the account list, locate the account that you want to delete and choose <strong id="ddm_05_0134__b91905541103">More</strong> > <strong id="ddm_05_0134__b319521517221">Delete</strong> in the <strong id="ddm_05_0134__b119613153224">Operation</strong> column.</span></li><li id="ddm_05_0134__li31613483616"><span>In the displayed dialog box, click <span class="uicontrol" id="ddm_05_0134__uicontrol16741322102218"><b>Yes</b></span>.</span></li></ol>
|
||||
<div class="section" id="ddm_05_0134__section1024831334916"><h4 class="sectiontitle">Procedure</h4><ol id="ddm_05_0134__ol172601822134919"><li id="ddm_05_0134__li18845223192514"><span>In the instance list, locate the DDM instance with the account that you want to delete and click its name.</span></li><li id="ddm_05_0134__li16917753172513"><span>In the navigation pane, choose <strong id="ddm_05_0134__b94571240145215">Accounts</strong>.</span></li><li id="ddm_05_0134__li877881015353"><span>In the account list, locate the account that you want to delete and choose <strong id="ddm_05_0134__b13897442185211">More</strong> > <strong id="ddm_05_0134__b2897442155218">Delete</strong> in the <strong id="ddm_05_0134__b4897342135216">Operation</strong> column.</span></li><li id="ddm_05_0134__li31613483616"><span>In the displayed dialog box, click <span class="uicontrol" id="ddm_05_0134__uicontrol16741322102218"><b>Yes</b></span>.</span></li></ol>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
</div>
|
||||
<div class="section" id="ddm_06_0007__s2e6dd0a283024771aff961abc306652a"><h4 class="sectiontitle">Precautions</h4><p id="ddm_06_0007__p598643802114">The destination DDM instance has no schemas with the same name.</p>
|
||||
</div>
|
||||
<div class="section" id="ddm_06_0007__section862114653117"><h4 class="sectiontitle">Procedure</h4><ol id="ddm_06_0007__ol12323258"><li id="ddm_06_0007__li1135718381115"><span>Log in to the DDM console, in the instance list, locate the DDM instance that you want to import schema information into and click its name.</span></li><li id="ddm_06_0007__li58659836"><span>On the displayed page, in the navigation pane, choose <strong id="ddm_06_0007__b54061697557">Schemas</strong>.</span></li><li id="ddm_06_0007__li3554164620431"><span>On the displayed page, click <strong id="ddm_06_0007__b91206152555">Import Schema Information</strong>.</span></li><li id="ddm_06_0007__li144106359409"><span>On the displayed page, click <strong id="ddm_06_0007__b1616517267243">Select File</strong> to select the required JSON file which has been exported in <a href="ddm_06_0015.html">Exporting Schema Information</a>.</span></li><li id="ddm_06_0007__li11342421162014"><span>Select the required data nodes, enter a database account with required permissions, and click <strong id="ddm_06_0007__b14811013192615">Finish</strong>.</span><p><div class="note" id="ddm_06_0007__note1089312310479"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="ddm_06_0007__ul1549091153420"><li id="ddm_06_0007__li1249081193415">The number of selected data nodes is the same as the number of data nodes imported into the DDM instance.</li><li id="ddm_06_0007__li104903163416">Required permissions: SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER WITH GRANT OPTION<p id="ddm_06_0007__p590985054416"><a name="ddm_06_0007__li104903163416"></a><a name="li104903163416"></a>You can create a database account for the RDS for MySQL instance and assign it the above permissions in advance.</p>
|
||||
<div class="section" id="ddm_06_0007__section862114653117"><h4 class="sectiontitle">Procedure</h4><ol id="ddm_06_0007__ol12323258"><li id="ddm_06_0007__li1135718381115"><span>Log in to the DDM console, in the instance list, locate the DDM instance that you want to import schema information into and click its name.</span></li><li id="ddm_06_0007__li58659836"><span>On the displayed page, in the navigation pane, choose <strong id="ddm_06_0007__b54061697557">Schemas</strong>.</span></li><li id="ddm_06_0007__li3554164620431"><span>On the displayed page, click <strong id="ddm_06_0007__b91206152555">Import Schema Information</strong>.</span></li><li id="ddm_06_0007__li144106359409"><span>On the displayed page, click <strong id="ddm_06_0007__b1616517267243">Select File</strong> to select the required JSON file which has been exported in <a href="ddm_06_0015.html">Exporting Schema Information</a>.</span></li><li id="ddm_06_0007__li11342421162014"><span>Select the required data nodes, enter a database account with required permissions, and click <strong id="ddm_06_0007__b1179929205115">Finish</strong>.</span><p><div class="note" id="ddm_06_0007__note1089312310479"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="ddm_06_0007__ul1549091153420"><li id="ddm_06_0007__li1249081193415">The number of selected data nodes is the same as the number of data nodes imported into the DDM instance.</li><li id="ddm_06_0007__li104903163416">Required permissions: SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER WITH GRANT OPTION<p id="ddm_06_0007__p590985054416"><a name="ddm_06_0007__li104903163416"></a><a name="li104903163416"></a>You can create a database account for the RDS for MySQL instance and assign it the above permissions in advance.</p>
|
||||
</li></ul>
|
||||
</div></div>
|
||||
</p></li></ol>
|
||||
|
||||
@ -5,11 +5,11 @@
|
||||
<p id="ddm_06_0017__p46473114585">When you delete a schema by mistake or your RDS for MySQL instances become abnormal, metadata restoration allows you to restore your DDM instance metadata and match the metadata with the RDS instance that has completed PITR to re-establish the relationship between your DDM instance and RDS instance. Metadata restoration supports only RDS for MySQL.</p>
|
||||
<p id="ddm_06_0017__p533912361448">To restore metadata of a DDM instance, you can specify a point in time by referring to <a href="#ddm_06_0017__section128315484524">Restoring Metadata to a Point in Time</a>, or using an available backup by referring to <a href="#ddm_06_0017__section132806221525">Restoring Metadata Using Backups</a>.</p>
|
||||
<p id="ddm_06_0017__p115306341696">Metadata restoration mainly restores the metadata of your DDM instance to a new DDM instance. It starts after a point-in-time recovery (PITR) for the associated data nodes is complete.</p>
|
||||
<div class="section" id="ddm_06_0017__section738494914255"><h4 class="sectiontitle">Prerequisites</h4><ul id="ddm_06_0017__ul935817253264"><li id="ddm_06_0017__li23588251267">There is a source DDM instance available and associated with an RDS for MySQL instance. This DB instance is the source data node.</li><li id="ddm_06_0017__li10969101072817">The source data node has been restored to a specified time point.</li></ul>
|
||||
<div class="section" id="ddm_06_0017__section738494914255"><h4 class="sectiontitle">Prerequisites</h4><ul id="ddm_06_0017__ul935817253264"><li id="ddm_06_0017__li23588251267">There is a source DDM instance available and associated with an RDS for MySQL instance. This DB instance is the source data node.</li><li id="ddm_06_0017__li10969101072817">The data from the source data node has been restored to the destination data node as it existed at a specified point in time.</li><li id="ddm_06_0017__li524210671711">The destination RDS for MySQL instance and the destination DDM instance must be deployed in the same VPC and subnet and have the same security group rules.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="ddm_06_0017__section16311542101119"><h4 class="sectiontitle">Constraints</h4><ul id="ddm_06_0017__ul1983134241110"><li id="ddm_06_0017__li884942201112">The destination DDM instance is not associated with any RDS for MySQL instance and has no schemas or accounts.</li><li id="ddm_06_0017__li02281512112">Restoration is not supported if the destination DDM instance is in the primary network and its associated RDS for MySQL instance is in the extended network.</li><li id="ddm_06_0017__li750871015158">The source DDM instance must be of the version 2.3.2.11 or later, and the destination DDM instance must be of the version 3.0.8 or later.</li><li id="ddm_06_0017__li3128144163914">Time points that data can be restored to depend on the backup policy set on original data nodes.</li></ul>
|
||||
<div class="section" id="ddm_06_0017__section16311542101119"><h4 class="sectiontitle">Constraints</h4><ul id="ddm_06_0017__ul1983134241110"><li id="ddm_06_0017__li884942201112">The destination DDM instance is not associated with any RDS for MySQL instance and has no schemas and accounts.</li><li id="ddm_06_0017__li02281512112">Restoration is not supported if the destination DDM instance is in the primary network and its associated RDS for MySQL instance is in the extended network.</li><li id="ddm_06_0017__li750871015158">The source DDM instance must be of the version 2.3.2.11 or later, and the destination DDM instance must be of the version 3.0.8 or later.</li><li id="ddm_06_0017__li3128144163914">Time points that data can be restored to depend on the backup policy set on original data nodes.</li><li id="ddm_06_0017__li4865172813345">The time point to which metadata is restored must be the same as that to which an RDS for MySQL instance is restored.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="ddm_06_0017__section128315484524"><a name="ddm_06_0017__section128315484524"></a><a name="section128315484524"></a><h4 class="sectiontitle">Restoring Metadata to a Point in Time</h4><ol id="ddm_06_0017__ol5371100162518"><li id="ddm_06_0017__li1366514210243"><span>Log in to the DDM console.</span></li><li id="ddm_06_0017__li4793191882712"><a name="ddm_06_0017__li4793191882712"></a><a name="li4793191882712"></a><span><a href="ddm_06_0132.html">Create a new DDM instance</a> and use it as the destination DDM instance. For details, see<a href="ddm_06_0132.html">Creating a DDM Instance</a>.</span></li><li id="ddm_06_0017__li203725032512"><span>In the DDM instance list, locate the source instance and click its name.</span></li><li id="ddm_06_0017__li382416102214"><span>In the navigation pane on the left, choose <strong id="ddm_06_0017__b2066407356621">Backups & Restorations</strong>.</span></li><li id="ddm_06_0017__li723812134011"><span>Click <strong id="ddm_06_0017__b698802603620">Restore Metadata</strong>.</span></li><li id="ddm_06_0017__li179543119201"><span>On the displayed page, specify a time point. The source DDM instance will select an appropriate DDM metadata backup closest to the time point.</span><p>
|
||||
<div class="section" id="ddm_06_0017__section128315484524"><a name="ddm_06_0017__section128315484524"></a><a name="section128315484524"></a><h4 class="sectiontitle">Restoring Metadata to a Point in Time</h4><ol id="ddm_06_0017__ol5371100162518"><li id="ddm_06_0017__li1366514210243"><span>Log in to the DDM console.</span></li><li id="ddm_06_0017__li4793191882712"><a name="ddm_06_0017__li4793191882712"></a><a name="li4793191882712"></a><span><a href="ddm_06_0132.html">Create a new DDM instance</a> and use it as the destination DDM instance. For details, see <a href="ddm_06_0132.html">Creating a DDM Instance</a>.</span></li><li id="ddm_06_0017__li203725032512"><span>In the DDM instance list, locate the source instance and click its name.</span></li><li id="ddm_06_0017__li382416102214"><span>In the navigation pane on the left, choose <strong id="ddm_06_0017__b2066407356621">Backups & Restorations</strong>.</span></li><li id="ddm_06_0017__li723812134011"><span>Click <strong id="ddm_06_0017__b698802603620">Restore Metadata</strong>.</span></li><li id="ddm_06_0017__li179543119201"><span>On the displayed page, specify a time point. The source DDM instance will select an appropriate DDM metadata backup closest to the time point.</span><p>
|
||||
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ddm_06_0017__table1941672742313" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="ddm_06_0017__row11417102722314"><th align="left" class="cellrowborder" valign="top" width="24.87%" id="mcps1.3.7.2.6.2.1.2.3.1.1"><p id="ddm_06_0017__p10417112762314">Parameter</p>
|
||||
</th>
|
||||
<th align="left" class="cellrowborder" valign="top" width="75.13%" id="mcps1.3.7.2.6.2.1.2.3.1.2"><p id="ddm_06_0017__p4417172715233">Description</p>
|
||||
@ -29,6 +29,7 @@
|
||||
<tr id="ddm_06_0017__row8417172719237"><td class="cellrowborder" valign="top" width="24.87%" headers="mcps1.3.7.2.6.2.1.2.3.1.1 "><p id="ddm_06_0017__p1941772722315">Destination Data Nodes</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="75.13%" headers="mcps1.3.7.2.6.2.1.2.3.1.2 "><p id="ddm_06_0017__p441719279239">Select the RDS for MySQL instances that have completed PITR. DDM will match the selected data nodes with shard information in the selected metadata backup.</p>
|
||||
<p id="ddm_06_0017__p3802721124213">For details about how to perform PITR on RDS for MySQL instances, see <a href="https://docs.otc.t-systems.com/relational-database-service/umn/working_with_rds_for_mysql/backups_and_restorations/restoring_a_db_instance_to_a_point_in_time.html" target="_blank" rel="noopener noreferrer">Restoring a DB Instance to a Point in Time</a>.</p>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
@ -36,7 +37,7 @@
|
||||
</div>
|
||||
</p></li><li id="ddm_06_0017__li160757161013"><span>Click <strong id="ddm_06_0017__b4545118193711">OK</strong>. If a message is displayed indicating that the metadata is restored successfully, the restoration is complete.</span></li></ol>
|
||||
</div>
|
||||
<div class="section" id="ddm_06_0017__section132806221525"><a name="ddm_06_0017__section132806221525"></a><a name="section132806221525"></a><h4 class="sectiontitle">Restoring Metadata Using Backups</h4><ol id="ddm_06_0017__ol5281162245210"><li id="ddm_06_0017__li106372611521"><span>Log in to the DDM console.</span></li><li id="ddm_06_0017__li881143742017"><a name="ddm_06_0017__li881143742017"></a><a name="li881143742017"></a><span><a href="ddm_06_0132.html">Create a new DDM instance</a> and use it as the destination DDM instance. For details, see<a href="ddm_06_0132.html">Creating a DDM Instance</a>.</span></li><li id="ddm_06_0017__li2065410461265"><span>In the DDM instance list, locate the source instance and click its name.</span></li><li id="ddm_06_0017__li1828118220524"><span>In the navigation pane on the left, choose <strong id="ddm_06_0017__b8953192322212">Backups</strong>. Alternatively, go to the instance details page and choose <strong id="ddm_06_0017__b1295319237225">Backups</strong> in the navigation pane on the left.</span></li><li id="ddm_06_0017__li1528112265210"><span>Locate the required backup based on the instance name and backup time and click <strong id="ddm_06_0017__b137151723143718">Restore</strong> in the <strong id="ddm_06_0017__b1371519233371">Operation</strong> column.</span><p><p id="ddm_06_0017__p42821322165212"></p>
|
||||
<div class="section" id="ddm_06_0017__section132806221525"><a name="ddm_06_0017__section132806221525"></a><a name="section132806221525"></a><h4 class="sectiontitle">Restoring Metadata Using Backups</h4><ol id="ddm_06_0017__ol5281162245210"><li id="ddm_06_0017__li106372611521"><span>Log in to the DDM console.</span></li><li id="ddm_06_0017__li881143742017"><a name="ddm_06_0017__li881143742017"></a><a name="li881143742017"></a><span><a href="ddm_06_0132.html">Create a new DDM instance</a> and use it as the destination DDM instance. For details, see <a href="ddm_06_0132.html">Creating a DDM Instance</a>.</span></li><li id="ddm_06_0017__li2065410461265"><span>In the DDM instance list, locate the source instance and click its name.</span></li><li id="ddm_06_0017__li1828118220524"><span>In the navigation pane on the left, choose <strong id="ddm_06_0017__b8953192322212">Backups</strong>. Alternatively, go to the instance details page and choose <strong id="ddm_06_0017__b1295319237225">Backups</strong> in the navigation pane on the left.</span></li><li id="ddm_06_0017__li1528112265210"><span>Locate the required backup based on the instance name and backup time and click <strong id="ddm_06_0017__b137151723143718">Restore</strong> in the <strong id="ddm_06_0017__b1371519233371">Operation</strong> column.</span><p><p id="ddm_06_0017__p42821322165212"></p>
|
||||
</p></li><li id="ddm_06_0017__li10282152255212"><span>On the displayed page, configure required parameters.</span><p>
|
||||
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ddm_06_0017__table5752162417313" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Parameter description</caption><thead align="left"><tr id="ddm_06_0017__row13753624163111"><th align="left" class="cellrowborder" valign="top" width="26.26%" id="mcps1.3.8.2.6.2.1.2.3.1.1"><p id="ddm_06_0017__p3753192410319">Parameter</p>
|
||||
</th>
|
||||
@ -57,6 +58,7 @@
|
||||
<tr id="ddm_06_0017__row7753524143120"><td class="cellrowborder" valign="top" width="26.26%" headers="mcps1.3.8.2.6.2.1.2.3.1.1 "><p id="ddm_06_0017__p10753824153111">Destination Data Nodes</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="73.74000000000001%" headers="mcps1.3.8.2.6.2.1.2.3.1.2 "><p id="ddm_06_0017__p375342483120">Select the RDS for MySQL instances that have completed PITR. DDM will match the selected data nodes with shard information in the selected metadata backup.</p>
|
||||
<p id="ddm_06_0017__p3244414373">For details about how to perform PITR on RDS for MySQL instances, see <a href="https://docs.otc.t-systems.com/relational-database-service/umn/working_with_rds_for_mysql/backups_and_restorations/restoring_a_db_instance_to_a_point_in_time.html" target="_blank" rel="noopener noreferrer">Restoring a DB Instance to a Point in Time</a>.</p>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
|
||||
14
docs/ddm/umn/ddm_12_0002.html
Normal file
14
docs/ddm/umn/ddm_12_0002.html
Normal file
@ -0,0 +1,14 @@
|
||||
<a name="ddm_12_0002"></a><a name="ddm_12_0002"></a>
|
||||
|
||||
<h1 class="topictitle1">Account Permissions</h1>
|
||||
<div id="body0000001142093626"><p id="ddm_12_0002__p15609163215619">DDM permissions management is based on MySQL permissions management. DDM supports most of MySQL syntax and permissions. For more information about MySQL accounts and permissions, see <a href="https://dev.mysql.com/doc/refman/5.7/en/access-control.html" target="_blank" rel="noopener noreferrer">MySQL documentation</a>.</p>
|
||||
<p id="ddm_12_0002__p1791825495916">This document describes DDM account rules, permission levels, permission items, and permission operations.</p>
|
||||
<div class="note" id="ddm_12_0002__note1844851819115"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="ddm_12_0002__p245221812117">DDM Accounts created by CREATE USER or GRANT have nothing to do with accounts in associated RDS instances, and will not be synchronized to RDS instances either.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="ddm_05_0020.html">Managing Permissions</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
16
docs/ddm/umn/ddm_12_0003.html
Normal file
16
docs/ddm/umn/ddm_12_0003.html
Normal file
@ -0,0 +1,16 @@
|
||||
<a name="ddm_12_0003"></a><a name="ddm_12_0003"></a>
|
||||
|
||||
<h1 class="topictitle1">Account Requirements</h1>
|
||||
<div id="body0000001141571578"><div class="section" id="ddm_12_0003__section1490895514220"><h4 class="sectiontitle">Account</h4><p id="ddm_12_0003__p56101328563">Different from MySQL, DDM identifies an account by username, not by username@host.</p>
|
||||
</div>
|
||||
<div class="section" id="ddm_12_0003__section821383116319"><h4 class="sectiontitle">Username</h4><ul id="ddm_12_0003__ul1761013211562"><li id="ddm_12_0003__li861093210563">Must be case-sensitive.</li><li id="ddm_12_0003__li2610203245615">Contains 1 to 32 characters and must start with a letter. Only letters, digits, and underscores (_) are allowed.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="ddm_12_0003__section1629416171062"><h4 class="sectiontitle">Password</h4><ul id="ddm_12_0003__ul0610832165619"><li id="ddm_12_0003__li96101432195610">Contains 8 to 32 characters.</li><li id="ddm_12_0003__li5610183213568">Must contain at least three of the following character types: uppercase letters, lowercase letters, digits, and special characters ~!@#%^*-_+?</li><li id="ddm_12_0003__li361017329564">Cannot be a weak password that is easily guessed.</li></ul>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="ddm_05_0020.html">Managing Permissions</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
169
docs/ddm/umn/ddm_12_0004.html
Normal file
169
docs/ddm/umn/ddm_12_0004.html
Normal file
File diff suppressed because it is too large
Load Diff
BIN
docs/ddm/umn/en-us_image_0000002454002032.png
Normal file
BIN
docs/ddm/umn/en-us_image_0000002454002032.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 109 KiB |
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -8,9 +8,14 @@
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody><tr id="dli_08_00005__row117793121078"><td class="cellrowborder" valign="top" width="26.91%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_08_00005__p8777202392813">2025-03-04</p>
|
||||
<tbody><tr id="dli_08_00005__row3388181562412"><td class="cellrowborder" valign="top" width="26.91%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_08_00005__p1020431922412">2025-04-30</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="73.09%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_08_00005__p177719238285">Added <a href="en-us_topic_0000002172127506.html">Flink OpenSource SQL 1.15 Syntax Reference</a>.</p>
|
||||
<td class="cellrowborder" valign="top" width="73.09%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_08_00005__p520414195244">Added descriptions that data multi-versioning will be deprecated and is no longer recommended to <a href="dli_08_0349.html">Backing Up and Restoring Multi-Versioning Data (Deprecated, Not Recommended)</a>.</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_00005__row117793121078"><td class="cellrowborder" valign="top" width="26.91%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_08_00005__p8777202392813">2025-03-04</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="73.09%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_08_00005__p177719238285">Added <a href="dli_08_15000.html">Flink OpenSource SQL 1.15 Syntax Reference</a>.</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_00005__row02881225102114"><td class="cellrowborder" valign="top" width="26.91%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_08_00005__p18857192716215">2025-02-08</p>
|
||||
|
||||
@ -1,7 +1,8 @@
|
||||
<a name="dli_08_0057"></a><a name="dli_08_0057"></a>
|
||||
|
||||
<h1 class="topictitle1">Overview</h1>
|
||||
<div id="body8662426"><p id="dli_08_0057__en-us_topic_0093947008_a60c0b18611f4456188c9308bc450734b">Data type is a basic attribute of data. It is used to distinguish different types of data. Different data types occupy different storage space and support different operations. Data is stored in data tables in the database. A data type is specified for each column of a data table. Therefore, data to be stored in a data table must comply with the attribute of the specific data type. Otherwise, errors may occur.</p>
|
||||
<div id="body8662426"><p id="dli_08_0057__p1625813109247">Data type is a basic attribute of data and used to distinguish different types of data. Different data types occupy different storage space and support different operations.</p>
|
||||
<p id="dli_08_0057__p1910192232411">Data is stored in data tables in the database. Each column of a table defines the data type. During storage, data must be stored according to data types.</p>
|
||||
<p id="dli_08_0057__en-us_topic_0093947008_a63de052ecbb24a88a5db1c9f680643b0">DLI only supports primitive data types.</p>
|
||||
</div>
|
||||
<div>
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
<h1 class="topictitle1">Creating an OBS Table Using the DataSource Syntax</h1>
|
||||
<div id="body8662426"><div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_sb2a2dad92c524677baa470c6a05dc948"><h4 class="sectiontitle">Function</h4><p id="dli_08_0076__p28131188218">Create an OBS table using the DataSource syntax.</p>
|
||||
<p id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_p374295012356">The main differences between the DataSource and the Hive syntax lie in the supported data formats and the number of supported partitions. For details, see syntax and precautions.</p>
|
||||
<div class="note" id="dli_08_0076__note2901719181211"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0076__p1990121912128">You are advised to use the OBS parallel file system for storage. A parallel file system is a high-performance file system that provides latency in milliseconds, TB/s-level bandwidth, and millions of IOPS. It applies to interactive big data analysis scenarios.</p>
|
||||
<div class="note" id="dli_08_0076__note2901719181211"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0076__p1990121912128">You are advised to use an OBS parallel file system for storage. A parallel file system is a high-performance file system that provides latency in milliseconds, TB/s-level bandwidth, and millions of IOPS. It applies to interactive big data analysis scenarios.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_s2af9615060a543218bb4f231acfcd929"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_u0132702d68ac4186be555e682b694899"><li id="dli_08_0076__dli_08_0076_li15318170215">The size of a table is not calculated when the table is created.</li><li id="dli_08_0076__dli_08_0076_li1453121719215">When data is added, the table size will be changed to 0.</li><li id="dli_08_0076__dli_08_0076_li25313179210">You can check the table size on OBS.</li><li id="dli_08_0076__li27231725165616">Table properties cannot be specified using CTAS table creation statements.</li><li id="dli_08_0076__li119261906338"><strong id="dli_08_0076__b8773182162519">An OBS directory containing subdirectories:</strong><p id="dli_08_0076__p19145415173313">If you specify an OBS directory that contains subdirectories when creating a table, all file types and content within those subdirectories will also be included as table content.</p>
|
||||
@ -448,6 +448,10 @@
|
||||
</pre></div></td></tr></table></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0076__section1216992824118"><h4 class="sectiontitle">FAQ</h4><ul id="dli_08_0076__ul14911104711418"><li id="dli_08_0076__li1791164720414"><strong id="dli_08_0076__b16496113704610">What should I do if the error message "xxx dli datasource v2 tables is only supported in spark3.3 or later version." appears when I create a DataSource table using the default queue?</strong><p id="dli_08_0076__p19155311418">Ensure that you use Spark 3.3.1 or a later version when creating such a table. If the error message appears, use the Hive syntax to create the table. For details, see <a href="dli_08_0077.html">Creating an OBS Table Using the Hive Syntax</a>.</p>
|
||||
</li><li id="dli_08_0076__li01781618194315"><strong id="dli_08_0076__b102412910474">What should I do if the error message "xxx don`t support dli v1 table." appears when I use Spark 3.3.1 to run a Jar job?</strong><p id="dli_08_0076__p111863544616">This error message indicates that table operations cannot be performed when Spark 3.3.1 is used to execute the Jar job. Use the Hive syntax to recreate the tables' data structure. For example, you can use <strong id="dli_08_0076__b41612316581">[STORED AS file_format] CTAS</strong> to recreate the table and then run the job. For details, see <a href="dli_08_0077.html">Creating an OBS Table Using the Hive Syntax</a>.</p>
|
||||
</li></ul>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
|
||||
@ -43,7 +43,7 @@
|
||||
</pre></div></td></tr></table></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0077__en-us_topic_0114776171_en-us_topic_0093946792_s3805b86231dd4c41b26864162d6c4ed5"><h4 class="sectiontitle">Keywords</h4><ul id="dli_08_0077__en-us_topic_0114776171_ul74261946687"><li id="dli_08_0077__en-us_topic_0114776171_en-us_topic_0093946792_l4409cf7808da45c6abd51a1e32c0b909">EXTERNAL: Creates an OBS table.</li><li id="dli_08_0077__en-us_topic_0114776171_li12814539191914">IF NOT EXISTS: Prevents system errors when the created table exists.</li><li id="dli_08_0077__en-us_topic_0114776171_li11143171444414">COMMENT: Field or table description.</li><li id="dli_08_0077__en-us_topic_0114776171_li572161314209">PARTITIONED BY: Partition field.</li><li id="dli_08_0077__li1761110581424">ROW FORMAT: Row data format.</li><li id="dli_08_0077__en-us_topic_0114776171_en-us_topic_0093946792_l2b4d833ae66e470dae553c300f0783b8">STORED AS: Specifies the format of the file to be stored. Currently, only the TEXTFILE, AVRO, ORC, SEQUENCEFILE, RCFILE, and PARQUET format are supported.</li><li id="dli_08_0077__en-us_topic_0114776171_en-us_topic_0093946792_le38d766b4f094c178071c05b9717391f">LOCATION: Specifies the path of OBS. This keyword is mandatory when you create OBS tables.</li><li id="dli_08_0077__li6331130191815">TBLPROPERTIES: Allows you to add the <strong id="dli_08_0077__b1550212295461">key/value</strong> properties to a table.<ul id="dli_08_0077__ul1611116494185"><li id="dli_08_0077__li7473164411185">You can use this statement to enable the multiversion function to back up and restore table data. After the multiversion function is enabled, the system automatically backs up table data when you delete or modify the data using <strong id="dli_08_0077__b17731842484">insert overwrite</strong> or <strong id="dli_08_0077__b18773204194816">truncate</strong>, and retains the data for a certain period. You can quickly restore data within the retention period. For details about the SQL syntax for the multiversion function, see <a href="dli_08_0354.html">Enabling or Disabling Multiversion Backup</a> and <a href="dli_08_0349.html">Backing Up and Restoring Data of Multiple Versions</a>.<p id="dli_08_0077__p1882095415365">When creating an OBS table, you can use <strong id="dli_08_0077__b12439113516404">TBLPROPERTIES ("dli.multi.version.enable"="true")</strong> to enable multiversion. For details, see the following example.</p>
|
||||
<div class="section" id="dli_08_0077__en-us_topic_0114776171_en-us_topic_0093946792_s3805b86231dd4c41b26864162d6c4ed5"><h4 class="sectiontitle">Keywords</h4><ul id="dli_08_0077__en-us_topic_0114776171_ul74261946687"><li id="dli_08_0077__en-us_topic_0114776171_en-us_topic_0093946792_l4409cf7808da45c6abd51a1e32c0b909">EXTERNAL: Creates an OBS table.</li><li id="dli_08_0077__en-us_topic_0114776171_li12814539191914">IF NOT EXISTS: Prevents system errors when the created table exists.</li><li id="dli_08_0077__en-us_topic_0114776171_li11143171444414">COMMENT: Field or table description.</li><li id="dli_08_0077__en-us_topic_0114776171_li572161314209">PARTITIONED BY: Partition field.</li><li id="dli_08_0077__li1761110581424">ROW FORMAT: Row data format.</li><li id="dli_08_0077__en-us_topic_0114776171_en-us_topic_0093946792_l2b4d833ae66e470dae553c300f0783b8">STORED AS: Specifies the format of the file to be stored. Currently, only the TEXTFILE, AVRO, ORC, SEQUENCEFILE, RCFILE, and PARQUET format are supported.</li><li id="dli_08_0077__en-us_topic_0114776171_en-us_topic_0093946792_le38d766b4f094c178071c05b9717391f">LOCATION: Specifies the path of OBS. This keyword is mandatory when you create OBS tables.</li><li id="dli_08_0077__li6331130191815">TBLPROPERTIES: Allows you to add the <strong id="dli_08_0077__b1550212295461">key/value</strong> properties to a table.<ul id="dli_08_0077__ul1611116494185"><li id="dli_08_0077__li7473164411185">(The multi-versioning function has been deprecated and is not recommended.) Enables data multi-versioning for table data backup and restoration. After the multiversion function is enabled, the system automatically backs up table data when you delete or modify the data using <strong id="dli_08_0077__b17731842484">insert overwrite</strong> or <strong id="dli_08_0077__b18773204194816">truncate</strong>, and retains the data for a certain period. You can quickly restore data within the retention period. For details about the SQL syntax for the multiversion function, see <a href="dli_08_0354.html">Enabling or Disabling Data Multi-Versioning (Deprecated, Not Recommended)</a> and <a href="dli_08_0349.html">Backing Up and Restoring Multi-Versioning Data (Deprecated, Not Recommended)</a>.<p id="dli_08_0077__p1882095415365">When creating an OBS table, you can use <strong id="dli_08_0077__b12439113516404">TBLPROPERTIES ("dli.multi.version.enable"="true")</strong> to enable multiversion. For details, see the following example.</p>
|
||||
|
||||
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0077__table2861727205115" frame="border" border="1" rules="all"><caption><b>Table 1 </b>TBLPROPERTIES parameters</caption><thead align="left"><tr id="dli_08_0077__row2862827195111"><th align="left" class="cellrowborder" valign="top" width="32.35%" id="mcps1.3.4.2.8.2.1.6.2.3.1.1"><p id="dli_08_0077__p1686272745115">Key</p>
|
||||
</th>
|
||||
|
||||
@ -3,7 +3,11 @@
|
||||
<h1 class="topictitle1">Inserting Data</h1>
|
||||
<div id="body8662426"><div class="section" id="dli_08_0095__en-us_topic_0114776189_en-us_topic_0093946901_s9964bb020c52461992d2c7711fd7feed"><h4 class="sectiontitle">Function</h4><p id="dli_08_0095__en-us_topic_0114776189_en-us_topic_0093946901_a453a1e6d2a6e4b05a42b498a72d224bc">This statement is used to insert the SELECT query result or a certain data record into a table.</p>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0095__section1480717121549"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dli_08_0095__ul1282125143915"><li id="dli_08_0095__li8821525103916">The <strong id="dli_08_0095__b19412132395515">INSERT OVERWRITE</strong> syntax is not suitable for "read-write" scenarios, where data is continuously processed and updated. Using this syntax in such scenarios may result in data loss.<p id="dli_08_0095__p127271127173912">"Read-write" refers to the ability to read data while generating new data or modifying existing data during data processing.</p>
|
||||
<div class="section" id="dli_08_0095__section1480717121549"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dli_08_0095__ul1282125143915"><li id="dli_08_0095__li194403272292">The <strong id="dli_08_0095__b11593143874813">insert overwrite</strong> syntax does not apply to self-read and self-write scenarios within the same table (including both partitioned and non-partitioned tables). Directly executing <strong id="dli_08_0095__b16963121124914">insert overwrite</strong> on the original table may lead to risks of data loss or inconsistency.<p id="dli_08_0095__p153511828172917">To implement data operations in self-read and self-write scenarios, you are advised to use a temporary table to handle the data. See <a href="#dli_08_0095__fig8181339957">Figure 1</a>.</p>
|
||||
<p id="dli_08_0095__p127271127173912">Self-read and self-write means that the destination table and the data source table are the same table. For example, suppose you want to extract information of students with <strong id="dli_08_0095__b17972543541">class_no = 1</strong> from the <strong id="dli_08_0095__b1166212579549">student</strong> table and overwrite the original table, the following statements represent typical operations in self-read and self-write scenarios:</p>
|
||||
<pre class="screen" id="dli_08_0095__screen8847151312282">INSERT OVERWRITE TABLE student
|
||||
SELECT name FROM student WHERE class_no = 1;</pre>
|
||||
<div class="fignone" id="dli_08_0095__fig8181339957"><a name="dli_08_0095__fig8181339957"></a><a name="fig8181339957"></a><span class="figcap"><b>Figure 1 </b>Alternative solution for self-read and self-write scenarios by running insert overwrite</span><br><span><img id="dli_08_0095__image16599286715" src="en-us_image_0000002272860560.png"></span></div>
|
||||
</li><li id="dli_08_0095__li181767227540">When using Hive and Datasource tables (excluding Hudi), executing data modification commands (such as <strong id="dli_08_0095__b236110285217">insert into</strong> and <strong id="dli_08_0095__b1467713331723">load data</strong>) may result in data duplication or inconsistency if the data source does not support transactions and there is a system failure or queue restart.<p id="dli_08_0095__p15597549191616">To avoid this situation, you are advised to prioritize data sources that support transactions, such as Hudi data sources. This type of data source has Atomicity, Consistency, Isolation, Durability (ACID) capabilities, which helps ensure data consistency and accuracy.</p>
|
||||
<p id="dli_08_0095__p1542252311541">To learn more, refer to <a href="#dli_08_0095__section1516329541">How Do I Handle Duplicate Records After Executing the INSERT INTO Statement?</a></p>
|
||||
</li></ul>
|
||||
@ -80,14 +84,14 @@
|
||||
</div>
|
||||
<div class="section" id="dli_08_0095__en-us_topic_0114776189_en-us_topic_0093946901_sc650d4ee825647c2a760ac5fcc4ab2c8"><h4 class="sectiontitle">Example</h4><div class="note" id="dli_08_0095__note1149934611312"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0095__p161225010319">Before importing data, you must create a table. For details, see <a href="dli_08_0223.html">Creating an OBS Table</a> or <a href="dli_08_0224.html">Creating a DLI Table</a>.</p>
|
||||
</div></div>
|
||||
<ul id="dli_08_0095__en-us_topic_0114776189_en-us_topic_0093946901_ul18471028173914"><li id="dli_08_0095__li193703191708">Insert the SELECT query result into a table.<ul id="dli_08_0095__ul4308115011213"><li id="dli_08_0095__li1494424416214">Use the DataSource syntax to create a parquet partitioned table.<pre class="screen" id="dli_08_0095__screen20881115535">CREATE TABLE data_source_tab1 (col1 INT, p1 INT, p2 INT)
|
||||
<ul id="dli_08_0095__en-us_topic_0114776189_en-us_topic_0093946901_ul18471028173914"><li id="dli_08_0095__li193703191708">Example 1: Insert the SELECT query result into a table.<ul id="dli_08_0095__ul4308115011213"><li id="dli_08_0095__li1494424416214">Use the DataSource syntax to create a parquet partitioned table.<pre class="screen" id="dli_08_0095__screen20881115535">CREATE TABLE data_source_tab1 (col1 INT, p1 INT, p2 INT)
|
||||
USING PARQUET PARTITIONED BY (p1, p2);</pre>
|
||||
</li><li id="dli_08_0095__li136969131212">Insert the query result to the partition (p1 = 3, p2 = 4).<pre class="screen" id="dli_08_0095__screen09951301751">INSERT INTO data_source_tab1 PARTITION (p1 = 3, p2 = 4)
|
||||
SELECT id FROM RANGE(1, 3);</pre>
|
||||
</li><li id="dli_08_0095__li28072014151212">Insert the new query result to the partition (p1 = 3, p2 = 4).<pre class="screen" id="dli_08_0095__screen623634219616">INSERT OVERWRITE TABLE data_source_tab1 PARTITION (p1 = 3, p2 = 4)
|
||||
SELECT id FROM RANGE(3, 5);</pre>
|
||||
</li></ul>
|
||||
</li><li id="dli_08_0095__li1928215175112">Insert a data record into a table.<ul id="dli_08_0095__ul9733124541111"><li id="dli_08_0095__li10508854201111">Create a Parquet partitioned table with Hive format<pre class="screen" id="dli_08_0095__screen11281528191410">CREATE TABLE hive_serde_tab1 (col1 INT, p1 INT, p2 INT)
|
||||
</li><li id="dli_08_0095__li1928215175112">Example 2: Insert a piece of data into a table.<ul id="dli_08_0095__ul9733124541111"><li id="dli_08_0095__li10508854201111">Create a Parquet partitioned table with Hive format<pre class="screen" id="dli_08_0095__screen11281528191410">CREATE TABLE hive_serde_tab1 (col1 INT, p1 INT, p2 INT)
|
||||
USING HIVE OPTIONS(fileFormat 'PARQUET') PARTITIONED BY (p1, p2);</pre>
|
||||
</li><li id="dli_08_0095__li19793643111416">Insert two data records into the partition (p1 = 3, p2 = 4).<pre class="screen" id="dli_08_0095__screen13173181071519">INSERT INTO hive_serde_tab1 PARTITION (p1 = 3, p2 = 4)
|
||||
VALUES (1), (2);</pre>
|
||||
|
||||
@ -254,6 +254,10 @@
|
||||
</div>
|
||||
</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0098__section1216992824118"><h4 class="sectiontitle">FAQ</h4><ul id="dli_08_0098__ul14911104711418"><li id="dli_08_0098__li1791164720414"><strong id="dli_08_0098__b97691718107">What should I do if the error message "xxx dli datasource v2 tables is only supported in spark3.3 or later version." appears when I create a DataSource table using the default queue?</strong><p id="dli_08_0098__p19155311418">Ensure that you use Spark 3.3.1 or a later version when creating such a table. If the error message appears, use the Hive syntax to create the table. For details, see <a href="dli_08_0204.html">Creating a DLI Table Using the Hive Syntax</a>.</p>
|
||||
</li><li id="dli_08_0098__li01781618194315"><strong id="dli_08_0098__b13969132412012">What should I do if the error message "xxx don`t support dli v1 table." appears when I use Spark 3.3.1 to run a Jar job?</strong><p id="dli_08_0098__p73279166509">This error message indicates that table operations cannot be performed when Spark 3.3.1 is used to execute the Jar job. Use the Hive syntax to recreate the tables' data structure. For example, you can use <strong id="dli_08_0098__b16148291106">[STORED AS file_format] CTAS</strong> to recreate the table and then run the job. For how to create a table, see <a href="dli_08_0204.html">Creating a DLI Table Using the Hive Syntax</a>.</p>
|
||||
</li></ul>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
|
||||
@ -34,7 +34,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="84.2%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0197__en-us_topic_0142907233_p216518281377">Create a datasource connection before you can obtain this RDS connection URL.</p>
|
||||
<p id="dli_08_0197__en-us_topic_0142907233_p17658829185311">After an enhanced datasource connection is created, use the internal network domain name or internal network address and database port number provided by RDS to connect to DLI. If MySQL is used, the format is <strong id="dli_08_0197__b197693862414"><em id="dli_08_0197__i1875143812410">protocol header</em>://<em id="dli_08_0197__i4757382246">internal IP address</em>:<em id="dli_08_0197__i2768386240">internal network port number</em></strong>. If PostgreSQL is used, the format is <strong id="dli_08_0197__b178838192411"><em id="dli_08_0197__i776193813242">protocol header</em>://<em id="dli_08_0197__i67733882412">internal IP address</em>:<em id="dli_08_0197__i577133872411">internal network port number</em>/<em id="dli_08_0197__i1478123852412">database name</em></strong>.</p>
|
||||
<p id="dli_08_0197__en-us_topic_0142907233_p14832153644513">For example: <strong id="dli_08_0197__b932218404249">jdbc:mysql://192.168.0.193:3306</strong> or <strong id="dli_08_0197__b8322184082413">jdbc:postgresql://192.168.0.193:3306/postgres</strong>.</p>
|
||||
<p id="dli_08_0197__en-us_topic_0142907233_p14832153644513">For example: <strong id="dli_08_0197__b932218404249">jdbc:mysql://192.168.0.193:3306</strong> or <strong id="dli_08_0197__b8322184082413">jdbc:postgresql://192.168.0.193:5432/postgres</strong>.</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_0197__en-us_topic_0142907233_en-us_topic_0114776213_en-us_topic_0103157088_row166137253465"><td class="cellrowborder" valign="top" width="15.8%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0197__en-us_topic_0142907233_en-us_topic_0114776213_en-us_topic_0103157088_p584237211576">driver</p>
|
||||
@ -128,7 +128,8 @@
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0197__en-us_topic_0142907233_en-us_topic_0114776213_en-us_topic_0103157088_section1856617202225"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0197__en-us_topic_0142907233_p912611523415">When creating a table associated with RDS, you do not need to specify the <strong id="dli_08_0197__b29521246102519">Schema</strong> of the associated table. DLI automatically obtains the schema of the table in the <strong id="dli_08_0197__b5362548122511">dbtable</strong> parameter of RDS.</p>
|
||||
<div class="section" id="dli_08_0197__en-us_topic_0142907233_en-us_topic_0114776213_en-us_topic_0103157088_section1856617202225"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0197__p19948131624612">When creating up an RDS association table for the first time, there is no need to define the table's schema. DLI will automatically retrieve the schema from the RDS parameter <strong id="dli_08_0197__b11207131645417">dbtable</strong> to create the association table.</p>
|
||||
<p id="dli_08_0197__p159481816144618">If you make changes to the fields in the RDS table, the associated table will not update automatically. In such cases, you need to recreate the association table to ensure its schema matches the modified RDS table.</p>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0197__en-us_topic_0142907233_en-us_topic_0114776213_en-us_topic_0103157088_section9892337112213"><h4 class="sectiontitle">Example</h4><p id="dli_08_0197__en-us_topic_0142907233_p14325175413169">Accessing MySQL</p>
|
||||
<div class="codecoloring" codetype="Sql" id="dli_08_0197__en-us_topic_0142907233_screen156701345141612"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
|
||||
@ -156,7 +157,7 @@
|
||||
<span class="normal">6</span>
|
||||
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">dli_to_rds</span>
|
||||
<span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">JDBC</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span>
|
||||
<span class="w"> </span><span class="s1">'url'</span><span class="o">=</span><span class="s1">'jdbc:postgresql://to-rds-1174405119-oLRHAGE7.datasource.com:3306/postgreDB'</span><span class="p">,</span>
|
||||
<span class="w"> </span><span class="s1">'url'</span><span class="o">=</span><span class="s1">'jdbc:postgresql://to-rds-1174405119-oLRHAGE7.datasource.com:5432/postgreDB'</span><span class="p">,</span>
|
||||
<span class="w"> </span><span class="s1">'driver'</span><span class="o">=</span><span class="s1">'org.postgresql.Driver'</span><span class="p">,</span>
|
||||
<span class="w"> </span><span class="s1">'dbtable'</span><span class="o">=</span><span class="s1">'pg_schema.test1'</span><span class="p">,</span>
|
||||
<span class="w"> </span><span class="s1">'passwdauth'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'xxx'</span><span class="p">,</span>
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
<a name="dli_08_0219"></a><a name="dli_08_0219"></a>
|
||||
|
||||
<h1 class="topictitle1">SQL Syntax Overview of Batch Jobs</h1>
|
||||
<h1 class="topictitle1">SQL Syntax Overview for Batch Jobs</h1>
|
||||
<div id="body1578361990117"><p id="dli_08_0219__p7811840729">This section describes the Spark SQL syntax list provided by DLI. For details about the parameters and examples, see the syntax description.</p>
|
||||
|
||||
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0219__en-us_topic_0178353918_table3712193011019" frame="border" border="1" rules="all"><caption><b>Table 1 </b>SQL syntax of batch jobs</caption><thead align="left"><tr id="dli_08_0219__en-us_topic_0178353918_row57131230101015"><th align="left" class="cellrowborder" valign="top" width="39.989999999999995%" id="mcps1.3.2.2.3.1.1"><p id="dli_08_0219__p12912191921618">Classification</p>
|
||||
@ -128,15 +128,15 @@
|
||||
<tr id="dli_08_0219__row894531915318"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p6930191975313"><a href="dli_08_0124.html">Querying an OpenTSDB Table</a></p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_0219__row16889357205419"><td class="cellrowborder" rowspan="3" valign="top" width="39.989999999999995%" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p6860175714542">Syntax for Datasource Connection to a DWS Table</p>
|
||||
<tr id="dli_08_0219__row16889357205419"><td class="cellrowborder" rowspan="3" valign="top" width="39.989999999999995%" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p6860175714542">Syntax for Datasource Connection to a GaussDB(DWS) Table</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="60.01%" headers="mcps1.3.2.2.3.1.2 "><p id="dli_08_0219__p88601057185417"><a href="dli_08_0193.html">Creating a Table and Associating It with DWS</a></p>
|
||||
<td class="cellrowborder" valign="top" width="60.01%" headers="mcps1.3.2.2.3.1.2 "><p id="dli_08_0219__p88601057185417"><a href="dli_08_0193.html">Creating a Table and Associating It with GaussDB(DWS)</a></p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_0219__row13889105735419"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p78611757155416"><a href="dli_08_0194.html">Inserting Data to a DWS Table</a></p>
|
||||
<tr id="dli_08_0219__row13889105735419"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p78611757155416"><a href="dli_08_0194.html">Inserting Data to a GaussDB(DWS) Table</a></p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_0219__row14888125775419"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p5861155755417"><a href="dli_08_0195.html">Querying a DWS Table</a></p>
|
||||
<tr id="dli_08_0219__row14888125775419"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p5861155755417"><a href="dli_08_0195.html">Querying a GaussDB(DWS) Table</a></p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_0219__row13909144265619"><td class="cellrowborder" rowspan="3" valign="top" width="39.989999999999995%" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p188621042195619">Syntax for Datasource Connection to an RDS Table</p>
|
||||
@ -246,7 +246,7 @@
|
||||
<tr id="dli_08_0219__row19781623131017"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p1254623191016"><a href="dli_08_0285.html">Displaying All Functions</a></p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_0219__row134000231125"><td class="cellrowborder" valign="top" width="39.989999999999995%" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p1040016235124">Multiversion-related Syntax</p>
|
||||
<tr id="dli_08_0219__row134000231125"><td class="cellrowborder" valign="top" width="39.989999999999995%" headers="mcps1.3.2.2.3.1.1 "><p id="dli_08_0219__p1040016235124">Data Multi-Versioning-related Syntax (Deprecated, Not Recommended)</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="60.01%" headers="mcps1.3.2.2.3.1.2 "><p id="dli_08_0219__p1340012311123"><a href="dli_08_0077.html">Enabling Multiversion Backup When Creating an OBS Table</a></p>
|
||||
<p id="dli_08_0219__p268881001318"><a href="dli_08_0354.html">Enabling or Disabling Multiversion Backup When Modifying a Table</a></p>
|
||||
|
||||
@ -7,7 +7,7 @@
|
||||
<ul class="ullinks">
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0266.html">Common Configuration Items of Batch SQL Jobs</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0219.html">SQL Syntax Overview of Batch Jobs</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0219.html">SQL Syntax Overview for Batch Jobs</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0477.html">Spark Open Source Commands</a></strong><br>
|
||||
</li>
|
||||
@ -35,7 +35,7 @@
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0205.html">Exporting Search Results</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0349.html">Backing Up and Restoring Data of Multiple Versions</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0349.html">Backing Up and Restoring Multi-Versioning Data (Deprecated, Not Recommended)</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0465.html">Table Lifecycle Management</a></strong><br>
|
||||
</li>
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
<a name="dli_08_0261"></a><a name="dli_08_0261"></a>
|
||||
|
||||
<h1 class="topictitle1">Creating an RDS Table</h1>
|
||||
<div id="body1574393121912"><p id="dli_08_0261__p7798335163614">Create an RDS/DWS table to connect to the source stream.</p>
|
||||
<div id="body1574393121912"><p id="dli_08_0261__p7798335163614">Create an RDS or GaussDB(DWS) table to connect to the source stream.</p>
|
||||
<p id="dli_08_0261__p174595520164">For details about the JOIN syntax, see <a href="dli_08_0106.html">JOIN</a>.</p>
|
||||
<div class="section" id="dli_08_0261__section149656154515"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0261__en-us_topic_0111499975_ul810551215398"><li id="dli_08_0261__en-us_topic_0111499975_li6425436163814">Ensure that you have created a PostgreSQL or MySQL RDS instance in RDS.<p id="dli_08_0261__en-us_topic_0111499975_p104771013203812"><a name="dli_08_0261__en-us_topic_0111499975_li6425436163814"></a><a name="en-us_topic_0111499975_li6425436163814"></a>For details about how to create an RDS instance, see <span class="filepath" id="dli_08_0261__filepath102767631292142"><b>Creating an Instance</b></span> in the <em id="dli_08_0261__i25507952392142">Relational Database Service User Guide</em>.</p>
|
||||
</li><li id="dli_08_0261__en-us_topic_0111499975_li14111191212396">In this scenario, jobs must run on the dedicated queue of DLI. Therefore, DLI must interconnect with the enhanced datasource connection that has been connected with RDS instance. You can also set the security group rules as required.<p id="dli_08_0261__p94091618103916"><a name="dli_08_0261__en-us_topic_0111499975_li14111191212396"></a><a name="en-us_topic_0111499975_li14111191212396"></a>For details about how to create an enhanced datasource connection, see <span class="filepath" id="dli_08_0261__dli_08_0249_en-us_topic_0132966325_filepath13726182511815"><b>Enhanced Datasource Connections</b></span> in the <i><cite id="dli_08_0261__dli_08_0249_cite534516e564224b13bad1de4471da3139181527">Data Lake Insight User Guide</cite></i>.</p>
|
||||
@ -73,7 +73,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p14859300387">Database connection address, for example, <strong id="dli_08_0261__b26002101492142">{database_type}://ip:port/database</strong>.</p>
|
||||
<p id="dli_08_0261__p193531452515">Currently, two types of database connections are supported: MySQL and PostgreSQL.</p>
|
||||
<ul id="dli_08_0261__ul18228175913311"><li id="dli_08_0261__li122812591335">MySQL: 'mysql://ip:port/database'</li><li id="dli_08_0261__li112281159133316">PostgreSQL: 'postgresql://ip:port/database'<div class="note" id="dli_08_0261__note19207195162811"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_08_0261__p62071951142817">To create a DWS dimension table, set the database connection address to a DWS database address. If the DWS database version is later than 8.1.0, the open-source PostgreSQL driver cannot be used for connection. You need to use the GaussDB driver for connection.</p>
|
||||
<ul id="dli_08_0261__ul18228175913311"><li id="dli_08_0261__li122812591335">MySQL: 'mysql://ip:port/database'</li><li id="dli_08_0261__li112281159133316">PostgreSQL: 'postgresql://ip:port/database'<div class="note" id="dli_08_0261__note19207195162811"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_08_0261__p62071951142817">To create a GaussDB(DWS) dimension table, set the database connection address to a GaussDB(DWS) database address. If the GaussDB(DWS) database version is later than 8.1.0, the open-source PostgreSQL driver cannot be used for connection. You need to use the GaussDB driver for connection.</p>
|
||||
</div></div>
|
||||
</li></ul>
|
||||
</td>
|
||||
@ -204,7 +204,7 @@
|
||||
</pre></div></td></tr></table></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="note" id="dli_08_0261__note13797314164316"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0261__p379716147437">To create a DWS dimension table, set the database connection address to a DWS database address. If the DWS database version is later than 8.1.0, the open-source PostgreSQL driver cannot be used for connection. You need to use the GaussDB driver for connection.</p>
|
||||
<div class="note" id="dli_08_0261__note13797314164316"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0261__p379716147437">To create a GaussDB(DWS) dimension table, set the database connection address to a GaussDB(DWS) database address. If the GaussDB(DWS) database version is later than 8.1.0, the open-source PostgreSQL driver cannot be used for connection. You need to use the GaussDB driver for connection.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0470.html">Modifying Column Comments</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0354.html">Enabling or Disabling Multiversion Backup</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dli_08_0354.html">Enabling or Disabling Data Multi-Versioning (Deprecated, Not Recommended)</a></strong><br>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
|
||||
@ -78,14 +78,18 @@
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.4.1.2 "><p id="dli_08_0266__p16441413464">false</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dli_08_0266__row11135125418304"><td class="cellrowborder" valign="top" width="28.349999999999998%" headers="mcps1.3.2.2.4.1.1 "><p id="dli_08_0266__p2013525463013">spark.sql.optimizer.dynamicPartitionPruning.enabled</p>
|
||||
<tr id="dli_08_0266__row101021133154210"><td class="cellrowborder" valign="top" width="28.349999999999998%" headers="mcps1.3.2.2.4.1.1 "><p id="dli_08_0266__p16103123320426">spark.sql.dli.job.shareLevel</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="9.56%" headers="mcps1.3.2.2.4.1.2 "><p id="dli_08_0266__p1135854183018">true</p>
|
||||
<td class="cellrowborder" valign="top" width="9.56%" headers="mcps1.3.2.2.4.1.2 "><p id="dli_08_0266__p2103123314427">Queue</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="62.09%" headers="mcps1.3.2.2.4.1.3 "><p id="dli_08_0266__p0120112114312">This parameter is used to control whether to enable dynamic partition pruning. Dynamic partition pruning can help reduce the amount of data that needs to be scanned and improve query performance when executing SQL queries.</p>
|
||||
<ul id="dli_08_0266__ul10431123219399"><li id="dli_08_0266__li34312325392">When set to <strong id="dli_08_0266__b1293118303259">true</strong>, dynamic partition pruning is enabled. SQL automatically detects and deletes partitions that do not meet the WHERE clause conditions during query. This is useful for tables that have a large number of partitions.</li><li id="dli_08_0266__li18431163263919">If SQL queries contain a large number of nested left join operations and the table has a large number of dynamic partitions, a large number of memory resources may be consumed during data parsing. As a result, the memory of the driver node is insufficient and there are frequent Full GCs.<p id="dli_08_0266__p9628429134018"><a name="dli_08_0266__li18431163263919"></a><a name="li18431163263919"></a>To avoid such issues, you can disable dynamic partition pruning by setting this parameter to <strong id="dli_08_0266__b155611420352">false</strong>.</p>
|
||||
<p id="dli_08_0266__p317112359398">However, disabling this optimization may reduce query performance. Once disabled, Spark does not automatically prun the partitions that do not meet the requirements.</p>
|
||||
<td class="cellrowborder" valign="top" width="62.09%" headers="mcps1.3.2.2.4.1.3 "><p id="dli_08_0266__p1449141415431">This configuration item is used to set the isolation level of SQL statements. Different isolation levels (job, user, project, queue) determine whether SQL jobs are executed by independent Spark Drivers and Executors or share existing ones.</p>
|
||||
<ul id="dli_08_0266__ul178673438527"><li id="dli_08_0266__li164481049155212"><strong id="dli_08_0266__b1522419919555">job</strong>:<ul id="dli_08_0266__ul166155519524"><li id="dli_08_0266__li198671343175215">Each SQL job will independently start a Spark Driver and a set of Executors for execution.</li><li id="dli_08_0266__li1486724335210">This is suitable for jobs that require complete isolation, ensuring that each job's execution environment is entirely independent.</li></ul>
|
||||
</li><li id="dli_08_0266__li18717333105311"><strong id="dli_08_0266__b19700611155510">user:</strong><ul id="dli_08_0266__ul8635137549"><li id="dli_08_0266__li172701757125318">If a Spark Driver started by this user already exists and can continue submitting tasks, the new SQL job will be submitted to this existing Driver for execution.</li><li id="dli_08_0266__li11270125713538">If there is no existing Driver or the current Driver cannot continue submitting tasks, a new Spark Driver will be started for this user.</li><li id="dli_08_0266__li2270125755315">This is suitable for scenarios where multiple jobs from the same user need to share resources.</li></ul>
|
||||
</li><li id="dli_08_0266__li16832196546"><strong id="dli_08_0266__b147091217185516">project:</strong><ul id="dli_08_0266__ul17995173016543"><li id="dli_08_0266__li1212311230546">If a Spark Driver started by this project already exists and can continue submitting tasks, the new SQL job will be submitted to this existing Driver for execution.</li><li id="dli_08_0266__li1212319231549">If there is no existing Driver or the current Driver cannot continue submitting tasks, a new Spark Driver will be started for this project.</li><li id="dli_08_0266__li912332311545">This is suitable for scenarios where multiple jobs within the same project need to share resources.</li></ul>
|
||||
</li><li id="dli_08_0266__li7712164012544"><strong id="dli_08_0266__b11265172216558">queue:</strong><ul id="dli_08_0266__ul213054919541"><li id="dli_08_0266__li202986440544">If a Spark Driver started by this queue already exists and can continue submitting tasks, the new SQL job will be submitted to this existing Driver for execution.</li><li id="dli_08_0266__li729824415540">If there is no existing Driver or the current Driver cannot continue submitting tasks, a new Spark Driver will be started for this queue.</li><li id="dli_08_0266__li192981044185413">This is suitable for scenarios where resources are managed by queues, allowing for more granular control over resource allocation.</li></ul>
|
||||
</li></ul>
|
||||
<div class="note" id="dli_08_0266__note070313588548"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_08_0266__p4703175812547">The maximum number of Spark Drivers that can be started (maximum Spark Driver instances) and the maximum number of concurrent SQL queries that can be executed by each Spark Driver (maximum concurrency per Spark Driver instance) can be configured in the queue properties.</p>
|
||||
</div></div>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
|
||||
@ -11,7 +11,7 @@ FROM (
|
||||
FROM table_name)
|
||||
WHERE rownum = 1</pre>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_section1627916611011"><h4 class="sectiontitle">Description</h4><ul id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_ul12220145108"><li id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_li82201045703">ROW_NUMBER(): Assigns a unique, sequential number to each row, starting with one.</li><li id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_li152201045504">PARTITION BY col1[, col2...]: Specifies the partition columns, for example, the deduplicate key.</li><li id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_li1422020451403">ORDER BY time_attr [asc|desc]: Specifies the ordering column, it must be a time attribute. Currently Flink supports proctime only. Ordering by ASC means to keep the first row, ordering by DESC means to keep the last row.</li><li id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_li1922012451800">WHERE rownum = 1: The rownum = 1 is required for Flink to recognize this query is deduplication.</li></ul>
|
||||
<div class="section" id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_section1627916611011"><h4 class="sectiontitle">Description</h4><ul id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_ul12220145108"><li id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_li82201045703">ROW_NUMBER(): Assigns a unique, sequential number to each row, starting with one.</li><li id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_li152201045504">PARTITION BY col1[, col2...]: Specifies the partition columns, for example, the deduplicate key.</li><li id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_li1422020451403">ORDER BY time_attr [asc|desc]: Specifies the ordering column, which must be a time attribute. Currently Flink supports proctime only. Ascending (ASC) sorting keeps only the first row, while descending (DESC) sorting keeps only the last row.</li><li id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_li1922012451800">WHERE rownum = 1: The rownum = 1 is required for Flink to recognize this query is deduplication.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_section17171113913120"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0328__en-us_topic_0000001119232092_en-us_topic_0000001132426601_p10805114812117">None</p>
|
||||
</div>
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user