yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity

dli_sqlreference_20240227

Browse Source 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
This commit is contained in:
Su, Xiaomeng
2024-03-27 22:02:33 +00:00
committed by zuul
parent 03485beee6
commit 76a5b1ee83
314 changed files with 21401 additions and 2348 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
docs/dli/sqlreference/dli_spark_trans_array.html Normal file
View File

@ -0,0 +1,97 @@
<a name="dli_spark_trans_array"></a><a name="dli_spark_trans_array"></a>
<h1 class="topictitle1">trans_array</h1>
<div id="body8662426"><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p18471163815215">This function is used to convert an array split by a fixed separator in a column into multiple rows.</p>
<div class="section" id="dli_spark_trans_array__en-us_topic_0000001703740713_section822644119147"><h4 class="sectiontitle">Restrictions</h4><ul id="dli_spark_trans_array__en-us_topic_0000001703740713_ul157361137123715"><li id="dli_spark_trans_array__en-us_topic_0000001703740713_li1973673763720">All columns used as keys must be placed before the columns to be transposed.</li><li id="dli_spark_trans_array__en-us_topic_0000001703740713_li1173620372378">Only one UDTF is allowed in a select statement.</li><li id="dli_spark_trans_array__en-us_topic_0000001703740713_li3736123783712">This function cannot be used together with <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b10405164313274">group by</strong>, <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b6277154614271">cluster by</strong>, <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b20501145014277">distribute by</strong>, or sort by.</li></ul>
</div>
<div class="section" id="dli_spark_trans_array__en-us_topic_0000001703740713_section45291954203217"><h4 class="sectiontitle">Syntax</h4><pre class="screen" id="dli_spark_trans_array__en-us_topic_0000001703740713_screen19928459112517">trans_array (&lt;num_keys&gt;, &lt;separator&gt;, &lt;key1&gt;,&lt;key2&gt;,…,&lt;col1&gt;,&lt;col2&gt;,&lt;col3&gt;) as (&lt;key1&gt;,&lt;key2&gt;,...,&lt;col1&gt;, &lt;col2&gt;)</pre>
</div>
<div class="section" id="dli_spark_trans_array__en-us_topic_0000001703740713_section992014913317"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_spark_trans_array__en-us_topic_0000001703740713_table1829154762513" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="dli_spark_trans_array__en-us_topic_0000001703740713_row8830104792517"><th align="left" class="cellrowborder" valign="top" width="19.220000000000002%" id="mcps1.3.4.2.2.5.1.1"><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p983074711252">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="13.44%" id="mcps1.3.4.2.2.5.1.2"><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p6830124732517">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="15.559999999999999%" id="mcps1.3.4.2.2.5.1.3"><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p13318164512614">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="51.78%" id="mcps1.3.4.2.2.5.1.4"><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p08301547132513">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_spark_trans_array__en-us_topic_0000001703740713_row15830184792511"><td class="cellrowborder" valign="top" width="19.220000000000002%" headers="mcps1.3.4.2.2.5.1.1 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p683034714250">num_keys</p>
</td>
<td class="cellrowborder" valign="top" width="13.44%" headers="mcps1.3.4.2.2.5.1.2 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p12830184752518">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="15.559999999999999%" headers="mcps1.3.4.2.2.5.1.3 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p0318545102615">BIGINT</p>
</td>
<td class="cellrowborder" valign="top" width="51.78%" headers="mcps1.3.4.2.2.5.1.4 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p9484232112711">The value is a constant of the BIGINT type and must be greater than or equal to 0. This parameter indicates the number of columns that are used as transposed keys when being converted to multiple rows.</p>
</td>
</tr>
<tr id="dli_spark_trans_array__en-us_topic_0000001703740713_row1271091217107"><td class="cellrowborder" valign="top" width="19.220000000000002%" headers="mcps1.3.4.2.2.5.1.1 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p2710181281012">separator</p>
</td>
<td class="cellrowborder" valign="top" width="13.44%" headers="mcps1.3.4.2.2.5.1.2 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p47102012141018">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="15.559999999999999%" headers="mcps1.3.4.2.2.5.1.3 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p1571013126106">STRING</p>
</td>
<td class="cellrowborder" valign="top" width="51.78%" headers="mcps1.3.4.2.2.5.1.4 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p771041201014">The value is a constant of the STRING type, which is used to split a string into multiple elements. If this parameter is left blank, an error is reported.</p>
</td>
</tr>
<tr id="dli_spark_trans_array__en-us_topic_0000001703740713_row17814025161018"><td class="cellrowborder" valign="top" width="19.220000000000002%" headers="mcps1.3.4.2.2.5.1.1 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p8814152516109">keys</p>
</td>
<td class="cellrowborder" valign="top" width="13.44%" headers="mcps1.3.4.2.2.5.1.2 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p1814182591014">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="15.559999999999999%" headers="mcps1.3.4.2.2.5.1.3 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p17814142512107">STRING</p>
</td>
<td class="cellrowborder" valign="top" width="51.78%" headers="mcps1.3.4.2.2.5.1.4 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p681402513107">Columns used as keys during transpose. The number of columns is specified by <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b76331022113115">num_keys</strong>. If <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b1561302810314">num_keys</strong> specifies that all columns are used as keys (that is, <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b1098303617314">num_keys</strong> is equal to the number of all columns), only one row is returned.</p>
</td>
</tr>
<tr id="dli_spark_trans_array__en-us_topic_0000001703740713_row134381636171014"><td class="cellrowborder" valign="top" width="19.220000000000002%" headers="mcps1.3.4.2.2.5.1.1 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p44381236101015">cols</p>
</td>
<td class="cellrowborder" valign="top" width="13.44%" headers="mcps1.3.4.2.2.5.1.2 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p164381836151016">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="15.559999999999999%" headers="mcps1.3.4.2.2.5.1.3 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p643883691012">STRING</p>
</td>
<td class="cellrowborder" valign="top" width="51.78%" headers="mcps1.3.4.2.2.5.1.4 "><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p12438103618104">Array to be converted to rows. All columns following <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b62494117329">keys</strong> are regarded as arrays to be transposed and must be of the STRING type.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_spark_trans_array__en-us_topic_0000001703740713_section210162513312"><h4 class="sectiontitle">Return Values</h4><p id="dli_spark_trans_array__en-us_topic_0000001703740713_p184124317231">The return value is of the data type of the parameter.</p>
<div class="note" id="dli_spark_trans_array__en-us_topic_0000001703740713_note61422099395"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_spark_trans_array__en-us_topic_0000001703740713_ul0191143961318"><li id="dli_spark_trans_array__en-us_topic_0000001703740713_li61918392133">Transposed rows are returned. The new column name is specified by <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b15231193328">as</strong>.</li><li id="dli_spark_trans_array__en-us_topic_0000001703740713_li1319116395137">The type of the key column does not change, and the type of other columns is <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b1811844153219">STRING</strong>.</li><li id="dli_spark_trans_array__en-us_topic_0000001703740713_li13191113912138">The number of rows after the split is subject to the array with more rows. If there are not enough rows, <strong id="dli_spark_trans_array__en-us_topic_0000001703740713_b1230184412327">NULL</strong> is added.</li></ul>
</div></div>
</div>
<div class="section" id="dli_spark_trans_array__en-us_topic_0000001703740713_section13277192233920"><h4 class="sectiontitle">Example Code</h4><div class="p" id="dli_spark_trans_array__en-us_topic_0000001703740713_p533119589453">To help you understand how to use functions, this example provides source data and function examples based on the source data. Run the following command to create the salary table and add data:<pre class="screen" id="dli_spark_trans_array__en-us_topic_0000001703740713_screen1633135814458">CREATE EXTERNAL TABLE salary (
dept_id STRING, -- Department
userid string, -- <em id="dli_spark_trans_array__en-us_topic_0000001703740713_i42271345163510">Employee ID</em>
sal INT -- Salary
) ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
stored as textfile;</pre>
</div>
<p id="dli_spark_trans_array__en-us_topic_0000001703740713_p1333731464618">Adds the following data:</p>
<pre class="screen" id="dli_spark_trans_array__en-us_topic_0000001703740713_screen14338181414463">d1,user1/user4,1000/6000
d1,user2/user5,2000/7000
d1,user3/user6,3000
d2,user4/user7,4000
d2,user5/user8,5000/8000</pre>
<p id="dli_spark_trans_array__en-us_topic_0000001703740713_p1920571774810">Executes the SQL statement</p>
<pre class="screen" id="dli_spark_trans_array__en-us_topic_0000001703740713_screen112921114114817">select trans_array(1, "/", dept_id, user_id, sal) as (dept_id, user_id, sal) from salary; </pre>
<p id="dli_spark_trans_array__en-us_topic_0000001703740713_p1992311118506">The command output is as follows:</p>
<pre class="screen" id="dli_spark_trans_array__en-us_topic_0000001703740713_screen191221132165017">d1,user1,1000
d1,user4,6000
d1,user2,2000
d1,user5,7000
d1,user3,3000
d1,user6,NULL
d2,user4,4000
d2,user7,NULL
d2,user5,5000
d2,user8,8000</pre>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0476.html">Other Functions</a></div>
</div>
</div>