vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0273.html
Su, Xiaomeng 76a5b1ee83 dli_sqlreference_20240227 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2024-03-27 22:02:33 +00:00

102 lines
9.5 KiB
HTML
Raw Blame History

<a name="dli_08_0273"></a><a name="dli_08_0273"></a>
<h1 class="topictitle1">Custom Source Stream</h1>
<div id="body1587982674377"><p id="dli_08_0273__p5831736182311">Compile code to obtain data from the desired cloud ecosystem or open-source ecosystem as the input data of Flink jobs.</p>
<div class="section" id="dli_08_0273__section820185715231"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0273__screen533213913485"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">SOURCE</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">stream_id</span><span class="w"> </span><span class="p">(</span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;user_defined&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">type_class_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">type_class_parameter</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;&quot;</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="p">(</span><span class="k">TIMESTAMP</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">timeindicator</span><span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">timeindicator</span><span class="p">)</span><span class="o">?</span><span class="p">);</span><span class="n">timeindicator</span><span class="p">:</span><span class="n">PROCTIME</span><span class="w"> </span><span class="s1">'.'</span><span class="w"> </span><span class="n">PROCTIME</span><span class="o">|</span><span class="w"> </span><span class="n">ID</span><span class="w"> </span><span class="s1">'.'</span><span class="w"> </span><span class="n">ROWTIME</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0273__section204091366497"><h4 class="sectiontitle">Keywords</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0273__table2972926145319" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Keywords</caption><thead align="left"><tr id="dli_08_0273__row29913265537"><th align="left" class="cellrowborder" valign="top" width="16.53%" id="mcps1.3.3.2.2.4.1.1"><p id="dli_08_0273__p17998132611534">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9.49%" id="mcps1.3.3.2.2.4.1.2"><p id="dli_08_0273__p12310279537">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="73.98%" id="mcps1.3.3.2.2.4.1.3"><p id="dli_08_0273__p10982717536">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0273__row816152795316"><td class="cellrowborder" valign="top" width="16.53%" headers="mcps1.3.3.2.2.4.1.1 "><p id="dli_08_0273__p202362713532">type</p>
</td>
<td class="cellrowborder" valign="top" width="9.49%" headers="mcps1.3.3.2.2.4.1.2 "><p id="dli_08_0273__p6281227165310">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="73.98%" headers="mcps1.3.3.2.2.4.1.3 "><p id="dli_08_0273__p143216277530">Data source type. The value <strong id="dli_08_0273__b154171615701">user_defined</strong> indicates that the data source is a user-defined data source.</p>
</td>
</tr>
<tr id="dli_08_0273__row0343279539"><td class="cellrowborder" valign="top" width="16.53%" headers="mcps1.3.3.2.2.4.1.1 "><p id="dli_08_0273__p13427275537">type_class_name</p>
</td>
<td class="cellrowborder" valign="top" width="9.49%" headers="mcps1.3.3.2.2.4.1.2 "><p id="dli_08_0273__p34712277532">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="73.98%" headers="mcps1.3.3.2.2.4.1.3 "><p id="dli_08_0273__p20521827195312">Name of the source class for obtaining source data. The value must contain the complete package path.</p>
</td>
</tr>
<tr id="dli_08_0273__row955727195316"><td class="cellrowborder" valign="top" width="16.53%" headers="mcps1.3.3.2.2.4.1.1 "><p id="dli_08_0273__p156112719538">type_class_parameter</p>
</td>
<td class="cellrowborder" valign="top" width="9.49%" headers="mcps1.3.3.2.2.4.1.2 "><p id="dli_08_0273__p1654277535">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="73.98%" headers="mcps1.3.3.2.2.4.1.3 "><p id="dli_08_0273__p10706276532">Input parameter of the user-defined source class. Only one parameter of the string type is supported.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0273__section167071940500"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0273__p67781735573">The user-defined source class needs to inherit the <strong id="dli_08_0273__b918415351308">RichParallelSourceFunction</strong> class and specify the data type as Row. For example, define MySource class: <strong id="dli_08_0273__b1718912350010">public class MySource extends RichParallelSourceFunction&lt;Row&gt;{}</strong>. It aims to implement the <strong id="dli_08_0273__b15190113517019">open</strong>, <strong id="dli_08_0273__b151908351015">run</strong>, and <strong id="dli_08_0273__b18190183519013">close</strong> functions.</p>
<p id="dli_08_0273__p84501124107">Dependency pom:</p>
<pre class="screen" id="dli_08_0273__screen476617221083">&lt;dependency&gt;
&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
&lt;artifactId&gt;flink-streaming-java_2.11&lt;/artifactId&gt;
&lt;version&gt;${flink.version}&lt;/version&gt;
&lt;scope&gt;provided&lt;/scope&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
&lt;artifactId&gt;flink-core&lt;/artifactId&gt;
&lt;version&gt;${flink.version}&lt;/version&gt;
&lt;scope&gt;provided&lt;/scope&gt;
&lt;/dependency&gt;</pre>
</div>
<div class="section" id="dli_08_0273__section7600402288"><h4 class="sectiontitle">Example</h4><p id="dli_08_0273__p19311143182815">A data record is generated in each period. The data record contains only one field of the INT type. The initial value is 1 and the period is 60 seconds. The period is specified by an input parameter.</p>
</div>
<div class="codecoloring" codetype="Sql" id="dli_08_0273__screen194421611162913"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span>
<span class="normal">8</span>
<span class="normal">9</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">SOURCE</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">user_in_data</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="k">count</span><span class="w"> </span><span class="nb">INT</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span><span class="w"> </span>
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;user_defined&quot;</span><span class="p">,</span><span class="w"> </span>
<span class="w"> </span><span class="n">type_class_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;mySourceSink.MySource&quot;</span><span class="p">,</span><span class="w"> </span>
<span class="w"> </span><span class="n">type_class_parameter</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;60&quot;</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="k">TIMESTAMP</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">car_timestamp</span><span class="p">.</span><span class="n">rowtime</span><span class="p">;</span><span class="w"> </span>
</pre></div></td></tr></table></div>
</div>
<div class="note" id="dli_08_0273__note156291340143015"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0273__p863974010300">To customize the implementation of the source class, you need to pack the class in a JAR package and upload the UDF function on the SQL editing page.</p>
</div></div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0272.html">Custom Stream Ecosystem</a></div>
</div>
</div>
View Git Blame Copy Permalink