forked from docs/doc-exports
Su, Xiaomeng
be9eabe464
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com> Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
269 lines
20 KiB
HTML
269 lines
20 KiB
HTML
<a name="dli_08_15019"></a><a name="dli_08_15019"></a>
|
|
|
|
<h1 class="topictitle1">CSV</h1>
|
|
<div id="body0000001262815694"><div class="section" id="dli_08_15019__section226021020326"><h4 class="sectiontitle">Function</h4><p id="dli_08_15019__p31920232323">The CSV format allows you to read and write CSV data based on a CSV schema. Currently, the CSV schema is derived from table schema. For details, see <a href="https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/table/formats/csv/" target="_blank" rel="noopener noreferrer">CSV Format</a>.</p>
|
|
</div>
|
|
<div class="section" id="dli_08_15019__section122491371116"><h4 class="sectiontitle">Supported Connectors</h4><ul id="dli_08_15019__ul188074312166"><li id="dli_08_15019__li18073310163">Kafka</li><li id="dli_08_15019__li16323181520161">Upsert Kafka</li><li id="dli_08_15019__li14357112884017">FileSystem</li></ul>
|
|
</div>
|
|
<div class="section" id="dli_08_15019__section15970162015401"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_15019__table6702534155617" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Description</caption><thead align="left"><tr id="dli_08_15019__row1270218346560"><th align="left" class="cellrowborder" valign="top" width="15.55%" id="mcps1.3.3.2.2.6.1.1"><p id="dli_08_15019__p2070203417568">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="11.19%" id="mcps1.3.3.2.2.6.1.2"><p id="dli_08_15019__p1270273412568">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="10.58%" id="mcps1.3.3.2.2.6.1.3"><p id="dli_08_15019__p1703634175615">Default value</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="13.5%" id="mcps1.3.3.2.2.6.1.4"><p id="dli_08_15019__p207031134135615">Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="49.18%" id="mcps1.3.3.2.2.6.1.5"><p id="dli_08_15019__p19703103414569">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dli_08_15019__row6703203412568"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p103551327175815">format</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p93551727105810">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p11355172712585">None</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p17355122775815">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p1035512713585">Format to be used. Set the value to <strong id="dli_08_15019__b1170314499458">csv</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row16703113495617"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p1874315505918">csv.field-delimiter</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p674310513592">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p20666171265910">,</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p157431051594">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p9743853595">Field delimiter character, which must be a single character. You can use backslash to specify special characters, for example, <strong id="dli_08_15019__b1946014101610">\t</strong> represents the tab character. You can also use unicode to specify them in plain SQL, for example, <strong id="dli_08_15019__b62953511617">'csv.field-delimiter' = U&'\0001'</strong> represents the 0x01 character.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row5943184510584"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p14743756594">csv.disable-quote-character</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p274318516591">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p16743115105912">false</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p67431451591">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p474319513597">Disabled quote character for enclosing field values. If you set this parameter to <strong id="dli_08_15019__b954151135412">true</strong>, <strong id="dli_08_15019__b0879115319544">csv.quote-character</strong> cannot be set.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row4216952155814"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p1174395115913">csv.quote-character</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p1174310512596">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p8813181815595">''</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p3743145155920">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p3743155135919">Quote character for enclosing field values.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row512914914587"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p1674385175912">csv.allow-comments</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p2074313515912">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p1774315165917">false</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p12743135205919">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p674314595919">Ignore comment lines that start with <strong id="dli_08_15019__b6879134515710">#</strong>. If you set this parameter to <strong id="dli_08_15019__b1141133411574">true</strong>, make sure to also ignore parse errors to allow empty rows.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row2614554145814"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p157437512598">csv.ignore-parse-errors</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p3743145175918">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p074455175910">false</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p274410516599">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p974445105910">Whether fields and rows with parse errors will be skipped or failed. The default value is <strong id="dli_08_15019__b1890513775917">false</strong>, indicating that an error will be thrown. Fields are set to null in case of errors.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row6703163435617"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p15744185105910">csv.array-element-delimiter</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p17448585914">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p894143665919">;</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p6744115155910">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p18744752598">Array element delimiter string for separating array and row element values.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row1370353475616"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p6744355597">csv.escape-character</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p147442555913">No</p>
|
|
<p id="dli_08_15019__p974418515591"></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p774405155911">None</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p1674410519597">String</p>
|
|
<p id="dli_08_15019__p12744135175910"></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p1774418515592">Escape character for escaping values</p>
|
|
<p id="dli_08_15019__p774495105918"></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row7742944207"><td class="cellrowborder" valign="top" width="15.55%" headers="mcps1.3.3.2.2.6.1.1 "><p id="dli_08_15019__p166610495216">csv.null-literal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.19%" headers="mcps1.3.3.2.2.6.1.2 "><p id="dli_08_15019__p1366616499212">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.58%" headers="mcps1.3.3.2.2.6.1.3 "><p id="dli_08_15019__p66663498211">None</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.5%" headers="mcps1.3.3.2.2.6.1.4 "><p id="dli_08_15019__p136661049728">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.18%" headers="mcps1.3.3.2.2.6.1.5 "><p id="dli_08_15019__p1866616491129">Null literal string that is interpreted as a null value.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_15019__section2152182012913"><h4 class="sectiontitle">Data Type Mapping</h4><p id="dli_08_15019__p14381844101018">Currently, the CSV schema is always derived from table schema. Explicitly defining a CSV schema is not supported yet. Flink CSV format uses <a href="https://github.com/FasterXML/jackson-databind" target="_blank" rel="noopener noreferrer">jackson databind API</a> to parse and generate CSV string.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_15019__table17149122651016" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Data type mapping</caption><thead align="left"><tr id="dli_08_15019__row61495264108"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.3.2.3.1.1"><p id="dli_08_15019__p11495267109">Flink SQL Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.3.2.3.1.2"><p id="dli_08_15019__p11149526111017">CSV Type</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dli_08_15019__row7149172631011"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p16149226151020">CHAR/VARCHAR/STRING</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p614972651018">String</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row181492260105"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p1514915269105">BOOLEAN</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p714962641017">Boolean</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row14149626191010"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p71495266105">BINARY/VARBINARY</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p5149026161016">string with encoding: base64</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row4149226181016"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p4149132661017">DECIMAL</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p1214912267108">Number</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row161493268108"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p614912615109">TINYINT</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p614972621018">Number</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row18149172661016"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p15149026161019">SMALLINT</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p4149162620108">Number</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row614982612106"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p20149726141017">INT</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p10149426171019">Number</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row161491026141020"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p6149526141014">BIGINT</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p1014972651010">Number</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row71491726191018"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p8149162614103">FLOAT</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p1014918262102">Number</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row71492260108"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p41491262103">DOUBLE</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p614902681012">Number</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row1614962681015"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p61498268104">DATE</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p1214962612107">string with format: date</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row4149526181014"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p1614902611015">TIME</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p2150026171017">string with format: time</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row71501026111020"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p181501026161020">TIMESTAMP</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p11506260104">string with format: date-time</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row141501726161015"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p1615012671013">INTERVAL</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p915092611015">Number</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row18150152619107"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p1915022618103">ARRAY</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p16150132610101">array</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_15019__row151505262102"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_08_15019__p121501426151015">ROW</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_08_15019__p1315012651014">object</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_15019__section13511182319310"><h4 class="sectiontitle">Example</h4><p id="dli_08_15019__p15881132116016">Use Kafka to send data and output the data to Print.</p>
|
|
<ol id="dli_08_15019__ol840395722311"><li id="dli_08_15019__li04031578234"><span>Create a datasource connection for the communication with the VPC and subnet where Kafka locates and bind the connection to the queue. Set a security group and inbound rule to allow access of the queue and test the connectivity of the queue using the Kafka IP address. For example, locate a general-purpose queue where the job runs and choose <strong id="dli_08_15019__b19637102614292">More</strong> > <strong id="dli_08_15019__b1663872619298">Test Address Connectivity</strong> in the <strong id="dli_08_15019__b186382026152915">Operation</strong> column. If the connection is successful, the datasource is bound to the queue. Otherwise, the binding fails.</span></li><li id="dli_08_15019__li1599913011242"><span>Create a Flink OpenSource SQL job. Copy the following statement and submit the job:</span><p><pre class="screen" id="dli_08_15019__screen299960162418">CREATE TABLE kafkaSource (
|
|
order_id string,
|
|
order_channel string,
|
|
order_time string,
|
|
pay_amount double,
|
|
real_pay double,
|
|
pay_time string,
|
|
user_id string,
|
|
user_name string,
|
|
area_id string
|
|
) WITH (
|
|
'connector' = 'kafka',
|
|
'topic' = '<em id="dli_08_15019__i1727019331823"><strong id="dli_08_15019__b158125326210">kafkaTopic</strong></em>',
|
|
'properties.bootstrap.servers' = '<em id="dli_08_15019__i366724417446"><strong id="dli_08_15019__b1466716449449">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
|
|
'properties.group.id' = '<em id="dli_08_15019__i1825515215318"><strong id="dli_08_15019__b11855512311">GroupId</strong></em>',
|
|
'scan.startup.mode' = 'latest-offset',
|
|
'format' = 'csv'
|
|
);
|
|
|
|
|
|
CREATE TABLE printSink (
|
|
order_id string,
|
|
order_channel string,
|
|
order_time string,
|
|
pay_amount double,
|
|
real_pay double,
|
|
pay_time string,
|
|
user_id string,
|
|
user_name string,
|
|
area_id string
|
|
) WITH (
|
|
'connector' = 'print'
|
|
);
|
|
insert into printSink select * from kafkaSource;</pre>
|
|
</p></li><li id="dli_08_15019__li1511420343241"><span>Insert the following data into the source Kafka topic:</span><p><pre class="screen" id="dli_08_15019__screen107391221112410">202103251505050001,appShop,2021-03-25 15:05:05,500.00,400.00,2021-03-25 15:10:00,0003,Cindy,330108
|
|
|
|
202103241606060001,appShop,2021-03-24 16:06:06,200.00,180.00,2021-03-24 16:10:06,0001,Alice,330106</pre>
|
|
</p></li><li id="dli_08_15019__li4353143193117"><span>Perform the following operations to view the data result in the <strong id="dli_08_15019__b33942589492658">taskmanager.out</strong> file:</span><p><ol type="a" id="dli_08_15019__ol864115198285"><li id="dli_08_15019__li10901621122819">Log in to the DLI console. In the navigation pane, choose <strong id="dli_08_15019__b91215867291920">Job Management</strong> > <strong id="dli_08_15019__b63944497691920">Flink Jobs</strong>.</li><li id="dli_08_15019__li1912163912282">Click the name of the corresponding Flink job, choose <strong id="dli_08_15019__b137504787291931">Run Log</strong>, click <strong id="dli_08_15019__b62589126191931">OBS Bucket</strong>, and locate the folder of the log you want to view according to the date.</li><li id="dli_08_15019__li0641191914285">Go to the folder of the date, find the folder whose name contains <strong id="dli_08_15019__b10927081684493">taskmanager</strong>, download the <strong id="dli_08_15019__b8551755404493">.out</strong> file, and view result logs.</li></ol>
|
|
<pre class="screen" id="dli_08_15019__screen14251955184812">+I[202103251505050001, appShop, 2021-03-25 15:05:05, 500.0, 400.0, 2021-03-25 15:10:00, 0003, Cindy, 330108]
|
|
+I[202103241606060001, appShop, 2021-03-24 16:06:06, 200.0, 180.0, 2021-03-24 16:10:06, 0001, Alice, 330106]</pre>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_15014.html">Formats</a></div>
|
|
</div>
|
|
</div>
|
|
|