yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
8f01f116a5ad22cc6ad22bdf6cdc797063a2f5f1
doc-exports/docs/dataartsstudio/umn/dataartsstudio_01_0590.html
chenxiaoxiong f9e2808b7c DataArts UMN 20250810 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
2025-09-02 10:44:13 +00:00

33 lines
13 KiB
HTML
Raw Blame History

<a name="dataartsstudio_01_0590"></a><a name="dataartsstudio_01_0590"></a>
<h1 class="topictitle1">Encryption and Decryption During File Migration</h1>
<div id="body32001227"><div class="p" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p8849103820410">When you migrate files to a file system, CDM can encrypt and decrypt those files. Currently, CDM supports the following encryption modes:<ul id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_ul18895123871319"><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li589514384136"><a href="#dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_section17832638131012">AES-256-GCM</a></li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li13320551141320"><a href="#dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_section9481412161214">KMS Encryption</a></li></ul>
</div>
<div class="section" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_section17832638131012"><a name="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_section17832638131012"></a><a name="en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_section17832638131012"></a><h4 class="sectiontitle">AES-256-GCM</h4><p id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p1790035810137">Currently, only AES-256-GCM (NoPadding) is supported. This algorithm is used for encryption at the migration destination and decryption at the migration source. The supported source and destination data sources are as follows:</p>
<ul id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_ul87701553153516"><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li207700538355">Data sources supported by the migration source: HDFS (supported in the binary format)</li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li73481236174317">Data sources supported by the migration destination: HDFS (supported in the binary format)</li></ul>
<p id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p34047400442">The following part describes how to use AES-256-GCM to decrypt the encrypted files to be exported from HDFS and encrypt the files to be imported to HDFS. </p>
<ul id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_ul1011363095919"><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li411333035918"><strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b11102019163011">Configure decryption at the migration source.</strong><div class="p" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p826312599585">When you use CDM to create a job for exporting files from HDFS, set the migration source to HDFS and file format to binary, and set the following parameters in the advanced settings of <span class="parmname" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_parmname15918522546"><b>Source Job Configuration</b></span>:<ol id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_ol172632591587"><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li22631659165817"><strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b14432733181113">Encryption</strong>: Select <span class="parmvalue" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_parmvalue4601112963119"><b>AES-256-GCM</b></span>.</li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li2264159135819"><strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b1242010542567">DEK</strong>: The key must be the same as that configured in encryption. Otherwise, the decrypted data is incorrect and the system does not display an error message.</li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li5264175915813"><strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b4216121211573">IV</strong>: The initialization vector must be the same as that configured in encryption. Otherwise, the decrypted data is incorrect and the system does not display an error message.</li></ol>
</div>
<p id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p152641459145813">In this way, after CDM exports encrypted files from HDFS, the files written to the migration destination are decrypted plaintext files.</p>
</li></ul>
<ul id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_ul1470483215916"><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li1170423214594"><strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b10478102043217">Configure encryption at the migration destination.</strong><div class="p" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p13358169609">When you create a CDM job to import files to HDFS, set the migration destination to HDFS and file format to binary, and set the following parameters in the advanced settings of <span class="parmname" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_parmname54400446570"><b>Destination Job Configuration</b></span>:<ol id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_ol0358693010"><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li103581891605"><strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b250539963">Encryption</strong>: Select <span class="parmvalue" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_parmvalue1433506328"><b>AES-256-GCM</b></span>.</li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li123581291010"><strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b47724268154">DEK</strong>: custom encryption key. The key consists of 64 hexadecimal numbers. It is case-insensitive but must contain 64 characters. For example, <span class="parmvalue" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_parmvalue452252617341"><b>DD0AE00DFECD78BF051BCFDA25BD4E320DB0A7AC75A1F3FC3D3C56A457DCDC1B</b></span>.</li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li133581291202"><strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b1746143816184">IV</strong>: custom initialization vector. The initialization vector consists of 32 hexadecimal numbers. It is case-insensitive but must contain 32 characters. For example, <span class="parmvalue" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_parmvalue20226105543811"><b>5C91687BA886EDCD12ACBC3FF19A3C3F</b></span>.</li></ol>
</div>
<p id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p235829309">In this way, after CDM imports files to HDFS, the files in the destination HDFS are encrypted using the AES-256-GCM algorithm.</p>
</li></ul>
</div>
<div class="section" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_section9481412161214"><a name="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_section9481412161214"></a><a name="en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_section9481412161214"></a><h4 class="sectiontitle">KMS Encryption</h4><div class="note" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_note61031511121"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p310411191218">The migration source does not support KMS encryption.</p>
</div></div>
<p id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p1046511584176">CDM supports KMS encryption if tables, files, or a whole database is migrated to OBS. In the <strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b771417293338">Advanced Attributes</strong> area of the <strong id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_b1715162953314">Destination Job Configuration</strong> page, set the parameters.</p>
<p id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p07362133410">A key must be created in KMS of DEW in advance. For details, see the <i><cite id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_cite126011018428">Data Encryption Workshop User Guide</cite></i>.</p>
<p id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_p43851213144210">After KMS encryption is enabled, objects to be uploaded will be encrypted and stored on OBS. When you download the encrypted objects, the encrypted data will be decrypted on the server and displayed in plaintext to users.</p>
<div class="note" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_note7695438161618"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_ul18850204021613"><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li15143195412715">If KMS encryption is enabled, <a href="dataartsstudio_01_0591.html">MD5 verification</a> cannot be used.</li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li376512535341">If the KMS ID of another project is used, change <span class="parmname" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_parmname10703121812381"><b>Project ID</b></span> to the ID of the project to which KMS belongs. If KMS and CDM are in the same project, retain the default value of <span class="parmname" id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_parmname15720220347"><b>Project ID</b></span>.</li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li68501440171613">After KMS encryption is performed, the encryption status of the objects on OBS cannot be changed.</li><li id="dataartsstudio_01_0590__en-us_topic_0000001197659213_en-us_topic_0000001197658801_en-us_topic_0108275362_li2085014017162">A key in use cannot be deleted. Otherwise, the object encrypted with this key cannot be downloaded.</li></ul>
</div></div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0110.html">Key Operation Guide</a></div>
</div>
</div>
View Git Blame Copy Permalink