forked from docs/doc-exports
weihongmin1
a41a4e0331
Reviewed-by: Sabelnikov, Dmitriy <dmitriy.sabelnikov@t-systems.com> Co-authored-by: weihongmin1 <weihongmin1@huawei.com> Co-committed-by: weihongmin1 <weihongmin1@huawei.com>
32 lines
4.2 KiB
HTML
32 lines
4.2 KiB
HTML
<a name="obs_11_0059"></a><a name="obs_11_0059"></a>
|
|
|
|
<h1 class="topictitle1">Using obsutil for Resumable Data Transfer</h1>
|
|
<div id="body1561967257980"><p id="obs_11_0059__p8060118">obsutil supports resumable data transfer (upload, download, and copy) for large files by using the multipart algorithms for upload, download, and copy. You can set the threshold size for starting a multipart upload, download, or copy task based on your actual requirements to resume the upload, download, or copy task if the task fails or is interrupted. You can specify the threshold size for starting a multipart task in either of the following ways:</p>
|
|
<div class="section" id="obs_11_0059__section7860192173611"><h4 class="sectiontitle">Method 1</h4><p id="obs_11_0059__p16453886320">Set <strong id="obs_11_0059__b1961772595212">defaultBigfileThreshold</strong> in the configuration file. For details, see <a href="obs_11_0035.html">Configuration Parameters</a>.</p>
|
|
</div>
|
|
<div class="section" id="obs_11_0059__section136078437367"><h4 class="sectiontitle">Method 2</h4><p id="obs_11_0059__p91117221329">Set <strong id="obs_11_0059__b255022115513">threshold</strong>, a command-level parameter, when you run commands for <a href="obs_11_0013.html">object uploads</a>, <a href="obs_11_0018.html">object downloads</a>, <a href="obs_11_0017.html">object copy</a>, <a href="obs_11_0042.html">synchronous uploads of incremental objects</a>, <a href="obs_11_0043.html">synchronous downloads of incremental objects</a>, and <a href="obs_11_0044.html">synchronous copy of incremental objects</a>.</p>
|
|
<p id="obs_11_0059__p13855124703420">Example: <strong id="obs_11_0059__b81914191346">obsutil cp d:\temp\test.txt obs://bucket-test/key</strong><strong id="obs_11_0059__b1819121983412"> -threshold=52428800</strong></p>
|
|
<p id="obs_11_0059__p16230052386">In this command:</p>
|
|
<ul id="obs_11_0059__ul1935911147582"><li id="obs_11_0059__li53597141589"><strong id="obs_11_0059__b113715235217">obsutil cp d:\temp\test.txt obs://bucket-test/key</strong> uploads file <strong id="obs_11_0059__b774084513215">test.txt</strong> in the <strong id="obs_11_0059__b19235752828">temp</strong> directory under D drive to bucket <strong id="obs_11_0059__b0149557313">bucket-test</strong> and renames the file <strong id="obs_11_0059__b18527111835">key</strong>.</li><li id="obs_11_0059__li19751917193510"><strong id="obs_11_0059__b1483474719345">-threshold=52428800</strong> starts a multipart upload when the threshold 50 MB is reached.</li></ul>
|
|
<div class="p" id="obs_11_0059__p5591182612353">The following example is based on a Windows OS:<pre class="screen" id="obs_11_0059__screen09781053183312"><strong id="obs_11_0059__b109783531332">obsutil cp d:\temp\test.txt obs://bucket-test/key -threshold=52428800</strong>
|
|
|
|
Parallel: 3 Jobs: 3
|
|
Threshold: 50.00MB PartSize: auto
|
|
VerifyLength: false VerifyMd5: false
|
|
CheckpointDir: xxxx
|
|
|
|
[====================================================] 100.00% 1.68 MB/s 5s
|
|
Upload successfully, 8.46MB, d:\temp\test.txt --> obs://bucket-test/key</pre>
|
|
</div>
|
|
</div>
|
|
<p id="obs_11_0059__p17428430142414"></p>
|
|
<div class="note" id="obs_11_0059__note1315172617294"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="obs_11_0059__ul31441297298"><li id="obs_11_0059__li12144122913293">Priority: Command level parameter <strong id="obs_11_0059__b1391231911116">threshold</strong> has higher priority than the <strong id="obs_11_0059__b1127519508119">defaultBigfileThreshold</strong> in the configuration file.</li><li id="obs_11_0059__li1518018267331">The threshold size of a multipart task applies to single files or objects. When the size of a file or object is greater than the threshold value, the multipart algorithm is applied to the file or object.</li><li id="obs_11_0059__li19641630143518">The multipart algorithm and resumable data transfer are forcibly bound together. That is, once the multipart algorithm is used, the resumable data transfer is enabled for the task.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="obs_11_0033.html">Best Practices</a></div>
|
|
</div>
|
|
</div>
|
|
|