yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
UCS_openapi
doc-exports/docs/obs/tool-obsutil/obs_11_0064.html
weihongmin1 c4291b1dd5 OBS Util 0309 Verion 
Reviewed-by: Sabelnikov, Dmitriy <dmitriy.sabelnikov@t-systems.com>
Co-authored-by: weihongmin1 <weihongmin1@huawei.com>
Co-committed-by: weihongmin1 <weihongmin1@huawei.com>
2026-03-27 12:15:19 +00:00

38 KiB
Raw Permalink Blame History

Downloading Objects by Using an Authorization Code

Function

You can use this command to download an object or download objects in a batch by object name prefix to your local PC.

Do not change the source objects in the OBS bucket when downloading a single object or objects in batches. Otherwise, the download may fail or data may be inconsistent.

Command Line Structure

  • Windows
    • Enter the authorization code to download a single object.
      obsutil share-cp authorization_code file_or_folder_url -key=xxx [-ac=xxx] [-dryRun] [-tempFileDir=xxx] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-ps=auto] [-cpd=xxx][-fr] [-o=xxx] [-config=xxx]
    • Use the file path to transfer the authorization code and download a single object.
      obsutil share-cp file://authorization_code_file_url file_or_folder_url -key=xxx [-ac=xxx] [-dryRun] [-tempFileDir=xxx] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-ps=auto] [-cpd=xxx][-fr] [-o=xxx] [-config=xxx]
    • Enter the authorization code to download objects in a batch.
      obsutil share-cp authorization_code folder_url -r [-key=xxx] [-ac=xxx] [-dryRun] [-tempFileDir=xxx] [-f] [-u] [-vlength] [-vmd5] [-flat] [-j=1] [-p=1] [-threshold=52428800] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx]
    • Use the file path to transfer the authorization code and download objects in a batch.
      obsutil share-cp file://authorization_code_file_url folder_url -r [-key=xxx] [-ac=xxx] [-dryRun] [-tempFileDir=xxx] [-f] [-u] [-vlength] [-vmd5] [-flat] [-j=1] [-p=1] [-threshold=52428800] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx]
  • macOS or Linux
    • Enter the authorization code to download a single object.
      ./obsutil share-cp authorization_code file_or_folder_url -key=xxx [-ac=xxx] [-dryRun] [-tempFileDir=xxx] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-ps=auto] [-cpd=xxx][-fr] [-o=xxx] [-config=xxx]
    • Use the file path to transfer the authorization code and download a single object.
      ./obsutil share-cp file://authorization_code_file_url file_or_folder_url -key=xxx [-ac=xxx] [-dryRun] [-tempFileDir=xxx] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-ps=auto] [-cpd=xxx][-fr] [-o=xxx] [-config=xxx]
    • Enter the authorization code to download objects in a batch.
      ./obsutil share-cp authorization_code folder_url -r [-key=xxx] [-ac=xxx] [-dryRun] [-tempFileDir=xxx] [-f] [-u] [-vlength] [-vmd5] [-flat] [-j=1] [-p=1] [-threshold=52428800] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx]
    • Use the file path to transfer the authorization code and download objects in a batch.
      ./obsutil share-cp file://authorization_code_file_url folder_url -r [-key=xxx] [-ac=xxx] [-dryRun] [-tempFileDir=xxx] [-f] [-u] [-vlength] [-vmd5] [-flat] [-j=1] [-p=1] [-threshold=52428800] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx]

Examples

  • In Windows, you can run the obsutil share-cp xxx d:\temp\test.txt -ac=123456 -key=src/test.txt command to download a single object.
    obsutil share-cp  xxx  d:\temp\test.txt -ac=123456 -key=test/test.txt
The authorized prefix is [test/test.txt]

Start at 2024-09-30 07:55:36.7698045 +0000 UTC

Parallel:      3                   Jobs:          3
Threshold:     50.00MB           PartSize:      auto
VerifyLength:  false               VerifyMd5:     false
CheckpointDir: xxxx
TempFileDir: xxxx
Waiting to prepare the temp file [xxx].
[==========================================] 100.00% 4.86 KB/s 0s
Download successfully, 19B, n/a, https://endpoint:443/test/test.txt --> d:\temp\test.txt
The authorized prefix is [test/test.txt]
  • In Windows, you can run the obsutil share-cp xxx d:\temp -ac=123456 -f -r command to download objects in a batch.
    obsutil share-cp xxx d:\temp -ac=123456 -f -r
The authorized prefix is [test]

Start at 2024-09-30 08:05:07.0097508 +0000 UTC

Parallel:      3                   Jobs:          3
Threshold:     50.00MB             PartSize:      auto
VerifyLength:  false               VerifyMd5:     false
CheckpointDir: xxxx

Task id: 1a50b1dd-3f92-42ae-a974-ff8fe514c2c2
OutputDir: xxxx
TempFileDir: xxxx

[======================================================] 100.00% 155.59 KB/s 0s
Succeed count:   6         Failed count:    0
Succeed bytes: xxx
Metrics [max cost:153 ms, min cost:129 ms, average cost:92.00 ms, average tps:17.86, transferred size:70B]

Task id: 1a50b1dd-3f92-42ae-a974-ff8fe514c2c2
The authorized prefix is [test/]

Parameter Description

Parameter

Optional or Mandatory

Description

authorization_code

or

file://authorization_code_file_url

Mandatory

The code for authorization

NOTE:

If the authorization code starts with file://, the authorization code is obtained from a local file.

file_or_folder_url

Mandatory for downloading an object

The local file/folder path

folder_url

Mandatory for downloading objects in batches

The local folder path

key

Mandatory for downloading an object (additional parameter)

Optional for downloading objects in batches

The name of the object to be downloaded, or the name prefix of the objects to be downloaded in batches

This parameter cannot be left blank when downloading an object. The saving and naming rules are as follows:

  • If this parameter specifies a file or folder path that does not exist, the tool checks whether the value ends with a slash (/) or backslash (\). If yes, a folder is created based on the path, and the object is downloaded to this newly created directory.
  • If this parameter specifies a file or folder path that does not exist and the value does not end with a slash (/) or backslash (\), the object is downloaded to your local PC with the value of the parameter as the file name.
  • If this parameter specifies an existing file, the object is downloaded to your local PC overwriting the existing file, with the value of the parameter as the file name.
  • If this parameter specifies an existing folder, the object is downloaded to the directory specified by file_or_folder_url with the object name as the file name.

During batch download, the saving rules are as follows:

  • If this parameter is left blank, all objects in the authorized path specified in the authorization code are downloaded to the folder specified by folder_url.
  • If this parameter is configured, objects whose name prefix is the configured value in the bucket are downloaded to the directory specified by folder_url.
NOTE:
  • If this parameter is specified, objects starting with this prefix are listed.
  • For batch downloads, if this parameter is configured and the flat parameter is not, the name of the downloaded file includes the parent object's name prefix. If flat is configured, the name of the downloaded file does not include the parent object's name prefix.
NOTICE:

During batch download, if the flat option is selected and the object name prefix is empty or does not end with a slash (/) during authorization code creation, the downloaded object list may be empty.

ac

Optional (additional parameter)

The access code

NOTE:
  • If no access code is specified using this parameter, obsutil tool prompts you to enter the access code in interactive mode.
  • An access code is a six-digit string.

r

Mandatory for downloading objects in batches (additional parameter)

Copies objects in batches based on a specified object name prefix.

fr

Optional for downloading an object (additional parameter)

Generates an operation result file when downloading an object.

flat

Optional for downloading objects in batches (additional parameter)

The name prefix of the parent object is excluded when downloading objects in a batch.

tempFileDir

Optional (additional parameter)

The directory for storing temporary files during multipart download. The default value is the value of defaultTempFileDir in the configuration file.

NOTE:
  • If this parameter is left blank and the defaultTempFileDir parameter in the configuration file is also left blank, temporary files generated during multipart download are saved in the directory where to-be-downloaded files are located and end with the suffix of .obs.temp.
  • Temporary files generated during multipart download are stored in this directory. Therefore, ensure that the user who executes obsutil has the write permission on the path.
  • The available space of the partition where the path is located must be greater than the size of the objects to be downloaded.

dryRun

Optional (additional parameter)

Conducts a dry run.

u

Optional (additional parameter)

Indicates incremental download. When this parameter is set, each object is compared with the file at the corresponding local path. An object is downloaded only when the file does not exist, the file size differs from the object size, or the file was last modified earlier than the object.

vlength

Optional (additional parameter)

Checks whether the sizes of the local files are the same as those of the objects in the bucket after the download is complete.

vmd5

Optional (additional parameter)

Checks whether MD5 values of the local files are the same as those of the objects in the bucket after the download is complete.

NOTE:

Objects in the bucket must contain metadata x-obs-meta-md5chksum, or MD5 verification will be skipped.

CAUTION:

If your object needs encryption, do not use this parameter.

p

Optional (additional parameter)

The maximum number of concurrent multipart download tasks when downloading an object. The default value is the value of defaultParallels in the configuration file.

threshold

Optional (additional parameter)

The threshold for enabling multipart download, in bytes. The default value is the value of defaultBigfileThreshold in the configuration file.

NOTE:
  • If the size of the object to be downloaded is smaller than the threshold, download the object directly. If not, a multipart download is required.
  • If you download an object directly, no part record is generated, and resumable transmission is not supported.
  • This parameter value can contain a unit, for example, 1MB (indicating 1,048,576 bytes).

ps

Optional (additional parameter)

The size of each part in a multipart download task, in bytes. The default value is the value of defaultPartSize in the configuration file.

NOTE:
  • This parameter value can contain a unit, for example, 1MB (indicating 1,048,576 bytes).
  • The parameter can be set to auto. In this case, obsutil automatically sets the part size for each multipart task based on the source object size.

cpd

Optional (additional parameter)

The folder where the part records reside. The default value is .obsutil_checkpoint, the subfolder in the home directory of the user who executes obsutil commands.

NOTE:

A part record is generated during a multipart download and saved to the down subfolder. After the download succeeds, its part record is deleted automatically. If the download fails or is suspended, the system attempts to resume the task according to its part record when you perform the download the next time.

f

Optional for downloading objects in batches (additional parameter)

Runs in force mode.

j

Optional for downloading objects in batches (additional parameter)

The maximum number of concurrent tasks for downloading objects in a batch. The default value is the value of defaultJobs in the configuration file.

NOTE:

The tool ensures that this value is at least 1.

exclude

Optional for downloading objects in batches (additional parameter)

The matching patterns of source objects that are excluded, for example: *.txt

NOTE:
  • The asterisk (*) represents any group of characters, and the question mark (?) represents any single character. For instance, abc*.txt indicates any file whose name starts with abc and ends with .txt.
  • You can use \* to represent * and \? to represent ?.
  • If the name of the object to be downloaded matches the value of this parameter, the object is skipped.
NOTICE:
  • You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and double quotation marks for Windows.
  • The matching pattern applies to the object's absolute path, which includes the object name prefix and the object name starting from the root directory. For example, if an object's path in the bucket is obs://bucket/src1/src2/test.txt, its absolute path is src1/src2/test.txt.
  • This matching pattern applies only to objects whose names do not end with a slash (/).
  • Multiple exclude parameters can be specified, for example, -exclude=*.xxx -exclude=*.xxx.

include

Optional for downloading objects in batches (additional parameter)

The matching patterns of source objects that are included, for example: *.jpg

NOTE:
  • The asterisk (*) represents any group of characters, and the question mark (?) represents any single character.
  • You can use \* to represent * and \? to represent ?.
  • The exclude rule is applied first. If the name of the object to be downloaded does not match the exclude rule, the system then checks whether the object name matches this parameter. If it does, the object is downloaded; if it does not, the object is skipped.
NOTICE:
  • You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and double quotation marks for Windows.
  • The matching pattern applies to the object's absolute path, which includes the object name prefix and the object name starting from the root directory. For example, if an object's path in the bucket is obs://bucket/src1/src2/test.txt, its absolute path is src1/src2/test.txt.
  • This matching pattern applies only to objects whose names do not end with a slash (/).
  • Multiple include parameters can be specified, for example, -include=*.xxx -include=*.xxx.

timeRange

Optional for downloading objects in batches (additional parameter)

The time range matching pattern when downloading objects. Only objects whose latest modification time is within the configured time range are downloaded.

This pattern has a lower priority than the object matching patterns (exclude/include). That is, the time range matching pattern is executed after the configured object matching patterns.

NOTE:
  • The matching time range is represented in time1-time2, where time1 must be earlier than or the same as time2. The time format is yyyyMMddHHmmss.
  • Automatic formatting is supported. For example, yyyyMMdd is equivalent to yyyyMMdd000000, and yyyyMM is equivalent to yyyyMM01000000.
  • If this parameter is set to *-time2, all files whose last modification time is earlier than time2 are matched. If it is set to time1-*, all files whose last modification time is later than time1 are matched.
NOTICE:
  • Time in the matching pattern is the UTC time.
  • This matching pattern applies only to objects whose names do not end with a slash (/).

mf

Optional (additional parameter)

Indicates that the name matching pattern (include or exclude) and the time matching pattern (timeRange) also take effect on objects whose names end with a slash (/).

o

Optional (additional parameter)

The folder that stores the result files. After the command is executed, result files (possibly success, failure, and warning files) will be created in the specified folder. The default value is .obsutil_output, a subfolder in the user's home directory where obsutil commands are executed.

NOTE:
  • A result file should be named as follows: share-cp_{succeed | failed | warning}_report_time_TaskId.txt.
  • By default, the maximum size of a single result file is 30 MB and the maximum number of result files that can be retained is 1,024. You can set the maximum size and number by configuring recordMaxLogSize and recordBackups in the configuration file.
  • If there are multiple folders and files and you need to confirm the details of a failed task, refer to the failure result file cp_failed_report_time_TaskId.txt in the result folder and the log files in the log path.

config

Optional (additional parameter)

The user-defined configuration file for executing the current command. For details about parameters that can be configured, see Configuration Parameters.

Response

Refer to Response for uploading an object.

Parent topic: Object Commands