# Creating a Dataset

#### Function

This API is used to create a dataset.

#### URI

POST /v2/{project_id}/datasets

Table 1 Path Parameters
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
project_id | Yes | String | Project ID. For details about how to obtain the project ID, see Obtaining a Project ID.

#### Request Parameters

Table 2 Request body parameters
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
data_format | No | String | Data format. The options are as follows: - Default: default format - CarbonData: CarbonData (supported only by table datasets)
data_sources | No | Array of DataSource objects | Input dataset path, which is used to synchronize source data (such as images, text files, and audio files) in the directory and its subdirectories to the dataset. For a table dataset, this parameter indicates the import directory. The work directory of a table dataset cannot be an OBS path in a KMS-encrypted bucket.
dataset_name | Yes | String | Dataset name. The value contains 1 to 100 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed, for example, dataset-9f3b.
dataset_type | No | Integer | Dataset type. The options are as follows: - 0: image classification - 1: object detection - 100: text classification - 101: named entity recognition - 102: text triplet - 200: sound classification - 201: speech content - 202: speech paragraph labeling - 400: table dataset - 600: video labeling - 900: custom format
description | No | String | Dataset description. The value is empty by default. The description contains 0 to 256 characters and does not support the following special characters: ^!<>=&"'
import_annotations | No | Boolean | Whether to automatically import the labeling information in the input directory, supporting detection, image classification, and text classification. The options are as follows: - true: Import labeling information in the input directory. (Default value) - false: Do not import labeling information in the input directory.
import_data | No | Boolean | Whether to import data. This parameter is used only for table datasets. The options are as follows: - true: Import data when creating a database. - false: Do not import data when creating a database. (Default value)
label_format | No | LabelFormat object | Label format information. This parameter is used only for text datasets.
labels | No | Array of Label objects | Dataset label list.
managed | No | Boolean | Whether to host a dataset. The options are as follows: - true: Host a dataset. - false: Do not host a dataset. (Default value)
schema | No | Array of Field objects | Schema list.
work_path | Yes | String | Output dataset path, which is used to store output files such as label files. - The format is /Bucket name/File path, for example, /obs-bucket/flower/rose/. (The directory is used as the path.)- A bucket cannot be directly used as a path.- The output dataset path is different from the input dataset path or its subdirectory.- The value contains 3 to 700 characters.
work_path_type | Yes | Integer | Type of the dataset output path. The options are as follows: - 0: OBS bucket (default value)
workforce_information | No | WorkforceInformation object | Team labeling information.
workspace_id | No | String | Workspace ID. If no workspace is created, the default value is 0. If a workspace is created and used, use the actual value.

Table 3 DataSource
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
data_path | No | String | Data source path.
data_type | No | Integer | Data type. The options are as follows: - 0: OBS bucket (default value) - 1: GaussDB(DWS) - 2: DLI - 3: RDS - 4: MRS - 5: AI Gallery - 6: Inference service
schema_maps | No | Array of SchemaMap objects | Schema mapping information corresponding to the table data.
source_info | No | SourceInfo object | Information required for importing a table data source.
with_column_header | No | Boolean | Whether the first row in the file is a column name. This field is valid for the table dataset. The options are as follows: - true: The first row in the file is the column name. - false: The first row in the file is not the column name.

Table 4 SchemaMap
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
dest_name | No | String | Name of the destination column.
src_name | No | String | Name of the source column.

Table 5 SourceInfo
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
cluster_id | No | String | ID of an MRS cluster.
cluster_mode | No | String | Running mode of an MRS cluster. The options are as follows: - 0: normal cluster - 1: security cluster
cluster_name | No | String | Name of an MRS cluster.
database_name | No | String | Name of the database to which the table dataset is imported.
input | No | String | HDFS path of a table dataset.
ip | No | String | IP address of your GaussDB(DWS) cluster.
port | No | String | Port number of your GaussDB(DWS) cluster.
queue_name | No | String | DLI queue name of a table dataset.
subnet_id | No | String | Subnet ID of an MRS cluster.
table_name | No | String | Name of the table to which a table dataset is imported.
user_name | No | String | Username, which is mandatory for GaussDB(DWS) data.
user_password | No | String | User password, which is mandatory for GaussDB(DWS) data.
vpc_id | No | String | ID of the VPC where an MRS cluster resides.

Table 6 LabelFormat
Parameter | Mandatory | Type | Description
--- | --- | --- | ---

label_type

No

String

Label type of text classification. The options are as follows: - 0: The label is separated from the text, and they are distinguished by the fixed suffix _result. For example, the text file is abc.txt, and the label file is abc_result.txt. - 1: Default value. Labels and texts are stored in the same file and separated by separators. You can use text_sample_separator to specify the separator between the text and label and text_label_separator to specify the separator between labels.

text_label_separator | No | String | Separator between labels. By default, a comma (,) is used as the separator. The separator needs to be escaped. The separator can contain only one character, such as a letter, a digit, or any of the following special characters: !@#$%^&*_=|?/':.;,
text_sample_separator | No | String | Separator between the text and label. By default, the Tab key is used as the separator. The separator needs to be escaped. The separator can contain only one character, such as a letter, a digit, or any of the following special characters: !@#$%^&*_=|?/':.;,

Table 7 Label
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
attributes | No | Array of LabelAttribute objects | Multi-dimensional attribute of a label. For example, if the label is music, attributes such as style and artist may be included.
name | No | String | Label name.
property | No | LabelProperty object | Basic attribute key-value pair of a label, such as color and shortcut keys.
type | No | Integer | Label type. The options are as follows: - 0: image classification - 1: object detection - 100: text classification - 101: named entity recognition - 102: text triplet relationship - 103: text triplet entity - 200: speech classification - 201: speech content - 202: speech paragraph labeling - 600: video classification

Table 8 LabelAttribute
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
default_value | No | String | Default value of a label attribute.
id | No | String | Label attribute ID.
name | No | String | Label attribute name.
type | No | String | Label attribute type. The options are as follows: - text: text - select: single-choice drop-down list
values | No | Array of LabelAttributeValue objects | List of label attribute values.

Table 9 LabelAttributeValue
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
id | No | String | Label attribute value ID.
value | No | String | Label attribute value.

Table 10 LabelProperty
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
@modelarts:color | No | String | Default attribute: Label color, which is a hexadecimal code of the color. By default, this parameter is left blank. Example: #FFFFF0.
@modelarts:default_shape | No | String | Default attribute: Default shape of an object detection label (dedicated attribute). By default, this parameter is left blank. The options are as follows: - bndbox: rectangle - polygon: polygon - circle: circle - line: straight line - dashed: dotted line - point: point - polyline: polyline
@modelarts:from_type | No | String | Default attribute: Type of the head entity in the triplet relationship label. This attribute must be specified when a relationship label is created. This parameter is used only for the text triplet dataset.
@modelarts:rename_to | No | String | Default attribute: The new name of the label.
@modelarts:shortcut | No | String | Default attribute: Label shortcut key. By default, this parameter is left blank. For example: D.
@modelarts:to_type | No | String | Default attribute: Type of the tail entity in the triplet relationship label. This attribute must be specified when a relationship label is created. This parameter is used only for the text triplet dataset.

Table 11 Field
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
description | No | String | Schema description.
name | No | String | Schema name.
schema_id | No | Integer | Schema ID.
type | No | String | Schema value type.

Table 12 WorkforceInformation
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
data_sync_type | No | Integer | Synchronization type. The options are as follows: - 0: not to be synchronized - 1: data to be synchronized - 2: label to be synchronized - 3: data and label to be synchronized
repetition | No | Integer | Number of persons who label each sample. The minimum value is 1.
synchronize_auto_labeling_data | No | Boolean | Whether to synchronously update auto labeling data. The options are as follows: - true: Update auto labeling data synchronously. - false: Do not update auto labeling data synchronously.
synchronize_data | No | Boolean | Whether to synchronize updated data, such as uploading files, synchronizing data sources, and assigning imported unlabeled files to team members. The options are as follows: - true: Synchronize updated data to team members. - false: Do not synchronize updated data to team members.
task_id | No | String | ID of a team labeling task.
task_name | Yes | String | Name of a team labeling task. The value contains 1 to 64 characters, including only letters, digits, underscores (_), and hyphens (-).
workforces_config | No | WorkforcesConfig object | Manpower assignment of a team labeling task. You can delegate the administrator to assign the manpower or do it by yourself.

Table 13 WorkforcesConfig
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
agency | No | String | Team administrator.
workforces | No | Array of WorkforceConfig objects | List of teams that execute labeling tasks.

Table 14 WorkforceConfig
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
workers | No | Array of Worker objects | List of labeling team members.
workforce_id | No | String | ID of a labeling team.
workforce_name | No | String | Name of a labeling team. The value contains 0 to 1024 characters and does not support the following special characters: !<>=&"'

Table 15 Worker
Parameter | Mandatory | Type | Description
--- | --- | --- | ---
create_time | No | Long | Creation time.
description | No | String | Labeling team member description. The value contains 0 to 256 characters and does not support the following special characters: ^!<>=&"'
email | No | String | Email address of a labeling team member.
role | No | Integer | Role. The options are as follows: - 0: labeling personnel - 1: reviewer - 2: team administrator - 3: dataset owner
status | No | Integer | Current login status of a labeling team member. The options are as follows: - 0: The invitation email has not been sent. - 1: The invitation email has been sent but the user has not logged in. - 2: The user has logged in. - 3: The labeling team member has been deleted.
update_time | No | Long | Update time.
worker_id | No | String | ID of a labeling team member.
workforce_id | No | String | ID of a labeling team.

#### Response Parameters

Status code: 201

Table 16 Response body parameters
Parameter | Type | Description
--- | --- | ---
dataset_id | String | Dataset ID.
error_code | String | Error code.
error_msg | String | Error message.
import_task_id | String | ID of an import task.

#### Example Requests

- Creating an Image Classification Dataset

      {
        "workspace_id" : "0",
        "dataset_name" : "dataset-457f",
        "dataset_type" : 0,
        "data_sources" : [ {
          "data_type" : 0,
          "data_path" : "/test-obs/classify/input/cat-dog/"
        } ],
        "description" : "",
        "work_path" : "/test-obs/classify/output/",
        "work_path_type" : 0,
        "labels" : [ {
          "name" : "Cat",
          "type" : 0,
          "property" : {
            "@modelarts:color" : "#3399ff"
          }
        }, {
          "name" : "Dog",
          "type" : 0,
          "property" : {
            "@modelarts:color" : "#3399ff"
          }
        } ]
      }

- Creating an Object Detection Dataset

      {
        "workspace_id" : "0",
        "dataset_name" : "dataset-95a6",
        "dataset_type" : 1,
        "data_sources" : [ {
          "data_type" : 0,
          "data_path" : "/test-obs/detect/input/cat-dog/"
        } ],
        "description" : "",
        "work_path" : "/test-obs/detect/output/",
        "work_path_type" : 0,
        "labels" : [ {
          "name" : "Cat",
          "type" : 1,
          "property" : {
            "@modelarts:color" : "#3399ff"
          }
        }, {
          "name" : "Dog",
          "type" : 1,
          "property" : {
            "@modelarts:color" : "#3399ff"
          }
        } ]
      }

- Creating a Table Dataset

      {
        "workspace_id" : "0",
        "dataset_name" : "dataset-de83",
        "dataset_type" : 400,
        "data_sources" : [ {
          "data_type" : 0,
          "data_path" : "/test-obs/table/input/",
          "with_column_header" : true
        } ],
        "description" : "",
        "work_path" : "/test-obs/table/output/",
        "work_path_type" : 0,
        "schema" : [ {
          "schema_id" : 1,
          "name" : "150",
          "type" : "STRING"
        }, {
          "schema_id" : 2,
          "name" : "4",
          "type" : "STRING"
        }, {
          "schema_id" : 3,
          "name" : "setosa",
          "type" : "STRING"
        }, {
          "schema_id" : 4,
          "name" : "versicolor",
          "type" : "STRING"
        }, {
          "schema_id" : 5,
          "name" : "virginica",
          "type" : "STRING"
        } ],
        "import_data" : true
      }

#### Example Responses

Status code: 201

Created

    {
      "dataset_id" : "WxCREuCkBSAlQr9xrde"
    }

#### Status Codes

Status Code | Description
--- | ---
201 | Created
401 | Unauthorized
403 | Forbidden
404 | Not Found

#### Error Codes

See Error Codes.

Parent topic: Dataset Management