yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
fix_python
doc-exports/docs/modelarts/umn/develop-modelarts-0009.html
Lai, Weijian 6aa966a79a ModelArts UMN 24.3.0 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lai, Weijian <laiweijian4@huawei.com>
Co-committed-by: Lai, Weijian <laiweijian4@huawei.com>
2024-11-02 09:04:52 +00:00

194 lines
36 KiB
HTML
Raw Permalink Blame History

<a name="EN-US_TOPIC_0000002079176585"></a><a name="EN-US_TOPIC_0000002079176585"></a>
<h1 class="topictitle1">Creating an Algorithm</h1>
<div id="body0000001165752072"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p1541010399193">Your locally developed algorithms or algorithms developed using other tools can be uploaded to ModelArts for unified management. Note the following when creating a custom algorithm:</p>
<ol id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_ol9127193520"><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li112719175219"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_section13181641194916">Prerequisites</a></li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li8127169195218"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1478003731615">Accessing the Algorithm Creation Page</a></li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li812718935217"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1534119614910">Setting Basic Information</a></li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li121273917523"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section058543274910">Setting the Boot Mode</a></li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li1469901112514"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_section19643153517516">Configuring Pipelines</a></li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li1412789205211"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1883311313516">Defining Hyperparameters</a></li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li15127209115216"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1959512404445">Supported Policies</a></li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li512749125211"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section11326178153512">Adding Training Constraints</a></li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li112719911524"><a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section743865313613">Follow-Up Operations</a></li></ol>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_section13181641194916"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_section13181641194916"></a><a name="en-us_topic_0000001133351332_section13181641194916"></a><h4 class="sectiontitle">Prerequisites</h4><ul id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_ul11181841194911"><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li2181641134918">Training data is available. You can create a dataset in ModelArts or upload an existing dataset used for training to the OBS directory.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li101834194913">Your training script has been uploaded to an OBS directory. For details about how to develop a training script, see <a href="develop-modelarts-0008.html">Developing a Custom Script</a>.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li121818419498">At least one empty folder has been created in OBS for storing the training output.</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1478003731615"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1478003731615"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1478003731615"></a><h4 class="sectiontitle">Accessing the Algorithm Creation Page</h4><ol id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_ol010031113195"><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li1955149103612">Log in to the ModelArts console and choose <strong id="EN-US_TOPIC_0000002079176585__b8225136045">Algorithm Management</strong> in the navigation pane on the left.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li1371141419434">On the <strong id="EN-US_TOPIC_0000002079176585__b1032616161249">My algorithm</strong> page, click <strong id="EN-US_TOPIC_0000002079176585__b1332616161745">Create</strong>. The <strong id="EN-US_TOPIC_0000002079176585__b153261916641">Create Algorithm</strong> page is displayed.</li></ol>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1534119614910"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1534119614910"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1534119614910"></a><h4 class="sectiontitle">Setting Basic Information</h4><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p133597225492">Enter basic information, including <strong id="EN-US_TOPIC_0000002079176585__b614017291540">Name</strong> and <strong id="EN-US_TOPIC_0000002079176585__b1714116292045">Description</strong>.</p>
<div class="fignone" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_fig43607224493"><span class="figcap"><b>Figure 1 </b>Setting basic information</span><br><span><img id="EN-US_TOPIC_0000002079176585__image1910519214126" src="figure/en-us_image_0000002079098321.png" width="469.49" height="128.7307" title="Click to enlarge" class="imgResize"></span></div>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section058543274910"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section058543274910"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_section058543274910"></a><h4 class="sectiontitle">Setting the Boot Mode</h4><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p540519466127">Select a preset image to create an algorithm.</p>
<div class="p" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p19666165274911">Set <strong id="EN-US_TOPIC_0000002079176585__b13331245843">Image</strong>, <strong id="EN-US_TOPIC_0000002079176585__b1333184518413">Code Directory</strong>, and <strong id="EN-US_TOPIC_0000002079176585__b203324511417">Boot File</strong> based on the algorithm code. Ensure that the framework of the AI image you select is the same as the one you use for editing algorithm code. For example, if TensorFlow is used for editing algorithm code, select a TensorFlow image when you create an algorithm.
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_table76663522497" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row9666145216491"><th align="left" class="cellrowborder" valign="top" width="21.14%" id="mcps1.3.6.3.4.2.3.1.1"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p13666155244910">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="78.86%" id="mcps1.3.6.3.4.2.3.1.2"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p966695214912">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row1266685211491"><td class="cellrowborder" valign="top" width="21.14%" headers="mcps1.3.6.3.4.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p1866685220491">Boot Mode &gt; Preset image</p>
</td>
<td class="cellrowborder" valign="top" width="78.86%" headers="mcps1.3.6.3.4.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p1966611526497">AI images supported by the new-version training are displayed by default. For details, see <a href="develop-modelarts-0007.html">Overview</a>.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row46663522494"><td class="cellrowborder" valign="top" width="21.14%" headers="mcps1.3.6.3.4.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p20666752194912">Code Directory</p>
</td>
<td class="cellrowborder" valign="top" width="78.86%" headers="mcps1.3.6.3.4.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p1092213146155">OBS path for storing the algorithm code. The files required for training, such as the training code, dependence installation packages, and pre-generated models, are uploaded to the code directory.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p2066675215491">The code directory cannot contain files and directories uploaded by others or irrelevant files or directories. Otherwise, uploading data may fail.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p171070261712">Do not store training data in the code directory. When the training job starts, the data stored in the code directory will be downloaded to the backend. A large amount of training data may lead to a download failure.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p3993141110148">After you create the training job, ModelArts downloads the code directory and its subdirectories to the training container.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p19451345113817">Take OBS path <strong id="EN-US_TOPIC_0000002079176585__b94036915160">obs://obs-bucket/training-test/demo-code</strong> as an example. The content in the OBS path will be automatically downloaded to <strong id="EN-US_TOPIC_0000002079176585__b24038941610">${MA_JOB_DIR}/demo-code</strong> in the training container, and <strong id="EN-US_TOPIC_0000002079176585__b1540318918166">demo-code</strong> (customizable) is the last-level directory of the OBS path.</p>
<div class="note" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_note195456219174"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_ul179321959191811"><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li7932155920186">Any programming language is supported.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li09327597180">The total number of both files and folders cannot exceed 1,000.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li493225911184">The total file size cannot exceed 5 GB.</li></ul>
</div></div>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row1466665219497"><td class="cellrowborder" valign="top" width="21.14%" headers="mcps1.3.6.3.4.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p166645212493">Boot File</p>
</td>
<td class="cellrowborder" valign="top" width="78.86%" headers="mcps1.3.6.3.4.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p8613171885117">The file must be stored in the code directory and end with .py. ModelArts supports boot files edited only in Python.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p566655204911">The boot file in the code directory is used to start a training job.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="fignone" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_fig15355127133011"><span class="figcap"><b>Figure 2 </b>Using a custom script to create an algorithm</span><br><span><img id="EN-US_TOPIC_0000002079176585__image617415487121" src="figure/en-us_image_0000002079176925.png" width="469.49" height="102.82230000000001" title="Click to enlarge" class="imgResize"></span></div>
</div>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_section19643153517516"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_section19643153517516"></a><a name="en-us_topic_0000001133351332_section19643153517516"></a><h4 class="sectiontitle">Configuring Pipelines</h4><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p3643163585119">A preset image-based algorithm obtains data from an OBS bucket or dataset for model training. The training output is stored in an OBS bucket. The input and output parameters in your algorithm code must be parsed to enable data exchange between ModelArts and OBS. For details about how to develop code for training on ModelArts, see <a href="develop-modelarts-0008.html">Developing a Custom Script</a>.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p0643163555117">When you use a preset image to create an algorithm, configure the input and output parameters defined in the algorithm code.</p>
<ul id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_ul11643435155116"><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li176431035115113">Input configurations
<div class="tablenoborder"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_table126437359515"></a><a name="en-us_topic_0000001133351332_table126437359515"></a><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_table126437359515" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Input configurations</caption><thead align="left"><tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row1664393512515"><th align="left" class="cellrowborder" valign="top" width="16.04%" id="mcps1.3.7.4.1.1.2.3.1.1"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p186436359513">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="83.96000000000001%" id="mcps1.3.7.4.1.1.2.3.1.2"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p12643835165115">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row19643163505113"><td class="cellrowborder" valign="top" width="16.04%" headers="mcps1.3.7.4.1.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p166436355517">Parameter Name</p>
</td>
<td class="cellrowborder" valign="top" width="83.96000000000001%" headers="mcps1.3.7.4.1.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p5643113515114">If you use <strong id="EN-US_TOPIC_0000002079176585__b138761135102111">argparse</strong> in the algorithm code to parse <strong id="EN-US_TOPIC_0000002079176585__b88768352216">data_url</strong> into the data input, set this parameter to <strong id="EN-US_TOPIC_0000002079176585__b4876035162112">data_url</strong> when creating the algorithm. Set this parameter based on the data input parameter in your algorithm code.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p1643835195116">The code path parameter must be the same as the data input parameter parsed in your algorithm code. Otherwise, the algorithm code cannot obtain the input data.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row24341328154114"><td class="cellrowborder" valign="top" width="16.04%" headers="mcps1.3.7.4.1.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p34351828204110">Description</p>
</td>
<td class="cellrowborder" valign="top" width="83.96000000000001%" headers="mcps1.3.7.4.1.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p205027406411">Customize the description of the input parameter.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row1010145144219"><td class="cellrowborder" valign="top" width="16.04%" headers="mcps1.3.7.4.1.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p2111453428">Obtained from</p>
</td>
<td class="cellrowborder" valign="top" width="83.96000000000001%" headers="mcps1.3.7.4.1.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p141112452421">Select a source of the input parameter, <strong id="EN-US_TOPIC_0000002079176585__b137402410247">Hyperparameters</strong> (default) or <strong id="EN-US_TOPIC_0000002079176585__b167408410249">Environment variables</strong>.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row3643133511511"><td class="cellrowborder" valign="top" width="16.04%" headers="mcps1.3.7.4.1.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p1764315356519">Constraints</p>
</td>
<td class="cellrowborder" valign="top" width="83.96000000000001%" headers="mcps1.3.7.4.1.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__p388117534316">Enable this parameter to specify the input source. You can select a storage path or ModelArts dataset. This parameter is optional.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p19643103535113">If you select a ModelArts dataset, set the following parameters:</p>
<ul id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_ul364312353513"><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li18643835115113"><strong id="EN-US_TOPIC_0000002079176585__b770820132269">Labeling Type</strong>: For details, see <a href="datalabel-modelarts_0004.html">Creating a Labeling Job</a>.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li19643163585114"><strong id="EN-US_TOPIC_0000002079176585__b20167235287">Data Format</strong>, which can be <span class="parmname" id="EN-US_TOPIC_0000002079176585__parmname4502168172816"><b>Default</b></span>, <span class="parmname" id="EN-US_TOPIC_0000002079176585__parmname3503208152814"><b>CarbonData</b></span>, or both. <span class="parmname" id="EN-US_TOPIC_0000002079176585__parmname76384145287"><b>Default</b></span> indicates the manifest format.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li364303525113"><strong id="EN-US_TOPIC_0000002079176585__b1597871916281">Data Segmentation</strong> is available only for image classification, object detection, text classification, and sound classification datasets.<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p9643735115112">The options are <span class="parmname" id="EN-US_TOPIC_0000002079176585__parmname131753152299"><b>Segmented dataset</b></span>, <span class="parmname" id="EN-US_TOPIC_0000002079176585__parmname9175111522913"><b>Dataset not segmented</b></span>, and <span class="parmname" id="EN-US_TOPIC_0000002079176585__parmname10176191517291"><b>Unlimited</b></span>. For details, see <a href="dataprepare-modelarts-0028.html">Publishing a Data Version</a>.</p>
</li></ul>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row5644193516517"><td class="cellrowborder" valign="top" width="16.04%" headers="mcps1.3.7.4.1.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p166441735145114">Add</p>
</td>
<td class="cellrowborder" valign="top" width="83.96000000000001%" headers="mcps1.3.7.4.1.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p13644103575119">Add multiple input data sources based on your algorithm.</p>
</td>
</tr>
</tbody>
</table>
</div>
</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_li1664433510517">Output configurations
<div class="tablenoborder"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_table8644335195117"></a><a name="en-us_topic_0000001133351332_table8644335195117"></a><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_table8644335195117" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Output configurations</caption><thead align="left"><tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row136441235185119"><th align="left" class="cellrowborder" valign="top" width="17.04%" id="mcps1.3.7.4.2.1.2.3.1.1"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p5644163565111">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="82.96%" id="mcps1.3.7.4.2.1.2.3.1.2"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p196445359517">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row46441035125117"><td class="cellrowborder" valign="top" width="17.04%" headers="mcps1.3.7.4.2.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p06441635165119">Parameter Name</p>
</td>
<td class="cellrowborder" valign="top" width="82.96%" headers="mcps1.3.7.4.2.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p206447358513">If you use <strong id="EN-US_TOPIC_0000002079176585__b142161934123114">argparse</strong> in the algorithm code to parse <strong id="EN-US_TOPIC_0000002079176585__b2217183420317">train_url</strong> into the data output, set this parameter to <strong id="EN-US_TOPIC_0000002079176585__b62171834143113">train_url</strong> when creating the algorithm. Set this parameter based on the data output parameter in your algorithm code.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p8644123517511">The code path parameter must be the same as the data output parameter parsed in your algorithm code. Otherwise, the algorithm code cannot obtain the output path.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row129021942144610"><td class="cellrowborder" valign="top" width="17.04%" headers="mcps1.3.7.4.2.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p1190224294617">Description</p>
</td>
<td class="cellrowborder" valign="top" width="82.96%" headers="mcps1.3.7.4.2.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p119023423467">Customize the description of the output parameter.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__row158331911763"><td class="cellrowborder" valign="top" width="17.04%" headers="mcps1.3.7.4.2.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__p5450114062">Obtained from</p>
</td>
<td class="cellrowborder" valign="top" width="82.96%" headers="mcps1.3.7.4.2.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__p1745051417616">Select a source of the output parameter, <strong id="EN-US_TOPIC_0000002079176585__b18621125820323">Hyperparameters</strong> (default) or <strong id="EN-US_TOPIC_0000002079176585__b262245833219">Environment variables</strong>.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row18644143555114"><td class="cellrowborder" valign="top" width="17.04%" headers="mcps1.3.7.4.2.1.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p15644163575116">Add</p>
</td>
<td class="cellrowborder" valign="top" width="82.96%" headers="mcps1.3.7.4.2.1.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p1264473513510">Add multiple output data paths based on your algorithm.</p>
</td>
</tr>
</tbody>
</table>
</div>
</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1883311313516"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1883311313516"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1883311313516"></a><h4 class="sectiontitle">Defining Hyperparameters</h4><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p1916351343213">When you use a preset image to create an algorithm on ModelArts, you can customize hyperparameters so you can review or modify them anytime. Defined hyperparameters are displayed in the boot command and passed to your boot file as CLI parameters.</p>
<ol id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_ol934965191214"><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li881313334135">Import hyperparameters.<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p118282050101313"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li881313334135"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_li881313334135"></a>You can click <span class="parmname" id="EN-US_TOPIC_0000002079176585__parmname125412026183420"><b>Add hyperparameter</b></span> to manually add hyperparameters.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p6286230154716"></p>
</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li78461510124915">Edit hyperparameters. For details, see <a href="#EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_table143901732155115">Table 4</a>.
<div class="tablenoborder"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_table143901732155115"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_table143901732155115"></a><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_table143901732155115" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Hyperparameter parameters</caption><thead align="left"><tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row103901032175115"><th align="left" class="cellrowborder" valign="top" width="13.69%" id="mcps1.3.8.3.2.2.2.3.1.1"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p63901432105119">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="86.31%" id="mcps1.3.8.3.2.2.2.3.1.2"><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p10390173220518">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row1539013285118"><td class="cellrowborder" valign="top" width="13.69%" headers="mcps1.3.8.3.2.2.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p1339019325514">Name</p>
</td>
<td class="cellrowborder" valign="top" width="86.31%" headers="mcps1.3.8.3.2.2.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p9390143285111">Enter the hyperparameter name.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p1966731173310">Enter 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row13390332125116"><td class="cellrowborder" valign="top" width="13.69%" headers="mcps1.3.8.3.2.2.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p53901532105111">Type</p>
</td>
<td class="cellrowborder" valign="top" width="86.31%" headers="mcps1.3.8.3.2.2.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p13907328518">Select the data type of the hyperparameter. The value can be <strong id="EN-US_TOPIC_0000002079176585__b263421815354">String</strong>, <strong id="EN-US_TOPIC_0000002079176585__b5635201883513">Integer</strong>, <strong id="EN-US_TOPIC_0000002079176585__b186351518193517">Float</strong>, or <strong id="EN-US_TOPIC_0000002079176585__b9635191843517">Boolean</strong></p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row1939153220517"><td class="cellrowborder" valign="top" width="13.69%" headers="mcps1.3.8.3.2.2.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p9391143216511">Default</p>
</td>
<td class="cellrowborder" valign="top" width="86.31%" headers="mcps1.3.8.3.2.2.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p1239123218517">Set the default value of the hyperparameter. This value will be used for training jobs by default.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row639103819578"><td class="cellrowborder" valign="top" width="13.69%" headers="mcps1.3.8.3.2.2.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p184016381577">Restrain</p>
</td>
<td class="cellrowborder" valign="top" width="86.31%" headers="mcps1.3.8.3.2.2.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p114016382574">Click <strong id="EN-US_TOPIC_0000002079176585__b35182147361">Restrain</strong> and set the range of the default value or enumerated value in the dialog box displayed.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_row8391163215519"><td class="cellrowborder" valign="top" width="13.69%" headers="mcps1.3.8.3.2.2.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p23912326512">Required</p>
</td>
<td class="cellrowborder" valign="top" width="86.31%" headers="mcps1.3.8.3.2.2.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p1239114321513">Whether the hyperparameter is mandatory. The value can be <strong id="EN-US_TOPIC_0000002079176585__b893061918364">Yes</strong> or <strong id="EN-US_TOPIC_0000002079176585__b1393151918369">No</strong>. If you select <strong id="EN-US_TOPIC_0000002079176585__b2938113615363">No</strong>, you can delete the hyperparameter on the training job creation page when using this algorithm to create a training job. If you select <strong id="EN-US_TOPIC_0000002079176585__b5597164217368">Yes</strong>, the hyperparameter cannot be deleted.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_row149993117496"><td class="cellrowborder" valign="top" width="13.69%" headers="mcps1.3.8.3.2.2.2.3.1.1 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p13951123514499">Description</p>
</td>
<td class="cellrowborder" valign="top" width="86.31%" headers="mcps1.3.8.3.2.2.2.3.1.2 "><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p69515358497">Enter the description of the hyperparameter.</p>
<p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p4952535124915">Only letters, digits, spaces, hyphens (-), underscores (_), commas (,), and periods (.) are allowed.</p>
</td>
</tr>
</tbody>
</table>
</div>
</li></ol>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1959512404445"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1959512404445"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_section1959512404445"></a><h4 class="sectiontitle">Supported Policies</h4><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p990745441112">Only the pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64 and tensorflow_2.1.0-cuda_10.1-py_3.7-ubuntu_18.04-x86_64 images are available for auto search.</p>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section11326178153512"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section11326178153512"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_section11326178153512"></a><h4 class="sectiontitle">Adding Training Constraints</h4><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_p14432978710">You can add training constraints of the algorithm based on your needs.</p>
<ul id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_ul94331871575"><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li194331171776"><strong id="EN-US_TOPIC_0000002079176585__b1986133063716">Resource Type</strong>: The options are <strong id="EN-US_TOPIC_0000002079176585__b786123018372">CPU</strong>, <strong id="EN-US_TOPIC_0000002079176585__b7862173020370">Ascend</strong>, and <strong id="EN-US_TOPIC_0000002079176585__b386219301376">GPU</strong>. You can select multiple options.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li204331174712"><strong id="EN-US_TOPIC_0000002079176585__b768203763716">Multicard Training</strong>: Select <strong id="EN-US_TOPIC_0000002079176585__b9682937123719">supported</strong> or <strong id="EN-US_TOPIC_0000002079176585__b17683137123715">not supported</strong>.</li><li id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_li1543310718713"><strong id="EN-US_TOPIC_0000002079176585__b557717416376">Distributed Training</strong>: Select <strong id="EN-US_TOPIC_0000002079176585__b165771341163719">supported</strong> or <strong id="EN-US_TOPIC_0000002079176585__b1957714417373">not supported</strong>.<div class="fignone" id="EN-US_TOPIC_0000002079176585__fig84606236442"><span class="figcap"><b>Figure 3 </b>Training constraints</span><br><span><img id="EN-US_TOPIC_0000002079176585__image1467303717192" src="figure/en-us_image_0000002043019396.png" width="469.49" height="157.1528" title="Click to enlarge" class="imgResize"></span></div>
</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_section134853752010"><h4 class="sectiontitle">Previewing the Runtime Environment</h4><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p17869127112111">When creating an algorithm, click the arrow on <span><img id="EN-US_TOPIC_0000002079176585__image95351211144410" src="figure/en-us_image_0000002079098381.png"></span> in the lower right corner of the page to know the paths of the code directory, boot file, and input and output data in the training container.</p>
<div class="fignone" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_fig167431218533"><span class="figcap"><b>Figure 4 </b>Preview Runtime Environment</span><br><span><img id="EN-US_TOPIC_0000002079176585__image662454319448" src="figure/en-us_image_0000002079176961.png"></span></div>
</div>
<div class="section" id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section743865313613"><a name="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_en-us_topic_0000001071986951_section743865313613"></a><a name="en-us_topic_0000001133351332_en-us_topic_0000001071986951_section743865313613"></a><h4 class="sectiontitle">Follow-Up Operations</h4><p id="EN-US_TOPIC_0000002079176585__en-us_topic_0000001133351332_p58695913613">After an algorithm is created, use it to create a training job. For details, see <a href="develop-modelarts-0011.html">Creating a Training Job</a>.</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="develop-modelarts-0006.html">Using a Preset Image (Custom Script)</a></div>
</div>
</div>
<script language="JavaScript">
<!--
image_size('.imgResize');
var msg_imageMax = "view original image";
var msg_imageClose = "close";
//--></script>
View Git Blame Copy Permalink