vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/modelarts/umn/modelarts_trouble_0015.html
Lai, Weijian 4e4b2d5f6d ModelArts UMN 23.3.0 Version. 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Lai, Weijian <laiweijian4@huawei.com>
Co-committed-by: Lai, Weijian <laiweijian4@huawei.com>
2024-06-26 07:03:02 +00:00

55 lines
11 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001909848532"></a><a name="EN-US_TOPIC_0000001909848532"></a>
<h1 class="topictitle1">Error Message "No module named .*" Displayed in Training Job Logs</h1>
<div id="body8662426"><p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_p9842103816374">Perform the following operations to locate the fault:</p>
<ol id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_ol7119240383"><li id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_li71195410389"><a href="#EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section9729952472">Checking Whether the Dependency Package Is Available</a></li><li id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_li1711919413388"><a href="#EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section11396125111268">Checking Whether the Dependency Package Path Can Be Detected</a></li><li id="EN-US_TOPIC_0000001909848532__li1215817411688"><a href="#EN-US_TOPIC_0000001909848532__section2182429896">Checking Whether the Selected Resource Flavor Is Correct</a></li><li id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_li512084123819"><a href="#EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section169401039184514">Summary and Suggestions</a></li></ol>
<div class="section" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section9729952472"><a name="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section9729952472"></a><a name="en-us_topic_0000001174943233_section9729952472"></a><h4 class="sectiontitle">Checking Whether the Dependency Package Is Available</h4><p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_p10626201214815">If the dependency package is unavailable, use either of the following methods to install it:</p>
<ul id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_ul7570618132210"><li id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_li597516374343">Method 1 (recommended): When you create an algorithm, place the required file or installation package in the code directory.<p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_p472912316566"><a name="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_li597516374343"></a><a name="en-us_topic_0000001174943233_li597516374343"></a>The required file varies depending on the dependency package type.</p>
<ul id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_ul1277854919572"><li id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_li1778114935710"><strong id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_b1384513542445">If the dependency package is an open-source installation package</strong><p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_p1051702281913">Create a file named <span class="filepath" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_filepath1551719224198"><b>pip-requirements.txt</b></span> in the code directory, and specify the dependency package name and version in the format of <em id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_i1253163144719">Package name</em><strong id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_b22534316477">==</strong><em id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_i1625320310470">Version</em> in the file.</p>
<p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_p14517922181914">For example, the OBS path specified by <span class="parmname" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_parmname1865511574525"><b>Code Directory</b></span> contains model files and the <span class="filepath" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_filepath18517192220193"><b>pip-requirements.txt</b></span> file. The code directory structure is as follows:</p>
<pre class="screen" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_screen951718224198">|---OBS path to the model boot file
|---model.py # Model boot file
|---pip-requirements.txt # Customized configuration file, which specifies the name and version of the dependency package</pre>
<p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_p3517132221911">The following shows the content of the <span class="filepath" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_filepath6517122151919"><b>pip-requirements.txt</b></span> file:</p>
<pre class="screen" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_screen7517112211917">alembic==0.8.6
bleach==1.4.3
click==6.6</pre>
</li><li id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_li106572405813"><strong id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_b36021643184612">If the dependency package is a WHL package</strong><p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_p72677538812">If the training backend does not support the download of open-source installation packages or the use of custom WHL packages, the system cannot automatically download and install the package. In this case, place the WHL package in the code directory, create a file named <span class="filepath" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_filepath82081126592"><b>pip-requirements.txt</b></span>, and specify the name of the WHL package in the file. The dependency package must be in WHL format.</p>
<p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_p8517195916108">For example, the OBS path specified by <span class="parmname" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_parmname82401330644"><b>Code Directory</b></span> contains model files, the WHL file, and the <span class="filepath" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_filepath05161459191012"><b>pip-requirements.txt</b></span> file. The code directory structure is as follows:</p>
<pre class="screen" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_screen0831354101214">|---OBS path to the model boot file
|---model.py # Model boot file
|---XXX.whl # Dependency package. If multiple dependencies are required, place all of them here.
|---pip-requirements.txt # Customized configuration file, which specifies the name of the dependency package</pre>
<p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_p921092518913">The following shows the content of the <span class="filepath" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_filepath86762393142"><b>pip-requirements.txt</b></span> file:</p>
<pre class="screen" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0274524271_screen1749471118109">numpy-1.15.4-cp36-cp36m-manylinux1_x86_64.whl
tensorflow-1.8.0-cp36-cp36m-manylinux1_x86_64.whl</pre>
</li></ul>
</li><li id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_li1939864112211">Method 2: Add the following code to the boot file to install the dependency package:<pre class="screen" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_screen717655282414">import os
os.system('pip install xxx')</pre>
</li></ul>
<p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_p128471013122517">In method 1, the dependency package can be downloaded and installed before the training job is started. In method 2, the dependency package is downloaded and installed during the running of the boot file.</p>
<p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_p17774702616"></p>
</div>
<div class="section" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section11396125111268"><a name="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section11396125111268"></a><a name="en-us_topic_0000001174943233_section11396125111268"></a><h4 class="sectiontitle">Checking Whether the Dependency Package Path Can Be Detected</h4><p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_p1540725112914">Before executing code locally, add <span class="parmname" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_parmname188986378322"><b>project_dir</b></span> to <strong id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_b193041125143820">PYTHONPATH</strong> or install <span class="parmname" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_parmname13862133515108"><b>project_dir</b></span> in <span class="parmname" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_parmname2123201193319"><b>site-package</b></span>. ModelArts enables you to add <span class="parmname" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_parmname19513159151519"><b>project_dir</b></span> to <span class="filepath" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_filepath8812145819332"><b>sys.path</b></span> to resolve this issue.</p>
<p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_p157956274309">Run <b><span class="cmdname" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_cmdname1084871919312">from module_dir import module_file</span></b> to import a package. The code structure is as follows:</p>
<pre class="screen" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_screen236214483317">project_dir
|- main.py
|- module_dir
| |- __init__.py
| |- module_file.py</pre>
</div>
<div class="section" id="EN-US_TOPIC_0000001909848532__section2182429896"><a name="EN-US_TOPIC_0000001909848532__section2182429896"></a><a name="section2182429896"></a><h4 class="sectiontitle">Checking Whether the Selected Resource Flavor Is Correct</h4><p id="EN-US_TOPIC_0000001909848532__p16410391199">Error message "No module named npu_bridge.npu_init" is displayed for a training job.</p>
<pre class="screen" id="EN-US_TOPIC_0000001909848532__screen62491543191112">from npu_bridge.npu_init import *
ImportError: No module named npu_bridge.npu_init</pre>
<p id="EN-US_TOPIC_0000001909848532__p3709018101111">Check whether the flavor used by the training job supports NPUs. The possible cause is that the job selected a non-NPU flavor, for example, a GPU flavor. As a result, an error occurs when NPUs are used.</p>
</div>
<div class="section" id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section169401039184514"><a name="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_section169401039184514"></a><a name="en-us_topic_0000001174943233_section169401039184514"></a><h4 class="sectiontitle">Summary and Suggestions</h4><p id="EN-US_TOPIC_0000001909848532__en-us_topic_0000001174943233_en-us_topic_0000001182115463_en-us_topic_0000001143177048_p1141444753416">Before creating a training job, use the ModelArts development environment to debug the training code to maximally eliminate errors in code migration.</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="modelarts_13_0071.html">In-Cloud Migration Adaptation Issues</a></div>
</div>
</div>
View Git Blame Copy Permalink