yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
fix_python
doc-exports/docs/modelarts/umn/modelarts_trouble_0054.html
Lai, Weijian 6aa966a79a ModelArts UMN 24.3.0 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lai, Weijian <laiweijian4@huawei.com>
Co-committed-by: Lai, Weijian <laiweijian4@huawei.com>
2024-11-02 09:04:52 +00:00

2.6 KiB
Raw Permalink Blame History

Error Message "retCode=0x91, [the model stream execute failed]" Displayed in MindSpore Logs

Symptom

When MindSpore is used for training, the following error message is displayed:
[ERROR] RUNTIME(3002)model execute error, retCode=0x91, [the model stream execute failed]

Possible Causes

The speed of reading data cannot keep up with the model iteration speed.

Solution

  1. Reduce shuffle operations during preprocessing.
    dataset = dataset.shuffle(buffer_size=x)
  2. Disable data preprocessing, which may affect system performance.
    NPURunConfig(enable_data_pre_proc=Flase)

Summary and Suggestions

Before creating a training job, use the ModelArts development environment to debug the training code to maximally eliminate errors in code migration.