Jiang, Beibei
781e07249c
Reviewed-by: gtema <artem.goncharov@gmail.com> Co-authored-by: Jiang, Beibei <beibei.jiang@t-systems.com> Co-committed-by: Jiang, Beibei <beibei.jiang@t-systems.com>
37 KiB
Viewing Service Details
After a model is deployed as a real-time service, you can access the service page to view its details.
- Log in to the ModelArts management console and choose Service Deployment > Real-Time Services.
- On the Real-Time Services page, click the name of the target service. The service details page is displayed.
You can view the service name and status. For details, see Table 1.
Table 1 Real-time service parameters Parameter
Description
Name
Name of the real-time service.
Status
Current status of the real-time service.
Source
Model source of the real-time service.
Failed Calls/Total Calls
Number of service calls, which is counted from the time when the service was created.
If the number of models is changed or a service is invoked when a model is not ready, the number of calls is not counted.
Network Configuration
Customized network configuration of the used dedicated resource pool.
Description
Service description, which can be edited after you click the edit button on the right side.
Custom Settings
Customized configurations based on real-time service versions. This allows version-based traffic distribution policies and configurations. Enable this option and click View Settings to customize the settings. For details, see Modifying Customized Settings.
Data Collection
Enable this option to store the data generated when the real-time service is invoked to a specified OBS path.
Filter
Enable this option so that the system automatically identifies hard examples in all sample data.
Synchronize Data
Synchronize the collected data to a dataset for centralized management and utilization.
Traffic Limit
Maximum number of times a service can be accessed within a second.
- You can switch between tabs on the details page of a real-time service to view more details. For details, see Table 2.
Table 2 Service details Parameter
Description
Usage Guides
Displays the API address, model information, input parameters, and output parameters. You can click
to copy the API address to call the service. Prediction
Performs a prediction test on the real-time service. For details, see Testing a Service.
Configuration Updates
Displays Existing Configuration and Historical Updates.
- Existing Configuration: includes the model name, version, status, traffic ratio, .
- Historical Updates: displays historical model information.
Monitoring
Displays Resource Usage and Model Calls.
- Resource Usage: includes the used and available CPU, memory, and GPU resources.
- Model Calls: indicates the number of model calls. The statistics collection starts after the model status changes to Ready.
Event
Displays key operations during service use, such as the service deployment progress, detailed causes of deployment exceptions, and time points when a service is started, stopped, or modified.
Logs
Displays the log information about each model in the service. You can view logs generated in the latest 5 minutes, latest 30 minutes, latest 1 hour, and user-defined time segment.
- You can select the start time and end time when defining the time segment.
Modifying Customized Settings
A customized configuration rule consists of the configuration condition (Setting), access version (Version), and customized running parameters (including Setting Name and Setting Value).
You can configure different settings with customized running parameters for different versions of a real-time service.
The priorities of customized configuration rules are in descending order. You can change the priorities by dragging the sequence of customized configuration rules.
After a rule is matched, the system will no longer match subsequent rules. A maximum of 10 configuration rules can be configured.
Parameter |
Mandatory |
Description |
|---|---|---|
Setting |
Yes |
Expression of the Spring Expression Language (SPEL) rule. Only the equal and matches expressions of the character type are supported. |
Version |
Yes |
Access version for a customized service configuration rule. When a rule is matched, the real-time service of the version is requested. |
Setting Name |
No |
Key of a customized running parameter, consisting of a maximum of 128 characters. Configure this parameter if the HTTP message header is used to carry customized running parameters to a real-time service. |
Setting Value |
No |
Value of a customized running parameter, consisting of a maximum of 256 characters. Configure this parameter if the HTTP message header is used to carry customized running parameters to a real-time service. |
Customized settings can be used in the following scenarios:
- If multiple versions of a real-time service are deployed for dark launch, customized settings can be used to distribute traffic by user.
Table 4 Built-in variables Built-in Variable
Description
DOMAIN_NAME
Account name that is used to invoke the inference request
DOMAIN_ID
Account ID that is used to invoke the inference request
PROJECT_NAME
Project name that is used to invoke the inference request
PROJECT_ID
Project ID that invokes the inference request
USER_NAME
Username that is used to invoke the inference request
USER_ID
User ID that is used to invoke the inference request
Pound key (#) indicates that a variable is referenced. The matched character string must be enclosed in single quotation marks.
#{Built-in variable} == 'Character string' #{Built-in variable} matches 'Regular expression'- Example 1:
If the account name for invoking the inference request is User A, the specified version is matched.
#DOMAIN_NAME == 'User A'
- Example 2:
If the account name in the inference request starts with op, the specified version is matched.
#DOMAIN_NAME matches 'op.*'
Table 5 Common regular expressions Character
Description
.
Match any single character except \n. To match any character including \n, use (.|\n).
*
Match the subexpression that it follows for zero or multiple times. For example, zo* can match z and zoo.
+
Match the subexpression that it follows for once or multiple times. For example, zo+ can match zo and zoo, but cannot match z.
?
Match the subexpression that it follows for zero or one time. For example, do(es)? can match does or do in does.
^
Match the start of the input string.
$
Match the end of the input string.
{n}
Match for the number specified by n, a non-negative integer. For example, o{2} cannot match o in Bob, but can match two os in food.
x|y
Match x or y. For example, z|food can match z or food, and (z|f)ood can match zood or food.
[xyz]
Match any single character contained in a character set. For example, [abc] can match a in plain.
Figure 1 Traffic distribution by user
- Example 1:
- If multiple versions of a real-time service are deployed for dark launch, customized settings can be used to access different versions through the header.Start with #HEADER_, indicating that the header is referenced as a condition.
#HEADER_{key} == '{value}' #HEADER_{key} matches '{value}'- Example 1:
If the header of an inference HTTP request contains a version and the value is 0.0.1, the condition is met. Otherwise, the condition is not met.
#HEADER_version == '0.0.1'
- Example 2:
If the header of an inference HTTP request contains testheader and the value starts with mock, the rule is matched.
#HEADER_testheader matches 'mock.*'
Figure 2 Using the header to access different versions
- Example 1:
- If a real-time service version supports different running configurations, you can use Setting Name and Setting Value to specify customized running parameters so that different users can use different running configurations.
Example:
When user A accesses the model, the user uses configuration A. When user B accesses the model, the user uses configuration B. When matching a running configuration, ModelArts adds a header to the request and also the customized running parameters specified by Setting Name and Setting Value.Figure 3 Customized running parameters added for a customized configuration rule