# Deploying a Model as a Real-Time Service

After a model is prepared, you can deploy the model as a real-time service and predict and call the service.

A maximum of one real-time service can be deployed.

#### Prerequisites

- Data has been prepared. Specifically, you have created a model in the Normal state in ModelArts.

#### Procedure

1. Log in to the ModelArts management console. In the left navigation pane, choose Service Deployment > Real-Time Services. By default, the system switches to the Real-Time Services page.
2. In the real-time service list, click Deploy in the upper left corner. The Deploy page is displayed.
3. Set parameters for a real-time service.
   1. Set basic information about model deployment. For details about the parameters, see Table 1.

      Table 1 Basic parameters of model deployment
      Parameter | Description
      --- | ---
      Name | Name of the real-time service. Set this parameter as prompted.
      Auto Stop | After this parameter is enabled and the auto stop time is set, a service automatically stops at the specified time. The auto stop function is enabled by default, and the default value is 1 hour later. The options are 1 hour later, 2 hours later, 4 hours later, 6 hours later, and Custom. If you select Custom, you can enter any integer from 1 to 24 hours in the textbox on the right.
      Description | Brief description of the real-time service.

      Figure 1 Basic information about deploying a model as a real-time service
   2. Enter key information including the resource pool and model configurations. For details, see Table 2.

      Table 2 Parameters
      Parameter | Sub-Parameter | Description
      --- | --- | ---
      Resource Pool | Public resource pools | Instances in the public resource pool can be of the CPU or GPU type.
      Resource Pool | Dedicated resource pools | For details about how to create a dedicated resource pool, see Creating a Dedicated Resource Pool. You can select a specification from the resource pool specifications.
      Model and Configuration | Model Source | You can select My Models or My Subscriptions based on site requirements. The models that match the model sources are displayed.
      Model | The system automatically associates with the list of available models. Select a model in the Normal status and its version.
      Traffic Ratio (%) | Set the traffic proportion of the current instance node. Service calling requests are allocated to the current version based on this proportion. If you deploy only one version of a model, set this parameter to 100%. If you select multiple versions for gated launch, ensure that the sum of the traffic ratios of multiple versions is 100%.
      Specifications | If you select Public resource pools, you can select the CPU or GPU resources based on site requirements. For details, see Table 3.
      Compute Nodes | Set the number of instances for the current model version. If you set Instances to 1, the standalone computing mode is used. If you set Instances to a value greater than 1, the distributed computing mode is used. Select a computing mode based on the actual requirements.
      Environment Variable | Set environment variables and inject them to the container instance. To ensure data security, do not enter sensitive information, such as plaintext passwords, in environment variables.
      Add Model and Configuration | ModelArts supports multiple model versions and flexible traffic policies. You can use gated launch to smoothly upgrade the model version. NOTE: If the selected model has only one version, the system does not display Add Model Version and Configuration.
      Traffic Limit | N/A | Maximum number of times a service can be accessed within a second. You can set this parameter as needed.

      Table 3 Supported specifications
      Specifications | Description
      --- | ---
      CPU: 8 vCPUs | 64 GiB GPU: 1 x V100 | Suitable for running GPU models.

      Figure 2 Setting model information
4. After confirming the entered information, complete service deployment as prompted. Generally, service deployment jobs run for a period of time, which may be several minutes or tens of minutes depending on the amount of your selected data and resources.

   After a real-time service is deployed, it is started immediately.

   You can go to the real-time service list to view the basic information about the real-time service. In the real-time service list, after the status of the newly deployed service changes from Deploying to Running, the service is deployed successfully.

Parent topic: Real-Time Services