Switching the Hive Execution Engine to Tez

Scenario

Hive can use the Tez engine to process data computing tasks. Before executing a task, you can manually switch the execution engine to Tez.

Prerequisites

The TimelineServer role of the Yarn service has been installed in the cluster and is running properly.

Switching the Execution Engine on the Client to Tez

  1. Install and log in to the Hive client. For details, see Using a Hive Client.
  2. Run the following commands for MRS 3.1.2 to switch the engine and enable the yarn.timeline-service.enabled parameter:

    set hive.execution.engine=tez;

    set yarn.timeline-service.enabled=true;
    • After yarn.timeline-service.enabled is enabled, you can view the details about the tasks executed by the Tez engine on TezUI. After this function is enabled, task information will be reported to TimelineServer. If the TimelineServer instance is faulty, the task will fail.
    • Tez uses the ApplicationMaster buffer pool. Therefore, yarn.timeline-service.enabled must be enabled before Tez tasks are submitted. Otherwise, this parameter cannot take effect and you need to log in to the client again to configure it.
    • When the execution engine needs to be switched to another engine, you need to run the set yarn.timeline-service.enabled=false command on the client to disable the yarn.timeline-service.enabled parameter.
    • To specify a Yarn running queue, run the set tez.queue.name=default command on the client.

  3. For MRS 3.2.0 and later versions, run the following command to switch the engine:

    set hive.execution.engine=tez;

    To specify a running Yarn queue, run the set tez.queue.name=default command on the client.

  4. Submit and execute the Tez tasks.
  5. Log in to FusionInsight Manager. For details, see Accessing FusionInsight Manager. Choose Cluster > Name of the desired cluster > Services > Tez > TezUI(host name) to view the task execution status on the TezUI page.

Switching the Default Execution Engine of Hive to Tez

  1. Log in to FusionInsight Manager. For details, see Accessing FusionInsight Manager. Choose Cluster > Name of the desired cluster > Services > Hive > Configurations > All Configurations > HiveServer(Role), and search for hive.execution.engine.
  2. Set hive.execution.engine to tez.
  3. Choose Hive(Service) > Customization and search for yarn.site.customized.configs for MRS 3.1.2.
  4. Add a customized parameter yarn.timeline-service.enabled next to yarn.site.customized.configs and set its value to true for MRS 3.1.2.

    • After yarn.timeline-service.enabled is enabled, you can view the details about the tasks executed by the Tez engine on TezUI. After this function is enabled, task information will be reported to TimelineServer. If the TimelineServer instance is faulty, the task will fail.
    • Tez uses the ApplicationMaster buffer pool. Therefore, yarn.timeline-service.enabled must be enabled before Tez tasks are submitted. Otherwise, this parameter cannot take effect and you need to log in to the client again to configure it.
    • When the execution engine needs to be switched to another one, you need to set the value of parameter yarn.timeline-service.enabled to false.

  5. Click Save. In the displayed confirmation dialog box, click OK.
  6. Choose Dashboard > More > Restart Service to restart the Hive service. Enter the password to restart the service.
  7. Install and log in to the Hive client. For details, see Using a Hive Client.
  8. Submit and execute the Tez tasks.
  9. Log in to FusionInsight Manager and choose Cluster > Name of the desired cluster > Services > Tez > TezUI(host name). On the displayed TezUI page, view the task execution status.
Parent topic: Using Hive