:original_name: mrs_01_1018.html

.. _mrs_01_1018:

Improving Real-time Data Read Performance
=========================================

Scenario
--------

HBase data needs to be read.

Prerequisites
-------------

The get or scan interface of HBase has been invoked and data is read in real time from HBase.

Procedure
---------

-  **Data reading server tuning**

   Parameter portal:

   Go to the **All Configurations** page of the HBase service. For details, see :ref:`Modifying Cluster Service Configuration Parameters <mrs_01_2125>`.

   .. table:: **Table 1** Configuration items that affect real-time data reading

       
      | Parameter                        | Description                                                                                                                                                                                                                                                                                                                                                     | Default Value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
       
      | GC_OPTS                          | You can increase HBase memory to improve HBase performance because read and write operations are performed in HBase memory.                                                                                                                                                                                                                                     | For versions earlier than MRS 3.x:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  | **HeapSize** and **NewSize** need to be adjusted. When you adjust **HeapSize**, set **Xms** and **Xmx** to the same value to avoid performance problems when JVM dynamically adjusts **HeapSize**. Set **NewSize** to 1/8 of **HeapSize**.                                                                                                                      | -  HMaster:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  | -  **HMaster**: If HBase clusters enlarge and the number of Regions grows, properly increase the **GC_OPTS** parameter value of the HMaster.                                                                                                                                                                                                                    |    -server -Xms2G -Xmx2G -XX:NewSize=256M -XX:MaxNewSize=256M -XX:MetaspaceSize=128M -XX:MaxMetaspaceSize=512M -XX:MaxDirectMemorySize=512M -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=65 -XX:+PrintGCDetails -Dsun.rmi.dgc.client.gcInterval=0x7FFFFFFFFFFFFFE -Dsun.rmi.dgc.server.gcInterval=0x7FFFFFFFFFFFFFE -XX:-OmitStackTraceInFastThrow -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M |
      |                                  | -  **RegionServer**: A RegionServer needs more memory than an HMaster. If sufficient memory is available, increase the **HeapSize** value.                                                                                                                                                                                                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 | -  RegionServer:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
      |                                  | .. note::                                                                                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |    -server -Xms4G -Xmx4G -XX:NewSize=512M -XX:MaxNewSize=512M -XX:MetaspaceSize=128M -XX:MaxMetaspaceSize=512M -XX:MaxDirectMemorySize=512M -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=65 -XX:+PrintGCDetails -Dsun.rmi.dgc.client.gcInterval=0x7FFFFFFFFFFFFFE -Dsun.rmi.dgc.server.gcInterval=0x7FFFFFFFFFFFFFE -XX:-OmitStackTraceInFastThrow -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M |
      |                                  |    When the value of **HeapSize** for the active HMaster is 4 GB, the HBase cluster can support 100,000 regions. Empirically, each time 35,000 regions are added to the cluster, the value of **HeapSize** must be increased by 2 GB. It is recommended that the value of **HeapSize** for the active HMaster not exceed 32 GB.                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 | For MRS 3.\ *x* or later:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 | -  HMaster                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |    -server -Xms4G -Xmx4G -XX:NewSize=512M -XX:MaxNewSize=512M -XX:MetaspaceSize=128M -XX:MaxMetaspaceSize=512M -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=65 -XX:+PrintGCDetails -Dsun.rmi.dgc.client.gcInterval=0x7FFFFFFFFFFFFFE -Dsun.rmi.dgc.server.gcInterval=0x7FFFFFFFFFFFFFE -XX:-OmitStackTraceInFastThrow -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M                              |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 | -  Region Server                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |    -server -Xms6G -Xmx6G -XX:NewSize=1024M -XX:MaxNewSize=1024M -XX:MetaspaceSize=128M -XX:MaxMetaspaceSize=512M -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=65 -XX:+PrintGCDetails -Dsun.rmi.dgc.client.gcInterval=0x7FFFFFFFFFFFFFE -Dsun.rmi.dgc.server.gcInterval=0x7FFFFFFFFFFFFFE -XX:-OmitStackTraceInFastThrow -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M                            |
       
      | hbase.regionserver.handler.count | Indicates the number of requests that RegionServer can process concurrently. If the parameter is set to an excessively large value, threads will compete fiercely. If the parameter is set to an excessively small value, requests will be waiting for a long time in RegionServer, reducing the processing capability. You can add threads based on resources. | 200                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
      |                                  |                                                                                                                                                                                                                                                                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
      |                                  | It is recommended that the value be set to 100 to 300 based on the CPU usage.                                                                                                                                                                                                                                                                                   |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
       
      | hfile.block.cache.size           | HBase cache sizes affect query efficiency. Set cache sizes based on query modes and query record distribution. If random query is used to reduce the hit ratio of the buffer, you can reduce the buffer size.                                                                                                                                                   | When **offheap** is disabled, the default value is **0.25**. When **offheap** is enabled, the default value is **0.1**.                                                                                                                                                                                                                                                                                                                                                                                               |
       

   .. note::

      If read and write operations are performed at the same time, the performance of the two operations affects each other. If flush and compaction operations are frequently performed due to data writes, a large number of disk I/O operations are occupied, affecting read performance. If a large number of compaction operations are blocked due to write operations, multiple HFiles exist in the region, affecting read performance. Therefore, if the read performance is unsatisfactory, you need to check whether the write configurations are proper.

-  **Data reading client tuning**

   When scanning data, you need to set **caching** (the number of records read from the server at a time. The default value is **1**.). If the default value is used, the read performance will be extremely low.

   If you do not need to read all columns of a piece of data, specify the columns to be read to reduce network I/O.

   If you only need to read the row key, add a filter (FirstKeyOnlyFilter or KeyOnlyFilter) that only reads the row key.

-  **Data table reading design optimization**

   .. table:: **Table 2** Parameters affecting real-time data reading

      +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
      | Parameter             | Description                                                                                                                                                                                                                                                              | Default Value         |
      +=======================+==========================================================================================================================================================================================================================================================================+=======================+
      | COMPRESSION           | The compression algorithm compresses blocks in HFiles. For compressible data, configure the compression algorithm to efficiently reduce disk I/Os and improve performance.                                                                                               | NONE                  |
      |                       |                                                                                                                                                                                                                                                                          |                       |
      |                       | .. note::                                                                                                                                                                                                                                                                |                       |
      |                       |                                                                                                                                                                                                                                                                          |                       |
      |                       |    Some data cannot be efficiently compressed. For example, a compressed figure can hardly be compressed again. The common compression algorithm is SNAPPY, because it has a high encoding/decoding speed and acceptable compression rate.                               |                       |
      +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
      | BLOCKSIZE             | Different block sizes affect HBase data read and write performance. You can configure sizes for blocks in an HFile. Larger blocks have a higher compression rate. However, they have poor performance in random data read, because HBase reads data in a unit of blocks. | 65536                 |
      |                       |                                                                                                                                                                                                                                                                          |                       |
      |                       | Set the parameter to 128 KB or 256 KB to improve data write efficiency without greatly affecting random read performance. The unit is byte.                                                                                                                              |                       |
      +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
      | DATA_BLOCK_ENCODING   | Encoding method of the block in an HFile. If a row contains multiple columns, set **FAST_DIFF** to save data storage space and improve performance.                                                                                                                      | NONE                  |
      +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
