DMS UMN 20250314 version
Reviewed-by: Liudmila Denisova <ldenisov@noreply.gitea.eco.tsi-dev.otc-service.com> Co-authored-by: chenjunjie <chenjunjie@huawei.com> Co-committed-by: chenjunjie <chenjunjie@huawei.com>
@ -1,12 +1,14 @@
|
|||||||
<a name="Kafka-client-best-practice"></a><a name="Kafka-client-best-practice"></a>
|
<a name="Kafka-client-best-practice"></a><a name="Kafka-client-best-practice"></a>
|
||||||
|
|
||||||
<h1 class="topictitle1">Suggestions on Using the Kafka Client</h1>
|
<h1 class="topictitle1">Suggestions on Using the Kafka Client</h1>
|
||||||
<div id="body0000001926355989"><div class="section" id="Kafka-client-best-practice__section85502807"><h4 class="sectiontitle">Consumers</h4><ol id="Kafka-client-best-practice__ol1577776118"><li id="Kafka-client-best-practice__li127707717">Ensure that the owner thread does not exit abnormally. Otherwise, the client may fail to initiate consumption requests and the consumption will be blocked.</li><li id="Kafka-client-best-practice__li13748161123917">Commit messages only after they have been processed. Otherwise, the messages may fail to be processed and cannot be polled again.</li><li id="Kafka-client-best-practice__li10779714118">Generally, do not commit every message. Otherwise, there will be many <strong id="Kafka-client-best-practice__b151141522115515">OFFSET_COMMIT</strong> requests, causing high CPU usage. For example, if a consumption request pulls 1000 messages and commits every one of them, TPS of the commit requests is 1000 times that of consumption. The smaller the message size, the larger the ratio. You can commit a specific number of messages in batches or enable <strong id="Kafka-client-best-practice__b1795161612581">enable.auto.commit</strong>. However, if the client is faulty, some cached consumption offset may be lost, resulting in repeated consumption. Therefore, you are advised to commit messages in batches based on service requirements.</li><li id="Kafka-client-best-practice__li57712713115">A consumer cannot frequently join or leave a group. Otherwise, the consumer will frequently perform rebalancing, which blocks consumption.</li><li id="Kafka-client-best-practice__li3771677119">The number of consumers cannot be greater than the number of partitions in the topic. Otherwise, some consumers may fail to poll for messages.</li><li id="Kafka-client-best-practice__li0771471212">Ensure that the consumer polls at regular intervals to keep sending heartbeats to the server. If the consumer stops sending heartbeats for long enough, the consumer session will time out and the consumer will be considered to have stopped. This will also block consumption.</li><li id="Kafka-client-best-practice__li1777171315">Ensure that there is a limitation on the size of messages buffered locally to avoid an out-of-memory (OOM) situation.</li><li id="Kafka-client-best-practice__li277677113">Set the timeout for the consumer session to 30 seconds: session.timeout.ms=30000.</li><li id="Kafka-client-best-practice__li477671112">Kafka supports exactly-once delivery. Therefore, ensure the idempotency of processing messages for services.</li><li id="Kafka-client-best-practice__li14781778111">Always close the consumer before exiting. Otherwise, consumers in the same group may be blocked within the timeout set by <strong id="Kafka-client-best-practice__b205502155918">session.timeout.ms</strong>.</li><li id="Kafka-client-best-practice__li689610431457">Do not start a consumer group name with a special character, such as a number sign (#). Otherwise, monitoring data of the consumer group cannot be displayed.</li></ol>
|
<div id="body0000001926355989"><div class="section" id="Kafka-client-best-practice__section85502807"><h4 class="sectiontitle">Consumers</h4><ol id="Kafka-client-best-practice__ol1577776118"><li id="Kafka-client-best-practice__li127707717">Ensure that the owner thread does not exit abnormally. Otherwise, the client may fail to initiate consumption requests and the consumption will be blocked.</li><li id="Kafka-client-best-practice__li13748161123917">Commit messages only after they have been processed. Otherwise, the messages may fail to be processed and cannot be polled again.</li><li id="Kafka-client-best-practice__li10779714118">Generally, do not commit every message. Otherwise, there will be many <strong id="Kafka-client-best-practice__b151141522115515">OFFSET_COMMIT</strong> requests, causing high CPU usage. For example, if a consumption request pulls 1000 messages and commits every one of them, TPS of the commit requests is 1000 times that of consumption. The smaller the message size, the larger the ratio. You can commit a specific number of messages in batches or enable <strong id="Kafka-client-best-practice__b1795161612581">enable.auto.commit</strong>. However, if the client is faulty, some cached consumption offset may be lost, resulting in repeated consumption. Therefore, you are advised to commit messages in batches based on service requirements.</li><li id="Kafka-client-best-practice__li57712713115">A consumer cannot frequently join or leave a group. Otherwise, the consumer will frequently perform rebalancing, which blocks consumption.</li><li id="Kafka-client-best-practice__li3771677119">The number of consumers in a consumer group must be within the total partitions subscribed by the consumer group. Otherwise, some consumers cannot pull messages.</li><li id="Kafka-client-best-practice__li0771471212">Ensure that the consumer polls at regular intervals to keep sending heartbeats to the server. If the consumer stops sending heartbeats for long enough, the consumer session will time out and the consumer will be considered to have stopped. This will also block consumption.</li><li id="Kafka-client-best-practice__li1777171315">Ensure that there is a limitation on the size of messages buffered locally to avoid an out-of-memory (OOM) situation.</li><li id="Kafka-client-best-practice__li277677113">Set the timeout for the consumer session to 30 seconds: session.timeout.ms=30000.</li><li id="Kafka-client-best-practice__li477671112">Kafka supports exactly-once delivery. Therefore, ensure the idempotency of processing messages for services.</li><li id="Kafka-client-best-practice__li14781778111">Always close the consumer before exiting. Otherwise, consumers in the same group may be blocked within the timeout set by <strong id="Kafka-client-best-practice__b205502155918">session.timeout.ms</strong>.</li><li id="Kafka-client-best-practice__li689610431457">Do not start a consumer group name with a special character, such as a number sign (#). Otherwise, monitoring data of the consumer group cannot be displayed.</li><li id="Kafka-client-best-practice__li1641816341398">Handle AuthorizationException in the consumption logic and set consumption retry. For example, in SpringKafka, add the following configuration.<pre class="screen" id="Kafka-client-best-practice__screen312620361113"># Set the authorization exception retry interval to 10 seconds.
|
||||||
|
spring.kafka.listener.authorizationExceptionRetryInterval=10000</pre>
|
||||||
|
</li></ol>
|
||||||
</div>
|
</div>
|
||||||
<div class="section" id="Kafka-client-best-practice__section65310184020"><h4 class="sectiontitle">Producers</h4><ol id="Kafka-client-best-practice__ol53887211062"><li id="Kafka-client-best-practice__li9388102113614">Synchronous replication: Set <strong id="Kafka-client-best-practice__b105761128192612">acks</strong> to <strong id="Kafka-client-best-practice__b1576122812612">all</strong>.</li><li id="Kafka-client-best-practice__li43881121866">Retry message sending: Set <strong id="Kafka-client-best-practice__b84721347102619">retries</strong> to <strong id="Kafka-client-best-practice__b24731747142611">3</strong>.</li><li id="Kafka-client-best-practice__li33888211962">Optimize message sending: For latency-sensitive messages, set <strong id="Kafka-client-best-practice__b1575215511269">linger.ms</strong> to <strong id="Kafka-client-best-practice__b175225117261">0</strong>. For latency-insensitive messages, set <strong id="Kafka-client-best-practice__b11162568262">linger.ms</strong> to a value ranging from <strong id="Kafka-client-best-practice__b1716125642616">100</strong> to <strong id="Kafka-client-best-practice__b16171156112618">1000</strong>.</li><li id="Kafka-client-best-practice__li133881421268">Ensure that the producer has sufficient JVM memory to avoid blockages.</li><li id="Kafka-client-best-practice__li18921123181613">Set the timestamp to the local time. Messages will fail to age if the timestamp is a future time.</li></ol>
|
<div class="section" id="Kafka-client-best-practice__section65310184020"><h4 class="sectiontitle">Producers</h4><ol id="Kafka-client-best-practice__ol53887211062"><li id="Kafka-client-best-practice__li9388102113614">Synchronous replication: Set <strong id="Kafka-client-best-practice__b105761128192612">acks</strong> to <strong id="Kafka-client-best-practice__b1576122812612">all</strong>.</li><li id="Kafka-client-best-practice__li43881121866">Retry message sending: Set <strong id="Kafka-client-best-practice__b84721347102619">retries</strong> to <strong id="Kafka-client-best-practice__b24731747142611">3</strong>.</li><li id="Kafka-client-best-practice__li33888211962">Optimize message sending: For latency-sensitive messages, set <strong id="Kafka-client-best-practice__b1575215511269">linger.ms</strong> to <strong id="Kafka-client-best-practice__b175225117261">0</strong>. For latency-insensitive messages, set <strong id="Kafka-client-best-practice__b11162568262">linger.ms</strong> to a value ranging from <strong id="Kafka-client-best-practice__b1716125642616">100</strong> to <strong id="Kafka-client-best-practice__b16171156112618">1000</strong>.</li><li id="Kafka-client-best-practice__li133881421268">Ensure that the producer has sufficient JVM memory to avoid blockages.</li><li id="Kafka-client-best-practice__li18921123181613">Set the timestamp to the local time. Messages will fail to age if the timestamp is a future time.</li><li id="Kafka-client-best-practice__li9875124612531">Try reusing producers. Do not create producers frequently. When idempotence is enabled (default for producer clients 3.0 and later), producing messages creates producer state objects on the server. Frequent creation results in too many objects to be reclaimed in time, causing server memory surges and performance deterioration. Set <strong id="Kafka-client-best-practice__b49441515102616">enable.idempotence</strong> to <strong id="Kafka-client-best-practice__b11880161822616">false</strong> if the idempotence is not required.</li><li id="Kafka-client-best-practice__li134383521680">Catch exception <strong id="Kafka-client-best-practice__b11011295299">AuthorizationException</strong>. You are advised to retry message production on the service side. Self-healing can be implemented through limited retry and backoff policies.</li></ol>
|
||||||
</div>
|
</div>
|
||||||
<div class="section" id="Kafka-client-best-practice__section785814360014"><h4 class="sectiontitle">Topics</h4><p id="Kafka-client-best-practice__p178951621165816">Recommended topic configurations: Use 3 replicas, enable synchronous replication, and set the minimum number of in-sync replicas to 2. The number of in-sync replicas cannot be the same as the number of replicas of the topic. Otherwise, if one replica is unavailable, messages cannot be produced.</p>
|
<div class="section" id="Kafka-client-best-practice__section785814360014"><h4 class="sectiontitle">Topics</h4><p id="Kafka-client-best-practice__p178951621165816">Recommended topic configurations: Use 3 replicas, enable synchronous replication, and set the minimum number of in-sync replicas to 2. The number of in-sync replicas cannot be the same as the number of replicas of the topic. Otherwise, if one replica is unavailable, messages cannot be produced.</p>
|
||||||
<p id="Kafka-client-best-practice__p07361810728">You can enable or disable automatic topic creation. If automatic topic creation is enabled, the system automatically creates a topic when a message is produced in or consumed from a topic that does not exist. This topic has the following default settings: 3 partitions, 3 replicas, aging time 72 hours, synchronous replication and flushing disabled, <strong id="Kafka-client-best-practice__b167644278297">CreateTime</strong> message timestamp, and maximum 10,485,760 bytes message size.</p>
|
<p id="Kafka-client-best-practice__p07361810728">You can enable or disable automatic topic creation. Enabling this function automatically creates a topic when a message is produced in or consumed from a topic that does not exist.</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="section" id="Kafka-client-best-practice__section194821855405"><h4 class="sectiontitle">Others</h4><p id="Kafka-client-best-practice__p13895172119589">Maximum number of connections: 3000</p>
|
<div class="section" id="Kafka-client-best-practice__section194821855405"><h4 class="sectiontitle">Others</h4><p id="Kafka-client-best-practice__p13895172119589">Maximum number of connections: 3000</p>
|
||||||
<p id="Kafka-client-best-practice__p9895521195815">Maximum size of a message: 10 MB</p>
|
<p id="Kafka-client-best-practice__p9895521195815">Maximum size of a message: 10 MB</p>
|
||||||
|
|||||||
@ -1,6 +1,6 @@
|
|||||||
<a name="ProductDescPrivilegeManagement"></a><a name="ProductDescPrivilegeManagement"></a>
|
<a name="ProductDescPrivilegeManagement"></a><a name="ProductDescPrivilegeManagement"></a>
|
||||||
|
|
||||||
<h1 class="topictitle1">Permission</h1>
|
<h1 class="topictitle1">Permissions</h1>
|
||||||
<div id="body8662426"><p id="ProductDescPrivilegeManagement__en-us_topic_0170871404_p7808122016303">If you need to grant your enterprise personnel permission to access your DMS resources, use Identity and Access Management (IAM). IAM provides identity authentication, fine-grained permissions management, and access control. IAM helps you secure access to your cloud resources.</p>
|
<div id="body8662426"><p id="ProductDescPrivilegeManagement__en-us_topic_0170871404_p7808122016303">If you need to grant your enterprise personnel permission to access your DMS resources, use Identity and Access Management (IAM). IAM provides identity authentication, fine-grained permissions management, and access control. IAM helps you secure access to your cloud resources.</p>
|
||||||
<p id="ProductDescPrivilegeManagement__en-us_topic_0170871404_p57386149361">You can create IAM users for your employees, and assign permissions to these users on a principle of least privilege (PoLP) basis to control their access to specific resource types. For example, you can create IAM users for software developers and assign specific permissions to allow them to use Kafka instance resources but prevent them from being able to delete resources or perform any high-risk operations.</p>
|
<p id="ProductDescPrivilegeManagement__en-us_topic_0170871404_p57386149361">You can create IAM users for your employees, and assign permissions to these users on a principle of least privilege (PoLP) basis to control their access to specific resource types. For example, you can create IAM users for software developers and assign specific permissions to allow them to use Kafka instance resources but prevent them from being able to delete resources or perform any high-risk operations.</p>
|
||||||
<p id="ProductDescPrivilegeManagement__en-us_topic_0170871404_p10590121414461">If your account does not require individual IAM users for permissions management, skip this section.</p>
|
<p id="ProductDescPrivilegeManagement__en-us_topic_0170871404_p10590121414461">If your account does not require individual IAM users for permissions management, skip this section.</p>
|
||||||
|
|||||||
@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
<h1 class="topictitle1">Configuring Kafka Instance Tags</h1>
|
<h1 class="topictitle1">Configuring Kafka Instance Tags</h1>
|
||||||
<div id="body1577263534920"><p id="TagManagement__p16892119416">Tags facilitate Kafka instance identification and management.</p>
|
<div id="body1577263534920"><p id="TagManagement__p16892119416">Tags facilitate Kafka instance identification and management.</p>
|
||||||
<p id="TagManagement__p164691516615">You can add tags to a Kafka instance when creating the instance or add tags on the <strong id="TagManagement__b68294410312">Tags</strong> tab page of the created instance. Up to 20 tags can be added to an instance. Tags can be deleted.</p>
|
<p id="TagManagement__p164691516615">You can add tags to a Kafka instance when creating the instance or add tags on the <strong id="TagManagement__b68294410312">Tags</strong> page of the created instance. Tags can be deleted.</p>
|
||||||
<p id="TagManagement__p8060118">A tag consists of a tag key and a tag value. <a href="#TagManagement__table193611920984">Table 1</a> lists the tag key and value requirements.</p>
|
<p id="TagManagement__p8060118">A tag consists of a tag key and a tag value. <a href="#TagManagement__table193611920984">Table 1</a> lists the tag key and value requirements.</p>
|
||||||
|
|
||||||
<div class="tablenoborder"><a name="TagManagement__table193611920984"></a><a name="table193611920984"></a><table cellpadding="4" cellspacing="0" summary="" id="TagManagement__table193611920984" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Tag key and value requirements</caption><thead align="left"><tr id="TagManagement__row143632201486"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.2.3.1.1"><p id="TagManagement__p536311201182">Parameter</p>
|
<div class="tablenoborder"><a name="TagManagement__table193611920984"></a><a name="table193611920984"></a><table cellpadding="4" cellspacing="0" summary="" id="TagManagement__table193611920984" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Tag key and value requirements</caption><thead align="left"><tr id="TagManagement__row143632201486"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.2.3.1.1"><p id="TagManagement__p536311201182">Parameter</p>
|
||||||
@ -13,22 +13,26 @@
|
|||||||
</thead>
|
</thead>
|
||||||
<tbody><tr id="TagManagement__row43636201815"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.2.3.1.1 "><p id="TagManagement__p336313201814">Tag key</p>
|
<tbody><tr id="TagManagement__row43636201815"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.2.3.1.1 "><p id="TagManagement__p336313201814">Tag key</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.2.3.1.2 "><ul id="TagManagement__ul133310231694"><li id="TagManagement__li833315236915">Cannot be left blank.</li><li id="TagManagement__li1733312319916">Must be unique for the same instance.</li><li id="TagManagement__li071752814194">Can contain 1 to 128 characters.</li><li id="TagManagement__li187665401268">Can contain letters, digits, spaces, and special characters _ . : = + - @</li><li id="TagManagement__li1933311235916">Cannot start or end with a space.</li><li id="TagManagement__li10155182510220">Cannot start with <strong id="TagManagement__b20870191531115">_sys_</strong>.</li></ul>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.2.3.1.2 "><ul id="TagManagement__ul133310231694"><li id="TagManagement__li833315236915">Cannot be left blank.</li><li id="TagManagement__li1733312319916">Must be unique for the same instance.</li><li id="TagManagement__li071752814194">Can contain 1 to 128 characters.</li><li id="TagManagement__li187665401268">Can contain letters, digits, spaces, and special characters _.:=+-@</li><li id="TagManagement__li1933311235916">Cannot start or end with a space.</li><li id="TagManagement__li10155182510220">Cannot start with <strong id="TagManagement__b20870191531115">_sys_</strong>.</li></ul>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="TagManagement__row193632201781"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.2.3.1.1 "><p id="TagManagement__p736315201282">Tag value</p>
|
<tr id="TagManagement__row193632201781"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.2.3.1.1 "><p id="TagManagement__p736315201282">Tag value</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.2.3.1.2 "><ul id="TagManagement__ul1216717306916"><li id="TagManagement__li2279188184015">Can contain 0 to 255 characters.</li><li id="TagManagement__li1169310184412">Can contain letters, digits, spaces, and special characters _ . : = + - @</li><li id="TagManagement__li358452535110">Cannot start or end with a space in instance creation.</li></ul>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.2.3.1.2 "><ul id="TagManagement__ul1216717306916"><li id="TagManagement__li2279188184015">Can contain 0 to 255 characters.</li><li id="TagManagement__li1169310184412">Can contain letters, digits, spaces, and special characters _.:=+-@</li><li id="TagManagement__li358452535110">Cannot start or end with a space in instance creation.</li></ul>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
</div>
|
</div>
|
||||||
<div class="section" id="TagManagement__section184051819123810"><h4 class="sectiontitle">Procedure</h4><ol id="TagManagement__ol3625154473819"><li id="TagManagement__li68751861819"><span>Log in to the console.</span></li><li id="TagManagement__li189561034172215"><span>Click <strong id="TagManagement__b14774858509570">Service List</strong> and choose <strong id="TagManagement__b2055053229570">Application</strong> > <strong id="TagManagement__b18820194239570">Distributed Message Service</strong>. The Kafka instance list is displayed.</span></li><li id="TagManagement__li1489192711531"><span>Click the name of an instance.</span></li><li id="TagManagement__li189991214141019"><span>In the navigation pane on the left, choose <strong id="TagManagement__b365705373516">Tags</strong>.</span><p><p id="TagManagement__p16898133410247">View the tags of the instance.</p>
|
<div class="section" id="TagManagement__section15121533164515"><h4 class="sectiontitle">Notes and Constraints</h4><p id="TagManagement__p1771119372455">Up to 20 tags can be added to a Kafka instance.</p>
|
||||||
</p></li><li id="TagManagement__li658882719391"><span>Perform the following operations as required:</span><p><ul id="TagManagement__ul143482985319"><li id="TagManagement__li184341129115317">Add a tag<ol type="a" id="TagManagement__ol15591750115315"><li id="TagManagement__li165913500532">Click <strong id="TagManagement__b192779879635755">Create/Delete Tag</strong>.</li><li id="TagManagement__li75973412544">Enter a tag key and a tag value, and click <strong id="TagManagement__b17953037173518">Add</strong>.<p id="TagManagement__p1580923115586">If you have predefined tags, select a predefined pair of tag key and value, and click <strong id="TagManagement__b99061025123610">Add</strong>.</p>
|
</div>
|
||||||
</li><li id="TagManagement__li1217122155815">Click <strong id="TagManagement__b13934927193610">OK</strong>.</li></ol>
|
<div class="section" id="TagManagement__section184051819123810"><h4 class="sectiontitle">Procedure</h4><ol id="TagManagement__ol3625154473819"><li id="TagManagement__li2186699118249"><span>Log in to the console.</span></li><li id="TagManagement__li189561034172215"><span>Click <strong id="TagManagement__b14774858509570">Service List</strong> and choose <strong id="TagManagement__b2055053229570">Application</strong> > <strong id="TagManagement__b18820194239570">Distributed Message Service</strong>. The Kafka instance list is displayed.</span></li><li id="TagManagement__li1489192711531"><span>Click the desired instance to go to the instance details page.</span></li><li id="TagManagement__li189991214141019"><span>In the navigation pane, choose <strong id="TagManagement__b365705373516">Tags</strong>.</span></li><li id="TagManagement__li658882719391"><span>Add or delete tags as required.</span><p><ul id="TagManagement__ul143482985319"><li id="TagManagement__li184341129115317">Add a tag<ol type="a" id="TagManagement__ol1639710122815"><li id="TagManagement__li173977182812">Click <strong id="TagManagement__b224383515">Create/Delete Tag</strong>.</li><li id="TagManagement__li75973412544">Enter a tag key and a tag value, and click <strong id="TagManagement__b17953037173518">Add</strong>.<p id="TagManagement__p1580923115586">If you have predefined tags, select a predefined pair of tag key and value, and click <strong id="TagManagement__b99061025123610">Add</strong>.</p>
|
||||||
|
</li><li id="TagManagement__li3397313281">Click <strong id="TagManagement__b981361501">OK</strong>.<p id="TagManagement__p173976113286">View the new tag on the tag list page.</p>
|
||||||
|
</li></ol>
|
||||||
</li><li id="TagManagement__li1931437145316">Delete a tag<p id="TagManagement__p10415343125317"><a name="TagManagement__li1931437145316"></a><a name="li1931437145316"></a>Delete a tag using either of the following methods:</p>
|
</li><li id="TagManagement__li1931437145316">Delete a tag<p id="TagManagement__p10415343125317"><a name="TagManagement__li1931437145316"></a><a name="li1931437145316"></a>Delete a tag using either of the following methods:</p>
|
||||||
<ul id="TagManagement__ul649151145418"><li id="TagManagement__li949311135418">In the row containing the tag to be deleted, click <strong id="TagManagement__b135611420124310">Delete</strong>. In the <strong id="TagManagement__b10212143711438">Delete Tag</strong> dialog box, click <strong id="TagManagement__b153592574422">Yes</strong>.</li><li id="TagManagement__li4750125155414">Click <strong id="TagManagement__b57028007435755">Create/Delete Tag</strong>. In the dialog box that is displayed, click <span><img id="TagManagement__image1543573125614" src="en-us_image_0000001427644729.png"></span> next to the tag to be deleted and click <strong id="TagManagement__b87193289449">OK</strong>.</li></ul>
|
<ul id="TagManagement__ul649151145418"><li id="TagManagement__li949311135418">In the row containing the tag to be deleted, click <strong id="TagManagement__b135611420124310">Delete</strong>. In the <strong id="TagManagement__b10212143711438">Delete Tag</strong> dialog box, click <strong id="TagManagement__b153592574422">Yes</strong>.</li><li id="TagManagement__li4750125155414">Click <strong id="TagManagement__b57028007435755">Create/Delete Tag</strong>. In the dialog box that is displayed, click <span><img id="TagManagement__image23696369314" src="en-us_image_0000002157871349.png"></span> next to the tag to be deleted and click <strong id="TagManagement__b87193289449">OK</strong>.<p id="TagManagement__p1521611111015">You can delete up to <strong id="TagManagement__b19932181222910">19</strong> tags in this way.</p>
|
||||||
|
</li><li id="TagManagement__li17363162010">Select one or more tags and click <strong id="TagManagement__b1168619817311">Delete</strong> above. The <strong id="TagManagement__b1818611013410">Delete Tag</strong> dialog box is displayed. Click <strong id="TagManagement__b613820186416">OK</strong>.</li></ul>
|
||||||
|
<p id="TagManagement__p1423620285485">The tags are deleted when they are no longer displayed in the tag list.</p>
|
||||||
</li></ul>
|
</li></ul>
|
||||||
</p></li></ol>
|
</p></li></ol>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@ -1,11 +1,11 @@
|
|||||||
<a name="UserPrivilegeManagement"></a><a name="UserPrivilegeManagement"></a>
|
<a name="UserPrivilegeManagement"></a><a name="UserPrivilegeManagement"></a>
|
||||||
|
|
||||||
<h1 class="topictitle1">Permission Management</h1>
|
<h1 class="topictitle1">Permissions Management</h1>
|
||||||
<div id="body8662426"><p id="UserPrivilegeManagement__en-us_topic_0170871335_p8060118"></p>
|
<div id="body8662426"><p id="UserPrivilegeManagement__en-us_topic_0170871335_p8060118"></p>
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<ul class="ullinks">
|
<ul class="ullinks">
|
||||||
<li class="ulchildlink"><strong><a href="CreateUserAndGrantPolicy.html">Creating a User and Granting DMS for Kafka Permissions</a></strong><br>
|
<li class="ulchildlink"><strong><a href="CreateUserAndGrantPolicy.html">Creating an IAM User and Granting DMS for Kafka Permissions</a></strong><br>
|
||||||
</li>
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 462 B |
|
Before Width: | Height: | Size: 401 B |
|
Before Width: | Height: | Size: 762 B |
|
Before Width: | Height: | Size: 407 B |
|
Before Width: | Height: | Size: 188 B After Width: | Height: | Size: 217 B |
|
Before Width: | Height: | Size: 416 B After Width: | Height: | Size: 461 B |
|
Before Width: | Height: | Size: 416 B |
|
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 23 KiB |
BIN
docs/dms/umn/en-us_image_0000002110644228.png
Normal file
|
After Width: | Height: | Size: 179 B |
BIN
docs/dms/umn/en-us_image_0000002157871349.png
Normal file
|
After Width: | Height: | Size: 222 B |
|
Before Width: | Height: | Size: 5.8 KiB After Width: | Height: | Size: 5.8 KiB |
|
Before Width: | Height: | Size: 6.0 KiB After Width: | Height: | Size: 6.0 KiB |
BIN
docs/dms/umn/en-us_image_0000002317148937.png
Normal file
|
After Width: | Height: | Size: 485 B |
BIN
docs/dms/umn/en-us_image_0000002317222029.png
Normal file
|
After Width: | Height: | Size: 485 B |
@ -27,7 +27,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.4.2.1.2.2.6.1.3 "><p id="kafka-bp-migration__p1921320288819">9094</p>
|
<td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.4.2.1.2.2.6.1.3 "><p id="kafka-bp-migration__p1921320288819">9094</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="18.02180218021802%" headers="mcps1.3.4.2.1.2.2.6.1.4 "><p id="kafka-bp-migration__p1121316281483">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="18.02180218021802%" headers="mcps1.3.4.2.1.2.2.6.1.4 "><p id="kafka-bp-migration__p1121316281483">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="38.46384638463846%" headers="mcps1.3.4.2.1.2.2.6.1.5 "><p id="kafka-bp-migration__p151381038801">Accessing a Kafka instance in a public network (in plaintext)</p>
|
<td class="cellrowborder" valign="top" width="38.46384638463846%" headers="mcps1.3.4.2.1.2.2.6.1.5 "><p id="kafka-bp-migration__p151381038801">Accessing a Kafka instance in a public network (in plaintext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -38,7 +38,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.4.2.1.2.2.6.1.3 "><p id="kafka-bp-migration__p21387381002">9095</p>
|
<td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.4.2.1.2.2.6.1.3 "><p id="kafka-bp-migration__p21387381002">9095</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="18.02180218021802%" headers="mcps1.3.4.2.1.2.2.6.1.4 "><p id="kafka-bp-migration__p151384381409">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="18.02180218021802%" headers="mcps1.3.4.2.1.2.2.6.1.4 "><p id="kafka-bp-migration__p151384381409">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="38.46384638463846%" headers="mcps1.3.4.2.1.2.2.6.1.5 "><p id="kafka-bp-migration__p1345124955213">Accessing a Kafka instance in a public network (in ciphertext)</p>
|
<td class="cellrowborder" valign="top" width="38.46384638463846%" headers="mcps1.3.4.2.1.2.2.6.1.5 "><p id="kafka-bp-migration__p1345124955213">Accessing a Kafka instance in a public network (in ciphertext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -56,8 +56,7 @@
|
|||||||
</div>
|
</div>
|
||||||
<div class="section" id="kafka-bp-migration__section163553541014"><h4 class="sectiontitle">Migration Scheme 2: Migrating the Production Later</h4><p id="kafka-bp-migration__p10292175955518">Use multiple consumers for the consumption service. Some consume messages from the original Kafka instance, and others consume messages from the new Kafka instances. Then, migrate the production service to the new Kafka instance so that all messages can be consumed in time.</p>
|
<div class="section" id="kafka-bp-migration__section163553541014"><h4 class="sectiontitle">Migration Scheme 2: Migrating the Production Later</h4><p id="kafka-bp-migration__p10292175955518">Use multiple consumers for the consumption service. Some consume messages from the original Kafka instance, and others consume messages from the new Kafka instances. Then, migrate the production service to the new Kafka instance so that all messages can be consumed in time.</p>
|
||||||
<p id="kafka-bp-migration__p11914729171817">For a certain period of time, the consumption service consumes messages from both the original and new Kafka instances. Before the migration, message consumption from the new Kafka instance has already started, so there is no latency. However, early on in the migration, data is consumed from both the original and new Kafka instances, so the messages may not be consumed in the order that they are produced. This scheme is <strong id="kafka-bp-migration__b166211273158">suitable for services that require low latency but do not require strict message sequence</strong>.</p>
|
<p id="kafka-bp-migration__p11914729171817">For a certain period of time, the consumption service consumes messages from both the original and new Kafka instances. Before the migration, message consumption from the new Kafka instance has already started, so there is no latency. However, early on in the migration, data is consumed from both the original and new Kafka instances, so the messages may not be consumed in the order that they are produced. This scheme is <strong id="kafka-bp-migration__b166211273158">suitable for services that require low latency but do not require strict message sequence</strong>.</p>
|
||||||
<ol id="kafka-bp-migration__ol240126202"><li id="kafka-bp-migration__li1741142611015"><span>Start new consumer clients, set the Kafka connection addresses to that of the new Kafka instance, and consume data from the new Kafka instance.</span><p><div class="note" id="kafka-bp-migration__note18111437128"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="kafka-bp-migration__p121143715219">Original consumer clients must continue running. Messages are consumed from both the original and new Kafka instances.</p>
|
<ol id="kafka-bp-migration__ol240126202"><li id="kafka-bp-migration__li1741142611015"><span>Start new consumer clients, set the Kafka connection addresses to that of the new Kafka instance, and consume data from the new Kafka instance.</span><p><p id="kafka-bp-migration__p47106565446">Original consumer clients must continue running. Messages are consumed from both the original and new Kafka instances.</p>
|
||||||
</div></div>
|
|
||||||
</p></li><li id="kafka-bp-migration__li34113261704"><span>Change the Kafka connection address of the producer to that of the new Kafka instance.</span></li><li id="kafka-bp-migration__li3417261805"><span>Restart the producer client to migrate the production service to the new Kafka instance.</span></li><li id="kafka-bp-migration__li141826403"><span>After the production service is migrated, check whether the consumption service connected to the new Kafka instance is normal.</span></li><li id="kafka-bp-migration__li134116261701"><span>After all data in the original Kafka is consumed, close the original consumption clients.</span></li><li id="kafka-bp-migration__li9418265012"><span>The migration is complete.</span></li></ol>
|
</p></li><li id="kafka-bp-migration__li34113261704"><span>Change the Kafka connection address of the producer to that of the new Kafka instance.</span></li><li id="kafka-bp-migration__li3417261805"><span>Restart the producer client to migrate the production service to the new Kafka instance.</span></li><li id="kafka-bp-migration__li141826403"><span>After the production service is migrated, check whether the consumption service connected to the new Kafka instance is normal.</span></li><li id="kafka-bp-migration__li134116261701"><span>After all data in the original Kafka is consumed, close the original consumption clients.</span></li><li id="kafka-bp-migration__li9418265012"><span>The migration is complete.</span></li></ol>
|
||||||
</div>
|
</div>
|
||||||
<div class="section" id="kafka-bp-migration__section20636193514107"><h4 class="sectiontitle">Migration Scheme 3: Migrating the Consumption First</h4><p id="kafka-bp-migration__p1678612717184">Use Smart Connect to synchronize the two Kafka instances, migrate the consumer first and then the producer to the new Kafka instance.</p>
|
<div class="section" id="kafka-bp-migration__section20636193514107"><h4 class="sectiontitle">Migration Scheme 3: Migrating the Consumption First</h4><p id="kafka-bp-migration__p1678612717184">Use Smart Connect to synchronize the two Kafka instances, migrate the consumer first and then the producer to the new Kafka instance.</p>
|
||||||
|
|||||||
@ -1,9 +1,8 @@
|
|||||||
<a name="kafka-faq-0001"></a><a name="kafka-faq-0001"></a>
|
<a name="kafka-faq-0001"></a><a name="kafka-faq-0001"></a>
|
||||||
|
|
||||||
<h1 class="topictitle1">How Do I Obtain the Public Access Address After Public Access Is Enabled?</h1>
|
<h1 class="topictitle1">How Do I Obtain the Public Access Address After Public Access Is Enabled?</h1>
|
||||||
<div id="body1600394208994"><p id="kafka-faq-0001__p8060118">Click the name of your Kafka instance. In the <strong id="kafka-faq-0001__b791934213818">Connection</strong> section on the <strong id="kafka-faq-0001__b12919542688">Basic Information</strong> tab page, view <strong id="kafka-faq-0001__b1862120185111">Instance Address (Public Network)</strong>.</p>
|
<div id="body1600394208994"><p id="kafka-faq-0001__p8060118">Click the name of your Kafka instance. In the <strong id="kafka-faq-0001__b791934213818">Connection</strong> section on the <strong id="kafka-faq-0001__b12919542688">Basic Information</strong> page, view <strong id="kafka-faq-0001__b1862120185111">Instance Address (Public Network)</strong>.</p>
|
||||||
<p id="kafka-faq-0001__p5569193615812">For details about how to connect to a Kafka instance, see <a href="kafka-ug190605003.html">Connecting to an Instance</a>.</p>
|
<p id="kafka-faq-0001__p5569193615812">For details about how to connect to a Kafka instance, see <a href="kafka-ug190605003.html">Connecting to an Instance</a>.</p>
|
||||||
<p id="kafka-faq-0001__p17734191149"></p>
|
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -1,8 +1,7 @@
|
|||||||
<a name="kafka-faq-0005"></a><a name="kafka-faq-0005"></a>
|
<a name="kafka-faq-0005"></a><a name="kafka-faq-0005"></a>
|
||||||
|
|
||||||
<h1 class="topictitle1">How Do I Increase the Partition Quantity?</h1>
|
<h1 class="topictitle1">How Do I Increase the Partition Quantity?</h1>
|
||||||
<div id="body1598670216256"><p id="kafka-faq-0005__p8934179194916">You can increase the partition quantity by adding brokers.</p>
|
<div id="body1598670216256"><p id="kafka-faq-0005__p8934179194916">You can increase the partition quantity by increasing broker flavors or adding brokers. For details, see <a href="kafka-ug-181221001.html">Modifying Cluster Kafka Instance Specifications</a>.</p>
|
||||||
<p id="kafka-faq-0005__p109345984919">To do so, go to the Kafka console, locate the row that contains the desired instance, and choose <strong id="kafka-faq-0005__b3119158141916">More</strong> > <strong id="kafka-faq-0005__b91203582192">Modify Specifications</strong>. On the page that is displayed, add brokers as required. For details, see <a href="kafka-ug-181221001.html">Modifying Kafka Instance Specifications</a>.</p>
|
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -11,7 +11,7 @@
|
|||||||
</thead>
|
</thead>
|
||||||
<tbody><tr id="kafka-faq-0033__row178116245715"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0033__p18117219572">100 MB/s</p>
|
<tbody><tr id="kafka-faq-0033__row178116245715"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0033__p18117219572">100 MB/s</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.2 "><p id="kafka-faq-0033__p128115215714">3000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.2 "><p id="kafka-faq-0033__p128115215714">3,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row181116295718"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0033__p38115245715">300 MB/s</p>
|
<tr id="kafka-faq-0033__row181116295718"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0033__p38115245715">300 MB/s</p>
|
||||||
@ -24,7 +24,7 @@
|
|||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.2 "><p id="kafka-faq-0033__p38116235714">20,000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.2 "><p id="kafka-faq-0033__p38116235714">20,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row1811325573"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0033__p581112214576">1200 MB/s</p>
|
<tr id="kafka-faq-0033__row1811325573"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0033__p581112214576">1,200 MB/s</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.2 "><p id="kafka-faq-0033__p8811142145712">20,000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.3.1.2 "><p id="kafka-faq-0033__p8811142145712">20,000</p>
|
||||||
</td>
|
</td>
|
||||||
@ -41,42 +41,42 @@
|
|||||||
</thead>
|
</thead>
|
||||||
<tbody><tr id="kafka-faq-0033__row154214190914"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p954214191592">kafka.2u4g.cluster.small</p>
|
<tbody><tr id="kafka-faq-0033__row154214190914"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p954214191592">kafka.2u4g.cluster.small</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p165422192097">2000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p165422192097">2,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row15542201914910"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p454215194915">kafka.2u4g.single.small</p>
|
<tr id="kafka-faq-0033__row15542201914910"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p454215194915">kafka.2u4g.single.small</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p8542719496">2000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p8542719496">2,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row135421619791"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p25423194919">kafka.2u4g.cluster</p>
|
<tr id="kafka-faq-0033__row135421619791"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p25423194919">kafka.2u4g.cluster</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p1654213192913">2000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p1654213192913">2,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row185428191392"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p6542111915912">kafka.2u4g.single</p>
|
<tr id="kafka-faq-0033__row185428191392"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p6542111915912">kafka.2u4g.single</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p554291918919">2000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p554291918919">2,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row65421619596"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p115421719394">kafka.4u8g.cluster</p>
|
<tr id="kafka-faq-0033__row65421619596"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p115421719394">kafka.4u8g.cluster</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p1054214193914">4000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p1054214193914">4,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row2449183716140"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p84496370149">kafka.8u16g.cluster</p>
|
<tr id="kafka-faq-0033__row2449183716140"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p84496370149">kafka.8u16g.cluster</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p245073712145">4000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p245073712145">4,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row75271243181419"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p752834361410">kafka.12u24g.cluster</p>
|
<tr id="kafka-faq-0033__row75271243181419"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p752834361410">kafka.12u24g.cluster</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p14528134319142">4000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p14528134319142">4,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0033__row1269553051514"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p14696103014151">kafka.16u32g.cluster</p>
|
<tr id="kafka-faq-0033__row1269553051514"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.1 "><p id="kafka-faq-0033__p14696103014151">kafka.16u32g.cluster</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p86961530191517">4000</p>
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.3.1.2 "><p id="kafka-faq-0033__p86961530191517">4,000</p>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
|
|||||||
@ -1,7 +1,7 @@
|
|||||||
<a name="kafka-faq-0035"></a><a name="kafka-faq-0035"></a>
|
<a name="kafka-faq-0035"></a><a name="kafka-faq-0035"></a>
|
||||||
|
|
||||||
<h1 class="topictitle1">Does Specification Modification Affect Services?</h1>
|
<h1 class="topictitle1">Does Specification Modification Affect Services?</h1>
|
||||||
<div id="body0000001086724891"><p id="kafka-faq-0035__p59461753018"><a href="#kafka-faq-0035__table28771010122913">Table 1</a> describes the impact of increasing specifications. It takes 5 to 10 minutes to modify specifications on one broker. The more brokers, the longer time the modification takes.</p>
|
<div id="body0000001086724891"><p id="kafka-faq-0035__p59461753018"><a href="#kafka-faq-0035__table28771010122913">Table 1</a> describes the impact of increasing specifications.</p>
|
||||||
|
|
||||||
<div class="tablenoborder"><a name="kafka-faq-0035__table28771010122913"></a><a name="table28771010122913"></a><table cellpadding="4" cellspacing="0" summary="" id="kafka-faq-0035__table28771010122913" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Impact of specification modification</caption><thead align="left"><tr id="kafka-faq-0035__kafka-ug-181221001_row19997194714410"><th align="left" class="cellrowborder" valign="top" width="20.03%" id="mcps1.3.2.2.3.1.1"><p id="kafka-faq-0035__kafka-ug-181221001_p1199711474444">Modified Object</p>
|
<div class="tablenoborder"><a name="kafka-faq-0035__table28771010122913"></a><a name="table28771010122913"></a><table cellpadding="4" cellspacing="0" summary="" id="kafka-faq-0035__table28771010122913" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Impact of specification modification</caption><thead align="left"><tr id="kafka-faq-0035__kafka-ug-181221001_row19997194714410"><th align="left" class="cellrowborder" valign="top" width="20.03%" id="mcps1.3.2.2.3.1.1"><p id="kafka-faq-0035__kafka-ug-181221001_p1199711474444">Modified Object</p>
|
||||||
</th>
|
</th>
|
||||||
@ -11,7 +11,7 @@
|
|||||||
</thead>
|
</thead>
|
||||||
<tbody><tr id="kafka-faq-0035__kafka-ug-181221001_row179971047184413"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0035__kafka-ug-181221001_p399710476449">Bandwidth or broker quantity</p>
|
<tbody><tr id="kafka-faq-0035__kafka-ug-181221001_row179971047184413"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0035__kafka-ug-181221001_p399710476449">Bandwidth or broker quantity</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="79.97%" headers="mcps1.3.2.2.3.1.2 "><ul id="kafka-faq-0035__kafka-ug-181221001_ul15253145113482"><li id="kafka-faq-0035__kafka-ug-181221001_li1725385134814">Increasing the bandwidth or adding brokers does not affect the original brokers or services.</li><li id="kafka-faq-0035__kafka-ug-181221001_li14341154514530">When you increase the bandwidth or change the broker quantity, the storage space is proportionally expanded based on the current disk space. For example, assume that the original number of brokers of an instance is 3 and the disk size of each broker is 200 GB. If the broker quantity changes to 10 and the disk size of each broker is still 200 GB, the total disk size becomes 2000 GB.</li><li id="kafka-faq-0035__kafka-ug-181221001_li131021316135411">New topics are created on new brokers, and the original topics are still on the original brokers, resulting in unbalanced partitions. You can <a href="kafka_ug_0023.html">reassign partitions</a> to migrate the replicas of the original topic partitions to the new brokers.</li></ul>
|
<td class="cellrowborder" valign="top" width="79.97%" headers="mcps1.3.2.2.3.1.2 "><ul id="kafka-faq-0035__kafka-ug-181221001_ul15253145113482"><li id="kafka-faq-0035__kafka-ug-181221001_li1725385134814">Increasing the bandwidth or adding brokers does not affect the original brokers or services.</li><li id="kafka-faq-0035__kafka-ug-181221001_li14341154514530">When you increase the bandwidth or change the broker quantity, the storage space is proportionally expanded based on the current disk space. For example, assume that the original number of brokers of an instance is 3 and the disk size of each broker is 200 GB. If the broker quantity changes to 10 and the disk size of each broker is still 200 GB, the total disk size becomes 2,000 GB.</li><li id="kafka-faq-0035__kafka-ug-181221001_li131021316135411">New topics are created on new brokers, and the original topics are still on the original brokers, resulting in unbalanced partitions. You can <a href="kafka_ug_0023.html">reassign partitions</a> to migrate the replicas of the original topic partitions to the new brokers.</li></ul>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr id="kafka-faq-0035__kafka-ug-181221001_row18997184794418"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0035__kafka-ug-181221001_p399874715449">Storage space</p>
|
<tr id="kafka-faq-0035__kafka-ug-181221001_row18997184794418"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.2.2.3.1.1 "><p id="kafka-faq-0035__kafka-ug-181221001_p399874715449">Storage space</p>
|
||||||
@ -24,7 +24,7 @@
|
|||||||
<td class="cellrowborder" valign="top" width="79.97%" headers="mcps1.3.2.2.3.1.2 "><ul id="kafka-faq-0035__kafka-ug-181221001_ul34811410173217"><li id="kafka-faq-0035__kafka-ug-181221001_li2481181033214">Single-replica topics do not support message production during this period. Services will be interrupted.</li><li id="kafka-faq-0035__kafka-ug-181221001_li8481121018322">If a topic has multiple replicas, modifying the broker flavor does not interrupt services, but may cause disorder of partition messages. Evaluate this impact and avoid peak hours.</li><li id="kafka-faq-0035__kafka-ug-181221001_li134821710153220">Broker rolling restarts will cause partition leader changes, interrupting connections for less than a minute when the network is stable. For multi-replica topics, configure the retry mechanism on the producer client. To do so:<ul id="kafka-faq-0035__kafka-ug-181221001_ul184821910143210"><li id="kafka-faq-0035__kafka-ug-181221001_li848211107328">If you use an open-source Kafka client, configure the <strong id="kafka-faq-0035__kafka-ug-181221001_b2426143111213">retries</strong> parameter to a value in the range from 3 to 5.</li><li id="kafka-faq-0035__kafka-ug-181221001_li248215102324">If you use Flink, configure the retry policy by referring to the following code:<pre class="screen" id="kafka-faq-0035__kafka-ug-181221001_screen74827109323">StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
|
<td class="cellrowborder" valign="top" width="79.97%" headers="mcps1.3.2.2.3.1.2 "><ul id="kafka-faq-0035__kafka-ug-181221001_ul34811410173217"><li id="kafka-faq-0035__kafka-ug-181221001_li2481181033214">Single-replica topics do not support message production during this period. Services will be interrupted.</li><li id="kafka-faq-0035__kafka-ug-181221001_li8481121018322">If a topic has multiple replicas, modifying the broker flavor does not interrupt services, but may cause disorder of partition messages. Evaluate this impact and avoid peak hours.</li><li id="kafka-faq-0035__kafka-ug-181221001_li134821710153220">Broker rolling restarts will cause partition leader changes, interrupting connections for less than a minute when the network is stable. For multi-replica topics, configure the retry mechanism on the producer client. To do so:<ul id="kafka-faq-0035__kafka-ug-181221001_ul184821910143210"><li id="kafka-faq-0035__kafka-ug-181221001_li848211107328">If you use an open-source Kafka client, configure the <strong id="kafka-faq-0035__kafka-ug-181221001_b2426143111213">retries</strong> parameter to a value in the range from 3 to 5.</li><li id="kafka-faq-0035__kafka-ug-181221001_li248215102324">If you use Flink, configure the retry policy by referring to the following code:<pre class="screen" id="kafka-faq-0035__kafka-ug-181221001_screen74827109323">StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
|
||||||
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.seconds(20)));</pre>
|
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.seconds(20)));</pre>
|
||||||
</li></ul>
|
</li></ul>
|
||||||
</li></ul>
|
</li><li id="kafka-faq-0035__kafka-ug-181221001_li11859154183815">It takes 5 to 10 minutes to modify specifications on one broker. The more brokers, the longer time the modification takes.</li></ul>
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
|
|||||||
@ -8,7 +8,6 @@
|
|||||||
<div class="fignone" id="kafka-faq-0038__fig34751040942"><span class="figcap"><b>Figure 1 </b>Consumer access permissions</span><br><span><img class="eddx" id="kafka-faq-0038__image1147512401942" src="en-us_image_0000001384665714.png"></span></div>
|
<div class="fignone" id="kafka-faq-0038__fig34751040942"><span class="figcap"><b>Figure 1 </b>Consumer access permissions</span><br><span><img class="eddx" id="kafka-faq-0038__image1147512401942" src="en-us_image_0000001384665714.png"></span></div>
|
||||||
<p id="kafka-faq-0038__p10835175711273"><strong id="kafka-faq-0038__b91574437481">Solution:</strong></p>
|
<p id="kafka-faq-0038__p10835175711273"><strong id="kafka-faq-0038__b91574437481">Solution:</strong></p>
|
||||||
<ul id="kafka-faq-0038__ul2081872924412"><li id="kafka-faq-0038__li18186296441">If all consumers must be in the same consumer group (<strong id="kafka-faq-0038__b1549719178494">group.id</strong> is the same), grant the same topic access permissions to all the consumers.</li><li id="kafka-faq-0038__li45446162174">If the consumers do not need to be in the same consumer group, change the value of <strong id="kafka-faq-0038__b1124519452492">group.id</strong> to ensure that each consumer is in a separate consumer group.</li></ul>
|
<ul id="kafka-faq-0038__ul2081872924412"><li id="kafka-faq-0038__li18186296441">If all consumers must be in the same consumer group (<strong id="kafka-faq-0038__b1549719178494">group.id</strong> is the same), grant the same topic access permissions to all the consumers.</li><li id="kafka-faq-0038__li45446162174">If the consumers do not need to be in the same consumer group, change the value of <strong id="kafka-faq-0038__b1124519452492">group.id</strong> to ensure that each consumer is in a separate consumer group.</li></ul>
|
||||||
<p id="kafka-faq-0038__p71511230104011"></p>
|
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -2,9 +2,8 @@
|
|||||||
|
|
||||||
<h1 class="topictitle1">How Do I View the Number of Accumulated Messages?</h1>
|
<h1 class="topictitle1">How Do I View the Number of Accumulated Messages?</h1>
|
||||||
<div id="body0000001181826821"><p id="kafka-faq-0041__p112522472014">View the number of accumulated messages using any of the following methods:</p>
|
<div id="body0000001181826821"><p id="kafka-faq-0041__p112522472014">View the number of accumulated messages using any of the following methods:</p>
|
||||||
<ul id="kafka-faq-0041__ul1657019511234"><li id="kafka-faq-0041__li11570185132316">On the <strong id="kafka-faq-0041__b499124111440">Consumer Groups</strong> page of an instance, click the name of the consumer group whose accumulated messages are to be viewed. The consumer group details page is displayed. On the <strong id="kafka-faq-0041__b1917019202457">Consumer Offset</strong> tab page, view the number of messages accumulated in each topic of your target consumer group. For details, see <a href="kafka-ug-0014.html">Viewing Consumer Offsets</a>.</li><li id="kafka-faq-0041__li26102327574">On the <strong id="kafka-faq-0041__b11827104364415">Monitoring</strong> tab page of an instance, click the <strong id="kafka-faq-0041__b5827194334412">By Consumer Group</strong> tab. Select the desired consumer group for <strong id="kafka-faq-0041__b0827104374412">Consumer Group</strong> and <strong id="kafka-faq-0041__b482744319443">All topics</strong> for <strong id="kafka-faq-0041__b158271543114415">Topic</strong>. The <strong id="kafka-faq-0041__b58274435449">Consumer Available Messages</strong> metric reflects the number of messages accumulated in all topics of this consumer group. For details about viewing the monitoring data, see <a href="kafka-ug-190605001.html">Viewing Kafka Metrics</a>.</li><li id="kafka-faq-0041__li20593381704">On the <strong id="kafka-faq-0041__b2824949165211">Consumer Groups</strong> tab page of the Cloud Eye console, click the <strong id="kafka-faq-0041__b1082494985210">By Consumer Group</strong> tab. Select the desired consumer group for <strong id="kafka-faq-0041__b1682417498528">Consumer Group</strong> and <strong id="kafka-faq-0041__b108251249145219">All topics</strong> for <strong id="kafka-faq-0041__b11825249165211">Topic</strong>. The <strong id="kafka-faq-0041__b4825104910529">Consumer Available Messages</strong> metric reflects the number of messages accumulated in all topics of this consumer group. For details about viewing the monitoring data, see <a href="kafka-ug-190605001.html">Viewing Kafka Metrics</a>.</li><li id="kafka-faq-0041__li589275520402">In the <strong id="kafka-faq-0041__b168614112118">/bin</strong> directory on the <a href="https://cwiki.apache.org/confluence/display/KAFKA/Clients" target="_blank" rel="noopener noreferrer">Kafka client</a>, run command <strong id="kafka-faq-0041__b102956215204">kafka-consumer-groups.sh --bootstrap-server $<em id="kafka-faq-0041__i162742719229">{connection-address}</em> --describe --group $<em id="kafka-faq-0041__i73992011192212">{consumer-group-name}</em></strong> to check number of accumulated messages of each topic in a consumer group. <strong id="kafka-faq-0041__b13303137155517">LAG</strong> indicates the total number of messages accumulated in each topic.<div class="fignone" id="kafka-faq-0041__fig158441312131415"><span class="figcap"><b>Figure 1 </b>Viewing the total number of messages accumulated in each topic</span><br><span><img id="kafka-faq-0041__image188441312201411" src="en-us_image_0000001435265813.png"></span></div>
|
<ul id="kafka-faq-0041__ul1657019511234"><li id="kafka-faq-0041__li11570185132316">On the <strong id="kafka-faq-0041__b499124111440">Consumer Groups</strong> page of an instance, click the name of the consumer group whose accumulated messages are to be viewed. The consumer group details page is displayed. On the <strong id="kafka-faq-0041__b1917019202457">Consumer Offset</strong> tab page, view the number of messages accumulated in each topic of your target consumer group. For details, see <a href="kafka-ug-0014.html">Viewing Consumer Offsets</a>.</li><li id="kafka-faq-0041__li26102327574">On the <strong id="kafka-faq-0041__b883301414198">Monitoring</strong> page of an instance, click the <strong id="kafka-faq-0041__b5827194334412">By Consumer Group</strong> tab. Select the desired consumer group for <strong id="kafka-faq-0041__b0827104374412">Consumer Group</strong> and <strong id="kafka-faq-0041__b482744319443">All topics</strong> for <strong id="kafka-faq-0041__b158271543114415">Topic</strong>. The <strong id="kafka-faq-0041__b58274435449">Consumer Available Messages</strong> metric reflects the number of messages accumulated in all topics of this consumer group. For details about viewing the monitoring data, see <a href="kafka-ug-190605001.html">Viewing Kafka Metrics</a>.</li><li id="kafka-faq-0041__li20593381704">On the <strong id="kafka-faq-0041__b2824949165211">Consumer Groups</strong> tab page of the Cloud Eye console, click the <strong id="kafka-faq-0041__b1082494985210">By Consumer Group</strong> tab. Select the desired consumer group for <strong id="kafka-faq-0041__b1682417498528">Consumer Group</strong> and <strong id="kafka-faq-0041__b108251249145219">All topics</strong> for <strong id="kafka-faq-0041__b11825249165211">Topic</strong>. The <strong id="kafka-faq-0041__b4825104910529">Consumer Available Messages</strong> metric reflects the number of messages accumulated in all topics of this consumer group. For details about viewing the monitoring data, see <a href="kafka-ug-190605001.html">Viewing Kafka Metrics</a>.</li><li id="kafka-faq-0041__li589275520402">In the <strong id="kafka-faq-0041__b168614112118">/bin</strong> directory on the <a href="https://cwiki.apache.org/confluence/display/KAFKA/Clients" target="_blank" rel="noopener noreferrer">Kafka client</a>, run command <strong id="kafka-faq-0041__b102956215204">kafka-consumer-groups.sh --bootstrap-server $<em id="kafka-faq-0041__i162742719229">{connection-address}</em> --describe --group $<em id="kafka-faq-0041__i73992011192212">{consumer-group-name}</em></strong> to check number of accumulated messages of each topic in a consumer group. <strong id="kafka-faq-0041__b13303137155517">LAG</strong> indicates the total number of messages accumulated in each topic.<div class="fignone" id="kafka-faq-0041__fig158441312131415"><span class="figcap"><b>Figure 1 </b>Viewing the total number of messages accumulated in each topic</span><br><span><img id="kafka-faq-0041__image188441312201411" src="en-us_image_0000001435265813.png"></span></div>
|
||||||
<div class="note" id="kafka-faq-0041__note1457918331113"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="kafka-faq-0041__p135791733101111">If SASL authentication is enabled for the Kafka instance, the <strong id="kafka-faq-0041__b3510834194511">--command-config <em id="kafka-faq-0041__i185101934184518">{SASL authentication configuration file consumer.properties}</em></strong> parameter must be added to the preceding commands. For details about the configuration file <strong id="kafka-faq-0041__b3510334184516">consumer.properties</strong>, see the CLI access instructions provided in <a href="kafka-ug-180801001.html">Connecting to Kafka Using the Client (Ciphertext Access)</a>.</p>
|
<p id="kafka-faq-0041__p39191724244">If SASL authentication is enabled for the Kafka instance, the <strong id="kafka-faq-0041__b1334183115917">--command-config <em id="kafka-faq-0041__i5334935598">{SASL authentication configuration file consumer.properties}</em></strong> parameter must be added to the preceding commands. For details about the configuration file <strong id="kafka-faq-0041__b18334737594">consumer.properties</strong>, see the CLI access instructions provided in <a href="kafka-ug-180801001.html">Connecting to Kafka Using the Client (Ciphertext Access)</a>.</p>
|
||||||
</div></div>
|
|
||||||
</li></ul>
|
</li></ul>
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
|
|||||||
11
docs/dms/umn/kafka-faq-0063.html
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
<a name="kafka-faq-0063"></a><a name="kafka-faq-0063"></a>
|
||||||
|
|
||||||
|
<h1 class="topictitle1">Why Do Metrics Fluctuate Significantly (Disk Read/Write Speed, Average Disk Read/Write Time, and CPU Usage)?</h1>
|
||||||
|
<div id="body0000002106620465"><p id="kafka-faq-0063__p1989232549">Metrics including <strong id="kafka-faq-0063__b180711278344">Disk Read Speed</strong>, <strong id="kafka-faq-0063__b1688593019349">Disk Write Speed</strong>, <strong id="kafka-faq-0063__b10273133317341">Average Disk Read Time</strong>, <strong id="kafka-faq-0063__b240116366340">Average Disk Write Time</strong>, and <strong id="kafka-faq-0063__b9738133873415">CPU Usage</strong> output instantaneous values. These metrics are for reference only. Generally, flushing Kafka data asynchronously to disk consumes disk I/O and CPU usage, causing fluctuations. They do not affect services.</p>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="familylinks">
|
||||||
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-191030004.html">Monitoring & Alarm</a></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
11
docs/dms/umn/kafka-faq-0064.html
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
<a name="kafka-faq-0064"></a><a name="kafka-faq-0064"></a>
|
||||||
|
|
||||||
|
<h1 class="topictitle1">Why Does JVM Heap Memory Usage of JVM Fluctuate Significantly?</h1>
|
||||||
|
<div id="body0000002070872392"><p id="kafka-faq-0064__p1989232549">It is normal that <strong id="kafka-faq-0064__b4476165294118">JVM Heap Memory Usage of JVM</strong> is displayed in a sawtooth shape on the monitoring page. The maximum value may further increase or the fluctuation may be greater after scale-out or an engine upgrade. Such fluctuations are caused by normal JVM heap memory reclamation, and do not affect services.</p>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="familylinks">
|
||||||
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-191030004.html">Monitoring & Alarm</a></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
12
docs/dms/umn/kafka-faq-0065.html
Normal file
@ -0,0 +1,12 @@
|
|||||||
|
<a name="kafka-faq-0065"></a><a name="kafka-faq-0065"></a>
|
||||||
|
|
||||||
|
<h1 class="topictitle1">Why Are Offsets Not Continuous?</h1>
|
||||||
|
<div id="body0000002106681041"><p id="kafka-faq-0065__p9998429174212">If you have enabled idempotence or transactions on the producer client, and produced messages, message offsets are not continuous on the consumer client or on the <strong id="kafka-faq-0065__b1841184575819">Message Query</strong> page on the Kafka console. This is because enabling idempotence or transactions generates some metadata control messages during message production. These control messages are produced to topics, and are invisible to consumers.</p>
|
||||||
|
<p id="kafka-faq-0065__p1023711518442">By default, idempotence is enabled for Kafka 3.0 and later producer clients. To disable it, set the parameter <strong id="kafka-faq-0065__b7497663319">enable.idempotence</strong> to <strong id="kafka-faq-0065__b13277169033">false</strong>.</p>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="familylinks">
|
||||||
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-190416001.html">Messages</a></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
11
docs/dms/umn/kafka-faq-0066.html
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
<a name="kafka-faq-0066"></a><a name="kafka-faq-0066"></a>
|
||||||
|
|
||||||
|
<h1 class="topictitle1">Why Is Production Rate Still 0 When There Are Produced Messages?</h1>
|
||||||
|
<div id="body0000002141826937"><p id="kafka-faq-0066__p5643961112">The metric data of the message production rate is reported once every minute. Message production rate = Number of messages produced per minute/60. The value is rounded down. The production rate is displayed as 0 if the number of messages produced within a minute is less than 60.</p>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="familylinks">
|
||||||
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-191030004.html">Monitoring & Alarm</a></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
15
docs/dms/umn/kafka-faq-0067.html
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
<a name="kafka-faq-0067"></a><a name="kafka-faq-0067"></a>
|
||||||
|
|
||||||
|
<h1 class="topictitle1">Why Is the Number of Stacked Messages Monitored as 0 when Messages Are Stacked?</h1>
|
||||||
|
<div id="body0000002243701642"><div class="section" id="kafka-faq-0067__section8946016191120"><h4 class="sectiontitle">Symptom</h4><p id="kafka-faq-0067__p978010204112">On the <strong id="kafka-faq-0067__b1927752032020">Consumer Offset</strong> tab page of the consumer group details page, there are stacked messages in a topic (total stack ≠ 0), whereas the <strong id="kafka-faq-0067__b3418418142215">Consumer Available Messages</strong> on the consumer group monitoring page is 0.</p>
|
||||||
|
</div>
|
||||||
|
<div class="section" id="kafka-faq-0067__section17671142912162"><h4 class="sectiontitle">Conclusion</h4><p id="kafka-faq-0067__p4182155113915">The total stack on the <strong id="kafka-faq-0067__b176211579238">Consumer Offset</strong> tab page and the <strong id="kafka-faq-0067__b222718211249">Consumer Available Messages</strong> are not synchronously updated.</p>
|
||||||
|
<p id="kafka-faq-0067__p9899165613319">The total stack is sampled immediately. The <strong id="kafka-faq-0067__b117716478257">Consumer Available Messages</strong> is sampled every minute. You see the stack number at your time. When a consumer consumes all the stacked messages within 1 minute, <strong id="kafka-faq-0067__b19169191902919">Consumer Available Messages</strong> displays 0.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="familylinks">
|
||||||
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-191030004.html">Monitoring & Alarm</a></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
11
docs/dms/umn/kafka-faq-0068.html
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
<a name="kafka-faq-0068"></a><a name="kafka-faq-0068"></a>
|
||||||
|
|
||||||
|
<h1 class="topictitle1">Can I Purchase Partitions Separately?</h1>
|
||||||
|
<div id="body0000002293023660"><p id="kafka-faq-0068__p8934179194916">No. However, you can increase the total number of partitions by increasing the broker quantity or flavor. For details, see <a href="kafka-ug-181221001.html">Modifying Cluster Kafka Instance Specifications</a>.</p>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="familylinks">
|
||||||
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-191030003.html">Topics and Partitions</a></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
12
docs/dms/umn/kafka-faq-0069.html
Normal file
@ -0,0 +1,12 @@
|
|||||||
|
<a name="kafka-faq-0069"></a><a name="kafka-faq-0069"></a>
|
||||||
|
|
||||||
|
<h1 class="topictitle1">Why Are Consumer Groups Missing?</h1>
|
||||||
|
<div id="body0000002293183328"><p id="kafka-faq-0069__p8060118">If you cannot find a consumer group from the console, the possible causes are as follows:</p>
|
||||||
|
<ul id="kafka-faq-0069__ul8867134115100"><li id="kafka-faq-0069__li4867441161019"><strong id="kafka-faq-0069__b187240133612">auto.create.groups.enable</strong> is set to <strong id="kafka-faq-0069__b118814406362">true</strong>, the consumer group status is <strong id="kafka-faq-0069__b78884017361">EMPTY</strong>, and no offset has been submitted. The system automatically deletes the consumer group 10 minutes after it is created.</li><li id="kafka-faq-0069__li8961454201011">If the consumer group has never committed an offset, it will be deleted after the Kafka instance restarts.</li></ul>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="familylinks">
|
||||||
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-200423001.html">Consumer Groups</a></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
17
docs/dms/umn/kafka-faq-0070.html
Normal file
@ -0,0 +1,17 @@
|
|||||||
|
<a name="kafka-faq-0070"></a><a name="kafka-faq-0070"></a>
|
||||||
|
|
||||||
|
<h1 class="topictitle1">Why Are the Earliest Messages Not Automatically Deleted When the Disk Capacity Reaches 95%?</h1>
|
||||||
|
<div id="body0000002327701097"><div class="section" id="kafka-faq-0070__section102111056112213"><h4 class="sectiontitle">Symptom</h4><p id="kafka-faq-0070__p7650171112319">Each broker of a Kafka instance has a 100 GB disk. The automatic deletion policy has been configured and a topic with 180 partitions has been created. The earliest messages are not automatically deleted when the disk usage reaches 95%.</p>
|
||||||
|
</div>
|
||||||
|
<div class="section" id="kafka-faq-0070__section174217226238"><h4 class="sectiontitle">Possible Cause</h4><p id="kafka-faq-0070__p5192185733510">Each broker of a Kafka instance uses a 33 GB disk to store logs and ZooKeeper data. Such a disk was formatted during instance creation and certain space is used. As a result, the data storage space is approximately 66 GB. Several segment files constitute each partition of a topic. Each segment file has maximum storage of 500 MB. To delete a message, Kafka deletes segment files instead of a message. Kafka reserves at least one segment file in each partition for message storage. In this way, at least one segment file is kept. Increasing partitions to 132 (66 GB/500 MB = 132) may cause the disk usage to reach the upper limit.</p>
|
||||||
|
<p id="kafka-faq-0070__p615213495212">When the disk usage reaches 95% (62.7 GB), the messages might be stored in one of those kept segment files. In this case, the messages are not deleted.</p>
|
||||||
|
</div>
|
||||||
|
<div class="section" id="kafka-faq-0070__section14818232152318"><h4 class="sectiontitle">Solution</h4><p id="kafka-faq-0070__p176278363237">Expand the storage. For details, see <a href="kafka-ug-181221001.html">Modifying Cluster Kafka Instance Specifications</a></p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="familylinks">
|
||||||
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-191030002.html">Instances</a></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
@ -25,7 +25,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.4 "><p id="kafka-faq-180604024__p1613812384012">9092</p>
|
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.4 "><p id="kafka-faq-180604024__p1613812384012">9092</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.5 "><p id="kafka-faq-180604024__p1313813810020">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.5 "><p id="kafka-faq-180604024__p1313813810020">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.6 "><p id="kafka-faq-180604024__p14138113816018">Accessing a Kafka instance over a private network within a VPC (in plaintext)</p>
|
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.6 "><p id="kafka-faq-180604024__p14138113816018">Accessing a Kafka instance over a private network within a VPC (in plaintext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -38,7 +38,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.4 "><p id="kafka-faq-180604024__p391712394215">9192</p>
|
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.4 "><p id="kafka-faq-180604024__p391712394215">9192</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.5 "><p id="kafka-faq-180604024__p89171039324">::/0</p>
|
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.5 "><p id="kafka-faq-180604024__p89171039324">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.6 "><p id="kafka-faq-180604024__p1544556114817">Accessing a Kafka instance within a VPC (with SSL encryption disabled)</p>
|
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.6 "><p id="kafka-faq-180604024__p1544556114817">Accessing a Kafka instance within a VPC (with SSL encryption disabled)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -51,7 +51,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.4 "><p id="kafka-faq-180604024__p1513818381009">9093</p>
|
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.4 "><p id="kafka-faq-180604024__p1513818381009">9093</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.5 "><p id="kafka-faq-180604024__p161387382010">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.5 "><p id="kafka-faq-180604024__p161387382010">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.6 "><p id="kafka-faq-180604024__p73451549155214">Accessing a Kafka instance over a private network within a VPC (in ciphertext)</p>
|
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.6 "><p id="kafka-faq-180604024__p73451549155214">Accessing a Kafka instance over a private network within a VPC (in ciphertext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -64,7 +64,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.4 "><p id="kafka-faq-180604024__p330115481727">9193</p>
|
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.4 "><p id="kafka-faq-180604024__p330115481727">9193</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.5 "><p id="kafka-faq-180604024__p03011048726">::/0</p>
|
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.5 "><p id="kafka-faq-180604024__p03011048726">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.6 "><p id="kafka-faq-180604024__p748825964817">Accessing a Kafka instance within a VPC (with SSL encryption enabled)</p>
|
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.2.2.1.2.1.1.2.2.7.1.6 "><p id="kafka-faq-180604024__p748825964817">Accessing a Kafka instance within a VPC (with SSL encryption enabled)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -155,7 +155,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="15.15%" headers="mcps1.3.3.3.2.6.1.3 "><p id="kafka-faq-180604024__p128124125311">9011</p>
|
<td class="cellrowborder" valign="top" width="15.15%" headers="mcps1.3.3.3.2.6.1.3 "><p id="kafka-faq-180604024__p128124125311">9011</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="18.029999999999998%" headers="mcps1.3.3.3.2.6.1.4 "><p id="kafka-faq-180604024__p13281114185318">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="18.029999999999998%" headers="mcps1.3.3.3.2.6.1.4 "><p id="kafka-faq-180604024__p13281114185318">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="38.46%" headers="mcps1.3.3.3.2.6.1.5 "><p id="kafka-faq-180604024__p82817475319">Accessing a Kafka instance using DNAT (in cipher- or plaintext)</p>
|
<td class="cellrowborder" valign="top" width="38.46%" headers="mcps1.3.3.3.2.6.1.5 "><p id="kafka-faq-180604024__p82817475319">Accessing a Kafka instance using DNAT (in cipher- or plaintext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -166,7 +166,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="15.15%" headers="mcps1.3.3.3.2.6.1.3 "><p id="kafka-faq-180604024__p4521518014">9092</p>
|
<td class="cellrowborder" valign="top" width="15.15%" headers="mcps1.3.3.3.2.6.1.3 "><p id="kafka-faq-180604024__p4521518014">9092</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="18.029999999999998%" headers="mcps1.3.3.3.2.6.1.4 "><p id="kafka-faq-180604024__p145219515014">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="18.029999999999998%" headers="mcps1.3.3.3.2.6.1.4 "><p id="kafka-faq-180604024__p145219515014">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="38.46%" headers="mcps1.3.3.3.2.6.1.5 "><p id="kafka-faq-180604024__p8224188618">Accessing a Kafka instance using a peering connection across VPCs (in plaintext)</p>
|
<td class="cellrowborder" valign="top" width="38.46%" headers="mcps1.3.3.3.2.6.1.5 "><p id="kafka-faq-180604024__p8224188618">Accessing a Kafka instance using a peering connection across VPCs (in plaintext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -177,7 +177,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="15.15%" headers="mcps1.3.3.3.2.6.1.3 "><p id="kafka-faq-180604024__p1258910461018">9093</p>
|
<td class="cellrowborder" valign="top" width="15.15%" headers="mcps1.3.3.3.2.6.1.3 "><p id="kafka-faq-180604024__p1258910461018">9093</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="18.029999999999998%" headers="mcps1.3.3.3.2.6.1.4 "><p id="kafka-faq-180604024__p1858913461114">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="18.029999999999998%" headers="mcps1.3.3.3.2.6.1.4 "><p id="kafka-faq-180604024__p1858913461114">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="38.46%" headers="mcps1.3.3.3.2.6.1.5 "><p id="kafka-faq-180604024__p1032716521111">Accessing a Kafka instance using a peering connection across VPCs (in ciphertext)</p>
|
<td class="cellrowborder" valign="top" width="38.46%" headers="mcps1.3.3.3.2.6.1.5 "><p id="kafka-faq-180604024__p1032716521111">Accessing a Kafka instance using a peering connection across VPCs (in ciphertext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -210,7 +210,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.4.3.2.7.1.4 "><p id="kafka-faq-180604024__p1921320288819">9094</p>
|
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.4.3.2.7.1.4 "><p id="kafka-faq-180604024__p1921320288819">9094</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.4.3.2.7.1.5 "><p id="kafka-faq-180604024__p1121316281483">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.4.3.2.7.1.5 "><p id="kafka-faq-180604024__p1121316281483">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.4.3.2.7.1.6 "><p id="kafka-faq-180604024__p151381038801">Accessing a Kafka instance over a public network (in plaintext)</p>
|
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.4.3.2.7.1.6 "><p id="kafka-faq-180604024__p151381038801">Accessing a Kafka instance over a public network (in plaintext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -223,7 +223,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.4.3.2.7.1.4 "><p id="kafka-faq-180604024__p21387381002">9095</p>
|
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.4.3.2.7.1.4 "><p id="kafka-faq-180604024__p21387381002">9095</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.4.3.2.7.1.5 "><p id="kafka-faq-180604024__p151384381409">0.0.0.0/0</p>
|
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.4.3.2.7.1.5 "><p id="kafka-faq-180604024__p151384381409">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.4.3.2.7.1.6 "><p id="kafka-faq-180604024__p1345124955213">Accessing a Kafka instance over a public network (in ciphertext)</p>
|
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.4.3.2.7.1.6 "><p id="kafka-faq-180604024__p1345124955213">Accessing a Kafka instance over a public network (in ciphertext)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -236,7 +236,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.4.3.2.7.1.4 "><p id="kafka-faq-180604024__p1220893212297">9192</p>
|
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.4.3.2.7.1.4 "><p id="kafka-faq-180604024__p1220893212297">9192</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.4.3.2.7.1.5 "><p id="kafka-faq-180604024__p730633134812">::/0</p>
|
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.4.3.2.7.1.5 "><p id="kafka-faq-180604024__p730633134812">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.4.3.2.7.1.6 "><p id="kafka-faq-180604024__p33061831124811">Accessing a Kafka instance over a public network (without SSL)</p>
|
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.4.3.2.7.1.6 "><p id="kafka-faq-180604024__p33061831124811">Accessing a Kafka instance over a public network (without SSL)</p>
|
||||||
</td>
|
</td>
|
||||||
@ -249,7 +249,7 @@
|
|||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.4.3.2.7.1.4 "><p id="kafka-faq-180604024__p772294195311">9193</p>
|
<td class="cellrowborder" valign="top" width="14.01%" headers="mcps1.3.4.3.2.7.1.4 "><p id="kafka-faq-180604024__p772294195311">9193</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.4.3.2.7.1.5 "><p id="kafka-faq-180604024__p27222049531">::/0</p>
|
<td class="cellrowborder" valign="top" width="16.669999999999998%" headers="mcps1.3.4.3.2.7.1.5 "><p id="kafka-faq-180604024__p27222049531">IP address or IP address group of the Kafka client</p>
|
||||||
</td>
|
</td>
|
||||||
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.4.3.2.7.1.6 "><p id="kafka-faq-180604024__p430120571108">Accessing a Kafka instance over a public network (with SSL)</p>
|
<td class="cellrowborder" valign="top" width="35.57%" headers="mcps1.3.4.3.2.7.1.6 "><p id="kafka-faq-180604024__p430120571108">Accessing a Kafka instance over a public network (with SSL)</p>
|
||||||
</td>
|
</td>
|
||||||
|
|||||||
@ -22,6 +22,8 @@
|
|||||||
</li>
|
</li>
|
||||||
<li class="ulchildlink"><strong><a href="kafka_faq_0058.html">How Do I Modify message.max.bytes?</a></strong><br>
|
<li class="ulchildlink"><strong><a href="kafka_faq_0058.html">How Do I Modify message.max.bytes?</a></strong><br>
|
||||||
</li>
|
</li>
|
||||||
|
<li class="ulchildlink"><strong><a href="kafka-faq-0065.html">Why Are Offsets Not Continuous?</a></strong><br>
|
||||||
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -52,6 +52,8 @@
|
|||||||
</li>
|
</li>
|
||||||
<li class="ulchildlink"><strong><a href="kafka_faq_0046.html">How Do I Bind an EIP Again?</a></strong><br>
|
<li class="ulchildlink"><strong><a href="kafka_faq_0046.html">How Do I Bind an EIP Again?</a></strong><br>
|
||||||
</li>
|
</li>
|
||||||
|
<li class="ulchildlink"><strong><a href="kafka-faq-0070.html">Why Are the Earliest Messages Not Automatically Deleted When the Disk Capacity Reaches 95%?</a></strong><br>
|
||||||
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -14,8 +14,6 @@
|
|||||||
</li>
|
</li>
|
||||||
<li class="ulchildlink"><strong><a href="kafka-faq-200426027.html">Do Kafka Instances Support Batch Importing Topics or Automatic Topic Creation?</a></strong><br>
|
<li class="ulchildlink"><strong><a href="kafka-faq-200426027.html">Do Kafka Instances Support Batch Importing Topics or Automatic Topic Creation?</a></strong><br>
|
||||||
</li>
|
</li>
|
||||||
<li class="ulchildlink"><strong><a href="kafka-faq-200426028.html">Why Do Deleted Topics Still Exist?</a></strong><br>
|
|
||||||
</li>
|
|
||||||
<li class="ulchildlink"><strong><a href="kafka-faq-200426030.html">Can I View the Disk Space Used by a Topic?</a></strong><br>
|
<li class="ulchildlink"><strong><a href="kafka-faq-200426030.html">Can I View the Disk Space Used by a Topic?</a></strong><br>
|
||||||
</li>
|
</li>
|
||||||
<li class="ulchildlink"><strong><a href="kafka-faq-200426032.html">Can I Add ACL Permissions for Topics?</a></strong><br>
|
<li class="ulchildlink"><strong><a href="kafka-faq-200426032.html">Can I Add ACL Permissions for Topics?</a></strong><br>
|
||||||
@ -28,6 +26,8 @@
|
|||||||
</li>
|
</li>
|
||||||
<li class="ulchildlink"><strong><a href="kafka-faq-0038.html">What Can I Do If a Consumer Fails to Retrieve Messages from a Topic Due to Insufficient Permissions?</a></strong><br>
|
<li class="ulchildlink"><strong><a href="kafka-faq-0038.html">What Can I Do If a Consumer Fails to Retrieve Messages from a Topic Due to Insufficient Permissions?</a></strong><br>
|
||||||
</li>
|
</li>
|
||||||
|
<li class="ulchildlink"><strong><a href="kafka-faq-0068.html">Can I Purchase Partitions Separately?</a></strong><br>
|
||||||
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -10,6 +10,14 @@
|
|||||||
</li>
|
</li>
|
||||||
<li class="ulchildlink"><strong><a href="kafka-faq-0022.html">Why Is a Consumer Group Still on the Monitoring Page After Being Deleted?</a></strong><br>
|
<li class="ulchildlink"><strong><a href="kafka-faq-0022.html">Why Is a Consumer Group Still on the Monitoring Page After Being Deleted?</a></strong><br>
|
||||||
</li>
|
</li>
|
||||||
|
<li class="ulchildlink"><strong><a href="kafka-faq-0063.html">Why Do Metrics Fluctuate Significantly (Disk Read/Write Speed, Average Disk Read/Write Time, and CPU Usage)?</a></strong><br>
|
||||||
|
</li>
|
||||||
|
<li class="ulchildlink"><strong><a href="kafka-faq-0064.html">Why Does JVM Heap Memory Usage of JVM Fluctuate Significantly?</a></strong><br>
|
||||||
|
</li>
|
||||||
|
<li class="ulchildlink"><strong><a href="kafka-faq-0066.html">Why Is Production Rate Still 0 When There Are Produced Messages?</a></strong><br>
|
||||||
|
</li>
|
||||||
|
<li class="ulchildlink"><strong><a href="kafka-faq-0067.html">Why Is the Number of Stacked Messages Monitored as 0 when Messages Are Stacked?</a></strong><br>
|
||||||
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -14,6 +14,8 @@
|
|||||||
</li>
|
</li>
|
||||||
<li class="ulchildlink"><strong><a href="kafka-faq-0031.html">Can I Delete Unnecessary Topics in a Consumer Group?</a></strong><br>
|
<li class="ulchildlink"><strong><a href="kafka-faq-0031.html">Can I Delete Unnecessary Topics in a Consumer Group?</a></strong><br>
|
||||||
</li>
|
</li>
|
||||||
|
<li class="ulchildlink"><strong><a href="kafka-faq-0069.html">Why Are Consumer Groups Missing?</a></strong><br>
|
||||||
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -1,7 +1,7 @@
|
|||||||
<a name="kafka-faq-200426003"></a><a name="kafka-faq-200426003"></a>
|
<a name="kafka-faq-200426003"></a><a name="kafka-faq-200426003"></a>
|
||||||
|
|
||||||
<h1 class="topictitle1">Why Can't I View the Subnet and Security Group Information When Creating a DMS Instance?</h1>
|
<h1 class="topictitle1">Why Can't I View the Subnet and Security Group Information When Creating a DMS Instance?</h1>
|
||||||
<div id="body1587865149955"><p id="kafka-faq-200426003__p851222919813">This may be because you do not have the <strong id="kafka-faq-200426003__b9940835155615">Server Administrator</strong> and <strong id="kafka-faq-200426003__b394163585613">VPC Administrator</strong> permissions. For details about how to add permissions to a user group, see <a href="https://docs.otc.t-systems.com/en-us/usermanual/iam/en-us_topic_0085605493.html" target="_blank" rel="noopener noreferrer">Viewing and Modifying User Group Information</a>.</p>
|
<div id="body1587865149955"><p id="kafka-faq-200426003__p851222919813">The possible cause is that you do not have the <strong id="kafka-faq-200426003__b9940835155615">Server Administrator</strong> and <strong id="kafka-faq-200426003__b394163585613">VPC Administrator</strong> permissions. For details about how to add permissions to a user group, see <a href="https://docs.otc.t-systems.com/en-us/usermanual/iam/en-us_topic_0085605493.html" target="_blank" rel="noopener noreferrer">Viewing and Modifying User Group Information</a>.</p>
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -2,8 +2,8 @@
|
|||||||
|
|
||||||
<h1 class="topictitle1">How Do I Select Storage Space for a Kafka Instance?</h1>
|
<h1 class="topictitle1">How Do I Select Storage Space for a Kafka Instance?</h1>
|
||||||
<div id="body1587865149955"><p id="kafka-faq-200426005__p98934012288">The storage space is the space for storing messages (including messages in replicas), logs and metadata. To select a storage space, specify the disk type and disk size. For more information, see <a href="https://docs.otc.t-systems.com/en-us/usermanual/evs/en-us_topic_0014580744.html" target="_blank" rel="noopener noreferrer">Disk Types and Performance</a>.</p>
|
<div id="body1587865149955"><p id="kafka-faq-200426005__p98934012288">The storage space is the space for storing messages (including messages in replicas), logs and metadata. To select a storage space, specify the disk type and disk size. For more information, see <a href="https://docs.otc.t-systems.com/en-us/usermanual/evs/en-us_topic_0014580744.html" target="_blank" rel="noopener noreferrer">Disk Types and Performance</a>.</p>
|
||||||
<p id="kafka-faq-200426005__p1966572819387">For example, if the required disk size to store data for the retention period is 100 GB, the disk capacity must be at least: <strong id="kafka-faq-200426005__b1796211590013">100 GB x Number of replicas + 100 GB (reserved space)</strong>. In a Kafka cluster, each node uses a 33 GB disk to store logs and ZooKeeper data. Therefore, the actual available storage space is less than the created storage space.</p>
|
<p id="kafka-faq-200426005__p1966572819387">For example, if the required disk size to store data for the retention period is 100 GB, the disk capacity must be at least: <strong id="kafka-faq-200426005__b1796211590013">100 GB × Number of replicas + 100 GB (reserved space)</strong>. In a Kafka cluster, each node uses a 33 GB disk to store logs and ZooKeeper data. Therefore, the actual available storage space is less than the created storage space.</p>
|
||||||
<p id="kafka-faq-200426005__p9278193231820">The number of replicas (3 by default) can be configured when you create a topic. If automatic topic creation has been enabled, each automatically created topic has three replicas by default. You can change this quantity by setting <strong id="kafka-faq-200426005__b165487220106">default.replication.factor</strong> on the <strong id="kafka-faq-200426005__b123410264103">Parameters</strong> tab page.</p>
|
<p id="kafka-faq-200426005__p9278193231820">The number of replicas (3 by default) can be configured when you create a topic. If automatic topic creation has been enabled, each automatically created topic has three replicas by default. You can change this quantity by setting <strong id="kafka-faq-200426005__b165487220106">default.replication.factor</strong> on the <strong id="kafka-faq-200426005__b123410264103">Parameters</strong> page.</p>
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||
@ -3,9 +3,9 @@
|
|||||||
<h1 class="topictitle1">Which Capacity Threshold Policy Should I Use?</h1>
|
<h1 class="topictitle1">Which Capacity Threshold Policy Should I Use?</h1>
|
||||||
<div id="body1587865149955"><p id="kafka-faq-200426007__p13694182919718">The following policies are supported:</p>
|
<div id="body1587865149955"><p id="kafka-faq-200426007__p13694182919718">The following policies are supported:</p>
|
||||||
<ul id="kafka-faq-200426007__ul128491101810"><li id="kafka-faq-200426007__li88491103817">Stop production<p id="kafka-faq-200426007__p42290210814"><a name="kafka-faq-200426007__li88491103817"></a><a name="li88491103817"></a>When the memory usage reaches the disk capacity threshold (95%), new messages will no longer be created, but existing messages can still be retrieved until they are discarded. The default retention time is three days. This policy is suitable for scenarios where no data losses can be tolerated.</p>
|
<ul id="kafka-faq-200426007__ul128491101810"><li id="kafka-faq-200426007__li88491103817">Stop production<p id="kafka-faq-200426007__p42290210814"><a name="kafka-faq-200426007__li88491103817"></a><a name="li88491103817"></a>When the memory usage reaches the disk capacity threshold (95%), new messages will no longer be created, but existing messages can still be retrieved until they are discarded. The default retention time is three days. This policy is suitable for scenarios where no data losses can be tolerated.</p>
|
||||||
</li><li id="kafka-faq-200426007__li88491910284">Automatically delete<p id="kafka-faq-200426007__p132759343816"><a name="kafka-faq-200426007__li88491910284"></a><a name="li88491910284"></a>When the memory usage reaches the disk capacity threshold (95%), messages can be created and retrieved, but 10% of the earliest messages will be deleted to ensure sufficient disk space. This policy is suitable for scenarios where no service interruption can be tolerated. Data may be lost.</p>
|
</li><li id="kafka-faq-200426007__li88491910284">Automatically delete<p id="kafka-faq-200426007__p132759343816"><a name="kafka-faq-200426007__li88491910284"></a><a name="li88491910284"></a>When the memory usage reaches the disk capacity threshold (95%), messages can be produced and consumed, but 10% of the earliest messages will be deleted to ensure sufficient disk space. This policy is suitable for scenarios where no service interruption can be tolerated. Data may be lost.</p>
|
||||||
</li></ul>
|
</li></ul>
|
||||||
<p id="kafka-faq-200426007__p195747551283">Select a proper policy based on requirements for data and service reliability. Both policies are only used for handling extreme scenarios. <strong id="kafka-faq-200426007__b1451116511152">To avoid extreme scenarios, create sufficient disk space in the first place.</strong></p>
|
<p id="kafka-faq-200426007__p195747551283">Select a proper policy as required for data and service reliability. Both policies are only used for handling extreme scenarios. <strong id="kafka-faq-200426007__b1451116511152">To avoid extreme scenarios, create sufficient disk space in the first place.</strong></p>
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<div class="familylinks">
|
<div class="familylinks">
|
||||||
|
|||||||