Скачать или смотреть The Performance of Kafka's New Consumer Rebalance Protocol (D. Jacot & J. Kim, Confluent) Current 24

More: https://current.confluent.io | KIP-848 (https://cwiki.apache.org/confluence/d...) proposed a new paradigm of the rebalance protocol for Kafka consumers. This prompted a large architectural change in the group coordinator, primarily moving to an event-driven architecture.

My co-speaker and I have been working on the KIP-848 implementation for some time, and we have started testing the performance of the new architecture.

At this session, we will highlight the major internal design changes in the group coordinator and the rationalization behind these decisions.

Next, we will discuss the primary load dimensions we wanted to explore as part of the performance testing. We will also describe use cases that KIP-848 proposed to improve upon, such as the time to complete a rebalance.

Lastly, We will share the intermediary and final results. We will discuss the findings from the intermediary results and the improvements we have made to confirm that KIP-848 brings no regressions and instead shows substantial improvements when compared against the old group coordinator.

If time allows, we will share how we have tested the correctness of the new architecture.

LEARN MORE
► Confluent Developer: https://developer.confluent.io

CONNECT
Subscribe on YouTube:    / @confluentdeveloper  
Community Slack: confluentcommunity.slack.com
X: https://x.com/confluentinc
Linkedin:   / confluent  
GitHub: https://github.com/confluentinc
Site: https://developer.confluent.io

ABOUT CONFLUENT DEVELOPER
Confluent Developer provides comprehensive resources for developers looking to learn about Apache Kafka®, Apache Flink®, Confluent Cloud, Confluent Platform, and any other technology related to the broader Data Streaming Platform. Content on Confluent Developer includes courses, getting started guides, topical deep-dives, patterns, tutorials, and listings of community events. Learn more at https://developer.confluent.io.

#apachekafka #apacheflink #confluent