Скачать или смотреть Kafka Consumer using Python & Concept of Offset-Commit

In this video, Kafka Consumer using Python & the concept of Offset-Commit is explained in detail.


Prerequisite:
-------------------------
Consumer & Consumer Group in Kafka
   • Consumer & Consumer Group in Kafka  
Kafka Consumer Groups CLI Demo | Kafka-Python
   • Kafka Consumer Groups CLI Demo | Kafka-Python  

Producer Code:
------------------------------------
from time import sleep
from json import dumps
from kafka import KafkaProducer

topic_name='hello_world4'
producer = KafkaProducer(bootstrap_servers=['localhost:9092'],value_serializer=lambda x: dumps(x).encode('utf-8'))

for e in range(1000):
data = {'number' : e}
print(data)
producer.send(topic_name, value=data)
sleep(2)

Consumer Code:
---------------------------------------
from kafka import KafkaConsumer


import json


consumer = KafkaConsumer ('hello_world4',bootstrap_servers = ['localhost:9092'],
value_deserializer=lambda m: json.loads(m.decode('utf-8')),group_id='demo112215sgtrjwrykvjh',auto_offset_reset='earliest')


for message in consumer:
print(message)


Check this playlist for more Data Engineering related videos:
   • Demystifying Data Engineering with Cloud C...  

Apache Kafka form scratch
   • Apache Kafka for Python Developers  

Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation--
https://doc.clickup.com/37466271/d/h/...

🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY YOUTUBE CHANNEL