Скачать или смотреть Log Ingester System and Query Interface | System Design | Demo | Code Explanation

💻 Github Repo: github.com/biswajit-k/log-ingester-elasticsearch

In this video we will see how to design and code a log ingester and query interface system that can handle large volume of log insertions and provide a query interface to search these logs in real-time.
The software is designed to work as a distributed system with client-facing server for logs ingestion(using FastAPI) and a UI frontend for searching logs(using Flask). Kafka brokers are used to asynchronously handle high volume of logs sent by the ingester server. Kafka-Connect is used to consume logs from Kafka topics, serialize the data and send for storage. Elasticsearch is used as the database for quick indexing and full-text search capabilities.

🔵 Linkedin: https://linkedin.com/in/biswajit-kaushik
📁 GitHub: https://github.com/biswajit-k
🐦 Twitter: https://twitter.com/speedo_sorted

🎥 Timestamps:
00:00 Introduction
00:51 Log Ingester System
03:14 System Design
07:28 Demo
17:13 Code