Скачать или смотреть Ved Prakash – Siphon : Modern Data Stack with SF-CH & Iceberg

More: https://2025.berlinbuzzwords.de/sessi...

Speaker: Ved Prakash

Tired of waiting for batch jobs? See how we transformed our data pipeline using Apache Iceberg to stream quality data into Snowflake and Clickhouse simultaneously. Learn about our battle-tested architecture, performance gains, and how we maintain data consistency across dual analytics engines

Ever wondered how to stream data reliably to multiple warehouses without compromising data quality? We'll show you how Siphon uses Apache Iceberg's time travel and ACID properties to ensure data consistency across Snowflake and Clickhouse. Dive into our journey from batch to streaming - covering architecture evolution, data quality frameworks, and performance optimizations. We'll share our battle-tested patterns for handling schema evolution, managing data contracts, and implementing quality gates. Learn how we achieved sub-minute latency while preventing bad data from corrupting our warehouses. Perfect for data engineers and architects looking to modernize their data infrastructure with real-world proven solutions.

###

Follow us on Social Media and join the Community!

Mastodon: https://floss.social/@BerlinBuzzwords
LinkedIn:   / berlin-buzzwords  
Website: https://berlinbuzzwords.de
Mail: [email protected]

Berlin Buzzwords is an event by Plain Schwarz – https://plainschwarz.com