Скачать или смотреть Scale By The Bay 2018: Jie Ren, Greg Methvin, Using Akka Streams for Web scale Data Ingestion

ai.bythebay.io Nov 2025, Oakland, full-stack AI conference Scale By the Bay 2019 is held on November 13-15 in sunny Oakland, California, on the shores of Lake Merritt: https://scale.bythebay.io. Join us!
-----

Iterable uses Akka Streams to manage streams of hundreds of millions of events daily for some of the world's largest consumer companies. In this talk we'll discuss how we use Akka Streams to ingest streams of event data into Elasticsearch using Kafka. We'll show how Akka Streams makes writing streaming-related code more elegant, flexible, and performant. We'll discuss the various stream operations such as `mapAsync` and `groupedWithin`. We'll look at how Akka Streams deals with common streaming data issues such as idempotency, ordering of updates, and backpressure. Finally, we'll discuss practical issues such as measuring performance, testing, and tuning an Akka Streams-based system in the context of massive data ingestion into a distributed, eventually consistent database.