Building a Real Time Metrics Database at Datadog

In the course of its eight years of existence, Datadog has grown its real time metrics systems that collect, process, and visualize data to the point they now handle trillions of points per day. This has been based on an architecture combining open source technologies, such as Apache Cassandra, Kafka, and PostgreSQL, with a lot of in-house software for in-memory data storing and querying. This architecture serves to efficiently compute distribution metrics. In this talk, Director of Distribution Metrics Joel Barciauskas speaks to the challenges Datadog faces, how the architecture has evolved to cope with them, and what Datadog is looking to in the future as they architect for a quadrillion points per day.