When ordering matters - Niels Basjes

In the last decades many systems have been used that were described as "queues" (AQ, ActiveMQ, RabbitMQ, etc.), yet from a computer science perspective these are not queues at all. Many of us have learned to work quite effectively with these messaging systems and we all understand that we cannot expect to receive the messages in any particular order and that we get all messages exactly once (which we can expect with a queue). With the arrival of Kafka and Flink a new class of applications became possible. In this talk I will go into several real applications from the bol.com context that all revolve around low latency behavioral analytics. I will talk about the entire end-to-end pipeline from the webbrowser and application server to application and discuss many of the things to think about when creating your analysis application. I will also touch upon using state machines as a way of doing this type of behavioral analysis using very simple software and show example algorithms from our context.