Personalized Recommendations and Search with Retrieval and Ranking at scale on Hopsworks

Speaker:
Jim Dowling, CEO, Hopsworks
Jim Dowling is CEO of Logical Clocks and an Associate Professor at KTH Royal Institute of Technology. He is lead architect of the open-source Hopsworks platform, a horizontally scalable data platform for machine learning that includes the industry’s first Feature Store.

Abstract:
Personalized recommendations and personalized search systems at scale are increasingly being built on retrieval and ranking architectures based on the two-tower embedding model. This architecture requires a lot of infrastructure. A single user query will cause a large fanout of traffic to the backend, with hundreds of database lookups in a feature store, similarity search in an embedding store, and model outputs from both a query embedding model and a ranking model. You will also need to index your items in the embedding store using an item embedding model, and instrument your existing systems to store observations of user queries and the items they select.

In this workshop, we will introduce the retrieval and ranking architecture based on the two-tower recommendation model, and we will walk through the implementation of a personalized recommendations service on the open-source Hopsworks platform. We will introduce first the offline infrastructure needed to train your models, index your items in an embedding store, and update your feature store. We will then walk through the online infrastructure needed to retrieve candidates with similarity search using an embedding store and ranking candidates with both a ranking model and features from the feature store. We will show you how you can keep end-to-end latencies below 100ms and ensure all components of the system are highly available.