SPLADE: the first search model to beat BM25

AI-powered search is heating up, but it's nothing new. Google, Netflix, Amazon, and many more big tech companies have all powered their search and recommendation systems with “vector search”.

In this video, we'll talk about sparse and dense vector search, their pros and cons, and how the latest sparse embedding model called SPLADE can help us eliminate many of the downsides of traditional sparse embedding methods like TF-IDF and BM25.

SPLADE can be used as an alternative to dense embedding models like OpenAI's text-embedding-ada-002, Cohere's embed endpoint, or sentence transformers. But more interestingly, it can be used alongside these dense embedding models to give us the best of both worlds.

🌲 Pinecone article:
https://pinecone.io/learn/splade

🎙️ AI Dev Studio:
https://aurelio.ai/

👾 Discord:
  / discord  

🎉 Subscribe for Article and Video Updates!
  / subscribe  
  / membership  

00:00 Sparse and dense vector search
00:44 Comparing sparse vs. dense vectors
03:59 Using sparse and dense together
06:46 What is SPLADE?
09:06 Vocabulary mismatch problem
09:51 How SPLADE works (transformers 101)
14:28 Masked language modeling (MLM)
15:57 How SPLADE builds embeddings with MLM
17:35 Where SPLADE doesn't work so well
20:14 Implementing SPLADE in Python
20:38 SPLADE with PyTorch and Hugging Face
24:08 Using the Naver SPLADE library
27:11 What's next for vector search?