Generalized Contrastive Learning and Transforming Video Production | Multimodal Weekly 50

​​​In the 50th session of Multimodal Weekly, we have two exciting presentations from startup founders building real-world products for Multimodal AI applications.

​​​​✅ Jesse Clark, the Co-Founder and CTO of Marqo AI, will discuss generalized contrastive learning for multimodal retrieval and ranking. They generalize the popular training method of CLIP to accommodate any number of text and images when representing documents and also encode relevance (or rank) to provide better first stage retrieval.
Follow Jesse:   / jessenclark  

Check out the following resources on Marqo AI:​
​Website: https://www.marqo.ai/
​Article on Generalized Contrastive Learning: https://www.marqo.ai/blog/generalized...
​GitHub: https://github.com/marqo-ai/marqo
​Community: https://community.marqo.ai/

​​​​​​​​✅ Alexandre Berkovic, the Co-Founder and CEO of Adorno AI, will dive into how video and audio understanding technologies from Twelve Labs and Adorno AI are transforming video production.
Follow Alex:   / alexandreberkovic  

Check out the following resources on Adorno AI:
​Website: https://adorno.ai/
​Discord:   / discord  

Timestamps:
00:10 Introduction
03:14 Jesse starts
03:47 About Jesse
04:07 Talk outline
05:05 What is vector search?
06:50 Marqo vector search
07:45 Vector search use cases
09:01 Some examples of vector search
11:30 Generalized contrastive learning
12:02 How are models trained in OpenAI's CLIP
15:38 How are models trained in Marqo's Generalized Contrastive Learning
16:57 Marqo-GS10M dataset with 100k unique queries and ~5M unique products
20:10 Matryoshka embeddings via truncation
23:23 Binary embeddings via truncation and binarization
25:16 Conclusions
26:25 Q&A with Jesse
31:10 Alex starts
31:30 The power of sound design
33:12 Post-production is essential, but slow
34:25 Multimodality in the creative AI landscape
35:35 Adorno AI is an accessible one-stop solution to perform sound design
36:20 Live demo of Adorno AI
38:19 Adorno's technical backbone: from video understanding to content-aware audio
42:18 Use cases for Twelve Labs and Adorno AI
44:05 Q&A with Alex
53:35 Conclusion

Join the Multimodal Minds community to receive an invite for future webinars:   / discord