LLaMA3 400B to beat GPT4? (& more) | Trends in AI - May 2024

In this month’s edition, we discuss the burning question: how good is LLaMA 3 really? Plus all the latest news and releases, like Phi-3, Reka Core & Snowflake Arctic, the new Atlas by Boston Dynamics, and developments in self-driving cars from Wayve and Mobileye. Are the famous Chinchilla scaling laws burnt toast? And as usual, we dive deeper into the most trending AI R&D papers of the month, including LLM2Vec, PromptReps, Multi-token Prediction, Replacing Judges with Juries, LongEmbed, Infini-attention, Megalodon, and much more, including an ICLR 2024 preview.

Dissecting the current Trends in AI: News, R&D breakthroughs, trending papers and code, and the latest gossip. Live talk show from LAB42 with the Zeta Alpha crew, and online on Zoom.

Dive deeper into the papers we covered in this episode: https://search.zeta-alpha.com/tags/80501

Sign up for the series, and catch us live on the next edition! https://us06web.zoom.us/webinar/regis...

Timestamps:
0:00 Intro by Jakub Zavrel and Dinos Papakostas
1:24 News from Big Tech
7:21 AI is powering the next wave of Browser Wars
10:18 AI Hardware News
12:28 AI Startups & Funding
15:14 AI for Robotics & Self-Driving Cars
21:07 LLaMA 3, the new open-weights king
26:51 All the latest model releases
32:19 Updates in the Chatbot Arena
33:58 Embeddings, Vector Search & RAG
37:05 Dutch AI News
38:00 Zeta Alpha: AI Discovery Platform
41:38 ICLR 2024
45:02 LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
48:04 PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval
50:54 DUQGen: Effective Unsupervised Domain Adaptation of Neural Rankers by Diversifying Synthetic Query Generation
53:01 Better & Faster Large Language Models via Multi-token Prediction
55:33 Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
58:16 Many-Shot In-Context Learning
59:35 Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
1:00:31 Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
1:00:53 LongEmbed: Extending Embedding Models for Long Context Retrieval
1:01:25 The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
1:02:55 Chinchilla Scaling: A replication attempt
1:04:16 Long-form music generation with latent diffusion
1:05:12 What's next & Outro