Kyutai's Moshi and Claude 3.5 challenge GPT-4o (& much more) | Trends in AI - July 2024

Anthropic launched Claude 3.5, undercutting GPT-4o’s price with competitive performance, and Google’s Gemma 2 - 27B emerges as the new strongest ‘somewhat open’ model. ARC-AGI has put a $1M bounty for solving a deceptively simple task where current AI falls short. AI Startup Etched scores $120M to develop a Transformer-specific ASIC, and CuspAI, a new European startup secures millions for carbon capture technology. In California discussion is heating up about SB 1047, the state’s newly proposed AI regulation Bill, as AI products from Apple and others are delayed in Europe in a staredown around the new legislation. Plus an overview of SIGIR, ICML, and a deep dive into trending research papers like Prompts as Auto-Optimized Hyperparameters, TextGrad, Judging the Judges, Mixture-of-Agents, MatMul-Free Language Modeling, and more!

Dissecting the current Trends in AI: News, R&D breakthroughs, trending papers and code, and the latest gossip. Live talk show from LAB42 with the Zeta Alpha crew, and online on Zoom.

Dive deeper into the papers we covered in this episode: https://search.zeta-alpha.com/tags/81650

Sign up for the series, and catch us live on the next edition! https://us06web.zoom.us/webinar/regis...

Timestamps:
0:00 Intro by Jakub Zavrel and Dinos Papakostas
1:07 Kyutai releases the first open-source AI voice assistant
3:48 OpenAI slams the door on China
5:16 Apple & NVIDIA riding the AI wave
8:30 The ARC-AGI $1M prize
10:10 Etched bets on a Transformer-specific ASIC
11:46 AI startups & funding
15:35 AI lawsuits & regulators
19:46 Local AI news
22:00 Model releases: Claude Sonnet 3.5
23:48 Open-source model releases
27:38 Updates in the chatbot arena
29:29 Cool AI code repositories
31:45 Upcoming AI conferences
34:13 Zeta Alpha: Neural discovery platform
34:50 The top-10 research papers of July 2024
35:55 Prompts as Auto-Optimized Training Hyperparameters: Training Best-in-Class IR Models from Scratch with 10 Gold Labels
40:46 TextGrad: Automatic "Differentiation" via Text
44:07 Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
49:58 Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?
53:32 Mixture-of-Agents Enhances Large Language Model Capabilities
56:45 Language Modeling with Editable External Knowledge
57:45 Scalable MatMul-free Language Modeling
58:46 Transformers meet Neural Algorithmic Reasoners
59:09 OpenVLA: An Open-Source Vision-Language-Action Model
59:39 Simulating 500 million years of evolution with a language model
1:01:19 Weekend read, what's next, & outro