Scaling LLMs and Accelerating Adoption: Interview with Aidan Gomez

On this episode, we’re joined by Aidan Gomez, Co-Founder and CEO at Cohere. Cohere develops and releases a range of innovative AI-powered tools and solutions for a variety of NLP use cases.

Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.

Chapters:
0:00 Intro
1:20 Aidan’s role in the “Attention Is All You Need” paper
10:52 What SSMs are & how they could be an alternative to transformers
14:30 What it means for an ML architecture to saturate compute
21:36 Data constraints for when LLMs scale
27:00 Challenges of measuring LLM performance
36:08 How Cohere is positioned within the LLM development space
45:00 Scaling down an LLM into a more domain-specific one
50:08 Concerns around synthetic content & AI changing public discourse
57:17 The importance of raising money at healthy milestones for AI development

Resources:
https://cohere.ai/
https://research.google/pubs/pub46201/

#OCR #DeepLearning #AI #Modeling #ML