LLMOps (LLM Bootcamp)

New course announcement ✨

We're teaching an in-person LLM bootcamp in the SF Bay Area on November 14, 2023. Come join us if you want to see the most up-to-date materials building LLM-powered products and learn in a hands-on environment.

https://www.scale.bythebay.io/llm-wor...

Hope to see some of you there!

--------------------------------------------------------------------------------------------- In this video, Josh gives a tour of the emerging discipline of LLMOps: principles and practices for continuous improvement of large language model-powered applications.

- Comparing and evaluating open source and proprietary models
- Workflows and tools for iteration and prompt management
- Principles for applying test-driven development to LLMs

Download slides and view lecture notes: https://fullstackdeeplearning.com/llm...

Intro and outro music made with Riffusion: https://github.com/riffusion/riffusion

Watch the rest of the LLM Bootcamp videos here:    • LLM Bootcamp - Spring 2023  

00:00 Why LLMOps?
01:55 Choosing your base LLM
04:20 Proprietary LLMs
09:15 Open-source LLMs
14:45 Iteration and prompt management
22:35 Testing LLMs: Why and why is it hard?
28:15 Testing LLMs: What works?
35:05 Evaluation metrics for LLMs
37:40 Deployment and monitoring
44:35 Test-driven development for LLMs