Скачать или смотреть vLLM Office Hours - vLLM’s 2024 Wrapped and 2025 Vision - December 19, 2024

In this session, we wrapped up 2024 with a comprehensive update on the vLLM project and shared exciting plans for 2025. Michael Goin, vLLM Committer, walked us through the latest updates in vLLM v0.6.5, including performant structured outputs, while Simon Mo, vLLM Maintainer, shared key insights from vLLM’s 2024 journey and the roadmap for 2025.

Highlights:

[00:00-02:45] A recap of 2024 vLLM Office Hours by the numbers
[02:46-09:03] About vLLM & Neural Magic
[09:04-15:58] What’s new in vLLM v0.6.5, including performant structured outputs
[15:59-25:59] vLLM’s 2024 milestones and achievements
[26:00-35:55] vLLM 2025 roadmap, including upcoming features and improvements
[35:56-56:03] Open discussion and Q&A

Audience Q&A included discussions on:

-Support for older GPU architectures like Pascal and V100s
-OpenAI API-compliance for tool calling in vLLM
-Deployment recipes for production-ready solutions
-Structured outputs and their role in vLLM’s evolution
-And more...

URL Pointers:
View the slides: https://docs.google.com/presentation/...
Register for upcoming vLLM Office Hours: https://hubs.li/Q02Y5Pbh0