Скачать или смотреть Scalable LLM Inference on Kubernetes With NVIDIA NIMS, LangChain, Milvus and Flu... Riccardo Freschi

Don't miss out! Join us at the next Open Source Summit in Seoul, South Korea (November 4-5). Join us at the premier vendor-neutral open source conference, where developers and technologists come together to collaborate, share knowledge, and explore the latest innovations and advancements in open source technology. Learn more at https://events.linuxfoundation.org/

Scalable LLM Inference on Kubernetes With NVIDIA NIMS, LangChain, Milvus and FluxCD - Riccardo Freschi, AWS

Join us for a deep dive into architecting and implementing a scalable LLM inference service on Amazon EKS as the foundation for workload orchestrating, while incorporating NVIDIA NIMS for optimal GPU utilization, LangChain for flexible LLM operations, Milvus for efficient vector storage, FluxCD for GitOps-driven deployments, Karpenter for horizontal scaling and Prometheus and Grafana for Observability.