Self hosted vs Managed LLMs: learnings from Freshworks by Sundeep

Strategies for Hosting Large Language Models (LLMs) and AI Features by Sundeep Paluru ‪@FreshworksInc‬ | Webinar by ‪@DrDroidDev‬ ‪@portkeyai‬

Description:

In this webinar recording from August 29th, hosted by Doctor Droid, we feature a speaker from Freshworks who dives into the AI features recently launched at Freshworks and the strategies behind hosting large language models (LLMs). The discussion covers the balance between using self-managed vs. managed LLMs, cost comparisons, and cutting-edge techniques for optimizing AI performance.

Key Topics Discussed:

AI Features at Freshworks: Overview of the AI-powered solutions recently launched by Freshworks.
Self-Managed vs. Managed LLMs: Pros and cons of each approach, including OpenAI, Azure OpenAI, and Bedrock.
Cost vs. Performance Analysis: Learn the economic implications of hosting LLMs in production, with real-world examples comparing models like Llama 2 and GPT-3.5 Turbo.
Advanced Hosting Techniques: Explore fractional GPUs, model composition, and model multiplexing to optimize the performance and reliability of LLMs.
Agent-Based AI Solutions: Insights into agentic platforms and their growing role in AI systems.

Don't forget to like, subscribe, and leave your questions in the comments!