Скачать или смотреть [Best LLM] Coding Assistant Challenge - Part 2: Multi-prompts, Jul 2024 Update

🚀 Coding LLM Showdown Part 2: Multi-Prompt Challenge! 🏆

In this part 2 of our best coding LLM video, we've compared top paid models on their performance with multiple prompts to create an application. Our goal: build a local solution using a scaling out approach.

📝 Our 3-step prompting technique:
1️⃣ Prompt 1: Build a text-based chatbot with no UI
2️⃣ Prompt 2: Add a ChatGPT-like UI with Streamlit
3️⃣ Prompt 3: Turn this app into a local RAG solution using LangChain

🧪 Evaluation method: We give each model three debugging turns for each prompt.

⏱️ This is a long video, so please use the timestamps for the parts you're interested in. If you're just after the results: Claude 3.5 Sonnet takes the crown as the best coding LLM, slightly outperforming GPT-4o.

Timestamps:
0:00 Prompt 1 Intro
0:50 Prompt 1: gpt-4o - failure
7:50 Prompt 1: gpt-3.5-turbo - failure
15:22 Prompt 1: gemini 1.5 flash - failure
19:13 Prompt 1: gemini 1.5 Pro - success
23:37 Prompt 1: Claude 3.5 Sonnet - success
25:52 Prompt 2 Intro
26:25 Prompt 2: gpt-3.5-turbo - failure
37:38 Prompt 2: gpt-4o - success
41:25 Prompt 2: gemini 1.5 flash - success
43:58 Prompt 2: gemini 1.5 Pro - success
45:36 Prompt 2: Claude 3.5 Sonnet - success
47:05 Prompt 3 Intro
48:08 Prompt 3: gpt-3.5-turbo - failure
51:04 Prompt 3: gpt-4o - success
01:05:23 Prompt 3: gemini 1.5 flash - failure
01:10:57 Prompt 3: gemini 1.5 Pro - failure
01:19:34 Prompt 3: Claude 3.5 Sonnet - success
01:33:33 Summary

#codingassistance #gpt4o #claude3 #gemini #AIChallenge #MachineLearning #RAGSolution #Streamlit #LangChain