Скачать или смотреть SLM in Action: Local Inference with Arcee Nova 72B and Ollama

In this video, you will learn about Arcee-Nova, a new state-of-the-art 72-billion parameter model created by Arcee.ai.

⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at   / julsimon   or Substack at https://julsimon.substack.com. ⭐️⭐️⭐️

At the time of recording, Nova sits at the #1 spot in the Hugging Face Open LLM leaderboard, outperforming all other open-source models out there.

The model is a merge of Qwen2-72B with additional training. It was built on Arcee Cloud and you can learn more at https://www.arcee.ai/product/arceecloud.

Here, I run a 3-bit quantized version on my M3 MacBook and ollama, with a couple of creative writing and multi-step reasoning prompts. Excellent results and very decent speed for a model this large!

00:00 Introduction
00:25 Introducing Arcee-Nova
03:10 Running Arcee-Nova locally with ollama
05:25 Trying a few simple questions
07:25 Looking at the MUSR benchmark (Multi-Step Reasoning)
09:15 Trying a sample question from the MUSR dataset
11:50 Conclusion: try out the Arcee.ai platform!

Blog post: https://blog.arcee.ai/introducing-arc...
Model page: https://huggingface.co/arcee-ai/Arcee...
Ollama: https://github.com/ollama/ollama

Merging the shards into a single GGUF file:
llama.cpp/llama-gguf-split --merge Arcee-Nova-Alpha-GGUF.Q3_K_S-00001-of-00008.gguf Arcee-Nova-Alpha-GGUF.Q3_K_S.gguf

Configuration file for ollama:
FROM ./Arcee-Nova-Alpha-GGUF.Q3_K_S.gguf

#ai #slm #llm #openai #chatgpt #opensource #huggingface

Sign up for Arcee Cloud at https://www.arcee.ai, and please follow Arcee.ai on LinkedIn to stay on top of the latest Small Language Model action!   / 99895334