Скачать или смотреть How to Host and Run LLMs Locally with Ollama & llama.cpp

In this tutorial I show you how you can run and host your own LLMs locally on your pc with Ollama which is a wrapper around llama.cpp. I guide you through the Ollama installation and show you how you can pull models from the Ollama registry. After this I demonstrate how you can import custom finetunes and quanitzations of models into Ollama. Finally I will explain the python API of Ollama and teach you how you can interact with your locally hosted models through the OpenAI python package and the Ollama python package. This way you can easily build genai applications without having to spend any money on OpenAI tokens.

Ollama: https://ollama.com/
llama.cpp: https://github.com/ggerganov/llama.cpp
Ollama model files: https://github.com/ollama/ollama/blob...
Huggingface GGUF models: https://huggingface.co/models?library...