Fine-Tune Your Own Tiny-Llama on Custom Dataset

What to learn how to customize tiny llama on your own dataset? Here is how to do it.

🦾 Discord:   / discord  
☕ Buy me a Coffee: https://ko-fi.com/promptengineering
|🔴 Patreon:   / promptengineering  
💼Consulting: https://calendly.com/engineerprompt/c...
📧 Business Contact: [email protected]
Become Member: http://tinyurl.com/y5h28s6h

💻 Pre-configured localGPT VM: https://bit.ly/localGPT (use Code: PromptEngineering for 50% off).


LINKS:
TinyLlama Github: https://github.com/jzhang38/TinyLlama
TinyLlama Chat-Demo: http://tinyurl.com/e48uftef
TinyLlama Paper: https://arxiv.org/pdf/2401.02385.pdf
Google Colab: http://tinyurl.com/4eny9cvc
Blogpost: http://tinyurl.com/krskjumf
Learn about LoRA: Mixtral:    • Fine-tune Mixtral 8x7B (MoE) on Custo...  
The Dataset: https://huggingface.co/datasets/burke...

TIMESTAMPS:
[00:00] Introduction
[01:07] Training dataset
[02:29] Setting Up the Code
[04:19] Formatting the Dataset
[06:49] Setting Up the Model and LoRAs
[09:58] Training the Model
[10:37] Merging the Model and LoRa Adapters
[11:31] Inference and Testing
[14:02] Conclusion and Future Prospects



All Interesting Videos:
Everything LangChain:    • LangChain  

Everything LLM:    • Large Language Models  

Everything Midjourney:    • MidJourney Tutorials  

AI Image Generation:    • AI Image Generation Tutorials