How to code long-context LLM: LongLoRA explained on LLama 2 100K

Описание к видео How to code long-context LLM: LongLoRA explained on LLama 2 100K

Code and theory how to fine-tune for long-context LLM, like LLama-2 100K.
Long sequence LLM are important for a long scientific article with more than 32K or 64K tokens. 3 days ago a new tech for creating long sequence LLM has been published, which looks finally usable (at first glance): Long LoRA.
Plus optimized for Flash Attention2.

Claude 100K, ChatGPT 32K, LLama2 100K, etc .... create long sequence LLMs.

LongLoRA explained in detailed and the code to extend your LLM to higher context length.

#ai
#coding
#explanation

Комментарии

Информация по комментариям в разработке

Похожие видео