AISC 9 - Delphi Small LM Training & Evals Made Easy

Описание к видео AISC 9 - Delphi Small LM Training & Evals Made Easy

Team members: Alice Rigg, Gonçalo Paulo, Jai Dhyani, Jannik Brinkmann, Jett Janiak, Joshua Wendland, Rai (Phan Anh Duong), Siwei Li, Víctor Abia Alonso

Project Summary: Using Delphi, researchers can easily train and evaluate small LMs.

We provide tools for standardized tokenizer training, dataset tokenization, model training and model evaluation. We took great care to make sure that the suite is user friendly and the results are reproducible.

Delphi supports all 🤗 CausalLM architectures and any dataset. As a proof of concept, we trained a suite of 10 🐍 mambas and 10 🦙 llamas, ranging from 50k to 50m parameters, on the Tiny Stories dataset.

Комментарии

Информация по комментариям в разработке

Похожие видео