Скачать или смотреть MANZANO: Unified Multimodal LLM with Hybrid Tokenizer

In this AI Research Roundup episode, Alex discusses the paper:
'MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer'
Manzano introduces a unified multimodal LLM that handles both image understanding (I2T) and text-to-image generation (T2I) via a hybrid vision tokenizer and a three-stage training recipe. It resolves the trade-off between discrete tokenizers (good for AR generation) and continuous embeddings (good for understanding) by using a shared ViT with two lightweight adapters—continuous for I2T and discrete FSQ tokens for T2I—aligned in a common semantic space. A single autoregressive decoder predicts joint text and 64K image tokens, with a separate diffusion image decoder (DiT-Air) rendering pixels. Models scale from 300M to 30B parameters and are trained on large mixed corpora across text-only, interleaved, I2T, and T2I data.
Paper URL: https://arxiv.org/abs/2509.16197

#AI #MachineLearning #DeepLearning #Multimodal #LLM #TextToImage