Скачать или смотреть JavisDiT++: Joint Audio-Video Generation from Text (2.1B Params, Token-Level Sync)

JavisDiT++ is a compact text-to-audiovisual generation model that tries to do the hard thing: generate video and sound together, with actual correspondence between the two. The project page presents it as an upgraded version of JavisDiT, built on Wan2.1-1.3B and expanded to 2.1B parameters.

The key idea is not just “video + audio in one model,” but token-level audio-video synchronization. The authors describe a cross-modal setup where audio and video tokens are aligned during generation, plus a preference-learning stage to improve perceived quality and alignment. In theory, that means the sound should not just exist alongside the visuals, but actually match what is happening on screen.

Another interesting claim: they say the jump from 1.3B to 2.1B comes without increasing inference cost, by adding capacity in a way that does not raise runtime in the same way a standard scale-up would.

Realistically, this is still a small model by current frontier standards, so it does not compete with the latest proprietary top-end systems like Veo-class models on overall fidelity. But that’s also what makes it impressive: for roughly 2B parameters, synchronized audio-video generation is already here in open form.

Links:
Project page: https://javisverse.github.io/JavisDiT...
GitHub: https://github.com/JavisVerse/JavisDiT
Hugging Face: https://huggingface.co/JavisVerse/Jav...