Скачать или смотреть SANE2024 | Zhiyao Duan - Frontiers of Speech Synthesis

Zhiyao Duan, associate professor in Electrical and Computer Engineering, Computer Science and Data Science at the University of Rochester and co-founder of Violy, presents his work on "Frontiers of Speech Synthesis: Controllability, Expressiveness, and Natural Conversations" at the SANE 2024 workshop at Google, Cambridge, MA, on October 17, 2024.
More info on the SANE workshop series: http://www.saneworkshop.org/

Abstract: Speech synthesis research has made profound progress in the last decade. State-of-the-art text-to-speech and voice conversion systems are able to synthesize speech with high quality that is often indistinguishable from bonafide speech by human ears. However, such systems still lack controllability and expressiveness, and they show limited naturalness in conversational settings. In this talk, I will argue that controllability, expressiveness, and natural conversations are the new frontiers of speech synthesis research. I will present our recent work on these frontiers. Specifically, I will introduce ControlVC, a voice conversion system allowing users to control pitch and speech dynamically; GTR-Voice, our attempt to extend the definition of expressiveness to articulatory phonetics as professional voice actors do; and Parakeet, a system that can synthesize conversational speech with natural pauses, interruptions, and nonverbal events.