Скачать или смотреть It's Not a Walk in the Park! Challenges of Idiom Translation in Speech-to-text Systems

Why do modern AI speech translation systems struggle with idioms?

In this research talk, computational linguistics researcher Iuliia Zaitova explores a key limitation of modern AI translation systems: non-compositional expressions such as idioms.

Idioms like “kick the bucket” or “it’s not a walk in the park” cannot be understood by simply translating each word individually. While this is already difficult for text-based translation systems, the problem becomes even more complex for speech-to-text translation models, which must simultaneously process acoustic signals, syntax, and semantics.

This talk is based on a recent ACL publication conducted in collaboration with researchers from Saarland University and Xi’an Jiaotong University.

🔬 What this talk covers

What idioms are and why they challenge AI translation
Comparison between speech translation models and text-based systems
Evaluation of models like Whisper, SeamlessM4T, NLLB, LLaMA-based translators, and DeepSeek
Differences between end-to-end speech translation vs cascaded pipelines
Human evaluation of AI translations
Interpretability analysis showing how meaning emerges inside neural models

📊 Key findings

Speech translation systems perform significantly worse on idioms compared to text-based models
Cascaded pipelines (ASR + text translation) outperform direct speech-to-text systems
Neural models tend to produce literal translations instead of figurative meaning

#ArtificialIntelligence #MachineTranslation #ComputationalLinguistics #NLP