Скачать или смотреть DITING: LLM Benchmark for Web Novel Translation

In this AI Research Roundup episode, Alex discusses the paper:
'DITING: A Multi-Agent Evaluation Framework for Benchmarking Web Novel Translation'
This work targets the shortcomings of standard metrics in assessing LLM translations of web novels, where idioms, culture, and narrative fidelity matter. It introduces DITING, a benchmark with six focused dimensions and 18,745 expert-annotated Chinese–English pairs. The authors also release MetricAlign, a meta-evaluation set with high inter-annotator reliability. Finally, AgentEval uses multiple reasoning agents and a judge to deliberate and score translations with task-grounded schemas.
Paper URL: https://arxiv.org/abs/2510.09116

#AI #MachineLearning #DeepLearning #LLM #MachineTranslation #Evaluation #Benchmark #NLP

Resources:
GitHub: https://github.com/WHUNextGen/DITING