How do Language Agents Perform in Translating Long-Text Novels? Meet TransAgents: A Multi-Agent Framework Using LLMs to Tackle the Complexities of Literary Translation

Machine translation (MT) has made impressive progress in recent years, driven by breakthroughs in deep learning and neural networks. However, the challenge of literary translations for MT systems is difficult to solve. Literary texts, known for their complex language, figurative expressions, cultural variations, and unique feature styles, create problems that are hard for machines to overcome  Due to this complexity, literary translation becomes one of the most challenging areas within machine translation, often referred to as “the last frontier of machine translation”

Large language models (LLMs) have transformed the field of AI. These models are pre-trained on a huge amount of text data, learning to predict the next word in a sentence. After pretraining, supervised fine-tuning (SFT) or instruction tuning (IT) is used for fine-tuning the models using the instructions, allowing them to adapt their general language knowledge. Another method is multi-agent systems intelligent agents are developed to understand their environments, make good decisions, and react with suitable actions. Moreover, MT has achieved more advancements recently, which include general-purpose MT, low-resource MT, multilingual MT, and non-autoregressive MT. 

Researchers from Monash University, the University of Macau, and Tencent AI Lab introduced TRANSAGENTS, a multi-agent system for literary translation that can tackle complex details of literary works by utilizing multi-agent methods. Despite the method showing bad performance in terms of d-BLEU scores, it is preferred by human evaluators and an LLM evaluator over human-written references and GPT-4 translations. TRANSAGENTS can generate translations with more detailed and diverse descriptions, and it is 80 times less costly compared to professional human translators during the cost analysis for literary text translation. 

Two evaluation strategies, Monolingual Human Preference (MHP) and Bilingual LLM Preference (BLP) are also introduced by the researchers to evaluate the quality of translations. MHP focuses on the effect of translation on the target audience, giving importance to fluidity and suitable culture, while BLP compares translations directly with the original texts using advanced LLMs. Researchers presented an in-depth analysis of the strengths and weaknesses, especially in translation systems based on LLM, including GPT-4 and TRANSAGENTS, showing certain limitations on problems related to content omission. 

TRANSAGENTS is compared with other methods, such as REFERENCE 1 and GPT-4- 1106-PREVIEW using monolingual human preference evaluations. Results show that human evaluators prefer the translations generated by TRANSAGENTS over the other two methods mentioned. Moreover, the models are evaluated using BLP, and the results show that GPT-4-0125-PREVIEW prefers translations produced by TRANSAGENTS more, showing its robust preference for detailed and diverse descriptions while evaluating literary translations. Also, REFERENCE 1 costs $168.48 per chapter for the translations, but TRANSAGENTS costs $500 for the entire test set, which is 80 times cheaper. 

In conclusion, researchers introduced  TRANSAGENTS, a multi-agent virtual company designed for literary translation that reflects the traditional translation publication process. Further, two strategies, MHP and BLP are introduced to evaluate the quality of translations. Despite the lower d-BLEU scores, the translations generated by TRANSAGENTS are preferred over human-written references by human evaluators and language models, and it is 80 times less costly compared to professional human translators for literary text translation. However, certain limitations of TRANSAGENTS highlight the problem in machine translation (MT) evaluation approaches like bad evaluation metrics and the reliability of reference translations

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our 42k+ ML SubReddit