United States

As part of the Open Language Data Initiative shared tasks, we have expanded the FLORES+ evaluation set to include Emakhuwa, a lowresource language widely spoken in Mozambique. We translated the dev and devtest sets from Portuguese into Emakhuwa, and we detail the translation process and quality assurance measures used. Our methodology involved various quality checks, including postediting and adequacy assessments. The resulting datasets consist of multiple reference sentences for each source. We present baseline results from training a Neural Machine Translation system and fine-tuning existing multilingual translation models. Our findings suggest that spelling inconsistencies remain a challenge in Emakhuwa. Additionally, the baseline models underperformed on this evaluation set, underscoring the necessity for further research to enhance machine translation quality for Emakhuwa. The data is publicly available at https://huggingface.co/ datasets/LIACC/Emakhuwa-FLORES

EMNLP 2024

Expanding FLORES+ Benchmark for More Low-Resource Settings: Portuguese-Emakhuwa Machine Translation Evaluation

low-resource languagues

flores+

emakhuwa

african languages

machine translation

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

We introduce a FLORES+ dataset as an evaluation benchmark for modern Wu Chinese machine translation models and showcase its compatibility with existing Wu data. Wu Chinese is mutually unintelligible with other Sinitic languages such as Mandarin and Yue (Cantonese), but uses a set of Hanzi (Chinese characters) that profoundly overlaps with others. The population of Wu speakers is the second largest among languages in China, but the language has been suffering from significant drop in usage especially among the younger generations. We identify Wu Chinese as a textually low-resource language and address challenges for its machine translation models. Our contributions include: (1) an open-source, manually translated dataset, (2) full documentations on the process of dataset creation and validation experiments, (3) preliminary tools for Wu Chinese normalization and segmentation, and (4) benefits and limitations of our dataset, as well as implications to other low-resource languages.

Machine Translation Evaluation Benchmark for Wu Chinese: Workflow and Analysis

This paper introduces a translation of the FLORES+ dataset into the endangered Erzya language, with the goal of evaluating machine translation between this language and any of the other 200 languages already included into FLORES+. This translation was carried out as a part of the Open Language Data shared task at WMT24. We also present a benchmark of existing translation models bases on this dataset and a new translation model that achieves the state-of-the-art quality of translation into Erzya from Russian and English.

FLORES+ Translation and Machine Translation Evaluation for the Erzya Language

This paper presents the Seed-CAT submission to the WMT24 Open Language Data Initiative shared task. We detail our data collection method, which involves a computer-aided translation tool developed explicitly for translating Seed corpora. We release a professionally translated Spanish corpus and a provenance dataset documenting the translation process. The quality of the data was validated on the FLORES+ benchmark with English-Spanish neural machine translation models, achieving an average chrF++ score of 34.9.

Spanish Corpus and Provenance with Computer-Aided Translation for the WMT24 OLDI Shared Task

This paper introduces our machine translation system (team sakura), developed for the 2024 WMT Patent Translation Task. Our system focuses on translations between Japanese-English, Japanese-Korean, and Japanese-Chinese. As large language models have shown good results for various natural language processing tasks, we have adopted the RakutenAI-7B-chat model, which has demonstrated effectiveness in English and Japanese. We fine-tune this model with patent-domain parallel texts and translate using multiple prompts.

Rakuten’s Participation in WMT 2024 Patent Translation Task

This paper presents SETU-ADAPT's submissions to the WMT 2024 Biomedical Shared Task, where we participated for the language pairs English-to-French and English-to-German. Our approach focused on fine-tuning Large Language Models (LLMs), using in-domain and synthetic data, employing different data retrieval strategies. We introduce a novel MT framework, involving three autonomous agents: a Translator Agent, an Evaluator Agent and a Reviewer Agent. We present our findings and report the quality of the outputs.

The SETU-ADAPT Submission for WMT 24 Biomedical Shared Task

The presentation explores the WMT 2024 English-to-Low-Resource Multimodal Translation Task, where seven teams submitted translations for both automatic and human evaluation. It delves into the various approaches adopted by each team, along with an analysis of their results.

Findings of WMT2024 English-to-Low Resource Multimodal Translation Task

An overview of non-repetitive translation task

Findings of the WMT 2024 Shared Task on Non-Repetitive Translation

Translating for low-resource languages is challenging due to limited quality data. We improved translation performance by integrating target sentence length as an additional feature to source sentence during training. We developed transformer models and evaluated across 8 language directions (English <=> Assamese, Manipuri, Khasi, and Mizo), using four different length encoding methods. Comparing these models to the baseline, we submitted two systems per language direction and present our findings here.​

A3-108 Controlling Token Generation in Low Resource Machine Translation Systems

This paper describes the submission systems of DLUT-NLP team for the WMT24 low-resource Indic language translation shared task. We participated in the translation task of four language pairs, including en-as, en-mz, en-kha, en-mni.

DLUT-NLP Machine Translation Systems for WMT24 Low-Resource Indic Language Translation

Machine Translation for low-resource languages poses significant challenges, primarily due to the limited availability of data.The WMT24 Low-Resource Indic Neural Machine Translation task challenges us to employ innovative techniques to improve machine translation for low-resource Indian languages. Our proposed solution leverages advancements in neural machine translation, focusing on methodologies such as back-translation and fine-tuning. By fine-tuning pretrained models like mBART, we achieved significant progress in translating languages such as Manipuri and Khasi. The best score was achieved for the English-to-Khasi (en-kh) primary model, with the highest BLEU score of 0.0492, chrF score of 0.3316, and METEOR score of 0.2589 (on scale of 0 to 1) and comparable scores for other language pairs.

Downloads

Next from EMNLP 2024

Machine Translation Evaluation Benchmark for Wu Chinese: Workflow and Analysis

Similar lecture

Building Resources for Emakhuwa: Machine Translation and News Classification Benchmarks

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from EMNLP 2024

Machine Translation Evaluation Benchmark for Wu Chinese: Workflow and Analysis

Similar lecture

Building Resources for Emakhuwa: Machine Translation and News Classification Benchmarks

Downloads