United Arab Emirates

Zero-resource cross-lingual transfer approaches aim to apply supervised models
from a source language to unlabelled target languages. In this paper we perform
an in-depth study of the two main techniques employed so far for cross-lingual
zero-resource sequence labelling, based either on data or model transfer.
Although previous research has proposed translation and annotation projection
(data-based cross-lingual transfer) as an effective technique for cross-lingual
sequence labelling, in this paper we experimentally demonstrate that high
capacity multilingual language models applied in a zero-shot (model-based
cross-lingual transfer) setting consistently outperform data-based
cross-lingual transfer approaches. A detailed analysis of our results suggests
that this might be due to important differences in language use. More
specifically, machine translation often generates a textual signal which is
different to what the models are exposed to when using gold standard data,
which affects both the fine-tuning and evaluation processes. Our results also
indicate that data-based cross-lingual transfer approaches remain a competitive
option when high-capacity multilingual language models are not available.

EMNLP 2022

Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings

opinion target

label projection

annotation projection

absa

roberta

low resource

named entity recognition

cross-lingual

zero-shot

multilingual

low-resource

machine translation

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

workshop paper

Automatic question generation (AQG) is the task of generating a question from a given passage and an answer. Most existing AQG methods aim at encoding the passage and the answer to generate the question. However, limited work has focused on modeling the correlation between the target answer and the generated question. Moreover, unseen or rare word generation has not been studied in previous works. In this paper, we propose a novel approach which incorporates question generation with its dual problem, question answering, into a unified primal-dual framework. Specifically, the question generation component consists of an encoder that jointly encodes the answer with the passage, and a decoder that produces the question. The question answering component then re-asks the generated question on the passage to ensure that the target answer is obtained. We further introduce a knowledge distillation module to improve the model generalization ability. We conduct an extensive set of experiments on SQuAD and HotpotQA benchmarks. Experimental results demonstrate the superior performance of the proposed approach over several state-of-the-art methods.

Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation

Prompting has shown impressive success in enabling large pre-trained language models (LMs) to perform diverse NLP tasks, especially with only few downstream data. Automatically finding the optimal prompt for each task, however, is challenging. Most existing work resorts to tuning *soft* prompts (e.g., embeddings) which fall short of interpretability, reusability across LMs, and applicability when gradients are not accessible. *Discrete* prompts, on the other hand, are difficult to optimize, and are often created by "enumeration (e.g., paraphrasing)-then-selection" heuristics that do not explore the prompt space systematically. This paper proposes RLPrompt, an efficient discrete prompt optimization approach with reinforcement learning (RL). RLPrompt formulates a parameter-efficient policy network that generates the optimized discrete prompt after training with reward. To harness the complex and stochastic reward signals from the large LM environment, we incorporate effective reward stabilization that substantially enhances training efficiency. RLPrompt is flexibly applicable to different types of LMs, such as masked (e.g., BERT) and left-to-right models (e.g., GPTs), for both classification and generation tasks. Experiments on few-shot classification and unsupervised text style transfer show superior performance over a wide range of existing fine-tuning or prompting methods. Interestingly, the resulting optimized prompts are often ungrammatical gibberish text; and surprisingly, those gibberish prompts are transferrable between different LMs to retain significant performance, indicating that LM prompting may not follow human language patterns.

RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning

Social media has increasingly played a key role in emergency response: first responders can use public posts to better react to ongoing crisis events and deploy the necessary resources where they are most needed. Timeline extraction and abstractive summarization are critical technical tasks to leverage large numbers of social media posts about events. Unfortunately, there are few datasets for benchmarking technical approaches for those tasks. This paper presents \datasetname{}, the largest dataset of local crisis event timelines available to date. \datasetname{} contains 1,000 crisis event timelines across four domains: wildfires, local fires, traffic, and storms. We built \datasetname{} using a semi-automated cluster-then-refine approach to collect data from the public Twitter stream. 
Our initial experiments indicate a significant gap between the performance of strong baselines compared to the human performance on both tasks.
Our dataset, code, and models are publicly available (https://github.com/CrisisLTLSum/CrisisTimelines).

CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization

A deployed question answering (QA) model can easily fail when the test data has a distribution shift compared to the training data. Robustness tuning (RT) methods have been widely studied to enhance model robustness against distribution shifts before model deployment. However, can we improve a model after deployment? To answer this question, we evaluate test-time adaptation (TTA) to improve a model after deployment. We first introduce ColdQA, a unified evaluation benchmark for robust QA against text corruption and changes in language and domain. We then evaluate previous TTA methods on ColdQA and compare them to RT methods. We also propose a novel TTA method called online imitation learning (OIL). Through extensive experiments, we find that TTA is comparable to RT methods, and applying TTA after RT can significantly boost the performance on ColdQA. Our proposed OIL improves TTA to be more robust to variation in hyper-parameters and test distributions over time.

Robust Question Answering against Distribution Shifts with Test-Time Adaption: An Empirical Study

Contrary to the notions of language as an amodal system, natural human communication is multimodal and combines speech, gestures, writing, and pictures. To account for this, recent work has proposed that our vocal, bodily, and graphic modalities persist in parallel in a multimodal language faculty, and both unimodal and multimodal expressions arise out of emergent states of a shared architecture. Such a model carries different expectations for the ways in which modalities may be similar or different from each other, and how they may interact. I will highlight these properties specifically for our graphic modality, which I argue can manifest in full visual languages when displaying both a systematic lexicon and complex grammar. I will use analysis of a corpus of several hundred annotated comics to show distinctive patterns that suggest they are drawn in different visual languages. Yet, I will also show that consistent “universal” linguistic principles persist across this structural diversity. Finally, I will argue that a multimodal language faculty requires us to change our conception of linguistic relativity, and I will show how subtle structures of spoken languages permeate across to visual languages. Altogether, this work argues for a multimodal basis of linguistic structure, and heralds a reconsideration of what constitutes the language system.

The multimodal language faculty and the visual languages of comics

This paper deals with the problem of incremental dialect identification. Our goal is to
reliably determine the dialect before the full utterance is given as input. The major part of the previous research on dialect identification has been model-centric, focusing on performance. We address a new question: How much input is needed to identify a dialect? Our approach is a data-centric analysis that results in general criteria for finding the shortest input needed to make a plausible guess. Working
with three sets of language dialects (Swiss German, Indo-Aryan and Arabic languages), we show that it is possible to generalize across dialects and datasets with two input shortening criteria: model confidence and minimal input length (adjusted for the input type).

Early Guessing for Dialect Identification

We propose EntityCS, a method that focuses on Entity-level Code-Switching to capture fine-grained cross-lingual semantics without corrupting syntax. We construct and release an EntityCS corpus with 93 languages based on English Wikipedia and Wikidata, and design novel masking strategies for entity prediction. XLM trained on the constructed EntityCS corpus with the proposed masking strategies shows consistent improvement on entity-centric downstream tasks.

EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching

Causal inference is becoming an increasingly important topic in deep learning, with the potential to help with critical deep learning problems such as model robustness, interpretability, and fairness. In addition, causality is naturally widely used in various disciplines of science, to discover causal relationships among variables and estimate causal effects of interest. In this tutorial, we introduce the fundamentals of causal discovery and causal effect estimation to the natural language processing (NLP) audience, provide an overview of causal perspectives to NLP problems, and aim to inspire novel approaches to NLP further. This tutorial is inclusive to a variety of audiences and is expected to facilitate the community’s developments in formulating and addressing new, important NLP problems in light of emerging causal principles and methodologies.

Tutorial on Causal Inference for Natural Language Processing

State-of-the-art language models in NLP perform best when fine-tuned even on small datasets, but due to their increasing size, fine-tuning and downstream usage have become extremely compute-intensive. Being able to efficiently and effectively fine-tune the largest pre-trained models is thus key in order to reap the benefits of the latest advances in NLP. In this tutorial, we provide a comprehensive overview of parameter-efficient fine-tuning methods. We highlight their similarities and differences by presenting them in a unified view. We explore the benefits and usage scenarios of a neglected property of such parameter-efficient models—modularity—such as composition of modules to deal with previously unseen data conditions. We finally highlight how both properties—parameter efficiency and modularity—can be useful in the real-world setting of adapting pre-trained models to under-represented languages and domains with scarce annotated data for several downstream applications. 

Modular and Parameter-Efficient Fine-Tuning for NLP Models

The autoregressive mechanism makes it convenient to train sequence generation models, but the generation is slow since autoregressive models can only generate the target sequence word-by-word. Non-autoregressive (NAR) models generate all tokens in parallel by removing the sequential dependencies within the target sequence. While NAR models have received much attention in sequence generation tasks, it is non-trivial for NAR models to obtain satisfactory performance, where the major challenge is to capture the multi-modal data distribution. In this tutorial, we will provide a comprehensive introduction to non-autoregressive sequence generation in four sections. First, we will briefly introduce the background of sequence generation, give the motivation of NAR generation and the multi-modality challenge faced by NAR models. Second, we will introduce efforts to improve NAR generation. We focus on non-autoregressive translation in this paper, which includes many representative methods from different aspects: training objectives, data distillation, latent modeling, upsampling, improved decoding approaches, etc. Third, we will introduce the application of NAR models on a wide range of sequence generation tasks, including language modeling, speech recognition, text-to-speech, etc. Finally, we will conclude this tutorial by summarizing the strengths and challenges of NAR models, and discussing current concerns and future directions of NAR generation. 


Downloads

Next from EMNLP 2022

Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation

Similar lecture

Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from EMNLP 2022

Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation

Similar lecture

Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings

Downloads