Malta

Image clustering divides a collection of images into meaningful groups, typically interpreted post-hoc via human-given annotations. Those are usually in the form of text, begging the question of using text as an abstraction for image clustering. Current image clustering methods, however, neglect the use of generated textual descriptions. We, therefore, propose \textit{Text-Guided Image Clustering}, i.e., generating text using image captioning and visual question-answering (VQA) models and subsequently clustering the generated text. Further, we introduce a novel approach to inject task- or domain knowledge for clustering by prompting VQA models. Across eight diverse image clustering datasets, our results show that the obtained text representations often outperform image features. Additionally, we propose a counting-based cluster explainability method. Our evaluations show that the derived keyword-based explanations describe clusters better than the respective cluster accuracy suggests. Overall, this research challenges traditional approaches and paves the way for a paradigm shift in image clustering, using generated text.

EACL 2024

Text-Guided Image Clustering

text-guided

text-to-image

multimodality

clustering

[![](https://assets.underline.io/uploads/markdown_image/1/image/9a88ae95fbde06e80f4bcf6d4f7d133a.png)](https://certificateaixia.paperform.co/)
The certificate will be provided by December 17. 
In case you need it urgently please contact: francesca.gasparini@unimib.it. 

 

The 20th International Conference of the Italian Association for Artificial Intelligence (**AIxIA 2021**) is organized by AIxIA (Associazione Italiana per l’Intelligenza Artificiale), which is a non-profit scientific society founded in 1988 and devoted to the promotion of Artificial Intelligence. The society aims to increase the public awareness of AI, encourage the teaching of it and promote research in the field.

 
 



**General Chair**

Stefania Bandini, University of Milano-Bicocca, Italyand RCAST - The University of Tokyo, Japan


**Program Chairs**

Viviana Mascardi, University of Genova, Italy; Matteo Palmonari, University of Milano-Bicocca, Italy; Giuseppe Vizzari, University of Milano-Bicocca, Italy

**Organizing Committee**

Francesca Gasparini, University of Milano-Bicocca, Italy (**Chair**); Stefano Ferilli, University of Bari, Italy; Marta Giltri, University of Milano-Bicocca, Italy; Daniela Briola, University of Insubria, Italy


**Publicity and Communication Committee**

Elisabetta Fersini, University of Milano-Bicocca, Italy (**Chair**); Angelo Ferrando, University of Genova, Italy; Francesca Alessandra Lisi, University of Bari, Italy; Giovanni Stilo, University of L'Aquila, Italy; Marco Viviani, University of Milano-Bicocca, Italy


**Workshop Chairs**

Gabriella Cortellessa, ISTC-CNR, Rome, Italy; Silvia Rossi, University of Naples Federico II, Italy; Fabio Stella, University of Milano-Bicocca, Italy


**PC Members**

Fulvio Mastrogiovanni, University of Genova, Italy; Roberto Basili, University of Rome Tor Vergata, Italy; Nicola Gatti, Polytechnic University of Milan, Italy; Stefano Bistarelli, University of Perugia, Italy; Piero Poccianti, Consorzio Operativo Gruppo MPS, Italy; Francesco Scarcello, University of Calabria, Italy; Flavio S. Correa Da Silva, University of São Paulo, Brazil; Elena Cabrio, Université Côte d'Azur, CNRS, Inria, I3S, France; Antonio Lieto, University of Turin, Italy; Diego Calvanese, Free University of Bozen-Bolzano, Italy; Alessio Micheli, University of Pisa, Italy; Francesca A. Lisi University of Bari Aldo Moro, Italy; Mauro Vallati, University of Huddersfield, United Kingdom; Bernardo Magnini, Fondazione Bruno Kessler (FBK), Italy; Luigia C. Aiello, Sapienza University of Rome, Italy; Rafael H. Bordini, Pontifical Catholic University of Rio Grande do Sul, Brazil; Chiara Ghidini, Fondazione Bruno Kessler (FBK), Italy; Federico Bergenti, University of Parma, Italy; Francesco Ricca, University of Calabria, Italy; Andrea Burattin, Technical University of Denmark, Denmark; Angelo Oddi, ISTC-CNR, Italian National Research Council, Italy; Farshad Badie, Aalborg University, Denmark; Emanuel Sallinger, University of Oxford, United Kingdom; Marin Lujak, University Rey Juan Carlos, Spain; Giancarlo Fortino, University of Calabria, Italy; Salvatore Gaglio, University of Palermo, Italy; Johannes K. Fichte, TU Dresden, Germany; Matteo Baldoni, University of Turin, Italy; Fabrizio Riguzzi, Univeristy of Ferrara, Italy; Ivana Dusparic, Trinity College Dublin, Ireland; Serena Villata, CNRS - Laboratoire d'Informatique, Signaux et Systèmes de Sophia-Antipolis, France; Paolo Rosso, Technical University of Valencia, Spain; Ana L. C. Bazzan, Federal University of Rio Grande do Sul, Brazil; Gianluigi Greco, University of Calabria, Italy; Stefano Cagnoni, University of Parma, Italy; Gian Luca Pozzato, University of Turin, Italy; Salvatore Ruggieri, University of Turin, Italy; Claudio Di Ciccio, Sapienza University of Rome, Italy; Eloisa Vargiu, Cetaqua, Water Technology Center, Spain; Marco Maratea, University of Genova, Italy; Andrea Roli, University of Bologna, Italy; Viviana Patti, University of Turin, Italy; Stefania Costantini, University of L'Aquila, Italy; Antonius Weinzierl, Vienna University of Technology, Austria; Silvia Rossi, University of Naples Federico II, Italy; Giovanni Semeraro, University of Bari, Italy; Luciano Serafini, Fondazione Bruno Kessler (FBK), Italy; Laura Giordano, University of Eastern Piedmont, Italy; Ernesto Jimenez-Ruiz, City, University of London, United Kingdom; Marco Gavanelli, University of Ferrara, Italy; Federico Chesani, University of Bologna, Italy

Please register with AIxIA to be able to enter the conference on Underline.io. If you have already registered, please login to Underline.io to be able to see full conference content. In case you do not have an Underline account, please create one. Thank you

Please login to your Underline account

AIxIA 2021

The 20th International Conference of the Italian Association for Artificial Intelligence (AIxIA 2021) is organized by AIxIA (Associazione Italiana per l’Intelligenza Artificiale), which is a non-profit scientific society founded in 1988 and devoted to the promotion of Artificial Intelligence. The society aims to increase the public awareness of AI, encourage the teaching of it and promote research in the field.

technical paper

Welcome to the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Continuing its mission of expanding and involving the science community of all European countries, EACL has selected the Malta community for the 18th EACL. Considering the importance of physical interaction among researchers, the conference will be held at the Hotel Radisson Blu, St. Julians, in Malta, from 17 to 22 of March, 2024. The conference will also feature the possibility to participate virtually. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language. 
#### [Conference Handbook](https://drive.google.com/file/d/13Wn38q0yev6U3RTgnf4Tn5T9GuAuZ-xW/view?usp=sharing) *downloadable*

To access the EACL 2024 event page you need to register [**here**](https://acl.swoogo.com/EACL2024). 

Please register!

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

Large language models (LLMs) with instruction fine-tuning demonstrate superior generative capabilities. However, these models are resource-intensive. To alleviate this issue, we explore distilling knowledge from instruction-tuned LLMs into much smaller ones. While other similar works have been done, they are often conducted on a limited set of (usually still large) models and are not accompanied by proper evaluations. To this end, we carefully develop a large set of 2.58M instructions based on both existing and newly-generated instructions. In addition to being sizable, we design our instructions to cover a broad set of topics to ensure diversity. Extensive analysis of our instruction dataset confirms its diversity, and we generate responses for these instructions using gpt-3.5-turbo. Leveraging these instructions, we fine-tune a diverse herd of models, collectively referred to as LaMini-LM, which includes models from both the encoder-decoder and decoder-only families, with varying sizes. We evaluate the performance of our models using automatic metrics on 15 different natural language processing (NLP) benchmarks, as well as through human assessment. We also assess the model for hallucination and toxicity, and for the former, we introduce a new benchmark dataset for hallucination-inducing QA. The results demonstrate that our proposed LaMini-LM models are comparable to strong baselines while being much smaller in size.

LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions

Modern language models often exhibit powerful but brittle behavior, leading to the development of larger and more diverse benchmarks to reliably assess their behavior. Here, we suggest that model performance can be benchmarked and elucidated with much smaller evaluation sets. We first show that in six popular language classification benchmarks, model confidence in the correct class on many pairs of points is strongly correlated across models. We build upon this phenomenon to propose Anchor Point Selection, a technique to select small subsets of datasets that capture model behavior across the entire dataset. Anchor points reliably rank models: across 87 diverse language model-prompt pairs, evaluating models using 1-30 anchor points outperforms uniform sampling and other baselines at accurately ranking models. Moreover, just a dozen anchor points can be used to estimate model per-class predictions on all other points in a dataset with low error, sufficient for gauging where the model is likely to fail. Lastly, we present Anchor Point Maps for visualizing these insights and facilitating comparisons of the performance of different models on various regions within the dataset distribution.

Anchor Points: Benchmarking Models with Much Fewer Examples

We study incremental constituent parsers to assess their capacity to output trees based on prefix representations alone. Guided by strictly left-to-right generative language models and tree-decoding modules, we build parsers that adhere to a strong definition of incrementality across languages. This builds upon work that asserted incrementality, but that mostly only enforced it on either the encoder or the decoder. Finally, we conduct an analysis against non-incremental and partially incremental models.

From Partial to Strictly Incremental Constituent Parsing

Contemporary multilingual dependency parsers can parse a diverse set of languages, but for Morphologically Rich Languages (MRLs), performance is attested to be lower than other languages. The key challenge is that, due to high morphological complexity and ambiguity of the space-delimited input tokens, the linguistic units that act as nodes in the tree are not known in advance. Pre-neural dependency parsers for MRLs subscribed to the {\em joint morpho-syntactic hypothesis}, stating that morphological segmentation and syntactic parsing should be solved jointly, rather than as a pipeline where segmentation precedes parsing. However, neural state-of-the-art parsers to date use a strict pipeline. 
In this paper we introduce a joint neural architecture where a lattice-based representation preserving all morphological ambiguity of the input is provided to an arc-factored model, which then solves the morphological segmentation and syntactic parsing tasks at once. Our experiments on Hebrew, a rich and highly ambiguous MRL, demonstrate state-of-the-art performance on parsing, tagging and segmentation of the Hebrew section of UD, using a single model. This proposed architecture is LLM-based and language agnostic, providing a solid foundation for MRLs to obtain further performance improvements and bridge the gap with other languages.

A Truly Joint Neural Architecture for Segmentation and Parsing

Graph-based semantic representations are popular in natural language processing (NLP), where it is often convenient to model linguistic concepts as nodes and relations as edges between them. Several attempts have been made to find a generative device that is sufficiently powerful to describe languages of semantic graphs, while at the same allowing efficient parsing. We contribute to this line of work by introducing graph extension grammar, a variant of the contextual hyperedge replacement grammars proposed by Hoffmann et al. Contextual hyperedge replacement can generate graphs with non-structural reentrancies, a type of node-sharing that is very common in formalisms such as abstract meaning representation, but which context-free types of graph grammars cannot model. To provide our formalism with a way to place reentrancies in a linguistically meaningful way, we endow rules with logical formulas in counting monadic second-order logic. We then present a parsing algorithm and show as our main result that this algorithm runs in polynomial time on graph languages generated by a subclass of our grammars, the so-called local graph extension grammars.

Generation and Polynomial Parsing of Graph Languages with Non-Structural Reentrancies

Speech dysfluency modeling is the bottleneck for both speech therapy and language learning. However, there is no AI solution to systematically tackle this problem. We first propose to define the concept of dysfluent speech and dysfluent speech modeling. We then present Hierarchical Unconstrained Dysfluency Modeling (H-UDM) approach that addresses both dysfluency transcription and detection to eliminate the need for extensive manual annotation. Furthermore, we introduce a simulated dysfluent dataset called VCTK++ to enhance the capabilities of H-UDM in phonetic transcription. Our experimental results demonstrate the effectiveness and robustness of our proposed methods in both transcription and detection tasks.

Towards Hierarchical Spoken Language Disfluency Modeling

Acoustic word embeddings (AWEs) are vector representations of spoken words. An effective method for obtaining AWEs is the Correspondence Auto-Encoder (CAE). In the past, the CAE method has been associated with traditional MFCC features. Representations obtained from self-supervised learning (SSL)-based speech models such as HuBERT, Wav2vec2, etc., are outperforming MFCC in many downstream tasks. However, they have not been well studied in the context of learning AWEs. This work explores the effectiveness of CAE with SSL-based speech representations to obtain improved AWEs. Additionally, the capabilities of SSL-based speech models are explored in cross-lingual scenarios for obtaining AWEs. Experiments are conducted on five languages: Polish, Portuguese, Spanish, French, and English. HuBERT-based CAE model achieves the best results for word discrimination in all languages, despite HuBERT being pre-trained on English only. Also, the HuBERT-based CAE model works well in cross-lingual settings. It outperforms MFCC-based CAE models trained on the target languages when trained on one source language and tested on target languages.

Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations

We present a new Hindi text-to-speech (TTS) dataset and demonstrate its utility for the expressive synthesis of children's audio stories. The dataset comprises narration by a single female speaker who modifies her voice to produce different story characters. Annotation for dialogue identification, character labelling, and character attribution are provided, all of which are expected to facilitate the learning of character voice and speaking styles. Experiments are conducted using different versions of the annotated dataset that enable training a multi-speaker TTS model on the single-speaker data. Subjective tests show that the multi-speaker model improves expressiveness and character voice consistency compared to the baseline single-speaker TTS. With the multi-speaker model, objective evaluations show comparable word error rates, better speaker voice consistency, and higher correlations with ground-truth emotion attributes. We release a new 16.8 hours storytelling speech dataset in Hindi and propose effective solutions for expressive TTS with narrator voice modulation and character voice consistency.

STORiCo: Storytelling TTS for Hindi with Character Voice Modulation

Much texts describe a changing world (e.g., procedures, stories, newswires), and understanding them requires tracking how entities change. An earlier dataset, OpenPI, provided crowdsourced annotations of entity state changes in text. However, a major limitation was that those annotations were free-form and did not identify salient changes, hampering model evaluation. To overcome these limitations, we present an improved dataset, OpenPI2.0, where entities and attributes are fully canonicalized and additional entity salience annotations are added. On our fairer evaluation setting, we find that current state-of-the-art language models are far from competent. We also show that using state changes of salient entities as a chain-of-thought prompt, downstream performance is improved on tasks such as question answering and classical planning, outperforming the setting involving all related entities indiscriminately. We offer OpenPI2.0 for the continued development of models that can understand the dynamics of entities in text.

OpenPI2.0: An Improved Dataset for Entity Tracking in Texts

Over 15 years ago, Ward & Birner (2006) suggested that non-canonical constructions in English can serve both to mark information status and to structure the information flow of discourse. One such construction is preposing, where a phrasal constituent appears to the left of its canonical position, typically sentence-initially. But computational work on discourse has, to date, ignored non-canonical syntax. We take account of non-canonical syntax by providing quantitative evidence relating NP/PP preposing to discourse relations. The evidence comes from an LLM mask-filling task that compares the predictions when a mask is inserted between the arguments of an implicit inter-sentential discourse relation --- first, when the right-hand argument (Arg2) starts with a preposed constituent, and again, when that constituent is in canonical (post-verbal) position. Results show that (1) the top-ranked mask-fillers in the preposed case agree more often with "gold" annotations in the Penn Discourse TreeBank than they do in the latter case, and (2) preposing in Arg2 can affect the distribution of discourse-relational senses.

Downloads

Next from EACL 2024

LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions

Similar lecture

Dialog Clustering: a framework for automatic text clustering

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from EACL 2024

LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions

Similar lecture

Dialog Clustering: a framework for automatic text clustering

Downloads