Thailand

$$Recently, there has been a growing interest in analyzing user-generated text to understand opinions expressed on social media. In NLP, this task is known as stance detection, where the goal is to predict whether the writer is in favor, against, or has no opinion on a given topic. Stance detection is crucial for applications such as sentiment analysis, opinion mining, and social media monitoring, as it helps in capturing the nuanced perspectives of users on various subjects. As part of the ArabicNLP 2024 program, we organized the first shared task on Arabic Stance Detection, StanceEval 2024. This initiative aimed to foster advancements in stance detection for the Arabic language, a relatively underrepresented area in Arabic NLP research. This overview paper provides a detailed description of the shared task, covering the dataset, the methodologies used by various teams, and a summary of the results from all participants. We received 28 unique team registrations, and during the testing phase, 16 teams submitted valid entries. The highest classification F-score obtained was 84.38.$$

ACL 2024

$StanceEval 2024: The First Arabic Stance Detection Shared Task$

stance detection

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

workshop paper

This research explores the effectiveness of using pre-trained language models (PLMs) as feature extractors for Arabic stance detection on social media, focusing on topics like women empowerment, COVID-19 vaccination, and digital transformation. By leveraging sentence transformers to extract embeddings and incorporating aggregation architectures on top of BERT, we aim to achieve high performance without the computational expense of fine-tuning. Our approach demonstrates significant resource and time savings while maintaining competitive performance, scoring an F1-score of 78.62 on the test set. This study highlights the potential of PLMs in enhancing stance detection in Arabic social media analysis, offering a resource-efficient alternative to traditional fine-tuning methods.

Team_Zero at StanceEval2024: Frozen PLMs for Arabic Stance Detection

As part of our study, we worked on three tasks: stance detection, sarcasm detection and senti- ment analysis using fine-tuning techniques on BERT-based models. Fine-tuning parameters were carefully adjusted over multiple iterations to maximize model performance. The three tasks are essential in the field of natural lan- guage processing (NLP) and present unique challenges. Stance detection is a critical task aimed at identifying a writer’s stances or view- points in relation to a topic. Sarcasm detection seeks to spot sarcastic expressions, while senti- ment analysis determines the attitude expressed in a text. After numerous experiments, we iden- tified Arabert-twitter as the model offering the best performance for all three tasks. In particu- lar, it achieves a macro F-score of 78.08% for stance detection, a macro F1-score of 59.51% for sarcasm detection and a macro F1-score of 64.57% for sentiment detection. . Our source code is available at https:// github.com/MezghaniAmal/Mawqif

ANLP RG at StanceEval2024: Comparative Evaluation of Stance, Sentiment and Sarcasm Detection

This study compares Term Frequency-Inverse Document Frequency (TF-IDF) features with Sentence Transformers for detecting writers' stances—favorable, opposing, or neutral—towards three significant topics: COVID-19 vaccine, digital transformation, and women empowerment. Through empirical evaluation, we demonstrate that Sentence Transformers outperform TF-IDF features across various experimental setups. Our team, dzStance, participated in a stance detection competition, achieving the 13th position (74.91%) among 15 teams in Women Empowerment, 10th (73.43%) in COVID Vaccine, and 12th (66.97%) in Digital Transformation. Overall, our team's performance ranked 13th (71.77%) among all participants. Notably, our approach achieved promising F1-scores, highlighting its effectiveness in identifying writers' stances on diverse topics. These results underscore the potential of Sentence Transformers to enhance stance detection models for addressing critical societal issues.

dzStance at StanceEval2024: Arabic Stance Detection based on Sentence Transformers

This paper presents our submission for the Stance Detection in Arabic Language (StanceEval) 2024 shared task conducted by Team SMASH of the University of Edinburgh. We evaluated the performance of various BERT-based and large language models (LLMs). MARBERT demonstrates superior performance among the BERT-based models, achieving F1 and macro-F1 scores of 0.570 and 0.770, respectively. In contrast, Command~R model outperforms all models with the highest overall F1 score of 0.661 and macro F1 score of 0.820.

SMASH at StanceEval 2024: Prompt Engineering LLMs for Arabic Stance Detection

In NLP, stance detection identifies a writer's position or viewpoint on a particular topic or entity from their text and social media activity, which includes preferences and relationships. Researchers have been exploring techniques and approaches to develop effective stance detection systems. Large language models' latest advancements offer a more effective solution to the stance detection problem. This paper proposes fine-tuning the newly released 8B-parameter Llama 3 model from Meta GenAI for Arabic text stance detection. The proposed method was ranked ninth in the StanceEval 2024 Task on stance detection in Arabic language achieving a Macro average $F_1$ score of 0.7647.

CUFE at StanceEval2024: Arabic Stance Detection with Fine-Tuned Llama-3 Model

Stance detection, an evolving task in natural language processing, involves understanding a writer's perspective on certain topics by analyzing his written text and interactions online, especially on social media platforms. In this paper, we outline our submission to the StanceEval task, leveraging the Mawqif dataset featured in The Second Arabic Natural Language Processing Conference. Our task is to detect writers' stances (Favor, Against, or None) towards three selected topics (COVID-19 vaccine, digital transformation, and women empowerment). We present our approach primarily relying on a contrastive loss ensemble strategy. Our proposed approach achieved an F1-score of 0.8438 and ranked first in the stanceEval 2024 task. The code and checkpoints are available at https://github.com/MBadran2000/Mawqif.git

AlexUNLP-BH at StanceEval2024: Multiple Contrastive Losses Ensemble Strategy with Multi-Task Learning For Stance Detection in Arabic

It is essential to understand the attitude of individuals towards specific topics in Arabic language for tasks like sentiment analysis, opinion mining, and social media monitoring. However, the diversity of the linguistic characteristics of the Arabic language presents several challenges to accurately evaluate the stance. In this study, we suggest ensemble approach to tackle these challenges. Our method combines different classifiers using the voting method. Through multiple experiments, we prove the effectiveness of our method achieving significant F1-score value equal to 0.7027. Our findings contribute to promoting NLP and offer treasured enlightenment for applications like sentiment analysis, opinion mining, and social media monitoring.

ISHFMG_TUN at StanceEval: Ensemble Method for Arabic Stance Evaluation System

This paper outlines our approach to the StanceEval 2024- Arabic Stance Evaluation shared task. The goal of the task was to identify the stance, one out of three (Favor, Against or None) towards tweets based on three topics, namely- COVID-19 Vaccine, Digital Transformation and Women Empowerment. Our approach consists of fine-tuning BERT-based models efficiently for both, Single-Task Learning as well as Multi-Task Learning, the details of which are discussed. Finally, an ensemble was implemented on the best-performing models to maximize overall performance. We achieved a macro F1 score of 78.02% in this shared task. Our codebase is available publicly.

PICT at StanceEval2024: Stance Detection in Arabic using Ensemble of Large Language Models

In this paper, we present a high-performing model for Arabic stance detection on the STANCEEVAL2024 shared task part of ARABICNLP2024. Our model leverages ARABERTV1; a pre-trained Arabic language model, within a single-task learning framework. We fine-tuned the model on stance detection data for three specific topics: COVID19 vaccine, digital transformation, and women empowerment, extracted from the MAWQIF corpus. In terms of performance, our model achieves 73.30 macro-F1 score for women empowerment, 70.51 for digital transformation, and 64.55 for COVID-19 vaccine detection.

TAO at StanceEval2024 Shared Task: Arabic Stance Detection using AraBERT

We present WojoodNER-$2024$, the second Arabic Named Entity Recognition (NER) Shared Task. In WojoodNER-$2024$, we focus on fine-grained Arabic NER. We provided participants with a new Arabic fine-grained NER dataset called Wojoodfine, annotated with subtypes of entities. WojoodNER-$2024$ encompassed three subtasks: ($i$) Closed-Track Flat Fine-Grained NER, ($ii$) Closed-Track Nested Fine-Grained NER, and ($iii$) an Open-Track NER for the Israeli War on Gaza. A total of $43$ unique teams registered for this shared task. Five teams participated in the Flat Fine-Grained Subtask, among which two teams tackled the Nested Fine-Grained Subtask and one team participated in the Open-Track NER Subtask. The winning teams achieved $F_1$ scores of $91%$ and $92%$ in the Flat Fine-Grained and Nested Fine-Grained Subtasks, respectively. The sole team in the Open-Track Subtask achieved an $F_1$ score of $73.7%$.

Downloads

Next from ACL 2024

Team_Zero at StanceEval2024: Frozen PLMs for Arabic Stance Detection

Similar lecture

CUFE at StanceEval2024: Arabic Stance Detection with Fine-Tuned Llama-3 Model

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2024

Team_Zero at StanceEval2024: Frozen PLMs for Arabic Stance Detection

Similar lecture

CUFE at StanceEval2024: Arabic Stance Detection with Fine-Tuned Llama-3 Model

Downloads