Indonesia

Machine Learning models in the field of Information Extraction for Scientific Publications
require high-quality labeled data. The large
amount of easily accessible LATEX source code
is a treasure trove of high-quality labeled data.
However, existing datasets comprised of document collections and PDF extraction tools have
limitations: (1) The hierarchical structure of
papers is lost because labeling is done in terms
of pages rather than documents; (2) The reading order is not extracted, which potentially
muddles the extracted contextual structure; (3)
Papers included in the datasets are not likely
to be up-to-date.To address these challenges,
we propose LATEX Rainbow, a framework that
bridges LATEX to PDF that can automatically annotate and extract semantic and layout information from LATEX source code. This framework
extends existing annotation methods by taking
into account the properties of different existing
approaches. It can produce token-level semantic structure annotations, preserve the paper’s
reading order, and extract the table of contents,
i.e., the article’s section structure. LATEX Rainbow enables anyone to extend their datasets
with the latest documents. The project is opensourced on GitHub1
for community contributions and use.


IJCNLP-AACL 2023

LaTeX Rainbow: Universal LaTeX to PDF Document Semantic &amp; Layout Annotation Framework

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

LaTeX Rainbow: Universal LaTeX to PDF Document Semantic & Layout Annotation Framework

workshop paper

### Welcome to IJCNLP-AACL 2023!
I am honored and happy to welcome you to IJCNLP-AACL 2023! The conference hosts both the 13th
International Joint Conference on Natural Language Processing (IJCNLP) and the 3rd Conference of
the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL). 
The joint conferences of IJCNLP and AACL have been organized by alternating leadership in the
Asia-Pacific region, in odd years by Asian Federation of Natural Language Processing (AFNLP) and
in even years by AACL. The conference is organized by AFNLP this year. 

Considering the current situation that we are not yet fully over the coronavirus pandemic, the
conference is offered in a hybrid mode, off-line in Nusa Dua, Bali, Indonesia, and on-line with the help
of Underline, during November 1-4, 2023. For those attending off-line, I hope that you will find time to
explore the exciting local area, which remains quite warm and pleasant at this time of the year. For
those attending on-line, I hope that the attendance is not much hampered, sometimes even enhanced
and comfortable, though the on-line arrangements have been made without ample time. 

Message from Jong C. Park, Korea Advanced Institute of Science and Technology (KAIST) 
IJCNLP-AACL 2023 General Chair 
###### [Conference Handbook](https://drive.google.com/file/d/1S2JSXa105xJxIkNBfhYmGH20Om6wARBx/view?usp=sharing)

In order to acces this event page, you need to register [here](http://www.ijcnlp-aacl2023.org/)

The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics

Weak supervision has emerged as a promising approach for rapid and large-scale
dataset creation in response to the increasing demand for accelerated NLP development. By
leveraging labeling functions, weak supervision allows practitioners to generate datasets
quickly by creating learned label models that produce soft-labeled datasets. This paper aims to
show how such an approach can be utilized to build an Indonesian NLP dataset from
conservation news text. We construct two types of datasets: multi-class classification and
sentiment classification. We then provide baseline experiments using various pretrained
language models. These baseline results demonstrate test performances of 59.79% accuracy
and 55.72% F1-score for sentiment classification, 66.87% F1-score-macro, 71.5% F1-score-
micro, and 83.67% ROC-AUC for multi-class classification. Additionally, we release the datasets
and labeling functions used in this work for further research and exploration.

Utilizing Weak Supervision to Generate Indonesian Conservation Datasets

Text classification is an important technique in
natural language processing for categorizing
text into appropriate domains. With the increasing amount of textual data, robust text classification is in high demand. This paper focuses on
single-label classification of text for scholarly
articles, aiming to analyze a large number of
papers. Inspired by the successful Fusion-inDecoder method used in question-answering
tasks, we propose an accurate method suitable
for long articles. We evaluate the effectiveness of our method through experiments on
single-label classification with scholarly articles, demonstrating its high F1 scores.

Leveraging the Fusion-in-Decoder for Label Classification

This study tackles the challenge of generating
appropriate academic titles based on the paper’s
abstract. We approach this task as a high-level
text summarization problem and introduce an
innovative post-processing method that combines a predictive model with a set of linguistic
rules to enhance the quality of the title generation. We start by evaluating three Natural
Language Generation models (BART, T5, Flan
T5), by identifying the top-performing model
and by configuring it to generate diverse titles.
We then conduct experiments employing various post-processing strategies -using SciBERT
and linguistic rules- to select the best title out
of all machine-generated options. Finally, we
assess our title selection methods in relation to
human evaluations.

Enhancing Academic Title Generation Using SciBERT and Linguistic Rules

Keeping track of all relevant recent publications and experimental results for a research area is a challenging task. Prior work has demonstrated the efficacy of information extraction models in various scientific areas. Recently, several datasets have been released for the yet understudied materials science domain. However, these datasets focus on sub-problems such as parsing synthesis procedures or on subdomains, e.g., solid oxide fuel cells. In this resource paper, we present MuLMS, a new dataset of 50 open-access articles, spanning seven sub-domains of materials science. The corpus has been annotated by domain experts with several layers ranging from named entities over relations to frame structures. We present competitive neural models for all tasks and demonstrate that multi-task training with existing related resources leads to benefits.

MuLMS: A Multi-Layer Annotated Text Corpus for Information Extraction in the Materials Science Domain

We introduce a comprehensive end-to-end pipeline designed to extract complete bibliography section from English scientific articles in digital-born PDF format and further split them into individual citations. At the heart of our pipeline lies the utilization of Languageindependent Layout Transformer (LiLT), a multimodal model that combines text and layout features to enhance the accuracy and robustness of bibliography extraction. By considering both text and visual structure, LiLT significantly improves the identification of bibliographic sections within scientific articles. To split the extracted full bibliography into individual citations, we employ a custom fine-tuned version of SciBERT, a Transformer-based model that excels at handling complex formatting variations common in scholarly bibliography. Having such end-to-end pipeline in-house allows us to bypass reliance on third-party black box tools, such as GROBID, offering greater control and transparency in the bibliography extraction process. Another highlight of our pipeline is its extensibility, as it can be seamlessly adapted to multilingual and image-based PDFs, hence allowing its utility across a wide range of scholarly content. When evaluated on an in-house dataset of digital-born English PDF articles published at Elsevier, we achieved an F1-score of 94.6%, a notable 3.1% improvement over GROBID, which is a well-regarded tool for bibliography parsing in the industry.

An End-to-End Pipeline for Bibliography Extraction from Scientific Articles

Citation graphs represent the citation relations between papers, and they are commonly used by researchers to identify relevant papers. However, citation graphs do not always represent how papers are related to each other. To make more effective use of citation graphs to discover relevant papers, we can consider identifying the functions of citations and label each edge in the citation graphs with its function. This paper proposes a method to identify the functions of citations automatically. The proposed model utilizes language models, e.g., SciBERT, to identify the description of citation functions. However, the language models are limited in terms of the number of input tokens; thus, the entire citing paragraph cannot be processed at once. To overcome this problem, we analyzed the distribution of the descriptions of citation functions in the citing paragraphs and determined the focusing part in identifying the citation functions. Experiments conducted on scientific paper data demonstrated the effectiveness of the proposed method.

On the Use of Language Models for Function Identification of Citations in Scholarly Papers

Scientific research relies heavily on the exchange of knowledge through citations in academic literature. In the domain of astrophysics, the precise classification of citation functions and the extraction of contextual information are critical for understanding the vast universe of research papers. This paper presents the system description for the WIESP 2023 FOCAL shared task. We introduce an automated approach that leverages state-of-the-art language models, including ALBERT, RoBERTa, BERT, and DistillBERT, to classify citation functions and extract context within astrophysical paragraphs. Our system combines paraphrasing and question-answering techniques to achieve accurate results. Through comprehensive experiments, we demonstrate the robustness of our approach, with ALBERT consistently delivering strong performance.

Automated Citation Function Classification and Context Extraction in Astrophysics: Leveraging Paraphrasing and Question Answering

Measurement extraction is an information extraction subtask focused on extracting quantities and their dependent entities within a
given scientific text. Quantity extraction is
the first and most important step in measurement extraction. Most existing approaches
model the problem as a sequence-labeling task
using pre-trained language models (PLMs).
However, none of the existing systems have
utilised explicit syntactic knowledge to extend the PLM-based modeling. We propose a
syntax-enriched extension by integrating dependency tree representations as syntactic knowledge into transformer-based language models
to address the task of quantity extraction. We
apply our approach to a range of established
transformer-based models to evaluate our approach and analyze its impact in experiments
on scientific literature datasets. Our experimental results and in-depth analysis show that our
approach, syntax-enriched RoBERTa, outperforms the other models, even in situations with
scarce training data in the scientific domain.
The results demonstrate the adaptability of the
proposed model to the tasks, especially useful
in low-resource scenarios.

Investigating the Impact of Syntax-Enriched Transformers on Quantity Extraction in Scientific Texts

In this paper, we address the task of extracting semantic relations between entities in scientific articles in Russian, with a focus on scientific terms as entities. We present a dataset that includes annotated abstracts of scientific articles in Russian. This dataset was used to train and test models and develop an algorithm for the automatic extraction of semantic relations. We conducted experiments and compared one zero-shot and one few-shot approach for relation extraction: one based on the perplexity score and the other based on the use of prototype vectors of relations. Our results show that both methods can achieve reasonable performance, demonstrating the potential of zeroshot and few-shot approaches for relation extraction in scientific texts in Russian. The developed tool and annotated dataset are publicly available and could be valuable resources for other researchers.

Relation Extraction from Scientific Texts in Russian with Limited Training Data

We consider automatically identifying the defined term within a mathematical definition from the text of an academic article. Inspired by the development of transformer-based natural language processing applications, we pose the problem as (a) a token-level classification task using fine-tuned pre-trained transformers; and (b) a question-answering task using a generalist large language model (GPT). We also propose a rule-based approach to build a labeled dataset from the LATEX source of papers. Experimental results show that it is possible to reach high levels of precision and recall using either recent (and expensive) GPT 4 or simpler pre-trained models fine-tuned on our task.

Downloads

Next from IJCNLP-AACL 2023

Utilizing Weak Supervision to Generate Indonesian Conservation Datasets

Similar lecture

LaTeX Rainbow: Open Source Document Layout Semantic Annotation Framework from LaTeX to PDF

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from IJCNLP-AACL 2023

Utilizing Weak Supervision to Generate Indonesian Conservation Datasets

Similar lecture

LaTeX Rainbow: Open Source Document Layout Semantic Annotation Framework from LaTeX to PDF

Downloads