Netherlands

With the increasing prevalence of digitalization, the digital transformation is also reaching the building sector, which opens new opportunities for social and economic develop-ment. Therefore, the smart building concept is expanding from intelligent buildings and energy-efficient buildings, which aims to not only control devices and appliances in-sides, but also integrate virtual power plants and adapt to grid&#39;s conditions. This broad definition of the term is intended to illustrate the diversity of the challenges related to the en-ergy transition in buildings. In this thesis, a series of innova-tive approaches are applied to analyze the impacts of smart buildings in the energy system and tries to maximize the ben-efits of smart buildings by means of Natural Language Pro-cessing (NLP) and Explainable Artificial Intelligence (XAI) with human computation. Apart from the social trend mining for smart buildings, this thesis focuses on optimizing energy consumption behaviors in smart buildings based on user in-teraction and engaging the public to apply smart building ser-vices. Furthermore, a big data platform is established to inte-grate the impact analysis of above functions with correspond-ing interactive visualization for better reliability and inter-pretability of policymaking.

HCOMP 2020

The Role of Energy Efficient Smart Buildings in the Transformation of the Energy System: a Data Science Analysis from Social and Technical Perspectives

human-in-the-loop machine learning

energy efficient smart buildings

user engagement

ECAI2020 featured a full programme of technical papers, as well as many AI-related events of interest for researchers, students and all attendants who are interested on contemporary AI, including:




**An extensive programme of workshops, tutorials, and invited speakers**

**The Prestigious Applications of Intelligent Systems conference (PAIS)**

**The Frontiers in AI track sessions**

**The Lab To Market event**

**The Women in AI time**

**The EU Challenges forum**

… and many more!

ECAI 2020

ECAI 2020 featured a full programme of technical papers, as well as many AI-related events of interest for researchers, students and all attendants who are interested on contemporary AI.

With the increasing prevalence of digitalization, the digital transformation is also reaching the building sector, which opens new opportunities for social and economic develop-ment. Therefore, the smart building concept is expanding from intelligent buildings and energy-efficient buildings, which aims to not only control devices and appliances in-sides, but also integrate virtual power plants and adapt to grid's conditions. This broad definition of the term is intended to illustrate the diversity of the challenges related to the en-ergy transition in buildings. In this thesis, a series of innova-tive approaches are applied to analyze the impacts of smart buildings in the energy system and tries to maximize the ben-efits of smart buildings by means of Natural Language Pro-cessing (NLP) and Explainable Artificial Intelligence (XAI) with human computation. Apart from the social trend mining for smart buildings, this thesis focuses on optimizing energy consumption behaviors in smart buildings based on user in-teraction and engaging the public to apply smart building ser-vices. Furthermore, a big data platform is established to inte-grate the impact analysis of above functions with correspond-ing interactive visualization for better reliability and inter-pretability of policymaking.

technical paper

The 8th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2020) will be held October 26–28th virtually at The Netherlands Institute for Sound and Vision in Hilversum, The Netherlands.

While artificial intelligence (AI) and human-computer interaction (HCI) represent traditional mainstays of the conference, HCOMP believes strongly in fostering and promoting broad, interdisciplinary research. Our field is particularly unique in the diversity of disciplines it draws upon and contributes to, including human-centered qualitative studies and HCI design, social computing, artificial intelligence, economics, computational social science, digital humanities, policy, and ethics. We promote the exchange of advances in human computation and crowdsourcing not only among researchers, but also engineers and practitioners, to encourage dialogue across disciplines and communities of practice.

This year, we especially invite works that will generate new insights into the quality (or qualities) of human-annotated datasets, including elements such as reliability and replicability of human computation and crowdsourcing experiments, novel metrics for aggregation of results, and holistic approaches to deal with bias, fairness and interpretability.

HCOMP believes strongly in fostering and promoting broad, interdisciplinary research. Our field is particularly unique in the diversity of disciplines it draws upon and contributes to, including human-centered qualitative studies and HCI design, social computing, artificial intelligence, economics, computational social science, digital humanities, policy, and ethics. 


Redaction of private information from images is the kind of tedious, yet context-independent, task for which crowdsourcing is especially well suited. Despite tremendous progress, machine learning is not keeping pace with the needs of sensitive applications in which inadvertent disclosure could have real-world consequences. Human workers can detect faces that machines cannot; however, an open call to crowds would entail disclosure. We present IntoFocus, a method for engaging crowd workers to redact faces from images without dis-closing the facial identities of people depicted. The method works iteratively, starting with a heavily filtered form of the image, and gradually reducing the strength of the filter, with a different set of workers reviewing the image at each step. IntoFocus exploits the gap between the filter level at which a face becomes unidentifiable and the level at which it becomes undetectable. To calibrate the algorithm, we performed a perceptual study of detection and identification of faces in images filtered with the median filter. We present the system design, the results of the perception study, and the results of a summative evaluation of the system.

Privacy-Preserving Face Redaction using Crowdsourcing

While most user content posted on social media is benign, other content, such as violent or adult imagery, must be detected and blocked. Unfortunately, such detection is difficult to automate, due to high accuracy requirements, costs of errors, and nuanced rules for acceptable content. Consequently, social media platforms today rely on a vast workforce of human moderators. However, mounting evidence suggests that exposure to disturbing content can cause lasting psychological and emotional damage to some moderators. To mitigate such harm, we investigate a set of blur-based moderation interfaces for reducing exposure to disturbing content
whilst preserving moderator ability to quickly and accurately flag it. We report experiments with Mechanical Turk workers to measure moderator accuracy, speed, and emotional well-being across six alternative designs. Our key findings show interactive blurring designs can reduce emotional impact without sacrificing moderation accuracy, and speed.

Fast, Accurate, and Healthier: Interactive Blurring Helps Moderators Reduce Exposure to Harmful Content

Earlier research has shown the promise of enabling worker interactions in crowdwork to mitigate worker biases and improve the quality of crowdwork. In this study, we focus on one characteristic of the interacting workers that may influence the effectiveness of worker interactions in enhancing crowdwork-the diversity of perspectives that the interacting workers bring together—and we explore whether and how interactions between a set of workers holding different perspectives can help mitigate biases in crowdwork. Through two sets of randomized experiments, we find that whether interactions between workers with different perspectives can help mitigate biases in crowdwork depends on task properties. We also find no conclusive evidence in our experimental settings suggest-ing that interactions among workers with diverse perspectives reduce biases in crowdwork to a larger extent compared to interactions among workers with similar perspectives.

Does Exposure to Diverse Perspectives Mitigate Biases in Crowdwork? An Explorative Study

The title of this presentation is "Modeling Annotator Perspective and Polarized Opinions to Improve Hate Speech Detection". In this presentation, An approach is introduced which exploit the fine grained knowledge expressed by individual human annotators during a hate speech (HS) detection task, before the aggregation of single judgments in a gold standard dataset eliminates non-majority perspectives. We automatically divide the annotators into groups, aiming at grouping them by similar personal characteristics (ethnicity, social background, culture etc.). 
To serve a multi-lingual perspective, we performed classification experiments on three different Twitter datasets in English and Italian languages. We created different gold standards, one for each group, and trained a state-of-the-art deep learning model on them, showing that supervised models informed by different perspectives on the target phenomena outperform a baseline represented by models trained on fully aggregated data. Finally, we implemented an ensemble approach that combines the single perspective-aware classifiers into an inclusive model. The results in the presentation show that this strategy further improves the classification performance, especially with a significant boost in the recall of HS prediction.

Modeling Annotator Perspective and Polarized Opinions to Improve Hate Speech Detection

Given a set of ideas collected from crowds with regard to an open- ended question, how can we organize and prioritize them in order to determine the preferred ones based on preference comparisons by crowd evaluators? As there are diverse latent criteria for the value of an idea, multiple ideas can be considered as “the best”. In addition, evaluators can have different preference criteria, and their comparison results often disagree. In this paper, we propose an analysis method for obtaining a subset of ideas, which we call frontier ideas, that are the best in terms of at least one latent evaluation criterion. We propose an approach, called CrowDEA, which estimates the embeddings of the ideas in the multiple-criteria preference space, the best viewpoint for each idea, and preference criterion for each evaluator, to obtain a set of frontier ideas. Experimental results using real datasets containing numerous ideas or designs demonstrate that the proposed approach can effectively prioritize 
ideas from multiple viewpoints, thereby detecting frontier ideas. The embeddings of ideas learned by the proposed approach provide a visualization that facilitates observation of the frontier ideas. In addition, the proposed approach prioritizes ideas from a wider variety of viewpoints, whereas the baselines tend to use to the same viewpoints; it can also handle various viewpoints and prioritize ideas in situations where only a limited number of evaluators or labels are available.

CrowDEA: Multi-view Idea Prioritization with Crowds

Mixed-initiative systems allow users to interactively provide feedback to potentially improve system performance. Human feedback can correct model errors and update model parameters to dynamically adapt to changing data. Additionally,many users desire the ability to have a greater level of control and fix perceived flaws in systems they rely on. However,how the ability to provide feedback to autonomous systems influences user trust is a largely unexplored area of research.Our research investigates how the act of providing feedback can affect user understanding of an intelligent system and its accuracy. We present a controlled experiment using a simulated object detection system with image data to study the effects of interactive feedback collection on user impressions.The results show that providing human-in-the-loop feedback lowered both participants’ trust in the system and their perception of system accuracy, regardless of whether the system accuracy improved in response to their feedback. These results highlight the importance of considering the effects of allowing end-user feedback on user trust when designing intelligent systems.

Soliciting Human-in-the-Loop User Feedback for Interactive Machine Learning Reduces User Trust and Impressions of Model Accuracy

Domain-specific intelligent systems are meant to help system users in their decision-making process. Many systems aim to simultaneously support different users with varying levels of domain expertise, but prior domain knowledge can
affect user trust and confidence in detecting system errors. While it is also known that user trust can be influenced by first impressions with intelligent systems, our research explores the relationship between ordering bias and domain expertise
when encountering errors in intelligent systems. In this paper, we present a controlled user study to explore the role of domain knowledge in establishing trust and susceptibility to the influence of first impressions on user trust.

The Role of Domain Expertise in User Trust and the Impact of First Impressions with Intelligent Systems

We present a series of ultimatum bargaining game experiments comprising 1178 participants. We find that users are willing to use a black-box decision support system and thereby make better decisions. This translates into higher levels of cooperation and better market outcomes. However, because users under-weigh algorithmic advice, market outcomes remain far from optimal. Explanations
increase the number of unique system inquiries, but users appear less willing to follow the system’s recommendation. People who negotiate with a user who has a decision support system, but cannot use one themselves, react to its introduction by demanding a better deal for themselves, thereby decreasing overall cooperation levels. This effect is largely driven by the percentage of participants who perceive the system’s availability as unfair. Interpretability mitigates perceptions of unfairness. Our findings highlight the potential for decision support systems to further human cooperation, but also the need for regulators to consider heterogeneous stakeholder reactions.

Impact of Algorithmic Decision Making on Human Behavior: Evidence from Ultimatum Bargaining

This presentation proposes a heuristic algorithm for effectively summarizing the work of novice robot operators, e.g., ones recruited through crowdsourcing platforms, in search and rescue-like tasks. Such summaries can be used for many purposes, perhaps most notably for monitoring and evaluating operator's performance in settings where information gaps preclude automatic evaluation. The underlying idea of our method is dividing the task timeline into intervals, and extracting a subset of high-scoring and low-scoring segments within, using a heuristic scoring function. This results in a short effective summary of the operator's work, based on which several other crowdworkers can evaluate her performance. The effectiveness of the proposed method was extensively evaluated and compared to a large set of alternative methods through a series of experiments in Amazon Mechanical Turk. The analysis of the results reveals that the proposed method outperforms all tested alternatives. Finally, we evaluate the performance one may achieve with the use of machine learning for predicting the operator's performance in our domain. While this approach manages to reach a performance level similar to the one achieved with summaries, it requires an order-of-magnitude greater effort for training (measured in terms of crowdworkers time).

Effective Operator Summaries Extraction

Firms and research organizations require humans to annotate raw data to make it compatible with machine learning algorithms. These tasks are outsourced to individuals worldwide through crowdsourcing platforms, or infrastructures where labour is exchanged as a commodity. The firms that operate them consider workers as “independent contractors,” paying them low wages and denying them any rights and social protections.
This dissertation studies how this commodification of labour affects “social reproduction,” defined as reproduction of social inequalities. It argues that, despite the individualized and alienated nature of platform labour, workers are embedded in networks of trust, emerging from households, online and local communities, that provide social and economic support in “disembedded” markets unconstrained by institutional regulations. In this view, this dissertation studies the extent to which labour commodification permeates artificial intelligence development and the social experiences of platform workers.
The dissertation studies workers of platforms specialized in data annotation for machine learning, and online surveys and semi-structured interviews with workers located in Latin America. The study aims to highlight the challenges that workers and their communities need to overcome and to provide empirical evidence necessary to improve current ethical principles and public policies on artificial intelligence.

Downloads

Next from HCOMP 2020

Privacy-Preserving Face Redaction using Crowdsourcing

Similar lecture

Charge/Discharge Scheduling of Electric Vehicles and Battery Energy Storage in Smart Building: a Mix BinaryLinear Programming model

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from HCOMP 2020

Privacy-Preserving Face Redaction using Crowdsourcing

Similar lecture

Charge/Discharge Scheduling of Electric Vehicles and Battery Energy Storage in Smart Building: a Mix BinaryLinear Programming model

Downloads