The Third Symposium on Biases in Human Computation & Crowdsourcing was held online between 10–12 November in Delft, The Netherlands. The symposium was organized by the Web Information Systems group, TU Delft under the umbrella of the Academic Fringe Festival (https://www.academicfringe.org) and CHI Nederland (https://chinederland.nl) with the support of the following sponsors – the Artificial Intelligence Journal (AIJ), Toloka, and the host university TU Delft.
Human Computation and Crowdsourcing have become ubiquitous in the world of algorithm augmentation and data management. However, humans have various cognitive biases that influence the way they make decisions, remember information, and interact with machines. It is thus important to identify human biases and analyze their effect on complex hybrid systems. On the other hand, the potential interaction with a large pool of human contributors gives the opportunity to detect and handle biases in existing data and systems.
The goal of this symposium was to analyze both existing human biases in hybrid systems, and methods to manage bias via crowdsourcing and human computation. Different types of biases, measures and methods to track bias, as well as methodologies to prevent and solve bias were discussed. An interdisciplinary approach is often required to capture the broad effects that these processes have on systems and people, and at the same time to improve model interpretability and systems’ fairness. BHCC 2021 provided a framework for discussion among scholars, practitioners and other interested parties, including industry, crowd workers, requesters and crowdsourcing platform managers. Contributions spanned ideas from different disciplines, including computer science, psychology, economics and social sciences.
The symposium included full and short paper presentations alongside pitches for abstracts. The program included 3 fantastic talks from invited speakers – Ming Yin from Purdue University who spoke about Mitigating Biases in Crowdsourced Data Collection, Saiph Savage from Northeastern University who spoke about the Future of AI for Social Good, and Alexandra Olteanu from Microsoft who spoke about the Foresight and Measurement of Computational Harms.
One of the most intriguing discussions over the 3 days took place at the interdisciplinary panel on “Open Science and Good Research Practice”. Samantha Teplitzsky from UC Berkley, Michael Hind from IBM, Denny Borsboom from University of Amsterdam, and Pietro Michelluci from the Human Computation Institute were the panelists who dove deep into the intricacies of the benefits of Open Science, the competing incentives that stall good research practice, and what each of us can do to propel open science as a movement that can best serve the society at large.
A number of social activities were organized to facilitate interaction among participants at the symposium, including a complete city tour of Delft from a professional tour guide held in a virtual capacity! Participants of the symposium received a pair of BHCC’21 socks to keep their feet warm and cozy during the unrelenting pandemic.
This successful event would not have been possible without the hard work behind the scenes from the organization committee; Tim Draws (social events chair), Nirmal Roy (publicity co-chair), Alisa Riger (sponsorship chair), Sihang Qiu (publicity co-chair), Kevin Roitero (proceedings chair), Sarah Luger (diversity and inclusion chair), and Ujwal Gadiraju and Jie Yang (general chairs).
Written by: Ujwal Gadiraju