The MBB Lunch Series is free and open to the Harvard community.

 

Mitigating One-Sided Biases in Natural Language Understanding Datasets
Yonatan Belinkov
Postdoctoral Fellow, Computer Science
A primary goal of computational linguistics and natural language processing is to design machines that can understand human language and perform specific natural language understanding (NLU) tasks. Many such tasks consist of identifying the relationship between two objects, such as a paragraph and a question (reading comprehension), an image and a question (visual question answering), or a premise and a hypothesis (natural language inference). These tasks supposedly require a deep understanding of the information in the two objects and inference of the relationship between them. However, recent work has demonstrated that many NLU datasets contain one-sided biases—artifacts that allow models to achieve non-trivial performance by only considering one of the two objects. For instance, in natural language inference (NLI), models trained only on the hypothesis significantly outperform majority baselines, without learning whether a premise entails a hypothesis, and in visual question answering (VQA) many questions can be answered without looking at the image. This state of affairs poses a significant challenge to the NLU community: How should we handle such biases? In this talk, I will present a strategy for training models that are more robust to such biases and better transfer across datasets. The key idea is to encourage the models not to ignore the other object in the relationship (such as the premise in NLI). In practice, this results in an adversarial game between two subnetworks, one learning the full task and one the one-sided task. Time permitting, I will demonstrate the effects of this approach in the context of NLI and VQA, by analyzing the learned representations and evaluating the ability of the proposed models to generalize to other datasets.

Skepticism (or Lack Thereof) in Small Scale Societies in Southwest China
Kevin Hong
Graduate Student, Human Evolutionary Biology
Skepticism is an important epistemic attitude in human knowledge acquisition. Though skepticism has been extensively studied in contemporary, western populations as well as historically literate cultures, little is known about the form and degree of skepticism in traditional, simple societies. Here I present data showing that a substantial proportion of people in rural, southwest China readily acknowledge that they are not certain whether other people's words is true, and frequently use personal experience to "verify" (and very occasionally "falsify") such transmitted beliefs in the domain of supernatural entities, divination/magic, dream and pregnancy taboos. Instead of active hypothesis testing, individuals in SW china mostly utilize a very passive form of hypothesis checking, and end up verifying most of the transmitted beliefs.