Research Assistant Position: Comparing Human and AI Performance on Sentence Processing Experiments
Ethan Wilcox, Linguistics/FAS
fall 2019
 

Project Description: What do AI models of language learn about grammar and syntax? An exciting new approach involves analyzing the behavior of neural-network language models using techniques from experimental psychology, treating the models as “psycholinguistic subjects” to determine what linguistic generalizations they learn. So far, research in this area has proceeded through deep analysis of many small corners of syntax (e.g. question formation, subject/verb agreement) and many studies present machine results only. A well-rounded dataset of human performance on sentence processing tasks would allow for more direct and broad-based comparison between humans and AI. To this end, we are looking for an undergraduate researcher to help us compile a dataset of human sentence processing performance culled from the linguistics literature. The dataset will be uploaded and maintained on a website specially built for this purpose (www.syntaxgym.org). This project will give you experience reading and interpreting primary sources in the human sentence processing literature, database management and basic programming skills. Requirements: Introductory linguistics coursework and basic knowledge of experimental methods in linguistics/psychology. Basic knowledge of computer programming (specifically Python) is a plus. Students with demonstrated interest in artificial intelligence and natural language processing preferred. Additional Information: The position is volunteer and researchers are not expected to work more than 10 hours per week on the project. Subject Area: Linguistics, Psychology, Computer Science. Keywords: Sentence Processing, Artificial Intelligence, Computational Syntax, Psycholinguistic Experiments. Relevant URLs: http://staging.syntaxgym.org, http://docs.syntaxgym.org, https://arxiv.org/abs/1809.01329, and http://cpl.mit.edu. To Apply: Contact Ethan Wilcox at wilcoxeg@g.harvard.edu. (posted 9/2019)