Maria Figueiredo
Pronouns: she/her
Research Mentor(s): Jonathan Brennan
Research Mentor School/College/Department: Linguistics / LSA
Program: UROP
Session: Session 3 (11:00am – 11:50am)
Authors: Maria Figueiredo, Jonathan Brennan
Abstract
Deep learning transformer neural networks have been gaining attention as computational models of artificial intelligence that seem to understand, process, and react to natural language. Thus, there is an increased interest in examining the underlying similarities between these neural networks and the human brain when presented with the same linguistic input. In this study, we developed a Python-based pipeline to explore this relationship by implementing a method called Representational Similarity Analysis (RSA), which relies on measures of cosine similarity values to quantify correlations between elements. Text from an audiobook story was presented to human participants and to an artificial neural network. Brain activity in the occurrence of single words, delivered as auditory stimuli, was examined using electroencephalography (EEG). The text was tokenized and processed by OpenaAI’s Generative Pre-Trained Transformer 2 (GPT-2) language model, with activation values extracted from output layers of its 12 decoder blocks. We then computed cosine similarity between all words for a given neural network layer and, separately, for a given time-point in the EEG data. Correlation between those two matrices was used to assess the similarity of representations between the artificial neural network and human brain signals. The study tests potential links between both types of datasets. We probe how the relationship between artificial neural network and human data changes across layers for the transformer, and temporal dynamics during language processing. For instance, we test the hypothesis that an activation value at higher neural network layers may show closer similarity with late, rather than early, brain activity.
