EPFL researchers have developed the first AI model of language in the brain that captures both how neurons are arranged and how they function.
Our brain is a very organized place. Neurons – the nerve cells responsible for transmitting electrical and chemical signals throughout the body – are organized on tissue, tending to cluster together in groups according to how they function. For example, language studies have found that there are clusters of neurons that seem to specialize in verbs and others that focus specifically on nouns, however researchers aren’t sure just how these functional groups form.
Previous AI language models have been able to successfully capture these individual clusters of functional neurons but haven’t looked at how these are spatially arranged in the brain.
Now, researchers from the NeuroAI Laboratory, part of both the School of Computer and Communication Sciences (IC) and the School of Life Sciences (SV), have created TopoLM, a new AI language model that mimics both the functional clustering of neurons and, for the first time, how these nerve cells are spatially arranged within the brain.
“Building on work around how the brain processes vision, we made some relatively small changes to how a language model organizes itself internally, adding a rule that encourages the model’s internal representations to be ‘smooth’ spatially. The resulting model TopoLM develops spatial clusters of its internal components that functionally match the activity we see in human brains when they process language,” explained Assistant Professor Martin Schrimpf, Head of the NeuroAI Lab.
In their paper, TopoLM: Brain-Like Spatio-Functional Organization in a Topographic Language Model, one of the less than 2% of papers selected to be presented orally this week at ICLR 2025, the International Conference on Learning Representations, the researchers outline how TopoLM successfully predicted how the language system in the brain’s outer layer, the cortex, develops its spatio-functional organization.
“This was basic research to understand how spatial clusters or functional clusters in the brain originate in the first place and what this new model suggests is that it might be driven by a single, basic rule about spatial organization where close by neurones just tend to behave similarly,” continued Schrimpf.
The researchers believe TopoLM provides a framework for improving AI’s functional alignment with human cognition, offering direct applications to developing brain-inspired computing and neurolinguistics.
“This is an exciting step towards building artificial intelligence systems that are organized more like the human brain. One of our main goals is to build better models of the brain in general and with TopoLM we are one step closer to clinical applications that might help us to help people with language disorders or similar language deficits,” said Badr AlKhamissi, a doctoral assistant in the NeuroAI Lab and IC’s Natural Language Processing Laboratory (NLP) and one of the authors of the paper.
This work also sheds light on interpretability, basically the understanding of what happens inside complex Large Language Models (LLMs) and how they actually work. Normally, LLMs have mathematical representations called ‘vectors’ associated with each artificial ‘neuron’. To understand what the model has learned, it would be necessary to analyse each vector individually or in small groups.
Because TopoLM organizes its internal components into clusters, it’s possible to look at these and see meaningful groupings emerge. This makes it more straightforward to get a sense of how the model is representing and processing language because the organization itself reflects meaningful categories.
So what’s next in this cutting-edge research? The EPFL research team will now work to test the predictions of the model in human brains.
“This model works so well, far better than we expected, that we want to test if its predictions are true. There are clusters in the model that we have not yet observed in the human brain because nobody has looked for them yet. We’ll be working with colleagues in the United States who experimentally work with humans to run a new imaging study to find out if these clusters also appear in the human brain,” concluded Schrimpf.
Badr AlKhamissi, Neil Rathi and Hannes Mehrer all worked with Martin Schrimpf on developing the TopoLM model.
The NeuroAI Laboratory is part of EPFL’s Neuro-X institute, a collaborative and interdisciplinary community that brings together teams from EPFL’s School of Computer and Communication Sciences, The School of Life Sciences and the School of Engineering.
Author: Tanya Petersen
Source: EPFL