Budding never planned to pursue a PhD. “I had done a technical master’s and spent all my time programming — I was pretty much done with it after that,” she says. Until she stumbled upon this project. Rather than building yet another model, the research focuses on understanding and explaining what existing models do. Less computation, more reflection — with clear ties to philosophy. Because how do you explain what a language model actually learns?

Large Language Models

Large Language Models, or LLMs, are computer programs trained to process and generate language. They can write texts, answer questions, or produce summaries. The best-known example is ChatGPT. What they fundamentally do is recognize patterns in language: by observing which words frequently appear together and how sentences are usually structured, they learn which words are likely to follow.

The learning process involves processing massive amounts of text — books, articles, websites. The model repeatedly tries to predict the most likely next word in a sequence, and by doing this millions of times, it becomes increasingly skilled at detecting linguistic patterns. Sometimes LLMs receive additional fine-tuning for specific tasks, such as translation or summarization.

Tacit knowledge

“These models can do so much that people sometimes assume they have knowledge — or even intelligence. But is that really the case? And if we do say they have knowledge, what exactly do we mean by that?” Budding explains. She examined the role knowledge plays in human linguistic behavior, diving into philosophical literature. That led her to the concept of tacit knowledge: knowledge we possess but find difficult to articulate. Humans, for example, rely on grammatical knowledge to speak a language, but they apply it unconsciously and rarely can explain its logic.

Language models aren’t given explicit rules or facts; everything they “know” comes from the patterns embedded in their training data. So the real question is: what, exactly, have they learned? And could language models also possess some form of tacit knowledge? If you ask ChatGPT for the capital of France, it replies “Paris.” But does it understand that Paris is located in France and that the two concepts are connected? Or is it simply reproducing a frequently occurring question–answer pattern?

Parrots or thinkers?

The question of whether AI models truly understand anything — or merely repeat patterns — has been around for a long time. Some researchers call AI language models stochastic parrots: parrots that mimic everything without true comprehension. Others attribute qualities like “smart” or “intelligent” to them. Budding argues against buying into the hype and instead urges careful examination of what the models actually do. “We need to study this thoroughly and justify our claims before we start putting labels on anything,” she says.

That’s no small task: these models contain millions of parameters, and the first challenge is identifying what to look for when talking about knowledge. “In my dissertation, I make a proposal for that,” Budding says. She presents a theoretical framework that sets out clear criteria for tacit knowledge. If a language model meets all of these criteria, we can say it has tacit knowledge.

According to her, we should focus on whether models form connections between different inputs — the way humans do. “That way, we can determine whether a model truly knows something about Paris, or whether it treats every question about Paris independently and simply predicts the correct answer.”

Interventions

To determine whether a model truly possesses knowledge, Budding argues we need to look for internal representations of concepts — such as “Paris” or a grammatical rule — inside the model itself. In her thesis, she proposes doing this through interventions. The idea is to modify a model in a controlled way and then observe how its predictions change.