Across from the Florence Cathedral in Italy stands a much older church, the Baptistery of San Giovanni. It is a beloved center of religious life, where many Florentines are baptized to this day. Staid columns and lively arches hug its eight sides, half-camouflaged in patterns of green and white marble. Without the baptistery’s emulation of the architecture of ancient Rome, it’s hard to imagine Florence birthing the architectural Renaissance that changed the face of Europe. Yet for centuries, there has been no compelling solution as to who built it and when and for what reasons. Decades ago, I gave tours of the baptistery and came to revere it, and in the early 2020s I began delving into its origins.
After years of poring over historical documents and reading voraciously, I made an important discovery that was published last year: The baptistery was built not by Florentines but for Florentines — specifically, as part of a collaborative effort led by Pope Gregory VII after his election in 1073. My revelation happened just before the explosion of artificial intelligence into public consciousness, and recently I began to wonder: Could a large language model like ChatGPT, with its vast libraries of knowledge, crack the mystery faster than I did?
So as part of a personal experiment, I tried running three A.I. chatbots — ChatGPT, Claude and Gemini — through different aspects of my investigation. I wanted to see if they could spot the same clues I had found, appreciate their importance and reach the same conclusions I eventually did. But the chatbots failed. Though they were able to parse dense texts for information relevant to the baptistery’s origins, they ultimately couldn’t piece together a wholly new idea. They lacked essential qualities for making discoveries.
There are a few reasons for this. Large language models have read more text than any human could ever hope to. But when A.I. reads text, it’s merely picking up patterns. Peculiar details, outlier data and unusual perspectives that can influence thinking can get lost. Without eccentric or contrarian ideas, I never would have made my discoveries. For example, in his 2006 book “Toscana Romanica,” Guido Tigler, a professor at the University of Florence, argued the baptistery was built later than generally believed. It’s an idea that’s not widely accepted, and I believe that’s the reason the chatbots never presented it to me when I asked them what they would read to solve the enigma of the baptistery. Although I ultimately found reason to reject the later dating, Mr. Tigler’s unorthodox ideas taught me to more strongly consider the possibility that past scholarship had gotten the timeline for the baptistery wrong.
For centuries, many people believed Pope Nicholas II consecrated the baptistery in 1059. There is actually no known record of such an event; its existence is based on an assumption drawn from documents that show his involvement with other Florentine churches that year. When I nudged the chatbots to discover this discrepancy themselves, ChatGPT and Claude found it but failed to observe that it was suspicious, whereas Gemini hallucinated evidence that would eliminate this discrepancy. To contribute to a field of knowledge, you need to accurately survey the landscape, sniff out what’s fishy and demonstrate why it’s rotten. Large language models have trouble on all three counts.
And here’s the deeper problem: Sometimes pattern recognition, human and machine, is wrong. Though there was no confirming evidence, most scholars had simply assumed the patrons of the baptistery were Florentine. After all, a vast majority of church building in the Middle Ages was driven by local people: bishops, abbots, wealthy families. But from my readings I began to agree more and more with a fringe view that the inhabitants of 11th-century Florence were still too poor and provincial to produce such an accomplished building.
The key to identifying who built the baptistery was how much its architecture is inspired by the ancient Pantheon in Rome. By the 11th century, the Pantheon had become a church officiated only by the pope. Once you take Pope Nicholas out of the equation and focus on pontiffs obsessed with ancient Rome, only one name for our mystery patron comes to mind: Gregory VII.
A few years before Gregory’s election in 1073, Florentines had stopped having their children baptized in Florence, fearful that a reputedly corrupt bishop could not protect their infants’ souls. After an event proved the bishop’s unworthiness and sent him packing, the formidable rulers of Florence (and all of Tuscany), Beatrice of Bar and her daughter Matilda, seem to have made amends to the city by working with Gregory to give it a magnificent new baptistery. The sumptuous evocation of Roman splendor in the heart of Florence is exactly the kind of church architecture Gregory would have patronized.
Synthesizing so many pieces of medieval history into a new interpretation required stepping back and reconsidering their importance and how they relate to one another. A.I. may be able to optimize the process of collecting those pieces, but discovery means drawing new connections — something far beyond current A.I. capabilities, as the tests I did confirmed to me.
Discovery remains a human endeavor and is propelled by the very human quality to see oddities that don’t fit patterns and by probing them more deeply.
Elon Danziger is the director of technology at the nonprofit Global Strategies and previously worked at the National Gallery of Art. His book “Florence Rising: The Birth of the Baptistery and the Making of a City” is forthcoming.
The Times is committed to publishing a diversity of letters to the editor. We’d like to hear what you think about this or any of our articles. Here are some tips. And here’s our email: [email protected].
Follow the New York Times Opinion section on Facebook, Instagram, TikTok, Bluesky, WhatsApp and Threads.
The post I Asked ChatGPT to Solve an 800-Year-Old Italian Mystery. What Happened Surprised Me. appeared first on New York Times.
