In July, Jacob Tsimerman, a professor at the University of Toronto, received what is arguably math’s top prize: the Fields Medal, which honors the rarefied pursuit of mathematical truths for their own sake, irrespective of real-world utility.
Upon receiving the prize, Dr. Tsimerman announced a professional pivot, to the dismay of many in the field: He was taking a leave from academia, mostly giving up pure mathematical pursuits and redirecting his research toward solving problems in A.I. safety. He starts a job in the safety department at OpenAI later this month.
And on Tuesday, Dr. Tsimerman announced the formation of the Mathematical A.I. Safety Institute (MAISI), an independent research institution unaffiliated with OpenAI. Located in the Bay Area, MAISI will begin its first full semester of research in January 2027, aiming to hire 10 to 30 mathematicians, and significantly more the following academic year.
With MAISI, Dr. Tsimerman, who will serve as scientific director, brings a message to the academic community: “A.I. safety is a meaningful area of research,” he said in an interview from Toronto. “It is a legitimate intellectual pursuit that is interesting, useful, and there is real progress to be made.”
Over the years, despite advancements in artificial intelligence, many mathematicians did not take the technology seriously, considering it instead to be the purview of software engineers. But, as Dr. Tsimerman noted, “Engineers can only implement ideas that have been developed, and we need new ideas.” Since A.I. systems are fundamentally made from math — linear algebra, calculus, probability, statistics — he and like-minded mathematicians view some problems in A.I. safety as fundamentally math problems.
“We need more of that sort of mathematical clarity and carefulness in the A.I. industry,” said Andrew Critch, MAISI’s executive director, an A.I. researcher and mathematician who has studied the algebraic geometry of machine-learning models. “That’s how nuclear energy works: We do a lot of math even before turning on a power plant for the first test run.”
Mathematically trained researchers see an opportunity for higher math to address concerns such as how A.I. systems interact and cooperate with one another; how agents verify (that is, prove mathematically) that they are behaving responsibly and that outputs are accurate; and how A.I. systems can be designed from the bottom up to be resilient against even currently unidentified vulnerabilities and risks.
Dr. Tsimerman pointed to a cryptography tool called zero-knowledge proofs — a mathematical way of providing verification about a system without revealing more than necessary, such as proprietary details about model parameters or training data. Zero-knowledge proofs could help underpin protocols for human trust of A.I. agents, allowing people to verify that a system has done what it claims.
The computer scientist Shafi Goldwasser, who in the 1980s co-invented zero-knowledge proofs (and many other important ideas in modern cryptography), said this tool was “essentially a way of proving facts without giving your secret sauce away.”
Dr. Goldwasser is a co-founder of a similar math-oriented A.I. safety institute — the Institute for Responsible Superintelligence (RESI) — which launched in August in Cambridge, Mass. RESI’s founding scientific team includes Vinod Vaikuntanathan, an M.I.T. cryptographer, and Adam Tauman Kalai, an A.I. safety and ethics researcher who recently left OpenAI. At their first meeting last week, the team discussed how the methods they contemplate should inherently be general enough to scale up with A.I. model capabilities and intelligence.
“It’s a giant, growing ecosystem,” Lionel Levine, a mathematician at Cornell University, said about the interconnected community of math thinkers now focusing on the risks of artificial intelligence. Dr. Levine recently created an online repository of open problems and research agendas, inviting mathematicians to start doing this kind of work. “We need a lot of third-party checks and balances,” he said.
In founding MAISI, Dr. Tsimerman also has a message for the public, about what standard of A.I. safety it should expect. Pushing back against the view that “it’s not reasonable to expect a high level of safety from A.I.,” he insists that “it’s reasonable and in fact necessary, and we should be demanding a much, much higher level of safety standard than we’re currently getting.”
Ravi Vaki, a mathematician at Stanford and a scientific adviser to MAISI, characterized the problem as one that every group of relevant experts needed to prioritize. “We need to approach this with everything at our disposal,” he said. “We need multiple points of attack.”
“Math might be the silver bullet, or it might not be,” Dr. Vakil added. “But maybe we just need a lot of bullets; this is one bullet.”
The post Top Mathematician Announces New Institute for A.I. Safety appeared first on New York Times.
