An elite Wall Street law firm has apologized to a federal judge for submitting a court filing replete with errors created by artificial intelligence, including “hallucinations” that fabricated case citations.
The A.I.-generated errors came in a recent motion in U.S. Bankruptcy Court in Manhattan and were discovered by lawyers from an opposing firm, Andrew Dietderich, a partner at Sullivan & Cromwell, wrote in a letter to Judge Martin Glenn on April 18.
“We deeply regret that this has occurred,” Mr. Dietderich wrote.
The firm provided a ledger of the errors, which spanned three pages and totaled around three dozen. A number of them involved the citation of seemingly imagined passages from real cases.
Sullivan & Cromwell is one of the oldest and most prestigious law firms in the country. It is representing President Trump in several appeals, including his criminal conviction in 2024 in a case that stemmed from a hush-money payment to a porn star. Jay Clayton, now the U.S. attorney for the Southern District of New York, was of counsel and formerly a partner at the firm.
The apology revealed the latest embarrassing blunder for lawyers found to have used A.I. in crafting erroneous arguments. The legal profession is undergoing a reckoning over the growing and widespread use of A.I., which is luring lawyers dealing with voluminous research even as it has a propensity to spit out legal falsehoods.
A spate of cases in recent years has illuminated the dangers that using A.I. poses to lawyers. In 2023, a federal judge in Manhattan fined two lawyers $5,000 after they submitted a brief of made-up cases, concocted by ChatGPT.
The American Bar Association has instructed lawyers to exercise caution when posing prompts to A.I. models or retrieving results. Mr. Dietderich wrote in his letter that the firm’s policies governing the use of A.I. were “not followed” in preparing the motion.
It is not clear which A.I. tools or program were used by Sullivan & Cromwell in generating the errors. A spokesman for the firm declined to comment. The news of the letter was reported earlier by Reuters.
The hallucinations filed by Sullivan & Cromwell came about in a case involving the Prince Group, a Cambodian conglomerate whose founder, Chen Zhi, was indicted in Federal District Court in Brooklyn last year on charges that he operated a global scam operation.
His lawyers and representatives have denied the charges. Mr. Chen, who was not in the United States when the indictment was announced, was extradited from Cambodia to China in January.
On April 8, the Prince Group, a Cambodian conglomerate with a number of business entities incorporated in the British Virgin Islands, filed for bankruptcy in Manhattan. Sullivan & Cromwell is representing a group of people appointed by the authorities in the British Virgin Islands to oversee the Prince Group’s liquidated assets in that territory.
Some of the errors were identified by lawyers from Boies Schiller Flexner, the law firm representing the Prince Group, in a public filing. A spokesman for the firm declined to comment. After learning of the errors, Mr. Dietderich wrote, the firm conducted a review of all other filings in the case. The A.I. hallucinations were contained to the single filing, he wrote.
According to Mr. Dietderich’s letter, Sullivan & Cromwell requires its lawyers to take a training course before it gains access to A.I. tools. Among the training’s exhortations, Mr. Dietderich wrote, is to “trust nothing and verify everything.”
Santul Nerkar is a Times reporter covering federal courts in Brooklyn.
The post A.I. ‘Hallucinations’ Created Errors in Court Filing, Top Law Firm Says appeared first on New York Times.
