Two federal judges admit to using AI in botched rulings after months of silence

Two federal judges have publicly acknowledged using artificial intelligence in their work, leading them to retract court orders after the AI introduced errors into their rulings.

The summer rulings by U.S. District Judges Julien Xavier Neals in New Jersey and Henry T. Wingate in Mississippi had all the hallmarks of AI “hallucinations.” That was speculation until they both confirmed it in response to an inquiry from Senate Judiciary Committee Chairman Charles E. Grassley, Iowa Republican.

One judge said a law school intern wrongly used the AI, and the other said it was a clerk in his office. In both cases, the judges signed the final rulings.

Both men wrote to the office that oversees U.S. courts to say they had taken steps to prevent a repeat.

Mr. Grassley praised the judges for their honesty and urged other judges to learn from the fiasco.

“We can’t allow laziness, apathy or overreliance on artificial assistance to upend the judiciary’s commitment to integrity and factual accuracy. As always, my oversight will continue,” he said.

In their responses, the judges made clear how much the AI world has become the Wild West.

Judge Wingate said his office didn’t have any rules about generative AI. Judge Neals said he has a policy forbidding generative AI, including the ChatGPT used by the law school intern. Judge Neals said the student’s university also had a policy barring AI use, and the student appears to have violated that policy.

AI hallucinations result when a generative AI tool fabricates a legal citation or argument. Common hallucinations include making up case names, misstating the outcomes of rulings and manufacturing false quotes from legitimate decisions.

For several years, the legal world has been rife with stories of AI hallucinations making their way into attorneys’ briefs, and some judges have been stern in imposing sanctions on lawyers who have filed AI-polluted briefs.

However, having two federal judges file rulings with AI-induced errors and then refuse for months to acknowledge the causes of the mistakes took the problems to a new level.

Mr. Grassley said it threatened to undermine the credibility of the courts and potentially undermine litigants’ rights.

Susan Tanner, a law professor at the University of Louisville in Kentucky, said the two judges’ explanations should be pivotal for the judiciary.

“The legal system depends on careful deliberation and verification. Generative AI, on the other hand, is built for speed and fluency,” she said. “That mismatch between AI’s quick confidence and the slow, careful work of legal reasoning means we need to be really thoughtful about how we integrate this technology, not just reactive.”

Damien Charlotin, a researcher and lecturer at HEC Paris, maintains a database of AI hallucinations in court cases. In addition to the two federal judges, he lists a case from a state judge in Georgia and 118 instances in which attorneys may have used AI fabrications in their briefs.

Judge Neal, a Biden appointee, issued an opinion in June that wrongly cited the outcomes of previous precedents and used quotes from other cases that never appeared in those earlier rulings. The judge also wrongly attributed statements to litigants.

Judge Wingate, a Reagan appointee, issued a restraining order in July against Mississippi’s law limiting the teaching of diversity, equity and inclusion principles in schools. His opinion fabricated text of state law and cited people who weren’t parties to the case as litigants.

He previously called the issues “clerical errors.”

Mr. Grassley’s letter prodded the judges to own up.

Judge Wingate said his erroneous opinion was posted prematurely and should have gone through more verification, including a citation-checking tool. He said that was why he called it a clerical error and didn’t attribute it to AI.

Judge Neal’s office previously leaked to a reporter that a “temporary assistant” had used AI. In his new letter, he said the law student used ChatGPT to do legal research. He said the draft opinion that included the research was then posted before going through review and citation-checking.

“It was a draft that should have never been docketed,” he said.

Mr. Grassley wondered why the judges removed the erroneous opinions and replaced them with updated rulings, which could be seen as an attempt to hide the mistakes.

Both judges said it would be wrong to leave flawed opinions on the public docket, but both said the original botched rulings remain with their clerks’ offices.

The Administrative Office of the U.S. Courts, which oversees federal judicial operations, is reviewing how AI is used in courts.

Director Robert J. Conrad Jr., a former federal judge, said AI presents “opportunities” and “concerns” for the courts. His office issued interim guidance in late July after the two judges’ bungled rulings, calling for accountability in using AI but not discouraging it.

The guidance called for courts to review all AI-generated content and to be wary of delegating to AI “core judicial functions,” such as actual decisions in cases. It also recommended that judges consider disclosing the use of AI.

Ms. Tanner said written policies aren’t enough.

“What’s really needed is ongoing training and a culture where people learn to use AI carefully and thoughtfully,” she said in an email. “Policies can set boundaries, but without real education about how these models actually work (what they’re good at, where they fall short, and how they can make stuff up while sounding totally confident), users are left thinking they’re safer than they really are.”