Adam Aleksic is the author of “Algospeak: How Social Media Is Transforming the Future of Language.”
As more of the internet is saturated with artificially generated writing, people are turning to artificial intelligence detection tools promising to verify human-written text. But human language is converging with machine language, meaning that these services will get less reliable over time.
AI companies say the detection tools are very reliable — at least in their controlled testing environments. Though critics question those claims, Substack recently signed a partnership with Pangram, the leader in the detection space, to help “scan for AI text” on certain posts.
The idea behind the software is to use the architecture powering large language models to identify a unique “linguistic fingerprint” behind each LLM. But the models are constantly improving, and there is mounting evidence that regular people are adopting AI linguistic patterns, such as overusing the words “delve” and “meticulous.” Assuming this observation holds across other dimensions of language, such as sentence structure, the human and chatbot dialects may continue to converge until they are much harder to distinguish.
A new paper from researchers at the Max Planck Institute for Human Development suggests that, as future models are trained on human language imitating that of AI, two possibilities emerge for language: Either human speech and AI speech progressively assimilate, causing a collapse in any meaningful distinction, or language becomes a contested social signal. This second scenario would essentially be a game of whack-a-mole across the English language, where humans keep coming up with new ways to differentiate themselves from the machines, and then the machines catch on, and then the AI detectors catch on.
If the arms race continues to escalate, AI detectors might accidentally reward those able to participate in the elite countersignaling against large language models. Meanwhile, people with less resources would be left behind. Those less able to understand the current litmus test become vulnerable to false positives, which have the potential to destroy reputations and careers.
To be clear, I think these tools are excellent stopgap solutions while society grapples with the cultural upheavals of AI. But even Pangram CEO Max Spero acknowledged to me that they “don’t have a great answer for how data drift is affecting the model.”
The company’s human-written training set is entirely based on pre-2022 data to avoid cross contamination since ChatGPT became available. In practice, this means that more innocent people will be accused of using AI as time progresses, all while Pangram’s accuracy claim, already under fire, becomes less reliable.
Spero assures me that the company will find clean human data to analyze, but this would ironically raise the false negative rate, since more people will be talking like AI and the model will associate those features with human authors.
Just as more AI text slips through the cracks, detectors will also be plagued with increasingly sophisticated adversarial tools. New research suggests that, when given access to Pangram’s API, an adaptive humanizer service can evade detection 76 percent of the time, and models fine-tuned on existing authors’ works pass as authentic 97 percent of the time. These advanced loopholes are currently out of scope for the service, meaning that the most malicious actors (such as foreign disinformation accounts) will avoid punishment.
Since there is now a market niche for sophisticated cheating software, I find it probable that such a service will eventually be widely available. Then future AI detection models will be forced to tackle these problems, where they risk accelerating a cat-and-mouse game that penalizes naiveté over anything else.
Between the constantly moving goalposts and the growing difficulty of accurate measurement, care should be taken to avoid conducting baseless witch hunts — particularly while the guiltiest people might go free. Instead, attention should be focused on the structural incentives behind LLM-generated text. Amid the infighting over textual purity, individual users are being blamed for AI slop instead of the social media platforms that motivate its distribution. This prioritizes the symptom over the disease.
The current interlude presents an opportunity to reevaluate human relationships with language, art and knowledge — making sure that the tools employed are in harmony with the way people experience meaning.
The post Why tools to detect AI-generated text are doomed appeared first on Washington Post.
