This summer, lawyers at United Airlines noticed that someone had built an almost perfect replica of the company’s website.
This digital clone offered all the same buttons and menus for booking flights, hotels and rental cars. It included the same blue links for tracking frequent flier miles and browsing discount deals. It even used the United brand name and logo.
So United’s lawyers sent a formal takedown notice accusing the site of violating its copyrights.
Div Garg, whose tiny company built the replica site, promptly changed the site’s name to “Fly Unified” and removed the United logo. He was not interested in stepping on United’s copyrights. He and his company built their United.com replica as a training ground for artificial intelligence.
Mr. Garg’s company, AGI, is among a number of Silicon Valley start-ups that have spent the past several months recreating popular websites so that A.I. systems can learn to navigate the internet and complete specific tasks on their own, like booking flights. If an A.I. system learns to use a replica of United.com, it can use the real site, too.
These new shadow sites are a significant part of the tech industry’s efforts to transform today’s chatbots into A.I. agents, which are systems designed to book travel, schedule meetings, build bar charts and complete other computing tasks. In the coming years, many companies believe, A.I. agents will become increasingly sophisticated and could replace some white-collar workers.
“We want to build training environments that capture entire jobs that people do,” said Robert Farlow, whose start-up, Plato, is among those recreating popular websites and other software applications.
The new trend, fueled by Silicon Valley venture capital, shows just how far the tech industry will go in search of the enormous amounts of digital data needed to advance artificial intelligence. First, Silicon Valley hoovered up text, sounds and images from across the internet. When many sites blocked these efforts, companies found new ways of getting their hands on other people’s data. Now, they are recreating websites as a way of generating new data from scratch.
In recent months, backed by $10 million in funding from Menlo Ventures and other investors, Mr. Garg and his company have also cloned sites like Amazon, Airbnb and Gmail. With names like Omnizon, Staynb and Go Mail, these replicas provide a way for A.I. systems to learn skills through trial and error — a technique that researchers call reinforcement learning. Rather than learning from data that shows how humans use websites, they learn from vast amounts of data they generate on their own.
Silicon Valley researchers also have the option of training A.I. systems on real websites. But in many cases, that is not possible. Sites like Amazon and Airbnb often bar online bots, particularly when bots repeat the same tasks over and over again — a process that is fundamental to reinforcement learning.
“When you’re doing training, you want to run thousands of A.I. agents at the same time, so that they can explore the website and visit its different pages and do all sorts of different things,” Mr. Garg said. “If you do that on a real website, you will get blocked.”
Today’s A.I. systems are driven by what scientists call neural networks, which are mathematical systems that can identify patterns in text, images and sounds. But about nine months ago, companies like OpenAI used up just about all the English language text on the internet. So they are leaning more heavily on reinforcement learning.
This process, which can extend over weeks or months, began in areas like math and computer programming. By working through thousands of math problems, for instance, A.I. systems can learn which actions lead to the right answer and which do not.
Now, companies like OpenAI, Google, Amazon and Anthropic are using the technique to build A.I. agents.
They started by using recordings of people using real websites. By analyzing the way these hired hands used their mouse and keyboard to order lunch on DoorDash or type numbers into Microsoft Excel, the systems learned to use the sites on their own.
To make that work go faster, A.I. companies are paying little-known start-ups like AGI and Plato to build replica websites where bots can learn through extreme trial and error.
“You want the A.I. to be able to experiment with all the possible ways of completing each task,” said John Qian, whose start-up, Matrices, builds replica websites for A.I. training.
Most of this work happens behind the scenes. But in some cases, start-ups have posted their replica websites to the public internet as a way of advertising their work to the big A.I. companies like OpenAI, Google and Amazon.
After removing company names and logos from the replica sites built by his start-up, Mr. Garg said, he is not worried about further legal action from copyright holders like United Airlines. Mr. Qian said much the same, though he acknowledged that A.I. research had ventured into new legal territory that was not completely settled. Mr. Farlow declined to comment.
Robin Feldman, a professor at U.C. Law San Francisco and the author of the book “AI Versus IP,” said that using these shadow sites to train A.I. technologies could violate the copyrights of companies like United Airlines. But the courts may eventually find, she added, that this is permitted under copyright law.
“These companies are shooting first and asking questions later,” Ms. Feldman said. “The field is expanding much faster than the legal system can keep up with. Some of the decisions being made along the way end up biting the companies that have made those decisions.”
Companies like OpenAI and Anthropic have already released experimental technologies that can shop on Instacart or take notes using online word processors like Google Docs. But these technologies frequently make mistakes. Sometimes, this prevents them from completing the requested task.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied the suit’s claims.)
“There is a big gap between what companies want these agents to do and what they are capable of today,” said Rayan Krishnan, chief executive of Vals AI, a company that tests the performance of the latest A.I. technologies. “Today, these systems are way too slow for them to be useful. You can just do the clicks yourself.”
Experts disagree on how quickly this work will progress, whether consumers and businesses want or need this kind of automation and whether popular websites will even allow it to happen. Last month, Amazon sued a start-up called Perplexity over A.I. that aimed to automate shopping on the Amazon site.
But the goal is to build systems that automate almost any white-collar work.
“If you can recreate all the software and websites that people use, you can train A.I. to do the jobs and start to do them even better than a human,” Mr. Farlow said.
Cade Metz is a Times reporter who writes about artificial intelligence, driverless cars, robotics, virtual reality and other emerging areas of technology.
The post Silicon Valley Builds Amazon and Gmail Copycats to Train A.I. Agents appeared first on New York Times.
