Police responded to the Florida middle school minutes after the alert arrived last week: Security cameras had detected a man in the building, dressed in camouflage with a “suspected weapon pointed down the hallway, being held in the position of a shouldered rifle.”
The Oviedo school went into lockdown. An officer searched classrooms but couldn’t find the person or hear any commotion, according to a police report.
Then dispatchers added another detail. Upon closer review of the image flagged to police, they told the officer, the suspected rifle might have been a band instrument.
The officer went to where students were hiding in the band room. He found the culprit — a student wearing a military costume for a themed dress-up day — and the “suspected weapon”: a clarinet.
The gaffe occurred because an artificial-intelligence-powered surveillance system used by Lawton Chiles Middle School mistakenly flagged the clarinet as a weapon, according to ZeroEyes, the security company that runs the system and contracts with Lawton Chiles’s school district.
Like a growing number of school districts across the country, Seminole County Public Schools has turned to AI-powered surveillance to bolster campus security. ZeroEyes sells a threat-detection system that scans video surveillance footage for signs of weapons or contraband and alerts law enforcement when they are spotted. The appetite for such systems has grown in an era of frequent, high-profile school shootings — such as the attack at Brown University on Saturday that killed two students and injured nine.
Some school safety and privacy experts said the recent incident at the Florida middle school is part of a trend in which threat detection systems used by schools misfire, putting students under undue suspicion and stress.
“These are unproven technologies that are marketed as providing a lot of certainty and security,” said David Riedman, founder of the K-12 School Shooting Database.
ZeroEyes said that trained employees review alerts before they are sent and that its software can make a lifesaving difference in averting mass shootings by alerting law enforcement to weapons on campus within seconds. At Lawton Chiles, the student flagged by ZeroEyes was holding his musical instrument like a rifle, co-founder Sam Alaimo told The Washington Post.
“We don’t think we made an error, nor does the school,” Alaimo said. “That was better to dispatch [police] than not dispatch.”
Seminole County Public Schools declined to comment on Tuesday’s incident, but it provided a copy of the letter it sent to parents of Lawton Chiles students after the incident.
“While there was no threat to campus, I’d like to ask you to speak with your student about the dangers of pretending to have a weapon on a school campus,” principal Melissa Laudani wrote.
Concerns about student safety have pushed school districts across the country to embrace a growing industry of AI-assisted security tools that proactively flag threats to administrators and law enforcement. ZeroEyes spokesperson Olga Shmuklyer said its product is used in 48 states and that it has detected more than 1,000 weapons in the last three years.
The systems are usually trained to detect a safety risk by reviewing volumes of sample data, such as images of people holding guns, to look for matches in real-time.
They have sometimes made mistakes. In October, parents and officials in Baltimore County, Maryland, called for a review of a different AI threat-detection system after it confused a bag of Doritos chips for a gun and sent an alert that led to a high-schooler being handcuffed. In 2023, a high school in Clute, Texas, went into lockdown after ZeroEyes falsely alerted that a person was carrying a rifle, according to News 4 San Antonio.
In one case, a different threat-detection system failed to avert a fatal school shooting. Antioch High School in Nashville was equipped with AI surveillance software to detect guns in January when a 17-year-old student killed a classmate in a shooting, according to CNN. The system missed the shooter because he was too far away from surveillance cameras to detect his weapon, CNN reported.
Other systems that monitor students’ activity on school devices have also been criticized for falsely accusing students and violating their privacy. In September, students at a Kansas high school sued their school district after a monitoring tool falsely flagged art projects as pornography.
ZeroEyes has worked closely with Seminole County Public Schools since 2021, according to news reports and the company. That year, it held a live demonstration of the ZeroEyes system’s ability to detect guns at Oviedo High School.
“We’ve been very very pleased with the technology,” Seminole County Schools Public Safety Director Richard Francis told Fox 35 News at the time.
Alaimo, the ZeroEyes co-founder, said the company hires employees with military or law enforcement experience who are “calm under pressure and … very good at identifying guns” to review potential threats flagged by AI,
The image ZeroEyes flagged at Lawton Chiles showed the student appearing to aim his clarinet like a gun at a door and strongly resembled “a shooter about to do something bad,” Alaimo said.
The officer who responded to the alert questioned the student with the clarinet, according to the police report. The student said he was dressed as a military character from the Christmas movie “Red One” for the school’s Christmas-themed dress-up day.
The student said he was “unaware” he was holding his clarinet in a way that would have triggered an alert, according to the report. Police took no further action.
Chad Marlow, a senior policy counsel at the American Civil Liberties Union who has studied school security systems, said incidents like the one at Lawton Chiles show that systems like ZeroEyes can still be fallible, even with humans reviewing the threats that AI flags.
“If a computer technology is telling a … human evaluator that they see a gun and that literally seconds may be critical, that person is going to err on the side of saying it’s a weapon,” he said.
Amanda Klinger, the director of operations at the Educator’s School Safety Network, added that false reports risk “alarm fatigue” and dangerous situations if armed police respond to a school looking for a shooter.
“We have to be really clear-eyed about what are the limitations of these technologies,” Klinger said.
Alaimo said ZeroEyes — and its partners at school districts — would rather be safe than sorry.
“A superintendent, a school resource officer, a chief of police, a director of security, they’re going to say, ‘Yes, be more proactive, be more inclined to give me the alert if you have a fraction of a doubt,’” Alaimo said. “Because they want to keep people safe.”
The post A school locked down after AI flagged a gun. It was a clarinet. appeared first on Washington Post.
