Generative AI products are getting so good that they can clone your voice with just a few samples. OpenAI has already produced such a tool, and I immediately thought voice cloning might be the most dangerous thing coming out of genAI app development. Sure, the ability to clone voices can come in handy in a variety of situations that have a noble goal at heart. But voice cloning can and will be abused. That’s probably why OpenAI has still not released it into the wild.
This brings us to TerifAI, a low-latency voice-based chatbot that steals your voice and clones it right from the browser while chatting with you about whatever you might have in mind.
The AI innovation is absolutely amazing, but, as with OpenAI’s voice cloning solution, it’s also incredibly scary. What’s even more concerning is that it takes only a few minutes of chatting to clone your voice. And it happens without a proper warning.
But that’s the whole point of TerifAI. It’s meant to be educational.
Tech. Entertainment. Science. Your inbox.
Sign up for the most interesting tech & entertainment news out there.
Email: SIGN UP
By signing up, I agree to the Terms of Use and have reviewed the Privacy Notice.
I have tested TerifAI briefly, but not long enough for it to clone my voice. All you have to do is give your browser microphone access permissions, load the chatbot, and start talking.
From the start, I was amazed by the incredibly fast response rate. The website brags that TerifAI is the world’s fastest chatbot, and I tend to believe it. The chatbot talks back like a human; everything happens in real-time. There’s no lag in processing the voice data. You just speak your mind, and the chatbot will reply. You’ll also see the latency on the screen while you talk to it.
The chatbot has a preinstalled default voice you can’t change, and it can cover various topics, though this isn’t ChatGPT. On that note, it’s unclear what AI models TerifAI uses.
The chatbot will ask questions about your hobbies and things you like. It’ll quickly detect when you’ve stopped speaking and reply. You can interrupt it and talk back over it, just like you would do in a regular conversation with another person.
I haven’t reached the point where the chatbot cloned my voice, but I saw Aman Ibrahim’s demo, and it terrified me. Once the chatbot clones your voice, it’ll speak back to you in your own voice and accent.
Thankfully, the developer of TerifAI says these demos are meant to raise awareness about the dangers of voice phishing in the age of genAI. TerifAI isn’t a commercial product for now, so you can’t abuse it to generate impersonations of famous people or politicians.
But, again, this type of technology has massive implications, especially if misused in the wild. Anyone with the right tools can create similar products that can be used to clone voices.
This is a demo meant to shock you, so it won’t ask your permission to clone your voice. That’s understandable. But I found disturbing the lack of warnings that the chatbot has just cloned your voice within minutes of using it. After the cloning has occurred you realize what’s happening. You should get some sort of notice. I might not have had my voice cloned, but there’s no visual indication in Ibrahim’s demo to signal that voice cloning has happened.
It’s also unclear what happens to my data and what privacy protections I can expect from TerifAI.
The whole website has a minimal design, with the chatbot’s smiley face dominating the experience. I get that it’s all meant to demo the capabilities of Pipecat, an open-source framework for building voice-based AI agents that can clone your voice and raise awareness about potential abuse. But TerifAI is the kind of AI product that can go viral. Many people will try it without knowing what happens with the voice samples the chatbot collects.
You’ll find more details about TerifAI on GitHub.
The post TerifAI is a terrifying AI chatbot that steals your voice while you talk to it appeared first on BGR.
