The steady drumbeat of big mistakes by customer support AI agents, for example from big names like Chevy, Air Canada, and even New York City, has brought a renewed focus on the need for more reliability.
If you’re an enterprise decision maker involved in building generative AI apps and strategy and you are having a hard time keeping up with the latest chatbot technology, and how to keep it accountable, you should apply to attend our exclusive AI event in New York on June 5 about the “AI Audit.”
At this networking event hosted by VentureBeat, catering to enterprise technical leaders who are engineering and developing AI products, we’ll be hearing from three key players in the ecosystem on the latest best practices for AI auditing.
We’ll hear from Michael Raj, VP of Verizon for AI and data, about how he’s using meticulous AI audits and employee training to shape a framework for using generative AI responsibly in customer interactions.
We’ll also hear from Rebecca Qian, co-founder and CTO of a company called Patronus AI, which is on the leading edge of creating strategies and technologies for AI audits, and which can help pinpoint and patch safety gaps. Qian worked at Meta for more than four years and led AI evaluation work at Meta AI Research (FAIR).
I’ll be hosting the conversations with my colleague Carl Franzen, executive editor at VentureBeat. We’re delighted to have UI Path as a sponsor of the event: Justin Greenberger, SVP of UiPath, will also be there to share insights into how auditing and compliance guidelines are changing as a result of the fast changes in AI, and how to manage these processes across the organization. The event is part of our AI Impact Tour series of events, designed to foster conversation and networking among enterprise decision makers seeking to put generative AI applications to work in real deployments.
So what is an AI audit exactly, and how is it different from AI governance? Well, once you’ve set up your broader governance rules around AI, you need to set up an audit of your generative AI apps to make sure they living by the rules you’ve set up. But that’s increasingly critical, given the rapid changes in technology. Leading LLM providers like Open AI and Google keep advancing their latest versions of ChatGPT and Gemini. As of last week, these AI models can see, hear and speak, and even inject emotion into their interactions. This, together with advances by other providers, including Meta (Llama 3), Anthropic (Claude) and Inflection (and its new empathy-driven AI), makes staying up with accuracy, privacy and other auditing needs challenging.
Notably, a host of new companies, including Patronus AI, have arisen to fill the void in this area, launching benchmarks, datasets, and diagnostics to help in areas like detecting sensitive personally identifiable information (PII) in bot information. It turns out that even grounding techniques like retrieval augmented generation (RAG), and extended context windows, and system prompts aren’t enough to mitigate mistakes. Sometimes the problems are inherent in the LLM model training datasets themselves, which often lack transparency. This makes auditing even more critical.
Don’t miss this essential gathering for enterprise AI decision-makers who are seeking to lead with integrity in the digital age. Apply to join us at the AI Impact Tour to secure your place at the forefront of AI innovation and governance.
The post Gen AI safety problems force enterprises to upgrade AI audit measures appeared first on Venture Beat.