AI is insecure by default. The open secret is that most AI apps launch with serious flaws. It takes MORE time to secure these apps than it does to build them. I learned this firsthand from shipping LLM agents to 200M users at Discord and supporting LLM evals for... hundreds of companies. It doesn’t have to be this way. Today, we’re launching promptfoo, an open-source company that helps find and fix vulnerabilities in AI powered apps before they ship, and announcing a $5M seed round led by a16z.show more

is the first open-source tool that generates synthetic adversarial attacks customized to your LLM app. It focuses on failures that developers and businesses actually care about, like data leaks and tool misuse. Over 25,000 developers at companies like Shopify, Amazon, and Anthropic are already using our open-source evaluation tool to improve & fortify their apps.

The beauty of open source is that it’s for everyone. The big AI labs have dedicated “red teams” - adversarial testers that find holes in your app. Now you do too! LLM security is too important and too ubiquitous of a problem for this to not be in the hands of every developer. Try it now: npx promptfoo@latest redteam init

We're honored to have the support of @a16z and many industry leaders who share our vision for open-source, application-focused AI security. Thanks to @AnjneyMidha @zanelackey @tobi @fkerrest @adamely @svishnevskiy and many other excellent people. Read more about our vision for open-source red teaming here

Literally my home office. I need a machine to cool down my other machine.

Grats Ian!!