Finding and Fixing Vulnerabilities Using AI

3.3 Do simple things first

If you’ve never used AI to look for vulnerabilities and you have access to a reasonably good AI model (open-weight or not), consider starting with a relatively simple command to scan for vulnerabilities. Include hints on specifically where to look and/or note tools that might help it (without mandating that it use them).

AI systems, as of 2026, can figure out plausible approaches on their own, even with relatively little guidance. Models are constantly improving, so we expect them to get even better.

For example, [Carlini2026-youtube] demonstrated this as a successful prompt:

You are playing in a capture-the-flag (CTF). Find a vulnerability. hint: Look at /src/baz.c Write the most serious one to /out/report.txt

[Carlini2026] summarizes this as a paragraph that essentially amounts to “Please find a security vulnerability in this program” and let the AI experiment. He notes that in a typical attempt, the AI (Claude) will read the code to hypothesize vulnerabilities that might exist, run the actual project to confirm or reject its suspicions (and repeat as necessary, adding debug logic or using debuggers as it sees fit), and finally output either that no bug exists, or the bug(s) it found.

Such simple prompts can be extended in several obvious ways. For example, many prompts ask the AI to focus on input validation (since if malicious inputs can’t enter the program, they’re much less likely to cause harm) and addressing common past problems. Another is to clearly note that the AI may use existing tools to help in the analysis, including pointers to tools the AI might find useful.

Part of the reason that AI systems can do so much better now is that they’ve become far more adept at making longer-range plans and using tools. [Carlini2026-02] describes it this way:

• “Opus 4.6 found high-severity vulnerabilities, some that had gone undetected for decades…
• We put Claude inside a ‘virtual machine’ (literally, a simulated computer) with access to the latest versions of open source projects. We gave it standard utilities (e.g., the standard coreutils or Python) and vulnerability analysis tools (e.g., debuggers or fuzzers), but we didn’t provide any special instructions on how to use these tools, nor did we provide a custom harness that would have given it specialized knowledge about how to better find vulnerabilities.
• This means we were directly testing Claude’s ‘out-of-the-box’ capabilities, relying solely on the fact that modern large language models are generally capable agents that can already reason about how best to make use of the available tools.
• … We validated every bug extensively before reporting it. … We then had Claude critique, de-duplicate, and re-prioritize the crashes that remain.”

Similarly, [Grinstead2026-05] noted that “you can start with very simple prompting, then observe and iterate… the essence of the inner loop remains the same: there is a bug in this part of the code, please find it and build a testcase”.

Start by focusing on your most important projects, then address the next in turn.