Handle external vulnerability reports as findings.
External people and organizations may report what they claim are vulnerabilities. Unfortunately, these reports may be false, especially if they were AI-generated (since AI may not understand the context). This means that such reports are essentially findings.
A widespread problem is “AI slop reports”, that is, reports generated by an AI that are nonsense and waste everyone’s time. There are some steps you can take to counteract this. For example:
Make it clear that you expect reporters to have a human review of an AI-proposed vulnerability report.
Encourage reporters to work together to deduplicate reports before they get to you (e.g., through Akrites, which we will discuss later).
Require evidence that suggests a report may be a vulnerability. In particular, require a sample input demonstrating the vulnerability.
If the code is available to the reporter, ask them to report specific filenames and line numbers and, if possible, propose a fix. The fix may not be directly useful (it often isn’t), but a proposed fix can often clarify what the reporter believes the problem is.
Several prominent programs changed their policies in 2026 because of AI-generated reports. Their experience is instructive:
The curl project ended its bug bounty on 2026-01-31. Daniel Stenberg explained that “Previous years we have had a rate of somewhere north of 15% of the submissions ending up confirmed vulnerabilities. Starting 2025, the confirmed-rate plummeted to below 5%” [Stenberg2026-01]. In March 2026 curl went back to receiving reports through HackerOne, still without paying bounties. Stenberg then reported that “The slop situation is not a problem anymore”: reports arrived at about double the 2025 rate, nearly all used AI, and the confirmed vulnerability rate was back to “somewhere in the 15-16% range” [Stenberg2026-04]. We believe this is evidence that AI slop is a problem that can be countered.
HackerOne paused submissions to its Internet Bug Bounty, which rewards reports of vulnerabilities in widely used open source software. They said that “The balance between findings and remediation capacity in open source has substantively shifted” [Cooter2026].
GitHub raised its bug bounty requirements in May 2026, requiring “A working proof of concept with demonstrated security impact.” GitHub welcomes AI-assisted research, but says “No matter what tools you use (scanners, static analysis, AI assistants), you need to validate the output before submitting” [Brown2026].
The lesson for your project: publish a clear vulnerability reporting policy (e.g., in SECURITY.md) that says what evidence you require (such as a reproducing input), that reporters must validate reports before sending them, and that you welcome AI-assisted reports that meet those requirements. If you offer rewards, watch whether they attract low-effort submissions. For guidance on setting up a coordinated vulnerability disclosure program, see [CISA2026-CVD].
Quiz
Q1. Per the material, what should a project’s vulnerability reporting policy say about AI-assisted reports?
Reject every report that used AI at any point, since such reports are almost always false positives
Accept AI-generated reports as-is, since modern models rarely report vulnerabilities that aren’t real
Offer larger bug bounty rewards for AI-assisted reports, since bigger rewards attract better-validated reports
Require a reproducing input and prior validation by the human reporter, and welcome reports that meet that bar
Finding and Fixing Vulnerabilities Using AI