It’s vital to define the security requirements so you know what constitutes a vulnerability. Neither humans nor AI can determine whether something is a vulnerability without a clear definition for a specific project. There are many ways to define security requirements. Still, a common way to define them is to develop a “threat model” of the system under analysis, as it also provides an approach to analyzing and addressing the requirements.
Some people refer to the need for “system context” and/or “trust boundaries”. For our purposes, this is part of the threat model. For an AI to find vulnerabilities, it needs to know what a vulnerability is, including the system context and trust boundaries. All of that is wrapped into the threat model.
Adam Shostack suggests four core questions a threat model should answer: what are we working on, what can go wrong, what are we going to do about it, and did we do a good job [Shostack2014]. If you’re not familiar with threat modeling, there are many resources to learn more. Here we’ll focus on AI-specific issues.
When using AI to help develop a system, there are at least two threat models that apply:
The development/build/test/release environments, including any AI systems and CI/CD pipeline. An AI system may choose to perform many activities you do not want it to, and your development environment must prevent the worst-case scenarios. Typically this involves preventing the AI from attacking developers’ systems, exfiltrating data (including credentials and private keys), making unauthorized modifications, and/or attacking external systems. These are typically addressed by sandboxing, a topic we’ll discuss soon.
The deployed environment. This threat model focuses on security while the system is in use. For the rest of this section, we’ll focus on the deployed environment.
Unsurprisingly, “AI (and external contributors) are more successful if the project shares how they desire the software to be used, acceptable scenarios to be deployed into, and what problems the project is aware of that could go wrong” [Aniszczyk2026].
Having a threat model for a system’s deployed environment is important when using AI:
One report stated that an AI model “performed best on systems with well-documented threat models, system design docs, requirements, and constraints. When the threat model was well-defined, the model’s findings were exploitable 90 percent of the time.” In addition, “The most common cause of false positives is that the model lacks a good understanding of [the] trust boundaries.” [Yan2026].
A Mythos Preview user reported that it was valuable, but it “needs precise prompts, explicit threat models, and validation infrastructure to turn strong reasoning into reliable security outcomes” [Ziegler2026].
If the AI is not given enough information on the specific context and structure of your system’s environment, its recommendations may fail to align with requirements, including regulatory requirements [Kholoosi2025].
If you don’t already have a threat model for your deployed system, the good news is that an AI can help you create one. The bad news is that you need to interact with the AI, then review and refine its results, not simply take the AI-generated threat models as truth. [Yan2026] suggests that when creating a threat model with AI, “bootstrap from the code, docs, and vulnerability history. Feed the model what you would hand a new security engineer on day one: architecture docs, wikis, entry points, git history, and past vulnerabilities. This helps overcome the challenge of inferring implicit knowledge, trade-offs, and design decisions from code alone. Then, ask the model to create a threat model that includes the system context, assets, entry points, and trust boundaries. Finally, have the model cluster past bugs and list the relevant vulnerability classes. Make sure the threat model documents what vulnerabilities you do and don’t care about, and why.”
Various tools use AI to help you create a threat model for a system’s deployed environment. These include:
When creating or updating a threat model for a world with AI, consider the following:
Design for prevention and containment. If an attacker takes over one system, try to put in place prevention mechanisms to limit lateral movement [CrowdStrike2026-FiveSteps]
Strongly limit privileges
Strongly control the identity of all entities (human and non-human). Consider using continuous identification [CrowdStrike2026-FiveSteps]
Use layered defenses, so attackers must surmount multiple mechanisms to gain top privileges [Grinstead2026-05]
Look at past vulnerabilities to identify patterns that could prevent success across whole categories of attacks [Grinstead2026-05]
Specifically identify what is trusted (admins, specific config files, and so on). “These assumptions help separate non-exploitable bugs from actual exploits.” [Yan2026]
Include the threat model with the code (e.g., as THREAT_MODEL.md), so they can be updated simultaneously [Yan2026].
Finding and Fixing Vulnerabilities Using AI