OpenSSF Finding and Fixing Vulnerabilities Using AI

2.1 Basic AI terminology

First, let’s go over a few basic AI terms. It’s likely you know many of these, but look over these terms to make sure you understand what we mean by them:

AI systems are not sentient. LLMs, for example, repeatedly generate likely next tokens (word fragments); they don’t “understand” in the sense that humans do. Yet scale matters. With many layers and parameters, modern AI systems can simulate intelligence, sometimes astonishingly.

There’s also strong evidence that AI models have improved. One way to measure AI models is the “50%-task-completion time horizon” defined as the “time humans typically take to complete tasks that AI models can complete with a 50% success rate” [Kwa2025]. As of 2025, “this metric has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months” [Kwa2025-blog].

Quiz

Q1. What distinguishes a “frontier model” from other AI models?

  1. It uses symbolic logic instead of a neural network
  2. It can only run on local, organizational hardware
  3. It has no trainable parameters
  4. It is among the best currently-available models
Show answer Answer: D
Quiz

Q1. What is the key difference between an “AI agent” and an “AI chatbot”?

  1. An agent uses deep learning, while a chatbot uses only hand-written rules
  2. An AI agent can execute multi-step actions using external tools to affect its external environment; a chatbot can’t affect an external environment
  3. An AI agent needs no training data, while a chatbot needs large datasets
  4. A chatbot can perceive its environment, while an AI agent can’t
Show answer Answer: B