Certified: The ISACA AAIA Audio Course

In this episode, we take three A I security ideas that often sound mysterious and make them feel simple and concrete: data poisoning, evasion, and model theft. If you are brand new to cybersecurity, it can be frustrating to hear advanced terms that seem to assume you already know machine learning, statistics, and coding. You do not need any of that to understand the core security story here, because each concept maps to a familiar idea: changing what a system learns, tricking what a system decides, and stealing what a system is. The key is learning what these attacks look like in the real world, why they matter even when the computers are locked down, and what questions you should ask when you are evaluating whether an organization has real protection. By the time you finish, you should be able to explain each concept in everyday language, connect it to practical risk, and recognize the early warning signs that these issues may be present.

Before we continue, a quick note: this audio course is a companion to our course companion books. The first book is about the exam and provides detailed information on how to pass it best. The second book is a Kindle-only eBook that contains 1,000 flashcards that can be used on your mobile device or Kindle. Check them both out at Cyber Author dot me, in the Bare Metal Study Guides Series.

A I systems are different from normal software because their behavior is shaped by data rather than being fully defined by human-written rules. In traditional I T, you can often point to a line of code that caused a mistake, and you can fix it by rewriting that logic. With a model, the behavior comes from patterns learned during training, and that training is like teaching rather than programming. If someone can mess with the teaching materials, they can influence what the model learns, even though the model’s code did not change. If someone can craft inputs that take advantage of how the model interprets patterns, they can make the model choose the wrong answer without breaking into a server. And if someone can copy the model’s behavior or extract its internals, they can steal value and also make attacks easier. Keep that mental frame in mind, because it will make each of these three topics feel less abstract and more like normal security work applied to a new kind of system.

Data poisoning is the idea of tampering with the data a model learns from so the model learns the wrong lessons. A plain-language way to think about it is this: imagine you are training a new employee by giving them examples of what to do, and someone slips bad examples into the training binder. The employee might still look competent most of the time, but they will make predictable mistakes in certain situations because they were taught incorrectly. In A I, the training data is that binder, and the model learns patterns from it. Poisoning can be obvious, like inserting clearly wrong labels, but it can also be subtle, like adding many slightly biased examples that nudge the model over time. The attacker’s goal might be broad, such as lowering accuracy overall, or very specific, such as making the model behave incorrectly only for certain inputs. What makes this a security issue is that the model can be deployed into important decisions, and a poisoned model can quietly cause harm without any system alarms going off.

There are two common flavors of data poisoning that beginners should be able to separate: general poisoning and targeted poisoning. General poisoning is like adding noise to the training binder so the trainee becomes sloppy everywhere, which can reduce quality and trust across the board. Targeted poisoning is more like planting a specific rule in the trainee’s mind, so that when a particular situation happens, the trainee makes a specific wrong choice. For example, a targeted poison might cause a model to treat one category as another, or to behave incorrectly when it sees a specific trigger pattern. In some cases, the trigger can be something tiny that humans might not notice, like a specific phrase, formatting pattern, or data artifact that becomes associated with the attacker’s desired outcome. When you evaluate defenses, you want to know whether the organization can trace where training data came from, whether they validate it before using it, and whether they can detect unusual changes in the makeup of the data over time.

A key beginner misconception is to treat data poisoning as only a data quality problem, like messy records or typos. Data quality matters, but poisoning is about intent and adversarial influence, which changes what you look for. If records are messy by accident, you focus on cleaning and standardizing. If records are messy on purpose, you focus on access control, provenance, and monitoring, because someone is trying to shape the model’s behavior for their benefit. Another misconception is to assume poisoning is only possible if an attacker already has deep internal access, but that is not always true. If an organization collects training data from public sources, user contributions, vendor feeds, or partner systems, then the attacker might only need influence over those sources, not direct access to the model training environment. The evaluation mindset is to ask where the data comes from, who can influence it, and what checks exist to block or flag changes that look suspicious.

Evasion is different, and it helps to define it in the simplest possible way: evasion means causing the model to make the wrong decision at runtime by crafting an input that fools it. Unlike data poisoning, evasion does not require changing training data; it happens when the model is already deployed and running. In everyday terms, it is like learning how a security guard thinks and then dressing in a way that slips past their attention, even though you did not change the guard’s training. The attacker probes the model’s behavior and looks for weak spots, like patterns the model misinterprets, blind areas where it makes wrong assumptions, or phrasing that steers it into unsafe responses. Evasion matters because it can happen during normal use, through normal interfaces, without any malware. If a model is used to classify content, detect threats, approve requests, or summarize alerts, evasion can turn it into a weak link that attackers deliberately exploit.

One reason evasion is so powerful is that models can be sensitive to small changes in the way information is presented. Humans often recognize intent even when the wording changes, but models can react differently to slight shifts in phrasing, structure, or context. An attacker can take advantage of that by trying many variations until they find one that slips through. If there are safety rules, the attacker might try to find a phrasing that avoids the triggers while still accomplishing the same harmful goal. If the model is doing classification, the attacker may craft an input that looks benign to the model but still carries a malicious meaning for the human or downstream system. When evaluating evasion defenses, you are looking for whether the system was tested with adversarial inputs, whether there are guardrails beyond the model itself, and whether monitoring can detect probing behavior, such as repeated attempts with small variations that suggest someone is searching for a bypass.

It also helps to recognize that evasion does not always look dramatic, and it is not limited to a single type of model. Evasion can show up as a model giving an unsafe answer, classifying something incorrectly, or retrieving the wrong information from a knowledge base and presenting it confidently. Sometimes the attacker’s goal is to get the model to reveal sensitive data or instructions, and sometimes the goal is to get the model to produce content that helps the attacker. Other times the attacker uses evasion to hide, such as crafting text that a detection model will misclassify as harmless. The common thread is that the attacker is not breaking the system in the classic sense; they are exploiting predictable weaknesses in the model’s decision boundaries. A strong evaluation asks whether the organization treats the model as a component that can be tricked, rather than assuming it behaves consistently across all inputs.

Now let’s talk about model theft, which sounds like someone stealing a box labeled model, but in practice it can happen in a few different ways. At a plain-language level, model theft means an attacker gets access to the model’s value without permission, either by stealing the model directly or by reproducing its behavior closely enough to compete with it. In classic I T, you might worry about someone stealing source code, and that is still relevant if the model files or training code are exposed. But A I adds an extra route: an attacker can query a model many times, collect the inputs and outputs, and use that to build a copycat model that behaves similarly. Even if the copycat is not perfect, it can be good enough to replace the original in many situations, especially if the original model is used in a narrow, predictable way. Model theft matters for business reasons, but it also matters for security because a stolen or copied model can be used to test attacks and find weaknesses more efficiently.

Model theft can also involve extracting sensitive information from the model, not just copying its behavior. If a model was trained on confidential data, it may leak pieces of that data in its outputs under certain conditions, especially if safeguards are weak. In that case, the attacker does not need to steal the model files; they can treat the model like a leaky container of information and pull secrets out through carefully crafted questions. This is not guaranteed to work in every system, but it is a real concern, and the risk increases when training data includes personal information, proprietary documents, or internal records that were not properly protected. An evaluator should ask what data went into training, whether sensitive data was minimized, whether the system was tested for leakage, and what monitoring exists to detect unusual requests that look like extraction attempts.

A useful way to keep these three ideas separate is to focus on what changes and when it changes. With data poisoning, the attacker changes the learning material, which changes the model’s behavior later, even for innocent users. With evasion, the attacker changes the input at the moment of use, trying to get a wrong answer without changing the model itself. With model theft, the attacker tries to take the model’s value, either by stealing the model files, copying the behavior through querying, or extracting sensitive content the model contains. Each one has different warning signs. Poisoning might show up as gradual drift or sudden performance changes after a data refresh. Evasion might show up as lots of probing attempts, repeated failures, or strange input patterns. Theft might show up as high-volume querying, systematic coverage of many input categories, or attempts to access model storage locations and related infrastructure. When you can label the pattern, you can also ask smarter questions about defenses.

A beginner-friendly evaluation mindset is to think in terms of control points rather than technical tricks. For poisoning, the control points are data sources, data pipelines, and approvals for what becomes training material. You are asking whether there is a clear boundary between trusted and untrusted sources, and whether that boundary is enforced with access control, validation, and change tracking. For evasion, the control points are the interface and the guardrails around the model, including input filtering, output constraints, and monitoring for probing patterns. You are asking whether the organization assumes attackers will experiment with prompts, and whether they can detect and respond when that happens. For theft, the control points are access to the model itself, the ability to query it at scale, and safeguards against leakage of sensitive information. You are asking whether the organization treats the model like a high-value asset and a potential data exposure channel at the same time.

Another misconception worth correcting is the idea that these problems are only relevant to big companies with advanced A I teams. In practice, many organizations adopt A I by using third-party services, embedding models into products, or connecting assistants to internal documents. That can create the same risks, even if the organization never trains a model from scratch. If a vendor trains a model using shared data, poisoning risks may show up in the vendor pipeline or in the customer’s fine-tuning data. If an assistant is exposed to end users, evasion becomes relevant immediately because the user interface is public. If the organization relies on a model as a competitive advantage, theft becomes relevant, especially if the model can be queried freely. The evaluation focus is not on how sophisticated the organization is; it is on how the model is used, what the attack surface is, and whether the controls match the real exposure.

As you bring these ideas together, try to hold onto a simple narrative: A I security is about protecting learning, protecting decisions, and protecting value. Data poisoning attacks learning by corrupting what the model is taught. Evasion attacks decisions by manipulating how the model interprets inputs in the moment. Model theft attacks value by taking the model’s behavior, internals, or embedded knowledge. None of these requires the attacker to do the classic movie-style hack of breaking into a server room, and that is exactly why they catch organizations off guard. If you can explain these ideas plainly, you can also recognize when someone is overpromising with vague statements like the model is safe or the model is secure. A better claim is specific and testable, such as the data pipeline is controlled, the system is monitored for probing, and high-volume access is limited and investigated. That kind of clarity is what turns confusing A I terms into practical security thinking.