Introduction
Start thinking about AI security: what needs protection, what can go wrong, and how to ask useful questions.
Knowing how AI works is the first step. The next step is asking what could make an AI system unsafe or unreliable.
This room introduces risks around AI models, attacks that use AI, defensive uses, and secure adoption. The notebook covers all seven tasks, with examples, diagrams, and explanations of the ten practical questions. Original TryHackMe room.
Four words to understand first
Imagine a fictional shop with an AI assistant that answers questions about orders.
| Word | Simple meaning | Shop example |
|---|---|---|
| Asset | Something worth protecting | A customer’s address |
| Vulnerability | A weakness that could allow harm | The app forgets to check whose order it is |
| Threat | Something that could cause harm | A stranger tries to read another person’s order |
| Risk | The chance of harm, considered with its impact | How likely an address leak is, and how serious it would be |
The model is one part of the shop’s system. The website, accounts, documents, tools, and database also matter. A useful security review follows information through the whole system. This is threat modeling: describe the system, consider possible harm, choose protections, and check whether they work. OWASP’s threat modeling guide.
Ask where an attacker can act
An attacker might change data before training. Another might send harmful input after the model is running. These are different opportunities. NIST organizes AI attacks partly by their stage, the attacker’s goal, and the access the attacker has. NIST’s adversarial machine learning overview.
For the fictional shop, write down three questions:
- Who can add information to the assistant’s knowledge?
- Which customer records can it read?
- Can it only suggest a refund, or can it send money?
Specific questions make the next task easier to understand. You can connect each threat to a real part of a system.
Check your understanding
Is “the chatbot sounds confident” evidence that the whole system is secure?
No. A confident answer tells you little about who can read private records or change data. You need evidence about those controls. Try explaining one control you would inspect in the fictional shop.