AI Security in 2025: What Every Security Engineer Needs to Know

Artificial intelligence is no longer a future concern for security teams — it is a present one. As organisations race to integrate large language models (LLMs), generative AI tools, and ML-powered systems into their products, security engineers face an entirely new threat landscape that traditional AppSec frameworks were not designed to address.

This post covers the most critical AI security risks, the emerging standards you need to know, and practical steps to start building secure AI systems today.

Why AI security is different

Traditional application security deals with deterministic systems. Given the same input, a well-written function produces the same output. LLMs are fundamentally non-deterministic — the same prompt can produce different outputs, and the model's behaviour is shaped by billions of parameters trained on data you likely did not curate or audit. This creates attack surfaces that have no direct equivalent in classical AppSec:

• The attack surface includes natural language — inputs that look benign to humans but manipulate model behaviour.

• The "code" (model weights) is opaque and cannot be audited like source code.

• Security boundaries blur when the model has access to tools, APIs, or data stores.

• Third-party model providers introduce supply chain risks analogous to — but distinct from — traditional software dependencies.

OWASP LLM Top 10: the essential starting point

OWASP published the LLM Top 10 to give security practitioners a structured view of the most critical risks in LLM-based applications. The top risks every engineer should understand are:

• LLM01 — Prompt Injection: Attackers craft inputs that override the model's system instructions or manipulate its behaviour. Direct injection targets the model directly; indirect injection embeds malicious instructions in data the model retrieves (e.g. from a website or document).

• LLM02 — Insecure Output Handling: Downstream components blindly trust LLM output. If an LLM generates HTML, SQL, or shell commands that are executed without sanitisation, classic injection vulnerabilities re-emerge.

• LLM03 — Training Data Poisoning: Malicious data injected into training sets can introduce backdoors or biases that persist silently in the deployed model.

• LLM06 — Sensitive Information Disclosure: Models may inadvertently reproduce sensitive training data, internal system prompts, or confidential user data from prior conversations when prompted correctly.

• LLM08 — Excessive Agency: When an LLM is given tools (APIs, file access, email sending), it may take unintended high-impact actions if not properly scoped and constrained.

Practical controls for AI security

Security teams building or evaluating AI-powered products should apply these controls as a baseline:

• Input validation and sanitisation: Treat all LLM inputs as untrusted. Apply length limits, content filters, and context-aware validation before passing data to the model.

• Output encoding: Never trust LLM output. Encode or escape all generated content before rendering it in a browser, executing it in a shell, or passing it to a downstream API.

• Least privilege for agents: If your LLM has tool access, scope permissions to the minimum necessary. An LLM that can read emails should not also be able to send or delete them.

• Human-in-the-loop for high-impact actions: Require explicit human confirmation before the model takes irreversible actions — sending messages, making payments, modifying data.

• Threat modelling AI components: Apply STRIDE or a purpose-built AI threat modelling framework (such as MITRE ATLAS) to every AI system before deployment, not after.

• Red teaming LLMs: Traditional pen testing is insufficient. Build or procure adversarial prompt testing capability — automated and manual — to probe model behaviour before and after deployment.

The regulatory dimension

The EU AI Act, now in force, introduces risk-based obligations for AI systems — with the highest requirements applied to systems classified as high-risk (including those used in critical infrastructure, employment, and financial services). Security engineers working at European companies — particularly in fintech — need to understand how AI Act obligations intersect with existing DORA and GDPR requirements. Data minimisation, auditability, and human oversight are themes common to all three frameworks.

Where to start

If you are a security engineer or leader trying to build an AI security capability from scratch, start here: read the OWASP LLM Top 10, review the MITRE ATLAS framework, and run a threat modelling exercise on the first AI-powered feature your organisation has deployed or is planning to deploy. The muscle memory from traditional threat modelling transfers well — the new skill is understanding the unique properties of LLMs and where classical assumptions break down.

AI security is not a niche specialisation anymore. It is rapidly becoming a core competency for every security team — and the engineers who invest in it now will be well ahead of the curve.