AI and Cybersecurity: The New Arms Race Between Attackers and Defenders

Security has always been an adversarial discipline: every defensive improvement invites an offensive adaptation. Artificial intelligence accelerates both sides of that loop. Over my years building backend systems and APIs, the question I hear most from engineering teams has shifted from should we use AI for security? to how do we secure the AI features we have already shipped?

Three distinct conversations are often blurred together, and it helps to separate them: AI used by attackers, AI used by defenders, and attacks against AI systems themselves.

The distinction matters because each calls for different owners and controls. Attacker use of AI is mostly a problem for security awareness, identity verification and detection teams. Defender use of AI is a procurement and operations question. Attacks on AI systems, however, land squarely on product and engineering teams, because they arise from how we design prompts, tools, data pipelines and permissions. In Nigeria's fast-growing fintech and health tech sectors, where AI features are shipping quickly, that third category is where I see the largest gap between adoption and preparedness.

• Social engineering at scale. Large language models produce fluent, personalised phishing messages in any language, removing the spelling and grammar errors that once gave scams away. Voice cloning makes fraudulent calls impersonating executives or relatives far more convincing.
• Faster reconnaissance. Models help summarise public information about targets, map exposed services and explain how to use known exploits, lowering the skill barrier for less experienced attackers.
• Malware assistance. AI coding tools can help adapt or obfuscate code. Commercial providers apply safeguards, but open models and determined misuse mean defenders should assume some uplift.
• Deepfakes for fraud and disinformation. Synthetic video and audio are used to bypass identity verification and manipulate employees into authorising payments.

It is worth keeping perspective. Most successful breaches still exploit familiar weaknesses: stolen credentials, unpatched systems and misconfiguration. AI makes old attacks cheaper and more convincing more often than it creates entirely new ones.

Detection and anomaly analysis

Machine learning has been in security products for years: spam filtering, malware classification from static and behavioural features, and user and entity behaviour analytics that flag unusual logins or data access. Unsupervised models baseline normal network and identity behaviour so that deviations stand out.

LLM copilots for security operations

Security operations centres drown in alerts. Language models now summarise incidents, correlate alerts across tools, translate natural language questions into query languages for log platforms, and draft incident reports. The best implementations use retrieval over the organisation's own runbooks, asset inventory and threat intelligence so answers are grounded.

Code and configuration security

AI-assisted static analysis finds vulnerable patterns, explains them to developers and proposes fixes in pull requests. Applied to infrastructure-as-code, it catches misconfigurations before deployment.

Threat intelligence

Models extract indicators of compromise and tactics from unstructured reports and map them to frameworks such as MITRE ATT&CK, speeding up how quickly intelligence becomes detection rules.

This is the category many teams underestimate. Once you ship a model, it becomes part of your attack surface.

• Adversarial examples. Small, carefully crafted input perturbations can cause misclassification, a phenomenon studied since early work such as Goodfellow and colleagues on explaining and harnessing adversarial examples.
• Data poisoning. Corrupting training or fine-tuning data can implant backdoors or degrade performance.
• Prompt injection. For LLM applications, the most pressing risk. Malicious instructions hidden in a web page, email or document can hijack a model that reads them, which Greshake and colleagues demonstrated in their work on indirect prompt injection. When the model can call tools, send emails or query databases, injection becomes a pathway to real damage.
• Sensitive data leakage. Models can reveal secrets present in prompts, retrieved documents or training data.
• Model theft and abuse. Exposed endpoints can be queried to extract model behaviour or abused to run up costs.

The OWASP Top 10 for Large Language Model Applications and MITRE ATLAS, a knowledge base of adversarial tactics against AI systems, are excellent starting points. NIST's taxonomy of adversarial machine learning provides a rigorous vocabulary for these threats.

• False confidence. An AI security tool that misses an attack while producing reassuring summaries can be worse than no tool at all. Measure detection rates against realistic red-team exercises.
• Privacy. Behavioural monitoring of employees raises legitimate concerns. Be transparent about what is monitored and why, and comply with data protection laws such as the Nigeria Data Protection Act.
• Dual-use research. Publishing offensive techniques helps defenders but can also help attackers. Responsible disclosure norms apply to AI vulnerabilities too.
• Skills displacement. AI handles routine triage, but junior analysts learn by doing that triage. Organisations should be deliberate about how they develop the next generation of security talent.

For startups and engineering teams, particularly in markets where dedicated security staff are scarce, I recommend:

1. Get the basics right first. Multi-factor authentication, patching, least-privilege access, secrets management and logging still stop most attacks. AI does not replace hygiene.
2. Treat all model input as untrusted. Anything an LLM reads, user messages, retrieved documents, web pages, tool outputs, may contain instructions. Never let the model's output alone authorise sensitive actions.
3. Constrain tools and permissions. Give AI agents the minimum permissions needed, require human confirmation for irreversible actions and sandbox code execution.
4. Validate outputs. Apply output encoding and schema validation before model output reaches a browser, database or shell.
5. Red-team your AI features. Test for prompt injection, jailbreaks and data leakage before launch and after every significant change.
6. Train people for AI-era social engineering. Establish out-of-band verification for payment requests and credential resets, regardless of how convincing a voice or video seems.
7. Use AI to help your small team. Copilots for log analysis and code review can give a small team leverage, as long as humans make the final call.

• AI makes familiar attacks such as phishing and fraud cheaper and more convincing; fundamentals still stop most breaches.
• Defenders gain from ML-based detection and LLM copilots that triage, summarise and query at machine speed.
• AI systems are a new attack surface: prompt injection, data poisoning and data leakage need explicit defences.
• Use OWASP's LLM Top 10, MITRE ATLAS and NIST's adversarial ML taxonomy to structure your threat model.
• Treat model input as untrusted, minimise agent permissions and keep humans in the loop for sensitive actions.

1. OWASP Top 10 for Large Language Model Applications — OWASP Foundation
2. MITRE ATLAS — MITRE
3. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2e2023) — National Institute of Standards and Technology
4. Explaining and Harnessing Adversarial Examples — arXiv
5. Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — arXiv

Cover image: original illustration created for this article. Cited works remain the property of their respective authors and publishers.