Knowledge Tree
adversarial attack
Also known as: adversarial attack, adversarial attacks, adversarial examples, adversarial perturbations, adversarial example
Facts (16)
Sources
A Comprehensive Review of Neuro-symbolic AI for Robustness ... link.springer.com Dec 9, 2025 7 facts
measurementThe minimum distance to an adversarial example quantifies the smallest perturbation necessary to alter a model’s prediction and serves as a direct indicator of robustness.
measurementThe L0 norm measures the number of modified features in an adversarial attack, indicating the sparsity of the attack.
measurementThe L1 norm evaluates the total magnitude of changes in an adversarial attack by summing the absolute values of all perturbations, capturing the total alteration regardless of which specific features are affected.
claimPerceptual metrics consider structural features such as edges, textures, and spatial patterns, as well as color characteristics like hue, saturation, and brightness, to assess how adversarial examples differ from clean inputs.
referenceResearch by [52] showed that confidence scores can serve as useful signals for detecting out-of-distribution (OOD) inputs and adversarial attacks.
referenceResearch by [51] demonstrated that adversarial examples often lie near decision boundaries, where machine learning models produce low confidence predictions.
claimUnder adversarial attacks or distributional shifts, increases in metrics such as predictive entropy and variance indicate a machine learning model's uncertainty and potential struggle to produce reliable predictions.
Cybersecurity Trends and Predictions 2025 From Industry Insiders itprotoday.com 2 facts
claimJimmy Mesta of RAD Security states that AI workload security will focus on protecting models from data poisoning, model evasion, and adversarial attacks, as attackers increasingly target foundational elements like training datasets.
claimIn 2025, AI security will become a boardroom imperative, leading to new investments in safeguarding AI models against adversarial attacks, model theft, and data poisoning.
Beyond Missile Deterrence: The Rise of Algorithmic Superiority trendsresearch.org Mar 16, 2026 2 facts
claimAdversarial attacks on military artificial intelligence systems can mislead models into misclassifying targets or distorting situational awareness, potentially leading to unlawful or unintended decisions when linked to weapons or command-and-control systems.
claimThe integration of AI into cyber defense introduces vulnerabilities such as adversarial attacks on machine-learning models and data poisoning, which can mislead or disable defensive systems.
Building Trustworthy NeuroSymbolic AI Systems - arXiv arxiv.org 2 facts
referencePrevious attempts to explain BlackBox language models have utilized surrogate models like LIME (Ribeiro, Singh, and Guestrin 2016), visualization methods, and adversarial perturbations to input data (Chapman-Rounds et al. 2021).
claimNeuro-Symbolic AI (NeSy-AI) for adversarial perturbations uses general-purpose knowledge graphs to modify sentences to examine the brittleness in Large Language Model (LLM) outcomes.
LLM Hallucinations: Causes, Consequences, Prevention - LLMs llmmodels.org May 10, 2024 1 fact
claimLarge language models are vulnerable to adversarial attacks or manipulation, which can cause them to generate hallucinated text.
Track: Poster Session 3 - aistats 2026 virtual.aistats.org 1 fact
claimIn the context of offline reinforcement learning with human feedback (RLHF), an ε-fraction of trajectory pairs in a dataset can be corrupted, representing either adversarial attacks or noisy human preferences.
A Survey on the Theory and Mechanism of Large Language Models arxiv.org Mar 12, 2026 1 fact
referenceThe research paper 'What features in prompts jailbreak LLMs? Investigating the mechanisms behind attacks' explores the mechanisms behind adversarial attacks on large language models.