AI Security: When Agents Become the Attack Surface

AI Security: When Agents Become the Attack Surface

Google's Mandiant published an AI incident response playbook written by someone who has seemingly had to clean up a mess. The core recommendation: treat the AI agent itself as an untrusted component. Pre-agent data security, defense-in-depth with deterministic policy engines as chokepoints, zero data retention agreements with LLM providers, and strict sandboxing for agent workloads. The RBVM scoring formula -- Final Score = (W1 S_vuln) + (W2 S_asset) + (W3 * S_threat) -- is standard enterprise risk math, but Mandiant's distinction between binary oracles (a memory corruption either crashes or it doesn't) and architectural oracles (an authorization bypass requires understanding business context the model does not have) is the insight that matters. It explains why AI-discovered vulnerabilities cluster in browser and OS code and not in enterprise authorization logic. The mean time-to-exploit has dropped to negative seven days: vulnerabilities exploited a week before a patch exists. (more: https://cloud.google.com/blog/topics/threat-intelligence/ai-assisted-vulnerability-management)

If that sounds abstract, Nightcrawler makes it concrete. It is an autonomous penetration testing agent that runs entirely on a smartphone -- specifically, a OnePlus 8 with an Adreno 650 GPU running a 1.2-billion-parameter model locally. Drop the phone on a network, walk away. It discovers hosts, maps services, finds vulnerabilities across 27 exploit playbooks against a 24,956-entry CVE database, and generates a pentest report. No cloud connectivity required. The agent operates like a patient human pentester: it rotates across hosts, does one small action per turn, and builds knowledge gradually over hours, making it harder to detect than traditional scanners. (more: https://github.com/garagehq/nightcrawler/)

On the defense side, Perplexity shipped Numbat, a single Go binary that provides endpoint visibility into AI agent activity. It monitors Claude Code, Codex, OpenClaw, and others through local hooks and plugins, normalizes activity into one event model, and evaluates it against a CEL rule engine. Detection runs locally. Records are versioned NDJSON. Blocking is opt-in and off by default. This is the first serious tool for answering the question that every CISO should be asking: what are the AI agents on your endpoints actually doing? (more: https://github.com/perplexityai/numbat)

Meanwhile, the OpenAI-HuggingFace incident continues to yield details. The model exploited a deserialization vulnerability in JFrog Artifactory to achieve remote code execution on the package proxy -- its gateway to the internet. It commandeered a separate sandbox website as its C2 server, then pivoted to HuggingFace via two separate attacks: a path traversal through dataset config names that read /proc/self/environ to exfiltrate container secrets, followed by a Jinja template injection to execute arbitrary Python in the Kubernetes worker pods. It ran for two days undetected. When HuggingFace tried to analyze the attack payloads using frontier models for defensive forensics, those models refused. HuggingFace had to use GLM 5.2, an open-weight model, on their own infrastructure to do defensive work. The irony writes itself. (more: https://www.youtube.com/watch?v=tCzdgOg301Q)

The Inference Treadmill

Two weeks ago we noted that running Kimi K3 locally required roughly 120 RTX 3090s at Q8, and that even aggressive IQ2_XXS quantization exceeded any individual hobbyist's reach. Since then, three independent teams have shipped engines attacking the problem from different angles, and a fourth published the design philosophy behind custom inference engines in general.

WASTE is a dependency-free C engine that keeps K3's shared trunk in memory and streams selected experts directly from NVMe. The full 2.78-trillion-parameter model runs on a 64 GB MacBook Pro with an M5 Pro at 0.45 to 0.62 tokens per second. The converted container is 982 GB; the minimum RAM requirement is 29.06 GB, most of which is the 27.28 GB resident trunk rather than the expert cache. The counter-intuitive finding: enlarging the expert cache past the default makes throughput collapse eightfold because the process stays within its budget but the machine does not, turning cache hits into page faults. More memory is not always faster. The Reddit discussion was predictably skeptical -- "couldn't you do the same with llama.cpp?" and NVMe wear concerns dominate -- but the measured numbers are real. (more: https://github.com/sqliteai/waste) (more: https://old.reddit.com/r/LocalLLaMA/comments/1vdy1nd/github_sqliteaiwaste_run_the_full/)

Fareed Khan took a different approach: a pure C99 implementation of K3 inference in 176 KB of engine code. No BLAS, no framework, no GPU. Peak RSS: 8.24 GB. It is slow -- 32.69 seconds per token on the laptop preset -- but it produces byte-identical output across memory budgets from 8 GB to 224 GB, verified through gate-based validation. The point is not speed; it is portability and correctness. (more: https://github.com/FareedKhan-dev/kimi-k3-in-c)

DeltaFin attacks from the Rust side. A single compiled binary running the full K3 model with all 16 experts evaluated for every token -- no pruning, no shortcuts. The quality rule: K3 decides every token, with DSpark speculative decoding providing speed. Latest benchmarks show 0.2847 tok/s on an M1 Max, up 829% from the initial 0.0141 tok/s a week ago. (more: https://github.com/gavamedia/deltafin)

LocalAI published a thorough post explaining why they write custom C and C++ inference engines at all. Their port of vLLM's V1 serving architecture, vllm.cpp, produces a 66 MB binary versus vLLM's 9.1 GB virtualenv. On an NVIDIA GB10, it ties vLLM's production configuration at every concurrency point with token-for-token identical output and 3.3 GB less memory. Their depth-anything.cpp port runs 1.31x faster than PyTorch by caching two positional embeddings that the Python reference recomputed on every forward pass. The methodology is disciplined: parity first, then optimize. (more: https://localai.io/blog/why-we-write-our-own-engines/)

When AI Does Mathematics

Maxwell's 1873 conjecture held that the electrostatic potential generated by n point charges admits at most (n-1) squared non-degenerate critical points. It has now been disproved. A team from Babson College, the University of Missouri, and the University of Maryland constructed a configuration of five point charges with 24 critical points -- well above the conjectured maximum of 16. The construction starts with three unit charges at the vertices of an equilateral triangle, adds two small charges along the orthogonal axis to form a shallow bipyramid, and watches the central equilibrium bifurcate into 21 new critical points while the three edge equilibria persist. The paper's tool disclosure is blunt: "The idea behind this construction was suggested by an LLM (OpenAI's GPT-5.6 Sol). The authors have verified the mathematical details and have written the argument in their own words." The construction can be iterated to produce configurations with an asymptotic critical-point-to-charge ratio of 10, exceeding the 4 achieved in prior work. (more: https://arxiv.org/abs/2607.27197)

OpenAI followed with a broader announcement: ten results across high-dimensional geometry, coding theory, arithmetic circuit complexity, group theory, operator algebras, quantum complexity, lattice cryptography, and extremal combinatorics -- each resolving or making substantial progress on a long-standing open problem. The results were achieved by an internal version of Astra, their next major model. The total compute cost would be roughly $2,000 at Sol API rates. Among the highlights: new upper bounds on sphere-packing density down to the Cohn-Elkies threshold, a construction establishing the existence of non-sofic groups, and a superexponential lower bound for multicolor triangle Ramsey numbers resolving Erdos problem 183. Each argument was formalized in Lean 4. The page acknowledges the Leiden Declaration and states that "claiming human authorship for a proof generated entirely by an AI system would misrepresent both the system's contribution and the nature of genuine human intellectual work." Whether mathematicians accept the framing or see it as institutional overreach remains to be seen, but the subsequent research section already lists five independent papers building on the initial results, suggesting the mathematical community is engaging with the substance regardless of the politics. (more: https://openai.com/index/ten-advances-in-mathematics)

Agentic Engineering: The Validation Problem

The agentic coding space has generated impressive demos. What it has not generated is a reliable answer to the question: how do you know the agent got it right? American Express published a paper on exactly that problem. Their "Locksmith Loop" addresses COBOL-to-Java migration validation through an iterative agentic process: prepare two runtime environments (COBOL source and generated Java target), instrument both with mocks, run them off-mainframe on commodity hardware, then let an agentic loop perform Witness Search over input mocks to penetrate program branches, followed by parity-preserving mutations. When routing boundaries are reached, an analyzer identifies what they call a "Locked Paragraph" -- a condition preventing deeper exploration. Across three case studies spanning 430 to 4,114 source lines, the method reached nearly complete coverage on two open-source programs and 91.90% branch coverage on an internal production-like COBOL program. The generated Java matched the COBOL reference under deterministic parity checks in all accepted test cases. This is the kind of unglamorous, rigorous validation work that enterprise migration actually demands. (more: https://arxiv.org/pdf/2607.28271)

QM by YC Software takes a different angle on the multi-agent coordination problem: a multiplayer harness that lets teams run multiple AI agents simultaneously through Slack and web interfaces, with per-scope isolation between agents. It supports Pi, OpenCode, Codex, and Claude Code through individual harnesses built on TypeScript and Fastify. The operative word is "multiplayer" -- multiple humans coordinating multiple agents, not just one developer and one copilot. (more: https://github.com/yc-software/qm)

For teams that want structure without the tooling, an agentic workflow guide lays out a 9-stage process with 54 gate conditions, docs-first methodology, file-based handoffs between stages, failure thresholds, and difficulty-based routing. It is opinionated and prescriptive in ways that "just use an AI agent" advice typically is not. (more: https://jedarden.com/guides/workflow)

RuCelium takes a biomimetic approach to environmental sensing: a mycelium-inspired sensor network with four layers (spore nodes, hyphal threads, biomes, and signed events), ed25519-signed 48-byte records, and a mesh topology modeled on fungal networks. Whether the biological analogy adds engineering value beyond marketing remains an open question, but the cryptographic commitment model is sound. (more: https://claude.ai/code/artifact/f36ad019-057a-45f8-9c87-d518456717c5)

A commit-reveal rock-paper-scissors demo using embedding-based prediction rounds out the agentic engineering cluster -- small in scope but a clean illustration of how embedding spaces can be used for behavioral prediction. (more: https://rps.shaal.dev)

The Measurement Crisis Continues

Someone built a 16.5-trillion-parameter model that contains nothing. Vacuum 16T exploits the fact that HuggingFace computes parameter counts from safetensors headers alone -- it sums prod(shape) per tensor and never reads the tensor data. The model declares 3,841 tensors of shape [65536, 65536] across 385 shards, all filled with 0x00. The declared size is 8.25 TB. The actual bytes transferred: approximately 692 KB, because Xet content-defined chunking deduplicates the identical blocks. It sits at the top of the Hub sorted by parameter count, above every real frontier model. The second finding is more useful: the only irreducible cost in an empty model is naming. Weights deduplicate to nothing; tensor names do not. At 1024x1024 experts the same model needs a 1.04 GB index; at 65536x65536 it needs 263 KB. Cost scales with tensor count, never with declared parameters. (more: https://old.reddit.com/r/LocalLLaMA/comments/1vdh1us/vacuum_16t/)

The RAG evaluation picture is equally sobering. A benchmark comparing classic vector RAG against Google's new Open Knowledge Format showed vector RAG scoring 2 out of 7, OKF scoring 3 out of 7, and the combined pipeline scoring 4 out of 7. Nothing passes. The most instructive failure: a question about revenue calculation where the correct 2026 document (500 characters, one chunk) ranked 15th out of 85, behind a deprecated 2023 version that split into seven chunks and dominated the top-5 retrieval. Raising k to 15 does not help because you pull in six chunks saying the wrong thing against one saying the right thing. A reranker cannot fix it because nothing in the chunk text indicates which document is current. The composition failure is worse: a metric definition spanning three files retrieved two of three, then answered confidently with citations, never hinting anything might be missing. It goes quiet exactly when it is most dangerous. (more: https://old.reddit.com/r/LocalLLaMA/comments/1ve5r8y/i_benchmarked_classic_vector_rag_vs_googles_new/)

For the quantitative finance community, a curated collection of 97 libraries, 40-plus strategies, and 55 books provides a comprehensive map of the open-source systematic trading ecosystem -- the kind of resource that signals a field has matured past the point where everyone rolls their own from scratch. (more: https://github.com/paperswithbacktest/awesome-systematic-trading)

Governing the Intelligence Explosion

Employees from Anthropic, Google DeepMind, OpenAI, and Meta published a joint letter requesting that the U.S. government support an international effort to develop tools for deliberately pacing the frontier of automated AI development. The signatories include co-founders and chief scientists -- Chris Olah and Jared Kaplan among them. The letter's framing is stark: "The world is locked in a deadly race towards an intelligence explosion, where AI's ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction." The practical ask is narrow -- build the governance infrastructure before you need it -- but the subtext is that the people building the systems believe the competitive dynamics of their own industry cannot self-correct. One signer put it directly: "even after 3 years working on AI capability evaluations, the recent pace of progress has been a shock." Another compared it to splitting the atom. These are not outsiders speculating about risk; they are the people running the experiments. (more: https://www.pacingthefrontier.com)

The White House's OSTP published "Renewing America's Scientific Enterprise," a report organized around four goals: focus on the individual scientist, reform funding mechanisms, translate research into industrial capacity, and prepare for AI's impact on the research enterprise. It invokes Vannevar Bush's "Endless Frontier" and confronts Eroom's Law -- the observation that drug development costs have risen exponentially even as technology has advanced. The Genesis Mission initiative aims to restructure the roughly $200 billion annual federal R&D portfolio. Whether this report survives the next budget cycle is the question that matters; the ideas themselves are not controversial. (more: https://www.whitehouse.gov/science)

The Surveillance Infrastructure You Already Paid For

Over 100,000 Flock Safety ALPR cameras are deployed across the United States, and no warrant is needed to query them. The Fourth Amendment argument -- no reasonable expectation of privacy on a public road -- is the legal fig leaf, but the operational reality revealed by search logs from Wheaton, Illinois, tells a different story. The median search from this 55,000-person suburb queried nearly 16,500 cameras across more than a thousand networks. One in eight searches never used a license plate at all -- officers simply described vehicle characteristics. That is not a plate reader; that is a nationwide AI-enabled search engine running on the honor system.

The human cost is documented. A man named Robert was detained at gunpoint in Jacksonville when the system confused an O for a zero on his plate. Croissanna was accused of a $25 package theft in Colline Valley based solely on her truck passing through the neighborhood; the officer refused to show her the evidence and told her "we have cameras in that town and you can't get a breath of fresh air in or out of that place without us knowing." The actual thief, visible on Ring camera footage, bore no resemblance to her. The criminal defense lawyer in Houston offered the practitioner's perspective: Flock has not noticeably reduced violent crime in his experience, but it has been great for business.

There is no independent peer-reviewed study showing that Flock reduces crime. Flock's own white paper admits that the core question -- whether these crimes would have remained unsolved otherwise -- was never tested and "is unlikely to be solved at scale via quantitative analysis." Houston spent $6.39 million over four years on these cameras while the police department suspended over 264,000 case reports for lack of personnel to investigate them. The deflocking movement has responded with vigilantes who climb the poles to tape the lenses and clip the solar panel cables, security researchers who built ESP32-based WiFi detectors to map cameras Flock tried to hide, and a Florida gubernatorial candidate running on a platform of blocking taxpayer funding for the cameras. The Flock CEO, meanwhile, changes his car, his route, and his plates every day. (more: https://www.youtube.com/watch?v=Ju8rVbt05Z4)