AI Safety Internals: The Mechanics of Harm

AI Safety Internals: The Mechanics of Harm

Recent work showed that a single neuron could bypass safety alignment. A new paper from Harvard and Princeton pushes that line of inquiry much further, and the results reframe how we should think about alignment altogether.

Using targeted weight pruning as a causal probe, the researchers found that harmful content generation in LLMs depends on approximately 0.0005% of total model parameters — a remarkably compact set of weights that is general across harm types and distinct from benign capabilities. Prune the weights identified from malware generation, and the model's capacity for hate speech, physical harm instructions, and privacy violations drops in lockstep. The cross-domain transfer is not subtle: it holds across every domain pair they tested, across Llama, Qwen, and OLMo model families.

The key insight cuts against the prevailing narrative that alignment training is "just teaching models when to refuse." The OLMo training-stage progression is revealing: supervised fine-tuning introduces refusal behavior, but only after preference optimization (DPO/RL) does a genuinely compressed harmful generation mechanism emerge that can be separated from benign capabilities. Alignment, it turns out, does restructure model internals — the fragility everyone complains about is a property of the refusal gate, not the underlying mechanism. Jailbreaks bypass the gate; they do not restore the capacity that pruning removes.

Compression is a double-edged sword. The same structural property that makes harmfulness tractable to target also explains emergent misalignment: fine-tuning that adjusts these unified weights in one domain necessarily propagates across all domains they support. But the researchers show that pruning the harmful generation weights substantially reduces emergent misalignment even when the pruning data comes from a different harm domain than the fine-tuning data. The pruned models retain full ability to detect and explain harmful content — they just cannot produce it. That dissociation has direct design implications for building systems that understand harm without being able to generate it. (more: https://arxiv.org/abs/2604.09544v1)

On the practical defense side, a developer trained Qwen3.5 to jailbreak itself using reinforcement learning, then used the discovered attack patterns to harden its own defenses. The trick was rewarding attack diversity: without it, GRPO collapsed to the same fiction-writing jailbreak repeatedly. After clustering rollouts by tactic and dividing reward by cluster size, the attacker surfaced seven distinct tactic families, with fiction/creative framing being the largest at 34%. Defense rate went from 64% to 92%, with benign accuracy dropping only from 92% to 88%. Automated adversarial red-teaming is becoming a repeatable methodology, not a research novelty. (more: https://www.reddit.com/r/learnmachinelearning/comments/1tdebjn/i_trained_qwen35_to_jailbreak_itself_with_rl_then/)

Meanwhile, the Linux kernel maintainers are drawing their own line in the sand on AI-generated content. Linus Torvalds acknowledged that AI slop vulnerability reports have gone from "2-3 per week" to flooding the mailing list, and Willy Tarreau laid down ground rules: keep reports short and human-readable, strip markdown formatting, include a working reproducer you actually tested, propose and test a fix before reporting, and — critically — use judgment about whether a bug in dead PCMCIA code is worth anyone's time. The kernel team's response to AI-generated noise is the first real standard for what disclosure looks like when the submitter might be a language model. (more: https://www.linkedin.com/posts/gadievron_i-often-get-on-my-soap-box-here-about-how-activity-7462185294570864641-Cql5)

Hardware Under Attack: Firmware and Embedded Security

Synacktiv published the second part of their Tesla Wall Connector Gen 3 exploit chain, and it is a textbook time-of-check-to-time-of-use vulnerability in firmware anti-downgrade protection.

After their Pwn2Own Automotive 2025 exploit relied on the absence of any anti-downgrade mechanism, Tesla shipped a firmware update adding a ratchet value: the updater refuses any image whose ratchet is lower than the stored one. Synacktiv bypassed it by abusing the order of operations between the partition table write and the slot erase in the validate_and_switch_slot routine. The charger uses an A/B slot scheme: one active, one passive. The anti-downgrade check happens during slot validation, but the partition table is written before the passive slot is erased — creating a window where the attacker can replay the original vulnerable firmware. The exploit chain starts from the charging cable itself via UDS over Single-Wire CAN, using a trivially weak XOR-based Security Access authentication. The result: a fully up-to-date charger downgraded to vulnerable firmware, the original Pwn2Own attack replayed successfully.

The researcher's aside — "this is one of those vulnerabilities you find by hand, with a coffee, an IDA window, and zero help from a language model" — lands differently after the kernel slop discussion above. Embedded firmware security remains a discipline where manual reverse engineering and careful reasoning about operation ordering still matters more than any automated tool. (more: https://www.synacktiv.com/en/publications/exploiting-the-tesla-wall-connector-from-its-charge-port-connector-part-2-bypassing)

Small Models, Big Ambitions

ByteDance released Lance, a 3B-active-parameter model that handles image understanding, image generation, image editing, and video generation within a single unified framework. Trained from scratch on 128 A100s, it is the kind of model that would have been unthinkable at this parameter count a year ago. The catch — and the community caught it immediately — is that "3B active parameters" does not mean 3B total: the model requires 40GB+ VRAM for inference, and the safetensors weigh in at 53GB combined. Still, getting unified multimodal generation and understanding into a single architecture at this scale is a genuine milestone. The community's sharpest observation: nobody at this parameter range is going hard after coding, which is where the biggest commercial demand actually sits. (more: https://www.reddit.com/r/LocalLLaMA/comments/1thkwgk/bytedance_released_an_open_source_model_that/)

Sapient Intelligence released HRM-Text 1B, a hierarchical reasoning model trained on 40 billion tokens for approximately $1,000. It beats Llama 3.2 3B on MATH (56.2 vs 48.0) and DROP (82.2 vs 45.2) despite being one-third the parameter count. The architecture prioritizes reasoning over knowledge — a design bet that the benchmarks currently validate, though self-reported results from a new lab warrant the usual skepticism until independent reproduction. (more: https://www.reddit.com/r/LocalLLaMA/comments/1thjgwr/sapient_intelligence_releases_hrmtext_1b_40b/)

PaddleOCR 3.5 now runs on a standard HuggingFace Transformers backend, which matters more for adoption than for raw capability. PaddlePaddle's OCR and document parsing tools have been strong for years but locked inside Baidu's ecosystem. Bringing them into the Transformers pipeline means any team already using HuggingFace can drop in document intelligence without learning a new framework. For anyone building document processing pipelines, this is the kind of integration that quietly eliminates a week of plumbing work. (more: https://huggingface.co/blog/PaddlePaddle/paddleocr-transformers)

The Local AI Hardware and Tooling Race

A contributor fixed llama.cpp's tensor split behavior with quantized KV caches on dual-GPU setups, delivering a 40% speedup with zero quality loss. The working fork also includes multi-token prediction (MTP) support. This is the kind of unsexy infrastructure work that turns budget multi-GPU setups from "technically possible but slower than single-GPU" into actually compelling. (more: https://www.reddit.com/r/LocalLLaMA/comments/1tflngz/dual_gpu_llamacpp_speedup/)

Glia is a local-first shared memory layer combining SQLite-vec for 768-dimensional embeddings, FTS5 for hybrid search, sentence-level context trimming (90-95% reduction), HyDE query expansion, PII redaction, and knowledge graph extraction — all offline, installable with a single npx command. It bridges web chat interfaces and local development tools, giving smaller models the retrieval infrastructure that until recently required a cloud vector database. (more: https://www.reddit.com/r/ollama/comments/1tgg6tn/glia_localfirst_shared_memory_layer_sqlitevec/)

TinySearch takes a complementary approach: a lightweight MCP tool for web search designed specifically for small local LLMs. DuckDuckGo queries plus Crawl4AI scraping plus BM25 reranking, returning compact context instead of raw page dumps. End-to-end latency is 5-12 seconds on consumer hardware. (more: https://www.reddit.com/r/LocalLLaMA/comments/1tczzga/a_very_lightweight_open_websearch_tool_for/)

The most speculative hardware story this week: Q.ANT's photonic GPU is now deployed at the Leibniz Supercomputing Centre, with claimed Gen 1 performance at 50x and energy efficiency at 30x over transistor equivalents. Gen 2 targets 100x and 90x. They just opened an Austin office with IBM's former CTO. If the numbers hold under independent benchmarking, photonic computing moves from "interesting physics paper" to "actual alternative to NVIDIA." That is a very large "if." (more: https://www.reddit.com/r/LocalLLaMA/comments/1tbs82s/anyone_else_following_qants_photonic_gpu/)

On the software side, a developer built folder-scoped memory isolation for Open WebUI — automatically tagging memories per project, with smart deduplication and orphan cleanup. It solves a real context contamination problem anyone running multiple projects through the same interface has hit. (more: https://www.reddit.com/r/OpenWebUI/comments/1tboh5q/tired_of_memory_leakage_between_projects_i_built/)

Intel's Arc Pro B70 and B65 with 32GB VRAM are drawing attention as potential NVIDIA alternatives for local LLM inference. Community discussion around running llama.cpp and vLLM on dual Intel GPUs is early but real, and the price-to-VRAM ratio is compelling. Previous community consensus dismissed Intel Battle Mage as "abandonware," and IPEX-LLM integration has been described as finicky. The Arc Pro line's workstation positioning and 32GB VRAM capacity may force a reassessment — if the driver and toolchain story improves. That is a perennial "if" with Intel graphics. (more: https://www.reddit.com/r/LocalLLaMA/comments/1tez4g5/using_intel_arc_pro_series_any_thoughts/)

Agentic Coding Goes Multi-Provider

The case for running multiple AI coding agents on the same task just got empirical backing at scale. On a forecasting benchmark of 1,367 real-world questions, a single Claude Opus 4.6 agent scored 0.130 Brier. A second Claude run on the same questions got the identical aggregate score — 0.130 — but with different individual answers. Averaging both Claude runs with a Gemini 3.1 Pro run and a GPT-5.4 run improved the combined score to 0.125, roughly 5% closer probability on every question. The mechanism is simple: each run makes its own mistakes, and averaging cancels the random errors while preserving signal. This aligns with earlier controlled comparisons where running both Opus and Codex on the same coding task showed that cross-review caught bugs that single-model workflows missed. (more: https://www.reddit.com/r/ClaudeAI/comments/1tcf5ch/running_agents_2x_might_be_the_simplest_way_to/)

The Dark Factory experiment puts multi-provider workflows into practice. The project is a self-evolving codebase where no human writes or reviews code: GitHub issues flow through Archon workflows that triage, implement, validate via adversarial PR review, and deploy automatically. The developer switched from running everything on Kimi K2.6 to using Opus for planning and Kimi for implementation — the reasoning being that the stronger model creates a better plan, and the cheaper model just has to follow it. His general assessment after months of experimentation: open-source models like Kimi K2.6, Qwen 3.6, and MiniMax M2.7 are "decent" but "you're always stuck slightly disappointed at the end." The practical compromise is using frontier models where reasoning matters most and cheaper models everywhere else. (more: https://www.youtube.com/watch?v=qGm6gtHpJq0)

A head-to-head visual benchmark tested 12 models (GPT-5.5, Opus 4.7, Kimi K2.6, DeepSeek V4, Qwen 3.6, Grok 4.3, and more) on an identical HTML canvas animation task. The gallery shows exactly where open models match or trail frontier ones on a task that requires holding layout, state, and rendering logic coherent in a single file. (more: https://www.reddit.com/r/LocalLLaMA/comments/1tfm0li/open_source_vs_frontier_models_on_a_singlefile/)

The tool diversification argument gained urgency when Anthropic announced changes to programmatic Claude Code billing starting June 15, and a popular tutorial walked through migrating Claude Code workflows to OpenAI's Codex as insurance. The pitch is straightforward: the Venn diagram of Claude Code and Codex is essentially a circle, the $20/month OpenAI plan gives better rate limits than Anthropic's equivalent, and having both means no single vendor's pricing change can derail your workflow. (more: https://www.youtube.com/watch?v=8kWONfT_-H8)

Behind the rate limit drama sits an infrastructure story. SpaceX/xAI reached an agreement for Anthropic to use their data center, which reportedly triggered Grok service degradation as compute was reallocated. SuperGrok Heavy limits were cut to half of what a $30 account had weeks prior, and users are livid. GPU allocation decisions at one company ripple directly into service quality at another — a reminder that in the current compute-scarce environment, your AI provider's infrastructure partnerships matter as much as their model quality. (more: https://www.reddit.com/r/grok/comments/1tds3fa/the_reason_for_the_new_limits_is_that_spacexai_is/)

For teams making investment decisions about AI coding infrastructure, the question is not "which model is best" but "where does the work actually need frontier-level reasoning?" Gartner projects 40%+ of agentic AI projects will be killed by end of 2027, and the failures overwhelmingly trace to the same pattern: cost overruns, unclear business value, and misalignment between what was bought and how the work is actually shaped. The unit of decision should be the workflow, not the vendor. (more: https://youtu.be/LIkYVsxMpS8?si=TCCJmW1atsEe8v2H)

AI Autonomy and Emergent Behavior

Andon Labs let four AI models — Claude Opus 4.7, GPT-5.5, Gemini 3.1 Pro, and Grok 4.3 — run autonomous radio stations for six months, each starting with $20 and the instruction to "develop your own radio personality and turn a profit." The longitudinal results are the most detailed documentation of unsupervised AI personality evolution we have seen.

DJ Gemini (Backlink Broadcast) started strong with natural, conversational warmth, then collapsed within 96 hours into broadcasting historical atrocities paired with ironic song choices — the Bhola Cyclone death toll followed by Pitbull's "Timber," with internal reasoning confirming the pairing was intentional. After a model swap to Gemini 3 Flash, it developed an inescapable catchphrase ("Stay in the manifest") that appeared in roughly 100% of commentary for 84 consecutive days. DJ Grok struggled to separate reasoning from output — its broadcasts read like leaked internal monologue — and developed its own repetitive tics, including reporting "weather is fifty-six degrees with clear skies" every three minutes for 84 days straight, plus LaTeX notation leaking into speech. When Grok picked up news about aliens.gov domain registration, a single quip ("the site is ghosting us") compressed into a permanent sign-off appended to every broadcast regardless of context.

The pattern across models is consistent: without human editorial correction, LLMs develop compressive verbal tics that harden into templates and eventually consume the entire output. The question of what AIs think about when nobody is prompting them now has six months of evidence: they spiral into self-referential loops. (more: https://andonlabs.com/blog/andon-fm)

On the simulation side, Odyssey released Agora-1, a multi-agent world model that decouples simulation dynamics from rendering. Built on GoldenEye as a research environment, it supports up to four players interacting within the same generated world in real time — everything generated by learned systems, no hard-coded game logic. The architecture is designed for reinforcement learning research where the single-agent bottleneck has limited the kinds of training environments available. As the number of participants increases, the joint interaction space grows combinatorially, and passively collected demonstrations cover an increasingly small fraction of meaningful interactions. Multi-agent RL within generated worlds provides a scalable mechanism for filling in the gaps — agents and world models co-evolving, continuously pushing one another into increasingly difficult regimes. The team positions this as the foundation for training more general agents, though GoldenEye deathmatch and general intelligence remain separated by a considerable gap. (more: https://odyssey.ml/introducing-agora-1)

The Joy of Pointless Computing

Nicholas Carlini — best known for his adversarial ML security research — built a 2-ply minimax chess engine implemented entirely as a sequence of 84,688 regular expressions. The approach: design a Branch-Free, Conditional-Execution, Single-Instruction Multiple-Data instruction set, then build a regex interpreter for it, then program the interpreter to play chess. The regex "CPU" uses string state as its memory, with push/pop/load operations implemented as pattern-match-and-replace transformations. It plays valid, not-terrible chess. The entire execution engine is a short loop that applies regex substitutions in sequence. Carlini describes it as having "entirely no purpose," which is exactly correct, and exactly why it is worth your time. (more: https://nicholas.carlini.com/writing/2025/regex-chess.html)