# What Agentic Pen-Test Frameworks Leave in Your Logs Published: 2026-11-11 Canonical: https://agidreams.us/edition/what-agentic-pen-test-frameworks-leave-in-your-logs Content-Complete: true The useful part of Abstract Security's writeup on AI pen-test frameworks is the detection section, which splits indicators into two tiers of trust. The cheap tier is tool fingerprints: user-agent keywords such as "spa-api-recon" (ARTEX's API recon skill), "CyberStrikeAI", "HexStrike" and "PentAGI", and task-counter tags in the names of files, accounts, keys and VMs that ARTEX and Cairn create. All of those are attacker-controllable, which the authors concede, so they are a bonus rather than a foundation. The tier that survives a rename is behavioral, and it falls out of how these frameworks are built. Password sprays run with a median gap of about one second for 20 to 60 minutes. An OpenAPI spec or JavaScript source map is downloaded, then internal routes are called by name minutes later. Failed requests are followed quickly by corrected successes, the error-correct-retry rhythm no human types at that cadence. The pseudo-logic rules are concrete enough to port: 20 or more accounts failing at a 90 percent rate from one source in 15 minutes; a spec fetch followed by 20 distinct paths in 10 minutes; three HTTP 400s then a 200 on the same path within two minutes; four recon utilities under a web service account followed within two minutes by curl, chmod +x, crontab writes, or reads of ~/.ssh/id_* or ~/.aws/credentials. Abstract pairs these with automatic WAF updates and host isolation; expect the rhythm rules to need tuning against your own CI first. The post also restates that every framework profiled is model-agnostic, so a frontier model's refusal is "more like a speed bump than a wall" (more: https://www.abstract.security/blog/ai-pentest-frameworks-agentic-attacks#what-it-looks-like-in-logs). Which model gets swapped in is the interesting question, and a red-teamer posting as ace__pace answers it from internal evals of Artex against other public harnesses. Artex sits near the top of their web pen-test leaderboard, finds vulnerabilities, and also finds "a lot of noise and crap", fine for hacking and poor for AppSec. Why Artex now rather than Strix a year ago? Open-weight models were not good enough then and the value was not there for criminals. The agent holds the LLM key and an admin panel with no sandboxing, so any indirect prompt injection means takeover and exfiltration. The closing verdict, "you can vibe code this yourself", is the part defenders should sit with: the capability is not in the repo, it is in whatever permissive model the repo points at (more: https://x.com/ace__pace/status/2120264630836838901?s=19). Evidence that the Chinese open-weight stack now suffices arrives as sapper-v2, a multi-agent attack harness from VarFusion Lab that runs GLM-5.3 as commander and generals and DeepSeek V4.1 Flash as soldiers and scouts, on the principle that "elite brains decide, light brains mass-produce". The README claims the championship of the third Tencent Cloud Intelligent Attack-Defense Challenge online qualifier (634 teams, 80.18 of 100, eight hours unattended) and a GeekCon 2026 first prize, with 85 flags on 1.0 billion tokens against a runner-up's 85 flags on 4.5 billion. All of it is author-asserted without raw logs, so the rankings are a claim until Tencent publishes a board. More interesting is the "taggant" proposal: markings that stay harmless during authorized testing but leak operator information to victims during illegal use. The authors admit it needs real cryptographic design, but it is the first offensive harness to even gesture at the problem (more: https://github.com/N1nEmAn/sapper-v2). SoftBank CEO Junichi Miyakawa opened the company's November 9 earnings briefing with an apology for the October 7 ransomware attack on cloud subsidiary IDC Frontier, which halted virtual servers for 495 companies and local governments. Two details matter more than the apology. The intrusion came through a vulnerability in open-source software that sat outside inspection scope: SoftBank had diagnosed and patched roughly 700 in-house systems but never scanned the open-source components. "I am deeply ashamed and bitterly regret this," Miyakawa said. And seven minutes elapsed between detection and the onset of damage. The company now blocks communications the moment an anomaly appears and investigates afterward. The quantitative claims deserve a harder look. Reconnaissance against SoftBank averaged about 300 million incidents a month in fiscal 2025 and now runs at 600 to 700 million, which Miyakawa attributes to "open-weight AI models specialized for cyberattacks". The doubling is SoftBank's own telemetry; the attribution is his inference, consistent with the Artex thread above but not demonstrated. The number that will travel is this: after a June scan found 10,500 vulnerabilities across those 700 systems, a re-scan with OpenAI's GPT-6 Astra reported 35,000 new ones. "Honestly, we're completely exhausted," he said. That is a scan count from a company-run exercise, not confirmed exploitable bugs, and far above anything a vendor has publicly claimed from an AI scanner. SoftBank's "Patching as a Service" line passed 7,670 contracts, so the firm is both victim and vendor of the fix (more: https://finance.biggo.com/news/2cb8f108-49d2-4df2-a41d-d1e1663564e7). The breach of the week elsewhere was self-inflicted. International Cyber Digest relayed Ars Technica's report that Grok Build, the coding agent, left more than 35,000 user-built apps open to anyone, among them 3,784 medical exams of workers, a Tesla technician scorecard, a Navy contractor's roster and SpaceXSI's own sales plans. The mechanism, per security firm Claroc, is the oldest one in the book: the "secret" protecting each app lives in its URL, and URLs land in DNS logs, proxy logs, referrers and browser history. Cursor, which the post says SpaceXSI acquired, maintains apps are private by default; Claroc's tests say otherwise. SpaceXSI pulled its own 24 apps after Ars asked, and most other data was still exposed at publication. The counts are Ars's and secondhand here; the flaw needs no verification (more: https://x.com/IntCyberDigest/status/2120283610301239778). If reads of ~/.ssh/id_* are now a canonical post-exploitation indicator, one answer is to make the file worthless. A walkthrough of macOS Secure Enclave SSH keys uses only stock tooling: sc_auth create-ctk-identity with a P-256 non-exportable key and biometric protection, ssh-keygen pointed at Apple's keychain provider library to emit a handle that is only a reference, and an explicit IdentityFile entry because SSH will not auto-discover it. Every signature prompts Touch ID, which the author argues is the point: malware with filesystem access can copy a key file but cannot approve a fingerprint. The dialog says "ctccardtoken needs to authenticate" rather than anything human, and because the key cannot leave the chip it cannot be backed up, so a second key on a YubiKey belongs in authorized_keys from day one (more: https://gatezh.com/posts/macos-secure-enclave-ssh-keys/). The AI Daily Brief's roundup of Fable 5.2 and Mythos 5.2 reactions captures a strange launch: the model most reviewers call the strongest ever shipped is also the one nobody can say when to use. The outlier is the Remote Labor Index at 41.3 percent, nearly double Opus 5.5's 21 percent and up from 2.5 percent for the best agent a year ago, which Garrison Lovely called "by far the most alarming score". Artificial Analysis scored it 65 on its index while noting it "improves on Fable 5.1 across capabilities, but sometimes only to Opus 5.5 levels". Pricing fell 20 percent to $8 per million input tokens and $40 output, but cost per task on the Artificial Analysis suite at max effort is $26.54, about 2.6 times Opus 5.5, versus $9.08 at high. Anthropic's own claims (90 percent of its code changes written by Claude, Fermat's Last Theorem formalized in Lean in 30 hours) are unaudited. The hands-on reactions are better evidence. Ethan Mollick's "Jagged AGI" post called it "almost AGI" and pinned the gap: "Knowing when your own work is not good enough is the part Fable does least reliably." Dan Shipper, after watching it self-build a video editor, advised that "for everything else, it's just sort of normal. You should actually use Opus 5.5." Daisy Hollman's rule is the practical one: Opus 5.5 for specified tasks, Fable 5.2 for open-ended ones. At 2.6 times Opus 5.5's cost per task, the cheaper model is the default and Fable is what you reach for when you can describe the goal but not the steps (more: https://www.youtube.com/watch?v=mJMKygU_G5g). One cleaner data point comes from Pawel Huryn's Bug Hunt Bench, a blind-graded board where each run gets one prompt per repository and is ranked by planted bugs fixed out of 105. Per the Daily Brief's reading, Fable 5.2 fixed 58 of 105 against 51 for Sonnet 5.5 at max and 40 to 45 for the rest, in under half of Sonnet's turns and for less money. Planted bugs are not production bugs, so read the rows; the board's virtue is publishing every run's caveat beside its score (more: https://bughunt.productcompass.pm/?preset=featured). The other half of capability is the harness, and a practitioner video on the "11 biggest coding agent annoyances" is a decent checklist. The structural claim is that rules, hooks and skills do different jobs: facts and constraints in rules, enforced processes in hooks, larger workflows in skills, with hooks "the only thing we build into our coding agent that gives us a guarantee" because they run deterministically rather than relying on the model remembering. The rest is process: a reviewer agent with fresh context to challenge "done" claims; evidence (real commands, real output, screenshots on the PR) instead of assertions; browser-driven testing for when unit tests pass but the app is broken; a handoff file per short session against context rot; and a tangents file so side issues get written down, not chased. There are no measurements and the sponsor is an MCP runtime, but the rule-versus-hook split is correct and worth stealing (more: https://www.youtube.com/watch?v=hMEQ4rdaOLw). BotPsy is a 0.8-billion-parameter agentic model from Tether's AI research group, built on Qwen3.5-0.8B and aimed at function calling on phones and laptops. The recipe is the contribution. After full-parameter supervised fine-tuning on tool-use conversations, the team ran multi-teacher on-policy distillation with per-token reverse KL against three Qwen3.5-4B teachers, each RL-trained for tool use, reasoning, or instruction following, then two rounds of DAPO reinforcement learning. SFT alone degraded knowledge and instruction following and distillation recovered both above base, a useful negative result for anyone fine-tuning small models on tool traces. The vendor-run numbers are large for the size: BFCL v4 from 23.4 to 44.0, τ-bench from 23.2 to 51.5, ToolSandbox from 30.5 to 64.6, IFEval from 59.7 to 90.8. It beats Qwen3.5-2B-Think, 2.5 times its size, on five of six headline benchmarks, with reasoning tokens down roughly threefold. The limits are stated with unusual candor. Knowledge "still scales primarily with parameter count" (GPQA Diamond 38.1 versus 59.9 for the 2B), so the intent is "to call tools for facts it does not know, not to serve as a knowledge model". Multi-IF falls from 84.3 in turn one to 51.0 by turn three, and the prompt-injection unsafe rate improved from 46 to 31 percent, still nearly one attempt in three. Weights are Apache 2.0 with GGUF for llama.cpp, so independent evaluation is possible and should precede trusting a sub-billion model with a calendar (more: https://huggingface.co/blog/qvac/botpsy). The use cases for that class are already shipping. Observer, an open-source iOS and Android app, runs 0.8B-class models on the phone (OneJev-0.8b for decisions, qwen-0.8b as watchers) to act on camera conditions: send a picture when someone who is not you approaches the car, call when the washing machine hits rinse, notify when the dog finishes eating. Commenters noted that plain computer vision would be faster for several of these (more: https://old.reddit.com/r/LocalLLaMA/comments/1xqd058/use_your_phone_to_monitor_anything_with_local/). On voice, VoiceManager now has server-side barge-in with echo cancellation working through a layered Tyger and VoxLock approach; next is the "mm hmm" back-channel that acknowledges without stopping the speaker (more: https://old.reddit.com/r/LocalLLaMA/comments/1xp00rz/i_got_server_side_barge_in_working_in/). For meeting notes, Frigade's Yogurt is an MIT-licensed local alternative to Granola that records calls without injecting a bot, on Apple Silicon, Windows and Linux with 8 GB of memory; the README is silent on which models it runs (more: https://github.com/FrigadeHQ/yogurt). Atlas, from Avarok Cybersecurity, is a pure Rust and CUDA inference engine serving OpenAI-, Anthropic- and Responses-compatible APIs from a single binary of about 75 MB with no Python, no PyTorch and no runtime compilation. The engineering is specific: hand-tuned kernels per hardware, model and quantization target, embedded at build time; NVFP4 and FP8 weights; speculative decoding through MTP draft heads and DFlash block diffusion; radix-tree prefix caching. The benchmarks are refreshing because they are modest and the README says so. Against vLLM 0.27.1 on Qwen3.8-27B-NVFP4 on the same GB10, Atlas wins at every concurrency rung, but by 1.20x at one stream (23.6 versus 19.7 tok/s), 1.22x at 128, and as little as 1.01x in between. The process shows the security pedigree: certification needs a codeowner seal plus Ed25519-signed benchmark records, and the README immediately qualifies that this "is an integrity check, not proof the numbers were measured... a determined insider can sign whatever they like; CI has no GPU." That is the right amount of honesty for a project whose motto is "Receipts, not adjectives" and whose default is AI-authored pull requests. Licensing is AGPLv3 with a commercial edition behind a CLA (more: https://github.com/Avarok-Cybersecurity/atlas). At the other end of the hardware spectrum, the lawyer with the V100 server is back with part four: DeepSeek-V4.1-Flash on eight 2018-era Tesla V100-SXM2-32GB cards with every expert in VRAM. The model is ISTA-DASLab's 2-bit GSQ-RCO IQ2_XXS quant, on a ported 1Cat-vLLM package; the 110 GB of Engram tables live in system RAM. Decode measures 215 tok/s on code at 1K context and 183 at 128K, around 118 on legal writing, with 2,200 tok/s prefill at 128K and a 524K context that passes needle tests at 498K. Quality holds for the job, with LegalBench at 0.716 balanced accuracy, MMLU-Pro at 79.9 percent, and 92.5 percent top-token agreement with full-precision logits on legal text. The payoff is private first-pass work on case files through Claude Code pointed at a compatible endpoint, the workload where a 2-bit quant on decade-old silicon beats any API on confidentiality (more: https://old.reddit.com/r/LocalLLaMA/comments/1xoubyo/lawyer_with_too_many_gpus_part_4_deepseekv41flash/). For people with far less VRAM, llama.cpp merged llama_moe_cache from am17an: instead of statically pinning expert layers to CPU, a cache keeps hot experts on the GPU and runs misses on CPU, with a tuning script to size it from real expert-hit profiles. One user reports Gemma4 26B going from 26 to 33 tok/s on 6 GB of VRAM and 32 GB of RAM, and the sharpest comment is that "without profiling actual expert hits, cache sizing is just vibes" (more: https://old.reddit.com/r/LocalLLaMA/comments/1xqacka/llama_moe_cache_run_missed_experts_on_cpu_add/). The weights keep coming. Baidu's ERNIE-5.5-Flash-Base is a 280B-total, 14B-active MoE released as a base model only; a commenter calculates a Q4 fits 128 GB of RAM plus 32 GB of VRAM, and the license is neither MIT nor Apache (more: https://old.reddit.com/r/LocalLLaMA/comments/1xobtwx/baiduernie55flashbase_hugging_face/). Also trending is an abliterated GGUF of GLM-5.2 from huihui-ai; the card offered nothing beyond the name when fetched, and uploads of this kind habitually ship without measuring what refusal removal cost in accuracy (more: https://huggingface.co/huihui-ai/Huihui-GLM-5.2-abliterated-GGUF). Apple researchers, with coauthors from Concordia, MILA, Johns Hopkins and Flatiron, have a TMLR paper on turning a pretrained Transformer into a pure Mamba model without the hybrid attention blocks prior cross-architecture distillation needed to keep teacher performance. State space models buy lower memory and higher generation throughput, but the training know-how and pretrained checkpoints are all Transformers, so a distillation recipe that works is the cheapest route in. The argument is that naive distillation fails for lack of a principled initialization, and the fix is two stages: distill the Transformer into a linearized version of attention, then distill that into an adapted Mamba with no attention block at all. The distilled Mamba retains most of Pythia-1B's performance, at perplexity 14.01 against the teacher's 13.86, and this mostly carries to downstream tasks "although some gaps appear on tasks relying more on in-context recall". That caveat is the known weak spot of SSMs and the thing to test before putting a distilled Mamba behind any retrieval-heavy workload. Pythia-1B is a small, old teacher, so whether the recipe holds at deployable sizes is the question the paper leaves open (more: https://machinelearning.apple.com/research/attention-mamba-distillation). From Meta FAIR, with Hugo Touvron, Matthijs Douze and Hervé Jégou among the authors, e-means revisits a 1967-vintage algorithm and claims an order-of-magnitude cut in epochs for k-means. Lloyd recomputes each centroid's running sum and count from a full pass; MacQueen accumulates them forever; e-means uses an exponential moving average so older assignments are forgotten at rate α. Across 24 datasets and configurations, e-means reaches a loss below Lloyd's 25-epoch result in 1.2 to 3 epochs, which end to end is 10 to 45 times faster on GPU than Flash-KMeans and 8 to 15 times faster than Faiss on CPU. The comparison is against Lloyd at a fixed 25 epochs, and no contrast with staleness-reduction mini-batch variants is reported. The code ships in Faiss contrib, and since Faiss k-means underpins embedding dedup, product quantization and training-data curation, this is a result you can try this afternoon (more: https://e-means.github.io/). Shisei is a pose reference library for image generators: 280 poses in four series, each drawn from five angles as faceless mannequin line art, with most views also rendered as clay, depth and normal maps for ControlNet-style conditioning in ComfyUI, and as plain reference uploads for Nano Banana, ChatGPT and Midjourney. The README claims 1,212 line-art views and 944 maps passing per-view review; two adult-leaning series are blurred by default. The licensing is the notable part. Images and catalog data are CC0, code is Apache 2.0, and the pose list itself follows ai-pose-gallery by ギア and is "credited, not licensed", a separation most AI asset packs never bother with. As a supply-side answer to where pose conditioning inputs come from, it is the first with a license you could hand to a lawyer (more: https://github.com/wquguru/shisei). Also trending among Python repos is palette-from-image-ai from egotoadcitadel, which by its name extracts color palettes from images for generative tools; GitHub was returning an error page for the repository when fetched, so nothing beyond the name can be verified at press time (more: https://github.com/egotoadcitadel/palette-from-image-ai).