PPhying中文
Curated security research, vulnerabilities, advisories and tools for practitioners.

Evaluating Pentesting Agents for the Real-World

ethiack.com | research | #ai-security | #pentesting | #agentic-ai | #benchmark | #llm | #ai-pentesting | #evaluation | #ethiack | #ethibench
📄 Raw .md⬇️ Download .md🔗 Original ↗

Summary

Ethiack launches EthiBench, an open evaluation protocol for AI pentesting agents using realistic targets, structured ground truth with LLM-as-a-judge matching, and richer metrics than success rates.
Published
Collected

Related coverage

research · ethiack.com
Evaluating Pentesting Agents for the Real-World Part 2
Ethiack's EthiBench part 2 evaluates AI pentesting agents beyond single runs, covering significance under budget limits, cumulative detection, temporal behavior, and cost-effective target subsets.
blog · xbow.com
Grok 4.7 Is Different. So Is the Best Way to Use It.
XBOW tested Grok 4.7 on exploit crafting and agentic pentests: inside xAI's Build it improved substantially over 4.6, outside Build it slipped slightly—orchestration now matters as much as prompts.
blog · aikido.dev
Aikido launches external network pentesting
TL;DR: Aikido now pentests your external network. Attackers already test your perimeter every day, and with AI agents doing the grunt work, they're faster than ever. Now you can run the same test first. Give our agents one domain and they'll map the internet-facing systems you run and attack them from the outside, exactly like a malicious actor would.
blog · aikido.dev
Adding Aikido AI pentesting to your existing security stack
How to add Aikido's AI pentesting on top of the SAST, SCA and cloud tools you already run, from your first pentest through whitebox testing, continuous testing and remediation.
blog · tenzai.com
Five Models Walk Into an App Builder. Security Gets Interesting.
# Five Models Walk Into an App Builder. Security Gets Interesting. Eran Machlev Security Researcher Share We built the same three apps with five models on Base44 and ran Tenzai's AI hacker against...
blog · netspi.com
Introducing EchoBench: A Human Calibrated Benchmark for Autonomous Pentesting
EchoBench is a human-calibrated benchmark for agentic AI web pentesting; a score of 100 marks parity with associate pentesters across fidelity, difficulty-reach, OWASP breadth, and repeatability.
Back