PPhying中文
Curated security research, vulnerabilities, advisories and tools for practitioners.

Introducing dfbench v1

depthfirst.com | research | #benchmark | #ai-agents | #vulnerability-detection | #evaluation | #defensive-security | #dfbench
📄 Raw .md⬇️ Download .md🔗 Original ↗

Summary

Depthfirst introduces dfbench, a held-out benchmark for defensive security agents: it scores detection, validation and differential analysis, measuring coverage, precision and cost as code changes.

Why it matters

This research provides technical context that security teams can use for monitoring and validation.
Published
Collected

Related coverage

research · nebusec.ai
Beyond the Model: Benchmarking Codex Security, Claude Security, and Nebu
Nebu benchmarks its security pipeline against Codex Security and Claude Security on 76 known bugs; with GPT-5.4 it matches the best Codex result in less time at lower cost.
blog · about.gitlab.com
GPT-6 Astra on GitLab: Faster runs, fewer tokens used
GitLab's internal eval of GPT-6 Astra on Duo Agent Platform: up to 43.4% faster typical runs than GPT-5.6 Sol (49.2% at p95) and 42.7% fewer tokens, traded against a lower resolution rate.
research · depthfirst.com
Training State of the Art Vulnerability Discovery Agents through Reinforcement Learning
depthfirst introduces dfs-mini1, a security model co-trained with its harness via reinforcement learning; it reaches Pareto optimality on OpenAI's EVMBench Detect and state-of-the-art pass@8.
research · theori.io
What are business logic vulnerabilities, and why are they so hard to catch?
Business logic flaws execute correctly but violate business intent — like a password reset token returned in the HTTP response — and Xint explains why SAST and human testers miss them.
research · theori.io
Building Effective LLM Agents | AI Cyber Challenge
Lessons from building LLM agents for DARPA’s AI Cyber Challenge: decomposing tasks, curating toolsets and adapting to models, applied to vulnerability detection, PoV generation and patching.
research · theori.io
Offensive Security vs. Defensive Security: Navigating the Two Pillars of Cybersecurity
Xint compares offensive and defensive security — red teams simulating attacks versus blue teams hardening defences — and explains why a holistic programme needs both disciplines working together.
Back