PPhying中文
Curated security research, vulnerabilities, advisories and tools for practitioners.

Can chatbots craft correct code?

blog.trailofbits.com | blog | #ai-security | #ai-safety | #llm | #formal-verification | #code-generation | #program-analysis | #determinism
📄 Raw .md⬇️ Download .md🔗 Original ↗

Summary

Unlike compilers, LLMs lack determinism and semantic guarantees, making them useful coding assistants but unreliable for autonomous code generation without human review and formal verification.
Published
Collected

Related coverage

blog · tenzai.com
Five Models Walk Into an App Builder. Security Gets Interesting.
# Five Models Walk Into an App Builder. Security Gets Interesting. Eran Machlev Security Researcher Share We built the same three apps with five models on Base44 and ran Tenzai's AI hacker against...
blog · blog.trailofbits.com
Auditing in the age of (good enough) AI
Trail of Bits explains how agents built tooling and a Lean model for the Miden zkVM audit, finding a flaw enabling Falcon signature forgery and yielding 95 machine-checked proofs.
blog · blog.trailofbits.com
A “proof” of Fermat’s Last Theorem that fits the margin
Trail of Bits found a Lean ≤4.33.1 bug: String.Pos.Raw.extract returns a full string where its spec says empty — enough to 'prove' Fermat's Last Theorem. Not a soundness flaw; fixed in 4.34.0-rc1.
blog · blog.trailofbits.com
VMs won't contain cyber-capable agents
This technical commentary challenges prevailing assumptions about virtualization boundaries when hosting autonomous cyber AI models.
blog · blog.trailofbits.com
Why fuzzing over formal verification?
Trail of Bits explains why it favors fuzzing over formal verification in most audits: proving bug absence is often unattainable, while fuzzing finds similar bugs with far less skill and time.
blog · blog.trailofbits.com
Cryptographic design review of Ockam
Trail of Bits reviewed Ockam's protocol design for end-to-end encrypted channels, examining its Noise XX-based secure channels and applying formal verification with Verifpal and CryptoVerif.
Back