We hid backdoors in binaries — Opus 4.6 found 49% of them - Quesma Blog

A benchmark tested AI agents on their ability to detect backdoors in binary executables, finding that Claude Opus 4.6 could identify them only 49% of the time, with a significant rate of false positives. While AI is making binary analysis more accessible, current models are not yet reliable for production environments due to limitations in handling complex code and a tendency to miss subtle threats or flag benign code.