Assertion 1
Trail of Bits
Assertion status: No spot-check verdict is published for this assertion.
Trail of Bits characterizes a report published by 1Password on August 6, 2026, regarding AI-generated patches as flawed and misleading.
No stored spot-check names this claim in this edition.
Trail of Bits asserts that 1Password's reported 26% clean-fix rate is misleading because it includes experiments where agents were instructed to apply incorrect fixes.
No stored spot-check names this claim in this edition.
Trail of Bits asserts that 1Password's reported 26% clean-fix rate is misleading because it combines trials where agents could test their patches with trials where testing was prohibited.
No stored spot-check names this claim in this edition.
Trail of Bits asserts that 1Password's reported 26% clean-fix rate is misleading because the study selected vulnerabilities specifically for their complexity rather than as a representative sample.
No stored spot-check names this claim in this edition.
Trail of Bits asserts that 1Password's reported 26% clean-fix rate is misleading because the models were run at default reasoning settings rather than their highest available settings.
No stored spot-check names this claim in this edition.
Trail of Bits states that in trials from 1Password's data where agents could test patches and were not instructed to apply wrong fixes, 86% of the generated patches blocked the supplied exploit.
No stored spot-check names this claim in this edition.
Trail of Bits argues that 1Password's grading criteria penalize intended behavior changes, citing that 8% of ActiveMQ verdicts were penalized for this reason.
No stored spot-check names this claim in this edition.
Trail of Bits states that automated grades from 1Password's study disagreed with human review, with models grading their own patches matching human reviewers on the full five-category outcome in 65.9% of cases.
No stored spot-check names this claim in this edition.
Trail of Bits states that two different models assigned different outcomes to 36.8% of the same patches in 1Password's study.
No stored spot-check names this claim in this edition.