BioSecBench-Structure: Can AI Agents Analyze Molecular Structures?
AI Summary: The paper introduces BioSecBench-Structure, a benchmark of 58 pathogen‑related structural biology tasks designed to evaluate AI reasoning on experimental data such as crystallography, cryo‑EM, predicted structures, and HDX‑MS. Across 20 models and 35 configurations, the highest overall pass rate was 43.7% achieved by the Fable 5.2 / Claude Code configuration, with other top scores ranging from 31.0% to 41.4% and confidence intervals overlapping substantially. Disabling model refusals increased countermeasure task performance by 14–15 percentage points for Fable 5.2 and Opus 5.5, while pathogen‑characterization scores changed little, highlighting persistent errors in density interpretation and reference‑structure selection. The benchmark also reports resource usage, showing wide cost and time variations among models.