OmicsBench v1, a Latch Meta-Benchmark - Part 1
AI Summary: LatchBio released OmicsBench v1, a meta‑benchmark aggregating 784 short‑ and long‑horizon tasks from eight domain‑specific bioinformatics benchmarks to evaluate agentic capabilities on molecular (“omics”) data. Each task includes a Docker container, prompt, time limit, and automatic grader, and must satisfy verifiability, scientific validity, durability, and anti‑shortcut criteria verified by human and automated review. Evaluation of 31 model‑harness combinations on 741 short‑horizon and 43 long‑horizon tasks showed pass rates between 32.3 % and 49.0 %, with the highest short‑horizon scores achieved by Grok 4.7 (49.4 %), GPT‑6 Astra (49.3 %), and Opus 5 (47.7 %). The results indicate that current models are not yet saturated on the benchmark and that native harnesses provide no substantial performance advantage.