Capability benchmark · Biology · Medicine

SpatialBench-Long probes agents on end-to-end spatial biology reasoning

A long-horizon spatial biology benchmark requiring agents to recover biological claims from raw or near-raw measurement data.

Summary

SpatialBench-Long introduced a benchmark for long-horizon spatial biology analysis across cancer, organoid, lineage-tracing, aging, and intervention systems. The authors hardened candidate claims through reproduction, independent scientist review, and trajectory inspection, and reported only 11.1% success for the best tied model-harness pairs.

AI role

Agents analyzed spatial and single-cell biology datasets, selected methods, and produced controlled-vocabulary answers graded against hardened claims.

Narrative role

This adds a current biology benchmark that targets the harder step from running analyses to deriving accurate biological conclusions from complex measurements.

Caveat

The benchmark is small and preprint-stage, and the low success rates reflect particular harnesses, rubrics, and curated biological claims.

Event

Date: 2026-05-27
Timeline role: Notable
Progress relevance: Capability indicator
Validation: Benchmarked

Evidence

Preprint · Benchmark

Observed metrics

evaluations: 24; SpatialBench-Long contains 24 spatial biology evaluations.
best tied result: 8/72 runs; Three model-harness pairs tied at 8 successful runs out of 72.
best tied success rate: 11.1%; The reported best tied result corresponds to 11.1% across the benchmark.

Source: arXiv ↗