Multimodal models can build a working lab simulation from a video, but not the right apparatus

Lab2Sim, a benchmark of 218 science experiment videos with 11,881 rubric items, asks models to turn a video into an interactive virtual experiment that enforces each step's preconditions. Models readily produce pages that run, the authors find, but reproduce the apparatus poorly; their own pipeline, which writes the procedure as a state machine first, lifts apparatus fidelity by 24.15 points. SourcesA

Sources

• ALab2Sim: turning real-world experiment videos into interactive virtual experiments (arXiv 2611.02928, via Hugging Face)huggingface.co

This story ran in the Morning Brief of November 3rd 2026.