Illustration, AI-generated from the story text. Tap a panel to read the story through it.

1 / 3A transformer that picks the order of its own layers at run time×

EPFL and NYU researchers, with Micah Goldblum, propose Library of Layers, a model that keeps a shared library of transformer blocks and decides at inference which to run and in what order.

It matches dense, depth-recurrent and mixture-of-experts baselines on text pretraining; on sudoku and mazes it uses different orderings for different strategies, and the researchers can force a strategy by intervening on the router.

Sources and the full story

‹›

A transformer that picks the order of its own layers at run time

EPFL and NYU researchers, with Micah Goldblum, propose Library of Layers, a model that keeps a shared library of transformer blocks and decides at inference which to run and in what order. It matches dense, depth-recurrent and mixture-of-experts baselines on text pretraining; on sudoku and mazes it uses different orderings for different strategies, and the researchers can force a strategy by intervening on the router. SourcesA

Sources

• ALibrary of Layers: adaptively ordering transformer layers at runtime (arXiv 2611.03526, via Hugging Face)huggingface.co

This story ran in the Morning Brief of November 3rd 2026.