Programmers build different algorithms by chaining the same reusable primitives — conditionals, loops — in different orders. Language models do the opposite: whatever the input, their dozens of layers always execute in one fixed order. A 32-page paper posted to arXiv on November 2, Library of Layers, breaks that rule. Its six authors, including Pascal Frossard (EPFL) and Micah Goldblum (NYU), let the model decide its own layer ordering at inference time.
A Library Instead of a Fixed Stack
LoL has three parts: a fixed prelude and coda at both ends, and a routed core in the middle that draws from a library of L transformer blocks. A router selects and orders those blocks per window to build the execution sequence of each forward pass. Library size L and execution depth D are independent knobs: shrink L and you get looped transformers (the same blocks reused over and over); grow L and you approach sparse MoE (most blocks idle). The paper folds dense, depth-recurrent, and MoE architectures into one family this way. The 360M model uses 4 prelude layers, 16 routed depths, and 4 coda layers; a "growing" schedule trains shallow first and deepens later, spending most compute on a shallow model.
Routing Paths Are Readable — and Steerable
The interpretability experiments are the most striking part. After training a 360M model for 10 billion tokens on an equal mix of 10 FineWeb-2 languages, PCA of the routing paths clusters cleanly by language, with Latin-script languages sharing similar paths — same layers, different order. On 8×8 mazes, five algorithms (BFS, DFS, A*, bidirectional search, dead-end filling) share identical output syntax and differ only in the order they visit cells; LoL learns five distinct layer orderings, and forcing one routing path makes the model follow that algorithm at nearly 100% accuracy. Sudoku works the same way: forcing a path switches among five solving strategies. In a safety experiment, the model trains on web text carrying harm scores but never sees the labels; a linear probe fitted afterwards on router probabilities alone separates safe from unsafe text at 0.95 sequence-level AUROC.
No Performance Giveaway
A 1B-parameter, 16-layer model (4+8+4) trained on a Nemotron + OpenMath mixture for roughly 90 ExaFLOPs lands within two standard errors of dense baselines on all five benchmarks — ARC-Easy 50.3, PIQA 59.7 — and clearly beats the full-attention vanilla model on ARC-Easy (40.8). The 360M library-size sweep shows LoL competitive with the looped-transformer baseline Huginn, and against a 3:2 sparse MoE it trains at 2.8 vs 2.75 loss; the authors note that slightly higher loss coexisting with strong downstream performance is not unusual in latent-reasoning models.
So What
This is far from production, but it opens two doors. First, test-time scaling: inference-time depth can exceed training-time depth, and the architecture leaves that door open by design. Second, controllability: switch a routing path and the model switches its solution strategy, so future alignment work might edit routes instead of weights. When models start composing their own computational graph, "architecture" stops being a fixed thing. Paper: arXiv:2611.03526.