AI-generated editorial illustration; not a documentary photograph.
Multiverse Computing has presented a method that treats transformer block removal as a constrained binary optimization problem equivalent to finding low-energy states in an Ising glass. In a September 21, 2026 post about its paper, the company said this formulation improves block selection when compressing large language models, particularly when many layers must be removed.
Modeling interactions between blocks
Conventional block-pruning methods often estimate each block's importance separately or restrict removal to one consecutive group. The proposed approach instead represents every block with a binary keep-or-remove variable and uses a second-order approximation of model loss. Its Hessian captures both individual block importance and pairwise interactions, allowing the optimizer to account for how one removal changes the effect of another.
A cheaper candidate search
The Hessian is calculated once through forward and backward passes on a small calibration dataset, according to the authors. Candidate block combinations can then be ranked with an energy calculation instead of repeatedly running and benchmarking the model. The same coupling matrix can also be reused for different compression targets.
Exact enumeration remains expensive at larger scales. Multiverse reported that checking roughly 29 billion configurations for removing eight of Llama-3.3-70B-Instruct's 80 blocks took about two days on one GPU. For larger searches, the formulation can be passed to QUBO-compatible classical, quantum or quantum-inspired solvers; the company said an open-source tabu solver found low-energy candidates in seconds on cases that could be checked against brute force.
Reported gains at deep compression
Across Llama-3.1-8B-Instruct, Qwen3-14B and Llama-3.3-70B-Instruct, Multiverse said its constrained binary optimization method matched or exceeded the block-removal baselines it evaluated. At 50% depth reduction for Llama-3.3-70B-Instruct without retraining, it reported an almost 23-percentage-point MMLU advantage over the strongest comparison method. For Qwen3-14B with 12 of 40 blocks removed, the reported lead was about 10 points.
Useful candidates beyond the minimum
Energy is presented as a quality proxy rather than a perfect predictor. The workflow therefore produces several low-energy configurations for evaluation instead of relying only on the minimum-energy result. In one Llama-3.1-8B-Instruct experiment, a higher-energy candidate that removed an early block reportedly surpassed the minimum-energy configuration on several benchmarks after light retraining.
The authors also applied the method without retraining to NVIDIA-Nemotron-3-Nano-30B-A3B-FP8, which mixes Mamba2, attention and mixture-of-experts layers. They reported improvements over block influence on AIME25 and GPQA for selected layer-removal settings. Source code has been published, but the post does not specify its license terms.
Source: Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem
SIGNAL TITANS
Leave a Reply