Image: Mistral AI / The Decoder Mistral's Leanstral 1.5 Is an Open-Source Model That Found Bugs Your Tests Missed
Mistral released Leanstral 1.5 on July 2 — a 119B open-weights model built for Lean 4 formal verification that found five previously unknown bugs across 57 real open-source repositories.
Mistral released Leanstral 1.5 on July 2, and while it didn’t land with much fanfare, the results warrant attention. The model is purpose-built for Lean 4 formal verification and found five previously unknown bugs while scanning 57 real open-source repositories.
Formal verification is the practice of mathematically proving that code behaves as specified — not just testing it, but proving it. Until recently, it required specialists. The gap between “tests pass” and “this is provably correct” has been wide and expensive to close.
Leanstral is Mistral’s attempt to shrink that gap.
What the model is
Leanstral 1.5 is a 119-billion-parameter Mixture of Experts model with 128 total experts and 4 active per inference token, meaning it activates roughly 6.5B parameters per call despite the large total count. Context window is 256K tokens.
It’s fully open-weights under Apache 2.0 and trained specifically on Lean 4, the proof assistant used in formal mathematics and increasingly in software verification. Training used a three-stage process: mid-training, supervised fine-tuning, and reinforcement learning with CISPO (Mistral’s process optimization approach).
Benchmark results
On miniF2F, a standard formal math benchmark, Leanstral 1.5 scores 100%. On PutnamBench, which covers 672 problems from the Putnam Mathematical Competition, it solves 587. On FATE-H and FATE-X (master’s and doctoral-level algebra tasks covering group theory and ring theory), it scores 87% and 34% respectively, placing first among open-source models on both.
These are math benchmarks, not software benchmarks, but Lean 4 bridges the two domains — you can write software specifications and proofs in the same language.
The bug results are the real story
Benchmark scores on curated datasets are one thing. Running against real code is another.
Mistral fed Leanstral through 57 open-source repositories and had it generate formal property checks — essentially asking the model to specify what should be true about the code and then verify those properties against the implementation. Out of 47 flagged violations, 11 turned out to be genuine bugs. Five of those had never been reported on GitHub.
One confirmed bug was an integer overflow in the Rust varinteger library. It’s the kind of issue that might never surface in typical testing because it only triggers at specific input sizes, but a formal proof of integer bounds would catch it immediately.
Who this is for
If your codebase uses Lean 4, this model is a direct tool for extending your proof coverage. Most codebases don’t, but Lean 4 adoption has been growing — particularly in cryptography, finance, and any software where correctness guarantees matter more than shipping speed.
For teams not already using Lean 4, the more interesting signal is directional: automated formal verification just got significantly more accessible. The tooling to translate specifications into Lean proofs is getting better, and a free frontier-quality model to discharge those proofs removes one of the main barriers.
Access
Leanstral 1.5 is available on Hugging Face and through a free API. There’s no commercial restriction on the weights — Apache 2.0 allows both research and production use.
Sources: Mistral AI · The Decoder · MLQ News