Archived edition

Updates on artificial intelligence for mathematical research, proof discovery and formal verification, with links to papers, code and tools.

Research updatesSelect an entry to read more

1 entry · 3 linked sources News found
Kumori revises its Lean proof-repair evaluation after separating model contributions from built-in automation Kumori Published: 3 sources

Kumori reports that its earlier comparison overstated the contribution of small language models: a no-model control showed that Lean tactics alone solved all 11 problems attributed to a proof-skeleton pilot. In a separate evaluation, its model-free repair chain converted 145 of 2,197 rejected answers from 30 model lanes into Lean-accepted proofs, across 49 problems; on a held-back half of the problems, 52 of 1,038 rejected answers were repaired across 22 problems. The report says these were previously solved problems, so the results measure recovery of failed attempts rather than new mathematics. It also corrects an earlier ledger statement: eight answers had passed as written, rather than zero.

Research relevance:

A concrete example of how to evaluate AI-assisted theorem proving more carefully: compare against tactic-only baselines, hold out problems, freeze outputs before repair, and count distinct problems as well as proof attempts. The reported repair workflow may help salvage model-generated Lean proofs even when the model’s original code fails.

Claim status:
These are claims and counts reported by Kumori, with links to a public ledger and run records. The report says Lean judged the outputs; this digest did not inspect the linked repository artifacts or independently reproduce the checks. The update explicitly corrects earlier claims about what the model contributed and distinguishes repaired proofs from new results.
Limitations:
Kumori’s evaluation is on a fixed set of known problems and repeated, non-independent attempts; its repair chain uses Lean automation. The held-back result is 52 of 1,038 rejected answers across 22 problems, not a discovery rate. The authors note differences in models and tool versions between runs, and that the baseline is limited by its 30 recipes and timeouts. One linked GitHub page could not be fetched by the web tool.

Sources & further reading

Coverage and limitations
  • Read Kumori’s November 14 report directly, including its methods, results, caveats, and correction notes. Its linked public GitHub run and ledger were identified, but the web tool could not fetch them; their contents were not independently inspected.
  • Read Terence Tao’s November archive and searched official mathematics and Lean sources, recent papers, and related blogs for in-window research updates. Most pertinent items found were outside the specified window or were announcements without enough mathematical detail for inclusion.
  • X API leads were supplied for 15 accounts. Direct posts were supplied for Scott Armstrong, Daniel Litt, Alex Kontorovich, Boaz Barak, and Axiom; the other 10 accounts had empty results. Collections were bounded, exclude replies and reposts, may include cached posts, and API resolution does not independently verify identities. A post’s presence does not establish identity or verify its claims.
  • Bluesky collection for xenaproject.bsky.social returned no posts, with a 300-post cap and replies and reposts excluded. The Mathstodon collection for tao@mathstodon.xyz was unavailable due to an HTTP error, with the same cap and exclusions. Empty or unavailable feeds are not evidence that no posts were made.
  • No exhaustive X timeline access was available. Social and web indexing can lag; indexing dates were not treated as post dates. No Lean code was run and no proof was verified by this digest.

Requested coverage window: 2026-11-14T18:15:01.790779+01:00 — 2026-11-16T06:15:01.286842+01:00.

Public web research; coverage is not exhaustive. A source link is not a certification of a claim.

Archives 79
Older editions (49)