Giving LLMs a Formal Reasoning Engine for Code Analysis

LLM coding assistants are great at writing code. They're not great at proving things about it. It has been reported that a new project called Chiasmus aims to change that by giving LLMs a formal reasoning engine for code analysis — bundling Z3 for constraint solving and Tau Prolog for logic programming to answer structural questions that simple greps and pattern matching miss.

The problem with grepping through source

Anyone who's asked an assistant "Can user input reach this SQL query?" knows the pain. Grep finds strings, not paths. The result is a token-heavy, brittle back-and-forth: the model greps, reasons a bit, decides what to grep next, and repeats — often still missing transitive callers and subtle control-flow edges. Frustration is the emotional core here. You want certainty. You want exhaustive answers. Instead you get best-effort guesses and a lot of second-guessing.

How Chiasmus works

Chiasmus takes a different tack. It parses source with tree-sitter into typed ASTs, walks those trees to emit declarative facts (method defs, call relationships, imports/exports), then serializes them as Prolog facts so a logic engine can run formal queries over the call graph. It has been reported that the LLM stays in the loop for perception — understanding the question and filling templates — while Z3 and Tau Prolog do the heavy logical lifting, enabling exhaustive reachability, dead-code queries, and constraint checks with far fewer tokens and higher confidence.

Takeaways

Grounded in the neurosymbolic idea of combining neural perception with symbolic cognition, Chiasmus is a neat example of the hybrid tooling trend: don't throw away provable reasoning just because your UI speaks English. Adoption questions remain — limited language support today (TypeScript, JavaScript, Clojure), tooling integration, and the usual engineering debt of maintaining parsers and facts — but the promise is simple and attractive: less guessing, more guarantees. Will this finally tame the LLM guessing game around code structure? Time — and some real-world adopters — will tell.

Sources: yogthos.net, Lobsters