Structured Outputs with Ollama

1. ollama

   1 788 178,549 9.8 Go

   Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

   

   I authored the blog with some other contributors and worked on the feature (PR: https://github.com/ollama/ollama/pull/7900).

   The current implementation uses llama.cpp GBNF grammars. The more recent research (Outlines, XGrammar) points to potentially speeding up the sampling process through FSTs and GPU parallelism.

2. SaaSHub

   www.saashub.com sponsored

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. guidance

   2 35 21,712 8.9 Jupyter Notebook

   A guidance language for controlling large language models.

   

   Hi, just wanted to say how much I appreciate your work.

   I'm curious if you have considered implementing Microsoft's Guidance (https://github.com/guidance-ai/guidance)? Their approach offers significant speed improvements, which I understand can sometimes be shortcoming of GBNF (e.g https://github.com/ggerganov/llama.cpp/issues/4218).

4. llama.cpp

   3 1,073 123,686 10.0 C++

   LLM inference in C/C++

   

   If anyone needs a more powerful constrain outputs, llama.cpp support gbnf:

   https://github.com/ggerganov/llama.cpp/blob/master/grammars/...

5. llguidance

   4 13 841 8.7 Rust

   Super-fast Structured Outputs

   

   If you want avoid startup cost, llguidance [0] has no compilation phase and by far the fullest JSON support [1] of any library. I did a PoC llama.cpp integration [2] though our focus is mostly server-side [3].

   [0] https://github.com/guidance-ai/llguidance

6. llgtrt

   5 2 71 9.0 Rust

   TensorRT-LLM server with Structured Outputs (JSON) built with Rust

   

Suggest a related project