Structured Output from LLMs (Without Reprompting!)

• guidance

  89 12,248 9.5 Jupyter Notebook

  Discontinued A guidance language for controlling large language models. [Moved to: https://github.com/guidance-ai/guidance] (by microsoft)

  

I am unclear on the status of the project but here is the conversation that seem to be tracking it: https://github.com/microsoft/guidance/discussions/201

• llama.cpp

  769 55,846 10.0 C++

  LLM inference in C/C++

> Which brings me to the other approach: steering the LLM's output __as it is generating tokens__

A relevant PR:

https://github.com/ggerganov/llama.cpp/pull/1773

The plan is to support arbitrary grammar files to constrain tokens as they are generated, like the ones here:

https://github.com/antlr/grammars-v4

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• grammars-v4

  29 9,786 9.6 ANTLR

  Grammars written for ANTLR v4; expectation that the grammars are free of actions.

  

> Which brings me to the other approach: steering the LLM's output __as it is generating tokens__

A relevant PR:

https://github.com/ggerganov/llama.cpp/pull/1773

The plan is to support arbitrary grammar files to constrain tokens as they are generated, like the ones here:

https://github.com/antlr/grammars-v4

• gpt-json

  7 726 6.8 Python

  Structured and typehinted GPT responses in Python

I did a POC project with it recently. The guidance on gpt-3.5-turbo and gpt-4 models isn't as functional as plain gpt-3. I found I had better results using https://github.com/piercefreeman/gpt-json and it doesn't require multiple calls to the API. Not as feature filled, but it may meet your needs

Suggest a related project