guardrails VS dynamic-gpt-ui

You're absolutely correct, and I agree that there's potentially a risk of quality loss. But likewise, since these are all intrinsically linked, it may be possible to leverage strength by combining these tasks. I'm unaware of a paper reviewing the reliability and/or performance of LLMs in this specific scenario. If you find any, do share :) With regards to generating JSON responses - there are simple ways to nudge the model and even validate it, using libraries such as https://github.com/promptslab/Promptify, https://github.com/eyurtsev/kor and https://github.com/ShreyaR/guardrails

There are a couple different approaches:

- Use multi-shot prompting with something like guardrails to try prompting a commercial model until it works. [1]

- Use a local model with something with a final layer that steers token selection towards syntactically valid tokens [2]

[1] https://github.com/ShreyaR/guardrails

[2] "Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema" @ https://github.com/newhouseb/clownfish.

👍 validate and correct the outputs of LLMs using guardrails

not vouching for it, but I know this is at least a thing that exists and I like the general idea: https://github.com/shreyar/guardrails