guardrails VS truss

You're absolutely correct, and I agree that there's potentially a risk of quality loss. But likewise, since these are all intrinsically linked, it may be possible to leverage strength by combining these tasks. I'm unaware of a paper reviewing the reliability and/or performance of LLMs in this specific scenario. If you find any, do share :) With regards to generating JSON responses - there are simple ways to nudge the model and even validate it, using libraries such as https://github.com/promptslab/Promptify, https://github.com/eyurtsev/kor and https://github.com/ShreyaR/guardrails

There are a couple different approaches:

- Use multi-shot prompting with something like guardrails to try prompting a commercial model until it works. [1]

- Use a local model with something with a final layer that steers token selection towards syntactically valid tokens [2]

[1] https://github.com/ShreyaR/guardrails

[2] "Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema" @ https://github.com/newhouseb/clownfish.

👍 validate and correct the outputs of LLMs using guardrails

not vouching for it, but I know this is at least a thing that exists and I like the general idea: https://github.com/shreyar/guardrails

When type checking is needed, I find the Truss library* does the trick quite well.

As for the syntax, there is very little, which can make it a harder lift but once you have the hang of it you won't deal with the issues identified in the parent comment.

* https://github.com/taoensso/truss

I use truss extensively throughout my code to prevent those types of errors.

I haven't tried it myself. I generally just use truss for runtime constraint checking. I use a modified version that integrates scope-capture. And malli validation for more complex cases, but I try to limit that. For me it is better to validate individual attributes as needed, vs validating an entire "type"/collection of attributes. So each function only cares about the attributes that it needs, and validates only as needed.

Use assertions for all data requirements inside functions - I use a modified version of https://github.com/ptaoussanis/truss to ensure that I never get NullReference exceptions, and this also helps make functions more self-documenting. Also use this to assert return data.