NeMo-Guardrails VS kor

- https://github.com/NVIDIA/NeMo-Guardrails/

Here's "jailbreak detection", in the NeMo-Guardrails project from Nvidia:

https://github.com/NVIDIA/NeMo-Guardrails/blob/327da8a42d5f8...

I.e. they ask the llm if the prompt will break the llm. (I believe that more data /some evaluation on how well this performs is intended to be released. Probably fair to call this stuff "not battle tested".)

I have been playing around with some models locally and creating a discord bot as a fun side project, and I wanted to setup some guardrails on inputs / outputs of the bot to make sure that it isn't violating any ethical boundaries. I was going to use Nvidia's Nemo guardrails, but they only support openai currently. Are there any other good ways to control inputs?

Thanks, I hadn't seen those. I did find https://github.com/NVIDIA/NeMo-Guardrails earlier but haven't looked into it yet.

I'm not sure it solves the problem of restricting the information it uses though. For example, as a proof of concept for a customer, I tried providing information from a vector database as context, but GPT would still answer questions that were not provided in that context. It would base its answers on information that was already crawled from the customer website and in the model. That is concerning because the website might get updated but you can't update the model yourself (among other reasons).

you can use guard rails with langchain - https://github.com/NVIDIA/NeMo-Guardrails

Check out kor

Extract structured data from text using LLMs (https://github.com/eyurtsev/kor)

Check out https://github.com/eyurtsev/kor

You're absolutely correct, and I agree that there's potentially a risk of quality loss. But likewise, since these are all intrinsically linked, it may be possible to leverage strength by combining these tasks. I'm unaware of a paper reviewing the reliability and/or performance of LLMs in this specific scenario. If you find any, do share :) With regards to generating JSON responses - there are simple ways to nudge the model and even validate it, using libraries such as https://github.com/promptslab/Promptify, https://github.com/eyurtsev/kor and https://github.com/ShreyaR/guardrails

Currently, I'm experimenting with GPT-3.5-turbo in conjunction with the kor library (langchain for information extraction) to define a prompt template with various examples of what I'm looking for.

yes. there are a few approaches which i intend to take and some helpful resources:

You could implement a Dual LLM Pattern Model https://simonwillison.net/2023/Apr/25/dual-llm-pattern/

You could also leverage a concept like Kor which is a kind of pydantic for LLMs: https://github.com/eyurtsev/kor

in short and as mentioned in the README.md this is absolutely vulnerable to prompt injection. I think this is not a fully solved issue but some interesting community research has been done to help address these things in production