motorhead VS kor

Metal [1] is another one on my radar. Their API looks super simple.

Disclosures: None

[1] https://getmetal.io

Any alternatives? I found this Rust based project that might be interesting: https://github.com/getmetal/motorhead

you could run motorhead on docker https://github.com/getmetal/motorhead

The code is found using GitHub search FT.SEARCH inside https://github.com/getmetal/motorhead/blob/main/src/models.rs and adapted.

All the examples that Langchain gives are for persisting memory locally which won't work in a serverless (statelesss) environment, and the one solution documented for stateless applications, getmetal/motorhead, is a containerized, Rust-based service we would have to run ourselves.

I've only just signed up and haven't had a chance to build anything with it yet, but this might be something to consider https://getmetal.io/

Check out kor

Extract structured data from text using LLMs (https://github.com/eyurtsev/kor)

Check out https://github.com/eyurtsev/kor

You're absolutely correct, and I agree that there's potentially a risk of quality loss. But likewise, since these are all intrinsically linked, it may be possible to leverage strength by combining these tasks. I'm unaware of a paper reviewing the reliability and/or performance of LLMs in this specific scenario. If you find any, do share :) With regards to generating JSON responses - there are simple ways to nudge the model and even validate it, using libraries such as https://github.com/promptslab/Promptify, https://github.com/eyurtsev/kor and https://github.com/ShreyaR/guardrails

Currently, I'm experimenting with GPT-3.5-turbo in conjunction with the kor library (langchain for information extraction) to define a prompt template with various examples of what I'm looking for.

yes. there are a few approaches which i intend to take and some helpful resources:

You could implement a Dual LLM Pattern Model https://simonwillison.net/2023/Apr/25/dual-llm-pattern/

You could also leverage a concept like Kor which is a kind of pydantic for LLMs: https://github.com/eyurtsev/kor

in short and as mentioned in the README.md this is absolutely vulnerable to prompt injection. I think this is not a fully solved issue but some interesting community research has been done to help address these things in production