llm-api VS jehuty

You don't need to look at the code: I looked at the release notes and the garbage fire of unrelated nonsense getting added and got to skip even installing it.

Langchain is almost required at this point to accept anything. They raised money, and now their growth metric is Github stars.

A simple wrapper around the APIs (I used llamaflow, which is now llm-api https://github.com/dzhng/llm-api) and a templating engine is most of what you need.

AI is not at a point where generalist prompts to do agent/memory/search things is a good idea for a real product. You need to integrate procedural guidance unless you want your UX to be awful.

I used Langchain before for a job interview and was not confident with how it works under the hood and how dangerous would it be if there’s some injection going on. So I used it as minimal as possible. It took me a lot of codes even though when I’m using it minimally. One of their example is to call an API by letting LLM parse a documentation and call the API from its understanding, which looks so unreliable if the LLM went offs a bit. I found it hard to give total control to Langchain.

I tried experimenting on building a library that makes it easy and transparent to use LLM https://github.com/adityapurwa/jehuty and tried the middleware approach that might be more familiar in general. Its an experiment so the API might changes a lot until we find a sweet spot. If you have an advice or suggestions it would be helpful and appreciated.