sendenv VS langroid

A utility to share environment variables with others P2P - https://github.com/viraniaman94/sendenv

Had posted it here as well: https://news.ycombinator.com/item?id=39098133

Got a lot more GitHub stars than I had anticipated. Still getting stars here and there.

In the Langroid[1] LLM library we have a clean, extensible RAG implementation in the DocChatAgent[2] -- it uses several retrieval techniques, including lexical (bm25, fuzzy search) and semantic (embeddings), and re-ranking (using cross-encoder, reciprocal-rank-fusion) and also re-ranking for diversity and lost-in-the-middle mitigation:

[1] Langroid - a multi-agent LLM framework from CMU/UW-Madison researchers https://github.com/langroid/langroid

[2] DocChatAgent Implementation -

Langroid: https://github.com/langroid/langroid

Langroid (2.7k stars, 20k downloads/mo) is an intuitive, lightweight, extensible and principled Python framework to easily build agent-oriented LLM-powered applications, from CMU and UW-Madison researchers. You set up Agents, equip them with optional components (LLM,

You can have a look at Langroid, an agent-oriented LLM framework from CMU/UW-Madison researchers (I am the lead dev). We are seeing companies using it in production in preference to other libs mentioned here.

https://github.com/langroid/langroid

https://github.com/langroid/langroid/tree/main/examples/basi...

[1] Langroid: https://github.com/langroid/langroid

This is very interesting. To build LLM chat-oriented WebApps in python, these days I use Chainlit[1], which I find is much better than Streamlit for this. I've integrated Chainlit into the Langroid[2] Multi-Agent LLM framework via a callback injection class[3] (i.e. hooks to display responses by various entities).

One of the key requirements in a multi-agent chat app is to be able to display steps of sub-tasks nested under parent tasks (to any level of nesting), with the option to fold/collapse sub-steps to only view the parent steps. I was able to get this to work with chainlit, though it was not easy, since their sub-step rendering mental model seemed more aligned to a certain other LLM framework with a partial name overlap with theirs.

That said, I am very curious if Mesop could be a viable alternative, for this type of nested chat implementation, especially if the overall layout can be much more flexible (which it seems like), and more production-ready.

[1] Chainlit https://github.com/Chainlit/chainlit

[2] Langroid: https://github.com/langroid/langroid

[3] Langroid ChainlitAgentCallback class: https://github.com/langroid/langroid/blob/main/langroid/agen...

This was indeed true in the beginning, and I don’t know if this has changed. Inserting messages with Assistant role is crucial for many reasons, such as if you want to implement caching, or otherwise edit/compress a previous assistant response for cost or other reason.

At the time I implemented a work-around in Langroid[1]: since you can only insert a “user” role message, prepend the content with ASSISTANT: whenever you want it to be treated as an assistant role. This actually works as expected and I was able to do caching. I explained it in this forum:

https://community.openai.com/t/add-custom-roles-to-messages-...

[1] the Langroid code that adds a message with a given role, using this above “assistant spoofing trick”:

https://github.com/langroid/langroid/blob/main/langroid/agen...

Same question here. Ollama is fantastic as it makes it very easy to run models locally, But if you already have a lot of code that processes OpenAI API responses (with retry, streaming, async, caching etc), it would be nice to be able to simply switch the API client to Ollama, without having to have a whole other branch of code that handles Alama API responses. One way to do an easy switch is using the litellm library as a go-between but it’s not ideal (and I also recently found issues with their chat formatting for mistral models).

For an OpenAI compatible API my current favorite method is to spin up models using oobabooga TGW. Your OpenAI API code then works seamlessly by simply switching out the api_base to the ooba endpoint. Regarding chat formatting, even ooba’s Mistral formatting has issues[1] so I am doing my own in Langroid using HuggingFace tokenizer.apply_chat_template [2]

[1] https://github.com/oobabooga/text-generation-webui/issues/53...

[2] https://github.com/langroid/langroid/blob/main/langroid/lang...

Related question - I assume ollama auto detects and applies the right chat formatting template for a model?

we (like simpleaichat from OP) leverage Pydantic to specify the desired structured output, and under the hood Langroid translates it to either the OpenAI function-calling params or (for LLMs that don’t natively support fn-calling), auto-insert appropriate instructions into tje system-prompt. We call this mechanism a ToolMessage:

https://github.com/langroid/langroid/blob/main/langroid/agen...

We take this idea much further — you can define a method in a ChatAgent to “handle” the tool and attach the tool to the agent. For stateless tools you can define a “handle” method in the tool itself and it gets patched into the ChatAgent as the handler for the tool.

Many services/platforms are careless/disingenuous when they claim they “train” on your documents, where they actually mean they do RAG.

An under-appreciate benefit of RAG is the ability to have the LLM cite sources for its answers (which are in principle automatically/manually verifiable). You lose this citation ability when you finetune on your documents.

In Langroid (the Multi-Agent framework from ex-CMU/UW-Madison researchers) https://github.com/langroid/langroid