pipeless VS trulens

After spending some months working on the Pipeless open-source framework, today I bring something new and really cool: Pipeless Agents

pipeless init my-project --template empty # Using the empty template we avoid the interactive shell cd my-project wget -O - https://github.com/pipeless-ai/pipeless/archive/main.tar.gz | tar -xz --strip=2 "pipeless-main/examples/yolo"

wget -O - https://github.com/pipeless-ai/pipeless/archive/main.tar.gz | tar -xz --strip=2 "pipeless-main/examples/onnx-yolo"

Check the live demo in the website

The code is available in the Pipeless GitHub repository: https://github.com/pipeless-ai/pipeless/tree/main/examples/yolo

Recently, a new alternative called Pipeless has appeared. Pipeless is an open-source framework that focuses on providing a great development experience and out-of-the-box performance. It offers a really easy stream management allowing you to add, edit, and remove streams on the fly as well as processing multiple streams. Furthermore, it offers you the possibility to deploy the applications either to the cloud or directly to embedded or edge devices.

You can find the repo here: https://github.com/pipeless-ai/pipeless

Pipeless is an open-source computer vision framework that ships everything you need to build and deploy computer vision applications really fast.

Retrieval using a single vector is called dense passage retrieval (DPR), because an entire passage (dozens to hundreds of tokens) is encoded as a single vector. ColBERT instead encodes a vector-per-token, where each vector is influenced by surrounding context. This leads to meaningfully better results; for example, here’s ColBERT running on Astra DB compared to DPR using openai-v3-small vectors, compared with TruLens for the Braintrust Coda Help Desk data set. ColBERT easily beats DPR at correctness, context relevance, and groundedness.

12. TruLens by TruEra | Github | tutorial

3) Hallucination is probably the biggest problem we solve for. To do evals for hallucination, we typically see our users use a combination of groundedness (does the context support the LLM response) and context relevance (is the retrieved context relevant to the query). There's also a bunch more for the evaluations you mentioned (moderation models, sentiment, usefulness, etc.) and it's pretty easy to add custom evals.

Also - my hot take is that gpt-3.5 is good enough for evals (sometimes better) than gpt-4 if you give the LLM enough instructions on how to do the eval.

website: https://www.trulens.org/

The team at TruEra recently released an open source project for evaluation & tracking of LLM applications called TruLens-Eval. We’ve specifically targeted retrieval-augmented QA as a core use case and so far we’ve seen it used for comparing different models and parameters, prompts, vector-db configurations and query planning strategies. I’d love to get your feedback on it.