RAG Logger: An Open-Source Alternative to LangSmith

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers
Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.
www.nutrient.io
featured
CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai
featured
  1. RAG-logger

    RAG Logger is an open-source logging tool designed specifically for Retrieval-Augmented Generation (RAG) applications. It serves as a lightweight, open-source alternative to LangSmith, focusing on RAG-specific logging needs.

  2. Nutrient

    Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.

    Nutrient logo
  3. llm_recorder

    llm_recorder is a Python library that helps you record and replay interactions with language models.

    I've just published to Github my own very simplistic LLM logging and debugging tool: https://github.com/zby/llm_recorder

  4. opik

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    Really awesome seeing more people work on this! I’m one of the founders of Opik https://github.com/comet-ml/opik which does similar things but also has a UI and supports massive scale. Curious to hear if you have any feedback!

  5. langfuse

    🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

    There is also LangFuse which has an integration with Flowise

    https://github.com/langfuse/langfuse

    https://github.com/FlowiseAI/Flowise

  6. Flowise

    Drag & drop UI to build your customized LLM flow

    There is also LangFuse which has an integration with Flowise

    https://github.com/langfuse/langfuse

    https://github.com/FlowiseAI/Flowise

  7. langtrace

    Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊

    Congrats on the launch. Cool to see a RAG specific tracing tool. Excited to try it out. Full disclosure, I am the cofounder and core-maintainer of Langtrace(https://github.com/Scale3-Labs/langtrace) which is also an open source tool for tracing and observing your LLM stack and our SDKs are OTEL based. Based on my experience, I think the biggest challenge right now specifically for RAG pipelines is the lack of flexibility in the current crop of tracing tools to not just visualize the entire retrieval flow across all the components of the stack - the framework calls, vectorDB retrievals, re-ranker i/o if any and the final LLM inference. But, also being able to do experiments by freezing a setup, iterate on it and measuring the performance and improving it to clearly know how the changes map to the performance end to end. This is what we think about mostly while we are building Langtrace as well.

  8. langwatch

    The ultimate LLM Ops platform - Monitoring, Analytics, Evaluations, Datasets and Prompt Optimization ✨

    Awesome to see more opensource tools in this space. In transparency we'r building the oss tool https://github.com/langwatch/langwatch which is tool for tracing and monitoring your LLM features and open telemetry is supported as well. Monitoring is key to any team building LLM-features, and still much can be done in this field. What i believe in is the power of optimizing when understanding your performance with these solutions. For ex we're using DSPy optimizers. Curious towards your thoughts int this too! Congrats on the launch and all the best!

  9. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Pydantic Logfire

    7 projects | news.ycombinator.com | 30 Apr 2024
  • langfuse VS Kiln - a user suggested alternative

    2 projects | 19 Dec 2024
  • Universal Personal Assistant with LLMs

    13 projects | dev.to | 11 Dec 2024
  • OpenPipe

    1 project | news.ycombinator.com | 27 Sep 2024
  • Promptfoo: Test your LLM app locally

    1 project | news.ycombinator.com | 13 Sep 2024

Did you know that TypeScript is
the 1st most popular programming language
based on number of references?