Patterns for Building LLM-Based Systems and Products

• blog

  3 5 9.5 HTML

  The Vespa blog - https://blog.vespa.ai (by vespa-engine)

  

• ragas

  10 4,668 9.6 Python

  Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines

  

We have build RAGAS framework for this https://github.com/explodinggradients/ragas

• SurveyJS

  surveyjs.io featured

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

• OpenPipe

  13 2,374 9.9 TypeScript

  Turn expensive prompts into cheap fine-tuned models

  

This is fantastic! I found myself nodding along in many places. I've definitely found in practice that evals are critical to shipping LLM-based apps with confidence. I'm actually working on an open-source tool in this space: https://github.com/openpipe/openpipe. Would love any feedback on ways to make it more useful. :)

• llama_index

  75 31,184 10.0 Python

  LlamaIndex is a data framework for your LLM applications

  

https://github.com/jerryjliu/llama_index. You can use something like that or another GitHub repo built with it to create a customized chatbot application in a few minutes or a few days. (It should not take two weeks and $15,000).

It would be good to see something detailed that demonstrates an actual use case for fine tuning. Also, I don't believe that the academic tests are appropriate in that case. If you really were dead set on avoiding a leading edge closed LLM, and doing actual fine-tuning, you would want a person to look at the outputs and judge them in their specific context such as handling customer support requests for that system.

• agenta

  9 836 10.0 Python

  The all-in-one LLM developer platform: prompt management, evaluation, human feedback, and deployment all in one place.

  

Great project! We're building an open-source platform for building robust LLM apps (https://github.com/Agenta-AI/agenta), we'd love to integrate your library into our evaluation!

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project