gpt_index VS sketch

LLAMA Index Github repository

Zuwendungen kommen auch so langsam ( LLamaIndex ) https://github.com/jerryjliu/gpt_index

This is exactly what LlamaIndex is meant to solve!

A set of data structures to augment LLM's with your data: https://github.com/jerryjliu/gpt_index

This is what we've designed LlamaIndex for! https://github.com/jerryjliu/gpt_index. Designed to help you "index" over a large doc corpus in different ways for use with LLM prompts.

https://github.com/jerryjliu/gpt_index might be close to what you need.

I would start with https://gpt-index.readthedocs.io/en/latest/ and https://langchain.readthedocs.io/en/latest/

People have been doing this with https://github.com/jerryjliu/gpt_index

Here is a link to the repository. Take a look at the overview section of the readme. https://github.com/jerryjliu/gpt_index

(creator of gpt index / llamaindex here https://github.com/jerryjliu/gpt_index)

Funny that we had just rebranded our tool from GPT Index to LlamaIndex about a week ago to avoid potential trademark issues with OpenAI, and turns out Meta has similar ideas around LLM+llama puns :). Must mean the name is good though!

Also very excited to try plugging in the LLaMa model into LlamaIndex, will report the results.

We've made a lot of data tooling things based on LLMs, and are in the process of rebranding and launching our main product.

1. sketch (in notebook, ai for pandas) https://github.com/approximatelabs/sketch

2. datadm (open source, "chat with data", with support for the open source LLMs (https://github.com/approximatelabs/datadm)

3. Our main product: julyp. https://julyp.com/ (currently under very active rebrand and cleanup) -- but a "chat with data" style app, with a lot of specialized features. I'm also streaming me using it (and sometimes building it) every weekday on twitch to solve misc data problems (https://www.twitch.tv/bluecoconut)

For your next question, about the stack and deploy:

This morning I added a "Related Projects" [3] Section to the Buckaroo docs. If Buckaroo doesn't solve your problem, look at one of the other linked projects (like Mito).

[1] https://github.com/approximatelabs/sketch

[2] https://github.com/paddymul/buckaroo

[3] https://buckaroo-data.readthedocs.io/en/latest/FAQ.html

For GPT/Copilot style help for pandas, in notebooks REPL flow (without needing to install plugins), I built sketch. I genuinely use it every-time I'm working on pandas dataframes for a quick one-off analysis. Just makes the iteration loop so much faster. (Specifically the `.sketch.howto`, anecdotally I actually don't use `.sketch.ask` anymore)

https://github.com/approximatelabs/sketch

https://github.com/approximatelabs/lambdaprompt It has served all of my personal use-cases since making it, including powering `sketch` (copilot for pandas) https://github.com/approximatelabs/sketch

Core things it does: Uses jinja templates, does sync and async, and most importantly treats LLM completion endpoints as "function calls", which you can compose and build structures around just with simple python. I also combined it with fastapi so you can just serve up any templates you want directly as rest endpoints. It also offers callback hooks so you can log & trace execution graphs.

All together its only ~600 lines of python.

I haven't had a chance to really push all the different examples out there, but most "complex behaviors", so there aren't many patterns to copy. But if you're comfortable in python, then I think it offers a pretty good interface.

I hope to get back to it sometime in the next week to introduce local-mode (eg. all the open source smaller models are now available, I want to make those first-class)

Please look at sketch and langchain pandas/SQL plugins. I have seen excellent results with both of these approaches. Both of these approaches will require you to send metadata to openAI.

👉 Understand your data through questions 👉 Create code from plain text Quick Read: https://www.marktechpost.com/2023/02/01/meet-sketch-an-ai-code-writing-assistant-for-pandas/ Github: https://github.com/approximatelabs/sketch

(3) Asking for re-writes of failed queries (happens occasionally) also helps

The main challenge I think with a lot of these "look it works" tools for data applications, is how do you get an interface that actually will be easy to adopt. The chat-bot style shown here (discord and slack integration) I can see being really valuable, as I believe there has been some traction with these style integrations with data catalog systems recently. People like to ask data questions to other people in slack, adding a bot that tries to answer might short-circuit a lot of this!

We built a prototype where we applied similar techniques to the pandas-code-writing part of the stack, trying to help keep data scientists / data analysts "in flow", integrating the code answers in notebooks (similar to how co-pilot puts suggestions in-line) -- and released https://github.com/approximatelabs/sketch a little while ago.

Python AI Helper https://github.com/approximatelabs/sketch