sketch
pandas-ai
Our great sponsors
sketch | pandas-ai | |
---|---|---|
20 | 14 | |
2,194 | 10,904 | |
1.8% | 9.9% | |
4.4 | 9.8 | |
3 months ago | 8 days ago | |
Python | Python | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sketch
-
Ask HN: What have you built with LLMs?
We've made a lot of data tooling things based on LLMs, and are in the process of rebranding and launching our main product.
1. sketch (in notebook, ai for pandas) https://github.com/approximatelabs/sketch
2. datadm (open source, "chat with data", with support for the open source LLMs (https://github.com/approximatelabs/datadm)
3. Our main product: julyp. https://julyp.com/ (currently under very active rebrand and cleanup) -- but a "chat with data" style app, with a lot of specialized features. I'm also streaming me using it (and sometimes building it) every weekday on twitch to solve misc data problems (https://www.twitch.tv/bluecoconut)
For your next question, about the stack and deploy:
-
Pandas AI โ The Future of Data Analysis
This morning I added a "Related Projects" [3] Section to the Buckaroo docs. If Buckaroo doesn't solve your problem, look at one of the other linked projects (like Mito).
[1] https://github.com/approximatelabs/sketch
[2] https://github.com/paddymul/buckaroo
[3] https://buckaroo-data.readthedocs.io/en/latest/FAQ.html
-
Ask HN: What's your favorite GPT powered tool?
For GPT/Copilot style help for pandas, in notebooks REPL flow (without needing to install plugins), I built sketch. I genuinely use it every-time I'm working on pandas dataframes for a quick one-off analysis. Just makes the iteration loop so much faster. (Specifically the `.sketch.howto`, anecdotally I actually don't use `.sketch.ask` anymore)
https://github.com/approximatelabs/sketch
-
RasaGPT: First headless LLM chatbot built on top of Rasa, Langchain and FastAPI
https://github.com/approximatelabs/lambdaprompt It has served all of my personal use-cases since making it, including powering `sketch` (copilot for pandas) https://github.com/approximatelabs/sketch
Core things it does: Uses jinja templates, does sync and async, and most importantly treats LLM completion endpoints as "function calls", which you can compose and build structures around just with simple python. I also combined it with fastapi so you can just serve up any templates you want directly as rest endpoints. It also offers callback hooks so you can log & trace execution graphs.
All together its only ~600 lines of python.
I haven't had a chance to really push all the different examples out there, but most "complex behaviors", so there aren't many patterns to copy. But if you're comfortable in python, then I think it offers a pretty good interface.
I hope to get back to it sometime in the next week to introduce local-mode (eg. all the open source smaller models are now available, I want to make those first-class)
-
[D] The best way to train an LLM on company data
Please look at sketch and langchain pandas/SQL plugins. I have seen excellent results with both of these approaches. Both of these approaches will require you to send metadata to openAI.
-
Meet Sketch: An AI code Writing Assistant For Pandas
๐ Understand your data through questions ๐ Create code from plain text Quick Read: https://www.marktechpost.com/2023/02/01/meet-sketch-an-ai-code-writing-assistant-for-pandas/ Github: https://github.com/approximatelabs/sketch
-
Replacing a SQL analyst with 26 recursive GPT prompts
(3) Asking for re-writes of failed queries (happens occasionally) also helps
The main challenge I think with a lot of these "look it works" tools for data applications, is how do you get an interface that actually will be easy to adopt. The chat-bot style shown here (discord and slack integration) I can see being really valuable, as I believe there has been some traction with these style integrations with data catalog systems recently. People like to ask data questions to other people in slack, adding a bot that tries to answer might short-circuit a lot of this!
We built a prototype where we applied similar techniques to the pandas-code-writing part of the stack, trying to help keep data scientists / data analysts "in flow", integrating the code answers in notebooks (similar to how co-pilot puts suggestions in-line) -- and released https://github.com/approximatelabs/sketch a little while ago.
-
FLiP Stack Weekly for 21 Jan 2023
Python AI Helper https://github.com/approximatelabs/sketch
- LangChain: Build AI apps with LLMs through composability
- Show HN: Sketch โ AI code-writing assistant that understands data content
pandas-ai
- PandasAI is great but is there a more general library?
- AI enhanced Pandas Python Library
-
[P] Project Asking for Feedbacks, Critiques, and Everything in Between!
I started working on a project recently to familiar myself with LLM's. The project is called matplotlib_ai, and it's inspired by pandas_ai. matplotlib_ai is a package that is able to take in a natural language prompt by the user and generate graphs based on it. As of right now, it uses OpenAI's GPT-3.5 service only (so yes, it requires an API key from OpenAI), but I plan on incorporating other free LLM's too. The GitHub repo is here: https://github.com/notY0rick/matplotlib_ai.
- Trying to run prompt through pandasai and receiving index error
- Pandas AI?
-
How do I get Local LLM to analyze an whole excel or CSV?
you could use this https://github.com/gventuri/pandas-ai
-
Pandas AI โ The Future of Data Analysis
The medium article is ok, if blocked at times. This is just a summary, not by the package author.
You can jump to the code at https://github.com/gventuri/pandas-ai to see more of what it's trying to do.
-
AI Weekly rundown (May 7- May 13, 2023): OpenAI's ShapยทE, Multimodal GPT, Anthropic's constitutional AI and a lot more.
Pandas gets an AI-powered upgrade. (Link)
- Pandas AI: make data conversational
- PandasAI: Query DataFrames Using LLMs
What are some alternatives?
RasaGPT - ๐ฌ RasaGPT is the first headless LLM chatbot platform built on top of Rasa and Langchain. Built w/ Rasa, FastAPI, Langchain, LlamaIndex, SQLModel, pgvector, ngrok, telegram
lmql - A language for constraint-guided and efficient LLM programming.
keras-ocr - A packaged and flexible version of the CRAFT text detector and Keras CRNN recognition model.
gpt_index - LlamaIndex (GPT Index) is a project that provides a central interface to connect your LLM's with external data. [Moved to: https://github.com/jerryjliu/llama_index]
pandasql - sqldf for pandas
langchain - โก Building applications with LLMs through composability โก [Moved to: https://github.com/langchain-ai/langchain]
mito - The mitosheet package, trymito.io, and other public Mito code.
rasa - ๐ฌ Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
PromptQueries - An open-source C# library for generating LINQ queries using natural language prompts. Uses OpenAI API for language processing. Supports IQueryable data sources. Prevents SQL injection attacks. Simple and easy-to-use API.
viper - Simple, expressive pipeline syntax to transform and manipulate data with ease
oxia - Oxia - Metadata store and coordination system