The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 23 Python prompt-engineering Projects
-
promptflow
Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
tree-of-thoughts
Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%
-
Awesome-Prompt-Engineering
This repository contains a hand-curated resources for Prompt Engineering with a focus on Generative Pre-trained Transformer (GPT), ChatGPT, PaLM etc
-
-
prompttools
Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
-
-
graph-of-thoughts
Official Implementation of "Graph of Thoughts: Solving Elaborate Problems with Large Language Models"
-
PromptCraft-Robotics
Community for applying LLMs to robotics and a robot simulator with ChatGPT integration
-
-
multimodal-maestro
Effective prompting for Large Multimodal Models like GPT-4 Vision, LLaVA or CogVLM. 🔥
-
-
stable-diffusion-prompt-reader
A simple standalone viewer for reading prompts from Stable Diffusion generated image outside the webui.
-
-
awesome-gpt-prompt-engineering
A curated list of awesome resources, tools, and other shiny things for GPT prompt engineering.
-
swarms
Orchestrate Swarms of Agents From Any Framework Like OpenAI, Langchain, and Etc for Real World Workflow Automation. Join our Community: https://discord.gg/DbjBMJTSWD
-
-
megabots
🤖 State-of-the-art, production ready LLM apps made mega-easy, so you don't have to build them from scratch 🤯 Create a bot, now 🫵
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: A suite of tools designed to streamline the development cycle of LLM-based apps | news.ycombinator.com | 2024-04-12
Project mention: [D] Potential scammer on github stealing work of other ML researchers? | /r/MachineLearning | 2023-08-17I checked the issues and found https://github.com/kyegomez/tree-of-thoughts/issues/78
Yes, there are a lot of different resources online, especially for generative AI. The Awesome Prompt Engineering github is probably a good place to start https://github.com/promptslab/Awesome-Prompt-Engineering. If you're focusing directly on OpenAI's models then the OpenAI Prompt Engineering Guide would be my recommendation https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api.
Project mention: YiVal——Unlocking Your Data's Power to Create Customized GenAI Apps | /r/u_YiVal | 2023-11-16- 🤖Github:https://github.com/YiVal/YiVal/pull/189
Project mention: Did GPT-4 really get worse? We built an evaluation framework so you can find out | /r/ChatGPT | 2023-07-24Here's an example where we compare a few versions of GPT-4 against a locally run Llama 2 model: https://github.com/hegelai/prompttools/blob/main/examples/notebooks/GPT4vsLlama2.ipynb
Currently seeking feedback for the developed tool. Would love it if you can check it out on: https://github.com/uptrain-ai/uptrain/blob/main/examples/assistants/assistant_evaluator.ipynb
Project mention: Show HN: Times faster LLM evaluation with Bayesian optimization | news.ycombinator.com | 2024-02-13Fair question.
Evaluate refers to the phase after training to check if the training is good.
Usually the flow goes training -> evaluation -> deployment (what you called inference). This project is aimed for evaluation. Evaluation can be slow (might even be slower than training if you're finetuning on a small domain specific subset)!
So there are [quite](https://github.com/microsoft/promptbench) [a](https://github.com/confident-ai/deepeval) [few](https://github.com/openai/evals) [frameworks](https://github.com/EleutherAI/lm-evaluation-harness) working on evaluation, however, all of them are quite slow, because LLM are slow if you don't have infinite money. [This](https://github.com/open-compass/opencompass) one tries to speed up by parallelizing on multiple computers, but none of them takes advantage of the fact that many evaluation queries might be similar and all try to evaluate on all given queries. And that's where this project might come in handy.
Project mention: Q* Could Be It - Forget AlphaGO - It's Diplomacy - Peg 1 May Have Fallen - Noam Brown May Have Achieved The Improbable - Is this Q* Leak 2.0? | /r/singularity | 2023-12-08We introduce Graph of Thoughts (GoT): a framework that advances prompting capabilities in large language models (LLMs) beyond those offered by paradigms such as Chain-ofThought or Tree of Thoughts (ToT). The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information (“LLM thoughts”) are vertices, and edges correspond to dependencies between these vertices. This approach enables combining arbitrary LLM thoughts into synergistic outcomes, distilling the essence of whole networks of thoughts, or enhancing thoughts using feedback loops. We illustrate that GoT offers advantages over state of the art on different tasks, for example increasing the quality of sorting by 62% over ToT, while simultaneously reducing costs by >31%. We ensure that GoT is extensible with new thought transformations and thus can be used to spearhead new prompting schemes. This work brings the LLM reasoning closer to human thinking or brain mechanisms such as recurrence, both of which form complex networks. Website & code: https://github.com/spcl/graph-of-thoughts
Project mention: DiffusionDB: A large-scale text-to-image prompt gallery dataset based on Stable Diffusion | /r/datasets | 2023-06-27
Project mention: Show HN: Multimodal Maestro – Prompt tools for use with LMMs | news.ycombinator.com | 2023-11-29
This is a subproject of the SD Prompt Reader. It helps you extract metadata from images in any format supported by the SD Prompt Reader and saves the images with additional metadata to ensure compatibility with metadata detection on websites such as Civitai.
1) https://github.com/snwfdhmp/awesome-gpt-prompt-engineering 2) https://www.europe.study/artificial-intelligence?twclid=273rg3p5g5umnt3g3ifxzysbin 3) https://coursera.org/projects/chat-gpt-for-beginners-using-ai-for-market-research
Project mention: Swarms – Automating all digital activities with millions of autonomous AI Agents | news.ycombinator.com | 2023-07-10
Project mention: Show HN: LLMFlows – LangChain alternative for explicit and transparent apps | news.ycombinator.com | 2023-07-29
Project mention: Promptmap – automatically tests prompt injection attacks on ChatGPT instances | news.ycombinator.com | 2023-07-17
Project mention: 🦋 ChainFury: open-source tool to create an LLM chatbot in 4 clicks! | /r/github | 2023-04-27Check out our repo at https://github.com/NimbleBoxAI/ChainFury and give us a star ⭐️ to show your support! Thanks!
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python prompt-engineering related posts
- Meta Llama 3
- Show HN: I made a library for LLM prompt injection/exploit/jailbreak detection
- Minimal implementation of Mamba, the new LLM architecture, in 1 file of PyTorch
- OpenAI Switch Kit: Swap OpenAI with any open-source model
- SDXL Turbo: A Real-Time Text-to-Image Generation Model
- TypeGPT - an open source Python library that makes GPT outputs consistent.
- Graph of Thoughts: Solving Elaborate Problems with Large Language Models
-
A note from our sponsor - WorkOS
workos.com | 27 Apr 2024
Index
What are some of the best open-source prompt-engineering projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | awesome-chatgpt-zh | 9,867 |
| 2 | promptflow | 8,074 |
| 3 | aim | 4,782 |
| 4 | tree-of-thoughts | 4,029 |
| 5 | Awesome-Prompt-Engineering | 3,196 |
| 6 | YiVal | 2,425 |
| 7 | prompttools | 2,420 |
| 8 | uptrain | 1,976 |
| 9 | promptbench | 1,969 |
| 10 | graph-of-thoughts | 1,845 |
| 11 | PromptCraft-Robotics | 1,709 |
| 12 | AutoPrompt | 1,637 |
| 13 | diffusiondb | 1,105 |
| 14 | multimodal-maestro | 942 |
| 15 | spacy-llm | 925 |
| 16 | stable-diffusion-prompt-reader | 888 |
| 17 | llm-guard | 821 |
| 18 | awesome-gpt-prompt-engineering | 784 |
| 19 | swarms | 650 |
| 20 | llmflows | 615 |
| 21 | promptmap | 516 |
| 22 | ChainFury | 396 |
| 23 | megabots | 335 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com