promptfoo
shap-e
| promptfoo | shap-e | |
|---|---|---|
| 5 | 46 | |
| 328 | 11,355 | |
| - | 1.5% | |
| 10.0 | 4.4 | |
| 11 months ago | 3 months ago | |
| TypeScript | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
promptfoo
-
Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
Jumping in because I'm a big believer in (1) local LLMs, and (2) evals specific to individual use cases.
[0] https://github.com/typpo/promptfoo
- Meta Llama 3
-
Launch HN: Talc AI (YC S23) – Test Sets for AI
Congrats on the launch!
I've been interested in automatic testset generation because I find that the chore of writing tests is one of the reasons people shy away from evals. Recently landed eval testset generation for promptfoo (https://github.com/typpo/promptfoo), but it is non-RAG so more simplistic than your implementation.
Was also eyeballing this paper https://arxiv.org/abs/2401.03038, which outlines a method for generating asserts from prompt version history that may also be useful for these eval tools.
-
GPT-Prompt-Engineer
Thanks for the promptfoo mention. For anyone else who might prefer deterministic, programmatic evaluation of LLM outputs, I've been building promptfoo: https://github.com/typpo/promptfoo
Example asserts include basic string checks, regex, is-json, cosine similarity, etc.
shap-e
-
Text to 3D(shap-e) in our VR worldbuilding app made with godot4. Each generation takes about 20 sec, the output is decent but I see huge potential for this to change asset creation as the results improve. More model to come in the upcoming days.
No. Its all generated with an AI model trained on 3D data from OpenAI (https://github.com/openai/shap-e) . I have an instance running on my machine and I query it from godot.
-
Getting Started With Shap-E
Finally, here is the Official Shap-E Github Repository. As Shap-E is a completely open source project, you can find everything that goes into the project here.
-
What AI-powered app/website is bugging you that it doesn't exist?
This could be what you’re after https://github.com/openai/shap-e
-
SD Webui new extension - Txt/Img to 3D Model
see here https://github.com/openai/shap-e, it also has the original paper
-
Why the Sam Altman hate?
OpenAI have been releasing open source models recently, including Whisper (which was SOTA for Speech-to-text) and Shap-E. It's just the large scale LLMs they're not releasing. Also people are choosing to ignore that in Sam's statement before congress he argued only for regulation of large models on the scale of GPT-4, explicitly arguing against regulation of open source efforts.
-
Are AI developers not paying enough attention to this? [D]
It's not that no one paying attention to it, it's just the area hasn't had the same results as others. Personally I've tried to build a floor plan generator several years ago and it did not turn out great. I'm not sure of anything architecture specific out there but one recent success for 3d generation was Shape-E
- Getting the slicer to design half of your part feels like cheating!
- shap-e: generate 3D images
-
AI Weekly rundown (May 7- May 13, 2023): OpenAI's Shap·E, Multimodal GPT, Anthropic's constitutional AI and a lot more.
(Source)
What are some alternatives?
rebuff - LLM Prompt Injection Detector
Auto-GPT-MetaTrader-Plugin - The AutoGPT MetaTrader Plugin is a software tool that enables traders to connect their MetaTrader 4 or 5 trading account to Auto-GPT.
gpt-engineer - Specify what you want it to build, the AI asks for clarification, and then builds it.
promptfoo - Test your prompts, models, and RAGs. Catch regressions and improve prompt quality. LLM evals for OpenAI, Azure, Anthropic, Gemini, Mistral, Llama, Bedrock, Ollama, and other local & private models with CI/CD integration.
ChainForge - An open-source visual programming environment for battle-testing prompts to LLMs.
sd-webui-txt-img-to-3d-model - A custom extension for sd-webui that allow you to generate 3D model from txt or image, basing on OpenAI Shap-E.
gateway - A Blazing Fast AI Gateway. Route to 100+ LLMs with 1 fast & friendly API.
nsfw-prompt-detection-sd - NSFW Prompt Detection for Stable Diffusion
sugarcane-ai - npm like package ecosystem for Prompts 🤖
WizardVicunaLM - LLM that combines the principles of wizardLM and vicunaLM
plandex - AI driven development in your terminal. Designed for large, real-world tasks.
pandas-ai - Chat with your database (SQL, CSV, pandas, polars, mongodb, noSQL, etc). PandasAI makes data analysis conversational using LLMs (GPT 3.5 / 4, Anthropic, VertexAI) and RAG.