promptr VS gish

I taught https://github.com/KillianLucas/open-interpreter how to use https://github.com/ferrislucas/promptr

Then I asked it to add a test suite to a rails side project. It created missing factories, corrected a broken test database configuration, and wrote tests for the classes and controllers that I asked it to.

I didn't have to get involved with mundane details. I did have to intervene here and there, but not much. The tests aren't the best in the world, but IMO they're adding value by at least covering the happy path. They're not as good as an experienced person would write.

I did spend a non-trivial amount of time fiddling with the prompts I used to teach OI about Promptr as well as the prompts I used to get it to successfully create the test suite.

The total cost was around $11 using GPT4 turbo.

I think in this case it was a fun experiment. I think in the future, this type of tooling will be ubiquitous.

I made a CLI tool called Promptr that allows you to make changes to a codebase via plain English instructions:

https://github.com/ferrislucas/promptr

There’s a templating system (liquidjs) included which is useful if you have a library of prompts that you want to reference often.

You can think of it as a junior engineer that needs explicit instructions.

Here are a few example PR’s implemented by Promptr - see the commits for the prompt that was used to produce the code:

https://github.com/ferrislucas/promptr/pull/38

https://github.com/ferrislucas/promptr/pull/41

https://github.com/ferrislucas/promptr

You just prompt it directly or with a file, and it applies the changes to your file system. There's also a templating system that allows you to reference other files from your prompt file if you want to have a shared prompt file that contains project conventions etc.

This is exactly the use that Promptr is intended for https://github.com/ferrislucas/promptr

* full disclosure: I’m the author of Promptr

I’ve been experimenting with pairing a tool I wrote called Promptr [1] with another tool called Open Interpreter [2].

I start with a prompt that teaches Open Interpreter how to use Promptr, and then I discuss what I’m trying to accomplish. It’s certainly not perfect, but there’s definitely something good that happens when you can iterate using dialog with a robot that can modify your file system and execute commands locally.

[1] Promptr: https://github.com/ferrislucas/promptr

[2] Open Interpreter: https://github.com/KillianLucas/open-interpreter

I wrote a great tool for this: https://github.com/ferrislucas/promptr

It’s great for making changes to existing code because it automatically includes the relevant files for context.

For those looking for specific examples of useful code being authored by AI, you can check out this tool:

https://github.com/ferrislucas/promptr

The README links to example PR’s comprised of commits written by GPT4. The prompts used to produce the code are noted in the commit messages.

Promptr is a coding assistant tool that allows you to ask GPT to produce or modify code, and the results will be automatically applied to your file system.

https://github.com/ferrislucas/promptr

From the README:

I'm also interested in this problem. It can be theoretically solved by giving GPT long-term memory about a specific codebase through vector embedding generation (using OpenAI's embeddings API). The semantic embeddings can then be stored in a vector (or vector-supported) database such as Pinecone alongside metadata for querying. Some of the key considerations are how to compare vectors for similarity (there are many algorithms) and how to use metadata to better support your use case. The following resources can be helpful to further understand this technique: - https://platform.openai.com/docs/guides/embeddings - https://www.mlq.ai/fine-tuning-gpt-3-question-answer-bot/ - https://www.pinecone.io/learn/javascript-chatbot/ A couple of semi-related projects I've been looking into: - https://github.com/ferrislucas/promptr - https://github.com/pashpashpash/vault-ai

Promptr is a CLI tool for operating on your codebase using GPT. (https://github.com/ferrislucas/promptr)

I've had good luck with both:

https://github.com/drorm/gish/blob/main/tasks/coding.txt

and

https://github.com/drorm/gish/blob/main/tasks/webapp.txt

With the second one, I reliably generated half a dozen apps with one command.

Not to say that it won't fail sometimes.

Well, if you're interested in something more lite-weight, I wrote

https://github.com/drorm/gish

which is a shell command that lets you interact with GPT with flags, pipes, etc. in a much more unixy way.

This TUI has some impressive features, like the bots and plugins, but I feel gish covers most of the use cases, specifically for software development.

GPT is significantly better at modifying code when following this "all code in, all code out" pattern. This pattern has downsides: you can quickly exhaust the context window, it's slow waiting for GPT to re-type your code (most of which it hasn't modified) and of course you're running up token costs. But the ability of GPT to understand and execute high level changes to the code is far superior with this approach.

I have tried quite a large number of alternative workflows. Outside the "all code in/out" pattern, GPT gets confused, makes mistakes, implements the requested change in different ways in different sections of the code, or just plain fails.

If you're asking for self contained modifications to a single function, that's all the code that needs to go in/out. On the other side of the spectrum, I had GPT build an entire small webapp using this pattern by repeatedly feeding it all the html/css/js along with a series of feature requests. Many feature requests required coordinated changes across html/css/js.

https://github.com/paul-gauthier/easy-chat#created-by-chatgp...

Another HN user has also released a command line tool along these lines called gish:

https://github.com/drorm/gish