llemmings VS llm-code

I did this experiment (a game) to see what's up and what's down around all this: https://github.com/romland/llemmings.

While there is some GPT4 in there, it's mostly ChatGPT and a small handful of LLaMA solutions.

That project is a contrived scenario and not realistic, but I wanted to experiment with _exactly_ what you are talking about.

Very often I could have done things a lot faster myself, but there is one aspect that was actually helpful, and I did not foresee it. When inspiration gets a bit low and you're not in the "zone"; throwing something into an LLM will very often give me a push to keep at it. Even if what is coming up is mostly grunt work.

The other day I threw together a script to show the commits in a reverse order and filter out (most of) the human commits (glue) over at https://llemmings.com/

I’m using it to do the mundane tasks of unit testing and (some) documentation. I find that the code it spits out isn’t perfect but getting some boiler plate and fixing it up is pretty fast compared to writing from scratch.

I’ve used this enough that I wrapped some cli glue around it and wrote https://github.com/radoshi/llm-code

I’ve used this mostly to write Python and bash, with some Makefiles and Dockerfiles thrown in.

GPT-4 is better, albeit slower, than 3.5-turbo. HTH!