agro VS typedai

I've actually written my own a homebrew framework like this which is a.) cli-coder agnostic and b.) leans heavily on git worktrees [0].

The secret weapon to this approach is asking for 2-4 solutions to your prompt running in parallel. This helps avoid the most time consuming aspect of ai-coding: reviewing a large commit, and ultimately finding the approach to the ai took is hopeless or requires major revision.

By generating multiple solutions, you can cutdown investing fully into the first solution and use clever ways to select from all the 2-4 candidate solutions and usually apply a small tweak at the end. Anyone else doing something like this?

[0]: https://github.com/sutt/agro

Not OP but I know it can be difficult to really difficult to measure or communicate this to people who aren't familiar with the codebase or the problem being solved.

Other than just dumping 10M tokens of chats into a gist and say read through everything I said back and forth with claude for a week.

But, I think I've got the start of a useful summary format: it that takes every prompt and points to the corresponding code commit produced by ai + adds a line diff amount and summary of the task. Check it out below.

https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...

(In this case it's an python cli ai-coding framework that I'm using to build the package itself)

└── swap/ # Temporary files (gitignored)

Every markdown file in guides/ gets copied to any agent (aider, claude, gemini) I kick-off.

I gitignore this .agdocs directory by default. I find this useful because otherwise you get into _please commit or stash your changes_ when trying to switch branches.

But I also run an rsync script before each release to copy the .agdocs to a git tracked mirrored directory [1].

[0]: https://github.com/sutt/agro/blob/master/README.md#layout

[1]: https://github.com/sutt/vidstr/tree/master/.public-agdocs/gu...

Yeah, I've been able to stack ~200 AI-generated commits on top of each other. Over 95% of the code is now AI generated.

You can see each prompt and each commit that was generated here: https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...

Since AI-Gen code is such a roll of the dice, the key is to roll the dice a lot - usually generate at least 3 potential solutions from at least two separate providers at the outset - and get good at quickly reviewing the offered solutions, or iterating on the prompt and regenerating.

Last year, I was actually working on a bounty platform for Github PR's.

The low quality human-authored PR's that came in (due to the incentive we offered) combined with the fact that a draft PR could be made for pennies with AI made this concept dead in the water as far as I'm concerned.

The pain point of getting some attention and action on your opensource codebase is really no longer relevant, in fact the pain point seems to be moving to how to optimize the limited reviewer / maintainer bandwidth under the onslaught of proposed suggestions.

To this end I've been experimenting with a framework that builds PR's from the major agents and but with a focus on how to structure the tasks and review process that optimize the review => accept/revise cycle. If you're interested I've been writing up some case studies here: https://github.com/sutt/agro/blob/master/docs/case-studies/a...

I think the Github-PR model for agent code suggestions is the path of least resistance for getting adoption from today's developers working in an existing codebase. It makes sense: these developers are already used to the idea and the ergonomics of doing code reviews this way.

But pushing this existing process - which was designed for limited participation of scarce people - onto a use-case of managing a potentially huge reservoir of agent suggestions is going to get brittle quickly. Basically more suggestions require a more streamlined and scriptable review workflow.

Which is why I think working in the command line with your agents - similar to Claude and Aider - is going to be where human maintainers can most leverage the deep scalability of async and parallel agents.

> is way better than having to set up git worktrees or any other type of sandbox yourself

I've built up a helper library that does this for you for either aider or claude here: https://github.com/sutt/agro. And for FOSS purposes, I want to prevent MS, OpenAI, etc from controlling the means of production for software where you need to use their infra for sandboxing your dev environment.

And I've been writing about how to use CLI tricks to review the outputs on some case studies as well: https://github.com/sutt/agro/blob/master/docs/case-studies/i...

Currently working on a python package: agro.

It helps devs build parallel agents from the cli. (think claude code, gemini, aider)

And simplifies the git workflow for testing and accepting the best solution.

Check it out: https://github.com/sutt/agro

I keep a little text file pr.txt and try to link my project to HN / X / other aggregators once a day...

This way I spend a few hours each day coding, and a few minutes each day promoting and refining me message.

Great point, I've found Sonnet really can't be beat on many tasks, but increasingly finding Gemini-Pro and o3 handle the tough bugs and refactors best.

That's why I've been using agro to launch agents from each of the main LLM vendors and checking their results when I'm stuck: https://github.com/sutt/agro/blob/master/docs/index.md

I can share my own ai-generated codebase:

- there's a devlog showing all the prompts and accepted outputs: https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...

- and you can look at the ai-generated tests (as is being discussed above) and see they aren't very well thought out for the behavior, but are syntactically impressive: https://github.com/sutt/agro/tree/master/tests

- check out the case-studies in the docs if you're interested in more ideas.

Yes I agree: highly-focused-scope + low-stakes + high-chorelike-task is the sweet spot for agents currently.

I wrote a little about one such task, getting agents to supplement my markdown dev-log here: https://github.com/sutt/agro/blob/master/docs/case-studies/a...

I added a key rotator to my AI coder, and asked a couple of friends to make keys for me. That helped code a good chunk of http://typedai.dev when 2.5 Pro came out

I get the same feeing when I first looked at the LangChain documentation when I wanted to first start tinkering with LLM apps.

I built my own TypeScript AI platform https://typedai.dev with an extensive feature list where I've kept iterating on what I find the most ergonomic way to develop, using standard constructs as much as possible. I've coded enough Java streams, RxJS chains, and JavaScript callbacks and Promise chains to know what kind of code I like to read and debug.

I was having a peek at xstate but after I came across https://docs.dbos.dev/ here recently I'm pretty sure that's that path I'll go down for durable execution to keep building everything with a simple programming model.

Check out https://sophia.dev Its AI tooling I've built on top of Aider for the code editing. I initially built it before Aider added support for running compile and lint commands, as it would often generate changes which wouldn't compile.

I'd added seperate design/implementation agents before that was added to Aider https://aider.chat/2024/09/26/architect.html

The other different is I have a file selection agent and a code review agent, which often has some good fixes/improvements.

I use both, I'll use Aider if its something I feel it will right the first time or I want control over the files in the context, otherwise I'll use the agent in Sophia.

https://github.com/TrafficGuard/nous/blob/main/src/swe/codeE...

This gets the diff and asks questions like:

- Are there any redundant changes in the diff?

It feels a step backwards developing with a such a minimal type system. Seeing so many projects in Python was one reason we're open sourcing our LLM/agentic framework, to give more options for the TypeScript community. We haven't launched it but the code it up at https://github.com/trafficguard/nous

I'm excited to test this out! I've been building an open source TypeScript agentic AI platform for work (DevOps related with an autonomous agent and software engineer workflow). The Claude 3 models had an influence on the design with their tuning on using XML and three levels of capabilities, and have been my preferred models to use.

I remember having moments looking at the plans Opus generated and being impressed with its capabilities.

The slow speed of requests I could deal with, but the costs could quickly add up in workflows and the autonomous agent control loop. I'd often thought if I could just have Opus capabilities at a fraction of the price, so its a nice surprise to have it here sooner that I imagined!

The agent platform isn't officially launched yet, but its up at https://github.com/trafficguard/nous

I never liked the Langchain API when I looked at the examples so I built it from scratch. It has an autonomous agent with a custom XML-based function calling, memory and call history. The software engineer agentic workflow I initially dog-fooded with a prompt like "Complete Jira XYZ-123". So it gets the Jira description, finds the appropriate terraform project in GitLab, clones, edits (delegating to Aider), creates a MR and messages on Slack. It also has a UI for running agents, human-in-the-loop interactions etc.