Our great sponsors
-
emerging-trajectories
Open source framework for using LLMs to forecast political, economic, and social events.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
promptr
Promptr is a CLI tool that lets you use plain English to instruct GPT3 or GPT4 to make changes to your codebase.
-
open-interpreter
This is a proof of concept experiment that enables Open Interpreter to use the Promptr CLI tool - see https://github.com/ferrislucas/open-interpreter/pull/1 (by ferrislucas)
I'm working on research agents to help with economic, financial, and political research. These agents are open source (see: https://github.com/wgryc/emerging-trajectories).
The use cases are pretty straight forward and low risk:
1. Run a Google web search.
2. Query a news API.
3. Write a document based on the above, while citing sources.
Here's an example of something written yesterday, where I'm forecasting whether July 2024 will be the hottest on record: https://emergingtrajectories.com/a/forecast/74
This is working well in that the writeups are great and there are some "aha" moments, like the agent finding and referencing the The National Snow and Ice Data Center (NSIDC)... Very cool! I wouldn't have thought of it.
Then there's the part where the agent also tells me that the Oregon Department of Transportation has holidays during the summer, which doesn't matter at all.
So, YMMV, as they say... But I am more productive with these agents. I wouldn't publish anything formally without confirming and reviewing the content, though.
My RSS reader is an A.I. agent, I have written a huge number of comments mentioning it
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
I taught https://github.com/KillianLucas/open-interpreter how to use https://github.com/ferrislucas/promptr
Then I asked it to add a test suite to a rails side project. It created missing factories, corrected a broken test database configuration, and wrote tests for the classes and controllers that I asked it to.
I didn't have to get involved with mundane details. I did have to intervene here and there, but not much. The tests aren't the best in the world, but IMO they're adding value by at least covering the happy path. They're not as good as an experienced person would write.
I did spend a non-trivial amount of time fiddling with the prompts I used to teach OI about Promptr as well as the prompts I used to get it to successfully create the test suite.
The total cost was around $11 using GPT4 turbo.
I think in this case it was a fun experiment. I think in the future, this type of tooling will be ubiquitous.
I taught https://github.com/KillianLucas/open-interpreter how to use https://github.com/ferrislucas/promptr
Then I asked it to add a test suite to a rails side project. It created missing factories, corrected a broken test database configuration, and wrote tests for the classes and controllers that I asked it to.
I didn't have to get involved with mundane details. I did have to intervene here and there, but not much. The tests aren't the best in the world, but IMO they're adding value by at least covering the happy path. They're not as good as an experienced person would write.
I did spend a non-trivial amount of time fiddling with the prompts I used to teach OI about Promptr as well as the prompts I used to get it to successfully create the test suite.
The total cost was around $11 using GPT4 turbo.
I think in this case it was a fun experiment. I think in the future, this type of tooling will be ubiquitous.
It's pretty easy to force a locally running model to always output valid JSON: when it gives you probabilities for the next tokens, discard all tokens that would result in invalid JSON at that point (basically reverse parsing), and then apply the usual techniques to pick the completion only from the remaining tokens. You can even validate against a JSON schema that way, so long as it is simple enough.
There are a bunch of libraries for this already, e.g.: https://github.com/outlines-dev/outlines
Here's the fork of Open Interpreter that I was experimenting with: https://github.com/ferrislucas/open-interpreter/pull/1/files
The system prompt that adds the Promptr CLI tool is here: https://github.com/ferrislucas/open-interpreter/pull/1/files...