OpenAI: Memory and New Controls for ChatGPT

• refactor-benchmark

  2 18 5.9 Python

  Aider's refactoring benchmark exercises based on popular python repos

1-2 sentences: Rather than writing code, GPT-4 Turbo often inserts comments like "... finish implementing function here ...". I made a benchmark that provokes and quantifies that behavior.

1-2 paragraphs:

I found that I could provoke lazy coding by giving GPT-4 Turbo refactoring tasks, where I ask it to refactor a large method out of a large class. I analyzed 9 popular open source python repos and found 89 such methods that were conceptually easy to refactor, and built them into a benchmark [0].

GPT succeeds on a task if it can remove the method from its original class and add it to the top level of the file with appropriate changes to the SIZE of the abstract syntax tree. By measuring the size of the AST, we infer that GPT didn't replace a bunch of code with a comment like "... insert original method here...". I also gathered other laziness metrics like counting the number of new comments that contained "...", which correlated well with the AST size test.

[0] https://github.com/paul-gauthier/refactor-benchmark

• replacefn

  1 0 4.4 Rust

Lazy coding is a feature not a bug. My guess is that it breaks aider automation, but by analyzing the AST that wouldn't be a problem. My experience with lazy coding, is it omits the irrelevant code, and focuses on the relevant part. That's good!

As a side note, i wrote a very simple small program to analyze Rust syntax, and single out functions and methods using the syn crate [1]. My purpose was exactly to make it ignore lazy-coded functions.

[1]https://github.com/pramatias/replacefn/tree/master/src

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• llama-cpp-python

  54 6,378 9.9 Python

  Python bindings for llama.cpp

I'll share the core bit that took a while to figure out the right format, my main script is a hot mess using embeddings with SentenceTransformer, so I won't share that yet. E.g: last night I did a PR for llama-cpp-python that shows how Phi might be used with JSON only for the author to write almost exactly the same code at pretty much the same time. https://github.com/abetlen/llama-cpp-python/pull/1184

• neuromorph

  1 3 8.0 Python

  Neuromorphic agents are AI assistants modeled after the human brain

Suggest a related project