transformer-debugger VS Example_Data

You can actively see a fresh "hype curve" in the transformer-debugger repo that was posted a couple days ago (https://github.com/openai/transformer-debugger) (star history https://star-history.com/#openai/transformer-debugger&Date).

Regardless of the repo's stars or how valuable it really is, at the time I saw it posted to HN, it had 1.6k stars/16 hours. What channel are people listening to to star it so quickly. I'm not implying any nefariousness, mind you, I'm only wondering where all the stargazers were referred from so fast and in such volume.

Interesting to see the use of ruff and black in the same project. https://github.com/openai/transformer-debugger/blob/main/.pr...

We may well look back in future years and view the underlying approach introduced in Reexpress as among the more significant results of the first quarter of the 21st century. With Reexpress, we can generate reliable probability estimates over high-dimensional objects (e.g., LLMs), including in the presence of a non-trivial subset of distribution shifts seen in practice. A non-vacuous argument can be made that this solves the alignment/super-alignment problem (the ultimate goal of the line of work in the post above, and why I mention this here), because we can achieve this behavior via composition with networks of arbitrary size.

Because the parameters of the large neural networks are non-identifiable (in the statistical sense), we operate at the unit of analysis of labeled examples/exemplars (i.e., the observable data), with a direct connection between the Training set and the Calibration set.

This has important practical implications. It works with essentially any generative AI model. For example, we can build an 'uncertainty-aware GPT-4' for use in enterprise and professional settings, such as law: https://github.com/ReexpressAI/Example_Data/blob/main/tutori...

(The need for reliable, controllable estimates is critical regardless of any notion of AGI, since the existing LLMs are already getting baked into higher-risk settings, such as medicine, finance, and law.)

We've been working to make uncertainty quantification and interpretability first-class properties of LLMs. Reexpress one, a macOS app, is our first effort to make these properties widely available.

Perhaps counter-intuitively, and contrary to common wisdom, LLMs can in fact be transformed to generate very reliable uncertainty estimates (i.e., "knowing what they do and don't know" by assigning a probability to the output).

Getting there is a bit complicated, with vector matching/databases, prediction-time data dependencies, complicated inference, and multiple models flying all over the place.

We've made it simple and efficient to use in practice with an on-device, no-code approach. Common document classification tasks can be handled with the on-device models (up to 3.2 billion parameters). Additionally, you can add these capabilities to another LLM (e.g., for QA or more complicated tasks) by connecting your existing model by simply uploading the output logits into the app. For example, if you're using an on-device Mistral AI model, or cloud-based genAI model, just upload the output logits into the app.

Would be great to get feedback. Also, if you have another use case with a scale that doesn't fully fit into the on-device setting, happy to discuss and collaborate for your setting.

And if anyone finds this interesting and wants to get involved more in building reliable AI, let us know!

(Note that an Apple silicon Mac is required; ideally M1 Max or better with 64gb of RAM. You train the model yourself, which requires labeled data. The tutorial 1 video has a link to sentiment data in the JSON lines format; it's a good place to start: https://github.com/ReexpressAI/Example_Data/blob/main/tutori...)