Thinking Like Transformers

• tracr

  2 465 7.8 Python
  

To quote someone: RASP is like Matlab, designed by Satan.

There is an interpreter for a RASP like language if you want to try it out: https://srush.github.io/raspy/

And deepmind published a compiler from RASP to Transformer weights: https://github.com/deepmind/tracr

• Transformer-Puzzles

  2 256 4.3 Jupyter Notebook

  Puzzles for exploring transformers

There is also a more interactive version if you want to challenge yourself. A Python notebook of interactive puzzles to build an adder with transformers.

https://github.com/srush/Transformer-Puzzles

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• RASP

  3 267 6.5 Python

  An interpreter for RASP as described in the ICML 2021 paper "Thinking Like Transformers"

  

there's also an interpreter for RASP as described in the paper :) https://github.com/tech-srl/RASP

Sasha's blog (your link) has a nice walkthrough of long addition with RASP though!

Suggest a related project