GPT-J-6B, a 6B Parameter Text Generation Model

• mesh-transformer-jax

  52 6,213 0.0 Python

  Model parallel transformers in JAX and Haiku

If you’re also curious how it does vs GPT3 here’s the Zero-shot performance evaluations table: https://github.com/kingoflolz/mesh-transformer-jax#zero-shot...

• jax

  82 28,004 10.0 Python

  Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

  

This is first time for me to see JAX [1] being used by non-Google employee. Is it getting some momentum? Or this person just happened to pick it up for some reason (for, say, better TPU support)?

[1] https://github.com/google/jax

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• the-pile

  15 1,403 0.0 Python
  

Still upset about the pile as a dataset because I wrote an issue asking if they would include my dataset debatesum into the dataset[1]. They mentioned that they planned to do it and even wrote another issue indicating that they would include it in the pile.

Months go by, and I ask why this dataset is still not included. They told me that the pile is complete and will not be updated anymore. I didn't have time to write my own PR for this and had no idea that there was a time limit to get it in.

Makes me sad. I don't know why this company has decided to stop updating the pile...

[1] https://github.com/EleutherAI/the-pile/issues/56

• mesh-transformer-jax

  1 0 0.0 Jupyter Notebook

  Model parallel transformers in JAX and Haiku (by AeroScripts)

I'm running it quite comfortably on my 3090, although it's a really snug fit for the VRAM, and that's with a number of fixes to significantly reduce its memory use from https://github.com/AeroScripts/mesh-transformer-jax .

Suggest a related project