Show HN: Speeding up LLM inference 2x times (possibly)

• effort

  3 206 9.7 Swift

  An implementation of bucketMul LLM inference

I think it was somewhere around that tag:

https://github.com/kolinko/effort/releases/tag/5.0-last-mixt...

Cannot rerun easily any more, because the underlying model/weight names changed in the meantime. It doesn't help that Mixtral's published .safetensor files seem messed up, and I needed to hack a conversion from pytorch - it added an extra layer of confusion into the project.

• cria

  4 77 2.5 Python

  Tiny inference-only implementation of LLaMA (by recmo)

It originally started as a fork to Recmo’s cria pure numpy llama impl :)

https://github.com/recmo/cria

Took a whole night to compute a few

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project