[D] Why are transformers still being used?

• flash-attention

  26 10,990 9.4 Python

  Fast and memory-efficient exact attention

  

• RWKV-v2-RNN-Pile

  6 65 0.0 Python

  RWKV-v2-RNN trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• quality

  2 103 5.8 Python
  

Totally agree. Almost all standard tasks which are used to measure the performance of nlp are using very short inputs. So nlp researcher have no real incentives to work on that. On the other hand, tasks related to longer texts are often concentrating on aspects which do not need a deeper understanding of text (often a bow representation does quite well in this context). But I think interest in more compex tasks in relation to longer texts is growing. See for example this dataset published 6 months ago: https://github.com/nyu-mll/quality

Suggest a related project