Is GPT actually using the encoder NOT the decoder part of the transformer?

• minGPT

  35 18,875 0.0 Python

  A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

In the original paper they mention they are only using the decoder part of the model. However, their description and implementations seem to be using the encoder part of the transformer not the encoder. For example, this implementation of the original transformer encoder layer matches what the one in the GPT implementation.

• transformer-pytorch

  2 2,187 2.1 Python

  Transformer: PyTorch Implementation of "Attention Is All You Need"

In the original paper they mention they are only using the decoder part of the model. However, their description and implementations seem to be using the encoder part of the transformer not the encoder. For example, this implementation of the original transformer encoder layer matches what the one in the GPT implementation.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project