[D] Applications for using reinforcement learning to fine-tune GPT-2

• trl

  13 8,023 9.6 Python

  Train transformer language models with reinforcement learning.

  

I'm investigating the pros and cons of a more naive approach that does not require collecting a dataset of human preferences. Using the trl library, I train a BERT-classifier to distinguish between sarcastic and non-sarcastic reddit comments, and that classifier then serves as a reward model that provides a reward signal for fine-tuning GPT-2 for text generation using PPO. I have applied the same method to the task of generating negative review, by training BERT on the IMDB-dataset. This method of course leads to extensive reward hacking, but investigating how to mitigate that is part of the fun!

• lm-human-preferences

  8 1,106 2.7 Python

  Code for the paper Fine-Tuning Language Models from Human Preferences

  

Code for https://arxiv.org/abs/1909.08593 found: https://github.com/openai/lm-human-preferences

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project