gpt-2-output-dataset VS mesh-transformer-jax

especially: "Our classifier is not fully reliable. In our evaluations on a “challenge set” of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier’s reliability typically improves as the length of the input text increases. Compared to our previously released classifier, this new classifier is significantly more reliable on text from more recent AI systems." Many other classifiers are similar, e.g.:

Links search found: https://github.com/openai/gpt-2-output-dataset (dataset? I want GPT, Pre-trained model).

Source: the literal source code and paper by the original creators of a detector that most of these knockoff detectors are based on.

OpenAI published a detector themselves that seems to be quite good. https://github.com/openai/gpt-2-output-dataset/tree/master/detector

Here is OpenAI's own detector, but it's not impossible to beat by just doing some fairly basic stuff with things like automatic paraphrasing.

To a certain extent, GPT-2 worked because of the smaller dataset of just 40GB. Even in that model, researchers running detection found accurate results only in the:

GPT-J is a LLM case study with two goals: Training a LLM with a data source containing unique material, and using the training frameworkMesh Transformer JAX to achieve a high training efficiency through parallelization. There is no research paper about GPT-J, but on its GitHub pages, the model, different checkpoints, and the complete source code for training is given.

This idea has already been proposed in ViT-22B and GPT-J-6B.

Sure. Here's the repo I used for the fine-tuning: https://github.com/kingoflolz/mesh-transformer-jax. I used 5 epochs, and appart from that I kept the default parameters in the repo.

You can skip to step 4 using something like GPT-J as far as I understand: https://github.com/kingoflolz/mesh-transformer-jax#links

The pretrained model is already available.

The Github repo suggests it's possible you can change the number of checkpoints to make it run on a GPU.

GPT-J, which uses mesh-transformer-jax: https://github.com/kingoflolz/mesh-transformer-jax

They hinted at it in the screenshot, but the goods are linked from the https://6b.eleuther.ai page: https://github.com/kingoflolz/mesh-transformer-jax#gpt-j-6b (Apache 2)