gpt-neo VS obsidian-textgenerator-plugin

By December 2020, EleutherAI had introduced The Pile, a comprehensive text dataset designed for training models. Subsequently, tech giants such as Microsoft, Meta, and Google used this dataset for training their models. In March 2021, they revealed GPT-Neo, an open-source model under Apache 2.0 license, which was unmatched in size at its launch. EleutherAI’s later projects include the release of GPT-J, a 6 billion parameter model, and GPT-NeoX, a 20 billion parameter model, unveiled in February 2022. Their work demonstrates the viability of high-quality open-source AI models.

Yeah, if that is your requirement you should definitely ignore chatterbot, as its older and probably not what your teacher wants. I'm looking at the gpt-neo docs right now: https://github.com/EleutherAI/gpt-neo

3.) EleutherAI's GPT-Neo and GPT-NeoX: EleutherAI is an independent research organization that aims to promote open research in artificial intelligence. They have released GPT-Neo, an open-source language model based on the GPT architecture, and are developing GPT-NeoX, a highly-scalable GPT-like model. You can find more information on their GitHub repositories: GPT-Neo: https://github.com/EleutherAI/gpt-neo GPT-NeoX: https://github.com/EleutherAI/gpt-neox

This is one of the first comprehensive plugins that has been rewritten to support multiple AI backends such as OpenAI GPT3+ and other custom sources in the future such as ChatGPT, GPT-J, GPT-neo and more.

GPT-Neo is newer than GPT-2 on the open source side of things. In my experience, it tends to give longer and more creative responses than GPT-2 but not on the level of GPT-3. I've not tried GPT-J or GPT-NeoX, but they're also open source and reportedly better than GPT-Neo (albeit less accessible).

Finally, yes-- it is trained via masked language modeling (text prediction). The approach has been fairly standard for years- the big difference with the GPT* models is the number of paramaters and volume of text-- we still haven't reached a ceiling with LLM parameters- they appear to keep improving with size. This training allows the model to learn a strong representation of language. Their training approach is published and open-source GPT* versions have already been made and released (https://github.com/EleutherAI/gpt-neo). However, the models are huge and can't be run locally for hobbyists. This gets at larger issues in democratization of ML.

- GPT-Neo 2.7B, released Mar/2021, and unmaintained/unsupported as of Aug/2021? or;

I think you can achieve something similar to notion ai with text generator plugin maybe not so polished, but should be good enough

Add the link https://github.com/nhaouari/obsidian-textgenerator-plugin

I would argue that plugins solve this issue and save the dev team from becoming bogged down with ai requests. I'm sure if you were to share this in the plugin discord channel you may find the solution already exists. Much of what you describe could be done with a bit of research into the other available ai plugins for obsidian including AVA, Gpt-3 Notes, and the extremely powerful Text Generator plugin... off the top of my head I believe you could assign a hot key to a particular text generator custom template that says something like "generate tags based on the note content" and insert it into the YAML, take that even further by integrating it into your note template with Templater/Quickadd by calling the command upon creation.

I just remembered there is currently an OPENAI plugin called Text Generator that I've used and works pretty well, as a current example of a GPT-3 implementation. Here's the repo if it's any use to you: https://github.com/nhaouari/obsidian-textgenerator-plugin

I would try to combine a few plugins in Obsidian, Text Generator and any of the OCR plugins.

Well this already exists for GPT-3 so it's only a matter of them opening up the API I figure.

Hi there, Thanks for your interest in the Text Generator plugin! It's certainly an incredible tool. The full documentation for the plugin can be found at https://www.text-gen.com/. Unfortunately, the fine-tuning of parameters to train the generator on your own vault is not yet available, but it is on the roadmap for future development of this plugin. In the meantime, you can still use the Text Generator to explore and generate new ideas from text related to whatever topics you are interested in! Hope this helps and good luck with your project!