BERTweet VS kiri

Not sure if that's what you are looking for, but BERTweet has model trained on tweets containing COVID keywords https://github.com/VinAIResearch/BERTweet

I'm one of the authors of Backprop, a library built for transfer learning.

I'd like to share Backprop, an open source library I've been co-authoring for the last few months.

We've also got an open-source library that makes using + finetuning these models possible in a few lines of code.

I'd like to share Backprop, a Python library I've been co-authoring for the last few months. Our goal is to make finetuning and using models as easy as possible, even without extensive ML experience.

You might get some really promising results with finetuning.

If anything, you could build writing assistance that almost automates responses.

I've been co-authoring a library that lets you finetune such models in a single line of code.

https://github.com/backprop-ai/backprop

In specific the text generation finetuning example should be what you are looking for: https://github.com/backprop-ai/backprop/blob/main/examples/F...

Hope this helps, happy to chat more about it. Pretty curious about the results.

Here's an example Jupyter Notebook for finetuning T5. Full disclosure, I work on this library myself -- but it could be helpful.

I'm working on a library that hopefully makes working with some of these a bit easier -- here's an example notebook for running text classification with the BART checkpoint, if you're interested. If you need more task-specific finetuning for text classification, that's going to be rolled out in the near future.

I'm working on a library that includes a few different ML tasks, including summarisation. It uses a pretrained version of Google's T5 transformer model, which we host on Hugging Face with some details on how it was trained.