LLM-Finetuning-Hub VS rivet

Hi everyone, we've been working on benchmarking different open-source LLMs. We measure, in particular, on the performance of these models once finetued (via QLoRA) on classic NLP downstream tasks like summarization and classification. We also put particular emphasis on benchmarking inference time/cost for these models once deployed.

Hello all,

I have been working on benchmarking different LLMs -- both open-source and closed-source.

Repo: https://github.com/georgian-io/LLM-Finetuning-Hub

Precisely, I am comparing their out-of-the-box capabilities (prompting) and their fine-tuned conterparts!

So far, the following models have been benchmarked:

Open-Source:

- Finetuning with QLoRA

I evaluate how Falcon does on classification tasks when compared to Bert and Distilbert.

Moreover, I talk about different ways you can deploy the model, and the associated costs!

The code for all of my experiments are available on: https://github.com/georgian-io/LLM-Finetuning-Hub

Happy reading and learning!

But now coming to today's tool, let's talk about a topic that comes and goes, but it's always fun to play around with. I'm referring to AI; as you know, I'm not an AI-addicted person, but I like to have my share of fun with it! And a few days ago I came across the program that I am presenting to you today which aims to be very interesting and fun! Rivet, this is the name of the tool of this week, is a visual development environment for AI agents. But in what sense? Well, once you download the software, you will have a drag and drop interface to create a flow of information that can interact with various LLMs, whether it's GPT, Google models, or even Open models like Ollama running locally. It can handle all the classic control flows (if, loop etc.) and can integrate JS functions directly into it or import them from the project where we decide to incorporate it.

https://rivet.ironcladapp.com