LLM-Finetuning-Hub VS sqllineage

Hi everyone, we've been working on benchmarking different open-source LLMs. We measure, in particular, on the performance of these models once finetued (via QLoRA) on classic NLP downstream tasks like summarization and classification. We also put particular emphasis on benchmarking inference time/cost for these models once deployed.

Hello all,

I have been working on benchmarking different LLMs -- both open-source and closed-source.

Repo: https://github.com/georgian-io/LLM-Finetuning-Hub

Precisely, I am comparing their out-of-the-box capabilities (prompting) and their fine-tuned conterparts!

So far, the following models have been benchmarked:

Open-Source:

- Finetuning with QLoRA

I evaluate how Falcon does on classification tasks when compared to Bert and Distilbert.

Moreover, I talk about different ways you can deploy the model, and the associated costs!

The code for all of my experiments are available on: https://github.com/georgian-io/LLM-Finetuning-Hub

Happy reading and learning!

For the open source there is this library https://github.com/reata/sqllineage.

Is the idea here that it's inspired by re_data due to using dbt transformations underneath or because it's reposted looking nearly the same? (or both?)

Looks like much of the lineage code is also largely a wrapper around this library: https://github.com/reata/sqllineage

Would be curious to understand the project's purpose and unique contributions vs. the underlying dependencies powering it as there seems to be some ambiguity. Is this just a wrapper around dbt transformations and a lineage library in one package? Can I just use them directly?