edenai-apis VS COMET

HackerNoon featured our latest article in the "Future of AI" category

We explain how Eden AI contributes to the AI ecosystem in structuring AI and LLM APIs by creating the most accomplished Open-Source wrapper possible.

You can support us in reaching 1000 stars on Github here: https://github.com/edenai/edenai-apis

PS: if you can star our project, we'd really appreciate the support: https://github.com/edenai/edenai-apis

I'm working on a project that regroups all best AI (AIaaS) from different providers (GCP, AWS, Azure, DeepL, etc.) in one API.

If you don't want third party and feel the need of hanving your own accounts with the different providers, then you can use the open source version of the aggregator : https://github.com/edenai/edenai-apis

There are hundreds of companies doing that. The github repo regroups the best ones in one place.

Now we’re open sourcing our aggregation layer as a python module that you can ⭐ find on github ⭐ .

As it's said in the article, the library used is Open Source : https://github.com/edenai/edenai-apis/tree/master/edenai_apis

It's definitely a hard task to evaluate. I think we can use models like https://github.com/Unbabel/COMET for translation to try and mimic human evaluation. I don't know if datasets exist for that. There are some research done about that : https://aclanthology.org/P19-1502/ https://arxiv.org/abs/2104.00054v1

2/ Evaluation We compare Open AI to DeepL, ModernMT, NeuralSpace, Amazon and Google. A lot of metrics exist for automatic machine translation evaluation. We chose COMET by Unbabel (wmt21-comet-da) which is based on a machine learning model trained to get state-of-the-art levels of correlation with human judgements. (read more on their paper ) .

I'm trying to understand how I can use COMET to evaluate translation models https://github.com/Unbabel/COMET ? I don't really understand how it was trained the meaning of the outputed values ? https://unbabel.github.io/COMET/html/faqs.html#which-comet-model-should-i-use