MindsDB VS postgresml

Check us out on GitHub.

If your data lacks uniform time intervals between consecutive entries, QuestDB offers a solution by allowing you to sample your data. After that, MindsDB facilitates creating, training, and deploying your time-series models.

MindsDB is an open-source AI platform for developers that connects AI/ML models with real-time data. It provides tools and automation to easily build and maintain personalized AI solutions.

MindsDB combines both AI and SQL functions in one; users can create, train, optimize, and deploy ML models without the need for external tools. Data analysts can create and visualize forecasts without having to navigate the complexities of ML pipelines.MindsDB is open-source and works with well-known databases like MySQL, Postgres, Redit, Snowflakes, etc.

Mindsdb is a good example. It abstracts everything related to an AI workflow as "virtual tables". For example, you can import OpenAI API as a "virtual table":

Repo : https://github.com/mindsdb/mindsdb

MindsDB is an AI Automation platform for building AI/ML powered features and applications. It works by connecting any data source with any AI/ML model or framework and automating how real-time data flows between them. MindsDB is integrated with LlamaIndex, which makes use of its data framework for connecting custom data sources to large language models. LlamaIndex data ingestion allows you to connect to data sources like PDF’s, webpages, etc., provides data indexing and a query interface that takes input prompts from your data and provides knowledge-augmented responses, thus making it easy to Q&A over documents and webpages.

Now, imagine if you can deploy these highly trained models in your database to get insights, make predictions, understand your users, auto-generate content, and more. MindsDB makes this possible! MindsDB is an open-source AI database middleware that allows you to supercharge your databases by integrating various machine learning (ML) engines.

The Python client SDK is so small, because it's just a wrapper around the Rust client SDK: https://github.com/postgresml/postgresml/tree/master/pgml-sdks/rust/pgml. Currently we also support JS/Typescript SDKs as well, all generated from the same safe and efficient underlying Rust implementation, using some fancy Rust macros.

I don't think you'd replace a materialized view with pg_later, but it might help you populate or update your materialized view if you are trying to do that asynchronously. pglater.exec() works with DDL too!

I use it a lot for long running queries when doing data science and machine learning work, and a lot of times when executing queries from a jupyter notebook or CLI. That way if my jupyter kernel dies, my query execution continues even if the network or my environment has an issue. I've started using it a bit more with https://github.com/postgresml/postgresml for model training tasks too, since those can be quite long running depending on the situation.

PostgresML comes with pgvector as a vector database. The cool thing is it can run your models in the same memory space as a database extension. We’re also working on ggml support for huggingface transformers, but could use some help testing more LLMs for compatibility. https://github.com/postgresml/postgresml/pull/748

We've been working on a Python SDK[1] for PostgresML to make it easier for application developers to get the performance and scalability benefits of integrated memory for LLMs, by combining embedding generation, vector recall and LLM tasks from HuggingFace in a single database query.

This work builds on our previous efforts that give a 10x performance improvement from generating the LLM embedding[2] from input text along with tuning vector recall[3] in a single process to avoid excessive network transit.

We'd love your feedback on our roadmap[4] for this extension, if you have other use cases for an ML application database. So far, we've implemented our best practices for scalable vector storage to provide an example reference implementation for interacting with an ML application database based on Postgres.

[1]: https://github.com/postgresml/postgresml/tree/master/pgml-sd...

We've been working on a Python SDK for PostgresML to make it easier for application developers to get the performance and scalability benefits of integrated memory for LLMs, by combining embedding generation, vector recall and LLM tasks from HuggingFace in a single database query.

Links:

[1]: https://huggingface.co/spaces/mteb/leaderboard

[2]: https://postgresml.org/blog/generating-llm-embeddings-with-o...

[3]: https://postgresml.org/blog/tuning-vector-recall-while-gener...

[4]: https://postgresml.org/blog/personalize-embedding-vector-sea...

Github: https://github.com/postgresml/postgresml

Github: https://github.com/postgresml/postgresml

I'd encourage you to do inference outside of PostgreSQL (use TF serving and make requests against it, or do batch inference), but if you're determined to do so, they have an extension that integrates with the transformers library and allows for calling models directly from SQL.