Engine recommendation for heavy and fast etl on single server

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • duckdb

    DuckDB is an in-process SQL OLAP Database Management System

  • - DuckDb doesn't support off memory group by: https://github.com/duckdb/duckdb/discussions/6696

  • ClickHouse

    ClickHouse® is a free analytics DBMS for big data

  • - clickhouse is apparently implemented off memory joins, but I somehow can't run even very simple queries without OOM and don't know what to do with it: https://github.com/ClickHouse/ClickHouse/issues/47521

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts