Big Data Is Dead

• blog

  11 1,975 7.2 JavaScript

  Some notes on things I find interesting and important. (by frankmcsherry)

  This reminds me of a great blog post by Frank McSherry (Materialize, timely dataflow, etc) talking about how using the right tools on a laptop could beat out a bunch of these JVM distributed querying tools because... data locality basically.

  https://github.com/frankmcsherry/blog/blob/master/posts/2015...

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• memray

  28 13,389 8.9 Python

  Memray is a memory profiler for Python

  

• ClickHouse

  230 37,813 10.0 C++

  ClickHouse® is a real-time analytics DBMS

  

  One great reason to use DuckDB was when ClickHouse took up too much memory on Parquet files.

  https://github.com/ClickHouse/ClickHouse/issues/45741#issuec... helps with that though.

  Also, clickhouse-local exists https://clickhouse.com/blog/extracting-converting-querying-l... as a thing.

  But, yes, when I think of DuckDB...I think embedded use cases...i'm also not a power user.

  I also think of this very much as a 'horses for courses' or 'different strokes, different folks' sort of scenario. There is, naturally, overlap because 'analytical data.' But also, there is naturally overlap with R and this giant scary mess of data-munging PERL code I maintain for a side project.

  The DuckDB team, the MotherDuck team, the ClickHouse team...we all want your experience interacting with data to be amazing. In some scenarios, ClickHouse is better. In some scenarios, DuckDB. I'm biased (as I work for ClickHouse in DevRel), but I <3 ClickHouse.

  Try both. Pick the one that is best for you. Then...you know...tell the other(s) why so that we all can get better at what we do.

Suggest a related project