perspective VS ClickHouse

Try this project I work on, https://perspective.finos.org/ . It comes with a Python library (that runs the exact same engine/code as in the wasm version), and even a JupyterLab plugin. You can run the engine fully in-python, in-Javascript (via wasm), or replicated between the two.

Standalone Python virtual server example https://github.com/finos/perspective/tree/master/examples/to...

JupyterLab demo on Binder https://mybinder.org/v2/gh/finos/perspective/master?urlpath=...

The best example of WASM being used to render to canvas (it's also visualizations) I've seen is "Perspective":

https://perspective.finos.org/

  "Perspective is an interactive analytics and data visualization component, which is especially well-suited for large and/or streaming datasets. Originally developed at J.P. Morgan and open-sourced through the Fintech Open Source Foundation (FINOS), Perspective makes it simple to build user-configurable analytics entirely in the browser, or in concert with Python and/or Jupyterlab. Use it to create reports, dashboards, notebooks and applications, with static data or streaming updates via Apache Arrow."

Man the a16z marketing machine is working hard unfortunately at cost of quality.

For those interested in FS and open source today, especially with a capital markets lens check out:

https://www.finos.org

Lots of great projects, one I used recently and a favourite was this:

https://github.com/finos/perspective

As far as customizing the Perspective datagrid, the story on this is evolving :) . With the 1.0 release, we've released an NFT demo with a more current version of the plugin API, as well as new plugin API docs. Replacing innerHTML is only costly if you trigger a relayout before the replacement, which you'd want to avoid - check the pudgy-penguins demo source for examples which replaces these with without the intermediate DOM tree being rendered (though this is brower-dependent). If you can't, e.g. the replacement is async or whatever, the underlying regular-table component has an API that allows you to return the DOM elements themselves per cell, but you'd need to write a simple plugin to integrate this as Perspective's version provides its own dataListener.

By the way, the link to the blog on the project website results in a 404.

No surprises here: deep learning is the most popular subcategory, with hugging face transformers repo, YOLOv5, Tensorflow and Deepmind’s Alphafold all in the mix. Surprisingly, the only proper infrastructure-ey repos on the list are Meilisearch and Clickhouse, a tad bit surprising given all the hype data infrastructure receives in VC-world, but again, probably just a question of size of end-user populations + whether data scientists spend tons of time on Github vs. Web Developers…

And by this do you mean this? https://clickhouse.com/

In this blog post, we will be reviewing how we can integrate predictive capabilities powered by machine learning with the ClickHouse database. ClickHouse is a fast, open-source, column-oriented SQL database that is very useful for data analysis and real-time analytics. The project is maintained and supported by ClickHouse, Inc. We will be exploring its features in tasks that require data preparation in support of machine learning.

There's ksqldb (open source, built with java) and materialize (there's standalone edition), both need to use Kafka/RedPanda, also Clickhouse (open source, with materialize view with specific engine, but need to buffer the inserts using proxy like KittenHouse or buffering library like ch-timed-buffer), is there any other alternative to those 3 (that similarly doesn't do full scan to do aggregation)?

Moreover, using open-source warehouse tools can allow unlocking additional insights from your data in real-time and at a lesser cost. PostgreSQL (website, repo) is a popular example of an efficient and low-cost data warehousing solution. Another example is ClickHouse (website, GitHub), an open-source, analytics-focused DBMS that allows generating analytical reports from data in real-time using SQL.

The most efficient way is to use column store databases as data sources for eMondrian. For example, ClickHouse could run as a powerful and fast query engine while eMondrian works as a proxy representing data as cubes and executing MDX queries.

As of writing, there's a feature request on Github for adding specific commands for materializing specific columns on ClickHouse data parts.