datastation-documentation VS superset

Hey folks! I quit my job at Oracle almost a year ago now to build DataStation. It's an app I've wanted as an engineering manager for years now.

DataStation helps you query a variety of data sources (conventional SQL like PostgreSQL and MySQL, non-SQL like Prometheus or Elasticsearch), files and HTTP APIs. It is not a SQL layer on top of these various APIs like FDW in Postgres or Apache Calcite. For Prometheus you query with PromQL. For Elasticsearch you query with Lucene. And for SQL databases you query with their SQL dialect.

DataStation is made of panels (other apps might call them cells) that each produce a result. Panels can refer to other panels. These allow you to build workflows that cross the boundary of a particular datasource. For example you might have some data in a CSV a product manager gave you and the bulk of your data is in PostgreSQL. DataStation helps you pull together these data sets and script them.

DataStation is mainly a desktop app today where the end result is that you export graph SVGS or HTML tables or markdown tables or just a CSV file. All this data stays on your laptop so it's as easy to use in a corporate environment as any existing SQL IDE or Jupyter Notebook.

In the last year it's reached 1.5k stars on Github, over 1000 unique users and currently on-average about 40 fairly active users per month (defined as having opened the app more than a few times).

DataStation is primarily an Electron app but the code that evaluates panels is written in Go.

You can find a ton of tutorials on how to interact with supported databases on the DataStation website: https://datastation.multiprocess.io/docs/.

Looking forward to your feedback!

There are primarily a bunch of tutorials for getting started [0] (these are up to date) and some old videos (not up to date) [1].

[0] https://datastation.multiprocess.io/docs/

[1] https://www.youtube.com/channel/UCGOQFKonPUVo5LgxQDW26yg/vid...

A markdown generator embeds markdown from a Github repo into the marketing site. This way the marketing site is kept private while anyone can easily contribute to docs.

Docs are kept in separate folders for each release.

https://github.com/multiprocessio/datastation-documentation

Superset is absolutely phenomenal. I really hope Microsoft eventually releases all of their customizations they made to it internally to the OS community someday.

https://www.youtube.com/watch?v=RY0SSvSUkMA

https://github.com/apache/superset/discussions/20094

I recently ran a little shootout between Superset, Metabase, and Lightdash. All have nontrivial weaknesses but I ended up picking Lightdash.

Superset the best of them at _data visualization_ but I honestly found it almost useless for self-serve _BI_ by business users. This issue on how to do joins in Superset (with stalebot making a mess XD) is everything difficult about Superset for BI in a nutshell. https://github.com/apache/superset/issues/8645

Metabase is pretty great and it's definitely the right choice for a startup looking to get low cost BI set up. It still has a very table centric view, but feels built for _BI_ rather than visualization alone.

Lightdash has significant warts (YAML, pivoting being done in the frontend, no symmetric aggregates) but the Looker inspiration is obvious and it makes it easy to present _groups of tables_ to business users ready to rock. I liked Looker before Google acquired it. My business users are comfortable with star and snowflake schemas (not that they know those words) and it was easy to drop Lightdash on top of our existing data warehouse.

Or like not being able to delete a user without running some SQL:

https://github.com/apache/superset/issues/13345

Almostl instantly run into this issue setting up a test instance of Superset. And the issue has been around for years.

Are you interested in trying out Superset, but you're intimidated by the local setup process? Worry not! Superset needs some initial setup to install locally, but I've got a streamlined way to get started - using the makefile! This file contains a set of scripts to simplify the setup process.

Recently I discovered BigQuery public datasets - just over 200 datasets available for directly querying via SQL. I think this is a great thing! I can connect these direct to an analytics platform (we use Apache Superset which uses Python SQLAlchemy under the hood) for example and just start dashboarding.

if they don't want to pay for powerbi, can try something like https://superset.apache.org/

In today's fast-paced data-driven world, organizations must analyze data in real-time to make timely and informed decisions. Real-time data analytics enables businesses to gain valuable insights, respond to real-time events, and stay ahead of the competition. Also, the analytics engine must be capable of running analytical queries and returning results in real-time. In this article, we will explore how you can build a real-time data analytics solution using the open-source tools Redpanda a distributed streaming platform, Apache Superset, a data visualization, and a business intelligence platform, combined with RisingWave a streaming database.