evidence VS Trino

If you are a data analyst interested in making web apps, I might point you to my own OSS project, Evidence, which is a Static Site Generator for building reports and data apps with SQL and markdown.

Repo (3K stars): https://github.com/evidence-dev/evidence

Previous discussions on HN:

https://news.ycombinator.com/item?id=28304781 - 91 comments

It’s interesting to me how far you have pushed the SQL language in this framework, such that it truly is “SQL only”.

The challenge as I see it with enabling analysts to build websites is that you need to build abstractions to get from familiar (SQL, yaml) - the language of analytics, to new (HTML, CSS, JS) - the language of the web browser

As one of the maintainers of Evidence (https://evidence.dev), one of the things I’ve often considered is how accessible our syntax is to analysts. Our syntax combines SQL and Markdown, with MDX style components e.g.

The are inherently webdev-ey, and I do think they put off potential users.

On the flip-side, by adhering to web standards, you get extensibility out of the box, and working out what to do is just a Google search away.

Anyway, thanks for the thought provoking piece.

Dataclips was my first experiences writing SQL.

Writing code was a markedly better DX that building dashboards in Tableau, which is why I'm now working on https://evidence.dev - a SSG for creating data from SQL and markdown

Previous HN discussions:

I'm one of the founders of Evidence (https://evidence.dev) - would be great to hear about your experience. Reaching out now!

Full fledged BI tools like Superset and Metabase are amazing for their intended use cases.

But they may be an overkill if your primary use case is to infrequently build semi-interactive reports for non-technical end-users and your use cases are are mostly covered by standard graphs & tables. Esp. so if you are familiar with SQL and have access to the underlying data source. Two nifty utilities I have found to be very useful for latter kind of use cases are SQLPage and Evidence.

They make it very convenient to whip out some SQL and convert that to a neat professional looking web ui that can be forwarded to an end user. In case of Evidence it is a statically generated site, and in case of SQLPage it is a web app that connects to a live database.

SQLPage: https://sql.ophir.dev/

Evidence: https://evidence.dev

We use ECharts in our open source BI tool (Evidence) and it's a great library. Has helped us build a declarative syntax for viz which can be version controlled (https://evidence.dev)

We used ECharts to build our charting library at Evidence and it’s been a great experience overall (https://evidence.dev).

We started with D3 and a few other tools, but felt that we get a lot more out of the box with ECharts, like interactivity and an events API. ECharts is also a lot more extensible than people give it credit for.

If anyone is curious, we documented the process of selecting a charting library after assessing several options: https://github.com/evidence-dev/evidence/issues/136

The new direction seems very similar to what evidence has been doing for a while

https://evidence.dev

I'm quite excited about this, and would also love to have it distributed as an NPM package.

I work on an OSS web framework for reporting/ decision support applications (https://github.com/evidence-dev/evidence), and we use WASM duckDB as our query engine. Several folks have asked for PRQL support, and this looks like it could be a pretty seamless way to add it.

> Not sure the language you choose matters as much as making the API usable by a wide audience.

Fully agree with this, and having typeset my masters thesis and later my resume using LaTeX, I think that the “authoring experience” is 100% the place to focus on improving.

If you’re interested in the “markup to document publishing” space, you might also be interested in the open-source report publishing tool I’m now working on, Evidence (https://github.com/evidence-dev/evidence)

It’s similarly based on markdown, though uses code fences to execute code, HTML style tags for charts and components, and {…} for JavaScript, i.e.

---

DevRel for StarRocks. Trino doesn't have a great caching layer (https://github.com/trinodb/trino/pull/16375) and performance (https://github.com/trinodb/trino/issues/14237) and https://github.com/oap-project/Gluten-Trino. In benchmarks and community user testing, StarRocks has outperformed.

What if my SQL engine is Presto, Trino [1], or a similar query engine? If it's federating multiple source databases we peel the SQL back and get... SQL? Or you peel the SQL back and get... S3 + Mongo + Hadoop? Junior analysts would work at 1/10th the speed if they had to use those raw.

[1] https://trino.io/

The good thing is that the concepts in terms to the SQL supported by Trino transfers between them all. So its completely reasonable to start with one and move to another. In fact that is something that happens regularly. I invite to you check out the talks from the Trino Fest event that is just wrapping up today. There are presentations about all these aspects and different scenarios users encounter. All videos and slides will go live on the Trino website soon. Also feel free to join the Trino slack to chat about about all this with other users.

Trino or Hive for SQL querying. Get Trino/Hive to talk to Nessie.

One place to look are the projects repo's and docs, once you have a good idea of how the system is architected poking around pieces of the codebase can be helpful in letting you really understand their internals. I personally enjoy going through spark repo and trino repo and the documentation for both projects is decent and can answer many of your questions.

However, once your data reaches a certain size or you reach the limits of vertical scaling, it may be necessary to distribute your queries across a cluster, or scale horizontally. This is where distributed query engines like Trino and Spark come in. Distributed query engines make use of a coordinator to plan the query and multiple worker nodes to execute them in parallel.

So, there are a lot of people involved which is why you try to find some common ground regarding coding style, documentation, programming language version, … Most larger project have some guidelines you can read to see what is necessary for a PR to be approved: Trino’s Guidelines or dbt.