evidence VS obsidian-dataview

We use echarts at https://evidence.dev and have been quite happy with it. We do a lot of embedded analytics and it's worked well for us.

It’s interesting to me how far you have pushed the SQL language in this framework, such that it truly is “SQL only”.

The challenge as I see it with enabling analysts to build websites is that you need to build abstractions to get from familiar (SQL, yaml) - the language of analytics, to new (HTML, CSS, JS) - the language of the web browser

As one of the maintainers of Evidence (https://evidence.dev), one of the things I’ve often considered is how accessible our syntax is to analysts. Our syntax combines SQL and Markdown, with MDX style components e.g.

The are inherently webdev-ey, and I do think they put off potential users.

On the flip-side, by adhering to web standards, you get extensibility out of the box, and working out what to do is just a Google search away.

Anyway, thanks for the thought provoking piece.

Dataclips was my first experiences writing SQL.

Writing code was a markedly better DX that building dashboards in Tableau, which is why I'm now working on https://evidence.dev - a SSG for creating data from SQL and markdown

Previous HN discussions:

I'm one of the founders of Evidence (https://evidence.dev) - would be great to hear about your experience. Reaching out now!

Full fledged BI tools like Superset and Metabase are amazing for their intended use cases.

But they may be an overkill if your primary use case is to infrequently build semi-interactive reports for non-technical end-users and your use cases are are mostly covered by standard graphs & tables. Esp. so if you are familiar with SQL and have access to the underlying data source. Two nifty utilities I have found to be very useful for latter kind of use cases are SQLPage and Evidence.

They make it very convenient to whip out some SQL and convert that to a neat professional looking web ui that can be forwarded to an end user. In case of Evidence it is a statically generated site, and in case of SQLPage it is a web app that connects to a live database.

SQLPage: https://sql.ophir.dev/

Evidence: https://evidence.dev

We used ECharts to build our charting library at Evidence and it’s been a great experience overall (https://evidence.dev).

We started with D3 and a few other tools, but felt that we get a lot more out of the box with ECharts, like interactivity and an events API. ECharts is also a lot more extensible than people give it credit for.

If anyone is curious, we documented the process of selecting a charting library after assessing several options: https://github.com/evidence-dev/evidence/issues/136

The new direction seems very similar to what evidence has been doing for a while

https://evidence.dev

I'm quite excited about this, and would also love to have it distributed as an NPM package.

I work on an OSS web framework for reporting/ decision support applications (https://github.com/evidence-dev/evidence), and we use WASM duckDB as our query engine. Several folks have asked for PRQL support, and this looks like it could be a pretty seamless way to add it.

> Not sure the language you choose matters as much as making the API usable by a wide audience.

Fully agree with this, and having typeset my masters thesis and later my resume using LaTeX, I think that the “authoring experience” is 100% the place to focus on improving.

If you’re interested in the “markup to document publishing” space, you might also be interested in the open-source report publishing tool I’m now working on, Evidence (https://github.com/evidence-dev/evidence)

It’s similarly based on markdown, though uses code fences to execute code, HTML style tags for charts and components, and {…} for JavaScript, i.e.

---

At the end of the day, I use Dataview, a plugin for Obsidian, which allows me to make queries to my notes similar to SQL to visualize the collected information:

https://github.com/blacksmithgu/obsidian-dataview

This whole ideas to have data, visualisations and knowledge base in one private offline place is very appealing

Since at least 2012 I've also been using a text file format from http://todotxt.org/ and more recently I wrote a program that takes a crontab-like list to pre-generate entries on a daily, by-day-name (every Sunday for example), and I also pull in a list of holidays from gov.uk, so they are also populated.

[^1]: (https://github.com/blacksmithgu/obsidian-dataview)

> Joplin is using md to.

The way it's handled can make the difference in control.

> by separating that in their DB, it's a big NO for me since it's a closed silo.

Joplin is using a popular open database with a healthy community and good tooling. It's as open as markdown. Maybe not for you, when you lack the knowledge, but markdown is similar closed for anyone not understanding filesystems and editors.

> This: https://github.com/blacksmithgu/obsidian-dataview works so wonderful for me

Good for you, but that is very low level in terms of data-handling. Dataview is really just an elaborated search, there is no good level of interaction. Datacore, the next project of the Dataview is supposed to bring this, but it's not even usable yet AFAIK. Coincidental, the Obsidian-devs are also working on that front, but nothing is finished yet.

> https://github.com/denolehov/obsidian-git and b) easy to fix since it's a text file. Gosh!

That's useless when the app itself is not working. And even worse if you are not realizing the errors early.

> Aha. I don't think so. Which authority says that?

My own experience. I've tested enough plugins over the years to know their dark corners.

> And even if It's like that, my markdown files would survive everything

The thing is, technically you are not even having proper markdown, but a fork with some extensions of Obsidian. So some features of your parts might break when switching away from Obsidian. And the reason for all this is also because markdown is lacking definitions for what obsidian-people are doing with it. Coincidentally, this seems also one of the reasons why Joplin is using a database.

> And gosh, this is a good thing!

Not if they all suck.

> Installing multiple task plugins shows that something is "broke" on the user side.

Sure, because the plugins are lacking features, its the users fault... Maybe some users have just very different levels of requirements from you.

Refer: Obsidian Search, How I Use Embedded Queries, Dataview, Excalibrain

Ref: https://github.com/blacksmithgu/obsidian-dataview/issues/544 (Bearbeitet)

For using it, having SQL or JavaScript knowledge is useful, but you can probably figure it out without that knowledge. The Github page has a lot of examples that you can cannabalize for simple things without really getting too deep into it.

Check it here.

I think this could be helpful https://github.com/blacksmithgu/obsidian-dataview/issues/1086

Have you heard of Obsidian? It's a note-taking app build on locally stored markdown files with bidirectional linking and a great ecosystem of third party plugins. One of the most popular plugins is https://github.com/blacksmithgu/obsidian-dataview which lets you treat your notes as databases and query them to form tables. The creator has been working on its successor, Datacore https://github.com/blacksmithgu/datacore for a while - Datacore might come close to what you're looking for, its goals include WYSIWYG views and live editing inside tables.