synth VS Metabase

SEEKING FREELANCER | London | Remote

Synth (YC S20) [1] is an open source declarative data generator written 100% in Rust.

We are looking for someone with prior experience writing Rust in production for a 1-to-3 months contract to work with us on our core open-source project.

- Proven experience writing production Rust code, preferably in a large code base

- Knowledge of PostgreSQL at a level sufficient to design and build reliable integration

- Strong knowledge of data structures and algorithms

- Track record of contribution to open-source projects, preferably on GitHub

- Ability to work quickly and rigorously in a fully remote setting

If that sounds interesting, we want to talk to you! Shoot me an email at damien [at] getsynth.com!

[1]: https://github.com/getsynth/synth

Synth | Rust Software Engineer | Full Time or Part Time | London | Onsite(London)/Remote

About us: Synth is an open source declarative data generator (https://github.com/getsynth/synth). We are building Synth with the intention of solving, once and for all, the problem of generating realistic data for testing - helping big companies and small developers avoid the use of production data in testing.

Our mission is to build amazing developer tools that solve data privacy without forcing users to compromise on productivity. We have a few exciting products in our pipeline and we're backed by YCombinator and other great investors. We're based in London and building a remote-friendly culture.

We work exclusively on open source software. This is great because our community is not confined to just our core team and the users, but also includes our contributors - we believe it is way more fun this way.

We're using Rust for our main line of products - and what we would like to see ideally is:

* You have some experience with Rust that has connected you with at least one of: asynchronous I/O, meta-programming or common patterns for concurrency. Having been involved in an open-source Rust project is a bonus!

Take a look at this rust library (which works very well with python modules which generate data in certain formats): https://github.com/getsynth/synth

Putting the finishing touches on a procedural macro to bind Rust code to koto we want to use in synth. Also a blog post about it is on the way.

I'm working on synth https://github.com/getsynth/synth . Also working on a personal project, implementing the tcp protocol in Rust for the fun of it.

We use Rust to build synth, the open source declarative data generator.

Hey HN! We're Synth - a bunch of engineers out of Europe building tooling for developers. We're very excited about what we're working on and wanted to share it with the community.

We've been quite frustrated with the status quo of test data generation - after speaking to tons of other devs we've realised that many people are struggling when it comes to generating realistic looking test data.

Also, where people don’t want to copy sensitive production data to testing environments, data obfuscation can be a huge time-sink.

Enter Synth: a declarative data generator (see our website: https://getsynth.com/, github: https://github.com/getsynth/synth)

Synth enables devs and dev teams to have their application data models as code (basically a hierarchy of files) in their repos. These files can then be used to generate data for a local dev environment, automated testing in CI or even for sharing across organisations. The parameters of generation can also be tweaked to push the data model to its limits for QA, and even scaled for load testing / performance testing.

We're now working on taking the next step, and building a DSL around Synth. The Synth DSL will enable users to concisely define what data should look like and get going.

We're open source and written 100% in Rust. We believe that by making test data be as easy as using production data, we can improve the security and privacy for all of us. We'd love to get more early users as the initial feedback is positive but limited.

Thank you and looking forward to any feedback / ideas about how we can build a better tool for you!

P.S. Synth [launched on HN a while back](https://news.ycombinator.com/item?id=24198114) as an ML solution to create realistic (and safe) copies of your sensitive production data as a service. This approach quickly hit several limitations which couldn't address the use-cases we are trying to solve, happy to go into more details on this if anyone is interested.

I'm currently trying to improve the vtable dispatch in koto (because I want to use it in synth).

Luckily, not all Rust jobs are crypto jobs. I'm in my third Rust job working on synth right now and am 100% happy with it.

Remote Code Execution via H2

We've used it for about a year - Blazer is okay if you need a quick SQL query console, but we found it lacking as an actual business intelligence tool. The support for graphs and dashboards is limited, for graphs it requires you to structure the query in an exact way as you can see in the Blazer readme.

After some research on available alternatives that don't break the bank, we decided to deploy a self-hosted instance of Metabase[0]. This took only a few minutes to set up using their Docker image[1] and it has much better graphing capabilities and you can easily put a custom layout together for dashboards. Upgrading is similarly easy (just redeploy). Also easy to configure: data sources, hiding or changing the data type of a column, G Suite sign-in for our domain. Highly recommend it if you need anything more than Blazer's table output.

[0]: https://github.com/metabase/metabase

I've never used Tableau, but heard a lot of hate about it. However, in my previous role, we were big fans of Metabase (https://metabase.com). You can also self-host it, which was a huge win for us.

It seems folks want a working example. Here's one in prod:

Metabase is a BI tool, backend written mostly in Clojure. Like basically all BI tools they have this intermediate representation language thing so you write the same thing in "MBQL (metabase query language)" and it theoretically becomes same query in like, Postgres and Mongo and whatever. End user does not usually write MBQL, it's a service for the frontend querybuilding UI thing and lots of other frontend UI stuff mainly in usage.

Whole processing from MBQL -> your SQL or whatever is done via a buncha big-ass transducers. Metabase is not materially faster than other BI tools (because all the other BI tools do something vaguely similar in their langs) but it's pretty comparable speed and the whole thing was materially written by like 5 peeps

https://github.com/metabase/metabase/blob/master/src/metabas...

(nb: I used to work for Metabase but currently do not. but open core is open core)

They haven't released the source, and the compiled versions are non-trivial to diff (e.g. there are nondeterministic numbers from the clojure compiler that seem to have changed from one to the other, and .clj files have been removed from the jar).

The old version has `hash=1bb88f5`, which is a public commit: https://github.com/metabase/metabase/commit/1bb88f5

We are unsure about the right license to use, so this is a great feedback. We had a MIT license one week ago that we know that we cannot hold on long term and we felt we were lying to the community by keeping an MIT license and changing it in one year.

By using AGPL, we feel it's the right level of restriction. It's the license used by Metabase for example (https://github.com/metabase/metabase) that many companies use internally.

The solution really depends on what sort of problems you are trying to solve and who your customers are.

There are a fair few low-code solutions out there for reporting and data visualisation that are great for finance and marketing teams for example. e.g. https://metabase.com/ , https://evidence.dev/

For multipurpose SMB workflows and organisational processes, I have used n8n in the recent past and found it was quite good and incredibly easy to maintain. https://n8n.io/engineering-resources/

For enterprise processes I'd go with Camunda (solely based on recommendations and not first hand experience). Although only parts of their platform are OSS https://github.com/camunda

Bear in mind that some of these are not suitable if you want to build something that competes with them while taking their OSS code. But are perfectly fine otherwise.

Anyway, I have a collection of scripts that merge ActivityWatch data from all my machines and WakaTime exports to a PostgreSQL database which I then query with a project called Metabase. If you're curious, the scripts are in a repository called sqrt-data. I've been playing with this for ~4-5 years already I think.

Metabase | https://metabase.com | REMOTE | Full-time | Backend, Frontend, Full Stack, and DevOps engineers

Metabase is open source analytics software that lets anyone in your company rummage around in the databases you have. It connects to a number of databases / data warehouses (BigQuery, Redshift, Snowflake, Postgres, MySQL, etc).