potygen VS jOOQ

OMG this is one of my favorite tools paid for it all the way back when it went out. Have used it so many times just to write documentation for things like:

https://github.com/ivank/potygen/blob/main/packages/potygen/...

ASCII is just so versatile and allows you to put nice graphics in places where one does not expect, making things more easily understandable.

I also wrote a parser (in typescript) for postgres (https://github.com/ivank/potygen), and it turned out quite the educational experience - Learned _a lot_ about the intricacies of SQL, and how to build parsers in general.

Turned out in webdev there are a lot of instances where you actually want a parser - legacy places where they used to save things in plane text for example, and I started seeing the pattern everywhere.

Where I would have reached for some monstrosity of a regex to solve this, now I just whip out a recursive decent parser and call it a day, takes surprisingly small amount of code! (https://github.com/dmaevsky/rd-parse)

I used to agree 100% with this sentiment, as dissatisfaction with available ORMs at the time (early days of doctrine in PHP) drove me to actually write my own. Turned out an amazing exercise in why orms are hard.

Anyway a few years later I was in a position to start things fresh with a new project so thought to myself, great lets try to do things right this time - so went all the way in the other direction - raw sql everywhere, with some great sql analyzer lib (https://github.com/ivank/potygen) that would strictly type and format with prettier all the queries - kinda plugged all the possible disadvantages of raw query usage and was a breeze to work with … for me.

What I learned was that ORMs have other purposes - they kinda force you to think about the data model (even if giving you fewer tools to do so) With the amount of docs and tutorials out there it allows even junior members of the team to feel confident about building the system. I’m pretty used to sql, and thinking in it and its abstractions is easy for me, but its a skill a lot of modern devs have not acquired with all of our document dbs and orms so it was really hard on them to switch from thinking in objects and the few ways orms allows you to link them, to thinking in tables and the vast amounts of operations and dependencies you can build with them. Indexable json fields, views, CTEs, window functions all that on top of the usual relation theory … it was quite a lot to learn.

And the thing is while you can solve a lot of problems with raw sql, orms usually have plugins and extensions that solve common problems, things like soft delete, i18n, logs and audit, etc. Its easy even if its far from simple. With raw sql you have to deal with all that yourself, and while it can be done and done cleanly, still require intuition about performance characteristics that a lot of new devs just don’t possess yet. You need to be an sql expert to solve those in a reasonable manner m, just an average dev could easily string along a few plugins and call it a day. Would it have great performance? Probably not. Would it hold some future pitfalls because they did not understand the underlying sql? Absolutely! But hay it will work, at least for a while. And to be fair they would easily do those mistakes with raw sql as well, but with far few resources to understand why it would fail, because orms fail in predictable ways and there is usually tons of relevant blog posts and such about how to fix it.

It just allows for an better learning curve - learn a bit, build, fail, learn more, fix, repeat. Whereas raw sql requires a big upfront “learn” cost, while still going through the “fail” step more often than not.

Now I’m trying out a fp query builder / ORM - elixir’s ecto with the hopes that it gives me the best of both worlds … time will tell.

> 2. ORMs do not hide SQL nastiness.

This is certainly true!

I mean: ORMs are now well known to "make the easy queries slightly more easy, while making intermediate queries really hard and complex queries impossible".

I think the are of ORMs is over. It simply did not deliver.

If a book on SQL is --say-- 100 pages, a book on Hibernate is 400 pages. So much to learn just to make the easy queries slightly easier to type? Just not worth it.

I prefer jooq any day over ORMs. And dont get me started over what tools like Hasuna have to offer.

There are also some languages (forgot the names) that are SQL-done-right. Select in the back, more type safe, more logic, more in the same steps as the query gets executed. These need to be adopted by PG and MySQL and we're good to go. (IMHO)

https://www.jooq.org/

https://hasura.io/

See: https://github.com/jOOQ/jOOQ/issues/5916

I've worked on a few systems that were stored procedure heavy with Microsoft SQL server, not so much because I am a .NET guy, some of these came when I was working on super-random projects for a consulting company.

My take is that it is a lot like building a system that has an API over the database except that instead of writing in API in, say, Java and exposing it through an http API with, say, JAX-RS, you are writing the API in stored procedures and accessing it through JDBC or ODBC or the native API of the database.

It seems very possible to build some thin layer that uses metadata to make an http API over stored procedures.

I'd say that systems like that can work very well.

The basic challenge is maintaining version control over your scripts, my coworkers were rubyists on my first such project and built a system inspired by migrations in Ruby on Rails where we wrote up and down migration scripts for every database change. I carried that approach to other projects where the people had less discipline to begin with. There is a little awkwardness there that the "down" script that reverts a procedure to a previous version has a cut-and-pasted copy of the old version of the stored procedure, if I had to do it over again I'd make something where each version of a stored proc is in a numbered file and the "migrations" just say "upgrade ABC proc to version 7" or maybe you could make something that looks into the VCS and finds the old version.

From what I've read, PL/SQL from Oracle looks a lot better than the Transact-SQL language in SQL server but I've never done a major project with Oracle. Most places I've worked at recently use PostgreSQL and I think this would be a viable architecture for that.

One area where it seems to be a hassle is with the "query builder" pattern, for instance where I work now we have a very complex search form with a huge number of options that builds a SQL query. I think you can do that kind of thing with what they call "Dynamic SQL", see

https://www.postgresql.org/docs/15/plpgsql-statements.html#P...

but it seems preferable to do that kind of thing with a real programming language, particularly if you have tools like

https://www.jooq.org/

I have a bachelor's in computer science, took a databases class in college (which I did poorly in), and worked as a backend developer for two years, but I always struggled with SQL. I can do the basic SELECT * FROM table WHERE column = 1 but when the SQL statement gets long with lots of joins I couldn't understand it and relied on another programmer for help. When I need to build a website myself I end up going with MongoDB because that allows me to write code instead of writing SQL. That being said, instead of doing all the data processing stuff in the backend, I'd like to try doing as much of it as I can in the database and learn some programmimg language that is SQL-esque (for my next personal project). I know Scala and am very comfortable doing functional programming stuff like List(1,2,3).filter(_.isEven()) to get the even numbers in a list or writing List(1,2,3).reduce(_+_) to apply the addition operation on all the numbers in a list. I know the Big Data framework Apache Spark is written in and works with Scala, but I really want to learn something that works with a traditional database that runs on one centralized server and not have to worry about the distributed computing MapReduce paradigm stuff (also installing the Big Data ecosystem on my personal computer is a pain). Like I want to try building something with a traditional database like Oracle, SQL Server, etc. Something I find really helpful is having IDE code error highlighting and auto-completion and the ability to run a static analysis code quality checker tool, which I can do with Scala code, but I don't know of a way to have those things with traditional SQL Strings sent from the backend to the database. I know of things like Java's JOOQ or C#'s LINQ, but I don't want to use one of those, I want to use something in the database that is database specific and pushes as much data processing as possible into the database. I heard of languages like PL/SQL, T-SQL, and PL/pgSQL to add procedural control to SQL, making it like a real programming language. On Wikipedia I found this list of PL/SQL editors but there are so many choices and I don't know which one to pick (it has to not cost me money and I would love one that has auto-completion with error highlighting and suggestions or maybe some sort of graphical query builder tool that gives me choices of what to put to compose my statement). I saw someone on Reddit say that more recently they added the ability to add database triggers in various programming languages. I also heard mentions online of this thing called "dbt" which adds software engineering tools like version control to SQL, but I don't know if that can help me get around my difficulty with SQL or if it is something I would want to use. Any advice would be appreciated.

Spring Boot, ili ako bi nesto vise lightweight u Javi Spark + jOOQ

> I've been doing ORM on Java since Hibernate was new, and it has always sucked.

Have you ever looked at something like myBatis? In particular, the XML mappers: https://mybatis.org/mybatis-3/dynamic-sql.html

Looking back, I actually quite liked it - you had conditionals and ability to build queries dynamically (including snippets, doing loops etc.), while still writing mostly SQL with a bit of XML DSL around it, which didn't suck as much as one might imagine. The only problem was that there was still writing some boilerplate, which I wasn't the biggest fan of.

Hibernate always felt like walking across a bridge that might collapse at any moment (one eager fetch away from killing the performance, or having some obscure issue related to the entity mappings), however I liked tooling that let you point towards your database and get a local set of entities mapped automatically, even though codegen also used to have some issues occasionally (e.g. date types).

That said, there's also projects like jOOQ which had a more code centric approach, although I recall it being slightly awkward to use in practice: https://www.jooq.org/ (and the autocomplete killed the performance in some IDEs because of all the possible method signatures)

More recently, when working on a Java project, I opted for JDBI3, which felt reasonably close to what you're describing, at the expense of not being able to build dynamic queries as easily, as it was with myBatis: https://jdbi.org/

That said, with the multi-line string support we have in Java now, it was rather pleasant regardless: https://blog.kronis.dev/tutorials/2-4-pidgeot-a-system-for-m...

I don't think there's a silver bullet out there, everything from lightweight ORMs, to heavy ORMs like Hibernate, or even writing pure SQL has drawbacks. You just have to make the tradeoffs that will see you being successful in your particular project.

Have a look at jooq - I know this has been used to rewrite SQL from one dialect to another, so it MUST be capable of collating code activity metrics. Look here. Otherwise, you might want to look into writing your own parser. ANTLR has a T-SQL dialect parser script here.

Feature requests go here: https://github.com/jOOQ/jOOQ/issues/new/choose. Looking forward!

You may have the fortune of being familiar with the jOOQ library for Java/JVM apps, that can generate domain models based on your database schema, as well as methods on these classes to perform queries. In case you're not, here an example Postgres schema: