quinn VS lowdefy

quinn is a library with PySpark helper functions. I need to work through all the open issues / PRs and bump all versions. I should do another release. This library gets around 600,000 monthly downloads.

Pandas syntax is far inferior to regular PySpark in my opinion. Goes to show how much data analysts value a syntax that they're already familiar with. Pandas syntax makes it harder to reason about queries, abstract DataFrame transformations, etc. I've authored some popular PySpark libraries like quinn and chispa and am not excited to add Pandas syntax support, haha.

They spent a lot of effort improving the catalyst engine under the hood too and making it easier to extend and improve it in the future. Making it easy to add your own native code to Spark itself. Shameless plug of a blog post I wrote on this subject which basically reiterates what Matthew Powers, author of Spark Daria and quinn, wrote here.

I built daria (https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.

quinn (https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa) are the PySpark equivalents.

Built bebe (https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.

Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8

I've built popular PySpark (quinn, chispa) and Scala Spark (spark-daria, spark-fast-tests) libraries.

Check out https://lowdefy.com/ they even have a sample survey app as one of their examples.

Lowdefy is a low code tool that uses AG Grid as a block component, allowing you to create apps which render data in AG Grid without a lot of coding knowledge. There is a Lowdefy example using AG Grid here.

Shameless plug, also check out Lowdefy - https://github.com/lowdefy/lowdefy

I’m also going to jump in here and say try Lowdefy https://github.com/lowdefy/lowdefy - co-founder here.

We take a different angle and believe that low code should still work like code. We focus on a developer first approach.

We’re building lowdefy: https://github.com/lowdefy/lowdefy

A low-code tool that let’s you build web apps using yaml. In v4, you config is run in a lightweight nextjs app, so should scale as well as nextjs for the most part.

A lot of the issues that people are talking about in this thread have been address, mostly because with lowdefy we’ve focused on making the schema easy to read, write and understand. Currently we write all our apps by hard coding the yaml, and it’s scaling really well in our team.

For writing apps in yaml, you get advantages like:

I don't know of a solution for ETL DSL -> UI but https://github.com/lowdefy/lowdefy goes from a DSL to internal web apps. Perhaps some inspiration could be taken from there. Good luck!

This is very helpful! Thanks for sharing. We’re planning a few marketing sprints for Lowdefy [0] and resources like this is really a timesaver. If someone has any other useful sources, please share.

[0] - https://github.com/lowdefy/lowdefy

PS: If anyone would like to make a contribution to an open-source project, we would be happy to help you out. https://github.com/lowdefy/lowdefy

Check out https://github.com/lowdefy/lowdefy which is open-source.

We created Lowdefy (co-founder here) to built web apps fast and it is really working well for us and our customers. We’ve already rolled out a few enterprise level apps in record time and the platform really scales well as apps evolve and get really complicated. Even developing apps as a team is a breeze since you are working with yaml / json which is easy to code review, copy, paste, find and replace.

l