flintrock
PySpark-Boilerplate
Our great sponsors
flintrock | PySpark-Boilerplate | |
---|---|---|
1 | 1 | |
630 | 390 | |
- | - | |
5.0 | 2.5 | |
16 days ago | 3 months ago | |
Python | Python | |
Apache License 2.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
flintrock
-
Why Databricks Is Winning
> * AWS has a managed Spark offering called EMR
There is also my rinky-dink open source project, Flintrock [0], that will launch open source Spark clusters on AWS for you.
It's probably not the right tool for production use (and you would be right to wonder why Flintrock exists when we have EMR [1]), but I know of several companies that have used Flintrock at one point or other in production at large scale (like, 400+ node clusters).
[0]: https://github.com/nchammas/flintrock
[1]: https://github.com/nchammas/flintrock#why-build-flintrock-wh...
PySpark-Boilerplate
What are some alternatives?
databricks-nutter-repos-demo - Demo of using the Nutter for testing of Databricks notebooks in the CI/CD pipeline
ibis - the portable Python dataframe library
spark-snowflake - Snowflake Data Source for Apache Spark.
Optimus - :truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
dask-gateway - A multi-tenant server for securely deploying and managing Dask clusters.
cookiecutter-django - Cookiecutter Django is a framework for jumpstarting production-ready Django projects quickly.
chispa - PySpark test helper methods with beautiful error messages
etl-markup-toolkit - ETL Markup Toolkit is a spark-native tool for expressing ETL transformations as configuration