ci-cd-serverless-spark
Demo for GitHub Universe 2022 (by dacort)
spark-local-environment
An example of using EMR Serverless container image for local environment (by dacort)
| ci-cd-serverless-spark | spark-local-environment | |
|---|---|---|
| 2 | 1 | |
| 11 | 2 | |
| - | - | |
| 10.0 | 2.2 | |
| over 1 year ago | 9 months ago | |
| Python | Dockerfile | |
| - | Creative Commons Zero v1.0 Universal |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ci-cd-serverless-spark
Posts with mentions or reviews of ci-cd-serverless-spark.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-03-14.
-
CI/CD for AWS Glue jobs ?
My second example is more recent but is for EMR Serverless. It might still be useful, though - it uses GitHub Actions to build/deploy versioned artifacts to s3. The full code is in this ci-cd-serverless-spark repo.
-
End to End Pyspark Testing CI/CD Example Repo
I’ve also got a similar repo for full CI/CD with EMR Serverless.
spark-local-environment
Posts with mentions or reviews of spark-local-environment.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-02-05.
-
End to End Pyspark Testing CI/CD Example Repo
Sweet, sounds good! Just posted a repo with the changes I made to run EMRS image locally as well. https://github.com/dacort/spark-local-environment
What are some alternatives?
When comparing ci-cd-serverless-spark and spark-local-environment you can also consider the following projects:
athena-glue-service-logs - Glue scripts for converting AWS Service Logs for use in Athena
pyspark-testing-env - Example Repo to have full end to end pyspark testing via docker-compose