soda-spark VS TypedPyspark

Compare soda-spark vs TypedPyspark and see what are their differences.

soda-spark

Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes (by sodadata)
Spark Pyspark data-engineering data-quality data-observability data-testing soda-sql Python
Source Code
docs.soda.io
Suggest alternative
Edit details

TypedPyspark

Type-annotate your spark dataframes and validate them (by getyourguide)
Python Spark Pyspark Typing
Source Code
Suggest alternative
Edit details
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
soda-spark TypedPyspark
1 1
60 14
- -
0.0 2.4
almost 2 years ago 8 months ago
Python Python
Apache License 2.0 Apache License 2.0
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.

soda-spark

Posts with mentions or reviews of soda-spark. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-01-23.
  • How do you test your pipelines?
    3 projects | /r/dataengineering | 23 Jan 2022
    Since you already have Spark setup, perhaps it would be easier to build a DataFrames by loading data from different tables and validate it in one go ? You can give soda-spark a try (disclosure: I'm one of the developers), using which you can specify your checks using YAML declaratively and run the validations in spark jobs.

TypedPyspark

Posts with mentions or reviews of TypedPyspark. We have used some of these posts to build our list of alternatives and similar projects.

What are some alternatives?

When comparing soda-spark and TypedPyspark you can also consider the following projects:

great_expectations - Always know what to expect from your data.

pyspark-starter - Starter pyspark code with a working combination of all versions

pyspark-example-project - Implementing best practices for PySpark ETL jobs and applications.

pyspark-on-aws-emr - The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on writing pyspark code.

monosi - Open source data observability platform

Traffic-Data-Analysis-with-Apache-Spark-Based-on-Mobile-Robot-Data - Mobile robot data were analyzed with Apache-Spark to extract five different statistical result such as travel time, waiting time, average speed, occupancy and density were produced.

PySpark-Boilerplate - A boilerplate for writing PySpark Jobs

soda-core - :zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io

ibis - the portable Python dataframe library

data-caterer - Data generation and validation tool for any data source

Optimus - :truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

soda-spark vs great_expectations TypedPyspark vs pyspark-starter soda-spark vs pyspark-example-project TypedPyspark vs pyspark-on-aws-emr soda-spark vs monosi TypedPyspark vs Traffic-Data-Analysis-with-Apache-Spark-Based-on-Mobile-Robot-Data soda-spark vs PySpark-Boilerplate TypedPyspark vs pyspark-example-project soda-spark vs soda-core TypedPyspark vs ibis soda-spark vs data-caterer TypedPyspark vs Optimus