Apache Spark VS Apache Cassandra

Recently I had to revisit the "JVM languages universe" again. Yes, language(s), plural! Java isn't the only language that uses the JVM. I previously used Scala, which is a JVM language, to use Apache Spark for Data Engineering workloads, but this is for another post 😉.

Consume data into third party software (then let Open Search or Apache Spark or Apache Pinot) for analysis/datascience, GIS systems (so you can put reports on a map) or any ticket management system

also, this knowledge applies to learning more about data engineering, as this field of software engineering relies heavily on the event-driven approach via tools like Spark, Flink, Kafka, etc.

Apache SeaTunnel is a data integration platform that offers the three pillars of data pipelines: sources, transforms, and sinks. It offers an abstract API over three possible engines: the Zeta engine from SeaTunnel or a wrapper around Apache Spark or Apache Flink. Be careful, as each engine comes with its own set of features.

Apache Spark - https://spark.apache.org/

A JVM based framework named "Spark", when https://spark.apache.org exists?

I recently saw a pull request that was merged to the Apache/Spark repository that apparently adds initial Python bindings for PySpark on K8s. I posted a comment to the PR asking a question about how to use spark-on-k8s in a Python Jupyter notebook, and was told to ask my question here.

Apache Cassandra{:target="_blank"} is a highly scalable and fault-tolerant database that can handle large volumes of data across multiple nodes or clusters. It provides fast read and write operations, making it suitable for real-time analytics or applications with high throughput requirements.

Wide Column: Apache Cassandra, ScyllaDB and DynamoDB

Cassandra, a replicated, fault-tolerant, decentralized and scalable database system.

A popular database you might already be familiar with is Apache Cassandra®, which powers high-performing applications for thousands of companies including Hulu, Netflix, Spotify, and Apple. While this free, open-source database is known for its high availability, scalability, and resilience; the downside is that it’s also notoriously complex to set up and manage.

In our previous article, we looked at augmenting our dashboard to store and display individual events from the Avengers using DataStax Astra, a serverless DBaaS powered by Apache Cassandra using Stargate to offer additional APIs for working with it.

Example: Apache Cassandra, CouchDB.

For our TikTok database, we’re using DataStax Astra DB: a cloud-based database that fully manages Apache Cassandra®, one of the most robust and scalable NoSQL databases around.

Data partitioning in Apache Cassandra.