SynapseML
Simple and Distributed Machine Learning (by microsoft)
spark-excel
A Spark plugin for reading and writing Excel files (by crealytics)
Our great sponsors
| SynapseML | spark-excel | |
|---|---|---|
| 18 | 8 | |
| 4,964 | 433 | |
| 0.5% | 2.1% | |
| 8.9 | 8.6 | |
| 2 days ago | 8 days ago | |
| Scala | Scala | |
| MIT License | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SynapseML
Posts with mentions or reviews of SynapseML.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-09-12.
- FLaNK Stack Weekly for 12 September 2023
-
Microsoft announces new tool for applying ChatGPT and GPT-4 at massive scales
Release Notes: https://github.com/microsoft/SynapseML/releases/tag/v0.11.0
-
Data science in Scala
b) There are libraries around e.g. Microsoft SynapseML, LinkedIn Photon ML
- [N] Microsoft Announces New Integrations with OpenAI and MLFlow
- [N] Microsoft Releases new Integrations with OpenAI and MLflow as part of SynapseML
-
[P] Microsoft releases SynapseML v0.9.5 with support for speech synthesis, anomaly detection, and geospatial analytics on large-scale data
Link to Release Notes: https://github.com/microsoft/SynapseML/releases/tag/v0.9.5
- Microsoft releases SynapseML v0.9.5 for distributed geospatial analytics, speech synthesis, and anomaly detection in PySpark.
- [P] SynapseML v0.9.5 announces support for geospatial analytics, speech synthesis, and anomaly detection on large-scale datasets
- Microsoft releases SynapseML v0.9.5 with support for speech synthesis, anomaly detection, and geospatial analytics on Apache Spark
spark-excel
Posts with mentions or reviews of spark-excel.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-06-17.
- Pandas was faster and less memory intensive then crealytics pyspark. How is it possible?
-
Automating Excel to Databricks Table
Not natively. But the com.crealytics.spark.excel library has had great results for us. There are still some cases where pandas manipulation is needed with Excel files that have weird header setups.
-
Can AWS Glue convert a JSON payload to Excel tab? (not csv)
Spark 3.2 has a to_excel() method, but not Spark 3.1, so you'll need to use an external library such as https://github.com/crealytics/spark-excel
-
Reading a xlsx file with PySpark
Have you checked spark-excel's documentation? The dataAddress option seems to be what you're looking for.
-
read percentage values in spark ( no casting )
Are you using this library to load xlsx files? https://github.com/crealytics/spark-excel
-
Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot modify the value of a Spark config: spark.executor.memory;
I found similiar issues on their github: https://github.com/crealytics/spark-excel/issues/227
-
How do I learn to read a plug-in?
Plug-in in question is GitHub - crealytics/spark-excel: A Spark plugin for reading Excel files via Apache POI , but I guess it could be any. Assuming that I can read the plain code in an individual .scala file how do I learn to understand how it all pieces together and what the underlying code being run is?
What are some alternatives?
When comparing SynapseML and spark-excel you can also consider the following projects:
mmlspark - Simple and Distributed Machine Learning [Moved to: https://github.com/microsoft/SynapseML]
frameless - Expressive types for Spark.
isolation-forest - A Spark/Scala implementation of the isolation forest unsupervised outlier detection algorithm.
Quill - Compile-time Language Integrated Queries for Scala
deequ - Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
Tensorflow_scala - TensorFlow API for the Scala Programming Language
Apache Spark - Apache Spark - A unified analytics engine for large-scale data processing
Breeze - Breeze is a numerical processing library for Scala.
cobrix - A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
azure-kusto-spark - Apache Spark Connector for Azure Kusto
metorikku - A simplified, lightweight ETL Framework based on Apache Spark