os-lib
OS-Lib is a simple, flexible, high-performance Scala interface to common OS filesystem and subprocess APIs (by com-lihaoyi)
mack
Delta Lake helper methods in PySpark (by MrPowers)
Our great sponsors
| os-lib | mack | |
|---|---|---|
| 4 | 5 | |
| 640 | 269 | |
| 1.1% | - | |
| 7.5 | 5.9 | |
| 1 day ago | 3 months ago | |
| Scala | Python | |
| GNU General Public License v3.0 or later | MIT License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
os-lib
Posts with mentions or reviews of os-lib.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-11-27.
- Engenharia de Dados com Scala: aprenda a fazer webscraping dos filmes mais assistidos da Netflix em cada país
-
Scala CLI v1.0.0 is out!
Then something like os-lib can probably cover your scripting needs.
-
Spark open source community is awesome
another dev is working on adding an elegant interface to perform Hadoop filesystem operations, similar to os-lib for regular filesystem operations
-
What learning path did you follow into Scala?
In particular, the ammonite REPL and os-lib are absolutely ace.
mack
Posts with mentions or reviews of mack.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-03-05.
-
Implementing and using SCD Type 2
There still library form databricks? But I have never used it: https://github.com/MrPowers/mack
-
Spark/databricks seems amazing?
I was a Databricks user for 5 years and spent almost all my time inside the IntelliJ IDE developing code. I wrote almost all code in a text editor, unit tested all code (actually authored the popular Scala Spark / PySpark testing libraries: https://github.com/MrPowers/) and had everything up with CI/CD. Lots of OSS PySpark/Scala Spark work too. I only used Databricks notebooks for data exploration and for lightweight notebooks that would invoke functions (that were defined in Python Wheel / JAR files). I am on the Delta Lake team at Databricks now and still do all my work in text editors (see this project: https://github.com/MrPowers/mack) and create lots of examples in Jupyter Notebooks. So I definitely think it's possible to limit notebook exposure.
-
PySpark OSS Contribution Opportunity
Great, would love your help. You can also check out the mack project if you'd like to work on a Delta Lake + PySpark project: https://github.com/MrPowers/mack/issues
-
Spark open source community is awesome
a couple devs just added a `find_compositite_keys_candidates` function so users can easily identify columns that could be used as a unique identifier in their Delta table.
-
How to append data to Delta tables without adding any duplicates
Fair points. Here's the code repo: https://github.com/MrPowers/mack
What are some alternatives?
When comparing os-lib and mack you can also consider the following projects:
Eso - A (mostly) purely functional console-based esoteric language interpreter.
chispa - PySpark test helper methods with beautiful error messages
pfps-shopping-cart - :shopping_cart: The Shopping Cart application developed in the book "Practical FP in Scala: A hands-on approach"
delta-rs - A native Rust library for Delta Lake, with bindings into Python
scala-cli - Scala CLI is a command-line tool to interact with the Scala language. It lets you compile, run, test, and package your Scala code (and more!)
jodie - Delta lake and filesystem helper methods