SaaSHub helps you find the best software and product alternatives Learn more →
Top 12 Python Databrick Projects
-
Redash
Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
-
Scout Monitoring
Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
-
-
-
optscale
FinOps and cloud cost optimization tool. Supports AWS, Azure, GCP, Alibaba Cloud and Kubernetes.
-
dbx
🧱 Databricks CLI eXtensions - aka dbx is a CLI tool for development and advanced Databricks workflows management.
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
xonai-dashboard
A Grafana-based application to assist Big Data infrastructure optimization initiatives where Spark applications are a dominant cost driver
Project mention: Redash: Connect to data source, easily visualize, dashboard and share your data | news.ycombinator.com | 2024-03-20
Project mention: "[D]" Using data from Alpaca for a commercial version of a Open LLM | /r/MachineLearning | 2023-07-02
Project mention: The Future of MySQL is PostgreSQL: an extension for the MySQL wire protocol | news.ycombinator.com | 2024-04-26This is probably referring to "zero changes to your driver code" and not "zero changes to the SQL you send over this driver".
Translating between SQL dialects is notoriously hard and attempts to translate [1] are working in 95% of cases. But the last 5% would require 5x amount of work. That's because "SQL dialect" also includes weird edge cases of type inference of things like COALESCE(5, FALSE) and emulation of system catalogs (pg_catalog, information_schema).
[1] https://github.com/tobymao/sqlglot
One thing I wanted to add and call attention to is the importance of licensing in open models. This is often overlooked when we blindly accept the vague branding of models as “open”, but I am noticing that many open weight models are actually using encumbered proprietary licenses rather than standard open source licenses that are OSI approved (https://opensource.org/licenses). As an example, Databricks’s DBRX model has a proprietary license that forces adherence to their highly restrictive Acceptable Use Policy by referencing a live website hosting their AUP (https://github.com/databricks/dbrx/blob/main/LICENSE), which means as they change their AUP, you may be further restricted in the future. Meta’s Llama is similar (https://github.com/meta-llama/llama/blob/main/LICENSE ). I’m not sure who can depend on these models given this flaw.
Project mention: Profile and instrument ML experiments and optimize their performance expenses | news.ycombinator.com | 2023-09-27
To build custom deployment scripts, that go beyond declarative definitions, you are welcome to use https://github.com/databricks/databricks-sdk-py, https://github.com/databricks/databricks-sdk-jvm, and https://github.com/databricks/databricks-sdk-go.
Project mention: PyJaws: A Pythonic Way to Define Databricks JaWs (Jobs and Workflows) | /r/pythontips | 2023-09-25🐍 Install 𝐩𝐲𝐣𝐚𝐰𝐬 from PyPi: https://pypi.org/project/pyjaws/
Project mention: Show HN: Open sourcing a Big Data monitoring tool | news.ycombinator.com | 2024-03-29
Python Databricks discussion
Python Databricks related posts
-
Hello OLMo: A Open LLM
-
DBRX: A New Open LLM
-
Databricks SDK for Python
-
Official Python SDK for Databricks
-
How much object orienteered do you use in your projects? Bonus points for integration and unit tests
-
how/where do you define your databricks jobs, tasks and workflows?
-
Any suggestions for building DBT project on DataBricks?
-
A note from our sponsor - SaaSHub
www.saashub.com | 17 Jun 2024
Index
What are some of the best open-source Databrick projects in Python? This list will help you:
Project | Stars | |
---|---|---|
1 | Redash | 25,246 |
2 | dolly | 10,793 |
3 | sqlglot | 5,882 |
4 | dbrx | 2,455 |
5 | optscale | 1,047 |
6 | dbx | 435 |
7 | databricks-sdk-py | 311 |
8 | nutter | 265 |
9 | dbt-databricks | 193 |
10 | pyjaws | 39 |
11 | xonai-dashboard | 11 |
12 | fastdbfs | 4 |