visions
DataProfiler
Our great sponsors
visions | DataProfiler | |
---|---|---|
6 | 61 | |
194 | 1,362 | |
0.0% | 2.5% | |
0.0 | 6.3 | |
over 1 year ago | 3 days ago | |
Python | Python | |
GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
visions
-
Complete Beginner tasked with ML at work - where do I start
This one works pretty well: https://github.com/dylan-profiler/visions
- Visions – User defined data type systems
- Show HN: Visions – User defined data type systems
-
Show HN: DataProfiler – What's in your data? Extract schema, stats and entities
This is really cool! I'm glad there's more work going into the area - visions[1] is a similar tool which embeds the type system (user defined schema) into a traversable graph rather than encoding all type information in the trained network. We originally wrote it as the backend for pandas-profiling[2].
I'm not sure if any of the authors are on here but y'all might look into Sherlock[3] as well. They've got pre-trained models for many other semantic types than you've currently implemented.
1. https://github.com/dylan-profiler/visions
DataProfiler
-
LongRoPE: Extending LLM Context Window Beyond 2M Tokens
It's been possible to skip tokenization for a long time, my team and I did it here - https://github.com/capitalone/DataProfiler
For what it's worth, we actually were working with LSTMs with nearly a billion params back in 2016-2017 area. Transformers made it far more effective to train and execute, but ultimately LSTMs are able to achieve similar results, though slow & require more training data.
- Data Profiler – What's in your data?
-
Data Profiler 0.9.0 -- offering a massive improvement to memory usage during profiling of large datasets
Great call out -- would you be willing to write up an issue for that on the repo? Thank you! https://github.com/capitalone/DataProfiler/issues/new/choose
- FLiPN-FLaNK Stack Weekly for 20 March 2023
- Release 0.8.3 · capitalone/DataProfiler
What are some alternatives?
ydata-profiling - 1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
superset - Apache Superset is a Data Visualization and Data Exploration Platform
pyWhat - 🐸 Identify anything. pyWhat easily lets you identify emails, IP addresses, and more. Feed it a .pcap file or some text and it'll tell you what it is! 🧙♀️
scikit-learn - scikit-learn: machine learning in Python
usaddress - :us: a python library for parsing unstructured United States address strings into address components
Apache Superset - Apache Superset is a Data Visualization and Data Exploration Platform [Moved to: https://github.com/apache/superset]
XlsxWriter - A Python module for creating Excel XLSX files.
calculadora-do-cidadao - 💵 Tool for Brazilian Reais monetary adjustment/correction
datacompy - Pandas and Spark DataFrame comparison for humans and more!
vtuber-livechat-dataset - 📊 VTuber 1B: Billion-scale Live Chat and Moderation Event Dataset