|2 months ago||over 4 years ago|
|Mozilla Public License 2.0||GNU General Public License v3.0 or later|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Rust and what it needs to gain space in computation-oriented applications
7 projects | reddit.com/r/rust | 24 Nov 2021
You should check out polars, datafusion, influxdb iox and databend, all written in native Rust and powered by the Apache Arrow format. Polars in particular is pretty dam fast and has bindings for Python.
Database-Like Ops Benchmark
1 project | news.ycombinator.com | 20 Nov 2021
A better dtypes for pandas dataframes pulled from Postgres
1 project | reddit.com/r/datascience | 14 Nov 2021
Here is a good comparison: https://h2oai.github.io/db-benchmark/
Introducing tidypolars - a Python data frame package with syntax familiar to R tidyverse users
4 projects | reddit.com/r/datascience | 10 Nov 2021
The biggest difference with this one is that it's built on top of the polars package, which is probably the fastest data frame manipulation library out there. All of the other dplyr-to-python packages are build on top of pandas (which is very slow in comparison).
Introducing tidypolars - a Python data frame package for R tidyverse users
9 projects | reddit.com/r/rstats | 10 Nov 2021
I think having a basic understanding of pandas, given how broadly it's used, is beneficial. That being said, polars seems to be matching or beating data.table in performance, so I think it'd be very worth it to take it up. Wes McKinney, creator of pandas, has been quite vocal about architecture flaws of pandas -- which is why he's been working on the Arrow project. polars is based on Arrow, so in principle it's kinda like pandas 2.0 (adopting the changes that Wes proposed).9 projects | reddit.com/r/rstats | 10 Nov 2021
tidypolars uses the polars package as a backend, which might be the fastest data frame manipulation library out there. (Faster even than R's data.table, which has been the king of speed for many years.)
Your perfect program/language for experience studies?
1 project | reddit.com/r/actuary | 4 Nov 2021
Julia has ExperienceStudies.jl to help with exposure calculations and MortalityTables.jl for mortality rate data. It also performs very well in data science benchmarks: https://h2oai.github.io/db-benchmark/
Comparing SQLite, DuckDB and Arrow
5 projects | news.ycombinator.com | 27 Oct 2021
this benchmark is more comprehensive for this type of analytical work:
1 project | reddit.com/r/datascience | 23 Oct 2021
Data too big to work with memory you can do in R too, using SparkR. I agree the documentation to something like PySpark is better though. For data within memory, data.table in R beats pandas. Loses to Polars (implemented in Rust that has bindings in Python) but that is not in use much as its new: https://github.com/h2oai/db-benchmark.
Turning database into a searchable dashboard?
3 projects | reddit.com/r/datascience | 21 Oct 2021
How to start a war
2 projects | reddit.com/r/ProgrammerHumor | 30 Sep 2021
Why does Julia adopt 1-based index?
3 projects | reddit.com/r/Julia | 10 Sep 2021
some may hate it, some may love it
5 projects | reddit.com/r/Julia | 27 Jun 2021
The one and only indexing is https://github.com/simonster/TwoBasedIndexing.jl
Why not Julia?
11 projects | reddit.com/r/Julia | 1 May 2021
Again on 0-based vs. 1-based indexing
5 projects | news.ycombinator.com | 19 Jan 2021
Do not add any more fuel to the flame and instead use 2-based indexing: https://github.com/simonster/TwoBasedIndexing.jl
Seriously, the exact value of the lower bound for indexing doesn't matter here (some algorithms are best described with the lower bound other than 0 or 1, for example). The fixed or preferred lower bound for indexing is the real problem. Any argument for/against 0-based and 1-based indexing tends to gloss over the real problem because those arguments only exist to make some languages look better than other languages. As we move away from forced explicit indexing (e.g. arr.first() or foreach instead of arr[$LBOUND] or `for (i=$LBOUND; ...)`), it becomes clear that there is no such thing like the preferred lower bound for sequences.
What are some alternatives?
arrow-datafusion - Apache Arrow DataFusion and Ballista query engines
polars - Fast multi-threaded DataFrame library in Rust and Python
DataFramesMeta.jl - Metaprogramming tools for DataFrames
csvs-to-sqlite - Convert CSV files into a SQLite database
sktime - A unified framework for machine learning with time series
Preql - An interpreted relational query language that compiles to SQL.
csvzip - A standalone CLI tool to reduce CSVs size by converting categorical columns in a list of unique integers.
arrow-rs - Official Rust implementation of Apache Arrow
databend - An elastic and reliable Serverless Data Warehouse, offers Blazing Fast Query and combines Elasticity, Simplicity, Low cost of the Cloud, built to make the Data Cloud easy
skorch - A scikit-learn compatible neural network library that wraps PyTorch
scientific-visualization-book - An open access book on scientific visualization using python and matplotlib