tidytable VS polars

If you need speed, you should consider using dtplyr (or tidytable), or even dbplyr with duckdb.

What do you think of this instead?

Here's an example - I use this in a package I've built called tidytable. Here is the as_tidytable() function I use that uses method dispatch.

If you're having performance issues with dplyr you can also try out tidytable

I’d say tidytable is a better option these days as it supports more functions. Although I think dtplyr has improved on this front recently, but still lags. The author of tidytable contributes to dtplyr as well.

I know you said it 'feels much faster' which isn't exactly a data oriented comparison, but tidymodels performs very well. You can see one of the dplyr functions as step_* in tidymodels, for example mutate vs. step_mutate under recipes library. The author of tidytable, which uses data.table, had some revisions due to this conversation, just as an example.

Tidytable is what you might be looking for: https://markfairbanks.github.io/tidytable/, this will require a bit of refactoring (e.g group-bys happen as arguments in summarise/mutate). You'll get data.table like speed in a very compact & complete package.

People can also use tidytable and keep the same workflow they're already used to 😄

If you want to get into Polars, the library is very well documented, and I’d recommend you check out their getting started tutorial, their API docs, and when you’re all set up, you can also check out their Cookbooks to learn about many of the standard operations within Polars.

This is false. The polars api has used smart string for a long time.

https://github.com/pola-rs/polars/blob/32a2325b55f9bce81d019...

This is because 0.1 is in actuality the floating point value value 0.1000000000000000055511151231257827021181583404541015625, and thus 1 divided by it is ever so slightly smaller than 10. Nevertheless, fpround(1 / fpround(1 / 10)) = 10 exactly.

I found out about this recently because in Polars I defined a // b for floats to be (a / b).floor(), which does return 10 for this computation. Since Python's correctly-rounded division is rather expensive, I chose to stick to this (more context: https://github.com/pola-rs/polars/issues/14596#issuecomment-...).

https://github.com/pola-rs/polars/releases/tag/py-0.19.0

That turned out to be related to pola-rs/polars#11912, and this linked comment provided a deceptively simple solution - use PARSE_DECLTYPES when creating the connection: