siuba VS dtplyr

I don't know what's best for you, but I can recommend Siuba, a tidy interface for Python to send queries to pandas and SQL-db.

You mean siuba?

For further inspiration, this is a pretty good-looking "dplyr for Python": https://github.com/machow/siuba

You should follow /u/the75th's advice. However, if you decide to buck that take, I'd look into siuba. I've never heard of those packages you've listed, and have doubts they'd be maintained.

Check out siuba

Can’t say I’ve used it, but isn’t that what dtplyr is supposed to provide?

https://dtplyr.tidyverse.org/

# I am trying to follow the example at https://dtplyr.tidyverse.org/

FYI the developer of tidytable has been developing dtplyr for the Tidyverse. You might like that too!

If you need speed, you should consider using dtplyr (or tidytable), or even dbplyr with duckdb.

https://dtplyr.tidyverse.org/ ?

If you want data.table speed, but using dplyr/tidy then dtplyr is a good package to have handy. Personally I love R, and choose R + NodeJS as my gotos for everything I do, and use Python only when I have to.

Also, that dataset is quite big, and the "base" Tidyverse will be excessively slow. You should supplement the "base" Tidyverse packages (i.e. dplyr and tidyr) with either dtplyr or dbplyr (+ duckDB). I'd suggest starting with dtplyr, which should handle 10M+ rows fine.

If you want, there's a "substitute" for dplyr called dtplyr (also part of the Tidyverse), which "translates" your dplyr/tidyr code into data.table behind the scenes, and allows you to make your modifications apply directly to the original dataset by default:

Install the dtplyr package and change your code to:

Get the best of both worlds (tidyverse + data.tables) with dtplyr, a data.table backend for dplyr.