box VS tidytable

As for organising code within a project, as mentioned packages really don’t allow this beyond collation order. The best solution in this space is the ‘box’ package which implements a fully-featured module system for R. ‘box’ notably gets used by some folks to implement large-scale Shiny applications; if this is what you’re after, I would recommend the ‘rhino’ framework, which builds upon Siny and ‘box’.

You can write amazing, clean, modular code with the box package.

I would look into box https://github.com/klmr/box if you haven’t heard of it already

Shapeless plug: check out the already mentioned ‘box’, I think it’s strictly superior to ‘import’ (but I’m biased).

By contrast, R code doesn’t need to change the working directory at all! Having to do so hides other flaws in the code. For instance, when trying to load code or data, use the tools provided by R. That is, write packages and use system.file or, when not writing packages, use ‘box’.

‘box’ fixes that.

I think your use-case is best addressed by the ‘targets’ package. But I would also recommend checking out the ‘box’ package for a more general way of structuring R projects in modules which isn’t supported well natively by R (disclaimer: I wrote that package). Writing R code as modules fundamentally side-steps the issue of having to deal with absolute paths. Instead, all code and data are either contained in the module or can be accessed relative to the working directory.

However, using ‘box’, as recommended in another comment, allows you to achieve the same effect with less (and cleaner) code, by declaring your imports locally with the box::use function.

If you're writing you're own code you can use packageName::functionName(), or the box package. Which is definitely useful on larger codebases.

If you need speed, you should consider using dtplyr (or tidytable), or even dbplyr with duckdb.

What do you think of this instead?

Here's an example - I use this in a package I've built called tidytable. Here is the as_tidytable() function I use that uses method dispatch.

If you're having performance issues with dplyr you can also try out tidytable

I’d say tidytable is a better option these days as it supports more functions. Although I think dtplyr has improved on this front recently, but still lags. The author of tidytable contributes to dtplyr as well.

I know you said it 'feels much faster' which isn't exactly a data oriented comparison, but tidymodels performs very well. You can see one of the dplyr functions as step_* in tidymodels, for example mutate vs. step_mutate under recipes library. The author of tidytable, which uses data.table, had some revisions due to this conversation, just as an example.

Tidytable is what you might be looking for: https://markfairbanks.github.io/tidytable/, this will require a bit of refactoring (e.g group-bys happen as arguments in summarise/mutate). You'll get data.table like speed in a very compact & complete package.

People can also use tidytable and keep the same workflow they're already used to 😄