fs-extra VS async

I haven't seen fs-extra mentioned yet. For my work it involves a fair bit of reading/writing to the filesystem, so this makes it quite nice to deal with everything in an async way.

There are a number of these modules, https://github.com/jprichardson/node-fs-extra being probably the most famous / popular. It does a better job of explaining the why.

Since updating nodejs@18 and switching to esm only, many libraries have been replaced to support esm import, but fs-extra has not been The use of esm is correctly supported, and no suitable replacement has been found. After the a PR proposed by us was rejected, I decided to re-release a [fs-extra-unified] that correctly supports the use of esm (https://www.npmjs.com/package/fs-extra-unified) module.

Are you seriously suggesting the creators of the software can't write a "save elsewhere" function using Electron and JavaScript? The system has versioning so it would be easy to ask the user to create a backup on first loading with a new version of the software, and the copySync function to make a backup. Foundry uses Node.js already.

We are dealing with a file that is far too large to be loaded into memory at once. Node cannot buffer the file for us because of the size of it.

Implementing those 3 features was quite simple. For metadata and markdown file, I mainly used an external library: showdown to convert from markdown files to html files as well as parsing metadata from markdown files. However, implementing the support for static assets such as images is a bit more interesting. I had to use another external library called fs-extra to copy assets folder from user to the dist folder. I also had to create another new class called CopyFolder where I used fs.copy method to copy assets folder

The zx project implements a shell script executor similar to Bash but using JavaScript modules. It provides an inbuilt asynchronous JavaScript API to call other commands similar to Bash. Besides that, it provides wrapper functions for several Node.js-based command-line helpers such as chalk, minimist, fs-extra, OS, and Readline.

Node.js has the built-in fs module (fs/promises with async/await) which makes it easy to interact with the file system. On top of that, there are a few convenient libraries such as fs-extra and findit that make common tasks straightforward.

For this purpose we'll use fs-extra since copy/paste seems like not supported by fs API.

Well you could just install the async package which has lots of useful functions like mapLimit which will reduce the burden and only run a number in parallel.

Have you checked out the docs for the async library they are using?

Edit:2. What's a good library that takes care of common async patterns? (Something like async)

Async/await is certainly not promises. In fact it would be much better implemented without promises as I proposed here: https://es.discourse.group/t/callback-based-simplified-async...

I would even say that async/await is anti-promise, it takes the main functionality of promises, a caching layer for results and errors that allows you to add the code continuation later and elsewhere (which is a major footgun imo) and coerces the execution flow back to going on the next line and provided immediately at compile time which results in a cleaner flow but not as clean, stateless, efficient or functional as if you were to remove the promises completely. Having an additional caching layer and state machine around every asynchronous function call is quite inefficient.

The essence of async/await is not promises, it's the underlying javascript generator (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...) functionality combined with asynchronous code to stop and start the generator. It's the ability to pause and resume function execution based on asynchronous operations.

The promise functionality, the caching layer and state machine for results is basically sanitized away with async/await, it becomes dead-weight computation. The only benefit of promises in async/await code is being able to more easily interface with other promise laden code which you don't need once you have async/await and a library like https://www.npmjs.com/package/async for more complex cases.

Note that promises based async/await is also a mess of an implementation that breaks stack traces and needs to support tons of odd statement corner cases (basically anything that can return an object that could be a promise) whereas a continuation passing style async/await would be a much simpler implementation that would only apply to function calls and maintain stack traces. We get that stack trace support automatically because of the great work of whoever implemented javascript generators which seem to already carry stack traces across paused/resumed functions (if you don't wrap in promises).

The nodejs async module: https://github.com/caolan/async provides 2 similar methods, async.waterfall and async.series.

With async I'd simply use async.series().

I don't understand. Sure you could spawn a ton of processes, but things might be bogged down. There are utilities out there for doing work queues.... so only N workers are running at any one time. The async library has some utilities for that. https://github.com/caolan/async

The documentation of asyncjs is quite straightforward and easy to read. As we've only seen a couple of use cases in this article, I'd recommend to go the asyncjs documentation and check out other possibilities with the library. You can also try to replicate the same using async/await to solidify your understanding of where the library might still make sense.

It basically was tons and tons of callbacks. They'd nest weirdly deep and be a pain to work with. If you're curious, here's a link to one of my favorite JavaScript libraries from those days - it gave you a bunch of neat utilities for dealing with async code.

api(function(result){ api2(function(result2){ api3(function(result3){ // do work }); });}); Which I could use a library like async for anyway, with something like: