robotjs VS nut.js

If we talk about the first approach, the following approaches will be useful: Nodejs Child Process: write to stdin from an already initialised process, RobotJS. It makes sense to note here that the approaches above are rather for e2e testing than integration.

If you want to interact with the OS, and move the mouse, you'll likely want to write JS that targets the NodeJS platform. With a quick google search I found RobotJS which you can use with Node to automate the mouse (amongst other things). If it's a browser you want to automate, look into selenium-webdriver.

Then there’s obviously https://github.com/robot/robot-js and https://github.com/octalmage/robotjs, but both of these frameworks are abandoned.

That said, you're in luck as I can give you a specific use case from the library I'm working on. It's a desktop automation library (something like RobotJS or Nut.js). I won't use the exact code, but a simplified version as an example.

So what u need to do is go to a mercs bounty and afk inside it for 29 mins, then finish the fight by beating it and repeat. U will get more than 400 xp per hour with 20% boost (i think around 430? smth like that). If u go over 30 mins in the fight, u will get 0 xp so don't. Put the script on a loop and leave ur pc overnight, set hs fps to like 5 or smth from the config file (targetframerate), so ur pc uses little resources (the more gpu/cpu used the more wattage the higher the electricity bill, tho obviously it's still very worth it but the less the bill is the better). Level 100 to 400 is 300 * 1500 xp = 450 000 xp. With 10 hrs of ur pc working during the night, that is let's say 4100 xp per day, so it will take u 110 days to get from level 100 to 400. If u have work, leaving the pc on for let's say 16 hours a day (8 while u sleep and 8 while u work) will get u 7560 xp per day, getting u from 100 to level 400 in just 60 days (2 months). Make sure ur cpu gpu etc. are put on power-saving mode and turn off ur monitor (preferably even disconnect it from the power cable). Now for the script u wanna use, based on this https://thenewstack.io/which-programming-languages-use-the-least-electricity/ the best one would be C, personally i use Javascript because it's what i know and it's very easy to write and it's still 4.45 which is not that far from 1, considering it's just very simple mouse movement and a for loop and not some complex algorithm problems like in the article, it won't make much of a difference anyway, tho it might if u use python since 75.88 is a lot. So to do it with JS, u can use Node js with Robot js https://github.com/octalmage/robotjs

Autohotkey is also a scripting language. If you want parity there, there's robotJS, pynput, and xdotool (though this one won't work on some future version of mint when a piece of tech it relies on gets upgraded).

I use the Johnny-Five JS library to connect to the board http://johnny-five.io/ and use RobotJS https://www.npmjs.com/package/robotjs to map the buttons on the board to keyboard buttons. Currently I just have it set up to map to audio buttons so I can prev/next/pause/play Spotify/Youtube or turn the OS volume up.

The number of Dislikes on that GitHub issue that the OP mentioned in the post has gone from 36 to the moon! https://github.com/nut-tree/nut.js/issues/577

Nut.js is great for this kind of thing, you could probably drop ahk with it

You could try nut.js. I am not sure if it includes any other dependencies, though.

One really cool little JS library I've been using for a bunch of desktop automation tasks lately is nut.js and the lower level libnut library it's implemented on top of:

https://github.com/nut-tree/nut.js

https://github.com/nut-tree/libnut

It provides a means to send user input (mouse movement/clicks and key presses) and read and react to changes in visual state (through screenshots), and works across Windows, Linux and MacOS. It automates at a much lower level of abstraction than the approaches mentioned in the article that script against programmatic APIs.

What I really like about this lower level approach is that you don't need to get anyone's permission to automate anything, since there's no programmatic API that the system owners has to provide for you and thus can limit or take away when it becomes inconvenient.

Any task that can be accomplished though looking at stuff on the screen and clicking the mouse and pressing keys on a keyboard (i.e. what a real person would do to accomplish the same task) can be automated, and it's actually surprisingly easy and effective to do this with nut.js. What really helps is that OpenCV has become ridiculously good and ridiculously fast at matching/identifying objects from a screenshot, with latencies usually in the low double digits, so latency-based flakiness isn't nearly as much of an issue as I remember it in the old days. I've also played around with OCR with tesseract but haven't had as much success with it in terms of perf, and remember seeing latencies of several seconds for even recognizing a single word from a tiny pre-cropped screenshot containing only the word itself.

The main tradeoff to this approach compared to automation through APIs is that because it works by simulating real user inputs, it's not very amenable to running in the background while a user is actively interacting with the same machine, so a separate machine or VM is often needed. That's an acceptable tradeoff for some use cases but complete deal breaker for others, so YMMV, but just wanted to bring this cool little tool to people's attention.