xgboost-node VS node-gyp

This chips away at one of the showstoppers for Deno for me, which is good.

But while "the vast majority" of npm packages don't require a gyp build step for native addons, some of those modules are pretty important, and I see no indication in the announcement that they're also going to be implementing the Node C API or the gyp build process.

Right now I'm working with a machine learning project, and XGBoost [0] is a direct Node.js extension [1] through the binary interface.

So this does bring things a step closer to being generally usable, but there are still significant roadblocks.

A WebAssembly build of XGBoost could work with Deno, but aside from some guy's unsupported side project/proof-of-concept for use in a browser, I'm not seeing an XGBoost WebAssembly build. And generally when deploying something like a machine learning model I'd rather use well-supported tools than to need to dive into the rabbit hole of maintaining my own.

And yes, XGBoost will likely eventually have that kind of support for Deno, but then the next bleeding-edge project will come along and only support Node.

Even assuming Deno eventually hits a tipping point in popularity where everyone wants to release Node _and_ Deno support in their bleeding-edge projects, there are still things that I miss from package.json that don't seem to exist in the Deno ecosystem.

Things like the "scripts" block: A nice centralized place to find all of the things that need to be done to a project, plus auto-run script entries that can trigger when a project is installed. And inheritable, overridable dependency maps (see the yarn "resolutions" block).

I'd love to jump into Deno, but I think there has been far too much "baby thrown out with the bathwater" to its design. It's the classic development problem of looking at a system and seeing a ton of complexity, but not really understanding that all of that complexity was there for a reason. Maybe when it re-evolves 80% of Node's and npm's features I'll be convinced to make the jump. I'm a huge TypeScript fan after all. But it still strikes me as a violation of "As simple as possible, but no simpler."

[0] XGBoost is a _very_ promising approach to machine learning, training models much faster and with much more accuracy than traditional approaches.

[1] https://github.com/nuanio/xgboost-node

The benefit of having it in JS is not speed but portability and access to the JS ecosystem of tools. Having the code run in the browser without needing a complex setup is a huge benefit for sharing demos. Node.js provides a way to use native code as well and it's quite commonly used https://github.com/nodejs/node-gyp so there's no reason you couldn't use those same or similar libraries in a JS implementation.

It is still possible to execute binaries made in other languages such as C/C++ by linking it dynamically as Native Addon Modules in Node.js using tools like node-gyp. But the execution in those case is done by the operating system, not the JS Engine.

npm ERR! A complete log of this run can be found in: C:\Users\ASUS\AppData\Local\npm-cache\_logs\2023-07-13T06_54_24_745Z-debug-0.log PS C:\Users\ASUS\Documents\Sample Project\Audio to Text\scripts> npm install @tensorflow/tfjs-node npm ERR! code 1 npm ERR! path C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node npm ERR! command failed npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node scripts/install.js npm ERR! CPU-windows-4.8.0.zip npm ERR! https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.9.1.zip npm ERR! node-pre-gyp install failed with error: Error: Command failed: node-pre-gyp install --fallback-to-build npm ERR! node-pre-gyp info it worked if it ends with ok npm ERR! node-pre-gyp info using [email protected] npm ERR! node-pre-gyp info using [email protected] | win32 | x64 npm ERR! node-pre-gyp info check checked for "C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node\lib\napi-v8\tfjs_binding.node" (not found) npm ERR! node-pre-gyp http GET https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.8.0/CPU-windows-4.8.0.zip npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.8.0/CPU-windows-4.8.0.zip npm ERR! node-pre-gyp WARN Pre-built binaries not installable for @tensorflow/[email protected] and [email protected] (node-v108 ABI, unknown) (falling back to source compile with node-gyp) npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.8.0/CPU-windows-4.8.0.zip npm ERR! gyp info it worked if it ends with ok npm ERR! gyp info using [email protected] npm ERR! gyp info using [email protected] | win32 | x64 npm ERR! gyp info ok npm ERR! gyp info it worked if it ends with ok npm ERR! gyp info using [email protected] npm ERR! gyp info using [email protected] | win32 | x64 npm ERR! gyp info find Python using Python version 3.7.9 found at "C:\Users\ASUS\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\python.exe" npm ERR! gyp ERR! find VS npm ERR! gyp ERR! find VS msvs_version not set from command line or npm config npm ERR! gyp ERR! find VS VCINSTALLDIR not set, not running in VS Command Prompt npm ERR! gyp ERR! find VS checking VS2022 (17.6.33815.320) found at: npm ERR! gyp ERR! find VS "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools" npm ERR! gyp ERR! find VS - found "Visual Studio C++ core features" npm ERR! gyp ERR! find VS - missing any VC++ toolset npm ERR! gyp ERR! find VS could not find a version of Visual Studio 2017 or newer to use npm ERR! gyp ERR! find VS looking for Visual Studio 2015 npm ERR! gyp ERR! find VS - not found npm ERR! gyp ERR! find VS not looking for VS2013 as it is only supported up to Node.js 8 npm ERR! gyp ERR! find VS npm ERR! gyp ERR! find VS ************************************************************** npm ERR! gyp ERR! find VS You need to install the latest version of Visual Studio npm ERR! gyp ERR! find VS including the "Desktop development with C++" workload. npm ERR! gyp ERR! find VS For more information consult the documentation at: npm ERR! gyp ERR! find VS https://github.com/nodejs/node-gyp#on-windows npm ERR! gyp ERR! find VS ************************************************************** npm ERR! gyp ERR! find VS npm ERR! gyp ERR! configure error npm ERR! gyp ERR! stack Error: Could not find any Visual Studio installation to use npm ERR! gyp ERR! stack at VisualStudioFinder.fail (C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:122:47) npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:75:16 npm ERR! gyp ERR! stack at VisualStudioFinder.findVisualStudio2013 (C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:380:14) npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:71:14 npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:401:16 npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\util.js:54:7 npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\util.js:33:16 npm ERR! gyp ERR! stack at ChildProcess.exithandler (node:child_process:427:5) npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:513:28) npm ERR! gyp ERR! stack at maybeClose (node:internal/child_process:1091:16) npm ERR! gyp ERR! System Windows_NT 10.0.22621 npm ERR! gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\ASUS\\AppData\\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "configure" "--fallback-to-build" "--module=C:\\Users\\ASUS\\Documents\\Sample Project\\Audio to Text\\node_modules\\@tensorflow\\tfjs-node\\lib\\napi-v8\\tfjs_binding.node" "--module_name=tfjs_binding" "--module_path=C:\\Users\\ASUS\\Documents\\Sample Project\\Audio to Text\\node_modules\\@tensorflow\\tfjs-node\\lib\\napi-v8" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=8" "--node_napi_label=napi-v8" npm ERR! gyp ERR! cwd C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node npm ERR! gyp ERR! node -v v18.16.0 npm ERR! gyp ERR! node-gyp -v v9.4.0 npm ERR! gyp ERR! not ok npm ERR! node-pre-gyp ERR! build error npm ERR! node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js configure --fallback-to-build --module=C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node\lib\napi-v8\tfjs_binding.node --module_name=tfjs_binding --module_path=C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node\lib\napi-v8 --napi_version=8 --node_abi_napi=napi --napi_build_version=8 --node_napi_label=napi-v8' (1) npm ERR! node-pre-gyp ERR! stack at ChildProcess. (C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@mapbox\node-pre-gyp\lib\util\compile.js:89:23) npm ERR! node-pre-gyp ERR! stack at ChildProcess.emit (node:events:513:28) npm ERR! node-pre-gyp ERR! stack at maybeClose (node:internal/child_process:1091:16) npm ERR! node-pre-gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:302:5) npm ERR! node-pre-gyp ERR! System Windows_NT 10.0.22621 npm ERR! node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\ASUS\\Documents\\Sample Project\\Audio to Text\\node_modules\\@mapbox\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build" npm ERR! node-pre-gyp ERR! cwd C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node npm ERR! node-pre-gyp ERR! node -v v18.16.0 npm ERR! node-pre-gyp ERR! node-pre-gyp -v v1.0.9 npm ERR! node-pre-gyp ERR! not ok npm ERR! * Downloading libtensorflow npm ERR! npm ERR! * Building TensorFlow Node.js bindings npm ERR! A complete log of this run can be found in: C:\Users\ASUS\AppData\Local\npm-cache\_logs\2023-07-13T07_03_46_164Z-debug-0.log

create a native Node package using node-gyp

These are the bare minimum. You can also target other API levels like @nodert-win10-au, etc. You'll need to set up node-gyp with Python and Windows Build Tools and install the Windows 10 SDK corresponding to the version of the NodeRT package you pulled in.

I assume it's use of node-gyp, the native plugin system.

https://github.com/nodejs/node-gyp

I got it from here. Using -std=c++20 gives me cl : command line warning D9002: ignoring unknown option '-std=c++20'

Read the docs https://github.com/nodejs/node-gyp#on-windows