node-gyp VS neon

The benefit of having it in JS is not speed but portability and access to the JS ecosystem of tools. Having the code run in the browser without needing a complex setup is a huge benefit for sharing demos. Node.js provides a way to use native code as well and it's quite commonly used https://github.com/nodejs/node-gyp so there's no reason you couldn't use those same or similar libraries in a JS implementation.

It is still possible to execute binaries made in other languages such as C/C++ by linking it dynamically as Native Addon Modules in Node.js using tools like node-gyp. But the execution in those case is done by the operating system, not the JS Engine.

npm ERR! A complete log of this run can be found in: C:\Users\ASUS\AppData\Local\npm-cache\_logs\2023-07-13T06_54_24_745Z-debug-0.log PS C:\Users\ASUS\Documents\Sample Project\Audio to Text\scripts> npm install @tensorflow/tfjs-node npm ERR! code 1 npm ERR! path C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node npm ERR! command failed npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node scripts/install.js npm ERR! CPU-windows-4.8.0.zip npm ERR! https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.9.1.zip npm ERR! node-pre-gyp install failed with error: Error: Command failed: node-pre-gyp install --fallback-to-build npm ERR! node-pre-gyp info it worked if it ends with ok npm ERR! node-pre-gyp info using [email protected] npm ERR! node-pre-gyp info using [email protected] | win32 | x64 npm ERR! node-pre-gyp info check checked for "C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node\lib\napi-v8\tfjs_binding.node" (not found) npm ERR! node-pre-gyp http GET https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.8.0/CPU-windows-4.8.0.zip npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.8.0/CPU-windows-4.8.0.zip npm ERR! node-pre-gyp WARN Pre-built binaries not installable for @tensorflow/[email protected] and [email protected] (node-v108 ABI, unknown) (falling back to source compile with node-gyp) npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.8.0/CPU-windows-4.8.0.zip npm ERR! gyp info it worked if it ends with ok npm ERR! gyp info using [email protected] npm ERR! gyp info using [email protected] | win32 | x64 npm ERR! gyp info ok npm ERR! gyp info it worked if it ends with ok npm ERR! gyp info using [email protected] npm ERR! gyp info using [email protected] | win32 | x64 npm ERR! gyp info find Python using Python version 3.7.9 found at "C:\Users\ASUS\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\python.exe" npm ERR! gyp ERR! find VS npm ERR! gyp ERR! find VS msvs_version not set from command line or npm config npm ERR! gyp ERR! find VS VCINSTALLDIR not set, not running in VS Command Prompt npm ERR! gyp ERR! find VS checking VS2022 (17.6.33815.320) found at: npm ERR! gyp ERR! find VS "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools" npm ERR! gyp ERR! find VS - found "Visual Studio C++ core features" npm ERR! gyp ERR! find VS - missing any VC++ toolset npm ERR! gyp ERR! find VS could not find a version of Visual Studio 2017 or newer to use npm ERR! gyp ERR! find VS looking for Visual Studio 2015 npm ERR! gyp ERR! find VS - not found npm ERR! gyp ERR! find VS not looking for VS2013 as it is only supported up to Node.js 8 npm ERR! gyp ERR! find VS npm ERR! gyp ERR! find VS ************************************************************** npm ERR! gyp ERR! find VS You need to install the latest version of Visual Studio npm ERR! gyp ERR! find VS including the "Desktop development with C++" workload. npm ERR! gyp ERR! find VS For more information consult the documentation at: npm ERR! gyp ERR! find VS https://github.com/nodejs/node-gyp#on-windows npm ERR! gyp ERR! find VS ************************************************************** npm ERR! gyp ERR! find VS npm ERR! gyp ERR! configure error npm ERR! gyp ERR! stack Error: Could not find any Visual Studio installation to use npm ERR! gyp ERR! stack at VisualStudioFinder.fail (C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:122:47) npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:75:16 npm ERR! gyp ERR! stack at VisualStudioFinder.findVisualStudio2013 (C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:380:14) npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:71:14 npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:401:16 npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\util.js:54:7 npm ERR! gyp ERR! stack at C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\util.js:33:16 npm ERR! gyp ERR! stack at ChildProcess.exithandler (node:child_process:427:5) npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:513:28) npm ERR! gyp ERR! stack at maybeClose (node:internal/child_process:1091:16) npm ERR! gyp ERR! System Windows_NT 10.0.22621 npm ERR! gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\ASUS\\AppData\\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "configure" "--fallback-to-build" "--module=C:\\Users\\ASUS\\Documents\\Sample Project\\Audio to Text\\node_modules\\@tensorflow\\tfjs-node\\lib\\napi-v8\\tfjs_binding.node" "--module_name=tfjs_binding" "--module_path=C:\\Users\\ASUS\\Documents\\Sample Project\\Audio to Text\\node_modules\\@tensorflow\\tfjs-node\\lib\\napi-v8" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=8" "--node_napi_label=napi-v8" npm ERR! gyp ERR! cwd C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node npm ERR! gyp ERR! node -v v18.16.0 npm ERR! gyp ERR! node-gyp -v v9.4.0 npm ERR! gyp ERR! not ok npm ERR! node-pre-gyp ERR! build error npm ERR! node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\ASUS\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js configure --fallback-to-build --module=C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node\lib\napi-v8\tfjs_binding.node --module_name=tfjs_binding --module_path=C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node\lib\napi-v8 --napi_version=8 --node_abi_napi=napi --napi_build_version=8 --node_napi_label=napi-v8' (1) npm ERR! node-pre-gyp ERR! stack at ChildProcess. (C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@mapbox\node-pre-gyp\lib\util\compile.js:89:23) npm ERR! node-pre-gyp ERR! stack at ChildProcess.emit (node:events:513:28) npm ERR! node-pre-gyp ERR! stack at maybeClose (node:internal/child_process:1091:16) npm ERR! node-pre-gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:302:5) npm ERR! node-pre-gyp ERR! System Windows_NT 10.0.22621 npm ERR! node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\ASUS\\Documents\\Sample Project\\Audio to Text\\node_modules\\@mapbox\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build" npm ERR! node-pre-gyp ERR! cwd C:\Users\ASUS\Documents\Sample Project\Audio to Text\node_modules\@tensorflow\tfjs-node npm ERR! node-pre-gyp ERR! node -v v18.16.0 npm ERR! node-pre-gyp ERR! node-pre-gyp -v v1.0.9 npm ERR! node-pre-gyp ERR! not ok npm ERR! * Downloading libtensorflow npm ERR! npm ERR! * Building TensorFlow Node.js bindings npm ERR! A complete log of this run can be found in: C:\Users\ASUS\AppData\Local\npm-cache\_logs\2023-07-13T07_03_46_164Z-debug-0.log

create a native Node package using node-gyp

These are the bare minimum. You can also target other API levels like @nodert-win10-au, etc. You'll need to set up node-gyp with Python and Windows Build Tools and install the Windows 10 SDK corresponding to the version of the NodeRT package you pulled in.

I assume it's use of node-gyp, the native plugin system.

https://github.com/nodejs/node-gyp

I got it from here. Using -std=c++20 gives me cl : command line warning D9002: ignoring unknown option '-std=c++20'

Read the docs https://github.com/nodejs/node-gyp#on-windows

Great interview!

Love how much thought is being put into what you “gold-plate”. I’ve always felt that my best work comes around on round two (or three or four…).

Curious what you are planning for the ability to script the configuration? I haven’t played with zed much yet; is it possible today? Would something like Neon [1] help bridge the gap from VSCode and old Atom users?

[1]: https://github.com/neon-bindings/neon

Neon

Is there a third option? Surely node has a way to directly call native code, similar to Python's C extensions? Some Node equivalent of PyO3? For example, I found neon which promises "safe and fast native Node.js modules".

Have a look at https://crates.io/crates/napi and https://crates.io/crates/neon which allow you to call rust from node. We went with napi but they're both pretty good.

Just to add: N-API is incredibly underrated. Then again, maybe the lack of a strong native modules ecosystem is an indicator that the pure JS ecosystem is just so good. But man, got something computationally intensive? Just offload it to Rust with Neon or something. Got some proprietary bit of code in your product? Build a native module.

> rust is not well-suited for interfacing with FFI

How so? Packages like neon [1] and rustler [2] suggest otherwise. I'm using both of those in a real product (I'm using neon directly, to write native modules for an Electron app; on the back-end, I depend on an Elixir package that uses rustler).

[1]: https://github.com/neon-bindings/neon

[2]: https://github.com/rusterlium/rustler

You can use https://github.com/infinyon/node-bindgen, https://github.com/neon-bindings/neon, or https://github.com/napi-rs/napi-rs for Node.js libraries, https://github.com/PyO3/pyo3 for Python libraries, https://rustwasm.github.io/wasm-bindgen/ for WebAssembly, and https://github.com/rust-lang/rust-bindgen for C libraries!

They can use Rust to speed up Nodejs through https://crates.io/crates/neon for example

I’m not prompted again while actively using my laptop. When it’s time to switch to an open source project, I’m seamlessly prompted for my GitHub key.