onnxruntime VS windows-rs

Embedding generation uses the ONNX runtime under the hood. This is a cross-platform inferencing library that supports multiple execution providers from CPU to specialized GPUs.

tfjs is dead, looking at the commit history. The standard now is to convert PyTorch to onnx, then use onnxruntime (https://github.com/microsoft/onnxruntime/tree/main/js/web) to run the model on the browsdr.

- slang[2] has the potential, but the meta programming part is not as strong as C++, existing libraries cannot be used.

The above conclusion is drawn from my work https://github.com/microsoft/onnxruntime/tree/dev/opencl, purely nightmare to work with thoes drivers and jit compilers. Hopefully Vcc can take compute shader more seriously.

[1]: https://www.circle-lang.org/

I did. It depends what you want, for an overview of how ONNX Runtime works then Microsoft have a bunch of things on https://onnxruntime.ai, but the Java content is a bit lacking on there as I've not had time to write much. Eventually I'll probably write something similar to the C# SD tutorial they have on there but for the Java API.

For writing ONNX models from Java we added an ONNX export system to Tribuo in 2022 which can be used by anything on the JVM to export ONNX models in an easier way than writing a protobuf directly. Tribuo doesn't have full coverage of the ONNX spec, but we're happy to accept PRs to expand it, otherwise it'll fill out as we need it.

What about models besides GPT? Most of the popular vector encoding models aren't using this architecture.

If you really didn't want PyTorch/Transformers, you could consider exporting your models to ONNX (https://github.com/microsoft/onnxruntime).

It's a shame that, unlike with Win32, using WinUI places pretty harsh restrictions on which programming languages and environments you can use. Only C# and C++ are supported, the latter only with Microsoft compilers. For everything else, including Rust[1], Python and MinGW C/C++, there is no answer for OP's question, and the effect of this on the visual consistency of the Windows desktop is obvious - there is none. Every third-party app uses a different toolkit with a different look and feel, because the library providing the standard look and feel simply isn't available to the majority of developers.

[1]: https://github.com/microsoft/windows-rs/pull/1836

[0] https://github.com/microsoft/windows-rs

On Windows you'd need to call the LoadLibraryEx method. You'd also need a crate to call Win32 functions, I suggest windows-rs.

windows-rs, Microsoft's crate wrapping the Windows API, already includes the WDK, the special sdk for creating kernel code.

On windows, I'll probably use https://github.com/gabdube/native-windows-gui or https://github.com/microsoft/windows-rs both of them seem pretty solid.

*nod* It's an official Microsoft thing generated from official Microsoft API definition files. (The repo is at microsoft/windows-rs on GitHub.)

Microsoft offers official "bindings" to Win32 APIs through win32metadata. However, until recently, it did not include metadata for kernel-level functions or WDK. In early 2021, an issue was raised through windows-rs regarding this limitation, but progress was slow until now. Microsoft has finally released official metadata for WDK, which can be found on the wdkmetadata repository. The latest comment on the issue thread can be found here:

Is something wrong with https://github.com/deviceplug/btleplug or you haven't found it? You could also use bindings to platform libraries like https://github.com/microsoft/windows-rs and https://github.com/rust-mobile/ndk if btleplug doesn't have something fundamental to you.