websurfx VS rust-gpu

GitHub release: https://github.com/neon-mmd/websurfx/releases/tag/v1.0.0

Sorry for being late to reach you out. Thanks for trying out Websurfx, but I can help you out in setting up Websurfx. Just a raise an issue here, or you can ask for help here at our discord, we would be glad in helping you out :).

Recently we decided to do error-handling and provide custom error-messages for the errors related to each engine code present under src/engines folder in one of my project websurfx when using error-stack and we wanted to use enums for it and we found that the error-stack project provides no guide, tutorial or example but thanks to one of our maintainers @xffxff, @xffxff provided a really cool solution to this problem and which helped me learn a lot so I decided to share with you all on what I learned from it in this post and how you can wrap errors with enums when using error-stack so stick till the end of the post.

Also, you can check out the docs for steps on how to do docker deployment. I have covered it over there in great depth.

git clone https://github.com/neon-mmd/websurfx.git

Before you look into the code, I would like to explain in brief what the code below does. The code below a function which builds a new random user-agent from the provided browsers and returns it as a string but it is too slow it takes 10 seconds to build and even with caching option enabled it takes around 5-7 seconds but I want to reduce the time it takes by as much as possible. Please provide me with some suggestions or code on how to improve this down in the comments or here in this issue I have opened in my project.

If you'd like to join the WebSurfx team and help shape the future of a privacy-centric search engine, please check out our project on GitHub: WebSurfx GitHub. Feel free to explore the code, open issues, or submit pull requests. Don't hesitate to introduce yourself and share your ideas!

Sounds cool, but this requires yet another language to learn[0]. As someone who only has limited knowledge in this space, could someone tell me how comparable is the compute functionality of rust-gpu[1], where I can just write rust?

[0] https://github.com/Hugobros3/shady#language-syntax

[1] https://github.com/EmbarkStudios/rust-gpu

I don't do anything related to data science, but I feel like doing it in Rust would be nice.

You get operator overloading, so you can have ergonomic matrix operations that are typed also. Processing data on the CPU is fast, and crates like https://github.com/EmbarkStudios/rust-gpu make it very ergonomic to leverage the GPU.

I like this library for creating typed coordinate spaces for graphics programming (https://github.com/servo/euclid), I imagine something similar could be done to create refined types for matrices so you don't do matrix multiplication matrices of invalid sizes

Do you mean rust-gpu?

And https://github.com/EmbarkStudios/rust-gpu/tree/main/examples with the wgpu runner (here it runs the compute shader)

I don't know how major they are considered, but Embark Studios is doing quite a bit of Rust in the open source space, most notably (IMO) rust-gpu and kajiya

The examples in the rust-gpu repository are a good place to start

There's another project that's similar that's being used by an actual game company: https://github.com/EmbarkStudios/rust-gpu

They see specific advantages here that would outweigh that negative. It's not my space (I play games, but know next to nothing about graphics programming), but there's at least one argument in the other direction.

Could this approach work for compute shaders (GPGPU) as well? So far, I think https://github.com/EmbarkStudios/rust-gpu is the state of the art in that area, but it adds a specific Rust compiler backend for generating SPIR-V rather than leaving that up to the driver. That seems more complicated than it needs to be... but maybe it has advantages too? Thoughts?

https://github.com/embarkstudios/rust-gpu Allows you to create shaders (kernals) in Rust.

Also Embark Studios (formers DICE people) is doing a lot of work with Rust, all open source like Rust GPU https://github.com/EmbarkStudios/rust-gpu