wasi-nn
distroless
wasi-nn | distroless | |
---|---|---|
3 | 122 | |
402 | 17,781 | |
4.7% | 1.4% | |
5.6 | 9.4 | |
7 days ago | 1 day ago | |
Rust | Starlark | |
- | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
wasi-nn
-
Self-Hosting Open Source LLMs: Cross Devices and Local Deployment of Mistral 7B
I really like the post that they mention (https://www.secondstate.io/articles/fast-llm-inference/). The reasons for avoiding python all resonate with me. I'm excited to play with WASI-NN (https://github.com/WebAssembly/wasi-nn) and that rust code is very readable to load up a GGUL model.
-
Run LLMs on my own Mac fast and efficient Only 2 MBs
Mmm…
The wasm-nn that this relies on (https://github.com/WebAssembly/wasi-nn) is a proposal that relies of arbitrary plugin backends sending arbitrarily chunks to some vendor implementation. The api is literally like set input, compute, set output.
…and that is totally non portable.
The reason this works, is because it’s relying on the abstraction already implemented in llama.cpp that allows it to take a gguf model and map it to multiple hardware targets,which you can see has been lifted here: https://github.com/WasmEdge/WasmEdge/tree/master/plugins/was...
So..
> Developers can refer to this project to write their machine learning application in a high-level language using the bindings, compile it to WebAssembly, and run it with a WebAssembly runtime that supports the wasi-nn proposal, such as WasmEdge.
Is total rubbish; no, you can’t.
This isn’t portable.
It’s not sandboxed.
If you have a wasm binary you might be able to run it if the version of the runtime you’re using happens to implement the specific ggml backend you need, which it probably doesn’t… because there’s literally no requirement for it to do so.
There’s a lot of “so portable” talk in this article which really seems misplaced.
-
The Promise of WASM
in machine learning (https://github.com/WebAssembly/wasi-nn)
distroless
-
Chainguard Images now available on Docker Hub
lots of questions here regarding what this product is. I guess i can provide some information for the context, from a perspective of an outside contributor.
Chainguard Images is a set of hardened container images.
They were built by the original team that brought you Google's Distroless (https://github.com/GoogleContainerTools/distroless)
However, there were few problems with Distroless:
1. distroless were based on Debian - which in turn, limited to Debian's release cadence for fixing CVE.
2. distroless is using bazelbuild, which is not exactly easy to contrib, customize, etc...
3. distroless images are hard to extend.
Chainguard built a new "undistro" OS for container workload, named Wolfi, using their OSS projects like melange (for packaging pkgs) and apko (for building images).
The idea is (from my understanding) is that
1. You don't have to rely on upstream to cut a release. Chainguard will be doing that, with lots of automation & guardrails in placed. This allow them to fix vulnerabilties extremely fast.
- Language focused Docker images, minus the operating system
-
Using Alpine can make Python Docker builds 50Ă— slower
> If you have one image based on Ubuntu in your stack, you may as well base them all on Ubuntu, because you only need to download (and store!) the common base image once
This is only true if your infrastructure is static. If your infrastructure is highly elastic, image size has an impact on your time to scale up.
Of course, there are better choices than Alpine to optimize image size. Distroless (https://github.com/GoogleContainerTools/distroless) is a good example.
- Smaller and Safer Clojure Containers: Minimizing the Software Bill of Materials
-
Long Term Ownership of an Event-Driven System
The same as our code dependencies, container updates can include security patches and bug fixes and improvements. However, they can also include breaking changes and it is crucial you test them thoroughly before putting them into production. Wherever possible, I recommend using the distroless base image which will drastically reduce both your image size, your risk vector, and therefore your maintenance version going forward.
-
Minimizing Nuxt 3 Docker Images
# Use a large Node.js base image to build the application and name it "build" FROM node:18-alpine as build WORKDIR /app # Copy the package.json and package-lock.json files into the working directory before copying the rest of the files # This will cache the dependencies and speed up subsequent builds if the dependencies don't change COPY package*.json /app # You might want to use yarn or pnpm instead RUN npm install COPY . /app RUN npm run build # Instead of using a node:18-alpine image, we are using a distroless image. These are provided by google: https://github.com/GoogleContainerTools/distroless FROM gcr.io/distroless/nodejs:18 as prod WORKDIR /app # Copy the built application from the "build" image into the "prod" image COPY --from=build /app/.output /app/.output # Since this image only contains node.js, we do not need to specify the node command and simply pass the path to the index.mjs file! CMD ["/app/.output/server/index.mjs"]
-
Build Your Own Docker with Linux Namespaces, Cgroups, and Chroot
Lots of examples without the entire OS as other comments mention, an example would be Googles distroless[0]
[0]: https://github.com/GoogleContainerTools/distroless
-
Reddit temporarily ban subreddit and user advertising rival self-hosted platform (Lemmy)
Docker doesn't do this all the time. Distroless Docker containers are relatively common. https://github.com/GoogleContainerTools/distroless
-
Why elixir over Golang
Deployment: https://github.com/GoogleContainerTools/distroless
-
Reviews
Or use distroless image as it includes one, among others. https://github.com/GoogleContainerTools/distroless/blob/main/base/README.md
What are some alternatives?
wasmer - 🚀 The leading Wasm Runtime supporting WASIX, WASI and Emscripten
iron-alpine - Hardened alpine linux baseimage for Docker.
wagi - Write HTTP handlers in WebAssembly with a minimal amount of work
spring-boot-jib - This project is about Containerizing a Spring Boot Application With Jib
WasmEdge-WASINN-examples
jib - 🏗 Build container images for your Java applications.
podman - Podman: A tool for managing OCI containers and pods.
dockerfiles - Various Dockerfiles I use on the desktop and on servers.
docker-alpine - Official Alpine Linux Docker image. Win at minimalism!
whalebrew - Homebrew, but with Docker images
example-bazel-monorepo - 🌿💚 Example Bazel-ified monorepo, supporting Golang, Java, Python, Scala, and Typescript
fpm - Effing package management! Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity.