Rapid-MLX VS standards-positions

💡 💡 Make the fallback cheap to operate. The whole point of using Nano on the supported path is reduced cost. If your fallback is GPT-5.5 at $5/M tokens, you've moved the bill, not deleted it. Two patterns work well: (1) route the fallback to a smaller hosted model (Haiku, Gemini Flash, Mistral Small) that matches Nano's "short summarization" sweet spot; (2) for Mac users specifically, run Rapid-MLX as your /api/llm endpoint — Apple Silicon owners get on-device performance via your server's Mac, not theirs. Same thesis as our DeepClaude guide: the harness is one product, the model is another, and you can swap them.

> Large-context requests auto-route to a cloud LLM (GPT-5, Claude, etc.) when local prefill would be slow. Routing based on new tokens after cache hit. --cloud-model openai/gpt-5 --cloud-threshold 20000

https://github.com/raullenchai/Rapid-MLX

I've verified this end-to-end with structured output (output_type=BaseModel), streaming, multi-turn conversations, and multi-tool workflows. Test suite here.

pip install git+https://github.com/raullenchai/vllm-mlx.git

Mozilla's position from when Chrome first dropped it to September 2024 was "the benefits it provides are not significant enough on their own to justify the cost of adding another raster image format to the Web" [0], which they say is a "neutral" stance. Then like Chrome they only agreed to try it with jxl-rs [1], which is still their present stance. They are a complete passenger in this whole affair, like all other standards, where they basically just copy one side or the other (usually the more conservative side).

It's really bizarre to me that this is presented as "killing the standard". Is Apple also killing mechanical keyboards and hobby electronics development because they're the only ones who don't support Web USB or Web Serial? I strongly prefer having JXL and Web USB/Serial in my browser (FF for the last 20 years), but come on. If we don't like how much power browsers have in software distribution, then maybe software distribution outside of browsers should get fixed.

* [0] https://github.com/mozilla/standards-positions/issues/522

* [1] https://github.com/mozilla/standards-positions/pull/1064

Mozilla's response to "Request for Mozilla Position on an Emerging Web Specification", June 2020:

> For raw device access as envisioned in a number of APIs (Web USB, Web Bluetooth, Web NFC, and Web MIDI), the risks of exposing those APIs to users cannot be reasonably conveyed. This is pretty much an intractable flaw of allowing raw, non-semantic access to devices regardless of the protocol used to do so.

> The specific issue is: it's not intuitive that allowing malicious-site.com to access your Bluetooth keyboard might give that site access to your stored passwords, give them the ability to hijack your DNS settings, or allow them to encrypt your hard drive and hold it ransom. And if it's not immediately obvious how those things are possible, that only serves to demonstrate how completely non-intuitive the risks are and how intractable trying to explain them in a permission prompt would be.

https://github.com/mozilla/standards-positions/issues/95#iss...

When you have questions like this, the best places to check are the "standards positions" Github repositories for Mozilla and WebKit.

https://github.com/mozilla/standards-positions/issues/1369

Mozilla hasn't officially weighed in, but one decision maker (hsivonen) did vote in favor of it.

Apple WebKit is officially in favor of it, as of just a few days ago.

https://github.com/WebKit/standards-positions/issues/628

> AFAICT, this is a web standard and expected to get buy in from Safari and Firefox before shipping to users.

If it hasn’t already got buy in then it isn’t a web standard, it’s just a Google proposal.

Here are Mozilla and WebKit positions on this:

> This proposal attempts to solve multiple problems with a single solution. We (Mozilla) recognize the motivation for solving some of the problems, but believe that this is not the right solution to each problem, or in some case a step in the wrong direction.

— https://github.com/mozilla/standards-positions/issues/1076

— https://github.com/WebKit/standards-positions/issues/630

As far as I can see, nobody outside of Google has committed to implementing this.

⚠️ ⚠️ Mozilla's pushback isn't just standards politics. Per The Register's coverage, Mozilla's formal opposition is that the Prompt API "encourages model-specific behavior that harms interoperability" — early-2000s browser-sniffing, but for LLM quirks. If you only target Chrome, you'll write prompts that work great on Nano and silently break on whatever Apple ships next. The fallback in Step 4 isn't only for unsupported browsers; it's also your hedge against being locked into Nano's prompt style.

Also note the Mozilla standards position on this API: https://github.com/mozilla/standards-positions/issues/1213#i...

Or this summary on its status:

> Mozilla: Opposed

> WebKit: Opposed

> Microsoft: Several concerns

> W3C TAG: Several concerns

> Developers: Mostly negative

From https://mastodon.social/@jaffathecake/116527007495775507

> The spec is still in draft because Apple refuses to let it move forward

This is untrue. Web standards need two independent implementations. Google can’t convince any other rendering engine besides their own to implement it.

It doesn't take a single no from Apple to veto it; it takes a single yes from anybody outside of Blink.

Here is what Mozilla have to say about WebUSB:

> Because many USB devices are not designed to handle potentially-malicious interactions over the USB protocols and because those devices can have significant effects on the computer they're connected to, we believe that the security risks of exposing USB devices to the Web are too broad to risk exposing users to them or to explain properly to end users to obtain meaningful informed consent. It also poses risks that sites could use USB device identity or data stored on USB devices as tracking identifiers.

— https://mozilla.github.io/standards-positions/#webusb

Until Google can convince anybody outside of Blink to implement it, it is not a standard it’s a Blink-only API.

Some of Mozilla's positions are based on Apple's, such as the refusal to implement Web NFC [0].

Since Webkit has been the only engine allowed on iOS, ultimately this is a disagreement on app distribution. I can see Apple and Mozilla's argument regarding Web NFC, but I also don't want to write a whole app so my friends and I can play around with NFC tags. I find it irresistible to draw comparisons to the new Android situation regarding non-Play Store apps. If there was a developer registration list for websites (that was better than DNS records and TLS certificates), would Apple and Mozilla find that acceptable? After all, I need to give my real name and payment details to Apple just to write an app.

But for good measure I will add one for Mozilla too. Firefox Android still doesn't support the Web Codecs API [1], so I need to use the "jpeg" codec on Selkies remote desktop sites, which I assume is rather poor for my bandwidth and battery.

[0] https://github.com/mozilla/standards-positions/issues/238