mediacapture-record VS ffmpeg.wasm

Courtesy of this GitHub thread, my currently working, but very inconvenient strategy to capture the video is to use mediaCapture(0) on the canvas element with MediaStreamTrackProcessor to pipe frames into WebM muxer from a package called webm-muxer.

WebRTC's RTCRtpSender.replaceTrack() method achieves "seamless" replacement of a MediaStreamTrack. I proposed the same be added to MediaRecorder, see Add replaceTrack method to MediaRecorder., Add replaceStream to MediaRecorder.

I guess this could be used to remux the broken files that are spitted by the MediaRecorder API, which have missing metadata that prevents from seeking and thus far has been ignored / sweeped away in Chrome [1], Firefox [2], and even the standard itself [3], which ignored in its design the basic fact that encoding any file should include a "closing" stage (where metadata is written) before yielding it as a finished file.

[1]: https://bugs.chromium.org/p/chromium/issues/detail?id=642012

[2]: https://bugzilla.mozilla.org/show_bug.cgi?id=1283464

[3]: https://github.com/w3c/mediacapture-record/issues/119

There's already ffmpeg wasm. I've used it in projects. Works great.

https://github.com/ffmpegwasm/ffmpeg.wasm

There's a low-hanging fruit that I think would make ffmpeg more helpful for regular people.

There's a million terrible websites that offer file conversion services. They're ad-ridden, with god-knows-what privacy/security postures. There's little reason for users to need to upload their files to a third-party when they can do it locally. But getting them to download fiddly technical software is tough - and they're right to mistrust it.

So, there's a WASM version of ffmpeg, already working and hosted at Netlify [1]. It downloads the WASM bundle to your browser and you can run conversions/transformations as you wish, in your browser. Sandboxed and pretty performant too!

If this tool a) was updated regularly b) had a nicer, non-CLI UI for everyday users and c) was available at an easily-Googlable domain name - it would solve all the problems I mentioned above.

[1]: https://ffmpegwasm.netlify.app/

As their github page says, based on https://ffmpegwasm.netlify.app ...

I'm guessing no one did GPU-optimizations? I saw a web app (not an ffmpeg transpilation) that went clever and used WebGL so it can access the GPU and use its parallel processing capabilities...

ffmpeg.wasm: The engine making all these happen. However, I’m a bit concerned about its current situation. The repo has not been updated for more than six months now, and that’s not a healthy sign for an open-source project. Clearly I was reading the commits wrong. The author mentioned “speed up x264 with SIMD intrinsics” in their roadmap (https://github.com/ffmpegwasm/ffmpeg.wasm/discussions/415), which, if landed, could make such on-device video editors much more competitive.

I use https://github.com/ffmpegwasm/ffmpeg.wasm and I want my FFMPEG commands to be hidden from others.

Possibly using this? https://github.com/ffmpegwasm/ffmpeg.wasm

You could try using something like ffmpeg wasm which is a way of using ffmpeg client side in browser. Unfortunately WebAssembly only supports files less than 2 gigabytes, which is a problem for videos. And I don't know if ffmpeg wasm contains ffprobe, so you might have to find another project or try to compile ffprobe to wasm yourself. This stuff is out of my wheelhouse so I can't offer much help.