web-codecs VS lossless-cut

Encoding alpha, please! https://github.com/w3c/webcodecs/issues/672

From developer.mozilla.org [0]

> WebCodecs API

> The WebCodecs API gives web developers low-level access to the individual frames of a video stream and chunks of audio. It is useful for web applications that require full control over the way media is processed. For example, video or audio editors, and video conferencing.

And from w3c [1]:

> The WebCodecs API allows web applications to encode and decode audio and video

All this looks really promising, I wouldn't have thought that we could use browsers directly to render videos. Maybe Puppeteer could then stream the content of the page it is rendering, for example a three.js animation.

[0] https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_A...

[1] https://github.com/w3c/webcodecs

As I understand it, Web Codecs is quite a ways away from being a web standard - it is currently just a draft[1] for a recommendation for a possible future standard.

Just beware of any major API changes or any indication that it might be dropped in its entirety!

[1]:https://w3c.github.io/webcodecs/

WebCodecs has Mozilla and Microsoft coauthors: https://w3c.github.io/webcodecs/

WebGPU was chartered in 2017, with more Apple people in the initial membership than Mozilla or Google people: https://gpuweb.github.io/admin/cg-charter.html

Google specifically has approached Apple for input about scheduler.postTask(), and they say they've presented the proposal before at a working group that Apple is part of: https://lists.webkit.org/pipermail/webkit-dev/2021-June/0319...

I won't disagree that there's a ton that can be improved about the process, but it seems pretty unsubstantiated to claim that Google is "rushing forwards as fast as possible" given that they've been spending years working on these APIs in public. You can see from the rest of the post that they're running some experiments which they're specifically choosing not to enable by default, though they could.

I work for PaperlessPost.com, and for our Flyer product we export mp4s from ffmpeg compiled to WASM, which is similar to ffmpeg.js but optimized for our use case which sounds similar. We have this WebAssembly method working well but iOS 15 (and the other major browsers) now support MediaExporter which might be a better way to go if you have something else to convert these files. This becomes and issue because you can't control the format MediaRecorder is recording to but the management of memory, the image quality, the compression, the performance hit will be more ideal than these other methods. OMGgif is very slow and will produce large files or very bad looking ones. Keep in mind that GIFs limit the colors so something that looks nice on the screen might not look the same after it is saved. The other thing to look out for is the WebCodecs APIs which should be the ultimate way to handle all of this in the future but it is only working in Chrome I think https://github.com/w3c/webcodecs

While in theory it might be possible to run ffmpeg through emscripten and run it in a web worker (I certainly assume someone has done this), it's not necessarily practical to try and use the same technique from node to transcode audio on the web. The good news is that the w3c has chartered a working group to focus on web codecs. While this is at the time of writing still in early stages, the working group is powering ahead on designing and proposing an API to enable media transcoding on the web, and hopefully that will become available to us in the near future.

For lossless cut, there's the LosslessCut [1] app, which even has an experimental but mostly working version of a "smart cut" feature [2] (aka. only re-encode the minimal mandatory amount of frames if you trim at a point between 2 key frames)

[1]: https://github.com/mifi/lossless-cut

[2]: https://github.com/mifi/lossless-cut/issues/126

> thought I cropped/edited a video, it is still there in full length and resolution

It's been possible to create a clip from a video file that merely changes what parts of the video are displayed without effecting the data in the original since the Classic Mac OS days.

If you want to completely remove unwanted portions of a video to reduce the size without a loss of quality, there are many options. LosslessCut is one option that is both free and open source.

https://github.com/mifi/lossless-cut

LosslessCut

Lossless Cut is probably the simplest way to cut out parts of a video without having to re-encode it or use an online service. It is open source, free and available for all platforms. It is in essence a frontend for FFMPEG and cuts without re-encoding, so the results are instantaneous.

Additionally I would like if possible to shorten (both to reduce dimension and to make them less boring) my videos by trimming the beginning and/or ends where often there is nothing interesting happening and possibly change the thumbnails for easier identification. Do you know of any tool that does the job with the minimum quality degradation possible? Many years ago I've tried with some videos and a tool I don't remember anymore, ending up with a bunch of corrupted files :( I've found https://github.com/mifi/lossless-cut for the point above.. need to try it. please let me know what experience you have with it.

Lossless Cut would probably be a better choice for this. Will let you cut out the commercials without having to re-encode the video.

>Of course, you could only re-encode only the GoPs that get broken while keeping the rest intact, and I guess this would be better and a lot faster than re-encoding everything. I don't know if any application tries to do this.

LosslessCut does have experimental support for this partial re-encode called "smart cut" [1]. Since it's using ffmpeg internally, the challenge become how to instruct ffmpeg to do this[2]?

[1]: https://github.com/mifi/lossless-cut/issues/126