libheif VS enable-chromium-hevc-hardware-deco

Apple doesn't own HEIF, there are open implementations for reading all variants of it. Here's a browser-side implementation: https://alexcorvi.github.io/heic2any/

The image crate is the de facto crate for raster images. It doesn't have HEIF support, which is probably why libheif-rs was brought in (it's simply a wrapper for libheif, which is what lots of programs, including GIMP and ImageMagick, use for HEIF support).

Libheif has Go bindings https://github.com/strukturag/libheif/tree/master/go/heif. I used this library for AVIF (via libaom) but it should work for HEIF (via libde265.)

libheif provides a pixbuf loader that should make it work.

Note that there is no checking for valid parameters when using the prefix. BUGS Please reports bugs or issues at https://github.com/strukturag/libheif AUTHORS Dirk Farin, struktur AG COPYRIGHT Copyright © 2017 struktur AG

They use https://github.com/strukturag/libheif to decode it and just ignore the patent or licensing requirements. This is similar to how vlc's libavcodec implements HEVC/h264 (they say "we're french, software patents have no jurisdiction here")

As far as I understand, imagemagick uses libheif under the hood, which can be installed with apt install libheif-dev

Most devices can indeed most likely handle software decode of more common resolutions, codecs and bitrates. But I'd really hope they'd pick the one that won't suck up all the battery, so H264. This line of thought is supported by the fact that YouTube still provides an H264 option with most if not all videos.

With higher bitrate things, HEVC seems to grow in popularity but even software decode support is not everywhere. Netflix for example requires the installation of HEVC support on Windows to play 4K content.

Actually hardware-accelerated video decode is even spottier and more unreliable across most platforms. The JS API for codec support (canPlayType) literally returns "maybe" and "probably". It's quite bad.

So far the best compatibility I've seen has been Edge with flags on Windows (MPEG-2, H264, HEVC, AV1, VP8, VP9 with most also supporting accelerated encode). It still fails with some content (Dolby Vision P5 colors are incorrect, HEVC Rext doesn't play - more info about HEVC is available here https://github.com/StaZhu/enable-chromium-hevc-hardware-deco...). Chrome on macOS is a close second in terms of codec support.

The worst in terms of HW acceleration being all the browsers on desktop Linux-s, few and fragile combinations that offer limited and janky support. But it's slowly improving. This combined with the not-the-latest hardware many use, means things like VP9 or AV1 tend to stutter.

I'd love to see some more generic stats, but considering the APIs aren't sufficient to determine actual support, these might be difficult to gather.

> The 'right' solution would be to just use system codecs for everything. Many apps need good implementations of image codecs. They just need to be implemented once by the OS vendor (or the toolkit on Linux).

Windows has done this and is still doing this, but the decade-long track history so far is that this does not work well. It can work, in a very limited scope and if you have a lot of influence.

Sure, it's really nice if an 8K@60Hz HDR HEVC video plays perfectly straight in your browser or desktop app, but more often than not, it just won't. You don't have the right browser, the extension installed (due to license agreements), good enough graphics drivers or someone has forgotten a flag yet again.

And we haven't even gotten to the immense amount of variation each codec introduces or the potential attack surface.

How shit the situation is with just HEVC (and thus also basically HEIC): https://github.com/StaZhu/enable-chromium-hevc-hardware-deco...

> Just file a bug against your OS.

In the end that "just" carries a lot of burden, it can't be the users reporting these issues.

It's just way easier to leech off of ffmpeg and similar, and let it deal with all the formats. Instead of hoping that maybe you can leverage what the OS gives you, that it works and works correctly in all your edge-cases.

Though not everything is that gloomy, there are Vulkan extensions that might (in the future) simplify cross-platform image and video decoding (and HW acceleration).

I think there are a few patches that can enable HEVC hardware decoding with chromium. Though I am a firefox user so I didn't test whether these patches works or not. https://github.com/StaZhu/enable-chromium-hevc-hardware-deco...