video-compare VS FFMetrics

As mentioned those are different frames you are testing. Personally I use veryslow but most people here don't spend as much time encoding a single item as I do. Also don't use PNGs for checking use Video Compare . You can run two different MKVs at the same time and compare them frame by frame both in playback and pausing them.

use the Video-Compare tool. watch the demo here how it works

https://github.com/pixop/video-compare This one is simple but really good

Well, I would use https://github.com/pixop/video-compare for that - it's the tool I normally use to compare videos (to find differences or to check image compression issues) but it can load the same video twice to watch it frame by frame. It even shows the type of frame I'm on so it's useful when I need to find a keyframe to do a video split with no recoding.

You can preview (export as preview) test of few presets, settings, etc. on TVAI and if you wish you could also use a compare tool to check the results to see for yourself if you notice any difference between them (that would save you tons of time). or just playback the 2s / 5s of the preview(s) you generate, right after the other to see if you spot any difference in quality.

If you need to compare images easily, consider slow.pics. Or ICAT, or Video Comparison Tool. And actually, even x264 and x265 don't have perfect CRF scales as well, they are offset by a constant iirc.

You can easily check the type of each frame with the use of https://github.com/pixop/video-compare/ software - just use the same video twice (as video-compare needs two video inputs) and look for I, B, P letters near the current frame time.

Try Video compare.

FFMetrics.

You mean this? But he seems to be using a 4.x version of FFmpeg. That's ancient history at this point, since 4.0 was released in 2018, and 5.0 and 6.0 have been released since. And the dav1d in his FFmpeg build is 2+ years old as well, which means it's even pre-1.0. No wonder basic shit doesn't work.

You could then use applications such as HandBrake (encoding) and FFmetrics (VMAF), or better ShutterEncoder (supports both encoding and VMAF) to first encode a few test clips and then calculate the VMAF score. Anything above 95% will be visually lossless in motion. 93% is lossless to many people too (not to me though) and 90% is about the quality you get on YouTube - so absolutely terrible. If you ever want to generate an FFmpeg encoding command, FastFlix is also nice.

https://github.com/fifonik/FFMetrics is a good place to start.

Someone tried your suggestions and apparently, at least with the current FFmpeg versions, the flag doesn't affect VMAF (see here). I can only conclude VMAF does already return the highest values possible with AV1 as is, though what you said made a lot of sense so am unsure where exactly something changed from you gaining your knowledge to the current state of VMAF in FFmpeg (I'm a little puzzled).

https://github.com/fifonik/FFMetrics is a gui to compare your own outputs if you're too lazy to learn the cli for a bunch of different programs.

I recommend you do some tests I'd your own with FFMetrics.