FFMetrics VS videobench

FFMetrics.

You mean this? But he seems to be using a 4.x version of FFmpeg. That's ancient history at this point, since 4.0 was released in 2018, and 5.0 and 6.0 have been released since. And the dav1d in his FFmpeg build is 2+ years old as well, which means it's even pre-1.0. No wonder basic shit doesn't work.

You could then use applications such as HandBrake (encoding) and FFmetrics (VMAF), or better ShutterEncoder (supports both encoding and VMAF) to first encode a few test clips and then calculate the VMAF score. Anything above 95% will be visually lossless in motion. 93% is lossless to many people too (not to me though) and 90% is about the quality you get on YouTube - so absolutely terrible. If you ever want to generate an FFmpeg encoding command, FastFlix is also nice.

https://github.com/fifonik/FFMetrics is a good place to start.

Someone tried your suggestions and apparently, at least with the current FFmpeg versions, the flag doesn't affect VMAF (see here). I can only conclude VMAF does already return the highest values possible with AV1 as is, though what you said made a lot of sense so am unsure where exactly something changed from you gaining your knowledge to the current state of VMAF in FFmpeg (I'm a little puzzled).

https://github.com/fifonik/FFMetrics is a gui to compare your own outputs if you're too lazy to learn the cli for a bunch of different programs.

I recommend you do some tests I'd your own with FFMetrics.

Netflix even has a method to measure perceivable quality difference. A little while ago I used this tool to calculate this.

Sidenote, you won't get an "real" answer by just visually inspecting them. There are tools like https://github.com/JNoDuq/videobench and https://github.com/fifonik/FFMetrics which makes running tests easy and boils quality down to number.