video-compare VS video-compress

As mentioned those are different frames you are testing. Personally I use veryslow but most people here don't spend as much time encoding a single item as I do. Also don't use PNGs for checking use Video Compare . You can run two different MKVs at the same time and compare them frame by frame both in playback and pausing them.

use the Video-Compare tool. watch the demo here how it works

https://github.com/pixop/video-compare This one is simple but really good

Well, I would use https://github.com/pixop/video-compare for that - it's the tool I normally use to compare videos (to find differences or to check image compression issues) but it can load the same video twice to watch it frame by frame. It even shows the type of frame I'm on so it's useful when I need to find a keyframe to do a video split with no recoding.

You can preview (export as preview) test of few presets, settings, etc. on TVAI and if you wish you could also use a compare tool to check the results to see for yourself if you notice any difference between them (that would save you tons of time). or just playback the 2s / 5s of the preview(s) you generate, right after the other to see if you spot any difference in quality.

If you need to compare images easily, consider slow.pics. Or ICAT, or Video Comparison Tool. And actually, even x264 and x265 don't have perfect CRF scales as well, they are offset by a constant iirc.

You can easily check the type of each frame with the use of https://github.com/pixop/video-compare/ software - just use the same video twice (as video-compare needs two video inputs) and look for I, B, P letters near the current frame time.

Try Video compare.

In case it's helpful at all, here's the program in question.