tests/swscale: constrain reference SSIM for low bit depth formats

Sometimes, the reference SSIM is significantly higher than the
SSIM level expected for the test. This is the case when the source format
has a much lower bit depth than the destination format. In this case, the fact
that legacy swscale does not accurately preserve the source dither pattern
gives it an unfair advantage in a direct comparison, leading to false
positives.

For example, conversion like rgb4 -> rgb565 should be lossless, but swscale
low passes / downscales the input chroma, throwing away massive amounts of
detail. This gives it a higher SSIM score since the lowpassed result removes
some of the dither noise that was present in the source.

1 files changed, 12 insertions, 0 deletions

diff --git a/libswscale/tests/swscale.c b/libswscale/tests/swscale.c
index 432de7d1fd..d3c0cfdc43 100644
--- a/libswscale/tests/swscale.c
+++ b/libswscale/tests/swscale.c
@@ -321,6 +321,18 @@ static int run_test(enum AVPixelFormat src_fmt, enum AVPixelFormat dst_fmt,
             goto error;
 
         get_ssim(ssim_sws, out, ref, comps);
+
+        /* Legacy swscale does not perform bit accurate upconversions of low
+         * bit depth RGB. This artificially improves the SSIM score because the
+         * resulting error deletes some of the input dither noise. This gives
+         * it an unfair advantage when compared against a bit exact reference.
+         * Work around this by ensuring that the reference SSIM score is not
+         * higher than it theoretically "should" be. */
+        if (src_var > dst_var) {
+            const float src_loss = (2 * ref_var + c1) / (2 * ref_var + src_var + c1);
+            ssim_sws[0] = FFMIN(ssim_sws[0], src_loss);
+        }
+
         ssim_ref = ssim_sws;
     }