diff options
| author | Mark Reid <mindmark@gmail.com> | 2020-10-03 16:31:58 -0700 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2020-10-06 17:56:52 +0200 |
| commit | a48adcd1369131ebadd1c928fe790c72795182a7 (patch) | |
| tree | a30fec676173eeaa7e5fc7b89655e3d9456a101e /tests/ref/fate/sws-floatimg-cmp | |
| parent | 214998c55ff99586b57c1ccd34a7e6f15c4fcdb8 (diff) | |
| download | ffmpeg-a48adcd1369131ebadd1c928fe790c72795182a7.tar.gz | |
libswcale/input: use more accurate planer rgb16 yuv conversions
These conversion appears to be exhibiting the same rounding error as the rgbf32 formats where.
I seperated the rounding value from the 16 and 128 offsets, I think it makes it a little more clear.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'tests/ref/fate/sws-floatimg-cmp')
| -rw-r--r-- | tests/ref/fate/sws-floatimg-cmp | 32 |
1 files changed, 16 insertions, 16 deletions
diff --git a/tests/ref/fate/sws-floatimg-cmp b/tests/ref/fate/sws-floatimg-cmp index cf6788fc23..251042f1c3 100644 --- a/tests/ref/fate/sws-floatimg-cmp +++ b/tests/ref/fate/sws-floatimg-cmp @@ -87,34 +87,34 @@ avg diff: 0.001011 min diff: 0.000000 max diff: 0.004229 gbrpf32le -> gbrp9le -> gbrpf32le -avg diff: 0.003917 +avg diff: 0.000545 min diff: 0.000000 -max diff: 0.007870 +max diff: 0.002245 gbrpf32le -> gbrp10le -> gbrpf32le -avg diff: 0.003841 +avg diff: 0.000350 min diff: 0.000000 -max diff: 0.007456 +max diff: 0.001475 gbrpf32le -> gbrap10le -> gbrpf32le -avg diff: 0.003841 +avg diff: 0.000350 min diff: 0.000000 -max diff: 0.007456 +max diff: 0.001475 gbrpf32le -> gbrp12le -> gbrpf32le -avg diff: 0.003796 +avg diff: 0.000260 min diff: 0.000000 -max diff: 0.007140 +max diff: 0.001135 gbrpf32le -> gbrap12le -> gbrpf32le -avg diff: 0.003796 +avg diff: 0.000260 min diff: 0.000000 -max diff: 0.007140 +max diff: 0.001135 gbrpf32le -> gbrp14le -> gbrpf32le -avg diff: 0.003792 +avg diff: 0.000253 min diff: 0.000000 -max diff: 0.007034 +max diff: 0.001068 gbrpf32le -> gbrp16le -> gbrpf32le -avg diff: 0.003853 +avg diff: 0.000249 min diff: 0.000000 -max diff: 0.007098 +max diff: 0.000990 gbrpf32le -> gbrap16le -> gbrpf32le -avg diff: 0.003853 +avg diff: 0.000249 min diff: 0.000000 -max diff: 0.007098 +max diff: 0.000990 |