diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2017-05-10 01:26:39 +0200 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2017-05-20 03:41:33 +0200 |
| commit | 4dacf6bc4ac2a8f244fa89f010ef5f4955163eee (patch) | |
| tree | 8e0d20f45421f5dbf31c26705b9657e7189b8c4a /libavcodec | |
| parent | 43bdff201d60d1482b6d0655f7916ccb2faa92be (diff) | |
| download | ffmpeg-4dacf6bc4ac2a8f244fa89f010ef5f4955163eee.tar.gz | |
avcodec/svq3: Fix multiple runtime error: signed integer overflow: -237341 * 24552 cannot be represented in type 'int'
Fixes: 1429/clusterfuzz-testcase-minimized-5959951610544128
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ae6fd1790f48c457a8cedb445dcac73f8f7b7698)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'libavcodec')
| -rw-r--r-- | libavcodec/svq3.c | 16 |
1 files changed, 8 insertions, 8 deletions
diff --git a/libavcodec/svq3.c b/libavcodec/svq3.c index 21e38f00a0..a862927936 100644 --- a/libavcodec/svq3.c +++ b/libavcodec/svq3.c @@ -224,16 +224,16 @@ void ff_svq3_add_idct_c(uint8_t *dst, int16_t *block, } for (i = 0; i < 4; i++) { - const int z0 = 13 * (block[i + 4 * 0] + block[i + 4 * 2]); - const int z1 = 13 * (block[i + 4 * 0] - block[i + 4 * 2]); - const int z2 = 7 * block[i + 4 * 1] - 17 * block[i + 4 * 3]; - const int z3 = 17 * block[i + 4 * 1] + 7 * block[i + 4 * 3]; + const unsigned z0 = 13 * (block[i + 4 * 0] + block[i + 4 * 2]); + const unsigned z1 = 13 * (block[i + 4 * 0] - block[i + 4 * 2]); + const unsigned z2 = 7 * block[i + 4 * 1] - 17 * block[i + 4 * 3]; + const unsigned z3 = 17 * block[i + 4 * 1] + 7 * block[i + 4 * 3]; const int rr = (dc + 0x80000); - dst[i + stride * 0] = av_clip_uint8(dst[i + stride * 0] + ((z0 + z3) * qmul + rr >> 20)); - dst[i + stride * 1] = av_clip_uint8(dst[i + stride * 1] + ((z1 + z2) * qmul + rr >> 20)); - dst[i + stride * 2] = av_clip_uint8(dst[i + stride * 2] + ((z1 - z2) * qmul + rr >> 20)); - dst[i + stride * 3] = av_clip_uint8(dst[i + stride * 3] + ((z0 - z3) * qmul + rr >> 20)); + dst[i + stride * 0] = av_clip_uint8(dst[i + stride * 0] + ((int)((z0 + z3) * qmul + rr) >> 20)); + dst[i + stride * 1] = av_clip_uint8(dst[i + stride * 1] + ((int)((z1 + z2) * qmul + rr) >> 20)); + dst[i + stride * 2] = av_clip_uint8(dst[i + stride * 2] + ((int)((z1 - z2) * qmul + rr) >> 20)); + dst[i + stride * 3] = av_clip_uint8(dst[i + stride * 3] + ((int)((z0 - z3) * qmul + rr) >> 20)); } memset(block, 0, 16 * sizeof(int16_t)); |