diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2020-02-02 21:01:12 +0100 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2020-07-01 12:49:26 +0200 |
| commit | 5e9fa514a451dacb254bd8512486d95b83d9fabb (patch) | |
| tree | cefd185a97e9692747a56e862eefd9942ee60ecf /libavcodec | |
| parent | ff3ea90d6b68071df7a49195fb459921fca6c9b4 (diff) | |
| download | ffmpeg-5e9fa514a451dacb254bd8512486d95b83d9fabb.tar.gz | |
avcodec/rv40dsp: Fix integer overflows in rv40_weight_func_*()
Fixes: signed integer overflow: 40550400 * 128 cannot be represented in type 'int'
Fixes: 20331/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RV40_fuzzer-5676685725007872
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13171ad2e304b2a7d959429527b98c68ec5ea320)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'libavcodec')
| -rw-r--r-- | libavcodec/rv40dsp.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/libavcodec/rv40dsp.c b/libavcodec/rv40dsp.c index 95ba0a9259..cdc61fc700 100644 --- a/libavcodec/rv40dsp.c +++ b/libavcodec/rv40dsp.c @@ -379,7 +379,7 @@ static void rv40_weight_func_rnd_ ## size (uint8_t *dst, uint8_t *src1, uint8_t \ for (j = 0; j < size; j++) {\ for (i = 0; i < size; i++)\ - dst[i] = (((w2 * src1[i]) >> 9) + ((w1 * src2[i]) >> 9) + 0x10) >> 5;\ + dst[i] = ((((unsigned)w2 * src1[i]) >> 9) + (((unsigned)w1 * src2[i]) >> 9) + 0x10) >> 5;\ src1 += stride;\ src2 += stride;\ dst += stride;\ @@ -391,7 +391,7 @@ static void rv40_weight_func_nornd_ ## size (uint8_t *dst, uint8_t *src1, uint8_ \ for (j = 0; j < size; j++) {\ for (i = 0; i < size; i++)\ - dst[i] = (w2 * src1[i] + w1 * src2[i] + 0x10) >> 5;\ + dst[i] = ((unsigned)w2 * src1[i] + (unsigned)w1 * src2[i] + 0x10) >> 5;\ src1 += stride;\ src2 += stride;\ dst += stride;\ |