diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2018-06-21 23:08:32 +0200 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2018-07-16 19:16:13 +0200 |
| commit | eedde18f1a8f5cb32883d0bf529f400b82d80d1c (patch) | |
| tree | a0c8605f4d13e8dffef25420bf5a923e4dd0897d | |
| parent | 3db270a0786207fc5692581ae2fe3f28b917e430 (diff) | |
| download | ffmpeg-eedde18f1a8f5cb32883d0bf529f400b82d80d1c.tar.gz | |
avcodec/ra144: Fix integer overflow in ff_eval_refl()
Fixes: signed integer overflow: -4096 * -524288 cannot be represented in type 'int'
Fixes: 8650/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RA_144_fuzzer-5734816036159488
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b31189881a4cf54b0057ecf3eab917ad56eecfea)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
| -rw-r--r-- | libavcodec/ra144.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/libavcodec/ra144.c b/libavcodec/ra144.c index c077b7b327..cf8127c236 100644 --- a/libavcodec/ra144.c +++ b/libavcodec/ra144.c @@ -1569,11 +1569,11 @@ int ff_eval_refl(int *refl, const int16_t *coefs, AVCodecContext *avctx) b = 0x1000000 / b; for (j=0; j <= i; j++) { #if CONFIG_FTRAPV - int a = bp2[j] - ((refl[i+1] * bp2[i-j]) >> 12); + int a = bp2[j] - ((int)(refl[i+1] * (unsigned)bp2[i-j]) >> 12); if((int)(a*(unsigned)b) != a*(int64_t)b) return 1; #endif - bp1[j] = (int)((bp2[j] - ((refl[i+1] * bp2[i-j]) >> 12)) * (unsigned)b) >> 12; + bp1[j] = (int)((bp2[j] - ((int)(refl[i+1] * (unsigned)bp2[i-j]) >> 12)) * (unsigned)b) >> 12; } if ((unsigned) bp1[i] + 0x1000 > 0x1fff) |