diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2017-05-21 02:12:21 +0200 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2017-06-02 01:14:38 +0200 |
| commit | d11c686204b40921152fa6fd56e6b4b171ea39c5 (patch) | |
| tree | 5a3b98520dced2f32a96bc53c17a933c7dac6276 | |
| parent | 0ea475942e751af3aa98021d226210104aeb8818 (diff) | |
| download | ffmpeg-d11c686204b40921152fa6fd56e6b4b171ea39c5.tar.gz | |
avcodec/vp9block: fix runtime error: signed integer overflow: 196675 * 20670 cannot be represented in type 'int'
Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d4ee76780869c659a5d3b0815c56024ab260a81d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
| -rw-r--r-- | libavcodec/vp9block.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c index ae2f0e4c6f..a16ccdccdb 100644 --- a/libavcodec/vp9block.c +++ b/libavcodec/vp9block.c @@ -915,9 +915,9 @@ skip_eob: if (!--band_left) band_left = band_counts[++band]; if (is_tx32x32) - STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) * qmul[!!i]) / 2); + STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val : val) * (unsigned)qmul[!!i]) / 2); else - STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * qmul[!!i]); + STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * (unsigned)qmul[!!i]); nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1; tp = p[band][nnz]; } while (++i < n_coeffs); |