diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2017-05-14 14:06:56 +0200 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2017-05-20 03:41:33 +0200 |
| commit | 12f969bd2c572c882fc2a1172582129543fa9f89 (patch) | |
| tree | 34d2b5e7fbdbab37b17f7592d350381732cb502a /libavcodec | |
| parent | 42f2cf15976795ee9ba7752b616e560448baa8c8 (diff) | |
| download | ffmpeg-12f969bd2c572c882fc2a1172582129543fa9f89.tar.gz | |
avcodec/hqxdsp: Fix runtime error: signed integer overflow: -196264 * 11585 cannot be represented in type 'int'
Fixes: 1568/clusterfuzz-testcase-minimized-5944868608147456
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b923213276777f33d6366b1cb9d1845a8658f365)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'libavcodec')
| -rw-r--r-- | libavcodec/hqxdsp.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/libavcodec/hqxdsp.c b/libavcodec/hqxdsp.c index 04a65e7767..7f8044e463 100644 --- a/libavcodec/hqxdsp.c +++ b/libavcodec/hqxdsp.c @@ -47,8 +47,8 @@ static inline void idct_col(int16_t *blk, const uint8_t *quant) t5 = t1 * 2 + t3; t6 = t2 - t3; t7 = t3 * 2 + t6; - t8 = (t6 * 11585) >> 14; - t9 = (t7 * 11585) >> 14; + t8 = (int)(t6 * 11585U) >> 14; + t9 = (int)(t7 * 11585U) >> 14; tA = (int)(s2 * 8867U - s6 * 21407U) >> 14; tB = (int)(s6 * 8867U + s2 * 21407U) >> 14; tC = (s0 >> 1) - (s4 >> 1); |