diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2020-10-26 21:30:19 +0100 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2022-04-07 16:27:00 +0200 |
| commit | 4a7f3467d800c096951a78a38088fdbf35ffcd83 (patch) | |
| tree | 7288c7eb43be51027fb81fa4529ee4cb00dcc17a | |
| parent | f79f5a97bdc563c629c1a9a64e5ac00cf53f0ca9 (diff) | |
| download | ffmpeg-4a7f3467d800c096951a78a38088fdbf35ffcd83.tar.gz | |
avcodec/pixlet: Avoid signed integer overflow in scaling in filterfn()
Fixes: signed integer overflow: 11494 * 1073741824000000 cannot be represented in type 'long'
Fixes: 26586/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PIXLET_fuzzer-5752633970917376
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c1f20c6c858b753effda274b58ef635d1924915)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
| -rw-r--r-- | libavcodec/pixlet.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/libavcodec/pixlet.c b/libavcodec/pixlet.c index 6cb6516227..937076b126 100644 --- a/libavcodec/pixlet.c +++ b/libavcodec/pixlet.c @@ -404,7 +404,7 @@ static void filterfn(int16_t *dest, int16_t *tmp, unsigned size, int64_t scale) (int64_t) low [i - 1] * -INT64_C(325392907) + (int64_t) high[i + 0] * INT64_C(1518500249) + (int64_t) high[i - 1] * INT64_C(1518500249); - dest[i * 2] = av_clip_int16(((value >> 32) * scale) >> 32); + dest[i * 2] = av_clip_int16(((value >> 32) * (uint64_t)scale) >> 32); } for (i = 0; i < hsize; i++) { @@ -415,7 +415,7 @@ static void filterfn(int16_t *dest, int16_t *tmp, unsigned size, int64_t scale) (int64_t) high[i + 1] * INT64_C(303700064) + (int64_t) high[i + 0] * -INT64_C(3644400640) + (int64_t) high[i - 1] * INT64_C(303700064); - dest[i * 2 + 1] = av_clip_int16(((value >> 32) * scale) >> 32); + dest[i * 2 + 1] = av_clip_int16(((value >> 32) * (uint64_t)scale) >> 32); } } |