diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2023-09-15 00:49:41 +0200 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2024-07-23 23:21:16 +0200 |
| commit | 56c334d732dbbce43b0c8fc0809ec545b7946832 (patch) | |
| tree | d8cfe16e10035181b64c59e588733701a431d0b8 /libavcodec/osq.c | |
| parent | 3cd077e2820679e8b9f8eb10954b4f5701191c48 (diff) | |
| download | ffmpeg-56c334d732dbbce43b0c8fc0809ec545b7946832.tar.gz | |
avcodec/osq: avoid using too large numbers for shifts and integers in update_residue_parameter()
Fixes: 2.96539e+09 is outside the range of representable values of type 'int'
Fixes: Assertion n>=0 && n<=32 failed at libavcodec/get_bits.h:423
Fixes: 62241/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_OSQ_fuzzer-4525761925873664
Fixes: 70406/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_OSQ_fuzzer-6545326804434944
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'libavcodec/osq.c')
| -rw-r--r-- | libavcodec/osq.c | 10 |
1 files changed, 7 insertions, 3 deletions
diff --git a/libavcodec/osq.c b/libavcodec/osq.c index 1bd4485f07..42a46b25ce 100644 --- a/libavcodec/osq.c +++ b/libavcodec/osq.c @@ -161,11 +161,15 @@ static int update_residue_parameter(OSQChannel *cb) sum = cb->sum; x = sum / cb->count; - rice_k = av_ceil_log2(x); + rice_k = ceil(log2(x)); if (rice_k >= 30) { - rice_k = floor(sum / 1.4426952 + 0.5); - if (rice_k < 1) + double f = floor(sum / 1.4426952 + 0.5); + if (f <= 1) { rice_k = 1; + } else if (f >= 31) { + rice_k = 31; + } else + rice_k = f; } return rice_k; |