diff options
author | Andreas Rheinhardt <andreas.rheinhardt@outlook.com> | 2022-07-11 00:46:16 +0200 |
---|---|---|
committer | Andreas Rheinhardt <andreas.rheinhardt@outlook.com> | 2022-07-23 22:53:29 +0200 |
commit | 1cb7fd317c84117bbb13b14851d62f77f57bb9ce (patch) | |
tree | a4e5f134cb9a887aa66953de6df2444dba29c1eb /libavcodec/binkdsp.h | |
parent | 890efee2b80d5b3b5eb087d1f2a472854f66afe3 (diff) | |
download | ffmpeg-1cb7fd317c84117bbb13b14851d62f77f57bb9ce.tar.gz |
avcodec/svq1enc: Use unsigned for parameter >= 0 to workaround GCC bug
encode_block() in svq1enc.c looks like the following:
static int encode_block(int block[7][256], int level)
{
int best_score = 0;
for (unsigned x = 0; x < level; x++) {
int v = block[1][x];
block[level][x] = 0;
best_score += v * v;
}
if (level > 0 && best_score > 64) {
int score = 0;
score += encode_block(block, level - 1);
score += encode_block(block, level - 1);
if (score < best_score) {
best_score = score;
}
}
return best_score;
}
When called from outside of encode_block(), it is always called with
level == 5.
This triggers a bug [1] in GCC: On -O3, it creates eight clones of
encode_block with different values of level inlined into it. The clones
with negative values are of course useless*, but they also lead to
-Warray-bounds warnings, because they access block[-1].
This has been mitigated in GCC 12: It no longer creates clones
for parameters that it knows are impossible. Somehow switching levels
to unsigned makes GCC know this. Therefore this commit does this.
(For GCC 11, this changes the warning to "array subscript 4294967295 is
above array bounds" from "array subscript -1 is below array bounds".)
[1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102513
*: These clones can actually be discarded when compiling with
-ffunction-sections.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Diffstat (limited to 'libavcodec/binkdsp.h')
0 files changed, 0 insertions, 0 deletions