diff options
| author | Andreas Rheinhardt <andreas.rheinhardt@gmail.com> | 2021-01-24 21:56:36 +0100 |
|---|---|---|
| committer | Andreas Rheinhardt <andreas.rheinhardt@gmail.com> | 2021-01-24 22:37:55 +0100 |
| commit | 2bcec40ccee6a848db4034c4d096a44bf10db957 (patch) | |
| tree | 3c71c1db3e436a4eeceaf131a995e6bd26793641 /libavfilter/vf_nnedi.c | |
| parent | 2687070d9b092d3a354a6963c65197054ddf7a75 (diff) | |
| download | ffmpeg-2bcec40ccee6a848db4034c4d096a44bf10db957.tar.gz | |
avfilter/vf_nnedi: Fix segfault when prescreening is disabled
Since c737f6edcef74a64f4d0ebcefa970bd31266d512 prescreening is
nevertheless run because of a wrong check: "if (s->prescreen > 0)".
s->prescreen is an array of two function pointers that is contained in
the context and comparing it with 0 (i.e. NULL) is actually undefined
behaviour, because NULL and s->prescreen do not point to the same
object (NULL after all never points to any object). Nevertheless both
Clang as well as GCC compile this to code that treat s->prescreen > 0 as
true, leading to segfaults, because the code then tries to access the
-1th member of an array.
This commit fixes the check as well as another such check a few lines
below.
(Found via compiler warnings enabled by -pedantic:
"ordered comparison between pointer and zero is an extension".)
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Diffstat (limited to 'libavfilter/vf_nnedi.c')
| -rw-r--r-- | libavfilter/vf_nnedi.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/libavfilter/vf_nnedi.c b/libavfilter/vf_nnedi.c index 1462ce1042..4229150668 100644 --- a/libavfilter/vf_nnedi.c +++ b/libavfilter/vf_nnedi.c @@ -637,7 +637,7 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) width, 1, in_scale); for (int y = 0; y < slice_end - slice_start; y += 2) { - if (s->prescreen > 0) + if (s->pscrn > 0) s->prescreen[s->pscrn > 1](ctx, srcbuf + (y / 2) * srcbuf_stride + 32, srcbuf_stride, prescreen_buf, width, &s->prescreener[s->pscrn - 1]); @@ -649,7 +649,7 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) prescreen_buf, width, &s->coeffs[s->etype][s->nnsparam][s->nsize], s->qual == 2); - if (s->prescreen > 0) + if (s->pscrn > 0) interpolation(srcbuf + (y / 2) * srcbuf_stride + 32, srcbuf_stride, dstbuf + (y / 2) * dstbuf_stride, |