aboutsummaryrefslogtreecommitdiffstats
path: root/libavcodec/vmnc.c
diff options
context:
space:
mode:
authorAndreas Rheinhardt <andreas.rheinhardt@gmail.com>2020-12-24 14:36:22 +0100
committerAndreas Rheinhardt <andreas.rheinhardt@outlook.com>2021-04-08 01:07:50 +0200
commitd4b9e117ceb6356cbcdc9ca81ec9c6c4b90efdae (patch)
tree984d9ffb217775ed66c0b5b52f52cf1e879d0dde /libavcodec/vmnc.c
parentfb4da90fecdefa2508618ca835cd0250be940e04 (diff)
downloadffmpeg-d4b9e117ceb6356cbcdc9ca81ec9c6c4b90efdae.tar.gz
Revert "avcodec: add FF_CODEC_CAP_INIT_CLEANUP for all codecs which use ff_mpv_common_init()"
This mostly reverts commit 4b2863ff01b1fe93d9a518523c9098d17a9d8c6f. Said commit removed the freeing code from ff_mpv_common_init(), ff_mpv_common_frame_size_change() and ff_mpeg_framesize_alloc() and instead added the FF_CODEC_CAP_INIT_CLEANUP to several codecs that use ff_mpv_common_init(). This introduced several bugs: a) Several decoders using ff_mpv_common_init() in their init function were forgotten: This affected FLV, Intel H.263, RealVideo 3.0 and V4.0 as well as VC-1/WMV3. b) ff_mpv_common_init() is not only called from the init function of codecs, it is also called from AVCodec.decode functions. If an error happens after an allocation has succeeded, it can lead to memleaks; furthermore, it is now possible for the MpegEncContext to be marked as initialized even when ff_mpv_common_init() returns an error and this can lead to segfaults because decoders that call ff_mpv_common_init() when decoding a frame can mistakenly think that the MpegEncContext has been properly initialized. This can e.g. happen with H.261 or MPEG-4. c) Removing code for freeing from ff_mpeg_framesize_alloc() (which can't be called from any init function) can lead to segfaults because the check for whether it needs to allocate consists of checking whether the first of the buffers allocated there has been allocated. This part has already been fixed in 76cea1d2ce3f23e8131c8664086a1daf873ed694. d) ff_mpv_common_frame_size_change() can also not be reached from any AVCodec.init function; yet the changes can e.g. lead to segfaults with decoders using ff_h263_decode_frame() upon allocation failure, because the MpegEncContext will upon return be flagged as both initialized and not in need of reinitialization (granted, the fact that ff_h263_decode_frame() clears context_reinit before the context has been reinited is a bug in itself). With the earlier version, the context would be cleaned upon failure and it would be attempted to initialize the context again in the next call to ff_h263_decode_frame(). While a) could be fixed by adding the missing FF_CODEC_CAP_INIT_CLEANUP, keeping the current approach would entail adding cleanup code to several other places because of b). Therefore ff_mpv_common_init() is again made to clean up after itself; the changes to the wmv2 decoder and the SVQ1 encoder have not been reverted: The former fixed a memleak, the latter allowed to remove cleanup code. Fixes: double free Fixes: ff_free_picture_tables.mp4 Fixes: ff_mpeg_update_thread_context.mp4 Fixes: decode_colskip.mp4 Fixes: memset.mp4 Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Diffstat (limited to 'libavcodec/vmnc.c')
0 files changed, 0 insertions, 0 deletions