avformat/hlsenc: Fix extradata length check

Commit a2b1dd0ce301450a47c972745a6b33c4c273aa5d added support for
parsing annex B HEVC extradata to extract profile and level information.
Yet it only checks for there to be enough data left for the startcode
and the first byte of the NAL unit header and not for the full NAL unit
header; it simply presumes the second byte of the NAL unit header to be
present and skips it. Then the remaining size of the extradata is calculated
which ends up negative if the second byte of the NAL unit header is not
present. Yet when calling ff_nal_unit_extract_rbsp() it
will be converted to an uint32_t and end up as UINT32_MAX which
will cause mayhem.

This is solved by making sure that there is always enough remaining
extradata that could (pending 0x03 escapes) contain the data that we
are interested in.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

0 files changed, 0 insertions, 0 deletions