diff options
| author | Xi Wang <xi.wang@gmail.com> | 2013-01-22 20:58:07 -0500 |
|---|---|---|
| committer | Michael Niedermayer <michaelni@gmx.at> | 2013-02-07 01:00:01 +0100 |
| commit | bc58fe0309a075b6f75aeef948a55eede927591c (patch) | |
| tree | 9428ca7a16ab7cc9f20432b48a579981cda85788 | |
| parent | 8eda88868399de00806cf21a966d9660db4ae9b4 (diff) | |
| download | ffmpeg-bc58fe0309a075b6f75aeef948a55eede927591c.tar.gz | |
rtpenc: fix overflow checking in avc_mp4_find_startcode()
The check `start + res < start' is broken since pointer overflow is
undefined behavior in C. Many compilers such as gcc/clang optimize
away this check.
Use `res > end - start' instead. Also change `res' to unsigned int
to avoid signed left-shift overflow.
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2f014567cfd63e58156f60666f1a61ba147276ab)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
| -rw-r--r-- | libavformat/rtpenc_h264.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/libavformat/rtpenc_h264.c b/libavformat/rtpenc_h264.c index 68f497590b..b6c16e17d8 100644 --- a/libavformat/rtpenc_h264.c +++ b/libavformat/rtpenc_h264.c @@ -31,14 +31,14 @@ static const uint8_t *avc_mp4_find_startcode(const uint8_t *start, const uint8_t *end, int nal_length_size) { - int res = 0; + unsigned int res = 0; if (end - start < nal_length_size) return NULL; while (nal_length_size--) res = (res << 8) | *start++; - if (start + res > end || res < 0 || start + res < start) + if (res > end - start) return NULL; return start + res; |