avformat/mpegtsenc: Changed Video PES packet length to 0.

The rational for this is another issue that plex has exposed.  When it is
conducting a transcode of video to HLS for streaming, my father noticed
artifacts when played on his GoogleTV (NSZ-GT1).  He sent me a test file
and I reproduced it on my device of the same model.  It is important to
note that the artifacts were not present when streaming to VLC or QuickTime
Player.  I copied the command-line that plex used, and conducted all of the
following tests using FFmpeg git.

Transcode to HLS: artifacts on playback
Transcode to TS: playback is fine
Cat HLS segments into a single TS: playback is fine
Segment single TS file to segments: artifacts on playback
Segment single TS file to segments using Apple's HLS segmenter: playback is
fine

At this point I carefully examined the differences between Apple's HLS
segmenter output and FFmpeg's.  Among the considerable differences, I
noticed that the video PES packets always had a 0 length.  So I continued:

Transcode to HLS using FFmpeg with 0 length PES packets: playback is fine.
Segment single TS to segments with 0 length PES packets: playback is fine.

All failures mentioned are only on the GTV since it is the only player on
which I could reproduce artifacts.  I only tested the GTV, VLC, and
QuickTime Player though, so my test case is limited.  I do not know if
other players exhibit this issue.

Since it was useful last time, I have uploaded the test file as
hls_pes_packet_length.m4v along with its associated txt file which contains
the transcode command-line that was used.

Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

1 files changed, 3 insertions, 0 deletions

diff --git a/libavformat/mpegtsenc.c b/libavformat/mpegtsenc.c
index 8d80e072a0..9cc17ccb1d 100644
--- a/libavformat/mpegtsenc.c
+++ b/libavformat/mpegtsenc.c
@@ -1066,6 +1066,9 @@ static void mpegts_write_pes(AVFormatContext *s, AVStream *st,
             }
             if (len > 0xffff)
                 len = 0;
+            if (st->codec->codec_type == AVMEDIA_TYPE_VIDEO) {
+                len = 0;
+            }
             *q++ = len >> 8;
             *q++ = len;
             val = 0x80;