diff options
| author | Uoti Urpala <uau@glyph.nonexistent.invalid> | 2012-08-07 14:48:06 +0300 |
|---|---|---|
| committer | Martin Storsjö <martin@martin.st> | 2012-08-17 20:08:53 +0300 |
| commit | 2e8f3cbcda536970aee03f5543185d7fc01232a1 (patch) | |
| tree | e333420caf1636cd6af399572ad74343fc838bc6 | |
| parent | ca00a7e809a4b9c9fb146403d278964b88d16b85 (diff) | |
| download | ffmpeg-2e8f3cbcda536970aee03f5543185d7fc01232a1.tar.gz | |
h264: vdpau: fix crash with unsupported colorspace
The h264_vdpau decoder crashed if output colorspace was not 8-bit 420.
Add a check to error out instead (current hardware does not support
other colorspaces, so successful decoding is not possible).
Signed-off-by: Martin Storsjö <martin@martin.st>
| -rw-r--r-- | libavcodec/h264.c | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/libavcodec/h264.c b/libavcodec/h264.c index 2019de26ba..a191bc74de 100644 --- a/libavcodec/h264.c +++ b/libavcodec/h264.c @@ -3876,6 +3876,15 @@ again: if (avctx->bits_per_raw_sample != h->sps.bit_depth_luma || h->cur_chroma_format_idc != h->sps.chroma_format_idc) { + if (s->avctx->codec->capabilities & CODEC_CAP_HWACCEL_VDPAU + && (h->sps.bit_depth_luma != 8 || + h->sps.chroma_format_idc > 1)) { + av_log(avctx, AV_LOG_ERROR, + "VDPAU decoding does not support video " + "colorspace\n"); + buf_index = -1; + goto end; + } if (h->sps.bit_depth_luma >= 8 && h->sps.bit_depth_luma <= 10) { avctx->bits_per_raw_sample = h->sps.bit_depth_luma; h->cur_chroma_format_idc = h->sps.chroma_format_idc; |