diff options
| author | Wenbin Chen <wenbin.chen@intel.com> | 2024-03-15 12:42:49 +0800 |
|---|---|---|
| committer | Guo Yejun <yejun.guo@intel.com> | 2024-03-19 14:48:58 +0800 |
| commit | f4e0664fd1bc94ee1a1afbd20c5a32ac33d93a2d (patch) | |
| tree | c3301646df0a2ec6c551cf5165015d39b05c8a31 /libavfilter/vf_dnn_processing.c | |
| parent | d24b136f539944e4ee2bdf62abe613b2caa4320f (diff) | |
| download | ffmpeg-f4e0664fd1bc94ee1a1afbd20c5a32ac33d93a2d.tar.gz | |
libavfi/dnn: add LibTorch as one of DNN backend
PyTorch is an open source machine learning framework that accelerates
the path from research prototyping to production deployment. Official
website: https://pytorch.org/. We call the C++ library of PyTorch as
LibTorch, the same below.
To build FFmpeg with LibTorch, please take following steps as
reference:
1. download LibTorch C++ library in
https://pytorch.org/get-started/locally/,
please select C++/Java for language, and other options as your need.
Please download cxx11 ABI version:
(libtorch-cxx11-abi-shared-with-deps-*.zip).
2. unzip the file to your own dir, with command
unzip libtorch-shared-with-deps-latest.zip -d your_dir
3. export libtorch_root/libtorch/include and
libtorch_root/libtorch/include/torch/csrc/api/include to $PATH
export libtorch_root/libtorch/lib/ to $LD_LIBRARY_PATH
4. config FFmpeg with ../configure --enable-libtorch \
--extra-cflag=-I/libtorch_root/libtorch/include \
--extra-cflag=-I/libtorch_root/libtorch/include/torch/csrc/api/include \
--extra-ldflags=-L/libtorch_root/libtorch/lib/
5. make
To run FFmpeg DNN inference with LibTorch backend:
./ffmpeg -i input.jpg -vf \
dnn_processing=dnn_backend=torch:model=LibTorch_model.pt -y output.jpg
The LibTorch_model.pt can be generated by Python with torch.jit.script()
api. https://pytorch.org/tutorials/advanced/cpp_export.html. This is
pytorch official guide about how to convert and load torchscript model.
Please note, torch.jit.trace() is not recommanded, since it does
not support ambiguous input size.
Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Reviewed-by: Guo Yejun <yejun.guo@intel.com>
Diffstat (limited to 'libavfilter/vf_dnn_processing.c')
| -rw-r--r-- | libavfilter/vf_dnn_processing.c | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/libavfilter/vf_dnn_processing.c b/libavfilter/vf_dnn_processing.c index e7d21eef32..fdac31665e 100644 --- a/libavfilter/vf_dnn_processing.c +++ b/libavfilter/vf_dnn_processing.c @@ -51,6 +51,9 @@ static const AVOption dnn_processing_options[] = { #if (CONFIG_LIBOPENVINO == 1) { "openvino", "openvino backend flag", 0, AV_OPT_TYPE_CONST, { .i64 = DNN_OV }, 0, 0, FLAGS, .unit = "backend" }, #endif +#if (CONFIG_LIBTORCH == 1) + { "torch", "torch backend flag", 0, AV_OPT_TYPE_CONST, { .i64 = DNN_TH }, 0, 0, FLAGS, "backend" }, +#endif DNN_COMMON_OPTIONS { NULL } }; |