mirror/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2024-11-23 18:50:21 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
Page: Deep Learning in OpenCV
Pages
.clang_format 2018 2019 2020 2021 2022 2023 2024 Archive Branches BuildOpenCV4OpenVINO CI configuration CPU optimizations build options ChangeLog CiteOpenCV Coding_Style_Guide Compact build advice Contributing to G API: PRs Custom OpenCV Android SDK and AAR package build DNN Efficiency Deep Learning in OpenCV Documentation improvement plan Enabling GStreamer source in G API Enabling oneVPL source in G API Evolution Proposals FAQ G API Ideas towards OpenCV 5.0 G API Python Bindings GSoC_2017 GSoC_2018 GSoC_2019 GSoC_2020 GSoC_2021 GSoC_2022 GSoC_2023 GSoC_2024 Graph API Home HowToUsePerfTests HowToWritePerfTests How_to_contribute Huawei CANN Backend Intel OpenVINO backend Linux_contribution_guide MediaSDK encode decode backend OE 0. Template OE 1. Old C API OE 10. FP16 OE 11. Logging OE 12. Lapack OE 13. DNN ONNX Importer OE 14. DNN FP16 OE 15. Bit Exactness OE 16. Mini Halide OE 17. New Filter Engine OE 18. Module Optflow OE 19. Module Stereo OE 2. OpenCV 2 OE 20. Module Calibration OE 21. Module Tracking OE 21. Tracking API OE 22. Module Point Cloud OE 23. Module GAPI OE 24. Module HighGUI OE 25. Fast VideoIO OE 26. Samples OE 27. Wide Universal Intrinsics OE 28. Image Stitching OE 29. Adding AVX512 Support OE 3. OpenCV 3 OE 30. Color Calibration OE 31. Julia Bindings OE 32. Change OpenCV License to Apache 2 OE 33. 3D Module OE 34. Named Parameters OE 35. TFLite support OE 4. OpenCV 4 OE 5. OpenCV 5 OpenCL optimizations OpenCV 5 OpenCV Debugging Facilities OpenCV Platforms OpenCV RISC V OpenCVLogo OpenCV_GSoC OpenCV_GSoC_Application Opencv3 Opencv4 Profiling OpenCV Applications QA_in_OpenCV Running native OpenCV code with Hardware Address Sanitizer TIM VX Backend For Running OpenCV On NPU Template TensorFlow Object Detection API TensorFlow text graphs Using G API with MS ONNX Runtime Using G API with OpenCV AI Kit (OAK) Using G API with OpenVINO Toolkit Using G API with PlaidML backend Video IO hardware acceleration Video capture and write benchmark [Deprecated] Tengine based acceleration
12 Deep Learning in OpenCV
Abduragim Shtanchaev edited this page 2024-01-09 18:06:37 +03:00
Table of Contents

Deep Learning is the most popular and the fastest growing area in Computer Vision nowadays. Since OpenCV 3.1 there is DNN module in the library that implements forward pass (inferencing) with deep networks, pre-trained using some popular deep learning frameworks, such as Caffe. In OpenCV 3.3 the module has been promoted from opencv_contrib repository to the main repository (https://github.com/opencv/opencv/tree/master/modules/dnn) and has been accelerated significantly.

The module has no any extra dependencies, except for libprotobuf, and libprotobuf is now included into OpenCV.

The supported frameworks:

The supported layers:

  • AbsVal
  • Accum
  • AveragePooling
  • BatchNormalization
  • BNLL
  • Concatenation
  • Convolution (1d, 2d, including dilated convolution, 3d)
  • Crop
  • CropAndResize (RCNN-specific layer)
  • Deconvolution, a.k.a. transposed convolution or full convolution
  • DetectionOutput (SSD-specific layer)
  • Dropout
  • Einsum
  • Eltwise (+, *, max)
  • ELU
  • Expand
  • Flatten
  • FullyConnected
  • FlowWarp
  • Gather
  • Interpolation
  • LRN
  • LSTM
  • MaxPooling
  • MaxUnpooling
  • Mish
  • MVN
  • NormalizeBBox (SSD-specific layer)
  • Padding
  • Permute
  • Power
  • PReLU (including ChannelPReLU with channel-specific slopes)
  • PriorBox (SSD-specific layer)
  • ReLU
  • ReduceL1
  • ReduceL2
  • ReduceLogSum
  • ReduceLogSumExp
  • ReduceMax
  • ReduceMean
  • ReduceMin
  • ReduceProd
  • ReduceSum
  • ReduceSumSquare
  • Region (for DarkNet models)
  • Reorg
  • Resize
  • RNN
  • ROI Pooling (RCNN-specific layer)
  • Scale
  • Shift
  • ShuffleChannel
  • Sigmoid
  • Slice
  • Softmax
  • Split
  • Swish
  • TanH

You also can write your own Custom layer.

The module includes some SSE, AVX, AVX2 and NEON acceleration of the performance-critical layers as well as support of CUDA for the most of the layers. There is also constantly-improved Halide backend. OpenCL (libdnn-based) backend is being developed and should be integrated after OpenCV 3.3 release. Here you may find the up-to-date benchmarking results: DNN Efficiency

The provided API (for C++ and Python) is very easy to use, just load the network and run it. Multiple inputs/outputs are supported. Here are the examples: https://github.com/opencv/opencv/tree/master/samples/dnn.

There is Habrahabr article describing the module: https://habrahabr.ru/company/intel/blog/333612/ (in Russian).

The following networks have been tested and known to work:

Image classification

Caffe
TensorFlow
Darknet: https://pjreddie.com/darknet/imagenet/
ONNX: https://github.com/onnx/models
  • AlexNet
  • GoogleNet
  • CaffeNet
  • RCNN_ILSVRC13
  • ZFNet512
  • VGG16, VGG16_bn
  • ResNet-18v1, ResNet-50v1
  • CNN Mnist
  • MobileNetv2
  • LResNet100E-IR
  • Emotion FERPlus
  • Squeezenet
  • DenseNet121
  • Inception v1, v2
  • Shufflenet
Torchvision: https://github.com/pytorch/vision
PyTorch Image Models: https://github.com/rwightman/pytorch-image-models
PyTorch EfficientNet

Object detection

Caffe
TensorFlow
Darknet
ONNX: https://github.com/onnx/models

Semantic segmentation

Pose estimation

Image processing

Person identification

Text detection and Recognition

Depth Estimation

© Copyright 2024, OpenCV team