opencv/modules/dnn/src/nms.cpp

141 lines
4.8 KiB
C++
Raw Normal View History

2017-10-17 00:16:52 +08:00
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
//
// Copyright (C) 2017, Intel Corporation, all rights reserved.
// Third party copyrights are property of their respective owners.
#include "precomp.hpp"
#include "nms.inl.hpp"
2017-10-17 00:16:52 +08:00
#include <opencv2/imgproc.hpp>
namespace cv { namespace dnn {
CV__DNN_INLINE_NS_BEGIN
Merge pull request #12249 from kopytjuk:feature/region-layer-batch-mode Feature/region layer batch mode (#12249) * Add batch mode for Darknet networks. Swap variables in test_darknet. Adapt reorg layer to batch mode. Adapt region layer. Add OpenCL implementation. Remove trailing whitespace. Bugifx reorg opencl implementation. Fix bug in OpenCL reorg. Fix modulo bug. Fix bug. Reorg openCL. Restore reorg layer opencl code. OpenCl fix. Work on openCL reorg. Remove whitespace. Fix openCL region layer implementation. Fix bug. Fix softmax region opencl bug. Fix opencl bug. Fix openCL bug. Update aff_trans.cpp When the fullAffine parameter is set to false, the estimateRigidTransform function maybe return empty, then the _localAffineEstimate function will be called, but the bug in it will result in incorrect results. core(libva): support YV12 too Added to CPU path only. OpenCL code path still expects NV12 only (according to Intel OpenCL extension) cmake: allow to specify own libva paths via CMake: - `-DVA_LIBRARIES=/opt/intel/mediasdk/lib64/libva.so.2\;/opt/intel/mediasdk/lib64/libva-drm.so.2` android: NDK17 support tested with NDK 17b (17.1.4828580) Enable more deep learning tests using Intel's Inference Engine backend ts: don't pass NULL for std::string() constructor openvino: use 2018R3 defines experimental version++ OpenCV version++ OpenCV 3.4.3 OpenCV version '-openvino' openvino: use 2018R3 defines Fixed windows build with InferenceEngine dnn: fix variance setting bug for PriorBoxLayer - The size of second channel should be size[2] of output tensor, - The Scalar should be {variance[0], variance[0], variance[0], variance[0]} for _variance.size() == 1 case. Signed-off-by: Wu Zhiwen <zhiwen.wu@intel.com> Fix lifetime of networks which are loaded from Model Optimizer IRs Adds a small note describing BUILD_opencv_world (#12332) * Added a mall note describing BUILD_opencv_world cmake option to the Installation in Windows tutorial. * Made slight changes in BUILD_opencv_world documentation. * Update windows_install.markdown improved grammar Update opengl_interop.cpp resolves #12307 java: fix LIST_GET macro fix typo Added option to fail on missing testdata Fixed that object_detection.py does not work in python3. cleanup: IPP Async (IPP_A) except header file with conversion routines (will be removed in OpenCV 4.0) imgcodecs: add null pointer check Include preprocessing nodes to object detection TensorFlow networks (#12211) * Include preprocessing nodes to object detection TensorFlow networks * Enable more fusion * faster_rcnn_resnet50_coco_2018_01_28 test countNonZero function reworked to use wide universal intrinsics instead of SSE2 intrinsics resolve #5788 imgcodecs(webp): multiple fixes - don't reallocate passed 'img' (test fixed - must use IMREAD_UNCHANGED / IMREAD_ANYCOLOR) - avoid memory DDOS - avoid reading of whole file during header processing - avoid data access after allocated buffer during header processing (missing checks) - use WebPFree() to free allocated buffers (libwebp >= 0.5.0) - drop unused & undefined `.close()` method - added checks for channels >= 5 in encoder ml: fix adjusting K in KNearest (#12358) dnn(perf): fix and merge Convolution tests - OpenCL tests didn't run any OpenCL kernels - use real configuration from existed models (the first 100 cases) - batch size = 1 dnn(test): use dnnBackendsAndTargets() param generator Bit-exact resize reworked to use wide intrinsics (#12038) * Bit-exact resize reworked to use wide intrinsics * Reworked bit-exact resize row data loading * Added bit-exact resize row data loaders for SIMD256 and SIMD512 * Fixed type punned pointer dereferencing warning * Reworked loading of source data for SIMD256 and SIMD512 bit-exact resize Bit-exact GaussianBlur reworked to use wide intrinsics (#12073) * Bit-exact GaussianBlur reworked to use wide intrinsics * Added v_mul_hi universal intrinsic * Removed custom SSE2 branch from bit-exact GaussianBlur * Removed loop unrolling for gaussianBlur horizontal smoothing doc: fix English gramma in tutorial out-of-focus-deblur filter (#12214) * doc: fix English gramma in tutorial out-of-focus-deblur filter * Update out_of_focus_deblur_filter.markdown slightly modified one sentence doc: add new tutorial motion deblur filter (#12215) * doc: add new tutorial motion deblur filter * Update motion_deblur_filter.markdown a few minor changes Replace Slice layer to Crop in Faster-RCNN networks from Caffe js: use generated list of OpenCV headers - replaces hand-written list imgcodecs(webp): use safe cast to size_t on Win32 * Put Version status back to -dev. follow the common codestyle Exclude some target engines. Refactor formulas. Refactor code. * Remove unused variable. * Remove inference engine check for yolov2. * Alter darknet batch tests to test with two different images. * Add yolov3 second image GT. * Fix bug. * Fix bug. * Add second test. * Remove comment. * Add NMS on network level. * Add helper files to dev. * syntax fix. * Fix OD sample. Fix sample dnn object detection. Fix NMS boxes bug. remove trailing whitespace. Remove debug function. Change thresholds for opencl tests. * Adapt score diff and iou diff. * Alter iouDiffs. * Add debug messages. * Adapt iouDiff. * Fix tests
2018-09-12 18:29:43 +08:00
template <typename T>
static inline float rectOverlap(const T& a, const T& b)
{
return 1.f - static_cast<float>(jaccardDistance(a, b));
}
2017-10-17 00:16:52 +08:00
void NMSBoxes(const std::vector<Rect>& bboxes, const std::vector<float>& scores,
const float score_threshold, const float nms_threshold,
std::vector<int>& indices, const float eta, const int top_k)
Merge pull request #12249 from kopytjuk:feature/region-layer-batch-mode Feature/region layer batch mode (#12249) * Add batch mode for Darknet networks. Swap variables in test_darknet. Adapt reorg layer to batch mode. Adapt region layer. Add OpenCL implementation. Remove trailing whitespace. Bugifx reorg opencl implementation. Fix bug in OpenCL reorg. Fix modulo bug. Fix bug. Reorg openCL. Restore reorg layer opencl code. OpenCl fix. Work on openCL reorg. Remove whitespace. Fix openCL region layer implementation. Fix bug. Fix softmax region opencl bug. Fix opencl bug. Fix openCL bug. Update aff_trans.cpp When the fullAffine parameter is set to false, the estimateRigidTransform function maybe return empty, then the _localAffineEstimate function will be called, but the bug in it will result in incorrect results. core(libva): support YV12 too Added to CPU path only. OpenCL code path still expects NV12 only (according to Intel OpenCL extension) cmake: allow to specify own libva paths via CMake: - `-DVA_LIBRARIES=/opt/intel/mediasdk/lib64/libva.so.2\;/opt/intel/mediasdk/lib64/libva-drm.so.2` android: NDK17 support tested with NDK 17b (17.1.4828580) Enable more deep learning tests using Intel's Inference Engine backend ts: don't pass NULL for std::string() constructor openvino: use 2018R3 defines experimental version++ OpenCV version++ OpenCV 3.4.3 OpenCV version '-openvino' openvino: use 2018R3 defines Fixed windows build with InferenceEngine dnn: fix variance setting bug for PriorBoxLayer - The size of second channel should be size[2] of output tensor, - The Scalar should be {variance[0], variance[0], variance[0], variance[0]} for _variance.size() == 1 case. Signed-off-by: Wu Zhiwen <zhiwen.wu@intel.com> Fix lifetime of networks which are loaded from Model Optimizer IRs Adds a small note describing BUILD_opencv_world (#12332) * Added a mall note describing BUILD_opencv_world cmake option to the Installation in Windows tutorial. * Made slight changes in BUILD_opencv_world documentation. * Update windows_install.markdown improved grammar Update opengl_interop.cpp resolves #12307 java: fix LIST_GET macro fix typo Added option to fail on missing testdata Fixed that object_detection.py does not work in python3. cleanup: IPP Async (IPP_A) except header file with conversion routines (will be removed in OpenCV 4.0) imgcodecs: add null pointer check Include preprocessing nodes to object detection TensorFlow networks (#12211) * Include preprocessing nodes to object detection TensorFlow networks * Enable more fusion * faster_rcnn_resnet50_coco_2018_01_28 test countNonZero function reworked to use wide universal intrinsics instead of SSE2 intrinsics resolve #5788 imgcodecs(webp): multiple fixes - don't reallocate passed 'img' (test fixed - must use IMREAD_UNCHANGED / IMREAD_ANYCOLOR) - avoid memory DDOS - avoid reading of whole file during header processing - avoid data access after allocated buffer during header processing (missing checks) - use WebPFree() to free allocated buffers (libwebp >= 0.5.0) - drop unused & undefined `.close()` method - added checks for channels >= 5 in encoder ml: fix adjusting K in KNearest (#12358) dnn(perf): fix and merge Convolution tests - OpenCL tests didn't run any OpenCL kernels - use real configuration from existed models (the first 100 cases) - batch size = 1 dnn(test): use dnnBackendsAndTargets() param generator Bit-exact resize reworked to use wide intrinsics (#12038) * Bit-exact resize reworked to use wide intrinsics * Reworked bit-exact resize row data loading * Added bit-exact resize row data loaders for SIMD256 and SIMD512 * Fixed type punned pointer dereferencing warning * Reworked loading of source data for SIMD256 and SIMD512 bit-exact resize Bit-exact GaussianBlur reworked to use wide intrinsics (#12073) * Bit-exact GaussianBlur reworked to use wide intrinsics * Added v_mul_hi universal intrinsic * Removed custom SSE2 branch from bit-exact GaussianBlur * Removed loop unrolling for gaussianBlur horizontal smoothing doc: fix English gramma in tutorial out-of-focus-deblur filter (#12214) * doc: fix English gramma in tutorial out-of-focus-deblur filter * Update out_of_focus_deblur_filter.markdown slightly modified one sentence doc: add new tutorial motion deblur filter (#12215) * doc: add new tutorial motion deblur filter * Update motion_deblur_filter.markdown a few minor changes Replace Slice layer to Crop in Faster-RCNN networks from Caffe js: use generated list of OpenCV headers - replaces hand-written list imgcodecs(webp): use safe cast to size_t on Win32 * Put Version status back to -dev. follow the common codestyle Exclude some target engines. Refactor formulas. Refactor code. * Remove unused variable. * Remove inference engine check for yolov2. * Alter darknet batch tests to test with two different images. * Add yolov3 second image GT. * Fix bug. * Fix bug. * Add second test. * Remove comment. * Add NMS on network level. * Add helper files to dev. * syntax fix. * Fix OD sample. Fix sample dnn object detection. Fix NMS boxes bug. remove trailing whitespace. Remove debug function. Change thresholds for opencl tests. * Adapt score diff and iou diff. * Alter iouDiffs. * Add debug messages. * Adapt iouDiff. * Fix tests
2018-09-12 18:29:43 +08:00
{
CV_Assert_N(bboxes.size() == scores.size(), score_threshold >= 0,
nms_threshold >= 0, eta > 0);
NMSFast_(bboxes, scores, score_threshold, nms_threshold, eta, top_k, indices, rectOverlap);
}
void NMSBoxes(const std::vector<Rect2d>& bboxes, const std::vector<float>& scores,
const float score_threshold, const float nms_threshold,
std::vector<int>& indices, const float eta, const int top_k)
2017-10-17 00:16:52 +08:00
{
CV_Assert_N(bboxes.size() == scores.size(), score_threshold >= 0,
nms_threshold >= 0, eta > 0);
NMSFast_(bboxes, scores, score_threshold, nms_threshold, eta, top_k, indices, rectOverlap);
2017-10-17 00:16:52 +08:00
}
static inline float rotatedRectIOU(const RotatedRect& a, const RotatedRect& b)
{
std::vector<Point2f> inter;
int res = rotatedRectangleIntersection(a, b, inter);
if (inter.empty() || res == INTERSECT_NONE)
return 0.0f;
if (res == INTERSECT_FULL)
return 1.0f;
float interArea = contourArea(inter);
return interArea / (a.size.area() + b.size.area() - interArea);
}
void NMSBoxes(const std::vector<RotatedRect>& bboxes, const std::vector<float>& scores,
const float score_threshold, const float nms_threshold,
std::vector<int>& indices, const float eta, const int top_k)
{
CV_Assert_N(bboxes.size() == scores.size(), score_threshold >= 0,
nms_threshold >= 0, eta > 0);
NMSFast_(bboxes, scores, score_threshold, nms_threshold, eta, top_k, indices, rotatedRectIOU);
}
2021-10-05 21:46:37 +08:00
void softNMSBoxes(const std::vector<Rect>& bboxes,
const std::vector<float>& scores,
std::vector<float>& updated_scores,
const float score_threshold,
const float nms_threshold,
std::vector<int>& indices,
size_t top_k,
const float sigma,
SoftNMSMethod method)
{
CV_Assert_N(bboxes.size() == scores.size(), score_threshold >= 0,
nms_threshold >= 0, sigma >= 0);
indices.clear();
updated_scores.clear();
std::vector<std::pair<float, size_t> > score_index_vec(scores.size());
for (size_t i = 0; i < scores.size(); i++)
{
score_index_vec[i].first = scores[i];
score_index_vec[i].second = i;
}
const auto score_cmp = [](const std::pair<float, size_t>& a, const std::pair<float, size_t>& b)
{
return a.first == b.first ? a.second > b.second : a.first < b.first;
};
top_k = top_k == 0 ? scores.size() : std::min(top_k, scores.size());
ptrdiff_t start = 0;
while (indices.size() < top_k)
{
auto it = std::max_element(score_index_vec.begin() + start, score_index_vec.end(), score_cmp);
float bscore = it->first;
size_t bidx = it->second;
if (bscore < score_threshold)
{
break;
}
indices.push_back(static_cast<int>(bidx));
updated_scores.push_back(bscore);
std::swap(score_index_vec[start], *it); // first start elements are chosen
for (size_t i = start + 1; i < scores.size(); ++i)
{
float& bscore_i = score_index_vec[i].first;
const size_t bidx_i = score_index_vec[i].second;
if (bscore_i < score_threshold)
{
continue;
}
float overlap = rectOverlap(bboxes[bidx], bboxes[bidx_i]);
switch (method)
{
case SoftNMSMethod::SOFTNMS_LINEAR:
if (overlap > nms_threshold)
{
bscore_i *= 1.f - overlap;
}
break;
case SoftNMSMethod::SOFTNMS_GAUSSIAN:
bscore_i *= exp(-(overlap * overlap) / sigma);
break;
default:
CV_Error(Error::StsBadArg, "Not supported SoftNMS method.");
}
}
++start;
}
}
CV__DNN_INLINE_NS_END
2017-10-17 00:16:52 +08:00
}// dnn
}// cv