opencv/modules/objdetect/src
FantasqueX 4efd52f676
Merge pull request #26650 from FantasqueX:fix-26642
Use size_t when calculating size of all_points #26650

Closes: #26642 

Asan log
```
=================================================================
==41401==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fc55a02a3fc at pc 0x7fc58e304131 bp 0x7ffd54787b00 sp 0x7ffd54787af8
WRITE of size 4 at 0x7fc55a02a3fc thread T0
    #0 0x7fc58e304130 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726
    #1 0x7fc58e3054b0 in cv::QRDetectMulti::localization() /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3829
    #2 0x7fc58e308020 in cv::ImplContour::detectMulti(cv::_InputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3987
    #3 0x7fc58e30b5b1 in cv::ImplContour::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:4176
    #4 0x7fc58e28922f in cv::GraphicalCodeDetector::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/graphical_code_detector.cpp:42
    #5 0x5954e8 in Body /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:48
    #6 0x594fc0 in TestBody /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:42
    #7 0x67ee6a in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
    #8 0x6734a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
    #9 0x641fe8 in testing::Test::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3993
    #10 0x6431ac in testing::TestInfo::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4169
    #11 0x643d15 in testing::TestCase::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4287
    #12 0x659ff3 in testing::internal::UnitTestImpl::RunAllTests() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6662
    #13 0x681205 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
    #14 0x675127 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
    #15 0x65734c in testing::UnitTest::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6271
    #16 0x5907f0 in RUN_ALL_TESTS() /home/fanta/source/opencv/modules/ts/include/opencv2/ts/ts_gtest.h:22240
    #17 0x590cdd in main (/home/fanta/source/opencv-build-4.x-clang/bin/opencv_test_objdetect+0x590cdd) (BuildId: a9363fc788d57c48225fc0559ac9199d07d415db)
    #18 0x7fc58ab242ad in __libc_start_call_main (/lib64/libc.so.6+0x2a2ad) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
    #19 0x7fc58ab24378 in __libc_start_main@@GLIBC_2.34 (/lib64/libc.so.6+0x2a378) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
    #20 0x417014 in _start ../sysdeps/x86_64/start.S:115

0x7fc55a02a3fc is located 0 bytes after 2938510332-byte region [0x7fc4aadc8800,0x7fc55a02a3fc)
allocated by thread T0 here:
    #0 0x7fc58e590298 in operator new(unsigned long) (/lib64/libasan.so.8+0xfd298) (BuildId: da72ee674d801ced58193987786b90646d94ff8d)
    #1 0x7fc58e34d010 in std::__new_allocator<cv::Vec<int, 3> >::allocate(unsigned long, void const*) /usr/include/c++/14/bits/new_allocator.h:151

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&)

Shadow bytes around the buggy address:
  0x7fc55a02a100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7fc55a02a300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x7fc55a02a380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04]
  0x7fc55a02a400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x7fc55a02a600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==41401==ABORTING
```

`(true_points_group[i].size()` is 1794 and `(true_points_group[i].size() - 2 ) * (true_points_group[i].size() - 1) * true_points_group[i].size())` is 5764222464 which overflows `int`

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2024-12-23 11:36:39 +03:00
..
aruco Merge pull request #25938 from vrabaud:charuco 2024-07-24 09:27:07 +03:00
barcode_decoder Fixed barcode to be built without DNN 2023-06-16 15:12:49 +03:00
barcode_detector Merge pull request #24903 from cristidbr-adapta:feature-barcode-detector-parameters 2024-05-20 09:53:22 +03:00
opencl fix test failure on ODROID-N2 2020-04-28 21:46:31 +09:00
barcode.cpp Merge pull request #24903 from cristidbr-adapta:feature-barcode-detector-parameters 2024-05-20 09:53:22 +03:00
cascadedetect_convert.cpp backport: checks and fixes from static code analyzers results 2020-09-02 19:05:47 +00:00
cascadedetect.cpp Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2021-12-03 12:32:49 +00:00
cascadedetect.hpp Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2019-07-25 19:21:47 +00:00
detection_based_tracker.cpp Merge pull request #21107 from take1014:remove_assert_21038 2021-11-27 18:34:52 +00:00
face_detect.cpp WinRT/UWP build: fix some specific warnings 2024-10-25 22:32:44 +03:00
face_recognize.cpp WinRT/UWP build: fix some specific warnings 2024-10-25 22:32:44 +03:00
graphical_code_detector_impl.hpp Merge pull request #24548 from dkurt:qrcode_struct_append_decode 2024-02-01 16:15:14 +03:00
graphical_code_detector.cpp added graphicalCodeDetector, remove QRCodeDetectorBase 2023-06-08 14:50:58 +03:00
hog.cpp Clean up the Universal Intrinsic API. 2023-10-13 19:23:30 +08:00
main.cpp IPPInitSingelton was added to contain IPP related global variables; 2015-10-01 09:58:48 +03:00
precomp.hpp Merge pull request #23736 from seanm:c++11-simplifications 2024-01-19 16:53:08 +03:00
qrcode_encoder_table.inl.hpp Merge pull request #24299 from dkurt:qrcode_decode 2023-11-24 11:35:36 +03:00
qrcode_encoder.cpp objdetect: fix invalid vector access in QR encoder 2024-11-29 14:40:53 +03:00
qrcode.cpp Merge pull request #26650 from FantasqueX:fix-26642 2024-12-23 11:36:39 +03:00