G-API: Implement InferROI, InferList, InferList2 for OpenVINO backend #23799
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
G-API: Align IE Backend with the latest OpenVINO version #23796
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
G-API: Expose explicit preprocessing for IE Backend #23786
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
[G-API] Implement OpenVINO 2.0 backend #23595
### Pull Request Readiness Checklist
Implemented basic functionality for `OpenVINO` 2.0 G-API backend.
#### Overview
- [x] Implement `Infer` kernel with some of essential configurable parameters + IR/Blob models format support.
- [ ] Implement the rest of kernels: `InferList`, `InferROI`, `Infer2` + other configurable params (e.g reshape)
- [x] Asyncrhonous execution support
- [ ] Remote context support
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
G-API: Integration branch for ONNX & Python-related changes #23597
# Changes overview
## 1. Expose ONNX backend's Normalization and Mean-value parameters in Python
* Since Python G-API bindings rely on `Generic` infer to express Inference, the `Generic` specialization of `onnx::Params` was extended with new methods to control normalization (`/255`) and mean-value; these methods were exposed in the Python bindings
* Found some questionable parts in the existing API which I'd like to review/discuss (see comments)
UPD:
1. Thanks to @TolyaTalamanov normalization inconsistencies have been identified with `squeezenet1.0-9` ONNX model itself; tests using these model were updated to DISABLE normalization and NOT using mean/value.
2. Questionable parts were removed and tests still pass.
### Details (taken from @TolyaTalamanov's comment):
`squeezenet1.0.*onnx` - doesn't require scaling to [0,1] and mean/std because the weights of the first convolution already scaled. ONNX documentation is broken. So the correct approach to use this models is:
1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44 but without normalization step:
```
# DON'T DO IT:
# mean_vec = np.array([0.485, 0.456, 0.406])
# stddev_vec = np.array([0.229, 0.224, 0.225])
# norm_img_data = np.zeros(img_data.shape).astype('float32')
# for i in range(img_data.shape[0]):
# norm_img_data[i,:,:] = (img_data[i,:,:]/255 - mean_vec[i]) / stddev_vec[i]
# # add batch channel
# norm_img_data = norm_img_data.reshape(1, 3, 224, 224).astype('float32')
# return norm_img_data
# INSTEAD
return img_data.reshape(1, 3, 224, 224)
```
2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters:
```
net = cv.gapi.onnx.params('squeezenet', model_filename)
net.cfgNormalize('data_0', False)
```
**Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution.
---
`squeezenet1.1.*onnx` - requires scaling to [0,1] and mean/std - onnx documentation is correct.
1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44
2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters:
```
net = cv.gapi.onnx.params('squeezenet', model_filename)
net.cfgNormalize('data_0', True) // default
net.cfgMeanStd('data_0', [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
```
**Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution.
## 2. Expose Fluid & kernel package-related functionality in Python
* `cv::gapi::combine()`
* `cv::GKernelPackage::size()` (mainly for testing purposes)
* `cv::gapi::imgproc::fluid::kernels()`
Added a test for the above.
## 3. Fixed issues with Python stateful kernel handling
Fixed error message when `outMeta()` of custom python operation fails.
## 4. Fixed various issues in Python tests
1. `test_gapi_streaming.py` - fixed behavior of Desync test to avoid sporadic issues
2. `test_gapi_infer_onnx.py` - fixed model lookup (it was still using the ONNX Zoo layout but was NOT using the proper env var we use to point to one).
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
G-API: replace GAPI_Assert() with 'false' and '0' to GAPI_Error()
* gapi: GAPI_Error() macro
* gapi: replace GAPI_Assert() with 'false' and '0' to GAPI_Error()
* build: eliminate 'unreachable code' after CV_Error() (MSVC 2015)
* build: eliminate 'unreachable code' warning for MSVS 2015/2017
- observed in constructors stubs with throwing exception
[G-API] Handle exceptions in streaming executor
* Handle exceptions in streaming executor
* Rethrow exception in non-streaming executor
* Clean up
* Put more tests
* Handle exceptions in IE backend
* Handle exception in IE callbacks
* Handle exception in GExecutor
* Handle all exceptions in IE backend
* Not only (std::exception& e)
* Fix comments to review
* Handle input exception in generic way
* Fix comment
* Clean up
* Apply review comments
* Put more comments
* Fix alignment
* Move test outside of HAVE_NGRAPH
* Fix compilation
G-API: Removing G-API test code that is a reflection of ts module
* gapi: don't hijack testing infrastructure
* Removed initDataPath functionality (ts module exists)
* Removed false for ocv_extra data from findDataFile
Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
[G-API] Fix bugs in GIEBackend
* Remove inputs/outputs map from IEUnit
* Add test
* Add NV12 test
* Reorganize setBlob function
* Check that backend don't overwrite blob precision
* Stop setting config to global IE::Core
* Replace mutable to const_cast
* Update modules/gapi/test/infer/gapi_infer_ie_test.cpp
* Update modules/gapi/test/infer/gapi_infer_ie_test.cpp
* Make blob parameter as const ref
* Cosmetic fixes
* Fix failed test on inferROI
* Removed double ref for ii
* Disable tests
* Skip tests if device not available
* Use Sync prim under shared_ptr to avoid issue on MAC
* Apply WA for IE::Core
* Apply WA for MAC build
* Try to apply another WA
* Not release IE::Core for apple
* Put comment
* Support PreprocInfo for
* InferROI
* InferList
* InferList2
* Remove empty line
* Fix alignment
Co-authored-by: Maxim Pashchenkov <maxim.pashchenkov@intel.com>
G-API: Extend MediaFrame to be able to extract additional info besides access
* Extend MediaFrame to be able to extract additional info besides access
* Add default implementation for blobParams()
* Add comment on the default blobParams()
G-API: ONNX. Adding INT64-32 conversion for output.
* Added int64 to 32 conversion
* Added warning
* Added type checks for all toCV
* Added type checks for tests
* Small fixes
* Const for fixture in test
* std::tuple if retutn value for toCV
* Mistake
* Changed toCV for tests
* Added Assert
* Fix for comments
* One conversion for ONNX and IE
* Clean up
* One more fix
* Added copyFromONNX
* Removed warning
* Apply review comments
G-API: Implement async version for InferList & Infer2
* Implement async version for InferList & Infer2
* Fix warning
* Fix bug with roi ordering
* Post input meta instead of empty
* Fix comments to review
[G-API] Support multiple asynchronous requests
* Support nireq option
* Disable tests to check CI
* Fix bug with hanging
* WA to green CI
* Snapshot
* Simplify RequestPool
* Add default values to id
* Fix win warning
G-API: ONNX. Support for networks with three dimensional input.
* Padding without tests
* Removed padding
* Some small fixes
* Added wstring_convert
* Alignment fix, m b
* Small fixes
* Moved include from onnx.hpp
G-API: Support GFrame for ONNX infer
* Added GFrame for ONNX
* Cut test
* Removed IE from assert
* Review comments
* Added const/bbot rstrt
* View instead unique_ptr in func. sig.
* Added extractMat function, ONNXCompiled contains exMat - cv::Mat with non processed input data
* Added meta check for inferList2
[G-API] Support GFrame for infer
* GInfer(GFrame), currently broken
* Fixed (API only)
* Support GFrame in GIEBackend
* Fix comments to review
* Fix comments to review
* Fix doxygen
* Fix building with different IE versions
* Fix warning on MacOS
Co-authored-by: Dmitry Matveev <dmitry.matveev@intel.com>
Co-authored-by: Smirnov Alexey <alexey.smirnov@intel.com>
G-API: ONNX. Support tensor input for CNN with dynamic input
* Added support for dynamic input tensor, refactored one input/output tests
* Added multiple input/output fixture, test for mobilenet
* Removed whitespace
* Removed mistake in inferROI
* Small fixes
* One more fix
* Code cleanup
* Code cleanup X2
* bb rstrt
* Fix review comments
* One more fix review comments
* Mistake
* G-API: Introduce ONNX backend for Inference
- Basic operations are implemented (Infer, -ROI, -List, -List2);
- Implemented automatic preprocessing for ONNX models;
- Test suite is extended with `OPENCV_GAPI_ONNX_MODEL_PATH` env for test data
(test data is an ONNX Model Zoo repo snapshot);
- Fixed kernel lookup logic in core G-API:
- Lookup NN kernels not in the default package, but in the associated
backend's aux package. Now two NN backends can work in the same graph.
- Added Infer SSD demo and a combined ONNX/IE demo;
* G-API/ONNX: Fix some of CMake issues
Co-authored-by: Pashchenkov, Maxim <maxim.pashchenkov@intel.com>
Expand ie::Params to support config
* Add config to IE params
* Add test
* Remove comments from tests
* Rename to pluginConfig
* Add one more overloads for pluginConfig
* Add more tests
GAPI: Migration to IE Core API
* Migration to IE Core API
- both versions are maintained
- checked building with all the OpenVINO versions (2019.R1, R2, R3, 2020.4 (newest))
* commit to awake builders
* Addressing comments
- migrated to Core API in 'gapi_ie_infer_test.cpp'
- made Core a singleton object
- dropped redundant steps
* Addressing comments
- modified Mutex locking
* Update
* Addressing comments
- remove getInitMutex()
- reduce amount of #ifdef by abstracting into functions
* return to single IE::Core
* Divide functions readNet and loadNet to avoid warnings on GCC
* Fix deprecated code warnings
* Fix deprecated code warnings on CMake level
* Functions wrapped
- All the functions depended on IE version wrapped into a cv::gapi::ie::wrap namesapace
- All this contained to a new "giebackend/gieapi.hpp" header
- The header shared with G-API infer tests to avoid code duplications
* Addressing comments
- Renamed `gieapi.hpp` -> `giewrapper.hpp`, `cv::gapi::ie::wrap` -> `cv::gimpl::ie::wrap`
- Created new `giewrapper.cpp` source file to avoid potential "multiple definition" problems
- removed unnecessary step SetLayout() in tests
* Enabling two NN infer teest
* Two-NN infer test change for CI
- deleted additional network
- inference of two identical NN used instead
* Fix CI fileNotFound
* Disable MYRIAD test not to fail Custom CI runs
* G-API: Introduce a new gapi::infer2 overload + gaze estimation sample
* G-API/infer2: Introduced static type checking for infer2
- Also added extra tests on the type check routine
* G-API/infer2: Addressed self-review comments in the sample app
- Also fix build on Linux;
* G-API/infer2: Remove incorrect SetLayout(HWC) + dead code
- Also fixed comments in the backend
* G-API/infer2: Continue with self-review
- Fix warnings/compile errors in gaze estimation
- Dropped the use of RTTI (VectorRef::holds()) from the giebackend
- Replaced it with a trait-based enums for GArray<T> and std::vector<T>
- The enums and traits are temporary and need to be unified with
the S11N when it comes
* G-API/infer2: Final self-review items
- Refactored ROIList test to cover 70% for infer<> and infer2<>;
- Fixed the model data discovery routine to be compatible with new
OpenVINO;
- Hopefully fixed the final issues (warnings) with the sample.
* G-API/infer2: address review problems
- Fixed typo in comments;
- Fixed public (Doxygen) comment on GArray<GMat> input case for infer2;
- Made model lookup more flexible to allow new & old OMZ dir layouts.
* G-API/infer2: Change the model paths again
* G-API/infer2: Change the lookup path for test data
* G-API/infer2: use randu instead of imread. CI war is over
