[G-API] Implement OpenVINO 2.0 backend #23595
### Pull Request Readiness Checklist
Implemented basic functionality for `OpenVINO` 2.0 G-API backend.
#### Overview
- [x] Implement `Infer` kernel with some of essential configurable parameters + IR/Blob models format support.
- [ ] Implement the rest of kernels: `InferList`, `InferROI`, `Infer2` + other configurable params (e.g reshape)
- [x] Asyncrhonous execution support
- [ ] Remote context support
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
G-API: Integration branch for ONNX & Python-related changes #23597
# Changes overview
## 1. Expose ONNX backend's Normalization and Mean-value parameters in Python
* Since Python G-API bindings rely on `Generic` infer to express Inference, the `Generic` specialization of `onnx::Params` was extended with new methods to control normalization (`/255`) and mean-value; these methods were exposed in the Python bindings
* Found some questionable parts in the existing API which I'd like to review/discuss (see comments)
UPD:
1. Thanks to @TolyaTalamanov normalization inconsistencies have been identified with `squeezenet1.0-9` ONNX model itself; tests using these model were updated to DISABLE normalization and NOT using mean/value.
2. Questionable parts were removed and tests still pass.
### Details (taken from @TolyaTalamanov's comment):
`squeezenet1.0.*onnx` - doesn't require scaling to [0,1] and mean/std because the weights of the first convolution already scaled. ONNX documentation is broken. So the correct approach to use this models is:
1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44 but without normalization step:
```
# DON'T DO IT:
# mean_vec = np.array([0.485, 0.456, 0.406])
# stddev_vec = np.array([0.229, 0.224, 0.225])
# norm_img_data = np.zeros(img_data.shape).astype('float32')
# for i in range(img_data.shape[0]):
# norm_img_data[i,:,:] = (img_data[i,:,:]/255 - mean_vec[i]) / stddev_vec[i]
# # add batch channel
# norm_img_data = norm_img_data.reshape(1, 3, 224, 224).astype('float32')
# return norm_img_data
# INSTEAD
return img_data.reshape(1, 3, 224, 224)
```
2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters:
```
net = cv.gapi.onnx.params('squeezenet', model_filename)
net.cfgNormalize('data_0', False)
```
**Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution.
---
`squeezenet1.1.*onnx` - requires scaling to [0,1] and mean/std - onnx documentation is correct.
1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44
2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters:
```
net = cv.gapi.onnx.params('squeezenet', model_filename)
net.cfgNormalize('data_0', True) // default
net.cfgMeanStd('data_0', [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
```
**Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution.
## 2. Expose Fluid & kernel package-related functionality in Python
* `cv::gapi::combine()`
* `cv::GKernelPackage::size()` (mainly for testing purposes)
* `cv::gapi::imgproc::fluid::kernels()`
Added a test for the above.
## 3. Fixed issues with Python stateful kernel handling
Fixed error message when `outMeta()` of custom python operation fails.
## 4. Fixed various issues in Python tests
1. `test_gapi_streaming.py` - fixed behavior of Desync test to avoid sporadic issues
2. `test_gapi_infer_onnx.py` - fixed model lookup (it was still using the ONNX Zoo layout but was NOT using the proper env var we use to point to one).
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
G-API: replace GAPI_Assert() with 'false' and '0' to GAPI_Error()
* gapi: GAPI_Error() macro
* gapi: replace GAPI_Assert() with 'false' and '0' to GAPI_Error()
* build: eliminate 'unreachable code' after CV_Error() (MSVC 2015)
* build: eliminate 'unreachable code' warning for MSVS 2015/2017
- observed in constructors stubs with throwing exception
Minor refactoring
Partially address review comments
Move DX-related stuff from the sample to a default source
Simplify the default OneVPL config
Address minor review comments
Add class for the default VPL source
WIP: Add initial stub for tests with description
Removing default vpl source and minor refactoring
Refactor default files
Fix build and application crash
Address review comments
Add test on VPL + OCL interaction compared to CPU behavior
Fix test
Fixed handling of new stream, especially for stateful OCV kernels
* Fixed handling of new stream, especially for stateful OCV kernels
* Removed duplication from StateInitOnce tests
* Addressed review comments for PR #21731
- Fixed explanation comments
- Expanded test for stateful OCV kernels in Regular mode
* Addressed review comments for PR #21731
- Moved notification about new stream to the constructor
- Added test on state reset for Regular mode
* Addresed review comments
* Addressed review comments
Co-authored-by: Ruslan Garnov <ruslan.garnov@intel.com>
[G-API] Handle exceptions in streaming executor
* Handle exceptions in streaming executor
* Rethrow exception in non-streaming executor
* Clean up
* Put more tests
* Handle exceptions in IE backend
* Handle exception in IE callbacks
* Handle exception in GExecutor
* Handle all exceptions in IE backend
* Not only (std::exception& e)
* Fix comments to review
* Handle input exception in generic way
* Fix comment
* Clean up
* Apply review comments
* Put more comments
* Fix alignment
* Move test outside of HAVE_NGRAPH
* Fix compilation
G-API: oneVPL DX11 inference
* Draft GPU infer
* Fix incorrect subresource_id for array of textures
* Fix for TheOneSurface in different Frames
* Turn on VPP param configuration
* Add cropIn params
* Remove infer sync sample
* Remove comments
* Remove DX11AllocResource extra init
* Add condition for NV12 processing in giebackend
* Add VPP frames pool param configurable
* -M Remove extra WARN & INFOs, Fix custom MAC
* Remove global vars from example, Fix some comments, Disable blobParam due to OV issue
* Conflict resolving
* Revert back pointer cast for cv::any
GAPI: Add OAK backend
* Initial tests and cmake integration
* Add a public header and change tests
* Stub initial empty template for the OAK backend
* WIP
* WIP
* WIP
* WIP
* Runtime dai hang debug
* Refactoring
* Fix hang and debug frame data
* Fix frame size
* Fix data size issue
* Move test code to sample
* tmp refactoring
* WIP: Code refactoring except for the backend
* WIP: Add non-camera sample
* Fix samples
* Backend refactoring wip
* Backend rework wip
* Backend rework wip
* Remove mat encoder
* Fix namespace
* Minor backend fixes
* Fix hetero sample and refactor backend
* Change linking logic in the backend
* Fix oak sample
* Fix working with ins/outs in OAK island
* Trying to fix nv12 problem
* Make both samples work
* Small refactoring
* Remove meta args
* WIP refactoring kernel API
* Change in/out args API for kernels
* Fix build
* Fix cmake warning
* Partially address review comments
* Partially address review comments
* Address remaining comments
* Add memory ownership
* Change pointer-to-pointer to reference-to-pointer
* Remove unnecessary reference wrappers
* Apply review comments
* Check that graph contains only one OAK island
* Minor refactoring
* Address review comments
G-API: oneVPL merge DX11 acceleration
* Merge DX11 initial
* Fold conditions row in MACRO in utils
* Inject DeviceSelector
* Turn on DeviceSelector in DX11
* Change sharedLock logic & Move FMT checking in FrameAdapter c-tor
* Move out NumSuggestFrame to configure params
* Drain file source fix
* Fix compilation
* Force zero initializetion of SharedLock
* Fix some compiler warnings
* Fix integer comparison warnings
* Fix integers in sample
* Integrate Demux
* Fix compilation
* Add predefined names for some CfgParam
* Trigger CI
* Fix MultithreadCtx bug, Add Dx11 GetBlobParam(), Get rif of ATL CComPtr
* Fix UT: remove unit test with deprecated video from opencv_extra
* Add creators for most usable CfgParam
* Eliminate some warnings
* Fix warning in GAPI_Assert
* Apply comments
* Add VPL wrapped header with MSVC pragma to get rid of global warning masking
[G-API] Fix issue of getting 1D Mat out of RMat::View
* Fix issue of getting 1D Mat out of RMat::View
- added test
- fixed for standalone too (removed Assert(dims.empty()))
* Fixed asVeiw() function for standalone
* Put more detailed comment
G-API: Removing G-API test code that is a reflection of ts module
* gapi: don't hijack testing infrastructure
* Removed initDataPath functionality (ts module exists)
* Removed false for ocv_extra data from findDataFile
Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
[G-API] Fix bugs in GIEBackend
* Remove inputs/outputs map from IEUnit
* Add test
* Add NV12 test
* Reorganize setBlob function
* Check that backend don't overwrite blob precision
* Stop setting config to global IE::Core
* Replace mutable to const_cast
* Update modules/gapi/test/infer/gapi_infer_ie_test.cpp
* Update modules/gapi/test/infer/gapi_infer_ie_test.cpp
* Make blob parameter as const ref
* Cosmetic fixes
* Fix failed test on inferROI
* Removed double ref for ii
* Disable tests
* Skip tests if device not available
* Use Sync prim under shared_ptr to avoid issue on MAC
* Apply WA for IE::Core
* Apply WA for MAC build
* Try to apply another WA
* Not release IE::Core for apple
* Put comment
* Support PreprocInfo for
* InferROI
* InferList
* InferList2
* Remove empty line
* Fix alignment
Co-authored-by: Maxim Pashchenkov <maxim.pashchenkov@intel.com>
