New dnn engine #26056
This is the 1st PR with the new engine; CI is green and PR is ready to be merged, I think.
Merge together with https://github.com/opencv/opencv_contrib/pull/3794
---
**Known limitations:**
* [solved] OpenVINO is temporarily disabled, but is probably easy to restore (it's not a deal breaker to merge this PR, I guess)
* The new engine does not support any backends nor any targets except for the default CPU implementation. But it's possible to choose the old engine when loading a model, then all the functionality is available.
* [Caffe patch is here: #26208] The new engine only supports ONNX. When a model is constructed manually or is loaded from a file of different format (.tf, .tflite, .caffe, .darknet), the old engine is used.
* Even in the case of ONNX some layers are not supported by the new engine, such as all quantized layers (including DequantizeLinear, QuantizeLinear, QLinearConv etc.), LSTM, GRU, .... It's planned, of course, to have full support for ONNX by OpenCV 5.0 gold release. When a loaded model contains unsupported layers, we switch to the old engine automatically (at ONNX parsing time, not at `forward()` time).
* Some layers , e.g. Expat, are only partially supported by the new engine. In the case of unsupported flavours it switches to the old engine automatically (at ONNX parsing time, not at `forward()` time).
* 'Concat' graph optimization is disabled. The optimization eliminates Concat layer and instead makes the layers that generate tensors to be concatenated to write the outputs to the final destination. Of course, it's only possible when `axis=0` or `axis=N=1`. The optimization is not compatible with dynamic shapes since we need to know in advance where to store the tensors. Because some of the layer implementations have been modified to become more compatible with the new engine, the feature appears to be broken even when the old engine is used.
* Some `dnn::Net` API is not available with the new engine. Also, shape inference may return false if some of the output or intermediate tensors' shapes cannot be inferred without running the model. Probably this can be fixed by a dummy run of the model with zero inputs.
* Some overloads of `dnn::Net::getFLOPs()` and `dnn::Net::getMemoryConsumption()` are not exposed any longer in wrapper generators; but the most useful overloads are exposed (and checked by Java tests).
* [in progress] A few Einsum tests related to empty shapes have been disabled due to crashes in the tests and in Einsum implementations. The code and the tests need to be repaired.
* OpenCL implementation of Deconvolution is disabled. It's very bad and very slow anyway; need to be completely revised.
* Deconvolution3D test is now skipped, because it was only supported by CUDA and OpenVINO backends, both of which are not supported by the new engine.
* Some tests, such as FastNeuralStyle, checked that the in the case of CUDA backend there is no fallback to CPU. Currently all layers in the new engine are processed on CPU, so there are many fallbacks. The checks, therefore, have been temporarily disabled.
---
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Fixes for Swift troubles
* Remove NS_SWIFT_NAME override for Point, Rect, and Size due to Darwin namespace conflict
* Fix swift_type overrides in objc generator
* Add backwards compatibility Swift typealiases for Point, Rect, Size
* Add disable-swift build option to iOS/macOS builds
* Add import directive to swift source when building with disable-swift
Co-authored-by: Chris Ballinger <cballinger@rightpoint.com>
Make xcframework output path argument explicit and required
* Make output path argument explicit and required
* Improve xcframework documentation
* Add TODOs for future breaking changes on build_framework.py scripts
Support XCFramework builds, Catalyst
* Early work on xcframework support
* Improve legibility
* Somehow this works
* Specify ABIs in a place where they won't get erased
If you pass in the C/CXX flags from the Python script, they won't be respected. By doing it in the actual toolchain, the options are respected and Catalyst successfully links.
* Clean up and push updates
* Actually use Catalyst ABI
Needed to specify EXE linker flags to get compiler tests to link to the Catalyst ABIs.
* Clean up
* Revert changes to common toolchain that don't matter
* Try some things
* Support Catalyst build in OSX scripts
* Remove unnecessary iOS reference to AssetsLibrary framework
* Getting closer
* Try some things, port to Python 3
* Some additional fixes
* Point Cmake Plist gen to osx directory for Catalyst targets
* Remove dynamic lib references for Catalyst, copy iOS instead of macos
* Add flag for building only specified archs, remove iOS catalyst refs
* Add build-xcframework.sh
* Update build-xcframework.sh
* Add presumptive Apple Silicon support
* Add arm64 iphonesimulator target
* Fix xcframework build
* Working on arm64 iOS simulator
* Support 2.7 (replace run with check_output)
* Correctly check output of uname_m against arch
* Clean up
* Use lipo for intermediate frameworks, add python script
Remove unneeded __init__.py
* Simplify python xcframework build script
* Add --only-64-bit flag
* Add --framework-name flag
* Document
* Commit to f-strings, improve console output
* Add i386 to iphonesimulator platform in xcframework generator
* Enable objc for non-Catalyst frameworks
* Fix xcframework builder for paths with spaces
* Use arch when specifying Catalyst build platform in build command
* Fix incorrect settings for framework_name argparse configuration
* Prefer underscores instead of hyphens in new flags
* Move Catalyst flags to where they'll actually get used
* Use --without=objc on Catalyst target for now
* Remove get_or_create_folder and simplify logic
* Remove unused import
* Tighten up help text
* Document
* Move common functions into cv_build_utils
* Improve documentation
* Remove old build script
* Add readme
* Check for required CMake and Xcode versions
* Clean up TODOs and re-enable `copy_samples()`
Remove TODO
Fixup
* Add missing print_function import
* Clarify CMake dependency documentation
* Revert python2 change in gen_objc
* Remove unnecessary builtins imports
* Remove trailing whitespace
* Avoid building Catalyst unless specified
This makes Catalyst support a non-breaking change, though defaults should be specified when a breaking change is possible.
* Prevent lipoing for the same archs on different platforms before build
* Rename build-xcframework.py to build_xcframework.py
* Check for duplicate archs more carefully
* Prevent sample copying error when directory already exists
This can happen when building multiple architectures for the same platform.
* Simplify code for checking for default archs
* Improve build_xcframework.py header text
* Correctly resolve Python script paths
* Parse only known args in ios/osx build_framework.py
* Pass through uncaptured args in build_xcframework to osx/ios build
* Fix typo
* Fix typo
* Fix unparameterized build path for intermediate frameworks
* Fix dyanmic info.plist path for catalyst
* Fix utf-8 Python 3 issue
* Add dynamic flag to osx script
* Rename platform to platforms, remove armv7s and i386
* Fix creation of dynamic framework on maccatalyst and macos
* Update platforms/apple/readme.md
* Add `macos_archs` flag and deprecate `archs` flag
* Allow specification of archs when generating xcframework from terminal
* Change xcframework platform argument names to match archs flag names
* Remove platforms as a concept and shadow archs flags from ios/osx .py
* Improve documentation
* Fix building of objc module on Catalyst, excluding Swift
* Clean up build folder logic a bit
* Fix framework_name flag
* Drop passthrough_args, use unknown_args instead
* minor: coding style changes
Co-authored-by: Chris Ballinger <cballinger@rightpoint.com>
* Universal Build for Big Sur
* Refactor MacOS/iOS build to only ever build one architecture at a time + improve code readability
* Workaround for CMake issue 20989
Objc binding
* Initial work on Objective-C wrapper
* Objective-C generator script; update manually generated wrappers
* Add Mat tests
* Core Tests
* Imgproc wrapper generation and tests
* Fixes for Imgcodecs wrapper
* Miscellaneous fixes. Swift build support
* Objective-C wrapper build/install
* Add Swift wrappers for videoio/objdetect/feature2d
* Framework build;iOS support
* Fix toArray functions;Use enum types whenever possible
* Use enum types where possible;prepare test build
* Update test
* Add test runner scripts for iOS and macOS
* Add test scripts and samples
* Build fixes
* Fix build (cmake 3.17.x compatibility)
* Fix warnings
* Fix enum name conflicting handling
* Add support for document generation with Jazzy
* Swift/Native fast accessor functions
* Add Objective-C wrapper for calib3d, dnn, ml, photo and video modules
* Remove IntOut/FloatOut/DoubleOut classes
* Fix iOS default test platform value
* Fix samples
* Revert default framework name to opencv2
* Add converter util functions
* Fix failing test
* Fix whitespace
* Add handling for deprecated methods;fix warnings;define __OPENCV_BUILD
* Suppress cmake warnings
* Reduce severity of "jazzy not found" log message
* Fix incorrect #include of compatibility header in ios.h
* Use explicit returns in subscript/get implementation
* Reduce minimum required cmake version to 3.15 for Objective-C/Swift binding
* Add debug build option to MacOs and iOS build scripts
* osx: allow selection of MACOSX_DEPLOYMENT_TARGET
* Add --debug_info flag to iOS and macOS builds for building with debug info
Add ability to build iOS and macOS frameworks with nonfree modules (#13424)
* Allow building ios framework with nonfree
* Allow building osx framework with nonfree
Without it set, the build will choose the current OS version as the deployment target. 10.9 is the earliest usable, since that is when OS X introduced libc++.
