3cd57ea09e
New dnn engine #26056 This is the 1st PR with the new engine; CI is green and PR is ready to be merged, I think. Merge together with https://github.com/opencv/opencv_contrib/pull/3794 --- **Known limitations:** * [solved] OpenVINO is temporarily disabled, but is probably easy to restore (it's not a deal breaker to merge this PR, I guess) * The new engine does not support any backends nor any targets except for the default CPU implementation. But it's possible to choose the old engine when loading a model, then all the functionality is available. * [Caffe patch is here: #26208] The new engine only supports ONNX. When a model is constructed manually or is loaded from a file of different format (.tf, .tflite, .caffe, .darknet), the old engine is used. * Even in the case of ONNX some layers are not supported by the new engine, such as all quantized layers (including DequantizeLinear, QuantizeLinear, QLinearConv etc.), LSTM, GRU, .... It's planned, of course, to have full support for ONNX by OpenCV 5.0 gold release. When a loaded model contains unsupported layers, we switch to the old engine automatically (at ONNX parsing time, not at `forward()` time). * Some layers , e.g. Expat, are only partially supported by the new engine. In the case of unsupported flavours it switches to the old engine automatically (at ONNX parsing time, not at `forward()` time). * 'Concat' graph optimization is disabled. The optimization eliminates Concat layer and instead makes the layers that generate tensors to be concatenated to write the outputs to the final destination. Of course, it's only possible when `axis=0` or `axis=N=1`. The optimization is not compatible with dynamic shapes since we need to know in advance where to store the tensors. Because some of the layer implementations have been modified to become more compatible with the new engine, the feature appears to be broken even when the old engine is used. * Some `dnn::Net` API is not available with the new engine. Also, shape inference may return false if some of the output or intermediate tensors' shapes cannot be inferred without running the model. Probably this can be fixed by a dummy run of the model with zero inputs. * Some overloads of `dnn::Net::getFLOPs()` and `dnn::Net::getMemoryConsumption()` are not exposed any longer in wrapper generators; but the most useful overloads are exposed (and checked by Java tests). * [in progress] A few Einsum tests related to empty shapes have been disabled due to crashes in the tests and in Einsum implementations. The code and the tests need to be repaired. * OpenCL implementation of Deconvolution is disabled. It's very bad and very slow anyway; need to be completely revised. * Deconvolution3D test is now skipped, because it was only supported by CUDA and OpenVINO backends, both of which are not supported by the new engine. * Some tests, such as FastNeuralStyle, checked that the in the case of CUDA backend there is no fallback to CPU. Currently all layers in the new engine are processed on CPU, so there are many fallbacks. The checks, therefore, have been temporarily disabled. --- - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| build_xcframework.py | ||
| cv_build_utils.py | ||
| readme.md | ||
Building for Apple Platforms
build_xcframework.py creates an xcframework supporting a variety of Apple platforms.
You'll need the following to run these steps:
- MacOS 10.15 or later
- Python 3.6 or later
- CMake 3.18.5/3.19.0 or later (make sure the
cmakecommand is available on your PATH) - Xcode 12.2 or later (and its command line tools)
You can then run build_xcframework.py, as below:
cd ~/<my_working_directory>
python opencv/platforms/apple/build_xcframework.py --out ./build_xcframework
Grab a coffee, because you'll be here for a while. By default this builds OpenCV for 8 architectures across 4 platforms:
- iOS (
--iphoneos_archs): arm64, armv7 - iOS Simulator (
--iphonesimulator_archs): x86_64, arm64 - macOS (
--macos_archs): x86_64, arm64 - Mac Catalyst (
--catalyst_archs): x86_64, arm64
If everything's fine, you will eventually get opencv2.xcframework in the output directory.
The script has some configuration options to exclude platforms and architectures you don't want to build for. Use the --help flag for more information.
How it Works
This script generates a fat .framework for each platform you specify, and stitches them together into a .xcframework. This file can be used to support the same architecture on different platforms, which fat .frameworks don't allow. To build the intermediate .frameworks, build_xcframework.py leverages the build_framework.py scripts in the ios and osx platform folders.
Passthrough Arguments
Any arguments that aren't recognized by build_xcframework.py will be passed to the platform-specific build_framework.py scripts. The --without flag mentioned in the examples is an example of this in action. For more info, see the --help info for those scripts.
Examples
You may override the defaults by specifying a value for any of the *_archs flags. For example, if you want to build for arm64 on every platform, you can do this:
python build_xcframework.py --out somedir --iphoneos_archs arm64 --iphonesimulator_archs arm64 --macos_archs arm64 --catalyst_archs arm64
If you want to build only for certain platforms, you can supply the --build_only_specified_archs flag, which makes the script build only the archs you directly ask for. For example, to build only for Catalyst, you can do this:
python build_xcframework.py --out somedir --catalyst_archs x86_64,arm64 --build_only_specified_archs
You can also build without OpenCV functionality you don't need. You can do this by using the --without flag, which you use once per item you want to go without. For example, if you wanted to compile without video or objc, you'd can do this:
python build_xcframework.py --out somedir --without video --without objc
(if you have issues with this, try using =, e.g. --without=video --without=objc, and file an issue on GitHub.)