Created OE 16. Mini Halide (markdown)

Vadim Pisarevsky 2018-03-02 19:31:03 +03:00
parent d3e0b6dc3d
commit e32654540b

33
OE-16.-Mini-Halide.md Normal file

@ -0,0 +1,33 @@
## Implement light-weight Halide or Halide-like compiler and integrate it into OpenCV
* Author: Vadim Pisarevsky
* Link: TBD
* Status: **Draft**
* Platforms: **All**
* Complexity: several man-months to 1-2 man-years.
## Introduction and Rationale
[Halide](http://halide-lang.org) has been a revolutionary technology for accelerated array processing, co-developed by MIT, Stanford, Adobe and Google. It includes the notion of stencil operator, expressed in a form of Halide function, which is implicitly propagated to the whole processed array. There is are quite elegant solutions for the border extrapolation (users just need to specify the extrapolation type and everything else is done automatically) and optimization of the processing pipelines, where intermediate results are computed and cached on demand. Essentially, an array processing pipeline (e.g. image processing pipeline) in Halide is expressed as superposition of functions. This superposition of functions is then translated to the actual CPU or GPU code. Users can do fine control over the generated code, including tiling, vectorization, parallelization, loop unrolling, loop transposition etc. It's done by providing something called "schedule". This explicit separation of the math-like expression of the algorithm in terms of functions and the "close-to-metal" platform-specific schedule has been one of the main novel, revolutionary features of Halide.
Halide was proved to be useful for image processing acceleration; we've also got some promising results using Halide for OpenCV DNN module at the early optimization phase of this project. There are new perspective projects, like [tvm](http://tvmlang.org) that started with Halide engine and transformed it to very efficient deep learning inference engine.
In OpenCV Halide support can be useful for various reasons, such as:
*
## Proposed solution
ONNX support code is open-source project under MIT license hosted at github [\[1\]](https://github.com/onnx/onnx). It's also quite well documented with several popular models converted to this format. It would be quite strait-forward task to create the parser for it in OpenCV similar to Caffe, TensorFlow, Torch and Darknet parsers.
## Impact on existing code, compatibility
It's extra functionality in DNN, so it will not affect any existing code.
## Possible alternatives
Possible alternative would be to have some python script that would parse ONNX models and convert them to TensorFlow. Eventually, we may want to limit the number of formats supported by OpenCV DNN and may be even exclude some (e.g. Torch, Darknet), but ONNX looks like some well-supported thing that should stay around for a long time.
## References
1. [ONNX source code](https://github.com/onnx/onnx)