Fixes#22799
Replaces #21559 which was taken as a base
Connected PR in contrib: [#3388@contrib](https://github.com/opencv/opencv_contrib/pull/3388)
### Changes
OK, now this is more Odometry-related PR than Volume-related. Anyway,
* `Volume` class gets wrapped
* The same was done for helper classes like `VolumeSettings`, `OdometryFrame` and `OdometrySettings`
* `OdometryFrame` constructor signature changed to more convenient where depth goes on 1st place, RGB image on 2nd.
This works better for depth-only `Odometry` algorithms.
* `OdometryFrame` is checked for amount of pyramid layers inside `Odometry::compute()`
* `Odometry` was fully wrapped + more docs added
* Added Python tests for `Odometry`, `OdometryFrame` and `Volume`
* Added Python sample for `Volume`
* Minor fixes including better var names
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
### Changes
* Duplicated code removal in TSDF tests by implementing them with fixtures and GTest params
* e.g. separate OCL tests file removed
* as a result, more test cases are covered
* the same's done for perf tests
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Corresponding contrib PR: #3382@contrib
Changes
- Volume::raycast(): camera intrinsics can be explicitly passed to the function. If not, the ones from current volume settings are used
- getVolumeDimensions() renamed to getVolumeStrides() because they are strides actually
- TSDF tests: OpenCLStatusRevert and parametrized fixture
- ColorTSDF::integrate(): extra RGB projector is redundant, removed
- Minor changes
This PR contains:
- a new property enableGrowth which controls should the HashTSDF be extended during integration by adding new volume units or not
- a new method getBoundingBox which calculates the size of currently occupied data
- a set of tests to check that new functionality
- a fix for TSDF GPU reset (data is correctly zeroed now using floatToTsdf() function)
minor changes
There's a bug which appears when volume pose contains non-trivial rotation.
It results in wrong depth integration which can be observed during raycasting
or points/normals export.
- This PR fixes the bug for both CPU and OpenCL
- There is a reproducer for the bug
- The copy behavior of VolumeSettings fixed (now copy constructor creates a deep copy)
- Minor changes (e.g. unused vars removed)
Complement PR: #3366@contrib
Changes
OdometryFrame losts its getters: a user can provide data at construction stage only, pyramids and other generated data is read-only now
OdometryFrame is based on UMats: no TMat templates inside, CPU operations are done with UMat::getMat() method, chaining issues are solved ad-hoc
No more Odometry::createOdometryFrame() method, frames are compatible with all odometry algorithms
Normals computer is cached inside Odometry and exposed to API as well as its settings
Volume::raycast() won't return the result in OdometryFrame anymore
Added test for Odometry::prepareFrame*() & other test fixes
Minor code improvements
TODOs:
fix TODOs in code
lower acceptable accuracy errors
variables refactoring
levmarq fix
initIntrinsicParams2D() refactoring
undo LM fix
cameraCalcJErr: made a lambda; warnings fixed; vars rearranged & renamed, jacobian buffers, perViewErrors and allErrors fix, etc.
stereoCalibrate: internal vars in callback
stereoCalibrate: capture only useful variables
stereoCalibrate: perViewError fix + minors
rvecs and tvecs are not pointers anymore
no extra lambda
newObjPoints: no pointers
stdDevs: no pointers
_Jo removed: not used
Range::all() -> rowRange, colRange
param and mask are std::vectors now
indices shortened
tabs
less func-scoped vars; TODOs
less formatting & renaming changes
trailing whitespaces
less diff
less changes
less changes
Range::all() back
perViewErr ptr fix
NINTRINSIC captured
try to fix warning
trying to fix a warning
fix warnings, another attempt
RgbdNormals: setMethod() removed as useless
RgbdNormals: tests + cross product, to be fixed
+ cross product
LINEMOD: diffThreshold param added + tests fixed
minor
diffThreshold fix
points3dToDepth16U fix
normals computer diffThreshold fix
random plane generation fixed + diffThreshold fix
Rendered normals test rewritten to GTest Params
random plane generation: scale
RGBD_Normals tests: thresholds tuned
Rendered normals tests: 64F support added
Random planes normal tests rewritten to GTest Params
LINEMOD and CrossProduct fix
SRI threshold raised
NormalsRandomPlanes: thresholds raised
assert on unknown alg; minor
fix
frame size reduced
TIFF replaced by YAML.GZ
depthTo3d test changed
cv::transform is used
fix warning
nanMask()
flipAxes()
absDotPixel() + forgotten code
helper functions removed
RGBDNormals: checkNormals() and compare LINEMOD's pts3d to depth input
Rendered: another criteria; thresholds; LINEMOD's pts3d to depth input comparison
thresholds raised a bit
SRI slightly optimized
assert change
normal tests refactored, parametrized, split
trailing namespace, thresholds raised
SRI caching optimized a lot
normal tests rewritten to fixture, no loop
minor
runCase() joined with testIt()
thresholds were put into GTest params
ternary operator
RgbdNormalsTest merged into NormalsRandomPlanes; RgbdPlanes moved closer to tests
normal test minor refactoring
plane finder tests refactored to GTest Params
skip tests
thresholds raised
plane test minor
plane tests: timers dropped, nPlanes put into GTest Params; refactoring
generated normals tests: minor refactoring
flipAxes() templated
rendered normals tests refactored: thresholds to GTest Params
CV_Error -> ASSERT_FALSE
Add normal estimation and region growing algorithm for point cloud
* Add normal estimation and region growing algorithm for point cloud
* 1.Modified documentation for normal estimation;2.Converted curvature in region growing to absolute values;3.Changed the data type of threshold from float to double;4.Fixed some bugs;
* Finished documentation
* Add tests for normal estimation. Test the normal and curvature of each point in the plane and sphere of the point cloud.
* Fix some warnings caused by to small numbers in test
* Change the test to calculate the average difference instead of comparing each normal and curvature
* Fixed the bugs found by testing
* Redesigned the interface and fixed problems:
1. Make the interface compatible with radius search.
2. Make region growing optionally sortable on results.
3. Modified the region growing interface.
4. Format reference.
5. Removed sphere test.
* Fix warnings
* Remove flann dependency
* Move the flann dependency to the corresponding test
GSoC module to save and load point cloud
* Add functionality to read point cloud data from files
* address issues found on review, add tests for mesh, refactor
* enable fail-safe execution and empty arrays as output
* Some improvements for point cloud io module
Co-authored-by: Julie Bareeva <julia.bareeva@xperience.ai>
Add support for 3D point cloud segmentation, using the USAC framework.
* Modify the RANSAC framework in usac such that RANSAC can be used in 3D point cloud segmentation.
* 1. Add support for 3D point cloud segmentation, using the USAC framework.
2. Add solvers, error estimators for plane model and sphere model.
* Added code samples to the comments of class SACSegmentation.
* 1. Update the segment interface parameters of SACSegmentation.
2. Fix some errors in variable naming.
* Add tests for plane detection.
* 1. Add tests for sphere segmentation.
2. Fix some bugs found by tests.
3. Rename "segmentation" to "sac segmentation".
4. Rename "detect" to "segment".
TODO: Too much duplicate code, the structure of the test needs to be rebuilt.
* 1. Use SIMD acceleration for plane model and sphere model error estimation.
2. Optimize the RansacQualityImpl#getScore function to avoid multiple calls to the error#getError function.
3. Fix a warning in test_sac_segmentation.cpp.
* 1. Fix the warning of ModelConstraintFunction ambiguity.
2. Fix warning: no previous declaration for'void cv::usac::modelParamsToUsacConfig(cv::Ptr<cv::usac::SimpleUsacConfig>&, const cv::Ptr<const cv::usac::Model>& )
* Fix a warning in test_sac_segmentation.cpp about direct comparison of different types of data.
* Add code comments related to the interpretation of model coefficients.
* Update the use of custom model constraint functions.
* Simplified test code structure.
* Update the method of checking plane models.
* Delete test for cylinder.
* Add some comments about UniversalRANSAC.
* 1. The RANSAC paper in the code comments is referenced using the bibtex format.
2. The sample code in the code comments is replaced using @snippet.
3. Change the public API class SACSegmentation to interface.
4. Clean up the old useless code.
* fix warning(no previous declaration) in 3d_sac_segmentation.cpp.
* Fix compilation errors caused by 3d_sac_segmentation.cpp.
* Move the function sacModelMinimumSampleSize() from ptcloud.hpp to sac_segmentation.cpp.
* 1. Change the interface for setting the number of threads to the interface for setting whether to be parallel.
2. Move interface implementation code in ptcloud_utils.hpp to ptcloud_utils.cpp.
* SACSegmentation no longer inherits Algorithm.
* Add the constructor and destructor of SACSegmentation.
* 1. For the declaration of the common API, the prefix and suffix of the parameter names no longer contain underscores.
2. Rename the function _getMatFromInputArray -> getPointsMatFromInputArray.
3. Change part of CV_CheckDepth to CV_CheckDepthEQ.
4. Remove the doxygen flag from the source code.
5. Update the loop termination condition of SIMD in the point cloud section of 3D module.
* fix warning: passing 'bool' chooses 'int' over 'size_t {aka unsigned int}' .
* fix warning: passing 'bool' chooses 'int' over 'size_t {aka unsigned int}' .
New odometry Pipeline
* first intergation
* tests run, but not pass
* add previous version of sigma calc
* add minor comment
* strange fixes
* fix fast ICP
* test changes; fast icp still not work correctly
* finaly, it works
* algtype fix
* change affine comparison
* boolean return
* fix bug with angle and cos
* test pass correctly
* fix for kinfu pipeline
* add compute points normals
* update for new odometry
* change odometry_evaluation
* odometry_evaluation works
* change debug logs
* minor changes
* change depth setting in odometryFrame
* fastICP works with 4num points
* all odometries work with 4mun points
* odometry full works on 4num points and normals
* replace ICP with DEPTH; comments replacements
* create prepareFrame; add docs for Odometry
* change getPyramids()
* delete extra code
* add intrinsics; but dont works
* bugfix with nan checking
* add gpu impl
* change createOdometryFrame func
* remove old fastICP code
* comments fix
* add comments
* minor fixes
* other minor fixes
* add channels assert
* add impl for odometry settings
* add pimpl to odometry
* linux warning fix
* linux warning fix 1
* linux warning fix 2
* linux error fix
* linux warning fix 3
* linux warning fix 4
* linux error fix 2
* fix test warnings
* python build fix
* doxygen fix
* docs fix
* change normal tests for 4channel point
* all Normal tests pass
* plane works
* add warp frame body
* minor fix
* warning fixes
* try to fix
* try to fix 1
* review fix
* lvls fix
* createOdometryFrame fix
* add comment
* const reference
* OPENCV_3D_ prefix
* const methods
* OdometryFramePyramidType ifx
* add assert
* precomp moved upper
* delete types_c
* add assert for get and set functions
* minor fixes
* remove core.hpp from header
* ocl_run add
* warning fix
* delete extra comment
* minor fix
* setDepth fix
* delete underscore
* odometry settings fix
* show debug image fix
* build error fix
* other minor fix
* add const to signatures
* fix
* conflict fix
* getter fix
Accelerated 3D point cloud Farthest Point Sampling calculation using SIMD.
* Add several 3D point cloud sampling functions: Random, VoxelGrid, FarthestPoint.
* Made some code detail changes and exposed the random number generator parameters at the interface.
* Add simple tests for sampling.
* Modify interface output parameters.
* Modify interface return value.
* The sampling test is modified for the new changes of function interface.
* Improved test of VoxelGridFilterSampling
* Improved test of VoxelGridFilterSampling and FPS.
* Add test for the dist_lower_limit arguments of FPS function.
* Optimization function _getMatFromInputArray.
* Optimize the code style and some details according to the suggestions.
* Clear prefix cv: in the source code.
* Change the initialization of Mat in the sampling test.
* 1. Unified code style
2. Optimize randomSampling method
* 1. Optimize code comments.
2. Remove unused local variables.
* Rebuild the structure of the test, make the test case more reliable, and change the code style.
* Update test_sampling.cpp
Fix a warning.
* Use SIMD to optimize the farthest point sampling.
* Optimize the farthest point sampling SIMD code.
* 1. remove `\n` from the ptcloud.hpp comment.
2. updated the default value of the argument arrangement_of_points in the _getMatFromInputArray function in ptcloud_utils.hpp from 0 to 1, since the latter is more commonly used (such arrangement is easier for SIMD acceleration).
3. removed two functions in ptcloud_utils.hpp that were not used.
* Remove the <br> in the comment.
* Fix whitespace issues.
Add 3D point cloud sampling functions to branch next
* Add several 3D point cloud sampling functions: Random, VoxelGrid, FarthestPoint.
* Made some code detail changes and exposed the random number generator parameters at the interface.
* Add simple tests for sampling.
* Modify interface output parameters.
* Modify interface return value.
* The sampling test is modified for the new changes of function interface.
* Improved test of VoxelGridFilterSampling
* Improved test of VoxelGridFilterSampling and FPS.
* Add test for the dist_lower_limit arguments of FPS function.
* Optimization function _getMatFromInputArray.
* Optimize the code style and some details according to the suggestions.
* Clear prefix cv: in the source code.
* Change the initialization of Mat in the sampling test.
* 1. Unified code style
2. Optimize randomSampling method
* 1. Optimize code comments.
2. Remove unused local variables.
* Rebuild the structure of the test, make the test case more reliable, and change the code style.
* Update test_sampling.cpp
Fix a warning.
Moving RGBD parts to 3d
* files moved from rgbd module in contrib repo
* header paths fixed
* perf file added
* lapack compilation fixed
* Rodrigues fixed in tests
* rgbd namespace removed
* headers fixed
* initial: rgbd files moved to 3d module
* rgbd updated from latest contrib master; less file duplication
* "std::" for sin(), cos(), etc.
* KinFu family -> back to contrib
* paths & namespaces
* removed duplicates, file version updated
* namespace kinfu removed from 3d module
* forgot to move test_colored_kinfu.cpp to contrib
* tests fixed: Params removed
* kinfu namespace removed
* it works without objc bindings
* include headers fixed
* tests: data paths fixed
* headers moved to/from public API
* Intr -> Matx33f in public API
* from kinfu_frame.hpp to utils.hpp
* submap: Intr -> Matx33f, HashTSDFVolume -> Volume; no extra headers
* no RgbdFrame class, no Mat fields & arg -> InputArray & pImpl
* get/setPyramidAt() instead of lots of methods
* Mat -> InputArray, TMat
* prepareFrameCache: refactored
* FastICPOdometry: +truncate threshold, +depthFactor; Mat/UMat choose
* Mat/UMat choose
* minor stuff related to headers
* (un)signed int warnings; compilation minor issues
* minors: submap: pyramids -> OdometryFrame; tests fix; FastICP minor; CV_EXPORTS_W for kinfu_frame.hpp
* FastICPOdometry: caching, rgbCameraMatrix
* OdometryFrame: pyramid%s% -> pyramids[]
* drop: rgbCameraMatrix from FastICP, RGB cache mode, makeColoredFrameFrom depth and all color-functions it calls
* makeFrameFromDepth, buildPyramidPointsNormals -> from public to internal utils.hpp
* minors
* FastICPOdometry: caching updated, init fields
* OdometryFrameImpl<UMat> fixed
* matrix building fixed; minors
* returning linemode back to contrib
* params.pose is Mat now
* precomp headers reorganized
* minor fixes, header paths, extra header removed
* minors: intrinsics -> utils.hpp; whitespaces; empty namespace; warning fixed
* moving declarations from/to headers
* internal headers reorganized (once again)
* fix include
* extra var fix
* fix include, fix (un)singed warning
* calibration.cpp: reverting back
* headers fix
* workaround to fix bindings
* temporary removed wrappers
* VolumeType -> VolumeParams
* (temporarily) removing wrappers for Volume and VolumeParams
* pyopencv_linemod -> contrib
* try to fix test_rgbd.py
* headers fixed
* fixing wrappers for rgbd
* fixing docs
* fixing rgbdPlane
* RgbdNormals wrapped
* wrap Volume and VolumeParams, VolumeType from enum to int
* DepthCleaner wrapped
* header folder "rgbd" -> "3d"
* fixing header path
* VolumeParams referenced by Ptr to support Python wrappers
* render...() fixed
* Ptr<VolumeParams> fixed
* makeVolume(... resolution -> [X, Y, Z])
* fixing static declaration
* try to fix ios objc bindings
* OdometryFrame::release...() removed
* fix for Odometry algos not supporting UMats: prepareFrameCache<>()
* preparePyramidMask(): fix to compile with TMat = UMat
* fixing debug guards
* removing references back; adding makeOdometryFrame() instead
* fixing OpenCL ICP hanging (some threads exit before reaching the barrier -> the rest threads hang)
* try to fix objc wrapper warnings; rerun builders
* VolumeType -> VolumeKind
* try to fix OCL bug
* prints removed
* indentation fixed
* headers fixed
* license fix
* WillowGarage licence notion removed, since it's in OpenCV's COPYRIGHT already
* KinFu license notion shortened
* debugging code removed
* include guards fixed
* KinFu license left in contrib module
* isValidDepth() moved to private header
* indentation fix
* indentation fix in src files
* RgbdNormals rewritten to pImpl
* minor
* DepthCleaner removed due to low code quality, no depthScale provided, no depth images found to be successfully filtered; can be replaced by bilateral filtering
* minors, indentation
* no "private" in public headers
* depthTo3d test moved from separate file
* Normals: setDepth() is useless, removing it
* RgbdPlane => findPlanes()
* rescaleDepth(): minor
* warpFrame: minor
* minor TODO
* all Odometries (except base abstract class) rewritten to pImpl
* FastICPOdometry now supports maxRotation and maxTranslation
* minor
* Odometry's children: now checks are done in setters
* get rid of protected members in Odometry class
* get/set cameraMatrix, transformType, maxRot/Trans, iters, minGradients -> OdometryImpl
* cameraMatrix: from double to float
* matrix exponentiation: Eigen -> dual quaternions
* Odometry evaluation fixed to reuse existing code
* "small" macro fixed by undef
* pixNorm is calculated on CPU only now (and then uploads on GPU)
* test registration: no cvtest classes
* test RgbdNormals and findPlanes(): no cvtest classes
* test_rgbd.py: minor fix
* tests for Odometry: no cvtest classes; UMat tests; logging fixed
* more CV_OVERRIDE to overriden functions
* fixing nondependent names to dependent
* more to prev commit
* forgotten fixes: overriden functions, (non)dependent names
* FastICPOdometry: fix UMat support when OpenCL is off
* try to fix compilation: missing namespaces
* Odometry: static const-mimicking functions to internal constants
* forgotten change to prev commit
* more forgotten fixes
* do not expose "submap.hpp" by default
* in-class enums: give names, CamelCase, int=>enums; minors
* namespaces, underscores, String
* std::map is used by pose graph, adding it
* compute()'s signature fixed, computeImpl()'s too
* RgbdNormals: Mat -> InputArray
* depth.hpp: Mat -> InputArray
* cameraMatrix: Matx33f -> InputArray + default value + checks
* "details" headers are not visible by default
* TSDF tests: rearranging checks
* cameraMatrix: no (realistic) default value
* renderPointsNormals*(): no wrappers for them
* debug: assert on empty frame in TSDF tests
* debugging code for TSDF GPU
* debug from integrate to raycast
* no (non-zero) default camera matrix anymore
* drop debugging code (does not help)
* try to fix TSDF GPU: constant -> global const ptr
