mirror of
https://github.com/opencv/opencv.git
synced 2024-12-11 14:39:11 +08:00
139 lines
6.3 KiB
Markdown
139 lines
6.3 KiB
Markdown
Using Kinect and other OpenNI compatible depth sensors {#tutorial_kinect_openni}
|
||
======================================================
|
||
|
||
Depth sensors compatible with OpenNI (Kinect, XtionPRO, ...) are supported through VideoCapture
|
||
class. Depth map, BGR image and some other formats of output can be retrieved by using familiar
|
||
interface of VideoCapture.
|
||
|
||
In order to use depth sensor with OpenCV you should do the following preliminary steps:
|
||
|
||
-# Install OpenNI library (from here <http://www.openni.org/downloadfiles>) and PrimeSensor Module
|
||
for OpenNI (from here <https://github.com/avin2/SensorKinect>). The installation should be done
|
||
to default folders listed in the instructions of these products, e.g.:
|
||
@code{.text}
|
||
OpenNI:
|
||
Linux & MacOSX:
|
||
Libs into: /usr/lib
|
||
Includes into: /usr/include/ni
|
||
Windows:
|
||
Libs into: c:/Program Files/OpenNI/Lib
|
||
Includes into: c:/Program Files/OpenNI/Include
|
||
PrimeSensor Module:
|
||
Linux & MacOSX:
|
||
Bins into: /usr/bin
|
||
Windows:
|
||
Bins into: c:/Program Files/Prime Sense/Sensor/Bin
|
||
@endcode
|
||
If one or both products were installed to the other folders, the user should change
|
||
corresponding CMake variables OPENNI_LIB_DIR, OPENNI_INCLUDE_DIR or/and
|
||
OPENNI_PRIME_SENSOR_MODULE_BIN_DIR.
|
||
|
||
-# Configure OpenCV with OpenNI support by setting WITH_OPENNI flag in CMake. If OpenNI is found
|
||
in install folders OpenCV will be built with OpenNI library (see a status OpenNI in CMake log)
|
||
whereas PrimeSensor Modules can not be found (see a status OpenNI PrimeSensor Modules in CMake
|
||
log). Without PrimeSensor module OpenCV will be successfully compiled with OpenNI library, but
|
||
VideoCapture object will not grab data from Kinect sensor.
|
||
|
||
-# Build OpenCV.
|
||
|
||
VideoCapture can retrieve the following data:
|
||
|
||
-# data given from depth generator:
|
||
- CAP_OPENNI_DEPTH_MAP - depth values in mm (CV_16UC1)
|
||
- CAP_OPENNI_POINT_CLOUD_MAP - XYZ in meters (CV_32FC3)
|
||
- CAP_OPENNI_DISPARITY_MAP - disparity in pixels (CV_8UC1)
|
||
- CAP_OPENNI_DISPARITY_MAP_32F - disparity in pixels (CV_32FC1)
|
||
- CAP_OPENNI_VALID_DEPTH_MASK - mask of valid pixels (not occluded, not shaded etc.)
|
||
(CV_8UC1)
|
||
|
||
-# data given from BGR image generator:
|
||
- CAP_OPENNI_BGR_IMAGE - color image (CV_8UC3)
|
||
- CAP_OPENNI_GRAY_IMAGE - gray image (CV_8UC1)
|
||
|
||
In order to get depth map from depth sensor use VideoCapture::operator \>\>, e. g. :
|
||
@code{.cpp}
|
||
VideoCapture capture( CAP_OPENNI );
|
||
for(;;)
|
||
{
|
||
Mat depthMap;
|
||
capture >> depthMap;
|
||
|
||
if( waitKey( 30 ) >= 0 )
|
||
break;
|
||
}
|
||
@endcode
|
||
For getting several data maps use VideoCapture::grab and VideoCapture::retrieve, e.g. :
|
||
@code{.cpp}
|
||
VideoCapture capture(0); // or CAP_OPENNI
|
||
for(;;)
|
||
{
|
||
Mat depthMap;
|
||
Mat bgrImage;
|
||
|
||
capture.grab();
|
||
|
||
capture.retrieve( depthMap, CAP_OPENNI_DEPTH_MAP );
|
||
capture.retrieve( bgrImage, CAP_OPENNI_BGR_IMAGE );
|
||
|
||
if( waitKey( 30 ) >= 0 )
|
||
break;
|
||
}
|
||
@endcode
|
||
For setting and getting some property of sensor\` data generators use VideoCapture::set and
|
||
VideoCapture::get methods respectively, e.g. :
|
||
@code{.cpp}
|
||
VideoCapture capture( CAP_OPENNI );
|
||
capture.set( CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE, CAP_OPENNI_VGA_30HZ );
|
||
cout << "FPS " << capture.get( CAP_OPENNI_IMAGE_GENERATOR+CAP_PROP_FPS ) << endl;
|
||
@endcode
|
||
Since two types of sensor's data generators are supported (image generator and depth generator),
|
||
there are two flags that should be used to set/get property of the needed generator:
|
||
|
||
- CAP_OPENNI_IMAGE_GENERATOR -- A flag for access to the image generator properties.
|
||
- CAP_OPENNI_DEPTH_GENERATOR -- A flag for access to the depth generator properties. This flag
|
||
value is assumed by default if neither of the two possible values of the property is not set.
|
||
|
||
Some depth sensors (for example XtionPRO) do not have image generator. In order to check it you can
|
||
get CAP_OPENNI_IMAGE_GENERATOR_PRESENT property.
|
||
@code{.cpp}
|
||
bool isImageGeneratorPresent = capture.get( CAP_PROP_OPENNI_IMAGE_GENERATOR_PRESENT ) != 0; // or == 1
|
||
@endcode
|
||
Flags specifying the needed generator type must be used in combination with particular generator
|
||
property. The following properties of cameras available through OpenNI interfaces are supported:
|
||
|
||
- For image generator:
|
||
|
||
- CAP_PROP_OPENNI_OUTPUT_MODE -- Three output modes are supported: CAP_OPENNI_VGA_30HZ
|
||
used by default (image generator returns images in VGA resolution with 30 FPS),
|
||
CAP_OPENNI_SXGA_15HZ (image generator returns images in SXGA resolution with 15 FPS) and
|
||
CAP_OPENNI_SXGA_30HZ (image generator returns images in SXGA resolution with 30 FPS, the
|
||
mode is supported by XtionPRO Live); depth generator's maps are always in VGA resolution.
|
||
|
||
- For depth generator:
|
||
|
||
- CAP_PROP_OPENNI_REGISTRATION -- Flag that registers the remapping depth map to image map
|
||
by changing depth generator's view point (if the flag is "on") or sets this view point to
|
||
its normal one (if the flag is "off"). The registration process’s resulting images are
|
||
pixel-aligned,which means that every pixel in the image is aligned to a pixel in the depth
|
||
image.
|
||
|
||
Next properties are available for getting only:
|
||
|
||
- CAP_PROP_OPENNI_FRAME_MAX_DEPTH -- A maximum supported depth of Kinect in mm.
|
||
- CAP_PROP_OPENNI_BASELINE -- Baseline value in mm.
|
||
- CAP_PROP_OPENNI_FOCAL_LENGTH -- A focal length in pixels.
|
||
- CAP_PROP_FRAME_WIDTH -- Frame width in pixels.
|
||
- CAP_PROP_FRAME_HEIGHT -- Frame height in pixels.
|
||
- CAP_PROP_FPS -- Frame rate in FPS.
|
||
|
||
- Some typical flags combinations "generator type + property" are defined as single flags:
|
||
|
||
- CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE = CAP_OPENNI_IMAGE_GENERATOR + CAP_PROP_OPENNI_OUTPUT_MODE
|
||
- CAP_OPENNI_DEPTH_GENERATOR_BASELINE = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_BASELINE
|
||
- CAP_OPENNI_DEPTH_GENERATOR_FOCAL_LENGTH = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_FOCAL_LENGTH
|
||
- CAP_OPENNI_DEPTH_GENERATOR_REGISTRATION = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_REGISTRATION
|
||
|
||
For more information please refer to the example of usage
|
||
[videocapture_openni.cpp](https://github.com/opencv/opencv/tree/3.4/samples/cpp/videocapture_openni.cpp) in
|
||
opencv/samples/cpp folder.
|