mirror of
https://github.com/opencv/opencv.git
synced 2024-11-27 20:50:25 +08:00
Merge pull request #25907 from Andy-SAKN:Andy-SAKN-add_orbbec
Add tutorial on using Orbbec 3D cameras (UVC) #25907 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
parent
1125755345
commit
325da2b2f4
BIN
doc/tutorials/app/images/orbbec_uvc_cpp.jpg
Normal file
BIN
doc/tutorials/app/images/orbbec_uvc_cpp.jpg
Normal file
Binary file not shown.
After Width: | Height: | Size: 32 KiB |
BIN
doc/tutorials/app/images/orbbec_uvc_python.jpg
Normal file
BIN
doc/tutorials/app/images/orbbec_uvc_python.jpg
Normal file
Binary file not shown.
After Width: | Height: | Size: 16 KiB |
@ -3,7 +3,7 @@ Using Creative Senz3D and other Intel RealSense SDK compatible depth sensors {#t
|
|||||||
|
|
||||||
@tableofcontents
|
@tableofcontents
|
||||||
|
|
||||||
@prev_tutorial{tutorial_orbbec_astra_openni}
|
@prev_tutorial{tutorial_orbbec_uvc}
|
||||||
@next_tutorial{tutorial_wayland_ubuntu}
|
@next_tutorial{tutorial_wayland_ubuntu}
|
||||||
|
|
||||||
![hardwares](images/realsense.jpg)
|
![hardwares](images/realsense.jpg)
|
||||||
|
@ -4,7 +4,7 @@ Using Orbbec Astra 3D cameras {#tutorial_orbbec_astra_openni}
|
|||||||
@tableofcontents
|
@tableofcontents
|
||||||
|
|
||||||
@prev_tutorial{tutorial_kinect_openni}
|
@prev_tutorial{tutorial_kinect_openni}
|
||||||
@next_tutorial{tutorial_intelperc}
|
@next_tutorial{tutorial_orbbec_uvc}
|
||||||
|
|
||||||
|
|
||||||
### Introduction
|
### Introduction
|
||||||
|
126
doc/tutorials/app/orbbec_uvc.markdown
Normal file
126
doc/tutorials/app/orbbec_uvc.markdown
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
Using Orbbec 3D cameras (UVC) {#tutorial_orbbec_uvc}
|
||||||
|
====================================================
|
||||||
|
|
||||||
|
@tableofcontents
|
||||||
|
|
||||||
|
@prev_tutorial{tutorial_orbbec_astra_openni}
|
||||||
|
@next_tutorial{tutorial_intelperc}
|
||||||
|
|
||||||
|
| | |
|
||||||
|
| -: | :- |
|
||||||
|
| Original author | Jinyue Chen |
|
||||||
|
| Compatibility | OpenCV >= 4.10 |
|
||||||
|
|
||||||
|
### Introduction
|
||||||
|
This tutorial is devoted to the Orbbec 3D cameras based on UVC protocol. For the use of the older
|
||||||
|
Orbbec 3D cameras which depends on OpenNI, please refer to the
|
||||||
|
[previous tutorial](https://github.com/opencv/opencv/blob/4.x/doc/tutorials/app/orbbec_astra_openni.markdown).
|
||||||
|
|
||||||
|
Unlike working with the OpenNI based Astra 3D cameras which requires OpenCV built with OpenNI2 SDK,
|
||||||
|
Orbbec SDK is not required to be installed for accessing Orbbec UVC 3D cameras via OpenCV. By using
|
||||||
|
`cv::VideoCapture` class, users get the stream data from 3D cameras, similar to working with USB
|
||||||
|
cameras. The calibration and alignment of the depth map and color image are done internally.
|
||||||
|
|
||||||
|
### Instructions
|
||||||
|
|
||||||
|
In order to use the 3D cameras with OpenCV. You can refer to [Get Started](https://opencv.org/get-started/)
|
||||||
|
to install OpenCV.
|
||||||
|
|
||||||
|
Note since 4.11 on, Mac OS users need to compile OpenCV from source with flag
|
||||||
|
`-DOBSENSOR_USE_ORBBEC_SDK=ON` in order to use the cameras:
|
||||||
|
```bash
|
||||||
|
cmake -DOBSENSOR_USE_ORBBEC_SDK=ON ..
|
||||||
|
make
|
||||||
|
sudo make install
|
||||||
|
```
|
||||||
|
|
||||||
|
Code
|
||||||
|
----
|
||||||
|
|
||||||
|
@add_toggle_python
|
||||||
|
This tutorial code's is shown lines below. You can also download it from
|
||||||
|
[here](https://github.com/opencv/opencv/blob/4.x/samples/python/videocapture_obsensor.py)
|
||||||
|
@include samples/python/videocapture_obsensor.py
|
||||||
|
@end_toggle
|
||||||
|
|
||||||
|
@add_toggle_cpp
|
||||||
|
This tutorial code's is shown lines below. You can also download it from
|
||||||
|
[here](https://github.com/opencv/opencv/blob/4.x/samples/cpp/videocapture_obsensor.cpp)
|
||||||
|
@include samples/cpp/videocapture_obsensor.cpp
|
||||||
|
@end_toggle
|
||||||
|
|
||||||
|
### Code Explanation
|
||||||
|
|
||||||
|
#### Python
|
||||||
|
|
||||||
|
- **Open Orbbec Depth Sensor**:
|
||||||
|
Using `cv.VideoCapture(0, cv.CAP_OBSENSOR)` to attempt to open the first Orbbec depth sensor device.
|
||||||
|
If the camera fails to open, the program will exit and display an error message.
|
||||||
|
|
||||||
|
- **Loop to Grab and Process Data**:
|
||||||
|
In an infinite loop, the code continuously grabs data from the camera. The `orbbec_cap.grab()`
|
||||||
|
method attempts to grab a frame.
|
||||||
|
|
||||||
|
- **Process BGR Image**:
|
||||||
|
Using `orbbec_cap.retrieve(None, cv.CAP_OBSENSOR_BGR_IMAGE)` to retrieve the BGR image data.
|
||||||
|
If successfully retrieved, the BGR image is displayed in a window using `cv.imshow("BGR", bgr_image)`.
|
||||||
|
|
||||||
|
- **Process Depth Image**:
|
||||||
|
Using `orbbec_cap.retrieve(None, cv.CAP_OBSENSOR_DEPTH_MAP)` to retrieve the depth image data.
|
||||||
|
If successfully retrieved, the depth image is first normalized to a range of 0 to 255, then a
|
||||||
|
false color image is applied, and the result is displayed in a window using `cv.imshow("DEPTH", color_depth_map)`.
|
||||||
|
|
||||||
|
- **Keyboard Interrupt**:
|
||||||
|
Using `cv.pollKey()` to detect keyboard events. If a key is pressed, the loop breaks and
|
||||||
|
the program ends.
|
||||||
|
|
||||||
|
- **Release Resources**:
|
||||||
|
After exiting the loop, the camera resources are released using `orbbec_cap.release()`.
|
||||||
|
|
||||||
|
#### C++
|
||||||
|
|
||||||
|
- **Open Orbbec Depth Sensor**:
|
||||||
|
Using `VideoCapture obsensorCapture(0, CAP_OBSENSOR)` to attempt to open the first Orbbec depth
|
||||||
|
sensor device. If the camera fails to open, an error message is displayed, and the program exits.
|
||||||
|
|
||||||
|
- **Retrieve Camera Intrinsic Parameters**:
|
||||||
|
Using `obsensorCapture.get()` to retrieve the intrinsic parameters of the camera, including focal
|
||||||
|
lengths (`fx`, `fy`) and principal points (`cx`, `cy`).
|
||||||
|
|
||||||
|
- **Loop to Grab and Process Data**:
|
||||||
|
In an infinite loop, the code continuously grabs data from the camera. The `obsensorCapture.grab()`
|
||||||
|
method attempts to grab a frame.
|
||||||
|
|
||||||
|
- **Process BGR Image**:
|
||||||
|
Using `obsensorCapture.retrieve(image, CAP_OBSENSOR_BGR_IMAGE)` to retrieve the BGR image data.
|
||||||
|
If successfully retrieved, the BGR image is displayed in a window using `imshow("BGR", image)`.
|
||||||
|
|
||||||
|
- **Process Depth Image**:
|
||||||
|
Using `obsensorCapture.retrieve(depthMap, CAP_OBSENSOR_DEPTH_MAP)` to retrieve the depth image data.
|
||||||
|
If successfully retrieved, the depth image is normalized and a false color image is applied, then
|
||||||
|
the result is displayed in a window using `imshow("DEPTH", adjDepthMap)`. The retrieved depth
|
||||||
|
values are in millimeters and are truncated to a range between 300 and 5000 (millimeter).
|
||||||
|
This fixed range can be interpreted as a truncation based on the depth camera's depth range,
|
||||||
|
removing invalid pixels on the depth map.
|
||||||
|
|
||||||
|
- **Overlay Depth Map on BGR Image**:
|
||||||
|
Convert the depth map to an 8-bit image, resize it to match the BGR image size, and overlay it
|
||||||
|
on the BGR image with a specified transparency (`alpha`). The overlaid image is displayed in
|
||||||
|
a window using `imshow("DepthToColor", image)`.
|
||||||
|
|
||||||
|
- **Keyboard Interrupt**:
|
||||||
|
Using `pollKey()` to detect keyboard events. If a key is pressed, the loop breaks and the program ends.
|
||||||
|
|
||||||
|
- **Release Resources**:
|
||||||
|
After exiting the loop, the camera resources are released.
|
||||||
|
### Results
|
||||||
|
#### Python
|
||||||
|
|
||||||
|
![BGR And DEPTH frame](images/orbbec_uvc_python.jpg)
|
||||||
|
|
||||||
|
#### C++
|
||||||
|
|
||||||
|
![BGR And DEPTH And DepthToColor frame](images/orbbec_uvc_cpp.jpg)
|
||||||
|
|
||||||
|
### Note
|
||||||
|
Mac users need sudo privileges to execute the code.
|
@ -7,5 +7,6 @@ Application utils (highgui, imgcodecs, videoio modules) {#tutorial_table_of_cont
|
|||||||
- @subpage tutorial_video_write
|
- @subpage tutorial_video_write
|
||||||
- @subpage tutorial_kinect_openni
|
- @subpage tutorial_kinect_openni
|
||||||
- @subpage tutorial_orbbec_astra_openni
|
- @subpage tutorial_orbbec_astra_openni
|
||||||
|
- @subpage tutorial_orbbec_uvc
|
||||||
- @subpage tutorial_intelperc
|
- @subpage tutorial_intelperc
|
||||||
- @subpage tutorial_wayland_ubuntu
|
- @subpage tutorial_wayland_ubuntu
|
||||||
|
Loading…
Reference in New Issue
Block a user