Merge pull request #25907 from Andy-SAKN:Andy-SAKN-add_orbbec

Add tutorial on using Orbbec 3D cameras (UVC) #25907

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Andy-SAKN 2024-07-18 22:27:00 +08:00 committed by GitHub
parent 1125755345
commit 325da2b2f4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 129 additions and 2 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

View File

@ -3,7 +3,7 @@ Using Creative Senz3D and other Intel RealSense SDK compatible depth sensors {#t
@tableofcontents @tableofcontents
@prev_tutorial{tutorial_orbbec_astra_openni} @prev_tutorial{tutorial_orbbec_uvc}
@next_tutorial{tutorial_wayland_ubuntu} @next_tutorial{tutorial_wayland_ubuntu}
![hardwares](images/realsense.jpg) ![hardwares](images/realsense.jpg)

View File

@ -4,7 +4,7 @@ Using Orbbec Astra 3D cameras {#tutorial_orbbec_astra_openni}
@tableofcontents @tableofcontents
@prev_tutorial{tutorial_kinect_openni} @prev_tutorial{tutorial_kinect_openni}
@next_tutorial{tutorial_intelperc} @next_tutorial{tutorial_orbbec_uvc}
### Introduction ### Introduction

View File

@ -0,0 +1,126 @@
Using Orbbec 3D cameras (UVC) {#tutorial_orbbec_uvc}
====================================================
@tableofcontents
@prev_tutorial{tutorial_orbbec_astra_openni}
@next_tutorial{tutorial_intelperc}
| | |
| -: | :- |
| Original author | Jinyue Chen |
| Compatibility | OpenCV >= 4.10 |
### Introduction
This tutorial is devoted to the Orbbec 3D cameras based on UVC protocol. For the use of the older
Orbbec 3D cameras which depends on OpenNI, please refer to the
[previous tutorial](https://github.com/opencv/opencv/blob/4.x/doc/tutorials/app/orbbec_astra_openni.markdown).
Unlike working with the OpenNI based Astra 3D cameras which requires OpenCV built with OpenNI2 SDK,
Orbbec SDK is not required to be installed for accessing Orbbec UVC 3D cameras via OpenCV. By using
`cv::VideoCapture` class, users get the stream data from 3D cameras, similar to working with USB
cameras. The calibration and alignment of the depth map and color image are done internally.
### Instructions
In order to use the 3D cameras with OpenCV. You can refer to [Get Started](https://opencv.org/get-started/)
to install OpenCV.
Note since 4.11 on, Mac OS users need to compile OpenCV from source with flag
`-DOBSENSOR_USE_ORBBEC_SDK=ON` in order to use the cameras:
```bash
cmake -DOBSENSOR_USE_ORBBEC_SDK=ON ..
make
sudo make install
```
Code
----
@add_toggle_python
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/blob/4.x/samples/python/videocapture_obsensor.py)
@include samples/python/videocapture_obsensor.py
@end_toggle
@add_toggle_cpp
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/blob/4.x/samples/cpp/videocapture_obsensor.cpp)
@include samples/cpp/videocapture_obsensor.cpp
@end_toggle
### Code Explanation
#### Python
- **Open Orbbec Depth Sensor**:
Using `cv.VideoCapture(0, cv.CAP_OBSENSOR)` to attempt to open the first Orbbec depth sensor device.
If the camera fails to open, the program will exit and display an error message.
- **Loop to Grab and Process Data**:
In an infinite loop, the code continuously grabs data from the camera. The `orbbec_cap.grab()`
method attempts to grab a frame.
- **Process BGR Image**:
Using `orbbec_cap.retrieve(None, cv.CAP_OBSENSOR_BGR_IMAGE)` to retrieve the BGR image data.
If successfully retrieved, the BGR image is displayed in a window using `cv.imshow("BGR", bgr_image)`.
- **Process Depth Image**:
Using `orbbec_cap.retrieve(None, cv.CAP_OBSENSOR_DEPTH_MAP)` to retrieve the depth image data.
If successfully retrieved, the depth image is first normalized to a range of 0 to 255, then a
false color image is applied, and the result is displayed in a window using `cv.imshow("DEPTH", color_depth_map)`.
- **Keyboard Interrupt**:
Using `cv.pollKey()` to detect keyboard events. If a key is pressed, the loop breaks and
the program ends.
- **Release Resources**:
After exiting the loop, the camera resources are released using `orbbec_cap.release()`.
#### C++
- **Open Orbbec Depth Sensor**:
Using `VideoCapture obsensorCapture(0, CAP_OBSENSOR)` to attempt to open the first Orbbec depth
sensor device. If the camera fails to open, an error message is displayed, and the program exits.
- **Retrieve Camera Intrinsic Parameters**:
Using `obsensorCapture.get()` to retrieve the intrinsic parameters of the camera, including focal
lengths (`fx`, `fy`) and principal points (`cx`, `cy`).
- **Loop to Grab and Process Data**:
In an infinite loop, the code continuously grabs data from the camera. The `obsensorCapture.grab()`
method attempts to grab a frame.
- **Process BGR Image**:
Using `obsensorCapture.retrieve(image, CAP_OBSENSOR_BGR_IMAGE)` to retrieve the BGR image data.
If successfully retrieved, the BGR image is displayed in a window using `imshow("BGR", image)`.
- **Process Depth Image**:
Using `obsensorCapture.retrieve(depthMap, CAP_OBSENSOR_DEPTH_MAP)` to retrieve the depth image data.
If successfully retrieved, the depth image is normalized and a false color image is applied, then
the result is displayed in a window using `imshow("DEPTH", adjDepthMap)`. The retrieved depth
values are in millimeters and are truncated to a range between 300 and 5000 (millimeter).
This fixed range can be interpreted as a truncation based on the depth camera's depth range,
removing invalid pixels on the depth map.
- **Overlay Depth Map on BGR Image**:
Convert the depth map to an 8-bit image, resize it to match the BGR image size, and overlay it
on the BGR image with a specified transparency (`alpha`). The overlaid image is displayed in
a window using `imshow("DepthToColor", image)`.
- **Keyboard Interrupt**:
Using `pollKey()` to detect keyboard events. If a key is pressed, the loop breaks and the program ends.
- **Release Resources**:
After exiting the loop, the camera resources are released.
### Results
#### Python
![BGR And DEPTH frame](images/orbbec_uvc_python.jpg)
#### C++
![BGR And DEPTH And DepthToColor frame](images/orbbec_uvc_cpp.jpg)
### Note
Mac users need sudo privileges to execute the code.

View File

@ -7,5 +7,6 @@ Application utils (highgui, imgcodecs, videoio modules) {#tutorial_table_of_cont
- @subpage tutorial_video_write - @subpage tutorial_video_write
- @subpage tutorial_kinect_openni - @subpage tutorial_kinect_openni
- @subpage tutorial_orbbec_astra_openni - @subpage tutorial_orbbec_astra_openni
- @subpage tutorial_orbbec_uvc
- @subpage tutorial_intelperc - @subpage tutorial_intelperc
- @subpage tutorial_wayland_ubuntu - @subpage tutorial_wayland_ubuntu