mirror of
https://github.com/opencv/opencv.git
synced 2024-11-27 20:50:25 +08:00
Merge pull request #23018 from stopmosk:move-aruco-tutorial
Move Aruco tutorials and samples to main repo #23018 merge with https://github.com/opencv/opencv_contrib/pull/3401 merge with https://github.com/opencv/opencv_extra/pull/1143 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake --------- Co-authored-by: AleksandrPanov <alexander.panov@xperience.ai> Co-authored-by: Alexander Smorkalov <alexander.smorkalov@xperience.ai>
This commit is contained in:
Sergei Shutov
GitHub
parent
faba2d8148
commit
6ca45a9d33
@ -0,0 +1,201 @@
|
||||
Detection of ArUco boards {#tutorial_aruco_board_detection}
|
||||
=========================
|
||||
|
||||
@prev_tutorial{tutorial_aruco_detection}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original authors | Sergio Garrido, Alexander Panov |
|
||||
| Compatibility | OpenCV >= 4.7.0 |
|
||||
|
||||
An ArUco board is a set of markers that acts like a single marker in the sense that it provides a
|
||||
single pose for the camera.
|
||||
|
||||
The most popular board is the one with all the markers in the same plane, since it can be easily printed:
|
||||
|
||||
|
||||
|
||||
However, boards are not limited to this arrangement and can represent any 2d or 3d layout.
|
||||
|
||||
The difference between a board and a set of independent markers is that the relative position between
|
||||
the markers in the board is known a priori. This allows that the corners of all the markers can be used for
|
||||
estimating the pose of the camera respect to the whole board.
|
||||
|
||||
When you use a set of independent markers, you can estimate the pose for each marker individually,
|
||||
since you dont know the relative position of the markers in the environment.
|
||||
|
||||
The main benefits of using boards are:
|
||||
|
||||
- The pose estimation is much more versatile. Only some markers are necessary to perform pose estimation.
|
||||
Thus, the pose can be calculated even in the presence of occlusions or partial views.
|
||||
- The obtained pose is usually more accurate since a higher amount of point correspondences (marker
|
||||
corners) are employed.
|
||||
|
||||
Board Detection
|
||||
---------------
|
||||
|
||||
A board detection is similar to the standard marker detection. The only difference is in the pose estimation step.
|
||||
In fact, to use marker boards, a standard marker detection should be done before estimating the board pose.
|
||||
|
||||
To perform pose estimation for boards, you should use `solvePnP()` function, as shown below
|
||||
in the `samples/cpp/tutorial_code/objectDetection/detect_board.cpp`.
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_board_full_sample
|
||||
|
||||
|
||||
The parameters are:
|
||||
|
||||
- `objPoints`, `imgPoints` object and image points, matched with `cv::aruco::GridBoard::matchImagePoints()`
|
||||
which, in turn, takes as input `markerCorners` and `markerIds` structures of detected markers from
|
||||
`cv::aruco::ArucoDetector::detectMarkers()` function.
|
||||
- `board` the `cv::aruco::Board` object that defines the board layout and its ids
|
||||
- `cameraMatrix` and `distCoeffs`: camera calibration parameters necessary for pose estimation.
|
||||
- `rvec` and `tvec`: estimated pose of the board. If not empty then treated as initial guess.
|
||||
- The function returns the total number of markers employed for estimating the board pose.
|
||||
|
||||
The drawFrameAxes() function can be used to check the obtained pose. For instance:
|
||||
|
||||
|
||||
|
||||
And this is another example with the board partially occluded:
|
||||
|
||||
|
||||
|
||||
As it can be observed, although some markers have not been detected, the board pose can still be
|
||||
estimated from the rest of markers.
|
||||
|
||||
Sample video:
|
||||
|
||||
@youtube{Q1HlJEjW_j0}
|
||||
|
||||
A full working example is included in the `detect_board.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples now take input via command line via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
-w=5 -h=7 -l=100 -s=10
|
||||
-v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_board_detection/gboriginal.jpg
|
||||
-c=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml
|
||||
-cd=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml
|
||||
@endcode
|
||||
Parameters for `detect_board.cpp`:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_board_keys
|
||||
|
||||
Grid Board
|
||||
----------
|
||||
|
||||
Creating the `cv::aruco::Board` object requires specifying the corner positions for each marker in the environment.
|
||||
However, in many cases, the board will be just a set of markers in the same plane and in a grid layout,
|
||||
so it can be easily printed and used.
|
||||
|
||||
Fortunately, the aruco module provides the basic functionality to create and print these types of markers
|
||||
easily.
|
||||
|
||||
The `cv::aruco::GridBoard` class is a specialized class that inherits from the `cv::aruco::Board`
|
||||
class and which represents a Board with all the markers in the same plane and in a grid layout,
|
||||
as in the following image:
|
||||
|
||||
|
||||
|
||||
Concretely, the coordinate system in a grid board is positioned in the board plane, centered in the bottom left
|
||||
corner of the board and with the Z pointing out, like in the following image (X:red, Y:green, Z:blue):
|
||||
|
||||
|
||||
|
||||
A `cv::aruco::GridBoard` object can be defined using the following parameters:
|
||||
|
||||
- Number of markers in the X direction.
|
||||
- Number of markers in the Y direction.
|
||||
- Length of the marker side.
|
||||
- Length of the marker separation.
|
||||
- The dictionary of the markers.
|
||||
- Ids of all the markers (X*Y markers).
|
||||
|
||||
This object can be easily created from these parameters using the `cv::aruco::GridBoard` constructor:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_create_board
|
||||
|
||||
- The first and second parameters are the number of markers in the X and Y direction respectively.
|
||||
- The third and fourth parameters are the marker length and the marker separation respectively.
|
||||
They can be provided in any unit, having in mind that the estimated pose for this board will be
|
||||
measured in the same units (in general, meters are used).
|
||||
- Finally, the dictionary of the markers is provided.
|
||||
|
||||
So, this board will be composed by 5x7=35 markers. The ids of each of the markers are assigned, by default,
|
||||
in ascending order starting on 0, so they will be 0, 1, 2, ..., 34.
|
||||
|
||||
After creating a grid board, we probably want to print it and use it.
|
||||
There are two ways to do this:
|
||||
1. By using the script `doc/patter_tools/gen_pattern.py `, see @subpage tutorial_camera_calibration_pattern.
|
||||
2. By using the function `cv::aruco::GridBoard::generateImage()`.
|
||||
|
||||
The function `cv::aruco::GridBoard::generateImage()` is provided in cv::aruco::GridBoard class and
|
||||
can be called by using the following code:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/create_board.cpp aruco_generate_board_image
|
||||
|
||||
- The first parameter is the size of the output image in pixels. In this case 600x500 pixels. If this is not proportional
|
||||
to the board dimensions, it will be centered on the image.
|
||||
- `boardImage`: the output image with the board.
|
||||
- The third parameter is the (optional) margin in pixels, so none of the markers are touching the image border.
|
||||
In this case the margin is 10.
|
||||
- Finally, the size of the marker border, similarly to `generateImageMarker()` function. The default value is 1.
|
||||
|
||||
A full working example of board creation is included in the `samples/cpp/tutorial_code/objectDetection/create_board.cpp`
|
||||
|
||||
The output image will be something like this:
|
||||
|
||||
|
||||
|
||||
The samples now take input via commandline via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
"_output_path_/aboard.png" -w=5 -h=7 -l=100 -s=10 -d=10
|
||||
@endcode
|
||||
|
||||
Refine marker detection
|
||||
-----------------------
|
||||
|
||||
ArUco boards can also be used to improve the detection of markers. If we have detected a subset of the markers
|
||||
that belongs to the board, we can use these markers and the board layout information to try to find the
|
||||
markers that have not been previously detected.
|
||||
|
||||
This can be done using the `cv::aruco::refineDetectedMarkers()` function, which should be called
|
||||
after calling `cv::aruco::ArucoDetector::detectMarkers()`.
|
||||
|
||||
The main parameters of this function are the original image where markers were detected, the board object,
|
||||
the detected marker corners, the detected marker ids and the rejected marker corners.
|
||||
|
||||
The rejected corners can be obtained from the `cv::aruco::ArucoDetector::detectMarkers()` function and
|
||||
are also known as marker candidates. This candidates are square shapes that have been found in the
|
||||
original image but have failed to pass the identification step (i.e. their inner codification presents
|
||||
too many errors) and thus they have not been recognized as markers.
|
||||
|
||||
However, these candidates are sometimes actual markers that have not been correctly identified due to high
|
||||
noise in the image, very low resolution or other related problems that affect to the binary code extraction.
|
||||
The `cv::aruco::ArucoDetector::refineDetectedMarkers()` function finds correspondences between these
|
||||
candidates and the missing markers of the board. This search is based on two parameters:
|
||||
|
||||
- Distance between the candidate and the projection of the missing marker. To obtain these projections,
|
||||
it is necessary to have detected at least one marker of the board. The projections are obtained using the
|
||||
camera parameters (camera matrix and distortion coefficients) if they are provided. If not, the projections
|
||||
are obtained from local homography and only planar board are allowed (i.e. the Z coordinate of all the
|
||||
marker corners should be the same). The `minRepDistance` parameter in `refineDetectedMarkers()`
|
||||
determines the minimum euclidean distance between the candidate corners and the projected marker corners
|
||||
(default value 10).
|
||||
|
||||
- Binary codification. If a candidate surpasses the minimum distance condition, its internal bits
|
||||
are analyzed again to determine if it is actually the projected marker or not. However, in this case,
|
||||
the condition is not so strong and the number of allowed erroneous bits can be higher. This is indicated
|
||||
in the `errorCorrectionRate` parameter (default value 3.0). If a negative value is provided, the
|
||||
internal bits are not analyzed at all and only the corner distances are evaluated.
|
||||
|
||||
This is an example of using the `cv::aruco::ArucoDetector::refineDetectedMarkers()` function:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_and_refine
|
||||
|
||||
It must also be noted that, in some cases, if the number of detected markers in the first place is
|
||||
too low (for instance only 1 or 2 markers), the projections of the missing markers can be of bad
|
||||
quality, producing erroneous correspondences.
|
||||
|
||||
See module samples for a more detailed implementation.
|
||||
BIN
doc/tutorials/objdetect/aruco_board_detection/images/board.png
Normal file
BIN
doc/tutorials/objdetect/aruco_board_detection/images/board.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
BIN
doc/tutorials/objdetect/aruco_board_detection/images/gbaxis.jpg
Normal file
BIN
doc/tutorials/objdetect/aruco_board_detection/images/gbaxis.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 61 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 70 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 59 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 61 KiB |
|
Before Width: | Height: | Size: 75 KiB After Width: | Height: | Size: 75 KiB |
702
doc/tutorials/objdetect/aruco_detection/aruco_detection.markdown
Normal file
702
doc/tutorials/objdetect/aruco_detection/aruco_detection.markdown
Normal file
@ -0,0 +1,702 @@
|
||||
Detection of ArUco Markers {#tutorial_aruco_detection}
|
||||
==========================
|
||||
|
||||
@next_tutorial{tutorial_aruco_board_detection}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original authors | Sergio Garrido, Alexander Panov |
|
||||
| Compatibility | OpenCV >= 4.7.0 |
|
||||
|
||||
Pose estimation is of great importance in many computer vision applications: robot navigation,
|
||||
augmented reality, and many more. This process is based on finding correspondences between points in
|
||||
the real environment and their 2d image projection. This is usually a difficult step, and thus it is
|
||||
common to use synthetic or fiducial markers to make it easier.
|
||||
|
||||
One of the most popular approaches is the use of binary square fiducial markers. The main benefit
|
||||
of these markers is that a single marker provides enough correspondences (its four corners)
|
||||
to obtain the camera pose. Also, the inner binary codification makes them specially robust, allowing
|
||||
the possibility of applying error detection and correction techniques.
|
||||
|
||||
The aruco module is based on the [ArUco library](http://www.uco.es/investiga/grupos/ava/node/26),
|
||||
a popular library for detection of square fiducial markers developed by Rafael Muñoz and Sergio Garrido @cite Aruco2014.
|
||||
|
||||
The aruco functionalities are included in:
|
||||
@code{.cpp}
|
||||
#include <opencv2/objdetect/aruco_detector.hpp>
|
||||
@endcode
|
||||
|
||||
|
||||
Markers and Dictionaries
|
||||
------------------------
|
||||
|
||||
An ArUco marker is a synthetic square marker composed by a wide black border and an inner binary
|
||||
matrix which determines its identifier (id). The black border facilitates its fast detection in the
|
||||
image and the binary codification allows its identification and the application of error detection
|
||||
and correction techniques. The marker size determines the size of the internal matrix. For instance
|
||||
a marker size of 4x4 is composed by 16 bits.
|
||||
|
||||
Some examples of ArUco markers:
|
||||
|
||||
|
||||
|
||||
It must be noted that a marker can be found rotated in the environment, however, the detection
|
||||
process needs to be able to determine its original rotation, so that each corner is identified
|
||||
unequivocally. This is also done based on the binary codification.
|
||||
|
||||
A dictionary of markers is the set of markers that are considered in a specific application. It is
|
||||
simply the list of binary codifications of each of its markers.
|
||||
|
||||
The main properties of a dictionary are the dictionary size and the marker size.
|
||||
|
||||
- The dictionary size is the number of markers that compose the dictionary.
|
||||
- The marker size is the size of those markers (the number of bits/modules).
|
||||
|
||||
The aruco module includes some predefined dictionaries covering a range of different dictionary
|
||||
sizes and marker sizes.
|
||||
|
||||
One may think that the marker id is the number obtained from converting the binary codification to
|
||||
a decimal base number. However, this is not possible since for high marker sizes the number of bits
|
||||
is too high and managing such huge numbers is not practical. Instead, a marker id is simply
|
||||
the marker index within the dictionary it belongs to. For instance, the first 5 markers in a
|
||||
dictionary have the ids: 0, 1, 2, 3 and 4.
|
||||
|
||||
More information about dictionaries is provided in the "Selecting a dictionary" section.
|
||||
|
||||
|
||||
Marker Creation
|
||||
---------------
|
||||
|
||||
Before their detection, markers need to be printed in order to be placed in the environment.
|
||||
Marker images can be generated using the `generateImageMarker()` function.
|
||||
|
||||
For example, lets analyze the following call:
|
||||
|
||||
@code{.cpp}
|
||||
cv::Mat markerImage;
|
||||
cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
|
||||
cv::aruco::generateImageMarker(dictionary, 23, 200, markerImage, 1);
|
||||
cv::imwrite("marker23.png", markerImage);
|
||||
@endcode
|
||||
|
||||
First, the `cv::aruco::Dictionary` object is created by choosing one of the predefined dictionaries in the aruco module.
|
||||
Concretely, this dictionary is composed of 250 markers and a marker size of 6x6 bits (`cv::aruco::DICT_6X6_250`).
|
||||
|
||||
The parameters of `cv::aruco::generateImageMarker()` are:
|
||||
|
||||
- The first parameter is the `cv::aruco::Dictionary` object previously created.
|
||||
- The second parameter is the marker id, in this case the marker 23 of the dictionary `cv::aruco::DICT_6X6_250`.
|
||||
Note that each dictionary is composed of a different number of markers. In this case, the valid ids
|
||||
go from 0 to 249. Any specific id out of the valid range will produce an exception.
|
||||
- The third parameter, 200, is the size of the output marker image. In this case, the output image
|
||||
will have a size of 200x200 pixels. Note that this parameter should be large enough to store the
|
||||
number of bits for the specific dictionary. So, for instance, you cannot generate an image of
|
||||
5x5 pixels for a marker size of 6x6 bits (and that is without considering the marker border).
|
||||
Furthermore, to avoid deformations, this parameter should be proportional to the number of bits +
|
||||
border size, or at least much higher than the marker size (like 200 in the example), so that
|
||||
deformations are insignificant.
|
||||
- The fourth parameter is the output image.
|
||||
- Finally, the last parameter is an optional parameter to specify the width of the marker black
|
||||
border. The size is specified proportional to the number of bits. For instance a value of 2 means
|
||||
that the border will have a width equivalent to the size of two internal bits. The default value
|
||||
is 1.
|
||||
|
||||
The generated image is:
|
||||
|
||||
|
||||
|
||||
A full working example is included in the `create_marker.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples now take input from the command line using cv::CommandLineParser. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
"marker23.png" -d=10 -id=23
|
||||
@endcode
|
||||
Parameters for `create_marker.cpp`:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/create_marker.cpp aruco_create_markers_keys
|
||||
|
||||
Marker Detection
|
||||
----------------
|
||||
|
||||
Given an image containing ArUco markers, the detection process has to return a list of
|
||||
detected markers. Each detected marker includes:
|
||||
|
||||
- The position of its four corners in the image (in their original order).
|
||||
- The id of the marker.
|
||||
|
||||
The marker detection process is comprised of two main steps:
|
||||
|
||||
1. Detection of marker candidates. In this step the image is analyzed in order to find square shapes
|
||||
that are candidates to be markers. It begins with an adaptive thresholding to segment the markers,
|
||||
then contours are extracted from the thresholded image and those that are not convex or do not
|
||||
approximate to a square shape are discarded. Some extra filtering is also applied (removing contours
|
||||
that are too small or too big, removing contours too close to each other, etc).
|
||||
|
||||
2. After the candidate detection, it is necessary to determine if they are actually markers by
|
||||
analyzing their inner codification. This step starts by extracting the marker bits of each marker.
|
||||
To do so, a perspective transformation is first applied to obtain the marker in its canonical form.
|
||||
Then, the canonical image is thresholded using Otsu to separate white and black bits. The image
|
||||
is divided into different cells according to the marker size and the border size. Then the number
|
||||
of black or white pixels in each cell is counted to determine if it is a white or a black bit.
|
||||
Finally, the bits are analyzed to determine if the marker belongs to the specific dictionary.
|
||||
Error correction techniques are employed when necessary.
|
||||
|
||||
|
||||
Consider the following image:
|
||||
|
||||
|
||||
|
||||
And a printout of this image in a photo:
|
||||
|
||||
|
||||
|
||||
These are the detected markers (in green). Note that some markers are rotated. The small red square
|
||||
indicates the marker’s top left corner:
|
||||
|
||||
|
||||
|
||||
And these are the marker candidates that have been rejected during the identification step (in pink):
|
||||
|
||||
|
||||
|
||||
In the aruco module, the detection is performed in the `cv::aruco::ArucoDetector::detectMarkers()`
|
||||
function. This function is the most important in the module, since all the rest of the functionality
|
||||
is based on the detected markers returned by `cv::aruco::ArucoDetector::detectMarkers()`.
|
||||
|
||||
An example of marker detection:
|
||||
|
||||
@code{.cpp}
|
||||
cv::Mat inputImage;
|
||||
// ... read inputImage ...
|
||||
std::vector<int> markerIds;
|
||||
std::vector<std::vector<cv::Point2f>> markerCorners, rejectedCandidates;
|
||||
cv::aruco::DetectorParameters detectorParams = cv::aruco::DetectorParameters();
|
||||
cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
|
||||
cv::aruco::ArucoDetector detector(dictionary, detectorParams);
|
||||
detector.detectMarkers(inputImage, markerCorners, markerIds, rejectedCandidates);
|
||||
@endcode
|
||||
|
||||
When you create an `cv::aruco::ArucoDetector` object, you need to pass the following parameters to the constructor:
|
||||
|
||||
- A dictionary object, in this case one of the predefined dictionaries (`cv::aruco::DICT_6X6_250`).
|
||||
- Object of type `cv::aruco::DetectorParameters`. This object includes all parameters that can be customized during the detection process.
|
||||
These parameters will be explained in the next section.
|
||||
|
||||
The parameters of `cv::aruco::ArucoDetector::detectMarkers()` are:
|
||||
|
||||
- The first parameter is the image containing the markers to be detected.
|
||||
- The detected markers are stored in the `markerCorners` and `markerIds` structures:
|
||||
- `markerCorners` is the list of corners of the detected markers. For each marker, its four
|
||||
corners are returned in their original order (which is clockwise starting with top left).
|
||||
So, the first corner is the top left corner, followed by the top right, bottom right and bottom left.
|
||||
- `markerIds` is the list of ids of each of the detected markers in `markerCorners`.
|
||||
Note that the returned `markerCorners` and `markerIds` vectors have the same size.
|
||||
- The final optional parameter, `rejectedCandidates`, is a returned list of marker candidates, i.e.
|
||||
shapes that were found and considered but did not contain a valid marker. Each candidate is also
|
||||
defined by its four corners, and its format is the same as the `markerCorners` parameter. This
|
||||
parameter can be omitted and is only useful for debugging purposes and for ‘refind’ strategies
|
||||
(see `cv::aruco::ArucoDetector::refineDetectedMarkers()`).
|
||||
|
||||
|
||||
The next thing you probably want to do after `cv::aruco::ArucoDetector::detectMarkers()` is check that
|
||||
your markers have been correctly detected. Fortunately, the aruco module provides a function to draw
|
||||
the detected markers in the input image, this function is `drawDetectedMarkers()`. For example:
|
||||
|
||||
@code{.cpp}
|
||||
cv::Mat outputImage = inputImage.clone();
|
||||
cv::aruco::drawDetectedMarkers(outputImage, markerCorners, markerIds);
|
||||
@endcode
|
||||
|
||||
- `outputImage ` is the input/output image where the markers will be drawn (it will normally be
|
||||
the same as the image where the markers were detected).
|
||||
- `markerCorners` and `markerIds` are the structures of the detected markers returned by the
|
||||
`cv::aruco::ArucoDetector::detectMarkers()` function.
|
||||
|
||||
|
||||
|
||||
Note that this function is only provided for visualization and its use can be omitted.
|
||||
|
||||
With these two functions we can create a basic marker detection loop to detect markers from our
|
||||
camera:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers
|
||||
|
||||
Note that some of the optional parameters have been omitted, like the detection parameter object and the
|
||||
output vector of rejected candidates.
|
||||
|
||||
A full working example is included in the `detect_markers.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples now take input from the command line using cv::CommandLineParser. For this file
|
||||
the example parameters will look like:
|
||||
@code{.cpp}
|
||||
-v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg -d=10
|
||||
@endcode
|
||||
Parameters for `detect_markers.cpp`:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers_keys
|
||||
|
||||
|
||||
Pose Estimation
|
||||
---------------
|
||||
|
||||
The next thing you'll probably want to do after detecting the markers is to use them to get the camera pose.
|
||||
|
||||
To perform camera pose estimation, you need to know your camera's calibration parameters. These are
|
||||
the camera matrix and distortion coefficients. If you do not know how to calibrate your camera,
|
||||
you can take a look at the `calibrateCamera()` function and the Calibration tutorial of OpenCV.
|
||||
You can also calibrate your camera using the aruco module as explained in the **Calibration with ArUco and ChArUco**
|
||||
tutorial. Note that this only needs to be done once unless the camera optics are modified
|
||||
(for instance changing its focus).
|
||||
|
||||
As a result of the calibration, you get a camera matrix: a matrix of 3x3 elements with the
|
||||
focal distances and the camera center coordinates (a.k.a intrinsic parameters), and the distortion
|
||||
coefficients: a vector of 5 or more elements that models the distortion produced by your camera.
|
||||
|
||||
When you estimate the pose with ArUco markers, you can estimate the pose of each marker individually.
|
||||
If you want to estimate one pose from a set of markers, use ArUco Boards (see the **Detection of ArUco
|
||||
Boards** tutorial). Using ArUco boards instead of single markers allows some markers to be occluded.
|
||||
|
||||
The camera pose relative to the marker is a 3d transformation from the marker coordinate system to the
|
||||
camera coordinate system. It is specified by rotation and translation vectors. OpenCV provides
|
||||
`cv::solvePnP()` function to do that.
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation1
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation2
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation3
|
||||
|
||||
- The `corners` parameter is the vector of marker corners returned by the `cv::aruco::ArucoDetector::detectMarkers()` function.
|
||||
- The second parameter is the size of the marker side in meters or in any other unit. Note that the
|
||||
translation vectors of the estimated poses will be in the same units.
|
||||
- `camMatrix` and `distCoeffs` are the camera calibration parameters that were created during
|
||||
the camera calibration process.
|
||||
- The output parameters `rvecs` and `tvecs` are the rotation and translation vectors respectively,
|
||||
for each of the detected markers in `corners`.
|
||||
|
||||
The marker coordinate system that is assumed by this function is placed in the center (by default) or
|
||||
in the top left corner of the marker with the Z axis pointing out, as in the following image.
|
||||
Axis-color correspondences are X: red, Y: green, Z: blue. Note the axis directions of the rotated
|
||||
markers in this image.
|
||||
|
||||
|
||||
|
||||
OpenCV provides a function to draw the axis as in the image above, so pose estimation can be
|
||||
checked:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_draw_pose_estimation
|
||||
|
||||
- `imageCopy` is the input/output image where the detected markers will be shown.
|
||||
- `camMatrix` and `distCoeffs` are the camera calibration parameters.
|
||||
- `rvecs[i]` and `tvecs[i]` are the rotation and translation vectors respectively, for each of the detected markers.
|
||||
- The last parameter is the length of the axis, in the same unit as tvec (usually meters).
|
||||
|
||||
Sample video:
|
||||
|
||||
@youtube{IsXWrcB_Hvs}
|
||||
|
||||
A full working example is included in the `detect_markers.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples now take input from the command line using cv::CommandLineParser. For this file
|
||||
the example parameters will look like:
|
||||
@code{.cpp}
|
||||
-v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg -d=10
|
||||
-c=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml
|
||||
@endcode
|
||||
Parameters for `detect_markers.cpp`:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers_keys
|
||||
|
||||
Selecting a dictionary
|
||||
----------------------
|
||||
|
||||
The aruco module provides the `Dictionary` class to represent a dictionary of markers.
|
||||
|
||||
In addition to the marker size and the number of markers in the dictionary, there is another important
|
||||
parameter of the dictionary - the inter-marker distance. The inter-marker distance is the minimum
|
||||
Hamming distance between dictionary markers that determines the dictionary's ability to detect and
|
||||
correct errors.
|
||||
|
||||
In general, smaller dictionary sizes and larger marker sizes increase the inter-marker distance and
|
||||
vice versa. However, the detection of markers with larger sizes is more difficult due to the higher
|
||||
number of bits that need to be extracted from the image.
|
||||
|
||||
For instance, if you need only 10 markers in your application, it is better to use a dictionary composed
|
||||
only of those 10 markers than using a dictionary composed of 1000 markers. The reason is that
|
||||
the dictionary composed of 10 markers will have a higher inter-marker distance and, thus, it will be
|
||||
more robust to errors.
|
||||
|
||||
As a consequence, the aruco module includes several ways to select your dictionary of markers, so that
|
||||
you can increase your system robustness:
|
||||
|
||||
### Predefined dictionaries
|
||||
|
||||
This is the easiest way to select a dictionary. The aruco module includes a set of predefined
|
||||
dictionaries in a variety of marker sizes and number of markers. For instance:
|
||||
|
||||
@code{.cpp}
|
||||
cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
|
||||
@endcode
|
||||
|
||||
`cv::aruco::DICT_6X6_250` is an example of predefined dictionary of markers with 6x6 bits and a total of 250
|
||||
markers.
|
||||
|
||||
From all the provided dictionaries, it is recommended to choose the smallest one that fits your application.
|
||||
For instance, if you need 200 markers of 6x6 bits, it is better to use `cv::aruco::DICT_6X6_250` than `cv::aruco::DICT_6X6_1000`.
|
||||
The smaller the dictionary, the higher the inter-marker distance.
|
||||
|
||||
The list of available predefined dictionaries can be found in the documentation for the `PredefinedDictionaryType` enum.
|
||||
|
||||
### Automatic dictionary generation
|
||||
|
||||
A dictionary can be generated automatically to adjust the desired number of markers and bits
|
||||
to optimize the inter-marker distance:
|
||||
|
||||
@code{.cpp}
|
||||
cv::aruco::Dictionary dictionary = cv::aruco::extendDictionary(36, 5);
|
||||
@endcode
|
||||
|
||||
This will generate a customized dictionary composed of 36 markers of 5x5 bits. The process can take several
|
||||
seconds, depending on the parameters (it is slower for larger dictionaries and higher numbers of bits).
|
||||
|
||||
Also you could use `aruco_dict_utils.cpp` sample inside the `opencv/samples/cpp`. This sample calculates
|
||||
the minimum Hamming distance for the generated dictionary and also allows you to create markers that are
|
||||
resistant to reflection.
|
||||
|
||||
### Manual dictionary definition
|
||||
|
||||
Finally, the dictionary can be configured manually, so that any encoding can be used. To do that,
|
||||
the `cv::aruco::Dictionary` object parameters need to be assigned manually. It must be noted that,
|
||||
unless you have a special reason to do this manually, it is preferable to use one of the previous alternatives.
|
||||
|
||||
The `cv::aruco::Dictionary` parameters are:
|
||||
|
||||
@code{.cpp}
|
||||
class Dictionary {
|
||||
public:
|
||||
|
||||
cv::Mat bytesList; // marker code information
|
||||
int markerSize; // number of bits per dimension
|
||||
int maxCorrectionBits; // maximum number of bits that can be corrected
|
||||
|
||||
...
|
||||
|
||||
}
|
||||
@endcode
|
||||
|
||||
`bytesList` is the array that contains all the information about the marker codes. `markerSize` is the size
|
||||
of each marker dimension (for instance, 5 for markers with 5x5 bits). Finally, `maxCorrectionBits` is
|
||||
the maximum number of erroneous bits that can be corrected during the marker detection. If this value is too
|
||||
high, it can lead to a high number of false positives.
|
||||
|
||||
Each row in `bytesList` represents one of the dictionary markers. However, the markers are not stored in their
|
||||
binary form, instead they are stored in a special format to simplify their detection.
|
||||
|
||||
Fortunately, a marker can be easily transformed to this form using the static method `Dictionary::getByteListFromBits()`.
|
||||
|
||||
For example:
|
||||
|
||||
@code{.cpp}
|
||||
cv::aruco::Dictionary dictionary;
|
||||
|
||||
// Markers of 6x6 bits
|
||||
dictionary.markerSize = 6;
|
||||
|
||||
// Maximum number of bit corrections
|
||||
dictionary.maxCorrectionBits = 3;
|
||||
|
||||
// Let's create a dictionary of 100 markers
|
||||
for(int i = 0; i < 100; i++)
|
||||
{
|
||||
// Assume generateMarkerBits() generates a new marker in binary format, so that
|
||||
// markerBits is a 6x6 matrix of CV_8UC1 type, only containing 0s and 1s
|
||||
cv::Mat markerBits = generateMarkerBits();
|
||||
cv::Mat markerCompressed = cv::aruco::Dictionary::getByteListFromBits(markerBits);
|
||||
|
||||
// Add the marker as a new row
|
||||
dictionary.bytesList.push_back(markerCompressed);
|
||||
}
|
||||
@endcode
|
||||
|
||||
Detector Parameters
|
||||
-------------------
|
||||
|
||||
One of the parameters of `cv::aruco::ArucoDetector` is a `cv::aruco::DetectorParameters` object. This object
|
||||
includes all the options that can be customized during the marker detection process.
|
||||
|
||||
This section describes each detector parameter. The parameters can be classified depending on
|
||||
the process in which they’re involved:
|
||||
|
||||
### Thresholding
|
||||
|
||||
One of the first steps in the marker detection process is adaptive thresholding of the input image.
|
||||
|
||||
For instance, the thresholded image for the sample image used above is:
|
||||
|
||||
|
||||
|
||||
This thresholding can be customized with the following parameters:
|
||||
|
||||
#### adaptiveThreshWinSizeMin, adaptiveThreshWinSizeMax, and adaptiveThreshWinSizeStep
|
||||
|
||||
The `adaptiveThreshWinSizeMin` and `adaptiveThreshWinSizeMax` parameters represent the interval where the
|
||||
thresholding window sizes (in pixels) are selected for the adaptive thresholding (see OpenCV
|
||||
`threshold()` and `adaptiveThreshold()` functions for more details).
|
||||
|
||||
The parameter `adaptiveThreshWinSizeStep` indicates the increments of the window size from
|
||||
`adaptiveThreshWinSizeMin` to `adaptiveThreshWinSizeMax`.
|
||||
|
||||
For instance, for the values `adaptiveThreshWinSizeMin` = 5 and `adaptiveThreshWinSizeMax` = 21 and
|
||||
`adaptiveThreshWinSizeStep` = 4, there will be 5 thresholding steps with window sizes 5, 9, 13, 17 and 21.
|
||||
On each thresholding image, marker candidates will be extracted.
|
||||
|
||||
Low values of window size can "break" the marker border if the marker size is too large, causing it to not be detected, as in the following image:
|
||||
|
||||
|
||||
|
||||
On the other hand, too large values can produce the same effect if the markers are too small, and can also
|
||||
reduce the performance. Moreover the process will tend to global thresholding, resulting in a loss of adaptive benefits.
|
||||
|
||||
The simplest case is using the same value for `adaptiveThreshWinSizeMin` and
|
||||
`adaptiveThreshWinSizeMax`, which produces a single thresholding step. However, it is usually better to use a
|
||||
range of values for the window size, although many thresholding steps can also reduce the performance considerably.
|
||||
|
||||
@see cv::aruco::DetectorParameters::adaptiveThreshWinSizeMin, cv::aruco::DetectorParameters::adaptiveThreshWinSizeMax,
|
||||
cv::aruco::DetectorParameters::adaptiveThreshWinSizeStep
|
||||
|
||||
#### adaptiveThreshConstant
|
||||
|
||||
The `adaptiveThreshConstant` parameter represents the constant value added in the thresholding operation (see OpenCV
|
||||
`threshold()` and `adaptiveThreshold()` functions for more details). Its default value is a good option in most cases.
|
||||
|
||||
@see cv::aruco::DetectorParameters::adaptiveThreshConstant
|
||||
|
||||
|
||||
### Contour filtering
|
||||
|
||||
After thresholding, contours are detected. However, not all contours
|
||||
are considered as marker candidates. They are filtered out in different steps so that contours that are
|
||||
very unlikely to be markers are discarded. The parameters in this section customize
|
||||
this filtering process.
|
||||
|
||||
It must be noted that in most cases it is a question of balance between detection capacity
|
||||
and performance. All the considered contours will be processed in the following stages, which usually have
|
||||
a higher computational cost. So, it is preferred to discard invalid candidates in this stage than in the later stages.
|
||||
|
||||
On the other hand, if the filtering conditions are too strict, the real marker contours could be discarded and,
|
||||
hence, not detected.
|
||||
|
||||
#### minMarkerPerimeterRate and maxMarkerPerimeterRate
|
||||
|
||||
These parameters determine the minimum and maximum size of a marker, specifically the minimum and
|
||||
maximum marker perimeter. They are not specified in absolute pixel values, instead they are specified
|
||||
relative to the maximum dimension of the input image.
|
||||
|
||||
For instance, a image with size 640x480 and a minimum relative marker perimeter of 0.05 will lead
|
||||
to a minimum marker perimeter of 640x0.05 = 32 pixels, since 640 is the maximum dimension of the
|
||||
image. The same applies for the `maxMarkerPerimeterRate` parameter.
|
||||
|
||||
If the `minMarkerPerimeterRate` is too low, detection performance can be significantly reduced,
|
||||
as many more contours will be considered for future stages.
|
||||
This penalization is not so noticeable for the `maxMarkerPerimeterRate` parameter, since there are
|
||||
usually many more small contours than big contours.
|
||||
A `minMarkerPerimeterRate` value of 0 and a `maxMarkerPerimeterRate` value of 4 (or more) will be
|
||||
equivalent to consider all the contours in the image, however this is not recommended for
|
||||
performance reasons.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minMarkerPerimeterRate, cv::aruco::DetectorParameters::maxMarkerPerimeterRate
|
||||
|
||||
#### polygonalApproxAccuracyRate
|
||||
|
||||
A polygonal approximation is applied to each candidate and only those that approximate to a square
|
||||
shape are accepted. This value determines the maximum error that the polygonal approximation can
|
||||
produce (see `approxPolyDP()` function for more information).
|
||||
|
||||
This parameter is relative to the candidate length (in pixels). So if the candidate has
|
||||
a perimeter of 100 pixels and the value of `polygonalApproxAccuracyRate` is 0.04, the maximum error
|
||||
would be 100x0.04=5.4 pixels.
|
||||
|
||||
In most cases, the default value works fine, but higher error values could be necessary for highly
|
||||
distorted images.
|
||||
|
||||
@see cv::aruco::DetectorParameters::polygonalApproxAccuracyRate
|
||||
|
||||
#### minCornerDistanceRate
|
||||
|
||||
Minimum distance between any pair of corners in the same marker. It is expressed relative to the marker
|
||||
perimeter. Minimum distance in pixels is Perimeter * minCornerDistanceRate.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minCornerDistanceRate
|
||||
|
||||
#### minMarkerDistanceRate
|
||||
|
||||
Minimum distance between any pair of corners from two different markers. It is expressed relative to
|
||||
the minimum marker perimeter of the two markers. If two candidates are too close, the smaller one is ignored.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minMarkerDistanceRate
|
||||
|
||||
#### minDistanceToBorder
|
||||
|
||||
Minimum distance to any of the marker corners to the image border (in pixels). Markers partially occluded
|
||||
by the image border can be correctly detected if the occlusion is small. However, if one of the corners
|
||||
is occluded, the returned corner is usually placed in a wrong position near the image border.
|
||||
|
||||
If the position of marker corners is important, for instance if you want to do pose estimation, it is
|
||||
better to discard any markers whose corners are too close to the image border. Elsewhere, it is not necessary.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minDistanceToBorder
|
||||
|
||||
### Bits Extraction
|
||||
|
||||
After candidate detection, the bits of each candidate are analyzed in order to determine if they
|
||||
are markers or not.
|
||||
|
||||
Before analyzing the binary code itself, the bits need to be extracted. To do this, perspective
|
||||
distortion is corrected and the resulting image is thresholded using Otsu threshold to separate
|
||||
black and white pixels.
|
||||
|
||||
This is an example of the image obtained after removing the perspective distortion of a marker:
|
||||
|
||||
|
||||
|
||||
Then, the image is divided into a grid with the same number of cells as the number of bits in the marker.
|
||||
In each cell, the number of black and white pixels are counted to determine the bit value assigned
|
||||
to the cell (from the majority value):
|
||||
|
||||
|
||||
|
||||
There are several parameters that can customize this process:
|
||||
|
||||
#### markerBorderBits
|
||||
|
||||
This parameter indicates the width of the marker border. It is relative to the size of each bit. So, a
|
||||
value of 2 indicates the border has the width of two internal bits.
|
||||
|
||||
This parameter needs to coincide with the border size of the markers you are using. The border size
|
||||
can be configured in the marker drawing functions such as `generateImageMarker()`.
|
||||
|
||||
@see cv::aruco::DetectorParameters::markerBorderBits
|
||||
|
||||
#### minOtsuStdDev
|
||||
|
||||
This value determines the minimum standard deviation of the pixel values to perform Otsu
|
||||
thresholding. If the deviation is low, it probably means that all the square is black (or white)
|
||||
and applying Otsu does not make sense. If this is the case, all the bits are set to 0 (or 1)
|
||||
depending on whether the mean value is higher or lower than 128.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minOtsuStdDev
|
||||
|
||||
#### perspectiveRemovePixelPerCell
|
||||
|
||||
This parameter determines the number of pixels (per cell) in the obtained image after correcting perspective
|
||||
distortion (including the border). This is the size of the red squares in the image above.
|
||||
|
||||
For instance, let’s assume we are dealing with markers of 5x5 bits and border size of 1 bit
|
||||
(see `markerBorderBits`). Then, the total number of cells/bits per dimension is 5 + 2*1 = 7 (the border
|
||||
has to be counted twice). The total number of cells is 7x7.
|
||||
|
||||
If the value of `perspectiveRemovePixelPerCell` is 10, then the size of the obtained image will be
|
||||
10*7 = 70 -> 70x70 pixels.
|
||||
|
||||
A higher value of this parameter can improve the bits extraction process (up to some degree),
|
||||
however it can penalize the performance.
|
||||
|
||||
@see cv::aruco::DetectorParameters::perspectiveRemovePixelPerCell
|
||||
|
||||
#### perspectiveRemoveIgnoredMarginPerCell
|
||||
|
||||
When extracting the bits of each cell, the numbers of black and white pixels are counted. In general, it is
|
||||
not recommended to consider all the cell pixels. Instead it is better to ignore some pixels in the
|
||||
margins of the cells.
|
||||
|
||||
The reason for this is that, after removing the perspective distortion, the cells’ colors are, in general, not
|
||||
perfectly separated and white cells can invade some pixels of black cells (and vice versa). Thus, it is
|
||||
better to ignore some pixels just to avoid counting erroneous pixels.
|
||||
|
||||
For instance, in the following image:
|
||||
|
||||
|
||||
|
||||
only the pixels inside the green squares are considered. It can be seen in the right image that
|
||||
the resulting pixels contain a lower amount of noise from neighbor cells.
|
||||
The `perspectiveRemoveIgnoredMarginPerCell` parameter indicates the difference between the red and
|
||||
the green squares.
|
||||
|
||||
This parameter is relative to the total size of the cell. For instance if the cell size is 40 pixels and the
|
||||
value of this parameter is 0.1, a margin of 40*0.1=4 pixels is ignored in the cells. This means that the total
|
||||
number of pixels that would be analyzed in each cell would actually be 32x32, instead of 40x40.
|
||||
|
||||
@see cv::aruco::DetectorParameters::perspectiveRemoveIgnoredMarginPerCell
|
||||
|
||||
|
||||
### Marker identification
|
||||
|
||||
After the bits have been extracted, the next step is checking whether the extracted code belongs to the marker
|
||||
dictionary and, if necessary, error correction can be performed.
|
||||
|
||||
#### maxErroneousBitsInBorderRate
|
||||
|
||||
The bits of the marker border should be black. This parameter specifies the allowed number of erroneous
|
||||
bits in the border, i.e. the maximum number of white bits in the border. It is represented
|
||||
relative to the total number of bits in the marker.
|
||||
|
||||
@see cv::aruco::DetectorParameters::maxErroneousBitsInBorderRate
|
||||
|
||||
#### errorCorrectionRate
|
||||
|
||||
Each marker dictionary has a theoretical maximum number of bits that can be corrected (`Dictionary.maxCorrectionBits`).
|
||||
However, this value can be modified by the `errorCorrectionRate` parameter.
|
||||
|
||||
For instance, if the allowed number of bits that can be corrected (for the used dictionary) is 6 and the value of `errorCorrectionRate` is
|
||||
0.5, the real maximum number of bits that can be corrected is 6*0.5=3 bits.
|
||||
|
||||
This value is useful to reduce the error correction capabilities in order to avoid false positives.
|
||||
|
||||
@see cv::aruco::DetectorParameters::errorCorrectionRate
|
||||
|
||||
|
||||
### Corner Refinement
|
||||
|
||||
After markers have been detected and identified, the last step is performing subpixel refinement
|
||||
of the corner positions (see OpenCV `cornerSubPix()` and `cv::aruco::CornerRefineMethod`).
|
||||
|
||||
Note that this step is optional and it only makes sense if the positions of the marker corners have to
|
||||
be accurate, for instance for pose estimation. It is usually a time-consuming step and therefore is disabled by default.
|
||||
|
||||
#### cornerRefinementMethod
|
||||
|
||||
This parameter determines whether the corner subpixel process is performed or not and which method to use
|
||||
if it is being performed. It can be disabled if accurate corners are not necessary. Possible values are
|
||||
`CORNER_REFINE_NONE`, `CORNER_REFINE_SUBPIX`, `CORNER_REFINE_CONTOUR`, and `CORNER_REFINE_APRILTAG`.
|
||||
|
||||
@see cv::aruco::DetectorParameters::cornerRefinementMethod
|
||||
|
||||
#### cornerRefinementWinSize
|
||||
|
||||
This parameter determines the maximum window size for the corner refinement process.
|
||||
|
||||
High values can cause close corners of the image to be included in the window area, so that the corner
|
||||
of the marker moves to a different and incorrect location during the process. Also, it may affect performance.
|
||||
The window size may decrease if the ArUco marker is too small, check cv::aruco::DetectorParameters::relativeCornerRefinmentWinSize.
|
||||
The final window size is calculated as: min(cornerRefinementWinSize, averageArucoModuleSize*relativeCornerRefinmentWinSize),
|
||||
where averageArucoModuleSize is average module size of ArUco marker in pixels.
|
||||
|
||||
@see cv::aruco::DetectorParameters::cornerRefinementWinSize
|
||||
|
||||
#### relativeCornerRefinmentWinSize
|
||||
|
||||
Dynamic window size for corner refinement relative to Aruco module size (default 0.3).
|
||||
|
||||
The final window size is calculated as: min(cornerRefinementWinSize, averageArucoModuleSize*relativeCornerRefinmentWinSize),
|
||||
where averageArucoModuleSize is average module size of ArUco marker in pixels.
|
||||
In the case of markers located far from each other, it may be useful to increase the value of the parameter to 0.4-0.5.
|
||||
In the case of markers located close to each other, it may be useful to decrease the parameter value to 0.1-0.2.
|
||||
|
||||
@see cv::aruco::DetectorParameters::relativeCornerRefinmentWinSize
|
||||
|
||||
#### cornerRefinementMaxIterations and cornerRefinementMinAccuracy
|
||||
|
||||
These two parameters determine the stop criteria of the subpixel refinement process. The
|
||||
`cornerRefinementMaxIterations` indicates the maximum number of iterations and
|
||||
`cornerRefinementMinAccuracy` the minimum error value before stopping the process.
|
||||
|
||||
If the number of iterations is too high, it may affect the performance. On the other hand, if it is
|
||||
too low, it can result in poor subpixel refinement.
|
||||
|
||||
@see cv::aruco::DetectorParameters::cornerRefinementMaxIterations, cv::aruco::DetectorParameters::cornerRefinementMinAccuracy
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 5.4 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
BIN
doc/tutorials/objdetect/aruco_detection/images/marker23.png
Normal file
BIN
doc/tutorials/objdetect/aruco_detection/images/marker23.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.2 KiB |
BIN
doc/tutorials/objdetect/aruco_detection/images/markers.jpg
Normal file
BIN
doc/tutorials/objdetect/aruco_detection/images/markers.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 28 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 3.2 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 33 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 79 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 28 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 38 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 9.2 KiB |
@ -0,0 +1,5 @@
|
||||
Object Detection (objdetect module) {#tutorial_table_of_content_objdetect}
|
||||
==========================================================
|
||||
|
||||
- @subpage tutorial_aruco_detection
|
||||
- @subpage tutorial_aruco_board_detection
|
||||
@ -6,6 +6,7 @@ OpenCV Tutorials {#tutorial_root}
|
||||
- @subpage tutorial_table_of_content_imgproc - image processing functions
|
||||
- @subpage tutorial_table_of_content_app - application utils (GUI, image/video input/output)
|
||||
- @subpage tutorial_table_of_content_calib3d - extract 3D world information from 2D images
|
||||
- @subpage tutorial_table_of_content_objdetect - INSERT OBJDETECT MODULE INFO
|
||||
- @subpage tutorial_table_of_content_features2d - feature detectors, descriptors and matching framework
|
||||
- @subpage tutorial_table_of_content_dnn - infer neural networks using built-in _dnn_ module
|
||||
- @subpage tutorial_table_of_content_gapi - graph-based approach to computer vision algorithms building
|
||||
|
||||
@ -77,6 +77,9 @@ public:
|
||||
* If camera parameters are provided, the process is based in an approximated pose estimation, else it is based on local homography.
|
||||
* Only visible corners are returned. For each corner, its corresponding identifier is also returned in charucoIds.
|
||||
* @sa findChessboardCorners
|
||||
* @note After OpenCV 4.6.0, there was an incompatible change in the ChArUco pattern generation algorithm for even row counts.
|
||||
* Use cv::aruco::CharucoBoard::setLegacyPattern() to ensure compatibility with patterns created using OpenCV versions prior to 4.6.0.
|
||||
* For more information, see the issue: https://github.com/opencv/opencv/issues/23152
|
||||
*/
|
||||
CV_WRAP void detectBoard(InputArray image, OutputArray charucoCorners, OutputArray charucoIds,
|
||||
InputOutputArrayOfArrays markerCorners = noArray(),
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 78 KiB |
@ -283,7 +283,7 @@ int main(int argc, char *argv[])
|
||||
int markerSize = parser.get<int>("markerSize");
|
||||
bool checkFlippedMarkers = parser.get<bool>("r");
|
||||
|
||||
aruco::Dictionary dictionary = aruco::getPredefinedDictionary(0);
|
||||
aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
|
||||
|
||||
if (parser.has("d")) {
|
||||
string arucoDictName = parser.get<string>("d");
|
||||
|
||||
@ -354,7 +354,7 @@ int main(int argc, char* argv[])
|
||||
}
|
||||
else {
|
||||
// default dictionary
|
||||
dictionary = cv::aruco::getPredefinedDictionary(0);
|
||||
dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
|
||||
}
|
||||
cv::aruco::CharucoBoard ch_board({s.boardSize.width, s.boardSize.height}, s.squareSize, s.markerSize, dictionary);
|
||||
cv::aruco::CharucoDetector ch_detector(ch_board);
|
||||
|
||||
@ -0,0 +1,48 @@
|
||||
#include <opencv2/highgui.hpp>
|
||||
#include <opencv2/objdetect/aruco_detector.hpp>
|
||||
#include <opencv2/calib3d.hpp>
|
||||
#include <ctime>
|
||||
|
||||
namespace {
|
||||
inline static bool readCameraParameters(const std::string& filename, cv::Mat &camMatrix, cv::Mat &distCoeffs) {
|
||||
cv::FileStorage fs(filename, cv::FileStorage::READ);
|
||||
if (!fs.isOpened())
|
||||
return false;
|
||||
fs["camera_matrix"] >> camMatrix;
|
||||
fs["distortion_coefficients"] >> distCoeffs;
|
||||
return true;
|
||||
}
|
||||
|
||||
inline static bool saveCameraParams(const std::string &filename, cv::Size imageSize, float aspectRatio, int flags,
|
||||
const cv::Mat &cameraMatrix, const cv::Mat &distCoeffs, double totalAvgErr) {
|
||||
cv::FileStorage fs(filename, cv::FileStorage::WRITE);
|
||||
if (!fs.isOpened())
|
||||
return false;
|
||||
|
||||
time_t tt;
|
||||
time(&tt);
|
||||
struct tm *t2 = localtime(&tt);
|
||||
char buf[1024];
|
||||
strftime(buf, sizeof(buf) - 1, "%c", t2);
|
||||
|
||||
fs << "calibration_time" << buf;
|
||||
fs << "image_width" << imageSize.width;
|
||||
fs << "image_height" << imageSize.height;
|
||||
|
||||
if (flags & cv::CALIB_FIX_ASPECT_RATIO) fs << "aspectRatio" << aspectRatio;
|
||||
|
||||
if (flags != 0) {
|
||||
sprintf(buf, "flags: %s%s%s%s",
|
||||
flags & cv::CALIB_USE_INTRINSIC_GUESS ? "+use_intrinsic_guess" : "",
|
||||
flags & cv::CALIB_FIX_ASPECT_RATIO ? "+fix_aspectRatio" : "",
|
||||
flags & cv::CALIB_FIX_PRINCIPAL_POINT ? "+fix_principal_point" : "",
|
||||
flags & cv::CALIB_ZERO_TANGENT_DIST ? "+zero_tangent_dist" : "");
|
||||
}
|
||||
fs << "flags" << flags;
|
||||
fs << "camera_matrix" << cameraMatrix;
|
||||
fs << "distortion_coefficients" << distCoeffs;
|
||||
fs << "avg_reprojection_error" << totalAvgErr;
|
||||
return true;
|
||||
}
|
||||
|
||||
}
|
||||
95
samples/cpp/tutorial_code/objectDetection/create_board.cpp
Normal file
95
samples/cpp/tutorial_code/objectDetection/create_board.cpp
Normal file
@ -0,0 +1,95 @@
|
||||
#include <opencv2/highgui.hpp>
|
||||
#include <opencv2/objdetect/aruco_detector.hpp>
|
||||
#include <iostream>
|
||||
#include "aruco_samples_utility.hpp"
|
||||
|
||||
using namespace cv;
|
||||
|
||||
namespace {
|
||||
const char* about = "Create an ArUco grid board image";
|
||||
const char* keys =
|
||||
"{@outfile |<none> | Output image }"
|
||||
"{w | | Number of markers in X direction }"
|
||||
"{h | | Number of markers in Y direction }"
|
||||
"{l | | Marker side length (in pixels) }"
|
||||
"{s | | Separation between two consecutive markers in the grid (in pixels)}"
|
||||
"{d | | dictionary: DICT_4X4_50=0, DICT_4X4_100=1, DICT_4X4_250=2,"
|
||||
"DICT_4X4_1000=3, DICT_5X5_50=4, DICT_5X5_100=5, DICT_5X5_250=6, DICT_5X5_1000=7, "
|
||||
"DICT_6X6_50=8, DICT_6X6_100=9, DICT_6X6_250=10, DICT_6X6_1000=11, DICT_7X7_50=12,"
|
||||
"DICT_7X7_100=13, DICT_7X7_250=14, DICT_7X7_1000=15, DICT_ARUCO_ORIGINAL = 16}"
|
||||
"{cd | | Input file with custom dictionary }"
|
||||
"{m | | Margins size (in pixels). Default is marker separation (-s) }"
|
||||
"{bb | 1 | Number of bits in marker borders }"
|
||||
"{si | false | show generated image }";
|
||||
}
|
||||
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
CommandLineParser parser(argc, argv, keys);
|
||||
parser.about(about);
|
||||
|
||||
if(argc < 7) {
|
||||
parser.printMessage();
|
||||
return 0;
|
||||
}
|
||||
|
||||
int markersX = parser.get<int>("w");
|
||||
int markersY = parser.get<int>("h");
|
||||
int markerLength = parser.get<int>("l");
|
||||
int markerSeparation = parser.get<int>("s");
|
||||
int margins = markerSeparation;
|
||||
if(parser.has("m")) {
|
||||
margins = parser.get<int>("m");
|
||||
}
|
||||
|
||||
int borderBits = parser.get<int>("bb");
|
||||
bool showImage = parser.get<bool>("si");
|
||||
|
||||
String out = parser.get<String>(0);
|
||||
|
||||
if(!parser.check()) {
|
||||
parser.printErrors();
|
||||
return 0;
|
||||
}
|
||||
|
||||
Size imageSize;
|
||||
imageSize.width = markersX * (markerLength + markerSeparation) - markerSeparation + 2 * margins;
|
||||
imageSize.height =
|
||||
markersY * (markerLength + markerSeparation) - markerSeparation + 2 * margins;
|
||||
|
||||
aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
|
||||
if (parser.has("d")) {
|
||||
int dictionaryId = parser.get<int>("d");
|
||||
dictionary = aruco::getPredefinedDictionary(aruco::PredefinedDictionaryType(dictionaryId));
|
||||
}
|
||||
else if (parser.has("cd")) {
|
||||
FileStorage fs(parser.get<std::string>("cd"), FileStorage::READ);
|
||||
bool readOk = dictionary.readDictionary(fs.root());
|
||||
if(!readOk)
|
||||
{
|
||||
std::cerr << "Invalid dictionary file" << std::endl;
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
else {
|
||||
std::cerr << "Dictionary not specified" << std::endl;
|
||||
return 0;
|
||||
}
|
||||
|
||||
aruco::GridBoard board(Size(markersX, markersY), float(markerLength), float(markerSeparation), dictionary);
|
||||
|
||||
// show created board
|
||||
//! [aruco_generate_board_image]
|
||||
Mat boardImage;
|
||||
board.generateImage(imageSize, boardImage, margins, borderBits);
|
||||
//! [aruco_generate_board_image]
|
||||
|
||||
if(showImage) {
|
||||
imshow("board", boardImage);
|
||||
waitKey(0);
|
||||
}
|
||||
|
||||
imwrite(out, boardImage);
|
||||
|
||||
return 0;
|
||||
}
|
||||
77
samples/cpp/tutorial_code/objectDetection/create_marker.cpp
Normal file
77
samples/cpp/tutorial_code/objectDetection/create_marker.cpp
Normal file
@ -0,0 +1,77 @@
|
||||
#include <opencv2/highgui.hpp>
|
||||
#include <opencv2/objdetect/aruco_detector.hpp>
|
||||
#include <iostream>
|
||||
#include "aruco_samples_utility.hpp"
|
||||
|
||||
using namespace cv;
|
||||
|
||||
namespace {
|
||||
const char* about = "Create an ArUco marker image";
|
||||
|
||||
//! [aruco_create_markers_keys]
|
||||
const char* keys =
|
||||
"{@outfile |<none> | Output image }"
|
||||
"{d | | dictionary: DICT_4X4_50=0, DICT_4X4_100=1, DICT_4X4_250=2,"
|
||||
"DICT_4X4_1000=3, DICT_5X5_50=4, DICT_5X5_100=5, DICT_5X5_250=6, DICT_5X5_1000=7, "
|
||||
"DICT_6X6_50=8, DICT_6X6_100=9, DICT_6X6_250=10, DICT_6X6_1000=11, DICT_7X7_50=12,"
|
||||
"DICT_7X7_100=13, DICT_7X7_250=14, DICT_7X7_1000=15, DICT_ARUCO_ORIGINAL = 16}"
|
||||
"{cd | | Input file with custom dictionary }"
|
||||
"{id | | Marker id in the dictionary }"
|
||||
"{ms | 200 | Marker size in pixels }"
|
||||
"{bb | 1 | Number of bits in marker borders }"
|
||||
"{si | false | show generated image }";
|
||||
}
|
||||
//! [aruco_create_markers_keys]
|
||||
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
CommandLineParser parser(argc, argv, keys);
|
||||
parser.about(about);
|
||||
|
||||
if(argc < 4) {
|
||||
parser.printMessage();
|
||||
return 0;
|
||||
}
|
||||
|
||||
int markerId = parser.get<int>("id");
|
||||
int borderBits = parser.get<int>("bb");
|
||||
int markerSize = parser.get<int>("ms");
|
||||
bool showImage = parser.get<bool>("si");
|
||||
|
||||
String out = parser.get<String>(0);
|
||||
|
||||
if(!parser.check()) {
|
||||
parser.printErrors();
|
||||
return 0;
|
||||
}
|
||||
|
||||
aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
|
||||
if (parser.has("d")) {
|
||||
int dictionaryId = parser.get<int>("d");
|
||||
dictionary = aruco::getPredefinedDictionary(aruco::PredefinedDictionaryType(dictionaryId));
|
||||
}
|
||||
else if (parser.has("cd")) {
|
||||
FileStorage fs(parser.get<std::string>("cd"), FileStorage::READ);
|
||||
bool readOk = dictionary.readDictionary(fs.root());
|
||||
if(!readOk) {
|
||||
std::cerr << "Invalid dictionary file" << std::endl;
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
else {
|
||||
std::cerr << "Dictionary not specified" << std::endl;
|
||||
return 0;
|
||||
}
|
||||
|
||||
Mat markerImg;
|
||||
aruco::generateImageMarker(dictionary, markerId, markerSize, markerImg, borderBits);
|
||||
|
||||
if(showImage) {
|
||||
imshow("marker", markerImg);
|
||||
waitKey(0);
|
||||
}
|
||||
|
||||
imwrite(out, markerImg);
|
||||
|
||||
return 0;
|
||||
}
|
||||
201
samples/cpp/tutorial_code/objectDetection/detect_board.cpp
Normal file
201
samples/cpp/tutorial_code/objectDetection/detect_board.cpp
Normal file
@ -0,0 +1,201 @@
|
||||
#include <iostream>
|
||||
#include <vector>
|
||||
#include <opencv2/highgui.hpp>
|
||||
#include <opencv2/objdetect/aruco_detector.hpp>
|
||||
#include "aruco_samples_utility.hpp"
|
||||
|
||||
using namespace std;
|
||||
using namespace cv;
|
||||
|
||||
namespace {
|
||||
const char* about = "Pose estimation using a ArUco Planar Grid board";
|
||||
|
||||
//! [aruco_detect_board_keys]
|
||||
const char* keys =
|
||||
"{w | | Number of squares in X direction }"
|
||||
"{h | | Number of squares in Y direction }"
|
||||
"{l | | Marker side length (in pixels) }"
|
||||
"{s | | Separation between two consecutive markers in the grid (in pixels)}"
|
||||
"{d | | dictionary: DICT_4X4_50=0, DICT_4X4_100=1, DICT_4X4_250=2,"
|
||||
"DICT_4X4_1000=3, DICT_5X5_50=4, DICT_5X5_100=5, DICT_5X5_250=6, DICT_5X5_1000=7, "
|
||||
"DICT_6X6_50=8, DICT_6X6_100=9, DICT_6X6_250=10, DICT_6X6_1000=11, DICT_7X7_50=12,"
|
||||
"DICT_7X7_100=13, DICT_7X7_250=14, DICT_7X7_1000=15, DICT_ARUCO_ORIGINAL = 16}"
|
||||
"{cd | | Input file with custom dictionary }"
|
||||
"{c | | Output file with calibrated camera parameters }"
|
||||
"{v | | Input from video or image file, if omitted, input comes from camera }"
|
||||
"{ci | 0 | Camera id if input doesnt come from video (-v) }"
|
||||
"{dp | | File of marker detector parameters }"
|
||||
"{rs | | Apply refind strategy }"
|
||||
"{r | | show rejected candidates too }";
|
||||
}
|
||||
//! [aruco_detect_board_keys]
|
||||
|
||||
static void readDetectorParamsFromCommandLine(CommandLineParser &parser, aruco::DetectorParameters& detectorParams) {
|
||||
if(parser.has("dp")) {
|
||||
FileStorage fs(parser.get<string>("dp"), FileStorage::READ);
|
||||
bool readOk = detectorParams.readDetectorParameters(fs.root());
|
||||
if(!readOk) {
|
||||
cerr << "Invalid detector parameters file" << endl;
|
||||
throw -1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static void readCameraParamsFromCommandLine(CommandLineParser &parser, Mat& camMatrix, Mat& distCoeffs) {
|
||||
if(parser.has("c")) {
|
||||
bool readOk = readCameraParameters(parser.get<string>("c"), camMatrix, distCoeffs);
|
||||
if(!readOk) {
|
||||
cerr << "Invalid camera file" << endl;
|
||||
throw -1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static void readDictionatyFromCommandLine(CommandLineParser &parser, aruco::Dictionary& dictionary) {
|
||||
if (parser.has("d")) {
|
||||
int dictionaryId = parser.get<int>("d");
|
||||
dictionary = aruco::getPredefinedDictionary(aruco::PredefinedDictionaryType(dictionaryId));
|
||||
}
|
||||
else if (parser.has("cd")) {
|
||||
FileStorage fs(parser.get<string>("cd"), FileStorage::READ);
|
||||
bool readOk = dictionary.readDictionary(fs.root());
|
||||
if(!readOk) {
|
||||
cerr << "Invalid dictionary file" << endl;
|
||||
throw -1;
|
||||
}
|
||||
}
|
||||
else {
|
||||
cerr << "Dictionary not specified" << endl;
|
||||
throw -1;
|
||||
}
|
||||
}
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
CommandLineParser parser(argc, argv, keys);
|
||||
parser.about(about);
|
||||
|
||||
if(argc < 7) {
|
||||
parser.printMessage();
|
||||
return 0;
|
||||
}
|
||||
|
||||
//! [aruco_detect_board_full_sample]
|
||||
int markersX = parser.get<int>("w");
|
||||
int markersY = parser.get<int>("h");
|
||||
float markerLength = parser.get<float>("l");
|
||||
float markerSeparation = parser.get<float>("s");
|
||||
bool showRejected = parser.has("r");
|
||||
bool refindStrategy = parser.has("rs");
|
||||
int camId = parser.get<int>("ci");
|
||||
|
||||
|
||||
Mat camMatrix, distCoeffs;
|
||||
readCameraParamsFromCommandLine(parser, camMatrix, distCoeffs);
|
||||
|
||||
aruco::DetectorParameters detectorParams;
|
||||
detectorParams.cornerRefinementMethod = aruco::CORNER_REFINE_SUBPIX; // do corner refinement in markers
|
||||
readDetectorParamsFromCommandLine(parser, detectorParams);
|
||||
|
||||
String video;
|
||||
if(parser.has("v")) {
|
||||
video = parser.get<String>("v");
|
||||
}
|
||||
|
||||
if(!parser.check()) {
|
||||
parser.printErrors();
|
||||
return 0;
|
||||
}
|
||||
|
||||
aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
|
||||
readDictionatyFromCommandLine(parser, dictionary);
|
||||
|
||||
aruco::ArucoDetector detector(dictionary, detectorParams);
|
||||
VideoCapture inputVideo;
|
||||
int waitTime;
|
||||
if(!video.empty()) {
|
||||
inputVideo.open(video);
|
||||
waitTime = 0;
|
||||
} else {
|
||||
inputVideo.open(camId);
|
||||
waitTime = 10;
|
||||
}
|
||||
|
||||
float axisLength = 0.5f * ((float)min(markersX, markersY) * (markerLength + markerSeparation) +
|
||||
markerSeparation);
|
||||
|
||||
// Create GridBoard object
|
||||
//! [aruco_create_board]
|
||||
aruco::GridBoard board(Size(markersX, markersY), markerLength, markerSeparation, dictionary);
|
||||
//! [aruco_create_board]
|
||||
|
||||
// Also you could create Board object
|
||||
//vector<vector<Point3f> > objPoints; // array of object points of all the marker corners in the board
|
||||
//vector<int> ids; // vector of the identifiers of the markers in the board
|
||||
//aruco::Board board(objPoints, dictionary, ids);
|
||||
|
||||
double totalTime = 0;
|
||||
int totalIterations = 0;
|
||||
|
||||
while(inputVideo.grab()) {
|
||||
Mat image, imageCopy;
|
||||
inputVideo.retrieve(image);
|
||||
|
||||
double tick = (double)getTickCount();
|
||||
|
||||
vector<int> ids;
|
||||
vector<vector<Point2f>> corners, rejected;
|
||||
Vec3d rvec, tvec;
|
||||
|
||||
//! [aruco_detect_and_refine]
|
||||
|
||||
// Detect markers
|
||||
detector.detectMarkers(image, corners, ids, rejected);
|
||||
|
||||
// Refind strategy to detect more markers
|
||||
if(refindStrategy)
|
||||
detector.refineDetectedMarkers(image, board, corners, ids, rejected, camMatrix,
|
||||
distCoeffs);
|
||||
|
||||
//! [aruco_detect_and_refine]
|
||||
|
||||
// Estimate board pose
|
||||
int markersOfBoardDetected = 0;
|
||||
if(!ids.empty()) {
|
||||
// Get object and image points for the solvePnP function
|
||||
cv::Mat objPoints, imgPoints;
|
||||
board.matchImagePoints(corners, ids, objPoints, imgPoints);
|
||||
|
||||
// Find pose
|
||||
cv::solvePnP(objPoints, imgPoints, camMatrix, distCoeffs, rvec, tvec);
|
||||
|
||||
markersOfBoardDetected = (int)objPoints.total() / 4;
|
||||
}
|
||||
|
||||
double currentTime = ((double)getTickCount() - tick) / getTickFrequency();
|
||||
totalTime += currentTime;
|
||||
totalIterations++;
|
||||
if(totalIterations % 30 == 0) {
|
||||
cout << "Detection Time = " << currentTime * 1000 << " ms "
|
||||
<< "(Mean = " << 1000 * totalTime / double(totalIterations) << " ms)" << endl;
|
||||
}
|
||||
|
||||
// Draw results
|
||||
image.copyTo(imageCopy);
|
||||
if(!ids.empty()) {
|
||||
aruco::drawDetectedMarkers(imageCopy, corners, ids);
|
||||
}
|
||||
|
||||
if(showRejected && !rejected.empty())
|
||||
aruco::drawDetectedMarkers(imageCopy, rejected, noArray(), Scalar(100, 0, 255));
|
||||
|
||||
if(markersOfBoardDetected > 0)
|
||||
cv::drawFrameAxes(imageCopy, camMatrix, distCoeffs, rvec, tvec, axisLength);
|
||||
|
||||
imshow("out", imageCopy);
|
||||
char key = (char)waitKey(waitTime);
|
||||
if(key == 27) break;
|
||||
//! [aruco_detect_board_full_sample]
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
177
samples/cpp/tutorial_code/objectDetection/detect_markers.cpp
Normal file
177
samples/cpp/tutorial_code/objectDetection/detect_markers.cpp
Normal file
@ -0,0 +1,177 @@
|
||||
#include <opencv2/highgui.hpp>
|
||||
#include <opencv2/objdetect/aruco_detector.hpp>
|
||||
#include <iostream>
|
||||
#include "aruco_samples_utility.hpp"
|
||||
|
||||
using namespace std;
|
||||
using namespace cv;
|
||||
|
||||
namespace {
|
||||
const char* about = "Basic marker detection";
|
||||
|
||||
//! [aruco_detect_markers_keys]
|
||||
const char* keys =
|
||||
"{d | | dictionary: DICT_4X4_50=0, DICT_4X4_100=1, DICT_4X4_250=2,"
|
||||
"DICT_4X4_1000=3, DICT_5X5_50=4, DICT_5X5_100=5, DICT_5X5_250=6, DICT_5X5_1000=7, "
|
||||
"DICT_6X6_50=8, DICT_6X6_100=9, DICT_6X6_250=10, DICT_6X6_1000=11, DICT_7X7_50=12,"
|
||||
"DICT_7X7_100=13, DICT_7X7_250=14, DICT_7X7_1000=15, DICT_ARUCO_ORIGINAL = 16,"
|
||||
"DICT_APRILTAG_16h5=17, DICT_APRILTAG_25h9=18, DICT_APRILTAG_36h10=19, DICT_APRILTAG_36h11=20}"
|
||||
"{cd | | Input file with custom dictionary }"
|
||||
"{v | | Input from video or image file, if ommited, input comes from camera }"
|
||||
"{ci | 0 | Camera id if input doesnt come from video (-v) }"
|
||||
"{c | | Camera intrinsic parameters. Needed for camera pose }"
|
||||
"{l | 0.1 | Marker side length (in meters). Needed for correct scale in camera pose }"
|
||||
"{dp | | File of marker detector parameters }"
|
||||
"{r | | show rejected candidates too }"
|
||||
"{refine | | Corner refinement: CORNER_REFINE_NONE=0, CORNER_REFINE_SUBPIX=1,"
|
||||
"CORNER_REFINE_CONTOUR=2, CORNER_REFINE_APRILTAG=3}";
|
||||
}
|
||||
//! [aruco_detect_markers_keys]
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
CommandLineParser parser(argc, argv, keys);
|
||||
parser.about(about);
|
||||
|
||||
if(argc < 2) {
|
||||
parser.printMessage();
|
||||
return 0;
|
||||
}
|
||||
|
||||
bool showRejected = parser.has("r");
|
||||
bool estimatePose = parser.has("c");
|
||||
float markerLength = parser.get<float>("l");
|
||||
|
||||
cv::aruco::DetectorParameters detectorParams;
|
||||
if(parser.has("dp")) {
|
||||
cv::FileStorage fs(parser.get<string>("dp"), FileStorage::READ);
|
||||
bool readOk = detectorParams.readDetectorParameters(fs.root());
|
||||
if(!readOk) {
|
||||
cerr << "Invalid detector parameters file" << endl;
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
if (parser.has("refine")) {
|
||||
// override cornerRefinementMethod read from config file
|
||||
detectorParams.cornerRefinementMethod = parser.get<aruco::CornerRefineMethod>("refine");
|
||||
}
|
||||
std::cout << "Corner refinement method (0: None, 1: Subpixel, 2:contour, 3: AprilTag 2): " << (int)detectorParams.cornerRefinementMethod << std::endl;
|
||||
|
||||
int camId = parser.get<int>("ci");
|
||||
|
||||
String video;
|
||||
if(parser.has("v")) {
|
||||
video = parser.get<String>("v");
|
||||
}
|
||||
|
||||
if(!parser.check()) {
|
||||
parser.printErrors();
|
||||
return 0;
|
||||
}
|
||||
|
||||
aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
|
||||
if (parser.has("d")) {
|
||||
int dictionaryId = parser.get<int>("d");
|
||||
dictionary = aruco::getPredefinedDictionary(aruco::PredefinedDictionaryType(dictionaryId));
|
||||
}
|
||||
else if (parser.has("cd")) {
|
||||
cv::FileStorage fs(parser.get<std::string>("cd"), FileStorage::READ);
|
||||
bool readOk = dictionary.readDictionary(fs.root());
|
||||
if(!readOk) {
|
||||
std::cerr << "Invalid dictionary file" << std::endl;
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
else {
|
||||
std::cerr << "Dictionary not specified" << std::endl;
|
||||
return 0;
|
||||
}
|
||||
|
||||
//! [aruco_pose_estimation1]
|
||||
cv::Mat camMatrix, distCoeffs;
|
||||
if(estimatePose) {
|
||||
// You can read camera parameters from tutorial_camera_params.yml
|
||||
bool readOk = readCameraParameters(parser.get<string>("c"), camMatrix, distCoeffs);
|
||||
if(!readOk) {
|
||||
cerr << "Invalid camera file" << endl;
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
//! [aruco_pose_estimation1]
|
||||
//! [aruco_detect_markers]
|
||||
cv::aruco::ArucoDetector detector(dictionary, detectorParams);
|
||||
cv::VideoCapture inputVideo;
|
||||
int waitTime;
|
||||
if(!video.empty()) {
|
||||
inputVideo.open(video);
|
||||
waitTime = 0;
|
||||
} else {
|
||||
inputVideo.open(camId);
|
||||
waitTime = 10;
|
||||
}
|
||||
|
||||
double totalTime = 0;
|
||||
int totalIterations = 0;
|
||||
|
||||
//! [aruco_pose_estimation2]
|
||||
// set coordinate system
|
||||
cv::Mat objPoints(4, 1, CV_32FC3);
|
||||
objPoints.ptr<Vec3f>(0)[0] = Vec3f(-markerLength/2.f, markerLength/2.f, 0);
|
||||
objPoints.ptr<Vec3f>(0)[1] = Vec3f(markerLength/2.f, markerLength/2.f, 0);
|
||||
objPoints.ptr<Vec3f>(0)[2] = Vec3f(markerLength/2.f, -markerLength/2.f, 0);
|
||||
objPoints.ptr<Vec3f>(0)[3] = Vec3f(-markerLength/2.f, -markerLength/2.f, 0);
|
||||
//! [aruco_pose_estimation2]
|
||||
|
||||
while(inputVideo.grab()) {
|
||||
cv::Mat image, imageCopy;
|
||||
inputVideo.retrieve(image);
|
||||
|
||||
double tick = (double)getTickCount();
|
||||
|
||||
//! [aruco_pose_estimation3]
|
||||
vector<int> ids;
|
||||
vector<vector<Point2f> > corners, rejected;
|
||||
|
||||
// detect markers and estimate pose
|
||||
detector.detectMarkers(image, corners, ids, rejected);
|
||||
|
||||
size_t nMarkers = corners.size();
|
||||
vector<Vec3d> rvecs(nMarkers), tvecs(nMarkers);
|
||||
|
||||
if(estimatePose && !ids.empty()) {
|
||||
// Calculate pose for each marker
|
||||
for (size_t i = 0; i < nMarkers; i++) {
|
||||
solvePnP(objPoints, corners.at(i), camMatrix, distCoeffs, rvecs.at(i), tvecs.at(i));
|
||||
}
|
||||
}
|
||||
//! [aruco_pose_estimation3]
|
||||
double currentTime = ((double)getTickCount() - tick) / getTickFrequency();
|
||||
totalTime += currentTime;
|
||||
totalIterations++;
|
||||
if(totalIterations % 30 == 0) {
|
||||
cout << "Detection Time = " << currentTime * 1000 << " ms "
|
||||
<< "(Mean = " << 1000 * totalTime / double(totalIterations) << " ms)" << endl;
|
||||
}
|
||||
//! [aruco_draw_pose_estimation]
|
||||
// draw results
|
||||
image.copyTo(imageCopy);
|
||||
if(!ids.empty()) {
|
||||
cv::aruco::drawDetectedMarkers(imageCopy, corners, ids);
|
||||
|
||||
if(estimatePose) {
|
||||
for(unsigned int i = 0; i < ids.size(); i++)
|
||||
cv::drawFrameAxes(imageCopy, camMatrix, distCoeffs, rvecs[i], tvecs[i], markerLength * 1.5f, 2);
|
||||
}
|
||||
}
|
||||
//! [aruco_draw_pose_estimation]
|
||||
|
||||
if(showRejected && !rejected.empty())
|
||||
cv::aruco::drawDetectedMarkers(imageCopy, rejected, noArray(), Scalar(100, 0, 255));
|
||||
|
||||
imshow("out", imageCopy);
|
||||
char key = (char)waitKey(waitTime);
|
||||
if(key == 27) break;
|
||||
}
|
||||
//! [aruco_detect_markers]
|
||||
return 0;
|
||||
}
|
||||
@ -0,0 +1,14 @@
|
||||
%YAML:1.0
|
||||
camera_matrix: !!opencv-matrix
|
||||
rows: 3
|
||||
cols: 3
|
||||
dt: d
|
||||
data: [ 628.158, 0., 324.099,
|
||||
0., 628.156, 260.908,
|
||||
0., 0., 1. ]
|
||||
distortion_coefficients: !!opencv-matrix
|
||||
rows: 5
|
||||
cols: 1
|
||||
dt: d
|
||||
data: [ 0.0995485, -0.206384,
|
||||
0.00754589, 0.00336531, 0 ]
|
||||
38
samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml
Normal file
38
samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml
Normal file
@ -0,0 +1,38 @@
|
||||
%YAML:1.0
|
||||
nmarkers: 35
|
||||
markersize: 6
|
||||
marker_0: "101011111011111001001001101100000000"
|
||||
marker_1: "000000000010011001010011111010111000"
|
||||
marker_2: "011001100000001010000101111101001101"
|
||||
marker_3: "001000111111000111011001110000011111"
|
||||
marker_4: "100110110100101111000000111101110011"
|
||||
marker_5: "010101101110111000111010111100010111"
|
||||
marker_6: "101001000110011110101001010100110100"
|
||||
marker_7: "011010100100110000011101110110100010"
|
||||
marker_8: "111110001000101000110001010010111101"
|
||||
marker_9: "011101101100110111001100100001010100"
|
||||
marker_10: "100001100001010001110001011000000111"
|
||||
marker_11: "110010010010011100101111111000001111"
|
||||
marker_12: "110101001001010110011111010110001101"
|
||||
marker_13: "001111000001000100010001101001010001"
|
||||
marker_14: "000000010010101010111110110011010011"
|
||||
marker_15: "110001110111100101110011111100111010"
|
||||
marker_16: "101011001110001010110011111011001110"
|
||||
marker_17: "101110111101110100101101011001010111"
|
||||
marker_18: "000100111000111101010011010101000101"
|
||||
marker_19: "001110001110001101100101110100000011"
|
||||
marker_20: "100101101100010110110110110001100011"
|
||||
marker_21: "010110001001011010000100111000110110"
|
||||
marker_22: "001000000000100100000000010100010010"
|
||||
marker_23: "101001110010100110000111111010010000"
|
||||
marker_24: "111001101010001100011010010001011100"
|
||||
marker_25: "101000010001010000110100111101101001"
|
||||
marker_26: "101010000001010011001010110110000001"
|
||||
marker_27: "100101001000010101001000111101111110"
|
||||
marker_28: "010010100110010011110001110101011100"
|
||||
marker_29: "011001000101100001101111010001001111"
|
||||
marker_30: "000111011100011110001101111011011001"
|
||||
marker_31: "010100001011000100111101110001101010"
|
||||
marker_32: "100101101001101010111111101101110100"
|
||||
marker_33: "101101001010111000000100110111010101"
|
||||
marker_34: "011111010000111011111110110101100101"
|
||||
Loading…
Reference in New Issue
Block a user