mirror/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2025-06-07 17:44:04 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #11707 from kdpatters:patch-1

Browse Source
This commit is contained in:
Alexander Alekhin 2018-06-13 15:51:10 +00:00
commit 54e8487f56
1 changed files with 45 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

94
doc/py_tutorials/py_calib3d/py_calibration/py_calibration.markdown
View File

@ -4,32 +4,34 @@ Camera Calibration {#tutorial_py_calibration}
Goal Goal
---- ----
In this section, In this section, we will learn about
- We will learn about distortions in camera, intrinsic and extrinsic parameters of camera etc.
- We will learn to find these parameters, undistort images etc. * types of distortion caused by cameras
* how to find the intrinsic and extrinsic properties of a camera
* how to undistort images based off these properties
Basics Basics
------ ------
Today's cheap pinhole cameras introduces a lot of distortion to images. Two major distortions are Some pinhole cameras introduce significant distortion to images. Two major kinds of distortion are
radial distortion and tangential distortion. radial distortion and tangential distortion.
Due to radial distortion, straight lines will appear curved. Its effect is more as we move away from Radial distortion causes straight lines to appear curved. Radial distortion becomes larger the farther points are from
the center of image. For example, one image is shown below, where two edges of a chess board are the center of the image. For example, one image is shown below in which two edges of a chess board are
marked with red lines. But you can see that border is not a straight line and doesn't match with the marked with red lines. But, you can see that the border of the chess board is not a straight line and doesn't match with the
red line. All the expected straight lines are bulged out. Visit [Distortion red line. All the expected straight lines are bulged out. Visit [Distortion
(optics)](http://en.wikipedia.org/wiki/Distortion_%28optics%29) for more details. (optics)](http://en.wikipedia.org/wiki/Distortion_%28optics%29) for more details.
![image](images/calib_radial.jpg) ![image](images/calib_radial.jpg)
This distortion is represented as follows: Radial distortion can be represented as follows:
\f[x_{distorted} = x( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6) \\ \f[x_{distorted} = x( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6) \\
y_{distorted} = y( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6)\f] y_{distorted} = y( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6)\f]
Similarly, another distortion is the tangential distortion which occurs because image taking lense Similarly, tangential distortion occurs because the image-taking lense
is not aligned perfectly parallel to the imaging plane. So some areas in image may look nearer than is not aligned perfectly parallel to the imaging plane. So, some areas in the image may look nearer than
expected. It is represented as below: expected. The amount of tangential distortion can be represented as below:
\f[x_{distorted} = x + [ 2p_1xy + p_2(r^2+2x^2)] \\ \f[x_{distorted} = x + [ 2p_1xy + p_2(r^2+2x^2)] \\
y_{distorted} = y + [ p_1(r^2+ 2y^2)+ 2p_2xy]\f] y_{distorted} = y + [ p_1(r^2+ 2y^2)+ 2p_2xy]\f]
@ -38,10 +40,9 @@ In short, we need to find five parameters, known as distortion coefficients give
\f[Distortion \; coefficients=(k_1 \hspace{10pt} k_2 \hspace{10pt} p_1 \hspace{10pt} p_2 \hspace{10pt} k_3)\f] \f[Distortion \; coefficients=(k_1 \hspace{10pt} k_2 \hspace{10pt} p_1 \hspace{10pt} p_2 \hspace{10pt} k_3)\f]
In addition to this, we need to find a few more information, like intrinsic and extrinsic parameters In addition to this, we need to some other information, like the intrinsic and extrinsic parameters
of a camera. Intrinsic parameters are specific to a camera. It includes information like focal of the camera. Intrinsic parameters are specific to a camera. They include information like focal
length (\f$f_x,f_y\f$), optical centers (\f$c_x, c_y\f$) etc. It is also called camera matrix. It depends on length (\f$f_x,f_y\f$) and optical centers (\f$c_x, c_y\f$). The focal length and optical centers can be used to create a camera matrix, which can be used to remove distortion due to the lenses of a specific camera. The camera matrix is unique to a specific camera, so once calculated, it can be reused on other images taken by the same camera. It is expressed as a 3x3
the camera only, so once calculated, it can be stored for future purposes. It is expressed as a 3x3
matrix: matrix:
\f[camera \; matrix = \left [ \begin{matrix} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{matrix} \right ]\f] \f[camera \; matrix = \left [ \begin{matrix} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{matrix} \right ]\f]
@ -49,20 +50,16 @@ matrix:
Extrinsic parameters corresponds to rotation and translation vectors which translates a coordinates Extrinsic parameters corresponds to rotation and translation vectors which translates a coordinates
of a 3D point to a coordinate system. of a 3D point to a coordinate system.
For stereo applications, these distortions need to be corrected first. To find all these parameters, For stereo applications, these distortions need to be corrected first. To find these parameters,
what we have to do is to provide some sample images of a well defined pattern (eg, chess board). We we must provide some sample images of a well defined pattern (e.g. a chess board). We
find some specific points in it ( square corners in chess board). We know its coordinates in real find some specific points of which we already know the relative positions (e.g. square corners in the chess board). We know the coordinates of these points in real world space and we know the coordinates in the image, so we can solve for the distortion coefficients. For better results, we need at least 10 test patterns.
world space and we know its coordinates in image. With these data, some mathematical problem is
solved in background to get the distortion coefficients. That is the summary of the whole story. For
better results, we need atleast 10 test patterns.
Code Code
---- ----
As mentioned above, we need atleast 10 test patterns for camera calibration. OpenCV comes with some As mentioned above, we need at least 10 test patterns for camera calibration. OpenCV comes with some
images of chess board (see samples/cpp/left01.jpg -- left14.jpg), so we will utilize it. For sake of images of a chess board (see samples/data/left01.jpg -- left14.jpg), so we will utilize these. Consider an image of a chess board. The important input data needed for calibration of the camera
understanding, consider just one image of a chess board. Important input datas needed for camera is the set of 3D real world points and the corresponding 2D coordinates of these points in the image. 2D image points
calibration is a set of 3D real world points and its corresponding 2D image points. 2D image points
are OK which we can easily find from the image. (These image points are locations where two black are OK which we can easily find from the image. (These image points are locations where two black
squares touch each other in chess boards) squares touch each other in chess boards)
@ -72,7 +69,7 @@ values. But for simplicity, we can say chess board was kept stationary at XY pla
and camera was moved accordingly. This consideration helps us to find only X,Y values. Now for X,Y and camera was moved accordingly. This consideration helps us to find only X,Y values. Now for X,Y
values, we can simply pass the points as (0,0), (1,0), (2,0), ... which denotes the location of values, we can simply pass the points as (0,0), (1,0), (2,0), ... which denotes the location of
points. In this case, the results we get will be in the scale of size of chess board square. But if points. In this case, the results we get will be in the scale of size of chess board square. But if
we know the square size, (say 30 mm), and we can pass the values as (0,0),(30,0),(60,0),..., we get we know the square size, (say 30 mm), we can pass the values as (0,0), (30,0), (60,0), ... . Thus, we get
the results in mm. (In this case, we don't know square size since we didn't take those images, so we the results in mm. (In this case, we don't know square size since we didn't take those images, so we
pass in terms of square size). pass in terms of square size).
@ -80,23 +77,22 @@ pass in terms of square size).
### Setup ### Setup
So to find pattern in chess board, we use the function, **cv.findChessboardCorners()**. We also So to find pattern in chess board, we can use the function, **cv.findChessboardCorners()**. We also
need to pass what kind of pattern we are looking, like 8x8 grid, 5x5 grid etc. In this example, we need to pass what kind of pattern we are looking for, like 8x8 grid, 5x5 grid etc. In this example, we
use 7x6 grid. (Normally a chess board has 8x8 squares and 7x7 internal corners). It returns the use 7x6 grid. (Normally a chess board has 8x8 squares and 7x7 internal corners). It returns the
corner points and retval which will be True if pattern is obtained. These corners will be placed in corner points and retval which will be True if pattern is obtained. These corners will be placed in
an order (from left-to-right, top-to-bottom) an order (from left-to-right, top-to-bottom)
@sa This function may not be able to find the required pattern in all the images. So one good option @sa This function may not be able to find the required pattern in all the images. So, one good option
is to write the code such that, it starts the camera and check each frame for required pattern. Once is to write the code such that, it starts the camera and check each frame for required pattern. Once
pattern is obtained, find the corners and store it in a list. Also provides some interval before the pattern is obtained, find the corners and store it in a list. Also, provide some interval before
reading next frame so that we can adjust our chess board in different direction. Continue this reading next frame so that we can adjust our chess board in different direction. Continue this
process until required number of good patterns are obtained. Even in the example provided here, we process until the required number of good patterns are obtained. Even in the example provided here, we
are not sure out of 14 images given, how many are good. So we read all the images and take the good are not sure how many images out of the 14 given are good. Thus, we must read all the images and take only the good
ones. ones.
@sa Instead of chess board, we can use some circular grid, but then use the function @sa Instead of chess board, we can alternatively use a circular grid. In this case, we must use the function
**cv.findCirclesGrid()** to find the pattern. It is said that less number of images are enough when **cv.findCirclesGrid()** to find the pattern. Fewer images are sufficient to perform camera calibration using a circular grid.
using circular grid.
Once we find the corners, we can increase their accuracy using **cv.cornerSubPix()**. We can also Once we find the corners, we can increase their accuracy using **cv.cornerSubPix()**. We can also
draw the pattern using **cv.drawChessboardCorners()**. All these steps are included in below code: draw the pattern using **cv.drawChessboardCorners()**. All these steps are included in below code:
@ -146,22 +142,23 @@ One image with pattern drawn on it is shown below:
### Calibration ### Calibration
So now we have our object points and image points we are ready to go for calibration. For that we Now that we have our object points and image points, we are ready to go for calibration. We can
use the function, **cv.calibrateCamera()**. It returns the camera matrix, distortion coefficients, use the function, **cv.calibrateCamera()** which returns the camera matrix, distortion coefficients,
rotation and translation vectors etc. rotation and translation vectors etc.
@code{.py} @code{.py}
ret, mtx, dist, rvecs, tvecs = cv.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None) ret, mtx, dist, rvecs, tvecs = cv.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)
@endcode @endcode
### Undistortion ### Undistortion
We have got what we were trying. Now we can take an image and undistort it. OpenCV comes with two Now, we can take an image and undistort it. OpenCV comes with two
methods, we will see both. But before that, we can refine the camera matrix based on a free scaling methods for doing this. However first, we can refine the camera matrix based on a free scaling
parameter using **cv.getOptimalNewCameraMatrix()**. If the scaling parameter alpha=0, it returns parameter using **cv.getOptimalNewCameraMatrix()**. If the scaling parameter alpha=0, it returns
undistorted image with minimum unwanted pixels. So it may even remove some pixels at image corners. undistorted image with minimum unwanted pixels. So it may even remove some pixels at image corners.
If alpha=1, all pixels are retained with some extra black images. It also returns an image ROI which If alpha=1, all pixels are retained with some extra black images. This function also returns an image ROI which
can be used to crop the result. can be used to crop the result.
So we take a new image (left12.jpg in this case. That is the first image in this chapter) So, we take a new image (left12.jpg in this case. That is the first image in this chapter)
@code{.py} @code{.py}
img = cv.imread('left12.jpg') img = cv.imread('left12.jpg')
h, w = img.shape[:2] h, w = img.shape[:2]
@ -169,7 +166,7 @@ newcameramtx, roi = cv.getOptimalNewCameraMatrix(mtx, dist, (w,h), 1, (w,h))
@endcode @endcode
#### 1. Using **cv.undistort()** #### 1. Using **cv.undistort()**
This is the shortest path. Just call the function and use ROI obtained above to crop the result. This is the easiest way. Just call the function and use ROI obtained above to crop the result.
@code{.py} @code{.py}
# undistort # undistort
dst = cv.undistort(img, mtx, dist, None, newcameramtx) dst = cv.undistort(img, mtx, dist, None, newcameramtx)
@ -181,7 +178,7 @@ cv.imwrite('calibresult.png', dst)
@endcode @endcode
#### 2. Using **remapping** #### 2. Using **remapping**
This is curved path. First find a mapping function from distorted image to undistorted image. Then This way is a little bit more difficult. First, find a mapping function from the distorted image to the undistorted image. Then
use the remap function. use the remap function.
@code{.py} @code{.py}
# undistort # undistort
@ -193,23 +190,22 @@ x, y, w, h = roi
dst = dst[y:y+h, x:x+w] dst = dst[y:y+h, x:x+w]
cv.imwrite('calibresult.png', dst) cv.imwrite('calibresult.png', dst)
@endcode @endcode
Both the methods give the same result. See the result below: Still, both the methods give the same result. See the result below:
![image](images/calib_result.jpg) ![image](images/calib_result.jpg)
You can see in the result that all the edges are straight. You can see in the result that all the edges are straight.
Now you can store the camera matrix and distortion coefficients using write functions in Numpy Now you can store the camera matrix and distortion coefficients using write functions in NumPy
(np.savez, np.savetxt etc) for future uses. (np.savez, np.savetxt etc) for future uses.
Re-projection Error Re-projection Error
------------------- -------------------
Re-projection error gives a good estimation of just how exact is the found parameters. This should Re-projection error gives a good estimation of just how exact the found parameters are. The closer the re-projection error is to zero, the more accurate the parameters we found are. Given the intrinsic, distortion, rotation and translation matrices,
be as close to zero as possible. Given the intrinsic, distortion, rotation and translation matrices, we must first transform the object point to image point using **cv.projectPoints()**. Then, we can calculate
we first transform the object point to image point using **cv.projectPoints()**. Then we calculate
the absolute norm between what we got with our transformation and the corner finding algorithm. To the absolute norm between what we got with our transformation and the corner finding algorithm. To
find the average error we calculate the arithmetical mean of the errors calculate for all the find the average error, we calculate the arithmetical mean of the errors calculated for all the
calibration images. calibration images.
@code{.py} @code{.py}
mean_error = 0 mean_error = 0