mirror/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2024-11-28 05:06:29 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Small modifications to the scan tutorial and added the video-input-psnr-ssim sample file.

Browse Source
This commit is contained in:
Bernat Gabor 2011-08-09 05:43:58 +00:00
parent 1f07209f6c
commit a5f38806ab
4 changed files with 412 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
doc/tutorials/core/how_to_scan_images/how_to_scan_images.rst
View File

@ -126,16 +126,18 @@ In case of the efficient way making sure that you pass through the right amount
In case of color images we have three uchar items per column. This may be considered a short vector of uchar items, that has been baptized in OpenCV with the *Vec3b* name. To access the n-th sub column we use simple operator[] access. It's important to remember that OpenCV iterators go through the columns and automatically skip to the next row. Therefore in case of color images if you use a simple *uchar* iterator you'll be able to access only the blue channel values.
Random Pixel Access
===================
On-the-fly address calculation with reference returning
=======================================================
The final method isn't recommended for scanning. It was made to acquire or modify random elements in the image. Its basic usage is to specify the row and column number of the item you want to access. During our earlier scanning methods you could already observe that is important through what type we are looking at the image. It's no different here as you need manually to specify what type to use at the automatic lookup. You can observe this in case of the gray scale images for the following source code (the usage of the + :basicstructures:`at() <mat-at>` function):
The final method isn't recommended for scanning. It was made to acquire or modify somehow random elements in the image. Its basic usage is to specify the row and column number of the item you want to access. During our earlier scanning methods you could already observe that is important through what type we are looking at the image. It's no different here as you need manually to specify what type to use at the automatic lookup. You can observe this in case of the gray scale images for the following source code (the usage of the + :basicstructures:`at() <mat-at>` function):
.. literalinclude:: ../../../../samples/cpp/tutorial_code/core/how_to_scan_images/how_to_scan_images.cpp
:language: cpp
:tab-width: 4
:lines: 175-207
The functions takes your input type and coordinates and calculates on the fly the address of the queried item. Then returns a reference to that. This may be a constant when you *get* the value and non-constant when you *set* the value. As a safety step in **debug mode only*** there is performed a check that your input coordinates are valid and does exist. If this isn't the case you'll get a nice output message of this on the standard error output stream. Compared to the efficient way in release mode the only difference in using this is that for every element of the image you'll get a new row pointer for what we use the C operator[] to acquire the column element.
If you need to multiple lookups using this method for an image it may be troublesome and time consuming to enter the type and the at keyword for each of the accesses. To solve this problem OpenCV has a :basicstructures:`Mat_ <id3>` data type. It's the same as Mat with the extra need that at definition you need to specify the data type through what to look at the data matrix, however in return you can use the operator() for fast access of items. To make things even better this is easily convertible from and to the usual :basicstructures:`Mat <id3>` data type. A sample usage of this you can see in case of the color images of the upper function. Nevertheless, it's important to note that the same operation (with the same runtime speed) could have been done with the :basicstructures:`at() <mat-at>` function. It's just a less to write for the lazy programmer trick.
The Core Function
@ -165,12 +167,12 @@ Efficient Way 79.4717 milliseconds
Iterator 83.7201 milliseconds
Random Access 93.7878 milliseconds
On-The-Fly RA 93.7878 milliseconds
LUT function 32.5759 milliseconds
============= ====================
We can conclude a couple of things. If possible, use the already made functions of OpenCV (instead reinventing these). The fastest method turns out to be the LUT function. This is because the OpenCV library is multi-thread enabled via Intel Threaded Building Blocks. However, if you need to write a simple image scan prefer the pointer method. The iterator is a safer bet, however quite slower. Using the random access method for full image scan is the most costly. Your compiler might observe what you want to do and optimize it out somewhat, however in general case this approach is with a little slower than the iterator method.
We can conclude a couple of things. If possible, use the already made functions of OpenCV (instead reinventing these). The fastest method turns out to be the LUT function. This is because the OpenCV library is multi-thread enabled via Intel Threaded Building Blocks. However, if you need to write a simple image scan prefer the pointer method. The iterator is a safer bet, however quite slower. Using the on-the-fly reference access method for full image scan is the most costly in debug mode. In the release mode it may beat the iterator approach or not, however it surely sacrifices for this the safety trait of iterators.
Finally, you may watch a sample run of the program on the `video posted <https://www.youtube.com/watch?v=fB3AN5fjgwc>`_ on our YouTube channel.

206
samples/cpp/tutorial_code/HighGUI/video-input-psnr-sssim/video-input-psnr-sssim.cpp Normal file
View File

@ -0,0 +1,206 @@
// Video Image PSNR and SSIM
#include <iostream> // for standard I/O
#include <string> // for strings
#include <iomanip> // for controlling float print precision
#include <sstream> // string to number conversion
#include <opencv2/imgproc/imgproc.hpp> // Gaussian Blur
#include <opencv2/core/core.hpp> // Basic OpenCV structures (cv::Mat, Scalar)
#include <opencv2/highgui/highgui.hpp> // OpenCV window I/O
using namespace std;
using namespace cv;
double getPSNR ( const Mat& I1, const Mat& I2);
Scalar getMSSIM( const Mat& I1, const Mat& I2);
void help()
{
cout
<< "\n--------------------------------------------------------------------------" << endl
<< "This program shows how to read a video file with OpenCV. In addition, it tests the"
<< " similarity of two input videos first with PSNR, and for the frames below a PSNR " << endl
<< "trigger value, also with MSSIM."<< endl
<< "Usage:" << endl
<< "./video-source referenceVideo useCaseTestVideo PSNR_Trigger_Value Wait_Between_Frames " << endl
<< "--------------------------------------------------------------------------" << endl
<< endl;
}
int main(int argc, char *argv[], char *window_name)
{
help();
if (argc != 5)
{
cout << "Not enough parameters" << endl;
return -1;
}
stringstream conv;
const string sourceReference = argv[1],sourceCompareWith = argv[2];
int psnrTriggerValue, delay;
conv << argv[3] << argv[4]; // put in the strings
conv >> psnrTriggerValue >> delay;// take out the numbers
char c;
int frameNum = -1; // Frame counter
VideoCapture captRefrnc(sourceReference),
captUndTst(sourceCompareWith);
if ( !captRefrnc.isOpened())
{
cout << "Could not open reference " << sourceReference << endl;
return -1;
}
if( !captUndTst.isOpened())
{
cout << "Could not open case test " << sourceCompareWith << endl;
return -1;
}
Size refS = Size((int) captRefrnc.get(CV_CAP_PROP_FRAME_WIDTH),
(int) captRefrnc.get(CV_CAP_PROP_FRAME_HEIGHT)),
uTSi = Size((int) captUndTst.get(CV_CAP_PROP_FRAME_WIDTH),
(int) captUndTst.get(CV_CAP_PROP_FRAME_HEIGHT));
if (refS != uTSi)
{
cout << "Inputs have different size!!! Closing." << endl;
return -1;
}
const char* WIN_UT = "Under Test";
const char* WIN_RF = "Reference";
// Windows
namedWindow(WIN_RF, CV_WINDOW_AUTOSIZE );
namedWindow(WIN_UT, CV_WINDOW_AUTOSIZE );
cvMoveWindow(WIN_RF, 400 , 0); //750, 2 (bernat =0)
cvMoveWindow(WIN_UT, refS.width, 0); //1500, 2
cout << "Frame resolution: Width=" << refS.width << " Height=" << refS.height
<< " of nr#: " << captRefrnc.get(CV_CAP_PROP_FRAME_COUNT) << endl;
cout << "PSNR trigger value " <<
setiosflags(ios::fixed) << setprecision(3) << psnrTriggerValue << endl;
Mat frameReference, frameUnderTest;
double psnrV;
Scalar mssimV;
while( true) //Show the image captured in the window and repeat
{
captRefrnc >> frameReference;
captUndTst >> frameUnderTest;
if( frameReference.empty() || frameUnderTest.empty())
{
cout << " < < < Game over! > > > ";
break;
}
++frameNum;
cout <<"Frame:" << frameNum;
///////////////////////////////// PSNR ////////////////////////////////////////////////////
psnrV = getPSNR(frameReference,frameUnderTest); //get PSNR
cout << setiosflags(ios::fixed) << setprecision(3) << psnrV << "dB";
//////////////////////////////////// MSSIM /////////////////////////////////////////////////
if (psnrV < psnrTriggerValue)
{
mssimV = getMSSIM(frameReference,frameUnderTest);
cout << " MSSIM: "
<< "R" << setiosflags(ios::fixed) << setprecision(3) << mssimV.val[2] * 100
<< "G" << setiosflags(ios::fixed) << setprecision(3) << mssimV.val[1] * 100
<< "B" << setiosflags(ios::fixed) << setprecision(3) << mssimV.val[0] * 100;
}
cout << endl;
////////////////////////////////// Show Image /////////////////////////////////////////////
imshow( WIN_RF, frameReference);
imshow( WIN_UT, frameUnderTest);
c = cvWaitKey(delay);
if (c == 27) break;
}
return 0;
}
double getPSNR(const Mat& I1, const Mat& I2)
{
Mat s1;
absdiff(I1, I2, s1); // |I1 - I2|
s1.convertTo(s1, CV_32F); // cannot make a square on 8 bits
s1 = s1.mul(s1); // |I1 - I2|^2
Scalar s = sum(s1); // sum elements per channel
double sse = s.val[0] + s.val[1] + s.val[2]; // sum channels
if( sse <= 1e-10) // for small values return zero
return 0;
else
{
double mse =sse /(double)(I1.channels() * I1.total());
double psnr = 10.0*log10((255*255)/mse);
return psnr;
}
}
Scalar getMSSIM( const Mat& i1, const Mat& i2)
{
const double C1 = 6.5025, C2 = 58.5225;
/***************************** INITS **********************************/
int d = CV_32F;
Mat I1, I2;
i1.convertTo(I1, d); // cannot calculate on one byte large values
i2.convertTo(I2, d);
Mat I2_2 = I2.mul(I2); // I2^2
Mat I1_2 = I1.mul(I1); // I1^2
Mat I1_I2 = I1.mul(I2); // I1 * I2
/*************************** END INITS **********************************/
Mat mu1, mu2; // PRELIMINARY COMPUTING
GaussianBlur(I1, mu1, Size(11, 11), 1.5);
GaussianBlur(I2, mu2, Size(11, 11), 1.5);
Mat mu1_2 = mu1.mul(mu1);
Mat mu2_2 = mu2.mul(mu2);
Mat mu1_mu2 = mu1.mul(mu2);
Mat sigma1_2, sigma2_2, sigma12;
GaussianBlur(I1_2, sigma1_2, Size(11, 11), 1.5);
sigma1_2 -= mu1_2;
GaussianBlur(I2_2, sigma2_2, Size(11, 11), 1.5);
sigma2_2 -= mu2_2;
GaussianBlur(I1_I2, sigma12, Size(11, 11), 1.5);
sigma12 -= mu1_mu2;
///////////////////////////////// FORMULA ////////////////////////////////
Mat t1, t2, t3;
t1 = 2 * mu1_mu2 + C1;
t2 = 2 * sigma12 + C2;
t3 = t1.mul(t2); // t3 = ((2*mu1_mu2 + C1).*(2*sigma12 + C2))
t1 = mu1_2 + mu2_2 + C1;
t2 = sigma1_2 + sigma2_2 + C2;
t1 = t1.mul(t2); // t1 =((mu1_2 + mu2_2 + C1).*(sigma1_2 + sigma2_2 + C2))
Mat ssim_map;
divide(t3, t1, ssim_map); // ssim_map = t3./t1;
Scalar mssim = mean( ssim_map ); // mssim = average of ssim map
return mssim;
}

4
samples/cpp/tutorial_code/core/how_to_scan_images/how_to_scan_images.cpp
View File

@ -12,7 +12,7 @@ void help()
<< "\n--------------------------------------------------------------------------" << endl
<< "This program shows how to scan image objects in OpenCV (cv::Mat). As use case"
<< " we take an input image and divide the native color palette (255) with the " << endl
<< "input. Shows C operator[] method, iterators and at function for random access"<< endl
<< "input. Shows C operator[] method, iterators and at function for on-the-fly item address calculation."<< endl
<< "Usage:" << endl
<< "./howToScanImages imageNameToUse divideWith [G]" << endl
<< "if you add a G parameter the image is processed in gray scale" << endl
@ -92,7 +92,7 @@ int main( int argc, char* argv[])
t = 1000*((double)getTickCount() - t)/getTickFrequency();
t /= times;
cout << "Time of reducing with the random access operator (averaged for "
cout << "Time of reducing with the on-the-fly address generation - at function (averaged for "
<< times << " runs): " << t << " milliseconds."<< endl;
Mat lookUpTable(1, 256, CV_8U);

212
samples/cpp/tutorial_code/introduction/windows_visual_studio_Opencv/Test.cpp
View File

@ -1,24 +1,206 @@
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
// Video Image PSNR and SSIM
#include <iostream> // for standard I/O
#include <string> // for strings
#include <iomanip> // for controlling float print precision
#include <sstream> // string to number conversion
#include <opencv2/imgproc/imgproc.hpp> // Gaussian Blur
#include <opencv2/core/core.hpp> // Basic OpenCV structures (cv::Mat, Scalar)
#include <opencv2/highgui/highgui.hpp> // OpenCV window I/O
using namespace std;
using namespace cv;
int main( int argc, char** argv )
double getPSNR ( const Mat& I1, const Mat& I2);
Scalar getMSSIM( const Mat& I1, const Mat& I2);
void help()
{
Mat image;
image = imread("opencv-logo.png"); // Read the file
cout
<< "\n--------------------------------------------------------------------------" << endl
<< "This program shows how to read a video file with OpenCV. In addition, it tests the"
<< " similarity of two input videos first with PSNR, and for the frames below a PSNR " << endl
<< "trigger value, also with MSSIM."<< endl
<< "Usage:" << endl
<< "./video-source referenceVideo useCaseTestVideo PSNR_Trigger_Value Wait_Between_Frames " << endl
<< "--------------------------------------------------------------------------" << endl
<< endl;
}
int main(int argc, char *argv[], char *window_name)
{
help();
if (argc != 5)
{
cout << "Not enough parameters" << endl;
return -1;
}
stringstream conv;
if(! image.data ) // Check for invalid input
{
std::cout << "Could not open or find the image" << std::endl ;
return -1;
}
const string sourceReference = argv[1],sourceCompareWith = argv[2];
int psnrTriggerValue, delay;
conv << argv[3] << argv[4]; // put in the strings
conv >> psnrTriggerValue >> delay;// take out the numbers
namedWindow( "Display window", CV_WINDOW_FREERATIO );// Create a window for display.
imshow( "Display window", image ); // Show our image inside it.
char c;
int frameNum = -1; // Frame counter
waitKey(0); // Wait for a keystroke in the window
VideoCapture captRefrnc(sourceReference),
captUndTst(sourceCompareWith);
return 0;
if ( !captRefrnc.isOpened())
{
cout << "Could not open reference " << sourceReference << endl;
return -1;
}
if( !captUndTst.isOpened())
{
cout << "Could not open case test " << sourceCompareWith << endl;
return -1;
}
Size refS = Size((int) captRefrnc.get(CV_CAP_PROP_FRAME_WIDTH),
(int) captRefrnc.get(CV_CAP_PROP_FRAME_HEIGHT)),
uTSi = Size((int) captUndTst.get(CV_CAP_PROP_FRAME_WIDTH),
(int) captUndTst.get(CV_CAP_PROP_FRAME_HEIGHT));
if (refS != uTSi)
{
cout << "Inputs have different size!!! Closing." << endl;
return -1;
}
const char* WIN_UT = "Under Test";
const char* WIN_RF = "Reference";
// Windows
namedWindow(WIN_RF, CV_WINDOW_AUTOSIZE );
namedWindow(WIN_UT, CV_WINDOW_AUTOSIZE );
cvMoveWindow(WIN_RF, 400 , 0); //750, 2 (bernat =0)
cvMoveWindow(WIN_UT, refS.width, 0); //1500, 2
cout << "Frame resolution: Width=" << refS.width << " Height=" << refS.height
<< " of nr#: " << captRefrnc.get(CV_CAP_PROP_FRAME_COUNT) << endl;
cout << "PSNR trigger value " <<
setiosflags(ios::fixed) << setprecision(3) << psnrTriggerValue << endl;
Mat frameReference, frameUnderTest;
double psnrV;
Scalar mssimV;
while( true) //Show the image captured in the window and repeat
{
captRefrnc >> frameReference;
captUndTst >> frameUnderTest;
if( frameReference.empty() || frameUnderTest.empty())
{
cout << " < < < Game over! > > > ";
break;
}
++frameNum;
cout <<"Frame:" << frameNum;
///////////////////////////////// PSNR ////////////////////////////////////////////////////
psnrV = getPSNR(frameReference,frameUnderTest); //get PSNR
cout << setiosflags(ios::fixed) << setprecision(3) << psnrV << "dB";
//////////////////////////////////// MSSIM /////////////////////////////////////////////////
if (psnrV < psnrTriggerValue)
{
mssimV = getMSSIM(frameReference,frameUnderTest);
cout << " MSSIM: "
<< "R" << setiosflags(ios::fixed) << setprecision(3) << mssimV.val[2] * 100
<< "G" << setiosflags(ios::fixed) << setprecision(3) << mssimV.val[1] * 100
<< "B" << setiosflags(ios::fixed) << setprecision(3) << mssimV.val[0] * 100;
}
cout << endl;
////////////////////////////////// Show Image /////////////////////////////////////////////
imshow( WIN_RF, frameReference);
imshow( WIN_UT, frameUnderTest);
c = cvWaitKey(delay);
if (c == 27) break;
}
return 0;
}
double getPSNR(const Mat& I1, const Mat& I2)
{
Mat s1;
absdiff(I1, I2, s1); // |I1 - I2|
s1.convertTo(s1, CV_32F); // cannot make a square on 8 bits
s1 = s1.mul(s1); // |I1 - I2|^2
Scalar s = sum(s1); // sum elements per channel
double sse = s.val[0] + s.val[1] + s.val[2]; // sum channels
if( sse <= 1e-10) // for small values return zero
return 0;
else
{
double mse =sse /(double)(I1.channels() * I1.total());
double psnr = 10.0*log10((255*255)/mse);
return psnr;
}
}
Scalar getMSSIM( const Mat& i1, const Mat& i2)
{
const double C1 = 6.5025, C2 = 58.5225;
/***************************** INITS **********************************/
int d = CV_32F;
Mat I1, I2;
i1.convertTo(I1, d); // cannot calculate on one byte large values
i2.convertTo(I2, d);
Mat I2_2 = I2.mul(I2); // I2^2
Mat I1_2 = I1.mul(I1); // I1^2
Mat I1_I2 = I1.mul(I2); // I1 * I2
/*************************** END INITS **********************************/
Mat mu1, mu2; // PRELIMINARY COMPUTING
GaussianBlur(I1, mu1, Size(11, 11), 1.5);
GaussianBlur(I2, mu2, Size(11, 11), 1.5);
Mat mu1_2 = mu1.mul(mu1);
Mat mu2_2 = mu2.mul(mu2);
Mat mu1_mu2 = mu1.mul(mu2);
Mat sigma1_2, sigma2_2, sigma12;
GaussianBlur(I1_2, sigma1_2, Size(11, 11), 1.5);
sigma1_2 -= mu1_2;
GaussianBlur(I2_2, sigma2_2, Size(11, 11), 1.5);
sigma2_2 -= mu2_2;
GaussianBlur(I1_I2, sigma12, Size(11, 11), 1.5);
sigma12 -= mu1_mu2;
///////////////////////////////// FORMULA ////////////////////////////////
Mat t1, t2, t3;
t1 = 2 * mu1_mu2 + C1;
t2 = 2 * sigma12 + C2;
t3 = t1.mul(t2); // t3 = ((2*mu1_mu2 + C1).*(2*sigma12 + C2))
t1 = mu1_2 + mu2_2 + C1;
t2 = sigma1_2 + sigma2_2 + C2;
t1 = t1.mul(t2); // t1 =((mu1_2 + mu2_2 + C1).*(sigma1_2 + sigma2_2 + C2))
Mat ssim_map;
divide(t3, t1, ssim_map); // ssim_map = t3./t1;
Scalar mssim = mean( ssim_map ); // mssim = average of ssim map
return mssim;
}