Commit Graph

158 Commits

Author SHA1 Message Date
mshabunin
f5bf1e510b Test run script: added long test filter for valgrind 2016-12-14 17:16:08 +03:00
Alexander Alekhin
c57f145e90 ts: update run.py
Add information about python bindings
2016-10-20 16:54:51 +03:00
Maksim Shabunin
c3cf1be344 fixup! run.py: issues with forced configuration fixed 2015-11-19 15:32:42 +03:00
Maksim Shabunin
7df392bfd8 run.py: issues with forced configuration fixed 2015-11-16 14:02:26 +03:00
Maksim Shabunin
97878645bc Fix run.py test detection issues for debug VS configurations 2015-11-13 18:55:19 +03:00
Maksim Shabunin
90c74e3be0 ts: refactor run.py script 2015-10-29 11:28:12 +03:00
Evgeny Agafonchikov
ae090fe10d Functionality for relocation of tests
This allows to start run.py with --move_tests parameter to place tests

to another location and keep them runnable from there
2015-04-01 11:50:05 +03:00
Evgeny Agafonchikov
1f04f9d63b Fixing help messages
--help message and starting run.py w/o parameters generate different messages

E. g. w/ and w/o build_path in the end

build_path is required, removing square brackets
2015-03-31 17:58:25 +03:00
Alexander Alekhin
e6f6905868 run.py: propagate OPENCV* env variables only with --android_propagate_opencv_env flag 2014-05-26 14:32:52 +04:00
Alexander Alekhin
766600529b run.py: added --android_env parameter 2014-05-26 13:27:31 +04:00
Alexander Alekhin
eeed9ce247 perf report: summary.py: added custom --regressions map 2014-03-31 16:04:18 +04:00
Vladimir Kolesnikov
60803afe8f Misprint fixed in table formatter 2014-03-24 11:04:49 +04:00
Roman Donchenko
a600267b9f Merge pull request #2432 from ilya-lavrenov:summary_py 2014-03-04 11:46:29 +04:00
Ilya Lavrenov
29deff8707 ability to merge logs with intersections only 2014-03-03 13:27:07 +04:00
Roman Donchenko
af8aa8a2fa Merge pull request #2433 from ilya-lavrenov:typos 2014-03-03 12:20:35 +04:00
Ilya Lavrenov
eedf86402d typos 2014-03-02 21:04:17 +04:00
Evgeniy
8236181c62 Added more colorizing options to XLS generating script 2014-02-17 17:01:23 +04:00
Roman Donchenko
4c5593b633 Merge pull request #1837 from asmorkalov:android_opencl 2013-11-25 12:19:20 +04:00
Alexander Smorkalov
514e97223c Some hacks to build and run OpenCL on Qualcomm S800.
Some of functions were enabled on Qualcomm S800 by changing grid size;
OpenCL kernel grid size unification for different platfroms;
Test pass rate improvements by inclreasing threshold;
Some tests were disabled for Android;
run.py was adopted for devices with brackets in in name.
2013-11-22 16:13:38 +04:00
Alexander Alekhin
c1541c9e79 perf report: summary.py: added --progress mode 2013-11-20 19:57:20 +04:00
Roman Donchenko
5bb6949bd6 Fix running Java tests with run.py on everything other than Windows.
Previously, run.py would assume that the opencv_java library is in the
same directory as the tests, which is only true on Windows.

The library path depends on the build configuration, which may not be
known until the actual build (e.g. with the Visual Studio generators),
so it can't be stored in the CMake cache for run.py to read. I didn't
want to hardcode into run.py where the library is on each platform,
either. So that's why I used the current scheme with the properties
file. It also makes running the tests without run.py a little easier.
2013-10-16 19:22:01 +04:00
Alexander Alekhin
3eba2011b9 test: run.py: return valid errorCode in case of the single test run 2013-10-10 17:17:10 +04:00
Roman Donchenko
e9a28f66ee Normalized file endings. 2013-08-21 18:59:25 +04:00
Roman Donchenko
e23f31311d xls-report.py: separated image width and height for easier analysis. 2013-08-07 13:43:03 +04:00
Roman Donchenko
1432f3a82c Fix xls-report.py not working with more than one comparison. 2013-07-22 12:48:43 +04:00
Roman Donchenko
ea3239a00e xls-report.py: Added an option to show per-pixel times 2013-07-10 14:58:51 +04:00
Roman Donchenko
5b2dc26f2c Made the crude ability less crude. 2013-07-10 14:58:51 +04:00
Roman Donchenko
215b3e749f Added to the test log parser a crude ability to detect non-implemented tests. 2013-07-10 14:58:51 +04:00
Roman Donchenko
1080c4295a xls-report.py: removed image size and type from the list of other parameters.
Also, shrunk the corresponding column and the test name column.
2013-07-10 14:58:18 +04:00
Roman Donchenko
63d151a29c xls-report.py: add ability to specify arbitrary sheet properties 2013-07-10 14:58:17 +04:00
Roman Donchenko
2b44172026 xls-report.py: make behaviour more intelligent in case of repeated results 2013-07-04 20:01:27 +04:00
Roman Donchenko
0732905cff xls-report.py: write tests in sorted order 2013-07-04 20:01:27 +04:00
Roman Donchenko
ea054e2888 xls-report.py: made most columns a little wider 2013-07-04 20:01:27 +04:00
Roman Donchenko
4e939e2fce xls-report.py: added an option to include data from unrecognized XMLs 2013-07-04 20:00:59 +04:00
Roman Donchenko
83fd385bbd xls-report.py: don't eat all errors when reading sheet.conf 2013-07-04 18:03:03 +04:00
Roman Donchenko
d4a8b87645 Wrote relevant docs. 2013-06-21 16:45:17 +04:00
Roman Donchenko
0e3a9eaf98 Made Collector render property sets as dicts instead of tuples of pairs. 2013-06-21 13:43:16 +04:00
Roman Donchenko
2688e22cb5 Made xls-report.py use global properties in XML files.
Now it can determine, without looking at the file name, both the module
name and the configuration name (the latter with a little help from the
configuration file).
2013-06-20 19:57:45 +04:00
Roman Donchenko
57317c3196 Use log formatting as intended. 2013-06-20 19:39:02 +04:00
Roman Donchenko
3e2c456313 A few minor improvements to the XLS report generator.
* In comparison column headers, switched the order of labels, so that
  it's "to" vs "from".

* When a test was present, but not run successfully, put its status in
  the corresponding cell instead of coloring it gray.
2013-06-20 14:38:01 +04:00
Roman Donchenko
584f0745d0 Made xls-report.py ignore tests that were not successful. 2013-06-18 13:41:41 +04:00
Roman Donchenko
0f1156bbb6 Made the order of tests in XLS reports deterministic. 2013-06-18 13:37:40 +04:00
Roman Donchenko
4d7b1b5ede In the XLS report, enabled word wrapping for header cells.
Otherwise, Excel ignores line breaks in them.
2013-06-18 12:31:43 +04:00
Roman Donchenko
6ff207b53a Added a new and improved version of the XLS report generator. 2013-06-18 12:31:43 +04:00
Roman Donchenko
f003e29dc0 Updated testlog_parser.py to the latest version from the private repo. 2013-06-18 12:31:43 +04:00
Andrey Pavlenko
4af104aaae 'aapt' tool moved in SDK r22, adding support to run.py 2013-05-19 14:57:21 +04:00
Roman Donchenko
57d4c86b2b Fixed the shebang lines on the Python scripts.
Also, removed the one from modules/python/src2/cv.py and cleared its
executable bit, since it's not a script.
2013-04-11 18:37:23 +04:00
Andrey Kamaev
a5d2253be8 Merge pull request #544 from AnnaKogan8:added-regressions-only-option 2013-02-28 00:38:47 +04:00
Andrey Kamaev
4d2c864bea Add option --with-score to summary.py to automatically classify speedups
Legend for new column is:
* FASTER - strong speedup
* faster - spedup is detected but it is unreliable
* <empty> - no change in speed
* slower - slowdown is detected but it is unreliable
* SLOWER - strong slowdown
2013-02-26 19:42:51 +04:00
Anna Kogan
a9b1c5db78 Added --regressions-only=X-FACTOR option 2013-02-26 11:09:17 +04:00
Anna Kogan
51e58aeb25 Added --failed-only option and multiple input files support 2013-02-07 19:08:31 +04:00
Andrey Kamaev
0773ab4d07 Merge pull request #315 from taka-no-me:java_on 2013-01-23 19:02:27 +04:00
Andrey Kamaev
3889b34ec3 Add option to run java tests with run.py 2013-01-23 18:08:08 +04:00
Anna Kogan
0587bef667 Switched time from seconds to minutes 2013-01-15 15:32:22 +04:00
Anna Kogan
3cdfa654ae Added 'Num of tests' col, changed headers, added 'overall time' row 2013-01-15 15:26:43 +04:00
Anna Kogan
3271e395c8 Improved console output 2013-01-10 13:19:59 +04:00
Anna Kogan
0d7ce141aa Cleaned code 2013-01-10 13:18:26 +04:00
Anna Kogan
0334cf11cc Fixed last testsuit disregarding 2013-01-10 13:16:46 +04:00
Andrey Kamaev
02d1e68c64 Merge pull request #257 from AnnaKogan8:perf-tests-timing-script 2012-12-24 19:38:49 +04:00
Anna Kogan
f57d692cd7 Improved code 2012-12-24 17:09:04 +04:00
Anna Kogan
e1331b44f5 Wrote script generating timing table 2012-12-24 17:09:04 +04:00
Andrey Kamaev
e1afb1409f Add --perf_verify_sanity option to performance tests
This option provides an easy way to identify tests having no regression data
for sanity checks
2012-12-12 19:42:24 +04:00
Andrey Kamaev
7c6191ec11 Refactored run.py script and temporary file generation logic
* use OPENCV_TEMP_PATH environment variable on all platforms
* fix cleanup after OpenCV tests on Windows
* add --list flag to output names of all tests found
* do not override user-passed --perf_min_samples and --perf_force_samples
  options by --check flag
* fix complier checks inside run.py
2012-12-08 15:43:23 +04:00
Andrey Kamaev
157cdeb443 Merge pull request #175 from taka-no-me:run_py 2012-11-28 12:44:05 +04:00
Andrey Kamaev
8daebeac8a run.py: add --check option for easy run of sanity checks 2012-11-24 14:12:32 +04:00
Andrey Kamaev
9db1d9ba13 run.py: improve handling of multiple Android devices
* do not lose auto-selected device while running several tests
* reduce output noise
* list available devices if unable to auto-select device
* fix error message when no devices connected
2012-11-24 14:11:07 +04:00
marina.kolpakova
285d6320be add #/usr/bin/env python to all python files 2012-11-23 22:57:22 +04:00
OpenCV Buildbot
04384a71e4 Normalize line endings and whitespace 2012-10-17 15:32:23 +04:00
Oleg Sklyarov
3f61b9792b fixed bug #2407: run.py now understands android device names with spaces 2012-10-10 16:59:00 +04:00
Andrey Kamaev
3d9018f019 Fix run.py version control detection when used outide of OpenCV build 2012-08-29 19:17:18 +04:00
Andrey Kamaev
a3527fc4d8 Add git support to run.py 2012-08-07 14:30:36 +04:00
Andrey Kamaev
50d13ba5c4 Fixed xml name generation in run.py 2012-08-06 19:10:55 +04:00
Andrey Kamaev
cfc593eb0d Fixed run.py bugs 2012-07-02 21:07:32 +00:00
Andrey Kamaev
565a8dc30d Fixed automatic temporary files cleaning with run.py on Windows 2012-07-02 12:23:57 +00:00
Andrey Kamaev
d9c74f63e1 All tests writing temporary files are updated to use cv::tempfile() function 2012-06-25 11:24:06 +00:00
Andrey Kamaev
860172c626 run.py - fixed java tests launch on Windows 2012-05-03 07:50:24 +00:00
Vadim Pisarevsky
d5a0088bbe merged 2.4 into trunk 2012-04-30 14:33:52 +00:00
Vadim Pisarevsky
2fd1e2ea57 merged all the latest changes from 2.4 to trunk 2012-04-13 21:50:59 +00:00
Andrey Kamaev
b3f10835a3 run.py: added option to show tests own help 2012-03-15 08:31:57 +00:00
Andrey Kamaev
aeaae8b057 Updated rst configuration to read OpenCV version from sources; fixed several sphinx' warnings 2012-03-02 20:59:13 +00:00
Andrey Kamaev
6f99447fbf Fixed compiler flags caching in cmake 2012-01-24 11:46:11 +00:00
Andrey Kamaev
504a93ed0a Alternative xml naming for performance logs 2012-01-23 16:28:46 +00:00
Andrey Kamaev
c326de43c1 More natural sorting of MatType constants in performance reports. 2012-01-16 07:04:26 +00:00
Andrey Kamaev
61cb83b481 Fixed small bug in summary.py 2012-01-13 12:37:10 +00:00
Andrey Kamaev
cdac77c2b2 Added test group separators to the html version of performance report. 2012-01-09 07:45:19 +00:00
Andrey Kamaev
40c70d127d Fixed bug in summary.py 2012-01-06 08:44:28 +00:00
Andrey Kamaev
e9901306c2 Fixed files picking code to preserve the order in summary.py script. 2012-01-06 07:41:04 +00:00
Andrey Kamaev
4aa47ad115 Added multiple connected Android devices support to the run.py script 2011-12-19 07:43:07 +00:00
Andrey Kamaev
0744092b21 Fixed build with gcc compiler; couple minor improvements in perf testing framework 2011-12-16 04:42:34 +00:00
Andrey Kamaev
a67a0c76c5 Fixed logging in perf tests; small bugfix in run.py 2011-11-30 11:30:45 +00:00
Andrey Morozov
59ea0d8ee8 fixed bug in chart.py 2011-11-29 15:43:46 +00:00
Andrey Kamaev
7fb5b5f218 Improved sorting in performance reports 2011-11-22 13:57:44 +00:00
Andrey Kamaev
7172c8cea9 Android cmake toolchain: fixed some compatibility problems with cmake 2.6.x; improved file header.
Fixed small problem in chart.py.
2011-11-19 18:02:00 +00:00
Andrey Kamaev
3bdb53e484 Android cmake toolchain is reworked for NDK r7; added experimental Android x86 support. 2011-11-16 14:32:10 +00:00
Andrey Kamaev
4136855ba8 Tegra-optimized Canny 2011-11-11 14:45:53 +00:00
Andrey Kamaev
a612fa1520 Performance testing: added option to get list of all values for CV_ENUM; added perf test for cv::Sobel 2011-11-01 15:41:43 +00:00
Andrey Kamaev
1d5e407211 minor 2011-10-31 13:19:12 +00:00
Andrey Kamaev
5eee9e6f6d Several improvements in performance reports 2011-10-18 12:31:08 +00:00
Andrey Kamaev
e3265b0cfe Added filters to html perf reports generated by summary.py 2011-10-12 14:41:36 +00:00
Andrey Kamaev
49d33f6109 Perf tests: linux version of adb search 2011-09-15 07:35:05 +00:00