videoio/FFmpeg: increased packet read attempt limit, allow configuring it
resolves#9455
related #3225
* Use different counters for wrong packets recieved by demuxer and errors from decoder
* Allow modifying these counters via environment variables `OPENCV_FFMPEG_READ_ATTEMPTS`/`OPENCV_FFMPEG_DECODE_ATTEMPTS`
* Added logging when reading breaks at one of error limits
Notes:
* I've been able to reproduce original issue with a video file with 14 total streams (video + audio + subtitles), at some point in the video only packets from the last stream are being sent by the demuxer, thus exceeding our limit. For my specific video total number of packets from wrong stream was about 2700. I've chosen 4096 as default value.
* Default limit of decoding attempts is quite low, because I'm not sure in which cases it can be exceeded (network stream?). I tried to read 8k video from the disk, but it did not cause break at decode point.
Support VideoCapture CAP_PROP_AUTO_WB and CV_CAP_PROP_WHITE_BALANCE_BLUE_U for DShow
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [OK] I agree to contribute to the project under Apache 2 License.
- [OK] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [OK] The PR is proposed to the proper branch
- [OK] There is a reference to the original bug report and related work
https://github.com/opencv/opencv/issues/19621https://github.com/opencv/opencv/issues/21408
### Before apply this pull request console output.
before AWB setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1
after AWB disable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1
after AWB enable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1
after Manual WB(and Disable AWB) setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1
### After apply this pull request console output.
before AWB setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: 0
after AWB disable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 4000
CAP_PROP_AUTO_WB: 0
after AWB enable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 4000
CAP_PROP_AUTO_WB: 1
after Manual WB(and Disable AWB) setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: 0
### Test Code
[OpenCvVideoCapTest.zip](https://github.com/opencv/opencv/files/10825399/OpenCvVideoCapTest.zip)
Merge with https://github.com/opencv/opencv_contrib/pull/3446
Related issue: https://github.com/opencv/opencv/issues/11810
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Adding HEVC/H265 FourCC support to MSMF video writer
* Adding HEVC/H265 fourcc to MSMF video writer
Adding HEVC/H265 fourcc to MSMF video writer. I have verified it with my own video input stream, and it works well on my workstation.
* Update video io testing
* Adding macro fence to get rid of compiler error
H265/HEVC encoder is only available in Windows or later. https://learn.microsoft.com/en-us/windows/win32/medfound/h-265---hevc-video-encoder
* Update test_video_io.cpp
Usage of imread(): magic number 0, unchecked result
* docs: rewrite 0/1 to IMREAD_GRAYSCALE/IMREAD_COLOR in imread()
* samples, apps: rewrite 0/1 to IMREAD_GRAYSCALE/IMREAD_COLOR in imread()
* tests: rewrite 0/1 to IMREAD_GRAYSCALE/IMREAD_COLOR in imread()
* doc/py_tutorials: check imread() result
videoio: add Orbbec Gemini 2 and Astra 2 camera support
### Test Result
| OS | Compiler | Camera | Result |
|-----|-----------|---------|--------|
|Windows11| (VS2022)MSVC17.3|Orbbec Gemini 2|Pass|
|Windows11| (VS2022)MSVC17.3|Orbbec Astra 2|Pass|
|Ubuntu22.04|GCC9.2|Orbbec Gemini 2|Pass|
|Ubuntu22.04|GCC9.2|Orbbec Astra 2|Pass|
### Pull Request Readiness Checklist
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] The feature is well documented and sample code can be built with the project CMake
Address https://github.com/opencv/opencv/issues/22868
Used the same defaults as it's done for FFmpeg
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
```
force_builders=Custom
build_image:Custom=gstreamer:16.04
buildworker:Custom=linux-1
```
Add Python bindings for VideoCapture::waitAny #21826
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Introduce libavdevice to make v4l2 available to the ffmpeg backend
* introduce libavdevice to make v4l2 available to the ffmpeg backend
* downgrade the min required libavdevice version to 53.2.0
* make libavdevice optional
* create OCV_OPTION OPENCV_FFMPEG_ENABLE_LIBAVDEVICE and add definition through ocv_add_external_target
* move OCV_OPTION 'OPENCV_FFMPEG_ENABLE_LIBAVDEVICE' to detect_ffmpeg.cmake
Setting CAP_PROP_AUTO_EXPOSURE on VideoCapture with backend DSHOW does not change anything. Now with this implementation the property can be used with value 1 for availability.
* Allow the number of threads FFMpeg uses to be selected during VideoCapture::open().
Reset interupt timer in grab if
err = avformat_find_stream_info(ic, NULL);
is interupted but open is successful.
* Correct the returned number of threads and amend test cases.
* Update container test case.
* Reverse changes added to existing videoio_container test case and include test combining thread change and raw read in the newly added videoio_read test case.
This fixes the following error with mingw toolchain:
opencv/modules/videoio/src/cap_msmf.cpp:1020: error: 'wstring_convert' is not a member of 'std'
1020 | std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> conv;
| ^~~~~~~~~~~~~~~
opencv/modules/videoio/src/cap_ffmpeg_hw.hpp:230:26: error: 'wstring_convert' is not a member of 'std'
230 | std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> conv;
| ^~~~~~~~~~~~~~~
The locale header is required according to C++ standard.
See https://en.cppreference.com/w/cpp/locale/wstring_convert
This fixes the following error with mingw toolchain:
opencv/modules/videoio/src/cap_obsensor/obsensor_stream_channel_msmf.hpp:160:10: error: 'condition_variable' in namespace 'std' does not name a type
160 | std::condition_variable streamStateCv_;
| ^~~~~~~~~~~~~~~~~~
libstdc++ that comes with gcc 4.8 doesn't
define `getline(basic_istream<char>&&, std::string&)`
even if it's part of the c++11 standard.
However we can still use the following:
`getline(basic_istream<char>&, std::string&)`.
* videoio: add support for obsensor (Orbbec RGB-D Camera )
* obsensor: code format issues fixed and some code optimized
* obsensor: fix typo and format issues
* obsensor: fix crosses initialization error
Replaced sprintf with safer snprintf
* Straightforward replacement of sprintf with safer snprintf
* Trickier replacement of sprintf with safer snprintf
Some functions were changed to take another parameter: the size of the buffer, so that they can pass that size on to snprintf.
Some GStreamer elements may produce buffers with very non
standard strides, offsets and/or even transport each plane
in different, non-contiguous pointers. This non-standard
layout is communicated via GstVideoMeta structures attached
to the buffers. Given this, when a GstVideoMeta is available,
one should parse the layout from it instead of generating
a generic one from the caps.
The GstVideoFrame utility does precisely this: if the buffer
contains a video meta, it uses that to fill the format and
memory layout. If there is no meta available, the layout is
inferred from the caps.
* Added support for 4B RGB V4L2 pixel formats
Added support for V4L2_PIX_FMT_XBGR32 and V4L2_PIX_FMT_ABGR32 pixel
formats.
* Added workaround for missing V4L2_PIX_FMT_ABGR32 and V4L2_PIX_FMT_XBGR32
defines
Default FFMPEG VideoCapture backend to rtsp_flags=prefer_tcp
* Make the VideoCapture ffmpeg backends default rtsp connection type prefer_tcp.
* Ensure that the ffmpeg version of avformat is checked.
Audio MSMF: added the ability to set sample per second
* Audio MSMF: added the ability to set sample per second
* changed the valid sampling rate check
* fixed docs
* add test
* fixed warning
* fixed error
* fixed error
Add capacity to Videocapture to return the extraData from FFmpeg when required
* Update rawMode to append any extra data recieved during the initial negotiation of an RTSP stream or during the parsing of an MPEG4 file header.
For h264[5] RTSP streams this ensures the parameter sets if available are always returned on the first call to grab()/read() and has two purposes:
1) To ensure the parameter sets are available even if they are not transmitted in band. This is common for axis ip camera's.
2) To allow callers of VideoCapture::grab()[read()] to write to split the raw stream over multiple files by appending the parameter sets to the begining of any new files.
For (1) there is no alternative, for (2) if the parameter sets were provided in band it would be possible to parse the raw bit stream and search for the parameter sets however that would be a lot of work when that information is already provided by FFMPEG.
For MPEG4 files this information is only suplied in the header and is required for decoding.
Two properties are also required to enable the raw encoded bitstream to be written to multiple files, these are;
1) an indicator as to whether the last frame was a key frame or not - each new file needs to start at a key frame to avoid storing unusable frame diffs,
2) the length in bytes of the paramater sets contained in the last frame - required to split the paramater sets from the frame without having to parse the stream. Any call to VideoCapture::get(CAP_PROP_LF_PARAM_SET_LEN) returning a number greater than zero indicates the presense of a parameter set at the begining of the raw bitstream.
* Adjust test data to account for extraData
* Address warning.
* Change added property names and remove paramater set start code check.
* Output extra data on calls to retrieve instead of appending to the first packet.
* Reverted old test case and added new one to evaluate new functionality.
* Add missing definition.
* Remove flag from legacy api.
Add property to determine if returning extra data is supported.
Always allow extra data to be returned on calls to cap.retrieve()
Update test case.
* Update condition which indicates CAP_PROP_CODEC_EXTRADATA_INDEX is not supported in test case.
* Include compatibility for windows dll if not updated.
Enforce existing return status convention.
* Fix return error and missing test constraints.
Fix the following build failure with gcc 4.8:
In file included from /home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/videoio/src/cap_ffmpeg_impl.hpp💯0,
from /home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/videoio/src/cap_ffmpeg.cpp:50:
/home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/videoio/src/cap_ffmpeg_hw.hpp: In constructor 'HWAccelIterator::HWAccelIterator(cv::VideoAccelerationType, bool, AVDictionary*)':
/home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/videoio/src/cap_ffmpeg_hw.hpp:939:23: error: use of deleted function 'std::basic_istringstream<char>& std::basic_istringstream<char>::operator=(const std::basic_istringstream<char>&)'
s_stream_ = std::istringstream(accel_list);
^
In file included from /home/buildroot/autobuild/instance-3/output-1/host/opt/ext-toolchain/arm-none-linux-gnueabi/include/c++/4.8.3/complex:45:0,
from /home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/core/include/opencv2/core/cvstd.inl.hpp:47,
from /home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/core/include/opencv2/core.hpp:3306,
from /home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/videoio/include/opencv2/videoio.hpp:46,
from /home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/videoio/src/precomp.hpp:57,
from /home/buildroot/autobuild/instance-3/output-1/build/opencv4-4.5.4/modules/videoio/src/cap_ffmpeg.cpp:42:
/home/buildroot/autobuild/instance-3/output-1/host/opt/ext-toolchain/arm-none-linux-gnueabi/include/c++/4.8.3/sstream:272:11: note: 'std::basic_istringstream<char>& std::basic_istringstream<char>::operator=(const std::basic_istringstream<char>&)' is implicitly deleted because the default definition would be ill-formed:
class basic_istringstream : public basic_istream<_CharT, _Traits>
^
/home/buildroot/autobuild/instance-3/output-1/host/opt/ext-toolchain/arm-none-linux-gnueabi/include/c++/4.8.3/sstream:272:11: error: use of deleted function 'std::basic_istream<char>& std::basic_istream<char>::operator=(const std::basic_istream<char>&)'
Fixes:
- http://autobuild.buildroot.org/results/60f8846b435dafda0ced412d59ffe15bdff0810d
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Fix: #21021
NDK API AMediaCodec_getOutputBuffer() returns MediaCodecBuffer::data()
which is actually ABuffer::data(). The returned buffer address is already
adjusted by offset.
More info:
ABuffer::base() returns base address without offset
ABuffer::data() returns base + offset
Change-Id: I2936339ce4fa9acf657a5a7d92adc1275d7b28a1
