mirror/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2025-01-06 18:48:00 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
2aad039b4f
opencv/modules/videoio/src/cap_msmf.cpp

3056 lines
102 KiB
C++
Raw Normal View History

videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
#include "precomp.hpp"
/*
Media Foundation-based Video Capturing module is based on
videoInput library by Evgeny Pereguda:
http://www.codeproject.com/Articles/559437/Capturing-of-video-from-web-camera-on-Windows-7-an
Misc. modules/ cont. pt2 Found via `codespell`
2018-02-14 00:28:11 +08:00
Originally licensed under The Code Project Open License (CPOL) 1.02:
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
http://www.codeproject.com/info/cpol10.aspx
*/
Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update and rename cap_msmf.h to cap_msmf.hpp Successful test - samples are grabbed Library updated and cleaned up with comments, marshaling, exceptions and linker settings Fixed trailing whitespace VS 2013 support and cleanup consistency plus C++/CX new object fixed Conflicts: modules/highgui/src/cap_msmf.cpp modules/highgui/src/cap_msmf.hpp modules/highgui/src/ppltasks_winrt.h Fix merge conflicts VS 2013 Update 2 library bug fix integrated a-wi's changed integrated
2013-12-14 16:53:30 +08:00
//require Windows 8 for some of the formats defined otherwise could baseline on lower version
MSMF-based VideoCapture reworked to use SourceReader
2018-03-28 19:03:22 +08:00
#if WINVER < _WIN32_WINNT_WIN8
Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update and rename cap_msmf.h to cap_msmf.hpp Successful test - samples are grabbed Library updated and cleaned up with comments, marshaling, exceptions and linker settings Fixed trailing whitespace VS 2013 support and cleanup consistency plus C++/CX new object fixed Conflicts: modules/highgui/src/cap_msmf.cpp modules/highgui/src/cap_msmf.hpp modules/highgui/src/ppltasks_winrt.h Fix merge conflicts VS 2013 Update 2 library bug fix integrated a-wi's changed integrated
2013-12-14 16:53:30 +08:00
#undef WINVER
MSMF-based VideoCapture reworked to use SourceReader
2018-03-28 19:03:22 +08:00
#define WINVER _WIN32_WINNT_WIN8
Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update and rename cap_msmf.h to cap_msmf.hpp Successful test - samples are grabbed Library updated and cleaned up with comments, marshaling, exceptions and linker settings Fixed trailing whitespace VS 2013 support and cleanup consistency plus C++/CX new object fixed Conflicts: modules/highgui/src/cap_msmf.cpp modules/highgui/src/cap_msmf.hpp modules/highgui/src/ppltasks_winrt.h Fix merge conflicts VS 2013 Update 2 library bug fix integrated a-wi's changed integrated
2013-12-14 16:53:30 +08:00
#endif
Merge pull request #13422 from mshabunin:split-videoio-cmake * Split videoio dependencies search into pieces, removed obsolete backends
2018-12-26 20:50:20 +08:00
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
#include <windows.h>
#include <guiddef.h>
#include <mfidl.h>
videoio(MSMF): backport WITH_MSMF_DXVA flag
2018-12-13 19:53:48 +08:00
#include <mfapi.h>
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
#include <mfplay.h>
#include <mfobjects.h>
Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update and rename cap_msmf.h to cap_msmf.hpp Successful test - samples are grabbed Library updated and cleaned up with comments, marshaling, exceptions and linker settings Fixed trailing whitespace VS 2013 support and cleanup consistency plus C++/CX new object fixed Conflicts: modules/highgui/src/cap_msmf.cpp modules/highgui/src/cap_msmf.hpp modules/highgui/src/ppltasks_winrt.h Fix merge conflicts VS 2013 Update 2 library bug fix integrated a-wi's changed integrated
2013-12-14 16:53:30 +08:00
#include <tchar.h>
Media Foundation-based code refactoring. I* + SafeRelease -> ComPtr.
2013-05-15 21:09:03 +08:00
#include <strsafe.h>
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
#include <codecvt>
Merge pull request #13422 from mshabunin:split-videoio-cmake * Split videoio dependencies search into pieces, removed obsolete backends
2018-12-26 20:50:20 +08:00
#include <mfreadwrite.h>
videoio(MSMF): backport WITH_MSMF_DXVA flag
2018-12-13 19:53:48 +08:00
#ifdef HAVE_MSMF_DXVA
#include <d3d11.h>
#include <d3d11_4.h>
videoio: Include missing locale header for std::wstring_convert This fixes the following error with mingw toolchain: opencv/modules/videoio/src/cap_msmf.cpp:1020: error: 'wstring_convert' is not a member of 'std' 1020 | std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> conv; | ^~~~~~~~~~~~~~~ opencv/modules/videoio/src/cap_ffmpeg_hw.hpp:230:26: error: 'wstring_convert' is not a member of 'std' 230 | std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> conv; | ^~~~~~~~~~~~~~~ The locale header is required according to C++ standard. See https://en.cppreference.com/w/cpp/locale/wstring_convert
2022-08-19 15:44:57 +08:00
#include <locale>
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
#endif
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
#include <new>
#include <map>
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
#include <queue>
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
#include <vector>
#include <string>
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
#include <algorithm>
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
#include <deque>
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
#include <stdio.h>
#include <stdarg.h>
#include <string.h>
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Remove ATL dependency from MSMF capture code. Use _com_ptr_t instead of CComPtr in ComPtr wrapper to avoid ATL dependency.
2014-10-01 18:02:28 +08:00
#ifdef _MSC_VER
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
#pragma warning(disable:4503)
#pragma comment(lib, "mfplat")
#pragma comment(lib, "mf")
#pragma comment(lib, "mfuuid")
#pragma comment(lib, "Strmiids")
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
#pragma comment(lib, "Mfreadwrite")
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
#pragma comment(lib, "dxgi")
videoio(MSMF): backport WITH_MSMF_DXVA flag
2018-12-13 19:53:48 +08:00
#ifdef HAVE_MSMF_DXVA
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
#pragma comment(lib, "d3d11")
videoio(MSMF): avoid OpenCV load failure on Win7 machines OpenCV binaries are compiled on Win10 environment
2018-06-30 03:54:58 +08:00
// MFCreateDXGIDeviceManager() is available since Win8 only.
// To avoid OpenCV loading failure on Win7 use dynamic detection of this symbol.
// Details: https://github.com/opencv/opencv/issues/11858
videoio(MSMF): fix 32-bit build crash
2018-07-03 20:28:55 +08:00
typedef HRESULT (WINAPI *FN_MFCreateDXGIDeviceManager)(UINT *resetToken, IMFDXGIDeviceManager **ppDeviceManager);
videoio(MSMF): avoid OpenCV load failure on Win7 machines OpenCV binaries are compiled on Win10 environment
2018-06-30 03:54:58 +08:00
static bool pMFCreateDXGIDeviceManager_initialized = false;
static FN_MFCreateDXGIDeviceManager pMFCreateDXGIDeviceManager = NULL;
static void init_MFCreateDXGIDeviceManager()
{
Fixed several issues found by static analysis, Windows-specific
2018-07-27 23:25:55 +08:00
HMODULE h = LoadLibraryExA("mfplat.dll", NULL, LOAD_LIBRARY_SEARCH_SYSTEM32);
videoio(MSMF): avoid OpenCV load failure on Win7 machines OpenCV binaries are compiled on Win10 environment
2018-06-30 03:54:58 +08:00
if (h)
{
pMFCreateDXGIDeviceManager = (FN_MFCreateDXGIDeviceManager)GetProcAddress(h, "MFCreateDXGIDeviceManager");
}
pMFCreateDXGIDeviceManager_initialized = true;
}
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
#endif
Remove MinCore_Downlevel, replace with Shlwapi On windows 7, MinCore_Downlevel does not work correctly. However, the only API used was QISearch, which can be found in Shlwapi. Closes #12010
2018-12-12 09:06:01 +08:00
#pragma comment(lib, "Shlwapi.lib")
Remove ATL dependency from MSMF capture code. Use _com_ptr_t instead of CComPtr in ComPtr wrapper to avoid ATL dependency.
2014-10-01 18:02:28 +08:00
#endif
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Update cap_msmf.cpp Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Update cap_msmf.cpp Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update cap_msmf.cpp Create cap_msmf.h Update cap_msmf.cpp Update cap_msmf.h Update cap_msmf.cpp Update cap_msmf.h Update and rename cap_msmf.h to cap_msmf.hpp Update cap_msmf.cpp Update CMakeLists.txt Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.cpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Successful test - samples are grabbed Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Create ppltasks_winrt.h Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Update ppltasks_winrt.h Library updated and cleaned up with comments, marshaling, exceptions and linker settings Update ppltasks_winrt.h Fixed trailing whitespace
2013-12-14 16:53:30 +08:00
#include <mferror.h>
Remove ATL dependency from MSMF capture code. Use _com_ptr_t instead of CComPtr in ComPtr wrapper to avoid ATL dependency.
2014-10-01 18:02:28 +08:00
#include <comdef.h>
Update cap_msmf.cpp Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Update cap_msmf.cpp Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update cap_msmf.cpp Create cap_msmf.h Update cap_msmf.cpp Update cap_msmf.h Update cap_msmf.cpp Update cap_msmf.h Update and rename cap_msmf.h to cap_msmf.hpp Update cap_msmf.cpp Update CMakeLists.txt Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.cpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Successful test - samples are grabbed Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Create ppltasks_winrt.h Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Update ppltasks_winrt.h Library updated and cleaned up with comments, marshaling, exceptions and linker settings Update ppltasks_winrt.h Fixed trailing whitespace
2013-12-14 16:53:30 +08:00
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
#include <shlwapi.h> // QISearch
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
struct IMFMediaType;
struct IMFActivate;
struct IMFMediaSource;
struct IMFAttributes;
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
videoio: drop VideoCaptureModes enum in favour of fourcc
2018-11-13 22:53:35 +08:00
#define CV_CAP_MODE_BGR CV_FOURCC_MACRO('B','G','R','3')
#define CV_CAP_MODE_RGB CV_FOURCC_MACRO('R','G','B','3')
#define CV_CAP_MODE_GRAY CV_FOURCC_MACRO('G','R','E','Y')
#define CV_CAP_MODE_YUYV CV_FOURCC_MACRO('Y', 'U', 'Y', 'V')
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
using namespace cv;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
namespace
{
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Remove unused code
2018-04-13 20:43:41 +08:00
template <class T>
class ComPtr
{
public:
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
ComPtr()
Remove unused code
2018-04-13 20:43:41 +08:00
{
}
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
ComPtr(T* lp)
Remove unused code
2018-04-13 20:43:41 +08:00
{
p = lp;
}
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
ComPtr(_In_ const ComPtr<T>& lp)
Remove unused code
2018-04-13 20:43:41 +08:00
{
p = lp.p;
}
virtual ~ComPtr()
{
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
void swap(_In_ ComPtr<T>& lp)
{
ComPtr<T> tmp(p);
p = lp.p;
lp.p = tmp.p;
tmp = NULL;
}
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
T** operator&()
Remove unused code
2018-04-13 20:43:41 +08:00
{
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
CV_Assert(p == NULL);
Remove unused code
2018-04-13 20:43:41 +08:00
return p.operator&();
}
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
T* operator->() const
Remove unused code
2018-04-13 20:43:41 +08:00
{
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
CV_Assert(p != NULL);
Remove unused code
2018-04-13 20:43:41 +08:00
return p.operator->();
}
operator bool()
{
return p.operator!=(NULL);
}
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
T* Get() const
Remove unused code
2018-04-13 20:43:41 +08:00
{
return p;
}
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
void Release()
Remove unused code
2018-04-13 20:43:41 +08:00
{
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
if (p)
p.Release();
Remove unused code
2018-04-13 20:43:41 +08:00
}
// query for U interface
template<typename U>
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
HRESULT As(_Out_ ComPtr<U>& lp) const
Remove unused code
2018-04-13 20:43:41 +08:00
{
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
lp.Release();
return p->QueryInterface(__uuidof(U), reinterpret_cast<void**>((T**)&lp));
Remove unused code
2018-04-13 20:43:41 +08:00
}
private:
_COM_SMARTPTR_TYPEDEF(T, __uuidof(T));
TPtr p;
};
#define _ComPtr ComPtr
Update cap_msmf.cpp Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Update cap_msmf.cpp Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update cap_msmf.cpp Create cap_msmf.h Update cap_msmf.cpp Update cap_msmf.h Update cap_msmf.cpp Update cap_msmf.h Update and rename cap_msmf.h to cap_msmf.hpp Update cap_msmf.cpp Update CMakeLists.txt Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.cpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Successful test - samples are grabbed Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Update cap_msmf.cpp Update cap_msmf.hpp Create ppltasks_winrt.h Update cap_msmf.hpp Update cap_msmf.hpp Update cap_msmf.cpp Update ppltasks_winrt.h Library updated and cleaned up with comments, marshaling, exceptions and linker settings Update ppltasks_winrt.h Fixed trailing whitespace
2013-12-14 16:53:30 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
template <typename T> inline T absDiff(T a, T b) { return a >= b ? a - b : b - a; }
//==================================================================================================
// Structure for collecting info about types of video which are supported by current video device
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
struct MediaType
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
//video param
Remove unused code
2018-04-13 20:43:41 +08:00
UINT32 width;
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
UINT32 height;
INT32 stride; // stride is negative if image is bottom-up
UINT32 isFixedSize;
UINT32 frameRateNum;
UINT32 frameRateDenom;
UINT32 aspectRatioNum;
UINT32 aspectRatioDenom;
UINT32 sampleSize;
UINT32 interlaceMode;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
//audio param
UINT32 bit_per_sample;
UINT32 nChannels;
UINT32 nAvgBytesPerSec;
UINT32 nSamplesPerSec;
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
GUID majorType; // video or audio
GUID subType; // fourCC
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
_ComPtr<IMFMediaType> Type;
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
MediaType(IMFMediaType *pType = 0) :
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
Type(pType),
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
width(0), height(0),
stride(0),
isFixedSize(true),
frameRateNum(1), frameRateDenom(1),
aspectRatioNum(1), aspectRatioDenom(1),
sampleSize(0),
interlaceMode(0),
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bit_per_sample(0),
nChannels(0),
nAvgBytesPerSec(0),
nSamplesPerSec(0),
majorType({ 0 }),//MFMediaType_Video
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
subType({ 0 })
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (pType)
{
pType->GetGUID(MF_MT_MAJOR_TYPE, &majorType);
pType->GetGUID(MF_MT_SUBTYPE, &subType);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (majorType == MFMediaType_Audio)
{
pType->GetUINT32(MF_MT_AUDIO_BITS_PER_SAMPLE, &bit_per_sample);
pType->GetUINT32(MF_MT_AUDIO_NUM_CHANNELS, &nChannels);
pType->GetUINT32(MF_MT_AUDIO_AVG_BYTES_PER_SECOND, &nAvgBytesPerSec);
pType->GetUINT32(MF_MT_AUDIO_FLOAT_SAMPLES_PER_SECOND, &nSamplesPerSec);
}
else if (majorType == MFMediaType_Video)
{
MFGetAttributeSize(pType, MF_MT_FRAME_SIZE, &width, &height);
pType->GetUINT32(MF_MT_DEFAULT_STRIDE, (UINT32*)&stride); // value is stored as UINT32 but should be casted to INT3)
pType->GetUINT32(MF_MT_FIXED_SIZE_SAMPLES, &isFixedSize);
MFGetAttributeRatio(pType, MF_MT_FRAME_RATE, &frameRateNum, &frameRateDenom);
MFGetAttributeRatio(pType, MF_MT_PIXEL_ASPECT_RATIO, &aspectRatioNum, &aspectRatioDenom);
pType->GetUINT32(MF_MT_SAMPLE_SIZE, &sampleSize);
pType->GetUINT32(MF_MT_INTERLACE_MODE, &interlaceMode);
pType->GetUINT32(MF_MT_INTERLACE_MODE, &interlaceMode);
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
static MediaType createDefault_Video()
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
MediaType res;
res.width = 640;
res.height = 480;
res.setFramerate(30.0);
return res;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
static MediaType createDefault_Audio()
{
MediaType res;
res.majorType = MFMediaType_Audio;
res.subType = MFAudioFormat_PCM;
res.bit_per_sample = 16;
res.nChannels = 1;
res.nSamplesPerSec = 44100;
return res;
}
inline bool isEmpty(bool flag = false) const
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (!flag)
return width == 0 && height == 0;
else
return nChannels == 0;
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
_ComPtr<IMFMediaType> createMediaType_Video() const
Remove unused code
2018-04-13 20:43:41 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
_ComPtr<IMFMediaType> res;
MFCreateMediaType(&res);
if (width != 0 || height != 0)
MFSetAttributeSize(res.Get(), MF_MT_FRAME_SIZE, width, height);
if (stride != 0)
res->SetUINT32(MF_MT_DEFAULT_STRIDE, stride);
res->SetUINT32(MF_MT_FIXED_SIZE_SAMPLES, isFixedSize);
if (frameRateNum != 0 || frameRateDenom != 0)
MFSetAttributeRatio(res.Get(), MF_MT_FRAME_RATE, frameRateNum, frameRateDenom);
if (aspectRatioNum != 0 || aspectRatioDenom != 0)
MFSetAttributeRatio(res.Get(), MF_MT_PIXEL_ASPECT_RATIO, aspectRatioNum, aspectRatioDenom);
if (sampleSize > 0)
res->SetUINT32(MF_MT_SAMPLE_SIZE, sampleSize);
res->SetUINT32(MF_MT_INTERLACE_MODE, interlaceMode);
if (majorType != GUID())
res->SetGUID(MF_MT_MAJOR_TYPE, majorType);
if (subType != GUID())
res->SetGUID(MF_MT_SUBTYPE, subType);
return res;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
_ComPtr<IMFMediaType> createMediaType_Audio() const
{
_ComPtr<IMFMediaType> res;
MFCreateMediaType(&res);
if (majorType != GUID())
res->SetGUID(MF_MT_MAJOR_TYPE, majorType);
if (subType != GUID())
res->SetGUID(MF_MT_SUBTYPE, subType);
if (bit_per_sample != 0)
res->SetUINT32(MF_MT_AUDIO_BITS_PER_SAMPLE, bit_per_sample);
if (nChannels != 0)
res->SetUINT32(MF_MT_AUDIO_NUM_CHANNELS, nChannels);
if (nSamplesPerSec != 0)
res->SetUINT32(MF_MT_AUDIO_SAMPLES_PER_SECOND, nSamplesPerSec);
return res;
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
void setFramerate(double fps)
{
frameRateNum = (UINT32)cvRound(fps * 1000.0);
frameRateDenom = 1000;
}
double getFramerate() const
{
return frameRateDenom != 0 ? ((double)frameRateNum) / ((double)frameRateDenom) : 0;
}
LONGLONG getFrameStep() const
{
const double fps = getFramerate();
return (LONGLONG)(fps > 0 ? 1e7 / fps : 0);
}
inline unsigned long resolutionDiff(const MediaType& other) const
{
const unsigned long wdiff = absDiff(width, other.width);
const unsigned long hdiff = absDiff(height, other.height);
return wdiff + hdiff;
}
// check if 'this' is better than 'other' comparing to reference
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool VideoIsBetterThan(const MediaType& other, const MediaType& ref) const
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
const unsigned long thisDiff = resolutionDiff(ref);
const unsigned long otherDiff = other.resolutionDiff(ref);
if (thisDiff < otherDiff)
return true;
if (thisDiff == otherDiff)
Remove unused code
2018-04-13 20:43:41 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (width > other.width)
return true;
if (width == other.width && height > other.height)
return true;
if (width == other.width && height == other.height)
{
const double thisRateDiff = absDiff(getFramerate(), ref.getFramerate());
const double otherRateDiff = absDiff(other.getFramerate(), ref.getFramerate());
if (thisRateDiff < otherRateDiff)
return true;
}
Remove unused code
2018-04-13 20:43:41 +08:00
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return false;
Remove unused code
2018-04-13 20:43:41 +08:00
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool AudioIsBetterThan(const MediaType& other, const MediaType& ref) const
{
double thisDiff = absDiff(nChannels, ref.nChannels);
double otherDiff = absDiff(other.nChannels, ref.nChannels);
if (otherDiff < thisDiff)
{
thisDiff = absDiff(bit_per_sample, ref.bit_per_sample);
otherDiff = absDiff(bit_per_sample, ref.bit_per_sample);
if (otherDiff < thisDiff)
{
thisDiff = absDiff(nSamplesPerSec, ref.nSamplesPerSec);
otherDiff = absDiff(nSamplesPerSec, ref.nSamplesPerSec);
if (otherDiff < thisDiff)
return true;
}
}
return false;
}
select correct MediaType in MSMF backend.
2022-08-06 01:52:33 +08:00
bool VideoIsAvailable() const
{
return ((subType == MFVideoFormat_RGB32) ||
(subType == MFVideoFormat_RGB24) ||
(subType == MFVideoFormat_YUY2));
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
};
GetProperty method for MSMF VideoCapture implemented.
2013-05-23 00:50:54 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
void printFormat(std::ostream& out, const GUID& fmt)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
#define PRNT(FMT) else if (fmt == FMT) out << #FMT;
if (fmt == MFVideoFormat_Base) out << "Base";
PRNT(MFVideoFormat_RGB32)
PRNT(MFVideoFormat_ARGB32)
PRNT(MFVideoFormat_RGB24)
PRNT(MFVideoFormat_RGB555)
PRNT(MFVideoFormat_RGB565)
PRNT(MFVideoFormat_RGB8)
else
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
char fourcc[5] = { 0 };
memcpy(fourcc, &fmt.Data1, 4);
out << fourcc;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
#undef PRNT
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
std::ostream& operator<<(std::ostream& out, const MediaType& mt)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
out << "(" << mt.width << "x" << mt.height << " @ " << mt.getFramerate() << ") ";
printFormat(out, mt.subType);
return out;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
//==================================================================================================
// Class for creating of Media Foundation context
class Media_Foundation
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
public:
~Media_Foundation(void) { /*CV_Assert(SUCCEEDED(MFShutdown()));*/ CoUninitialize(); }
static Media_Foundation& getInstance()
{
static Media_Foundation instance;
return instance;
}
private:
Media_Foundation(void) { CoInitialize(0); CV_Assert(SUCCEEDED(MFStartup(MF_VERSION))); }
};
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
//==================================================================================================
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
class SourceReaderCB : public IMFSourceReaderCallback
{
public:
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
static const size_t MSMF_READER_MAX_QUEUE_SIZE = 3;
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
SourceReaderCB() :
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
m_nRefCount(0), m_hEvent(CreateEvent(NULL, FALSE, FALSE, NULL)), m_bEOS(FALSE), m_hrStatus(S_OK), m_reader(NULL), m_dwStreamIndex(0)
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
{
}
// IUnknown methods
STDMETHODIMP QueryInterface(REFIID iid, void** ppv) CV_OVERRIDE
{
#ifdef _MSC_VER
#pragma warning(push)
#pragma warning(disable:4838)
#endif
static const QITAB qit[] =
{
QITABENT(SourceReaderCB, IMFSourceReaderCallback),
{ 0 },
};
#ifdef _MSC_VER
#pragma warning(pop)
#endif
return QISearch(this, qit, iid, ppv);
}
STDMETHODIMP_(ULONG) AddRef() CV_OVERRIDE
{
return InterlockedIncrement(&m_nRefCount);
}
STDMETHODIMP_(ULONG) Release() CV_OVERRIDE
{
ULONG uCount = InterlockedDecrement(&m_nRefCount);
if (uCount == 0)
{
delete this;
}
return uCount;
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
STDMETHODIMP OnReadSample(HRESULT hrStatus, DWORD dwStreamIndex, DWORD dwStreamFlags, LONGLONG llTimestamp, IMFSample *pSample) CV_OVERRIDE
{
HRESULT hr = 0;
cv::AutoLock lock(m_mutex);
if (SUCCEEDED(hrStatus))
{
if (pSample)
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): got frame at " << llTimestamp);
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
if (m_capturedFrames.size() >= MSMF_READER_MAX_QUEUE_SIZE)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
#if 0
CV_LOG_DEBUG(NULL, "videoio(MSMF): drop frame (not processed). Timestamp=" << m_capturedFrames.front().timestamp);
m_capturedFrames.pop();
#else
// this branch reduces latency if we drop frames due to slow processing.
// avoid fetching of already outdated frames from the queue's front.
CV_LOG_DEBUG(NULL, "videoio(MSMF): drop previous frames (not processed): " << m_capturedFrames.size());
std::queue<CapturedFrameInfo>().swap(m_capturedFrames); // similar to missing m_capturedFrames.clean();
#endif
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
m_capturedFrames.emplace(CapturedFrameInfo{ llTimestamp, _ComPtr<IMFSample>(pSample), hrStatus });
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
}
else
{
CV_LOG_WARNING(NULL, "videoio(MSMF): OnReadSample() is called with error status: " << hrStatus);
}
if (MF_SOURCE_READERF_ENDOFSTREAM & dwStreamFlags)
{
// Reached the end of the stream.
m_bEOS = true;
}
m_hrStatus = hrStatus;
if (FAILED(hr = m_reader->ReadSample(dwStreamIndex, 0, NULL, NULL, NULL, NULL)))
{
CV_LOG_WARNING(NULL, "videoio(MSMF): async ReadSample() call is failed with error status: " << hr);
m_bEOS = true;
}
if (pSample || m_bEOS)
{
SetEvent(m_hEvent);
}
return S_OK;
}
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
STDMETHODIMP OnEvent(DWORD, IMFMediaEvent *) CV_OVERRIDE
{
return S_OK;
}
STDMETHODIMP OnFlush(DWORD) CV_OVERRIDE
{
return S_OK;
}
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
HRESULT Wait(DWORD dwMilliseconds, _ComPtr<IMFSample>& mediaSample, LONGLONG& sampleTimestamp, BOOL& pbEOS)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
pbEOS = FALSE;
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
for (;;)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
{
cv::AutoLock lock(m_mutex);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
pbEOS = m_bEOS && m_capturedFrames.empty();
if (pbEOS)
return m_hrStatus;
if (!m_capturedFrames.empty())
{
CV_Assert(!m_capturedFrames.empty());
CapturedFrameInfo frameInfo = m_capturedFrames.front(); m_capturedFrames.pop();
CV_LOG_DEBUG(NULL, "videoio(MSMF): handle frame at " << frameInfo.timestamp);
mediaSample = frameInfo.sample;
CV_Assert(mediaSample);
sampleTimestamp = frameInfo.timestamp;
ResetEvent(m_hEvent); // event is auto-reset, but we need this forced reset due time gap between wait() and mutex hold.
return frameInfo.hrStatus;
}
}
CV_LOG_DEBUG(NULL, "videoio(MSMF): waiting for frame... ");
DWORD dwResult = WaitForSingleObject(m_hEvent, dwMilliseconds);
if (dwResult == WAIT_TIMEOUT)
{
return E_PENDING;
}
else if (dwResult != WAIT_OBJECT_0)
{
return HRESULT_FROM_WIN32(GetLastError());
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
private:
// Destructor is private. Caller should call Release.
virtual ~SourceReaderCB()
{
videoio(msmf): use info message in SourceReaderCB destructor
2022-01-10 17:16:01 +08:00
CV_LOG_INFO(NULL, "terminating async callback");
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
}
public:
long m_nRefCount; // Reference count.
cv::Mutex m_mutex;
HANDLE m_hEvent;
BOOL m_bEOS;
HRESULT m_hrStatus;
Fix https://github.com/opencv/opencv/issues/12301
2018-09-03 20:56:38 +08:00
IMFSourceReader *m_reader;
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
DWORD m_dwStreamIndex;
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
struct CapturedFrameInfo {
LONGLONG timestamp;
_ComPtr<IMFSample> sample;
HRESULT hrStatus;
};
std::queue<CapturedFrameInfo> m_capturedFrames;
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
};
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
//==================================================================================================
// Enumerate and store supported formats and finds format which is most similar to the one requested
class FormatStorage
{
public:
struct MediaID
{
DWORD stream;
DWORD media;
MediaID() : stream(0), media(0) {}
void nextStream()
{
stream++;
media = 0;
}
void nextMedia()
{
media++;
}
bool operator<(const MediaID& other) const
{
return (stream < other.stream) || (stream == other.stream && media < other.media);
}
};
void read(IMFSourceReader* source)
{
HRESULT hr = S_OK;
MediaID cur;
while (SUCCEEDED(hr))
{
_ComPtr<IMFMediaType> raw_type;
hr = source->GetNativeMediaType(cur.stream, cur.media, &raw_type);
if (hr == MF_E_NO_MORE_TYPES)
{
hr = S_OK;
cur.nextStream();
}
else if (SUCCEEDED(hr))
{
formats[cur] = MediaType(raw_type.Get());
cur.nextMedia();
}
}
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
void countNumberOfAudioStreams(DWORD &numberOfAudioStreams)
{
std::pair<MediaID, MediaType> best;
std::map<MediaID, MediaType>::const_iterator i = formats.begin();
for (; i != formats.end(); ++i)
{
if(i->second.majorType == MFMediaType_Audio)
{
if(best.second.isEmpty() || i->first.stream != best.first.stream)
{
numberOfAudioStreams++;
best = *i;
}
}
}
}
videoio/MSMF: fixed audio stream handling
2020-05-27 22:52:17 +08:00
std::pair<MediaID, MediaType> findBestVideoFormat(const MediaType& newType)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
std::pair<MediaID, MediaType> best;
std::map<MediaID, MediaType>::const_iterator i = formats.begin();
for (; i != formats.end(); ++i)
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (i->second.majorType == MFMediaType_Video)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
select correct MediaType in MSMF backend.
2022-08-06 01:52:33 +08:00
if (best.second.isEmpty() || (i->second.VideoIsBetterThan(best.second, newType) && i->second.VideoIsAvailable()))
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
{
best = *i;
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
}
return best;
}
std::pair<MediaID, MediaType> findBestAudioFormat(const MediaType& newType)
{
std::pair<MediaID, MediaType> best;
std::map<MediaID, MediaType>::const_iterator i = formats.begin();
best = *i;
for (; i != formats.end(); ++i)
{
if (i->second.majorType == MFMediaType_Audio)
{
if ( i->second.AudioIsBetterThan(best.second, newType))
{
best = *i;
}
}
}
return best;
}
std::pair<MediaID, MediaType> findAudioFormatByStream(const DWORD StreamIndex)
{
std::pair<MediaID, MediaType> best;
std::map<MediaID, MediaType>::const_iterator i = formats.begin();
for (; i != formats.end(); ++i)
{
if (i->second.majorType == MFMediaType_Audio)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if ((*i).first.stream == StreamIndex)
{
best = *i;
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
}
return best;
}
private:
std::map<MediaID, MediaType> formats;
};
//==================================================================================================
// Enumerates devices and activates one of them
class DeviceList
{
public:
DeviceList() : devices(NULL), count(0) {}
~DeviceList()
{
if (devices)
{
for (UINT32 i = 0; i < count; ++i)
if (devices[i])
devices[i]->Release();
CoTaskMemFree(devices);
}
}
UINT32 read(IID sourceType = MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_GUID)
{
_ComPtr<IMFAttributes> attr;
if (FAILED(MFCreateAttributes(&attr, 1)) ||
FAILED(attr->SetGUID(MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE, sourceType)))
{
CV_Error(CV_StsError, "Failed to create attributes");
}
if (FAILED(MFEnumDeviceSources(attr.Get(), &devices, &count)))
{
CV_LOG_DEBUG(NULL, "Failed to enumerate MSMF devices");
return 0;
}
return count;
}
_ComPtr<IMFMediaSource> activateSource(UINT32 index)
{
_ComPtr<IMFMediaSource> result;
if (count == 0 || index >= count || FAILED(devices[index]->ActivateObject(__uuidof(IMFMediaSource), (void**)&result)))
{
CV_LOG_DEBUG(NULL, "Failed to activate media source (device " << index << ")");
}
return result;
}
private:
IMFActivate** devices;
UINT32 count;
};
} // namespace::
//==================================================================================================
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
/******* Capturing video from camera or file via Microsoft Media Foundation **********/
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
class CvCapture_MSMF : public cv::IVideoCapture
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
public:
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
typedef enum {
MODE_SW = 0,
MODE_HW = 1
} MSMFCapture_Mode;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
CvCapture_MSMF();
virtual ~CvCapture_MSMF();
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
bool configureHW(const cv::VideoCaptureParameters& params);
virtual bool open(int, const cv::VideoCaptureParameters* params);
virtual bool open(const cv::String&, const cv::VideoCaptureParameters* params);
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
virtual void close();
virtual double getProperty(int) const CV_OVERRIDE;
virtual bool setProperty(int, double) CV_OVERRIDE;
Merge pull request #20942 from MaximMilashchenko:AudioPatch Audio patch * fixed microphone, audio position * fixed docs * changed AudioOpenCheck
2021-10-26 22:33:53 +08:00
bool configureAudioFrame();
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool grabAudioFrame();
bool grabVideoFrame();
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
virtual bool grabFrame() CV_OVERRIDE;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool retrieveAudioFrame(int, OutputArray);
bool retrieveVideoFrame(OutputArray);
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
virtual bool retrieveFrame(int, cv::OutputArray) CV_OVERRIDE;
virtual bool isOpened() const CV_OVERRIDE { return isOpen; }
videoio: added '.getBackendName()' method
2018-09-16 04:54:03 +08:00
virtual int getCaptureDomain() CV_OVERRIDE { return CV_CAP_MSMF; }
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
protected:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool configureOutput();
bool configureAudioOutput(MediaType newType);
bool configureVideoOutput(MediaType newType, cv::uint32_t outFormat);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
bool setTime(double time, bool rough);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool setTime(int numberFrame);
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
bool configureHW(bool enable);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool configureStreams(const cv::VideoCaptureParameters&);
bool setAudioProperties(const cv::VideoCaptureParameters&);
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
bool checkAudioProperties();
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
template <typename CtrlT>
bool readComplexPropery(long prop, long& val) const;
template <typename CtrlT>
bool writeComplexProperty(long prop, double val, long flags);
_ComPtr<IMFAttributes> getDefaultSourceConfig(UINT32 num = 10);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool initStream(DWORD streamID, const MediaType mt);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
bool openFinalize_(const VideoCaptureParameters* params);
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
Media_Foundation& MF;
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
cv::String filename;
int camid;
MSMFCapture_Mode captureMode;
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
VideoAccelerationType va_type;
int hwDeviceIndex;
videoio(MSMF): backport WITH_MSMF_DXVA flag
2018-12-13 19:53:48 +08:00
#ifdef HAVE_MSMF_DXVA
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
_ComPtr<ID3D11Device> D3DDev;
_ComPtr<IMFDXGIDeviceManager> D3DMgr;
#endif
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
_ComPtr<IMFSourceReader> videoFileSource;
static analyzer issues
2021-03-03 17:10:00 +08:00
_ComPtr<IMFSourceReaderCallback> readCallback; // non-NULL for "live" streams (camera capture)
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
std::vector<DWORD> dwStreamIndices;
std::vector<_ComPtr<IMFSample>> audioSamples;
_ComPtr<IMFSample> impendingVideoSample;
_ComPtr<IMFSample> usedVideoSample;
DWORD dwVideoStreamIndex;
DWORD dwAudioStreamIndex;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
MediaType nativeFormat;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
MediaType captureVideoFormat;
MediaType captureAudioFormat;
bool device_status; //on or off
int videoStream; // look at CAP_PROP_VIDEO_STREAM
int audioStream; // look at CAP_PROP_AUDIO_STREAM
bool vEOS;
bool aEOS;
unsigned int audioBaseIndex;
int outputVideoFormat;
int outputAudioFormat;
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
UINT32 audioSamplesPerSecond;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
bool convertFormat;
MFTIME duration;
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
LONGLONG frameStep;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
LONGLONG nFrame;
LONGLONG impendingVideoSampleTime;
LONGLONG usedVideoSampleTime;
LONGLONG videoStartOffset;
LONGLONG videoSampleDuration;
LONGLONG requiredAudioTime;
LONGLONG audioSampleTime;
LONGLONG audioStartOffset;
LONGLONG audioSampleDuration;
LONGLONG audioTime;
LONGLONG chunkLengthOfBytes;
LONGLONG givenAudioTime;
LONGLONG numberOfAdditionalAudioBytes; // the number of additional bytes required to align the audio chunk
double bufferedAudioDuration;
LONGLONG audioSamplePos;
DWORD numberOfAudioStreams;
Mat audioFrame;
std::deque<BYTE> bufferAudioData;
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
bool isOpen;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool grabIsDone;
bool syncLastFrame;
bool lastFrame;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
};
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
CvCapture_MSMF::CvCapture_MSMF():
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
MF(Media_Foundation::getInstance()),
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
filename(""),
camid(-1),
captureMode(MODE_SW),
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
va_type(VIDEO_ACCELERATION_NONE),
hwDeviceIndex(-1),
videoio(MSMF): backport WITH_MSMF_DXVA flag
2018-12-13 19:53:48 +08:00
#ifdef HAVE_MSMF_DXVA
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
D3DDev(NULL),
D3DMgr(NULL),
#endif
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
videoFileSource(NULL),
static analyzer issues
2021-03-03 17:10:00 +08:00
readCallback(NULL),
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
impendingVideoSample(NULL),
usedVideoSample(NULL),
dwVideoStreamIndex(0),
dwAudioStreamIndex(0),
device_status(false),
videoStream(0),
audioStream(-1),
vEOS(false),
aEOS(false),
audioBaseIndex(1),
outputVideoFormat(CV_CAP_MODE_BGR),
outputAudioFormat(CV_16S),
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
audioSamplesPerSecond(0),
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
convertFormat(true),
static analyzer issues
2021-03-03 17:10:00 +08:00
duration(0),
frameStep(0),
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
nFrame(0),
impendingVideoSampleTime(0),
usedVideoSampleTime(0),
videoStartOffset(-1),
videoSampleDuration(0),
requiredAudioTime(0),
audioSampleTime(0),
audioStartOffset(-1),
audioSampleDuration(0),
audioTime(0),
chunkLengthOfBytes(0),
givenAudioTime(0),
numberOfAdditionalAudioBytes(0),
bufferedAudioDuration(0),
audioSamplePos(0),
numberOfAudioStreams(0),
isOpen(false),
grabIsDone(false),
syncLastFrame(true),
lastFrame(false)
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
{
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
CvCapture_MSMF::~CvCapture_MSMF()
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
close();
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
configureHW(false);
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
void CvCapture_MSMF::close()
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
if (isOpen)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
isOpen = false;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
usedVideoSample.Release();
for (auto item : audioSamples)
item.Release();
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
videoFileSource.Release();
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
device_status = false;
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
camid = -1;
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
filename.clear();
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
readCallback.Release();
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool CvCapture_MSMF::initStream(DWORD streamID, const MediaType mt)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
CV_LOG_DEBUG(NULL, "Init stream " << streamID << " with MediaType " << mt);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
_ComPtr<IMFMediaType> mediaTypesOut;
if (mt.majorType == MFMediaType_Audio)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
captureAudioFormat = mt;
mediaTypesOut = mt.createMediaType_Audio();
}
if (mt.majorType == MFMediaType_Video)
{
captureVideoFormat = mt;
mediaTypesOut = mt.createMediaType_Video();
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
if (FAILED(videoFileSource->SetStreamSelection(streamID, true)))
{
CV_LOG_WARNING(NULL, "Failed to select stream " << streamID);
return false;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
HRESULT hr = videoFileSource->SetCurrentMediaType(streamID, NULL, mediaTypesOut.Get());
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (hr == MF_E_TOPO_CODEC_NOT_FOUND)
{
CV_LOG_WARNING(NULL, "Failed to set mediaType (stream " << streamID << ", " << mt << "(codec not found)");
return false;
}
else if (hr == MF_E_INVALIDMEDIATYPE)
{
CV_LOG_WARNING(NULL, "Failed to set mediaType (stream " << streamID << ", " << mt << "(unsupported media type)");
return false;
}
else if (FAILED(hr))
{
CV_LOG_WARNING(NULL, "Failed to set mediaType (stream " << streamID << ", " << mt << "(HRESULT " << hr << ")");
return false;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return true;
}
_ComPtr<IMFAttributes> CvCapture_MSMF::getDefaultSourceConfig(UINT32 num)
{
CV_Assert(num > 0);
Merge pull request #20327 from tibaes:MSMF-Slow-Webcam-Startup * fixes MSMF slow webcam startup * add variable to change MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS at runtime
2021-07-01 06:08:24 +08:00
const bool OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS = utils::getConfigurationParameterBool("OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS", true);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
_ComPtr<IMFAttributes> res;
if (FAILED(MFCreateAttributes(&res, num)) ||
Merge pull request #20327 from tibaes:MSMF-Slow-Webcam-Startup * fixes MSMF slow webcam startup * add variable to change MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS at runtime
2021-07-01 06:08:24 +08:00
FAILED(res->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS)) ||
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
FAILED(res->SetUINT32(MF_SOURCE_READER_DISABLE_DXVA, false)) ||
FAILED(res->SetUINT32(MF_SOURCE_READER_ENABLE_VIDEO_PROCESSING, false)) ||
FAILED(res->SetUINT32(MF_SOURCE_READER_ENABLE_ADVANCED_VIDEO_PROCESSING, true))
)
{
CV_Error(CV_StsError, "Failed to create attributes");
}
#ifdef HAVE_MSMF_DXVA
if (D3DMgr)
{
if (FAILED(res->SetUnknown(MF_SOURCE_READER_D3D_MANAGER, D3DMgr.Get())))
{
CV_Error(CV_StsError, "Failed to create attributes");
}
}
#endif
return res;
}
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
bool CvCapture_MSMF::configureHW(bool enable)
{
videoio(MSMF): backport WITH_MSMF_DXVA flag
2018-12-13 19:53:48 +08:00
#ifdef HAVE_MSMF_DXVA
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
if ((enable && D3DMgr && D3DDev) || (!enable && !D3DMgr && !D3DDev))
return true;
videoio(MSMF): avoid OpenCV load failure on Win7 machines OpenCV binaries are compiled on Win10 environment
2018-06-30 03:54:58 +08:00
if (!pMFCreateDXGIDeviceManager_initialized)
init_MFCreateDXGIDeviceManager();
if (enable && !pMFCreateDXGIDeviceManager)
return false;
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
bool reopen = isOpen;
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
int prevcam = camid;
cv::String prevfile = filename;
close();
if (enable)
{
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
_ComPtr<IDXGIAdapter> pAdapter;
if (hwDeviceIndex >= 0) {
_ComPtr<IDXGIFactory2> pDXGIFactory;
if (FAILED(CreateDXGIFactory(__uuidof(IDXGIFactory2), (void**)& pDXGIFactory)) ||
FAILED(pDXGIFactory->EnumAdapters(hwDeviceIndex, &pAdapter))) {
return false;
}
}
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
D3D_FEATURE_LEVEL levels[] = { D3D_FEATURE_LEVEL_11_1, D3D_FEATURE_LEVEL_11_0,
D3D_FEATURE_LEVEL_10_1, D3D_FEATURE_LEVEL_10_0,
D3D_FEATURE_LEVEL_9_3, D3D_FEATURE_LEVEL_9_2, D3D_FEATURE_LEVEL_9_1 };
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
D3D_DRIVER_TYPE driverType = pAdapter ? D3D_DRIVER_TYPE_UNKNOWN : D3D_DRIVER_TYPE_HARDWARE;
if (SUCCEEDED(D3D11CreateDevice(pAdapter.Get(), driverType, NULL, D3D11_CREATE_DEVICE_BGRA_SUPPORT | D3D11_CREATE_DEVICE_VIDEO_SUPPORT,
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
levels, sizeof(levels) / sizeof(*levels), D3D11_SDK_VERSION, &D3DDev, NULL, NULL)))
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
{
// NOTE: Getting ready for multi-threaded operation
_ComPtr<ID3D11Multithread> D3DDevMT;
UINT mgrRToken;
if (SUCCEEDED(D3DDev->QueryInterface(IID_PPV_ARGS(&D3DDevMT))))
{
D3DDevMT->SetMultithreadProtected(TRUE);
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
D3DDevMT.Release();
videoio(MSMF): avoid OpenCV load failure on Win7 machines OpenCV binaries are compiled on Win10 environment
2018-06-30 03:54:58 +08:00
if (SUCCEEDED(pMFCreateDXGIDeviceManager(&mgrRToken, &D3DMgr)))
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
{
if (SUCCEEDED(D3DMgr->ResetDevice(D3DDev.Get(), mgrRToken)))
{
captureMode = MODE_HW;
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
if (hwDeviceIndex < 0)
hwDeviceIndex = 0;
// Log adapter description
_ComPtr<IDXGIDevice> dxgiDevice;
if (SUCCEEDED(D3DDev->QueryInterface(__uuidof(IDXGIDevice), reinterpret_cast<void**>(&dxgiDevice)))) {
_ComPtr<IDXGIAdapter> adapter;
if (SUCCEEDED(dxgiDevice->GetAdapter(&adapter))) {
DXGI_ADAPTER_DESC desc;
if (SUCCEEDED(adapter->GetDesc(&desc))) {
std::wstring name(desc.Description);
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> conv;
CV_LOG_INFO(NULL, "MSMF: Using D3D11 video acceleration on GPU device: " << conv.to_bytes(name));
}
}
}
// Reopen if needed
return reopen ? (prevcam >= 0 ? open(prevcam, NULL) : open(prevfile.c_str(), NULL)) : true;
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
}
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
D3DMgr.Release();
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
}
}
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
D3DDev.Release();
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
}
return false;
}
else
{
if (D3DMgr)
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
D3DMgr.Release();
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
if (D3DDev)
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
D3DDev.Release();
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
captureMode = MODE_SW;
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
return reopen ? (prevcam >= 0 ? open(prevcam, NULL) : open(prevfile.c_str(), NULL)) : true;
Added DXVA hardware acceleration to MSMF-based VideoCapture
2018-04-23 21:48:57 +08:00
}
#else
return !enable;
#endif
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
bool CvCapture_MSMF::configureHW(const VideoCaptureParameters& params)
{
va_type = params.get<VideoAccelerationType>(CAP_PROP_HW_ACCELERATION, VIDEO_ACCELERATION_ANY);
hwDeviceIndex = params.get<int>(CAP_PROP_HW_DEVICE, -1);
#ifndef HAVE_MSMF_DXVA
if (va_type != VIDEO_ACCELERATION_NONE && va_type != VIDEO_ACCELERATION_ANY)
{
CV_LOG_INFO(NULL, "VIDEOIO/MSMF: MSMF backend is build without DXVA acceleration support. Can't handle CAP_PROP_HW_ACCELERATION parameter: " << va_type);
}
#endif
return configureHW(va_type == VIDEO_ACCELERATION_D3D11 || va_type == VIDEO_ACCELERATION_ANY);
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool CvCapture_MSMF::configureAudioOutput(MediaType newType)
{
FormatStorage formats;
formats.read(videoFileSource.Get());
std::pair<FormatStorage::MediaID, MediaType> bestMatch;
formats.countNumberOfAudioStreams(numberOfAudioStreams);
if (device_status)
bestMatch = formats.findBestAudioFormat(newType);
else
bestMatch = formats.findAudioFormatByStream(audioStream);
if (bestMatch.second.isEmpty(true))
{
CV_LOG_DEBUG(NULL, "Can not find audio stream with requested parameters");
Merge pull request #20942 from MaximMilashchenko:AudioPatch Audio patch * fixed microphone, audio position * fixed docs * changed AudioOpenCheck
2021-10-26 22:33:53 +08:00
isOpen = false;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return false;
}
dwAudioStreamIndex = bestMatch.first.stream;
dwStreamIndices.push_back(dwAudioStreamIndex);
MediaType newFormat = bestMatch.second;
newFormat.majorType = MFMediaType_Audio;
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
newFormat.nSamplesPerSec = (audioSamplesPerSecond == 0) ? 44100 : audioSamplesPerSecond;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
switch (outputAudioFormat)
{
case CV_8S:
newFormat.subType = MFAudioFormat_PCM;
newFormat.bit_per_sample = 8;
break;
case CV_16S:
newFormat.subType = MFAudioFormat_PCM;
newFormat.bit_per_sample = 16;
break;
case CV_32S:
newFormat.subType = MFAudioFormat_PCM;
newFormat.bit_per_sample = 32;
case CV_32F:
newFormat.subType = MFAudioFormat_Float;
newFormat.bit_per_sample = 32;
break;
default:
break;
}
return initStream(dwAudioStreamIndex, newFormat);
}
bool CvCapture_MSMF::configureVideoOutput(MediaType newType, cv::uint32_t outFormat)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
FormatStorage formats;
formats.read(videoFileSource.Get());
videoio/MSMF: fixed audio stream handling
2020-05-27 22:52:17 +08:00
std::pair<FormatStorage::MediaID, MediaType> bestMatch = formats.findBestVideoFormat(newType);
if (bestMatch.second.isEmpty())
{
CV_LOG_DEBUG(NULL, "Can not find video stream with requested parameters");
return false;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
dwVideoStreamIndex = bestMatch.first.stream;
dwStreamIndices.push_back(dwVideoStreamIndex);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
nativeFormat = bestMatch.second;
MediaType newFormat = nativeFormat;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
videoio: fixed conversion in MSMF backend
2020-04-01 19:40:11 +08:00
if (convertFormat)
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
switch (outFormat)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
case CV_CAP_MODE_BGR:
case CV_CAP_MODE_RGB:
newFormat.subType = captureMode == MODE_HW ? MFVideoFormat_RGB32 : MFVideoFormat_RGB24;
newFormat.stride = (captureMode == MODE_HW ? 4 : 3) * newFormat.width;
newFormat.sampleSize = newFormat.stride * newFormat.height;
break;
case CV_CAP_MODE_GRAY:
newFormat.subType = MFVideoFormat_YUY2;
newFormat.stride = newFormat.width;
newFormat.sampleSize = newFormat.stride * newFormat.height * 3 / 2;
break;
case CV_CAP_MODE_YUYV:
newFormat.subType = MFVideoFormat_YUY2;
newFormat.stride = 2 * newFormat.width;
newFormat.sampleSize = newFormat.stride * newFormat.height;
break;
default:
return false;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
newFormat.interlaceMode = MFVideoInterlace_Progressive;
newFormat.isFixedSize = true;
if (nativeFormat.subType == MFVideoFormat_MP43) //Unable to estimate FPS for MP43
newFormat.frameRateNum = 0;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
// we select native format first and then our requested format (related issue #12822)
if (!newType.isEmpty()) // camera input
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
{
initStream(dwVideoStreamIndex, nativeFormat);
}
return initStream(dwVideoStreamIndex, newFormat);
}
bool CvCapture_MSMF::configureOutput()
{
if (FAILED(videoFileSource->SetStreamSelection((DWORD)MF_SOURCE_READER_ALL_STREAMS, false)))
{
CV_LOG_WARNING(NULL, "Failed to reset streams");
return false;
}
bool tmp = true;
if (videoStream != -1)
tmp = (!device_status)? configureVideoOutput(MediaType(), outputVideoFormat) : configureVideoOutput(MediaType::createDefault_Video(), outputVideoFormat);
if (audioStream != -1)
tmp &= (!device_status)? configureAudioOutput(MediaType()) : configureAudioOutput(MediaType::createDefault_Audio());
return tmp;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
bool CvCapture_MSMF::open(int index, const cv::VideoCaptureParameters* params)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
close();
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (index < 0)
Disable MSMF VideoCapture fallback to existing camera in case provided index is out of range.
2018-06-26 19:04:02 +08:00
return false;
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
if (params)
{
configureHW(*params);
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
if (!(configureStreams(*params) && setAudioProperties(*params)))
return false;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
}
if (videoStream != -1 && audioStream != -1 || videoStream == -1 && audioStream == -1)
{
CV_LOG_DEBUG(NULL, "Only one of the properties CAP_PROP_AUDIO_STREAM " << audioStream << " and " << CAP_PROP_VIDEO_STREAM << " must be different from -1");
return false;
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
DeviceList devices;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
UINT32 count = 0;
if (audioStream != -1)
count = devices.read(MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_AUDCAP_GUID);
if (videoStream != -1)
count = devices.read(MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_GUID);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (count == 0 || static_cast<UINT32>(index) > count)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
CV_LOG_DEBUG(NULL, "Device " << index << " not found (total " << count << " devices)");
return false;
}
_ComPtr<IMFAttributes> attr = getDefaultSourceConfig();
_ComPtr<IMFSourceReaderCallback> cb = new SourceReaderCB();
attr->SetUnknown(MF_SOURCE_READER_ASYNC_CALLBACK, cb.Get());
_ComPtr<IMFMediaSource> src = devices.activateSource(index);
if (!src.Get() || FAILED(MFCreateSourceReaderFromMediaSource(src.Get(), attr.Get(), &videoFileSource)))
{
CV_LOG_DEBUG(NULL, "Failed to create source reader");
return false;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
isOpen = true;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
device_status = true;
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
camid = index;
readCallback = cb;
duration = 0;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (configureOutput())
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
frameStep = captureVideoFormat.getFrameStep();
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
if (isOpen && !openFinalize_(params))
{
close();
return false;
}
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
if (isOpen)
{
if (audioStream != -1)
if (!checkAudioProperties())
return false;
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
return isOpen;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
bool CvCapture_MSMF::open(const cv::String& _filename, const cv::VideoCaptureParameters* params)
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
{
close();
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
if (_filename.empty())
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
return false;
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
if (params)
{
configureHW(*params);
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
if (!(configureStreams(*params) && setAudioProperties(*params)))
return false;
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
}
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
// Set source reader parameters
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
_ComPtr<IMFAttributes> attr = getDefaultSourceConfig();
cv::AutoBuffer<wchar_t> unicodeFileName(_filename.length() + 1);
MultiByteToWideChar(CP_ACP, 0, _filename.c_str(), -1, unicodeFileName.data(), (int)_filename.length() + 1);
if (SUCCEEDED(MFCreateSourceReaderFromURL(unicodeFileName.data(), attr.Get(), &videoFileSource)))
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
isOpen = true;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
usedVideoSampleTime = 0;
if (configureOutput())
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
filename = _filename;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
frameStep = captureVideoFormat.getFrameStep();
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
PROPVARIANT var;
HRESULT hr;
if (SUCCEEDED(hr = videoFileSource->GetPresentationAttribute((DWORD)MF_SOURCE_READER_MEDIASOURCE, MF_PD_DURATION, &var)) &&
var.vt == VT_UI8)
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
duration = var.uhVal.QuadPart;
PropVariantClear(&var);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
else
duration = 0;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
}
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
if (isOpen && !openFinalize_(params))
{
close();
return false;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (isOpen)
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
{
if (audioStream != -1)
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
{
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
if (!checkAudioProperties())
return false;
if (videoStream != -1)
{
isOpen = grabFrame();
if (isOpen)
grabIsDone = true;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
}
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
}
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
return isOpen;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
bool CvCapture_MSMF::openFinalize_(const VideoCaptureParameters* params)
{
if (params)
{
std::vector<int> unused_params = params->getUnused();
for (int key : unused_params)
{
if (!setProperty(key, params->get<double>(key)))
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: can't set property " << key);
return false;
}
}
}
VideoAccelerationType actual_va_type = (captureMode == MODE_HW) ? VIDEO_ACCELERATION_D3D11 : VIDEO_ACCELERATION_NONE;
if (va_type != VIDEO_ACCELERATION_NONE && va_type != VIDEO_ACCELERATION_ANY)
{
if (va_type != actual_va_type)
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: Can't select requested video acceleration through CAP_PROP_HW_ACCELERATION: "
<< va_type << " (actual is " << actual_va_type << "). Bailout");
return false;
}
}
else
{
va_type = actual_va_type;
}
return true;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool CvCapture_MSMF::configureStreams(const cv::VideoCaptureParameters& params)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (params.has(CAP_PROP_VIDEO_STREAM))
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
double value = params.get<double>(CAP_PROP_VIDEO_STREAM);
if (value == -1 || value == 0)
videoStream = static_cast<int>(value);
else
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: CAP_PROP_VIDEO_STREAM parameter value is invalid/unsupported: " << value);
return false;
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
}
if (params.has(CAP_PROP_AUDIO_STREAM))
{
double value = params.get<double>(CAP_PROP_AUDIO_STREAM);
if (value == -1 || value > -1)
audioStream = static_cast<int>(value);
else
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: CAP_PROP_AUDIO_STREAM parameter value is invalid/unsupported: " << value);
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
return false;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
}
return true;
}
bool CvCapture_MSMF::setAudioProperties(const cv::VideoCaptureParameters& params)
{
if (params.has(CAP_PROP_AUDIO_DATA_DEPTH))
{
int value = static_cast<int>(params.get<double>(CAP_PROP_AUDIO_DATA_DEPTH));
if (value != CV_8S && value != CV_16S && value != CV_32S && value != CV_32F)
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: CAP_PROP_AUDIO_DATA_DEPTH parameter value is invalid/unsupported: " << value);
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
return false;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
else
{
outputAudioFormat = value;
}
videoio: MSMF async reader for camera stream Synchronized reading from camera with heavy frame processing provides bad effects (huge frame latency, processing frames from the past). Generally, there is no way to process each frame and some frames will be dropped. Allow preventive frame dropping to reduce lag of processed frames. This mode is applied to cameras only (opened by 'index').
2018-06-13 17:47:53 +08:00
}
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
if (params.has(CAP_PROP_AUDIO_SAMPLES_PER_SECOND))
{
int value = static_cast<int>(params.get<double>(CAP_PROP_AUDIO_SAMPLES_PER_SECOND));
if (value < 0)
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: CAP_PROP_AUDIO_SAMPLES_PER_SECOND parameter can't be negative: " << value);
return false;
}
else
{
audioSamplesPerSecond = value;
}
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (params.has(CAP_PROP_AUDIO_SYNCHRONIZE))
{
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
int value = static_cast<UINT32>(params.get<double>(CAP_PROP_AUDIO_SYNCHRONIZE));
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
syncLastFrame = (value != 0) ? true : false;
}
return true;
}
Merge pull request #21145 from MaximMilashchenko:AudioUpdate Audio MSMF: added the ability to set sample per second * Audio MSMF: added the ability to set sample per second * changed the valid sampling rate check * fixed docs * add test * fixed warning * fixed error * fixed error
2021-12-04 18:37:10 +08:00
bool CvCapture_MSMF::checkAudioProperties()
{
if (audioSamplesPerSecond != 0)
{
_ComPtr<IMFMediaType> type;
UINT32 actualAudioSamplesPerSecond = 0;
HRESULT hr = videoFileSource->GetCurrentMediaType(dwAudioStreamIndex, &type);
if (SUCCEEDED(hr))
{
type->GetUINT32(MF_MT_AUDIO_SAMPLES_PER_SECOND , &actualAudioSamplesPerSecond);
if (actualAudioSamplesPerSecond != audioSamplesPerSecond)
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: CAP_PROP_AUDIO_SAMPLES_PER_SECOND parameter value is invalid/unsupported: " << audioSamplesPerSecond
<< ". Current value of CAP_PROP_AUDIO_SAMPLES_PER_SECOND: " << actualAudioSamplesPerSecond);
close();
return false;
}
return true;
}
return false;
}
return true;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool CvCapture_MSMF::grabVideoFrame()
{
DWORD streamIndex, flags;
HRESULT hr;
usedVideoSample.Release();
bool returnFlag = false;
bool stopFlag = false;
if (audioStream != -1)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
usedVideoSample.swap(impendingVideoSample);
std::swap(usedVideoSampleTime, impendingVideoSampleTime);
}
while (!stopFlag)
{
for (;;)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
CV_TRACE_REGION("ReadSample");
if (!SUCCEEDED(hr = videoFileSource->ReadSample(
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
dwVideoStreamIndex, // Stream index.
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
0, // Flags.
&streamIndex, // Receives the actual stream index.
&flags, // Receives status flags.
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
&impendingVideoSampleTime, // Receives the time stamp.
&impendingVideoSample // Receives the sample or NULL.
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
)))
break;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (streamIndex != dwVideoStreamIndex)
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
if (flags & (MF_SOURCE_READERF_ERROR | MF_SOURCE_READERF_ALLEFFECTSREMOVED | MF_SOURCE_READERF_ENDOFSTREAM))
break;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (impendingVideoSample)
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
if (flags & MF_SOURCE_READERF_STREAMTICK)
{
videoio(msmf): replace custom debug print function
2018-06-09 22:38:32 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream tick detected. Retrying to grab the frame");
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
}
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (SUCCEEDED(hr))
{
if (streamIndex != dwVideoStreamIndex)
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): Wrong stream read. Abort capturing");
close();
}
else if (flags & MF_SOURCE_READERF_ERROR)
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream reading error. Abort capturing");
close();
}
else if (flags & MF_SOURCE_READERF_ALLEFFECTSREMOVED)
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream decoding error. Abort capturing");
close();
}
else if (flags & MF_SOURCE_READERF_ENDOFSTREAM)
{
vEOS = true;
lastFrame = true;
stopFlag = true;
if (audioStream == -1)
returnFlag = false;
else if (usedVideoSample)
returnFlag = true;
CV_LOG_DEBUG(NULL, "videoio(MSMF): End of video stream detected");
}
else
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): got video frame with timestamp=" << impendingVideoSampleTime);
if (audioStream != -1)
{
if (!usedVideoSample)
{
usedVideoSample.swap(impendingVideoSample);
std::swap(usedVideoSampleTime, impendingVideoSampleTime);
videoStartOffset = usedVideoSampleTime;
}
else
{
stopFlag = true;
}
if (impendingVideoSample)
{
nFrame++;
videoSampleDuration = impendingVideoSampleTime - usedVideoSampleTime;
requiredAudioTime = impendingVideoSampleTime - givenAudioTime;
givenAudioTime += requiredAudioTime;
}
}
else
{
usedVideoSample.swap(impendingVideoSample);
std::swap(usedVideoSampleTime, impendingVideoSampleTime);
stopFlag = true;
nFrame++;
}
if (flags & MF_SOURCE_READERF_NEWSTREAM)
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): New stream detected");
}
if (flags & MF_SOURCE_READERF_NATIVEMEDIATYPECHANGED)
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream native media type changed");
}
if (flags & MF_SOURCE_READERF_CURRENTMEDIATYPECHANGED)
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream current media type changed");
}
returnFlag = true;
}
}
}
return returnFlag;
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Merge pull request #20942 from MaximMilashchenko:AudioPatch Audio patch * fixed microphone, audio position * fixed docs * changed AudioOpenCheck
2021-10-26 22:33:53 +08:00
bool CvCapture_MSMF::configureAudioFrame()
{
if (!audioSamples.empty() || !bufferAudioData.empty() && aEOS)
{
_ComPtr<IMFMediaBuffer> buf = NULL;
std::vector<BYTE> audioDataInUse;
BYTE* ptr = NULL;
DWORD maxsize = 0, cursize = 0;
CV_TRACE_REGION("get_contiguous_buffer");
for (auto item : audioSamples)
{
if (!SUCCEEDED(item->ConvertToContiguousBuffer(&buf)))
{
CV_TRACE_REGION("get_buffer");
DWORD bcnt = 0;
if (!SUCCEEDED(item->GetBufferCount(&bcnt)))
break;
if (bcnt == 0)
break;
if (!SUCCEEDED(item->GetBufferByIndex(0, &buf)))
break;
}
if (!SUCCEEDED(buf->Lock(&ptr, &maxsize, &cursize)))
break;
size_t lastSize = bufferAudioData.size();
bufferAudioData.resize(lastSize+cursize);
for (unsigned int i = 0; i < cursize; i++)
{
bufferAudioData[lastSize+i]=*(ptr+i);
}
CV_TRACE_REGION_NEXT("unlock");
buf->Unlock();
buf = NULL;
}
audioSamples.clear();
audioSamplePos += chunkLengthOfBytes/((captureAudioFormat.bit_per_sample/8)*captureAudioFormat.nChannels);
chunkLengthOfBytes = (videoStream != -1) ? (LONGLONG)((requiredAudioTime*captureAudioFormat.nSamplesPerSec*captureAudioFormat.nChannels*(captureAudioFormat.bit_per_sample)/8)/1e7) : cursize;
if ((videoStream != -1) && (chunkLengthOfBytes % ((int)(captureAudioFormat.bit_per_sample)/8* (int)captureAudioFormat.nChannels) != 0))
{
if ( (double)audioSamplePos/captureAudioFormat.nSamplesPerSec + audioStartOffset * 1e-7 - usedVideoSampleTime * 1e-7 >= 0 )
chunkLengthOfBytes -= numberOfAdditionalAudioBytes;
numberOfAdditionalAudioBytes = ((int)(captureAudioFormat.bit_per_sample)/8* (int)captureAudioFormat.nChannels)
- chunkLengthOfBytes % ((int)(captureAudioFormat.bit_per_sample)/8* (int)captureAudioFormat.nChannels);
chunkLengthOfBytes += numberOfAdditionalAudioBytes;
}
if (lastFrame && !syncLastFrame || aEOS && !vEOS)
{
chunkLengthOfBytes = bufferAudioData.size();
audioSamplePos += chunkLengthOfBytes/((captureAudioFormat.bit_per_sample/8)*captureAudioFormat.nChannels);
}
CV_Check((double)chunkLengthOfBytes, chunkLengthOfBytes >= INT_MIN || chunkLengthOfBytes <= INT_MAX, "MSMF: The chunkLengthOfBytes is out of the allowed range");
copy(bufferAudioData.begin(), bufferAudioData.begin() + (int)chunkLengthOfBytes, std::back_inserter(audioDataInUse));
bufferAudioData.erase(bufferAudioData.begin(), bufferAudioData.begin() + (int)chunkLengthOfBytes);
if (audioFrame.empty())
{
switch (outputAudioFormat)
{
case CV_8S:
cv::Mat((int)chunkLengthOfBytes/(captureAudioFormat.nChannels), captureAudioFormat.nChannels, CV_8S, audioDataInUse.data()).copyTo(audioFrame);
break;
case CV_16S:
cv::Mat((int)chunkLengthOfBytes/(2*captureAudioFormat.nChannels), captureAudioFormat.nChannels, CV_16S, audioDataInUse.data()).copyTo(audioFrame);
break;
case CV_32S:
cv::Mat((int)chunkLengthOfBytes/(4*captureAudioFormat.nChannels), captureAudioFormat.nChannels, CV_32S, audioDataInUse.data()).copyTo(audioFrame);
break;
case CV_32F:
cv::Mat((int)chunkLengthOfBytes/(4*captureAudioFormat.nChannels), captureAudioFormat.nChannels, CV_32F, audioDataInUse.data()).copyTo(audioFrame);
break;
default:
break;
}
}
audioDataInUse.clear();
audioDataInUse.shrink_to_fit();
return true;
}
else
{
return false;
}
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool CvCapture_MSMF::grabAudioFrame()
{
DWORD streamIndex, flags;
HRESULT hr;
_ComPtr<IMFSample> audioSample = NULL;
audioSamples.clear();
bool returnFlag = false;
audioTime = 0;
int numberOfSamples = -1;
if (bufferedAudioDuration*1e7 > requiredAudioTime)
return true;
while ((!vEOS) ? audioTime <= requiredAudioTime : !aEOS)
{
if (audioStartOffset - usedVideoSampleTime > videoSampleDuration)
return true;
for (;;)
{
CV_TRACE_REGION("ReadSample");
if (!SUCCEEDED(hr = videoFileSource->ReadSample(
dwAudioStreamIndex, // Stream index.
0, // Flags.
&streamIndex, // Receives the actual stream index.
&flags, // Receives status flags.
&audioSampleTime, // Receives the time stamp.
&audioSample // Receives the sample or NULL.
)))
break;
if (streamIndex != dwAudioStreamIndex)
break;
if (flags & (MF_SOURCE_READERF_ERROR | MF_SOURCE_READERF_ALLEFFECTSREMOVED | MF_SOURCE_READERF_ENDOFSTREAM))
break;
if (audioSample)
break;
if (flags & MF_SOURCE_READERF_STREAMTICK)
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream tick detected. Retrying to grab the frame");
}
}
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
if (SUCCEEDED(hr))
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (streamIndex != dwAudioStreamIndex)
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
{
Fix modules/ typos Found using `codespell -q 3 -S ./3rdparty -L activ,amin,ang,atleast,childs,dof,endwhile,halfs,hist,iff,nd,od,uint`
2019-08-16 06:02:09 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): Wrong stream read. Abort capturing");
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
close();
}
else if (flags & MF_SOURCE_READERF_ERROR)
{
videoio(msmf): replace custom debug print function
2018-06-09 22:38:32 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream reading error. Abort capturing");
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
close();
}
else if (flags & MF_SOURCE_READERF_ALLEFFECTSREMOVED)
{
videoio(msmf): replace custom debug print function
2018-06-09 22:38:32 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream decoding error. Abort capturing");
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
close();
}
else if (flags & MF_SOURCE_READERF_ENDOFSTREAM)
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
aEOS = true;
if (videoStream != -1 && !vEOS)
returnFlag = true;
Merge pull request #20942 from MaximMilashchenko:AudioPatch Audio patch * fixed microphone, audio position * fixed docs * changed AudioOpenCheck
2021-10-26 22:33:53 +08:00
if (videoStream == -1)
audioSamplePos += chunkLengthOfBytes/((captureAudioFormat.bit_per_sample/8)*captureAudioFormat.nChannels);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): End of audio stream detected");
break;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
}
else
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
audioSamples.push_back(audioSample);
audioSample = NULL;
numberOfSamples++;
audioSamples[numberOfSamples]->GetSampleDuration(&audioSampleDuration);
CV_LOG_DEBUG(NULL, "videoio(MSMF): got audio frame with timestamp=" << audioSampleTime << " duration=" << audioSampleDuration);
audioTime += (LONGLONG)(audioSampleDuration + bufferedAudioDuration*1e7);
if (nFrame == 1 && audioStartOffset == -1)
{
audioStartOffset = audioSampleTime - audioSampleDuration;
requiredAudioTime -= audioStartOffset;
}
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
if (flags & MF_SOURCE_READERF_NEWSTREAM)
Code review notes fixed.
2013-04-03 18:48:23 +08:00
{
videoio(msmf): replace custom debug print function
2018-06-09 22:38:32 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): New stream detected");
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
if (flags & MF_SOURCE_READERF_NATIVEMEDIATYPECHANGED)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio(msmf): replace custom debug print function
2018-06-09 22:38:32 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream native media type changed");
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
if (flags & MF_SOURCE_READERF_CURRENTMEDIATYPECHANGED)
{
videoio(msmf): replace custom debug print function
2018-06-09 22:38:32 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): Stream current media type changed");
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
returnFlag = true;
}
}
else
{
CV_LOG_DEBUG(NULL, "videoio(MSMF): ReadSample() method is not succeeded");
return false;
}
}
Merge pull request #20942 from MaximMilashchenko:AudioPatch Audio patch * fixed microphone, audio position * fixed docs * changed AudioOpenCheck
2021-10-26 22:33:53 +08:00
returnFlag &= configureAudioFrame();
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return returnFlag;
}
bool CvCapture_MSMF::grabFrame()
{
CV_TRACE_FUNCTION();
if (grabIsDone)
{
grabIsDone = false;
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): return pre-grabbed frame " << usedVideoSampleTime);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return true;
}
audioFrame = Mat();
if (readCallback) // async "live" capture mode
{
audioSamples.push_back(NULL);
HRESULT hr = 0;
SourceReaderCB* reader = ((SourceReaderCB*)readCallback.Get());
DWORD dwStreamIndex = 0;
if (videoStream != -1)
dwStreamIndex = dwVideoStreamIndex;
if (audioStream != -1)
dwStreamIndex = dwAudioStreamIndex;
if (!reader->m_reader)
{
// Initiate capturing with async callback
reader->m_reader = videoFileSource.Get();
reader->m_dwStreamIndex = dwStreamIndex;
if (FAILED(hr = videoFileSource->ReadSample(dwStreamIndex, 0, NULL, NULL, NULL, NULL)))
{
CV_LOG_ERROR(NULL, "videoio(MSMF): can't grab frame - initial async ReadSample() call failed: " << hr);
reader->m_reader = NULL;
return false;
Code review notes fixed.
2013-04-03 18:48:23 +08:00
}
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
BOOL bEOS = false;
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
LONGLONG timestamp = 0;
if (FAILED(hr = reader->Wait( videoStream == -1 ? INFINITE : 10000, (videoStream != -1) ? usedVideoSample : audioSamples[0], timestamp, bEOS))) // 10 sec
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
{
CV_LOG_WARNING(NULL, "videoio(MSMF): can't grab frame. Error: " << hr);
return false;
}
if (bEOS)
{
CV_LOG_WARNING(NULL, "videoio(MSMF): EOS signal. Capture stream is lost");
return false;
}
if (videoStream != -1)
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
usedVideoSampleTime = timestamp;
Merge pull request #20942 from MaximMilashchenko:AudioPatch Audio patch * fixed microphone, audio position * fixed docs * changed AudioOpenCheck
2021-10-26 22:33:53 +08:00
if (audioStream != -1)
return configureAudioFrame();
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): grabbed frame " << usedVideoSampleTime);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return true;
}
else if (isOpen)
{
if (vEOS)
return false;
bool returnFlag = true;
if (videoStream != -1)
{
if (!vEOS)
returnFlag &= grabVideoFrame();
if (!returnFlag)
return false;
}
if (audioStream != -1)
{
bufferedAudioDuration = (double)(bufferAudioData.size()/((captureAudioFormat.bit_per_sample/8)*captureAudioFormat.nChannels))/captureAudioFormat.nSamplesPerSec;
audioFrame.release();
if (!aEOS)
returnFlag &= grabAudioFrame();
}
return returnFlag;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
return false;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool CvCapture_MSMF::retrieveVideoFrame(cv::OutputArray frame)
Code review notes fixed.
2013-04-03 18:48:23 +08:00
{
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
CV_TRACE_FUNCTION();
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): retrieve video frame start...");
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
do
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (!usedVideoSample)
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
_ComPtr<IMFMediaBuffer> buf = NULL;
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
CV_TRACE_REGION("get_contiguous_buffer");
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (!SUCCEEDED(usedVideoSample->ConvertToContiguousBuffer(&buf)))
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
{
CV_TRACE_REGION("get_buffer");
DWORD bcnt = 0;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (!SUCCEEDED(usedVideoSample->GetBufferCount(&bcnt)))
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
if (bcnt == 0)
break;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (!SUCCEEDED(usedVideoSample->GetBufferByIndex(0, &buf)))
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
}
bool lock2d = false;
BYTE* ptr = NULL;
LONG pitch = 0;
DWORD maxsize = 0, cursize = 0;
// "For 2-D buffers, the Lock2D method is more efficient than the Lock method"
// see IMFMediaBuffer::Lock method documentation: https://msdn.microsoft.com/en-us/library/windows/desktop/bb970366(v=vs.85).aspx
_ComPtr<IMF2DBuffer> buffer2d;
if (convertFormat)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
if (SUCCEEDED(buf.As<IMF2DBuffer>(buffer2d)))
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
CV_TRACE_REGION_NEXT("lock2d");
if (SUCCEEDED(buffer2d->Lock2D(&ptr, &pitch)))
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
lock2d = true;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
}
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
}
}
if (ptr == NULL)
{
CV_Assert(lock2d == false);
CV_TRACE_REGION_NEXT("lock");
if (!SUCCEEDED(buf->Lock(&ptr, &maxsize, &cursize)))
{
break;
}
}
if (!ptr)
break;
if (convertFormat)
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (lock2d || (unsigned int)cursize == captureVideoFormat.sampleSize)
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
switch (outputVideoFormat)
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
{
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
case CV_CAP_MODE_YUYV:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
cv::Mat(captureVideoFormat.height, captureVideoFormat.width, CV_8UC2, ptr, pitch).copyTo(frame);
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
case CV_CAP_MODE_BGR:
if (captureMode == MODE_HW)
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
cv::cvtColor(cv::Mat(captureVideoFormat.height, captureVideoFormat.width, CV_8UC4, ptr, pitch), frame, cv::COLOR_BGRA2BGR);
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
else
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
cv::Mat(captureVideoFormat.height, captureVideoFormat.width, CV_8UC3, ptr, pitch).copyTo(frame);
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
case CV_CAP_MODE_RGB:
if (captureMode == MODE_HW)
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
cv::cvtColor(cv::Mat(captureVideoFormat.height, captureVideoFormat.width, CV_8UC4, ptr, pitch), frame, cv::COLOR_BGRA2BGR);
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
else
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
cv::cvtColor(cv::Mat(captureVideoFormat.height, captureVideoFormat.width, CV_8UC3, ptr, pitch), frame, cv::COLOR_BGR2RGB);
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
case CV_CAP_MODE_GRAY:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
cv::Mat(captureVideoFormat.height, captureVideoFormat.width, CV_8UC1, ptr, pitch).copyTo(frame);
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
break;
default:
frame.release();
break;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
}
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
else
frame.release();
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
else
{
cv::Mat(1, cursize, CV_8UC1, ptr, pitch).copyTo(frame);
}
CV_TRACE_REGION_NEXT("unlock");
if (lock2d)
buffer2d->Unlock2D();
else
buf->Unlock();
videoio(MSMF): add queue for async ReadSample()
2021-12-06 14:54:48 +08:00
CV_LOG_DEBUG(NULL, "videoio(MSMF): retrieve video frame done!");
videoio(MSMF): refactor grabFrame() / retrieveFrame() methods - no behavior changes in grabFrame() - added Lock2D() support into retrieveFrame()
2018-06-08 22:06:45 +08:00
return !frame.empty();
} while (0);
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
frame.release();
return false;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation based VideoCapture improved Code formating fixed; GrabFrame method implemented correclty.
2013-04-16 21:25:10 +08:00
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool CvCapture_MSMF::retrieveAudioFrame(int index, cv::OutputArray frame)
{
CV_TRACE_FUNCTION();
if (audioStartOffset - usedVideoSampleTime > videoSampleDuration)
{
frame.release();
return true;
}
do
{
if (audioFrame.empty())
{
frame.release();
if (aEOS)
return true;
}
cv::Mat data;
switch (outputAudioFormat)
{
case CV_8S:
data = cv::Mat(1, audioFrame.rows, CV_8S);
for (int i = 0; i < audioFrame.rows; i++)
data.at<char>(0,i) = audioFrame.at<char>(i,index-audioBaseIndex);
break;
case CV_16S:
data = cv::Mat(1, audioFrame.rows, CV_16S);
for (int i = 0; i < audioFrame.rows; i++)
data.at<short>(0,i) = audioFrame.at<short>(i,index-audioBaseIndex);
break;
case CV_32S:
data = cv::Mat(1, audioFrame.rows, CV_32S);
for (int i = 0; i < audioFrame.rows; i++)
data.at<int>(0,i) = audioFrame.at<int>(i,index-audioBaseIndex);
break;
case CV_32F:
data = cv::Mat(1, audioFrame.rows, CV_32F);
for (int i = 0; i < audioFrame.rows; i++)
data.at<float>(0,i) = audioFrame.at<float>(i,index-audioBaseIndex);
break;
default:
frame.release();
break;
}
if (!data.empty())
data.copyTo(frame);
return !frame.empty();
} while (0);
return false;
}
bool CvCapture_MSMF::retrieveFrame(int index, cv::OutputArray frame)
{
CV_TRACE_FUNCTION();
if (index < 0)
return false;
if ((unsigned int)index < audioBaseIndex)
{
if (videoStream == -1)
{
frame.release();
return false;
}
else
return retrieveVideoFrame(frame);
}
else
{
if (audioStream == -1)
{
frame.release();
return false;
}
else
return retrieveAudioFrame(index, frame);
}
}
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
bool CvCapture_MSMF::setTime(double time, bool rough)
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (videoStream == -1)
return false;
if (videoStream != -1 && audioStream != -1)
if (time != 0)
return false;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
PROPVARIANT var;
if (SUCCEEDED(videoFileSource->GetPresentationAttribute((DWORD)MF_SOURCE_READER_MEDIASOURCE, MF_SOURCE_READER_MEDIASOURCE_CHARACTERISTICS, &var)) &&
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
var.vt == VT_UI4 && var.ulVal & MFMEDIASOURCE_CAN_SEEK)
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
usedVideoSample.Release();
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
bool useGrabbing = time > 0 && !rough && !(var.ulVal & MFMEDIASOURCE_HAS_SLOW_SEEK);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
PropVariantClear(&var);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
usedVideoSampleTime = (useGrabbing) ? 0 : (LONGLONG)floor(time + 0.5);
nFrame = (useGrabbing) ? 0 : usedVideoSampleTime/frameStep;
givenAudioTime = (useGrabbing) ? 0 : nFrame*frameStep;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
var.vt = VT_I8;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
var.hVal.QuadPart = usedVideoSampleTime;
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
bool resOK = SUCCEEDED(videoFileSource->SetCurrentPosition(GUID_NULL, var));
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
PropVariantClear(&var);
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
if (resOK && useGrabbing)
{
LONGLONG timeborder = (LONGLONG)floor(time + 0.5) - frameStep / 2;
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
do { resOK = grabFrame(); usedVideoSample.Release(); } while (resOK && usedVideoSampleTime < timeborder);
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
}
return resOK;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
}
return false;
}
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
bool CvCapture_MSMF::setTime(int numberFrame)
{
if (videoStream == -1)
return false;
if (videoStream != -1 && audioStream != -1)
if (numberFrame != 0)
return false;
PROPVARIANT var;
if (SUCCEEDED(videoFileSource->GetPresentationAttribute((DWORD)MF_SOURCE_READER_MEDIASOURCE, MF_SOURCE_READER_MEDIASOURCE_CHARACTERISTICS, &var)) &&
var.vt == VT_UI4 && var.ulVal & MFMEDIASOURCE_CAN_SEEK)
{
usedVideoSample.Release();
PropVariantClear(&var);
usedVideoSampleTime = 0;
nFrame = 0;
givenAudioTime = 0;
var.vt = VT_I8;
var.hVal.QuadPart = usedVideoSampleTime;
bool resOK = SUCCEEDED(videoFileSource->SetCurrentPosition(GUID_NULL, var));
PropVariantClear(&var);
while (resOK && nFrame < numberFrame) { resOK = grabFrame(); usedVideoSample.Release(); };
return resOK;
}
return false;
}
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
template <typename CtrlT>
bool CvCapture_MSMF::readComplexPropery(long prop, long & val) const
{
_ComPtr<CtrlT> ctrl;
if (FAILED(videoFileSource->GetServiceForStream((DWORD)MF_SOURCE_READER_MEDIASOURCE, GUID_NULL, IID_PPV_ARGS(&ctrl))))
{
CV_LOG_DEBUG(NULL, "Failed to get service for stream");
return false;
}
long paramVal, paramFlag;
if (FAILED(ctrl->Get(prop, &paramVal, &paramFlag)))
{
CV_LOG_DEBUG(NULL, "Failed to get property " << prop);
// we continue
}
// fallback - get default value
long minVal, maxVal, stepVal;
if (FAILED(ctrl->GetRange(prop, &minVal, &maxVal, &stepVal, &paramVal, &paramFlag)))
{
CV_LOG_DEBUG(NULL, "Failed to get default value for property " << prop);
return false;
}
val = paramVal;
return true;
}
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
double CvCapture_MSMF::getProperty( int property_id ) const
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
long cVal = 0;
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
if (isOpen)
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
switch (property_id)
{
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
case CV_CAP_PROP_MODE:
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
return captureMode;
case cv::CAP_PROP_HW_DEVICE:
return hwDeviceIndex;
case cv::CAP_PROP_HW_ACCELERATION:
return static_cast<double>(va_type);
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
case CV_CAP_PROP_CONVERT_RGB:
return convertFormat ? 1 : 0;
case CV_CAP_PROP_SAR_NUM:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return captureVideoFormat.aspectRatioNum;
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
case CV_CAP_PROP_SAR_DEN:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return captureVideoFormat.aspectRatioDenom;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
case CV_CAP_PROP_FRAME_WIDTH:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return captureVideoFormat.width;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
case CV_CAP_PROP_FRAME_HEIGHT:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return captureVideoFormat.height;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
case CV_CAP_PROP_FOURCC:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return captureVideoFormat.subType.Data1;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
case CV_CAP_PROP_FPS:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return captureVideoFormat.getFramerate();
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_FRAME_COUNT:
if (duration != 0)
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return floor(((double)duration / 1e7)* captureVideoFormat.getFramerate() + 0.5);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
else
break;
case CV_CAP_PROP_POS_FRAMES:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return (double)nFrame;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_POS_MSEC:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return (double)usedVideoSampleTime / 1e4;
case CAP_PROP_AUDIO_POS:
return (double)audioSamplePos;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_POS_AVI_RATIO:
if (duration != 0)
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return (double)usedVideoSampleTime / duration;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
else
break;
case CV_CAP_PROP_BRIGHTNESS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_Brightness, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_CONTRAST:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_Contrast, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_SATURATION:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_Saturation, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_HUE:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_Hue, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_GAIN:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_Gain, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_SHARPNESS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_Sharpness, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_GAMMA:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_Gamma, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_BACKLIGHT:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_BacklightCompensation, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_MONOCHROME:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_ColorEnable, cVal))
return cVal == 0 ? 1 : 0;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_TEMPERATURE:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMVideoProcAmp>(VideoProcAmp_WhiteBalance, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_PAN:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMCameraControl>(CameraControl_Pan, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_TILT:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMCameraControl>(CameraControl_Tilt, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_ROLL:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMCameraControl>(CameraControl_Roll, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_IRIS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMCameraControl>(CameraControl_Iris, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_EXPOSURE:
case CV_CAP_PROP_AUTO_EXPOSURE:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMCameraControl>(CameraControl_Exposure, cVal))
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (property_id == CV_CAP_PROP_EXPOSURE)
return cVal;
else
return cVal == VideoProcAmp_Flags_Auto;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
}
break;
case CV_CAP_PROP_ZOOM:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMCameraControl>(CameraControl_Zoom, cVal))
return cVal;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_FOCUS:
case CV_CAP_PROP_AUTOFOCUS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (readComplexPropery<IAMCameraControl>(CameraControl_Focus, cVal))
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
{
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
if (property_id == CV_CAP_PROP_FOCUS)
return cVal;
else
return cVal == VideoProcAmp_Flags_Auto;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
}
break;
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
case CV_CAP_PROP_WHITE_BALANCE_BLUE_U:
case CV_CAP_PROP_WHITE_BALANCE_RED_V:
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_RECTIFICATION:
case CV_CAP_PROP_TRIGGER:
case CV_CAP_PROP_TRIGGER_DELAY:
case CV_CAP_PROP_GUID:
case CV_CAP_PROP_ISO_SPEED:
case CV_CAP_PROP_SETTINGS:
case CV_CAP_PROP_BUFFERSIZE:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
case CAP_PROP_AUDIO_BASE_INDEX:
return audioBaseIndex;
case CAP_PROP_AUDIO_TOTAL_STREAMS:
return numberOfAudioStreams;
case CAP_PROP_AUDIO_TOTAL_CHANNELS:
return captureAudioFormat.nChannels;
case CAP_PROP_AUDIO_SAMPLES_PER_SECOND:
return captureAudioFormat.nSamplesPerSec;
case CAP_PROP_AUDIO_DATA_DEPTH:
return outputAudioFormat;
case CAP_PROP_AUDIO_SHIFT_NSEC:
return (double)(audioStartOffset - videoStartOffset)*1e2;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
default:
break;
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
}
return -1;
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
template <typename CtrlT>
bool CvCapture_MSMF::writeComplexProperty(long prop, double val, long flags)
{
_ComPtr<CtrlT> ctrl;
if (FAILED(videoFileSource->GetServiceForStream((DWORD)MF_SOURCE_READER_MEDIASOURCE, GUID_NULL, IID_PPV_ARGS(&ctrl))))
{
CV_LOG_DEBUG(NULL, "Failed get service for stream");
return false;
}
if (FAILED(ctrl->Set(prop, (long)val, flags)))
{
CV_LOG_DEBUG(NULL, "Failed to set property " << prop);
return false;
}
return true;
}
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
bool CvCapture_MSMF::setProperty( int property_id, double value )
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
MediaType newFormat = captureVideoFormat;
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
if (isOpen)
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
switch (property_id)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
case CV_CAP_PROP_MODE:
switch ((MSMFCapture_Mode)((int)value))
{
case MODE_SW:
return configureHW(false);
case MODE_HW:
return configureHW(true);
default:
return false;
}
videoio: drop VideoCaptureModes enum in favour of fourcc
2018-11-13 22:53:35 +08:00
case CV_CAP_PROP_FOURCC:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return configureVideoOutput(newFormat, (int)cvRound(value));
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
case CV_CAP_PROP_FORMAT:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return configureVideoOutput(newFormat, (int)cvRound(value));
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
case CV_CAP_PROP_CONVERT_RGB:
videoio: fixed conversion in MSMF backend
2020-04-01 19:40:11 +08:00
convertFormat = (value != 0);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return configureVideoOutput(newFormat, outputVideoFormat);
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
case CV_CAP_PROP_SAR_NUM:
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
if (value > 0)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
newFormat.aspectRatioNum = (UINT32)cvRound(value);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return configureVideoOutput(newFormat, outputVideoFormat);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
break;
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
case CV_CAP_PROP_SAR_DEN:
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
if (value > 0)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
newFormat.aspectRatioDenom = (UINT32)cvRound(value);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return configureVideoOutput(newFormat, outputVideoFormat);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
break;
case CV_CAP_PROP_FRAME_WIDTH:
if (value >= 0)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
newFormat.width = (UINT32)cvRound(value);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return configureVideoOutput(newFormat, outputVideoFormat);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
Fixed setting of frame size properties for MSMF-based VideoCapture
2018-06-06 21:00:55 +08:00
break;
case CV_CAP_PROP_FRAME_HEIGHT:
if (value >= 0)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
newFormat.height = (UINT32)cvRound(value);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return configureVideoOutput(newFormat, outputVideoFormat);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
break;
case CV_CAP_PROP_FPS:
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
if (value >= 0)
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
{
newFormat.setFramerate(value);
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
return configureVideoOutput(newFormat, outputVideoFormat);
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
}
MSMF-based CameraCapture reworked to use SourceReader
2018-04-12 20:26:18 +08:00
break;
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_FRAME_COUNT:
break;
case CV_CAP_PROP_POS_AVI_RATIO:
if (duration != 0)
return setTime(duration * value, true);
break;
case CV_CAP_PROP_POS_FRAMES:
Merge pull request #19721 from MaximMilashchenko:Audio add audio support in cap_msmf * audio msmf * fixed warnings * minor fix * fixed SampleTime MSMF * minor fix, fixed audio test, retrieveAudioFrame * fixed warnings * impelemented sync audio and video stream with start offset * fixed error * fixed docs * fixed audio sample * CAP_PROP_AUDIO_POS, minor fixed * fixed warnings * videoio(MSMF): update audio test checks, add debug logging * fixed * fixed desynchronization of time positions, warnings * fixed warnings * videoio(audio): tune tests checks * videoio(audio): update properties description * build warnings Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-10-20 21:18:24 +08:00
if (std::fabs(captureVideoFormat.getFramerate()) > 0)
return setTime((int)value);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
break;
case CV_CAP_PROP_POS_MSEC:
Added precise seek to MSMF-based VideoCapture::set(CAP_PROP_POS_FRAMES)
2018-04-24 21:18:39 +08:00
return setTime(value * 1e4, false);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_BRIGHTNESS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_Brightness, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_CONTRAST:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_Contrast, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_SATURATION:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_Saturation, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_HUE:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_Hue, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_GAIN:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_Gain, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_SHARPNESS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_Sharpness, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_GAMMA:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_Gamma, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_BACKLIGHT:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_BacklightCompensation, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_MONOCHROME:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_ColorEnable, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_TEMPERATURE:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMVideoProcAmp>(VideoProcAmp_WhiteBalance, value, VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_PAN:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Pan, value, CameraControl_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_TILT:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Tilt, value, CameraControl_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_ROLL:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Roll, value, CameraControl_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_IRIS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Iris, value, CameraControl_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_EXPOSURE:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Exposure, value, CameraControl_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_AUTO_EXPOSURE:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Exposure, value, value != 0 ? VideoProcAmp_Flags_Auto : VideoProcAmp_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_ZOOM:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Zoom, value, CameraControl_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_FOCUS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Focus, value, CameraControl_Flags_Manual);
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_AUTOFOCUS:
videoio/MSMF: refactored format handling and selection, property reading and writing
2020-02-07 20:03:20 +08:00
return writeComplexProperty<IAMCameraControl>(CameraControl_Focus, value, value != 0 ? CameraControl_Flags_Auto : CameraControl_Flags_Manual);
case CV_CAP_PROP_WHITE_BALANCE_BLUE_U:
case CV_CAP_PROP_WHITE_BALANCE_RED_V:
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
case CV_CAP_PROP_RECTIFICATION:
case CV_CAP_PROP_TRIGGER:
case CV_CAP_PROP_TRIGGER_DELAY:
case CV_CAP_PROP_GUID:
case CV_CAP_PROP_ISO_SPEED:
case CV_CAP_PROP_SETTINGS:
case CV_CAP_PROP_BUFFERSIZE:
default:
break;
MSMF-based VideoCapture reworked to use SourceReader
2018-03-28 19:03:22 +08:00
}
Integrated camera and file MSMF-based VideoCaptures and implemented configuration properties get/set
2018-04-19 23:24:02 +08:00
return false;
GetProperty method for MSMF VideoCapture implemented.
2013-05-23 00:50:54 +08:00
}
VideoCapture for video files implemented. Set and Get methods are not implemented; Camera based video capture is broken due to modifications.
2013-05-14 20:17:34 +08:00
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
cv::Ptr<cv::IVideoCapture> cv::cvCreateCapture_MSMF( int index, const cv::VideoCaptureParameters& params)
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
{
videoio(msmf): avoid using of C++11 code build fails with MSVS 2012 without additional flags
2018-06-30 04:34:14 +08:00
cv::Ptr<CvCapture_MSMF> capture = cv::makePtr<CvCapture_MSMF>();
if (capture)
{
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
capture->open(index, &params);
videoio(msmf): avoid using of C++11 code build fails with MSVS 2012 without additional flags
2018-06-30 04:34:14 +08:00
if (capture->isOpened())
return capture;
}
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
return cv::Ptr<cv::IVideoCapture>();
Experimental MS Media Foundation API support added
2013-04-03 09:01:20 +08:00
}
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
cv::Ptr<cv::IVideoCapture> cv::cvCreateCapture_MSMF (const cv::String& filename, const cv::VideoCaptureParameters& params)
VideoCapture for video files implemented. Set and Get methods are not implemented; Camera based video capture is broken due to modifications.
2013-05-14 20:17:34 +08:00
{
videoio(msmf): avoid using of C++11 code build fails with MSVS 2012 without additional flags
2018-06-30 04:34:14 +08:00
cv::Ptr<CvCapture_MSMF> capture = cv::makePtr<CvCapture_MSMF>();
if (capture)
{
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
capture->open(filename, &params);
videoio(msmf): avoid using of C++11 code build fails with MSVS 2012 without additional flags
2018-06-30 04:34:14 +08:00
if (capture->isOpened())
return capture;
}
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
return cv::Ptr<cv::IVideoCapture>();
VideoCapture for video files implemented. Set and Get methods are not implemented; Camera based video capture is broken due to modifications.
2013-05-14 20:17:34 +08:00
}
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
//
//
// Media Foundation-based Video Writer
//
//
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
class CvVideoWriter_MSMF : public cv::IVideoWriter
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
{
public:
CvVideoWriter_MSMF();
virtual ~CvVideoWriter_MSMF();
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
virtual bool open(const cv::String& filename, int fourcc,
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
double fps, cv::Size frameSize, const cv::VideoWriterParameters& params);
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
virtual void close();
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
virtual void write(cv::InputArray);
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
virtual double getProperty(int) const override;
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
virtual bool setProperty(int, double) { return false; }
virtual bool isOpened() const { return initiated; }
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
videoio: added '.getBackendName()' method
2018-09-16 04:54:03 +08:00
int getCaptureDomain() const CV_OVERRIDE { return cv::CAP_MSMF; }
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
private:
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
Media_Foundation& MF;
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
VideoAccelerationType va_type;
int va_device;
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
UINT32 videoWidth;
UINT32 videoHeight;
double fps;
UINT32 bitRate;
UINT32 frameSize;
GUID encodingFormat;
GUID inputFormat;
DWORD streamIndex;
Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update and rename cap_msmf.h to cap_msmf.hpp Successful test - samples are grabbed Library updated and cleaned up with comments, marshaling, exceptions and linker settings Fixed trailing whitespace VS 2013 support and cleanup consistency plus C++/CX new object fixed Conflicts: modules/highgui/src/cap_msmf.cpp modules/highgui/src/cap_msmf.hpp modules/highgui/src/ppltasks_winrt.h Fix merge conflicts VS 2013 Update 2 library bug fix integrated a-wi's changed integrated
2013-12-14 16:53:30 +08:00
_ComPtr<IMFSinkWriter> sinkWriter;
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
bool initiated;
LONGLONG rtStart;
UINT64 rtDuration;
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
static const GUID FourCC2GUID(int fourcc);
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
};
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
CvVideoWriter_MSMF::CvVideoWriter_MSMF():
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
MF(Media_Foundation::getInstance()),
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
va_type(VIDEO_ACCELERATION_NONE),
va_device(-1),
Fixed several issues found by static analysis, Windows-specific
2018-07-27 23:25:55 +08:00
videoWidth(0),
videoHeight(0),
fps(0),
bitRate(0),
frameSize(0),
encodingFormat(),
inputFormat(),
streamIndex(0),
initiated(false),
rtStart(0),
rtDuration(0)
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
{
}
CvVideoWriter_MSMF::~CvVideoWriter_MSMF()
{
close();
}
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
const GUID CvVideoWriter_MSMF::FourCC2GUID(int fourcc)
{
switch(fourcc)
{
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('d', 'v', '2', '5'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_DV25; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('d', 'v', '5', '0'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_DV50; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('d', 'v', 'c', ' '):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_DVC; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('d', 'v', 'h', '1'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_DVH1; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('d', 'v', 'h', 'd'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_DVHD; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('d', 'v', 's', 'd'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_DVSD; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('d', 'v', 's', 'l'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_DVSL; break;
Fixed compilation errors under VS 2010 and Win 7 Applied in PR3240 to 2.4 branch.
2014-09-25 17:42:07 +08:00
#if (WINVER >= 0x0602)
Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update and rename cap_msmf.h to cap_msmf.hpp Successful test - samples are grabbed Library updated and cleaned up with comments, marshaling, exceptions and linker settings Fixed trailing whitespace VS 2013 support and cleanup consistency plus C++/CX new object fixed Conflicts: modules/highgui/src/cap_msmf.cpp modules/highgui/src/cap_msmf.hpp modules/highgui/src/ppltasks_winrt.h Fix merge conflicts VS 2013 Update 2 library bug fix integrated a-wi's changed integrated
2013-12-14 16:53:30 +08:00
case CV_FOURCC_MACRO('H', '2', '6', '3'): // Available only for Win 8 target.
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_H263; break;
Add support for WinRT in the MF capture framework by removing the disallowed calls to enumerate devices and create a sample grabber sink and adding framework for the MediaCapture interface and a custom sink which interfaces with the sample grabber callback interface. The change requires discussion for making it completely functional as redundancy is required given that if the source is a video file, the old code pathways must be used. Otherwise all IMFMediaSession, IMFMediaSource, and IMFActivate code must use a MediaCapture code path and all sink code must use the CMediaSink custom sink. Support for the custom sink is extended to non-WinRT not for compatibility as Windows Vista client is a minimum regardless, but because it offers more flexibility, could be faster and is able to be used as an optionally different code path during sink creation based on a future configuration parameter. My discussion and proposal to finish this change: Devices are so easily enumerated through WinRT Windows.Devices namespace that wrapping the calls in a library is quite a chore for little benefit though to get the various modes and formats could still be a worthwhile project. For now conditional compilation to remove videodevices and any offending non-video file related activity in videodevice. In my opinion, this is a different , far less fundamental and important change which can possibly be done as a future project and also much more easily implemented in C++/CX. ImageGrabber has the IMFSampleGrabberSinkCallback replaced with a base class (SharedSampleGrabber) which also be is base class for ImageGrabberRT. This change is necessary as the custom sink does not require a thread to pump events which is done through MediaCapture already. IMFSampleGrabberSinkCallback is the common element between both models and that piece can be shared. Initializing the new ImageGrabberRT is as simple as passing an already initialized MediaCapture object and any video format/encoding parameters. The concurrency event is necessary to wait for completion and is the way the underlying, IAsyncAction wrappers in the task library work as well. Native WIN32 event objects would be an option if HAVE_CONCURRENCY is not defined. I could even imagine doing it with sleep/thread yield and InterlockedCompareExchange yet I am not enthusiastic about that approach either. Since there is a specific compiler HAVE_ for concurrency, I do not like pulling it in though I think for WinRT it is safe to say we will always have it available though should probably conditionally compile with the Interlocked option as WIN32 events would require HAVE_WIN32. It looks like C++/CX cannot be used for the IMediaExtension sink (which should not be a problem) as using COM objects requires WRL and though deriving from IMediaExtension can be done, there is little purpose without COM. Objects from C++/CX can be swapped to interact with objects from native C++ as Inspectable* can reinterpret_cast to the ref object IInspectable^ and vice-versa. A solution to the COM class with C++/CX would be great so we could have dual support. Also without #define for every WRL object in use, the code will get quite muddy given that the */^ would need to be ifdef'd everywhere. Fixed bugs and completed the change. I believe the new classes need to be moved to a header file as the file has become to large and more classes need to be added for handling all the asynchronous problems (one wrapping IAsyncAction in a task and another for making a task out of IAsyncAction). Unfortunately, blocking on the UI thread is not an option in WinRT so a synchronous architecture is considered "illegal" by Microsoft's standards even if implementable (C++/CX ppltasks library throws errors if you try it). Worse, either by design or a bug in the MF MediaCapture class with Custom Sinks causes a crash if stop/start previewing without reinitializing (spPreferredPreviewMediaType is fatally nulled). After decompiling Windows.Media.dll, I worked around this in my own projects by using an activate-able custom sink ID which strangely assigns 1 to this pointer allowing it to be reinitialized in what can only be described as a hack by Microsoft. This would add additional overhead to the project to implement especially for static libraries as it requires IDL/DLL exporting followed by manifest declaration. Better to document that it is not supported. Furthermore, an additional class for IMFAttributes should be implemented to make clean architecture for passing around attributes as opposed to directly calling non-COM interface calls on the objects and making use of SetProperties which would also be a set up for an object that uses the RuntimeClass activation ID. The remaining changes are not difficult and will be complete soon along with debug tracing messages. Update and rename cap_msmf.h to cap_msmf.hpp Successful test - samples are grabbed Library updated and cleaned up with comments, marshaling, exceptions and linker settings Fixed trailing whitespace VS 2013 support and cleanup consistency plus C++/CX new object fixed Conflicts: modules/highgui/src/cap_msmf.cpp modules/highgui/src/cap_msmf.hpp modules/highgui/src/ppltasks_winrt.h Fix merge conflicts VS 2013 Update 2 library bug fix integrated a-wi's changed integrated
2013-12-14 16:53:30 +08:00
#endif
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('H', '2', '6', '4'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_H264; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('M', '4', 'S', '2'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_M4S2; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('M', 'J', 'P', 'G'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_MJPG; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('M', 'P', '4', '3'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_MP43; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('M', 'P', '4', 'S'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_MP4S; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('M', 'P', '4', 'V'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_MP4V; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('M', 'P', 'G', '1'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_MPG1; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('M', 'S', 'S', '1'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_MSS1; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('M', 'S', 'S', '2'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_MSS2; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('W', 'M', 'V', '1'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_WMV1; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('W', 'M', 'V', '2'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_WMV2; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('W', 'M', 'V', '3'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_WMV3; break;
Media foundation video i/o fixes. Bug in Video for Windows capture init fixed; Media Foundation based capture finalization fixed; Highgui tests for video i/o updated.
2013-05-22 19:21:23 +08:00
case CV_FOURCC_MACRO('W', 'V', 'C', '1'):
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
return MFVideoFormat_WVC1; break;
default:
return MFVideoFormat_H264;
}
}
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
bool CvVideoWriter_MSMF::open( const cv::String& filename, int fourcc,
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
double _fps, cv::Size _frameSize, const cv::VideoWriterParameters& params)
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
{
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
if (initiated)
close();
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
if (params.has(VIDEOWRITER_PROP_HW_ACCELERATION))
{
va_type = params.get<VideoAccelerationType>(VIDEOWRITER_PROP_HW_ACCELERATION);
if (va_type != VIDEO_ACCELERATION_NONE && va_type != VIDEO_ACCELERATION_ANY)
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: MSMF backend doesn't support writer acceleration support. Can't handle VIDEOWRITER_PROP_HW_ACCELERATION parameter. Bailout");
return false;
}
}
if (params.has(VIDEOWRITER_PROP_HW_DEVICE))
{
va_device = params.get<int>(VIDEOWRITER_PROP_HW_DEVICE);
if (va_type == VIDEO_ACCELERATION_NONE && va_device != -1)
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: Invalid usage of VIDEOWRITER_PROP_HW_DEVICE without requested H/W acceleration. Bailout");
return false;
}
if (va_type == VIDEO_ACCELERATION_ANY && va_device != -1)
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: Invalid usage of VIDEOWRITER_PROP_HW_DEVICE with 'ANY' H/W acceleration. Bailout");
return false;
}
if (va_device != -1)
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: VIDEOWRITER_PROP_HW_DEVICE is not supported. Specify -1 (auto) value. Bailout");
return false;
}
}
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
videoWidth = _frameSize.width;
videoHeight = _frameSize.height;
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
fps = _fps;
GetProperty method for MSMF VideoCapture implemented.
2013-05-23 00:50:54 +08:00
bitRate = (UINT32)fps*videoWidth*videoHeight; // 1-bit per pixel
Media Foundation-based VideoWriter improvements. FourCC parameter handlig added; Smart pointers instead SafeRelease call; Windows RT support (vertical mirroring).
2013-05-06 22:17:53 +08:00
encodingFormat = FourCC2GUID(fourcc);
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
inputFormat = MFVideoFormat_RGB32;
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
_ComPtr<IMFMediaType> mediaTypeOut;
_ComPtr<IMFMediaType> mediaTypeIn;
_ComPtr<IMFAttributes> spAttr;
if (// Set the output media type.
SUCCEEDED(MFCreateMediaType(&mediaTypeOut)) &&
SUCCEEDED(mediaTypeOut->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video)) &&
SUCCEEDED(mediaTypeOut->SetGUID(MF_MT_SUBTYPE, encodingFormat)) &&
SUCCEEDED(mediaTypeOut->SetUINT32(MF_MT_AVG_BITRATE, bitRate)) &&
SUCCEEDED(mediaTypeOut->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive)) &&
SUCCEEDED(MFSetAttributeSize(mediaTypeOut.Get(), MF_MT_FRAME_SIZE, videoWidth, videoHeight)) &&
videoio: fixed MSMF writer fps handling
2020-04-02 17:49:46 +08:00
SUCCEEDED(MFSetAttributeRatio(mediaTypeOut.Get(), MF_MT_FRAME_RATE, (UINT32)(fps * 1000), 1000)) &&
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
SUCCEEDED(MFSetAttributeRatio(mediaTypeOut.Get(), MF_MT_PIXEL_ASPECT_RATIO, 1, 1)) &&
// Set the input media type.
SUCCEEDED(MFCreateMediaType(&mediaTypeIn)) &&
SUCCEEDED(mediaTypeIn->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video)) &&
SUCCEEDED(mediaTypeIn->SetGUID(MF_MT_SUBTYPE, inputFormat)) &&
SUCCEEDED(mediaTypeIn->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive)) &&
SUCCEEDED(mediaTypeIn->SetUINT32(MF_MT_DEFAULT_STRIDE, 4 * videoWidth)) && //Assume BGR32 input
SUCCEEDED(MFSetAttributeSize(mediaTypeIn.Get(), MF_MT_FRAME_SIZE, videoWidth, videoHeight)) &&
videoio: fixed MSMF writer fps handling
2020-04-02 17:49:46 +08:00
SUCCEEDED(MFSetAttributeRatio(mediaTypeIn.Get(), MF_MT_FRAME_RATE, (UINT32)(fps * 1000), 1000)) &&
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
SUCCEEDED(MFSetAttributeRatio(mediaTypeIn.Get(), MF_MT_PIXEL_ASPECT_RATIO, 1, 1)) &&
// Set sink writer parameters
SUCCEEDED(MFCreateAttributes(&spAttr, 10)) &&
SUCCEEDED(spAttr->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, true)) &&
SUCCEEDED(spAttr->SetUINT32(MF_SINK_WRITER_DISABLE_THROTTLING, true))
)
{
// Create the sink writer
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
cv::AutoBuffer<wchar_t> unicodeFileName(filename.length() + 1);
opencv: Use cv::AutoBuffer<>::data()
2018-06-11 06:42:00 +08:00
MultiByteToWideChar(CP_ACP, 0, filename.c_str(), -1, unicodeFileName.data(), (int)filename.length() + 1);
HRESULT hr = MFCreateSinkWriterFromURL(unicodeFileName.data(), NULL, spAttr.Get(), &sinkWriter);
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
if (SUCCEEDED(hr))
{
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
// Configure the sink writer and tell it start to start accepting data
if (SUCCEEDED(sinkWriter->AddStream(mediaTypeOut.Get(), &streamIndex)) &&
SUCCEEDED(sinkWriter->SetInputMediaType(streamIndex, mediaTypeIn.Get(), NULL)) &&
SUCCEEDED(sinkWriter->BeginWriting()))
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
{
initiated = true;
rtStart = 0;
videoio: fixed MSMF writer fps handling
2020-04-02 17:49:46 +08:00
MFFrameRateToAverageTimePerFrame((UINT32)(fps * 1000), 1000, &rtDuration);
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
VideoAccelerationType actual_va_type = VIDEO_ACCELERATION_NONE;
if (va_type != VIDEO_ACCELERATION_NONE && va_type != VIDEO_ACCELERATION_ANY)
{
if (va_type != actual_va_type)
{
CV_LOG_ERROR(NULL, "VIDEOIO/MSMF: Can't select requested video acceleration through VIDEOWRITER_PROP_HW_ACCELERATION: "
<< va_type << " (actual is " << actual_va_type << "). Bailout");
close();
return false;
}
}
else
{
va_type = actual_va_type;
}
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
return true;
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
}
}
}
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
return false;
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
}
void CvVideoWriter_MSMF::close()
{
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
if (initiated)
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
{
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
initiated = false;
sinkWriter->Finalize();
videoio(msmf): cleanup unused methods from ComPtr wrapper Use Win32 TRUE/FALSE in SetUINT32() calls for bool parameters
2018-06-10 20:32:09 +08:00
sinkWriter.Release();
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
}
}
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
void CvVideoWriter_MSMF::write(cv::InputArray img)
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
{
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
if (img.empty() ||
(img.channels() != 1 && img.channels() != 3 && img.channels() != 4) ||
(UINT32)img.cols() != videoWidth || (UINT32)img.rows() != videoHeight)
return;
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
const LONG cbWidth = 4 * videoWidth;
const DWORD cbBuffer = cbWidth * videoHeight;
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
_ComPtr<IMFSample> sample;
_ComPtr<IMFMediaBuffer> buffer;
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
BYTE *pData = NULL;
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
// Prepare a media sample.
if (SUCCEEDED(MFCreateSample(&sample)) &&
// Set sample time stamp and duration.
SUCCEEDED(sample->SetSampleTime(rtStart)) &&
SUCCEEDED(sample->SetSampleDuration(rtDuration)) &&
// Create a memory buffer.
SUCCEEDED(MFCreateMemoryBuffer(cbBuffer, &buffer)) &&
// Set the data length of the buffer.
SUCCEEDED(buffer->SetCurrentLength(cbBuffer)) &&
// Add the buffer to the sample.
SUCCEEDED(sample->AddBuffer(buffer.Get())) &&
// Lock the buffer.
SUCCEEDED(buffer->Lock(&pData, NULL, NULL)))
{
// Copy the video frame to the buffer.
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
cv::cvtColor(img.getMat(), cv::Mat(videoHeight, videoWidth, CV_8UC4, pData, cbWidth), img.channels() > 1 ? cv::COLOR_BGR2BGRA : cv::COLOR_GRAY2BGRA);
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
buffer->Unlock();
Merge pull request #11092 from terfendail:msmf_videocapture Update for MSMF-based VideoCapture and VideoWriter (#11092) * MSMF based VideoCapture updated to handle video stream formats different from RGB24 * MSMF based VideoWriter updated to handle video frame top-bottom line ordering regardless of output format * Fixed race condition in MSMF based VideoCapture * Refactored MSMF based VideoCapture and VideoWriter * Disabled frame rate estimation for MP43 * Removed test for unsupported avi container from MSMF VideoWriter tests * Enabled MSMF-based VideoIO by default
2018-04-05 18:55:42 +08:00
// Send media sample to the Sink Writer.
if (SUCCEEDED(sinkWriter->WriteSample(streamIndex, sample.Get())))
{
rtStart += rtDuration;
}
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
}
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
double CvVideoWriter_MSMF::getProperty(int propId) const
{
if (propId == VIDEOWRITER_PROP_HW_ACCELERATION)
{
return static_cast<double>(va_type);
}
else if (propId == VIDEOWRITER_PROP_HW_DEVICE)
{
return static_cast<double>(va_device);
}
return 0;
}
Framework for dynamic videoio backends, FFmpeg and GStreamer plugins
2019-01-14 18:33:38 +08:00
cv::Ptr<cv::IVideoWriter> cv::cvCreateVideoWriter_MSMF( const std::string& filename, int fourcc,
Merge pull request #16766 from VadimLevin:dev/vlevin/video_writer_params_constructor * feature: Extend VideoWriter to accept vector of parameters - Add additional constructor and `open` method for `VideoWriter` those accept a vector of parameters - Move actual implementation of the `VideoWriter::open` to general method which accepts vector of parameters - Propagate parsed parameters map up to actual video backend construction * fix: Change VideoWriter constructor description to suppress doc warning * refactor: Rollback newlines changes * feature: Changed VideoWriter parameters workflow * feature: Log unused parameters in VideoWriter open * doc: Fix VideoWriter `isColor` parameter description * fix: int to bool VC++ conversion warning * doc: Remove information about `isColor` flag usage.
2020-04-28 16:38:39 +08:00
double fps, const cv::Size& frameSize,
const VideoWriterParameters& params)
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
{
videoio(msmf): avoid using of C++11 code build fails with MSVS 2012 without additional flags
2018-06-30 04:34:14 +08:00
cv::Ptr<CvVideoWriter_MSMF> writer = cv::makePtr<CvVideoWriter_MSMF>();
if (writer)
{
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
writer->open(filename, fourcc, fps, frameSize, params);
videoio(msmf): avoid using of C++11 code build fails with MSVS 2012 without additional flags
2018-06-30 04:34:14 +08:00
if (writer->isOpened())
return writer;
}
MSMF-based VideoCapture and VideoWriter backend changed to C++ interface
2018-04-25 23:19:14 +08:00
return cv::Ptr<cv::IVideoWriter>();
Media Foundation-based VideoWriter added
2013-05-06 18:36:51 +08:00
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
#if defined(BUILD_PLUGIN)
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
#define NEW_PLUGIN
#ifndef NEW_PLUGIN
videoio: plugins API versioning
2020-12-04 11:18:16 +08:00
#define ABI_VERSION 0
#define API_VERSION 0
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
#include "plugin_api.hpp"
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
#else
#define CAPTURE_ABI_VERSION 1
#define CAPTURE_API_VERSION 1
#include "plugin_capture_api.hpp"
#define WRITER_ABI_VERSION 1
#define WRITER_API_VERSION 1
#include "plugin_writer_api.hpp"
#endif
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
namespace cv {
typedef CvCapture_MSMF CaptureT;
typedef CvVideoWriter_MSMF WriterT;
static
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
CvResult CV_API_CALL cv_capture_open_with_params(
const char* filename, int camera_index,
int* params, unsigned n_params,
CV_OUT CvPluginCapture* handle
)
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
{
if (!handle)
return CV_ERROR_FAIL;
*handle = NULL;
if (!filename)
return CV_ERROR_FAIL;
CaptureT* cap = 0;
try
{
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
cv::VideoCaptureParameters parameters(params, n_params);
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
cap = new CaptureT();
bool res;
if (filename)
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
res = cap->open(std::string(filename), &parameters);
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
else
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
res = cap->open(camera_index, &parameters);
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
if (res)
{
*handle = (CvPluginCapture)cap;
return CV_ERROR_OK;
}
}
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
catch (const std::exception& e)
{
CV_LOG_WARNING(NULL, "MSMF: Exception is raised: " << e.what());
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
catch (...)
{
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
CV_LOG_WARNING(NULL, "MSMF: Unknown C++ exception is raised");
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
}
if (cap)
delete cap;
return CV_ERROR_FAIL;
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
static
CvResult CV_API_CALL cv_capture_open(const char* filename, int camera_index, CV_OUT CvPluginCapture* handle)
{
return cv_capture_open_with_params(filename, camera_index, NULL, 0, handle);
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
static
CvResult CV_API_CALL cv_capture_release(CvPluginCapture handle)
{
if (!handle)
return CV_ERROR_FAIL;
CaptureT* instance = (CaptureT*)handle;
delete instance;
return CV_ERROR_OK;
}
static
CvResult CV_API_CALL cv_capture_get_prop(CvPluginCapture handle, int prop, CV_OUT double* val)
{
if (!handle)
return CV_ERROR_FAIL;
if (!val)
return CV_ERROR_FAIL;
try
{
CaptureT* instance = (CaptureT*)handle;
*val = instance->getProperty(prop);
return CV_ERROR_OK;
}
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
catch (const std::exception& e)
{
CV_LOG_WARNING(NULL, "MSMF: Exception is raised: " << e.what());
return CV_ERROR_FAIL;
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
catch (...)
{
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
CV_LOG_WARNING(NULL, "MSMF: Unknown C++ exception is raised");
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
return CV_ERROR_FAIL;
}
}
static
CvResult CV_API_CALL cv_capture_set_prop(CvPluginCapture handle, int prop, double val)
{
if (!handle)
return CV_ERROR_FAIL;
try
{
CaptureT* instance = (CaptureT*)handle;
return instance->setProperty(prop, val) ? CV_ERROR_OK : CV_ERROR_FAIL;
}
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
catch (const std::exception& e)
{
CV_LOG_WARNING(NULL, "MSMF: Exception is raised: " << e.what());
return CV_ERROR_FAIL;
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
catch (...)
{
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
CV_LOG_WARNING(NULL, "MSMF: Unknown C++ exception is raised");
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
return CV_ERROR_FAIL;
}
}
static
CvResult CV_API_CALL cv_capture_grab(CvPluginCapture handle)
{
if (!handle)
return CV_ERROR_FAIL;
try
{
CaptureT* instance = (CaptureT*)handle;
return instance->grabFrame() ? CV_ERROR_OK : CV_ERROR_FAIL;
}
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
catch (const std::exception& e)
{
CV_LOG_WARNING(NULL, "MSMF: Exception is raised: " << e.what());
return CV_ERROR_FAIL;
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
catch (...)
{
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
CV_LOG_WARNING(NULL, "MSMF: Unknown C++ exception is raised");
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
return CV_ERROR_FAIL;
}
}
static
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
CvResult CV_API_CALL cv_capture_retrieve(CvPluginCapture handle, int stream_idx, cv_videoio_capture_retrieve_cb_t callback, void* userdata)
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
{
if (!handle)
return CV_ERROR_FAIL;
try
{
CaptureT* instance = (CaptureT*)handle;
Mat img;
if (instance->retrieveFrame(stream_idx, img))
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
#ifndef NEW_PLUGIN
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
return callback(stream_idx, img.data, (int)img.step, img.cols, img.rows, img.channels(), userdata);
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
#else
return callback(stream_idx, img.data, (int)img.step, img.cols, img.rows, img.type(), userdata);
#endif
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
return CV_ERROR_FAIL;
}
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
catch (const std::exception& e)
{
CV_LOG_WARNING(NULL, "MSMF: Exception is raised: " << e.what());
return CV_ERROR_FAIL;
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
catch (...)
{
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
CV_LOG_WARNING(NULL, "MSMF: Unknown C++ exception is raised");
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
return CV_ERROR_FAIL;
}
}
static
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
CvResult CV_API_CALL cv_writer_open_with_params(
const char* filename, int fourcc, double fps, int width, int height,
int* params, unsigned n_params,
CV_OUT CvPluginWriter* handle)
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
{
WriterT* wrt = 0;
try
{
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
VideoWriterParameters parameters(params, n_params);
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
wrt = new WriterT();
Size sz(width, height);
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
if (wrt && wrt->open(filename, fourcc, fps, sz, parameters))
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
{
*handle = (CvPluginWriter)wrt;
return CV_ERROR_OK;
}
}
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
catch (const std::exception& e)
{
CV_LOG_WARNING(NULL, "MSMF: Exception is raised: " << e.what());
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
catch (...)
{
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
CV_LOG_WARNING(NULL, "MSMF: Unknown C++ exception is raised");
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
}
if (wrt)
delete wrt;
return CV_ERROR_FAIL;
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
static
CvResult CV_API_CALL cv_writer_open(const char* filename, int fourcc, double fps, int width, int height, int isColor,
CV_OUT CvPluginWriter* handle)
{
int params[2] = { VIDEOWRITER_PROP_IS_COLOR, isColor };
return cv_writer_open_with_params(filename, fourcc, fps, width, height, params, 1, handle);
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
static
CvResult CV_API_CALL cv_writer_release(CvPluginWriter handle)
{
if (!handle)
return CV_ERROR_FAIL;
WriterT* instance = (WriterT*)handle;
delete instance;
return CV_ERROR_OK;
}
static
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
CvResult CV_API_CALL cv_writer_get_prop(CvPluginWriter handle, int prop, CV_OUT double* val)
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
{
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
if (!handle)
return CV_ERROR_FAIL;
if (!val)
return CV_ERROR_FAIL;
try
{
WriterT* instance = (WriterT*)handle;
*val = instance->getProperty(prop);
return CV_ERROR_OK;
}
catch (...)
{
return CV_ERROR_FAIL;
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
}
static
CvResult CV_API_CALL cv_writer_set_prop(CvPluginWriter /*handle*/, int /*prop*/, double /*val*/)
{
return CV_ERROR_FAIL;
}
static
CvResult CV_API_CALL cv_writer_write(CvPluginWriter handle, const unsigned char* data, int step, int width, int height, int cn)
{
if (!handle)
return CV_ERROR_FAIL;
try
{
CV_Assert(step >= 0);
WriterT* instance = (WriterT*)handle;
Size sz(width, height);
Mat img(sz, CV_MAKETYPE(CV_8U, cn), (void*)data, (size_t)step);
instance->write(img);
return CV_ERROR_OK;
}
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
catch (const std::exception& e)
{
CV_LOG_WARNING(NULL, "MSMF: Exception is raised: " << e.what());
return CV_ERROR_FAIL;
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
catch (...)
{
videoio: dump exception's messages in plugins
2021-02-25 01:45:19 +08:00
CV_LOG_WARNING(NULL, "MSMF: Unknown C++ exception is raised");
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
return CV_ERROR_FAIL;
}
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
} // namespace
#ifndef NEW_PLUGIN
videoio: plugins API versioning
2020-12-04 11:18:16 +08:00
static const OpenCV_VideoIO_Plugin_API_preview plugin_api =
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
{
{
videoio: plugins API versioning
2020-12-04 11:18:16 +08:00
sizeof(OpenCV_VideoIO_Plugin_API_preview), ABI_VERSION, API_VERSION,
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
CV_VERSION_MAJOR, CV_VERSION_MINOR, CV_VERSION_REVISION, CV_VERSION_STATUS,
"Microsoft Media Foundation OpenCV Video I/O plugin"
},
videoio: plugins API versioning
2020-12-04 11:18:16 +08:00
{
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
/* 1*/cv::CAP_MSMF,
/* 2*/cv::cv_capture_open,
/* 3*/cv::cv_capture_release,
/* 4*/cv::cv_capture_get_prop,
/* 5*/cv::cv_capture_set_prop,
/* 6*/cv::cv_capture_grab,
/* 7*/cv::cv_capture_retrieve,
/* 8*/cv::cv_writer_open,
/* 9*/cv::cv_writer_release,
/* 10*/cv::cv_writer_get_prop,
/* 11*/cv::cv_writer_set_prop,
/* 12*/cv::cv_writer_write
videoio: plugins API versioning
2020-12-04 11:18:16 +08:00
}
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
};
const OpenCV_VideoIO_Plugin_API_preview* opencv_videoio_plugin_init_v0(int requested_abi_version, int requested_api_version, void* /*reserved=NULL*/) CV_NOEXCEPT
{
videoio: plugins API versioning
2020-12-04 11:18:16 +08:00
if (requested_abi_version == ABI_VERSION && requested_api_version <= API_VERSION)
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
return &plugin_api;
return NULL;
}
#else // NEW_PLUGIN
static const OpenCV_VideoIO_Capture_Plugin_API capture_plugin_api =
{
{
sizeof(OpenCV_VideoIO_Capture_Plugin_API), CAPTURE_ABI_VERSION, CAPTURE_API_VERSION,
CV_VERSION_MAJOR, CV_VERSION_MINOR, CV_VERSION_REVISION, CV_VERSION_STATUS,
"Microsoft Media Foundation OpenCV Video I/O plugin"
},
{
/* 1*/cv::CAP_MSMF,
/* 2*/cv::cv_capture_open,
/* 3*/cv::cv_capture_release,
/* 4*/cv::cv_capture_get_prop,
/* 5*/cv::cv_capture_set_prop,
/* 6*/cv::cv_capture_grab,
/* 7*/cv::cv_capture_retrieve,
},
{
/* 8*/cv::cv_capture_open_with_params,
}
};
const OpenCV_VideoIO_Capture_Plugin_API* opencv_videoio_capture_plugin_init_v1(int requested_abi_version, int requested_api_version, void* /*reserved=NULL*/) CV_NOEXCEPT
{
if (requested_abi_version == CAPTURE_ABI_VERSION && requested_api_version <= CAPTURE_API_VERSION)
return &capture_plugin_api;
videoio: plugins API versioning
2020-12-04 11:18:16 +08:00
return NULL;
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
}
Merge pull request #19460 from mikhail-nikolskiy:videoio-hw videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * HW acceleration in FFMPEG backend * fixes on Windows, remove D3D9 * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * improve va test * Copyright * check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+ * CAP_MSMF test on .mp4 * .mp4 in test * .avi for GStreamer test * revert changes around seek() * cv_writer_open_with_params * params.warnUnusedParameters * VideoCaptureParameters in GStreamer * open_with_params * params->getUnused * Reduce PSNR threshold 33->32 (other tests use 30) * require FFMPEG 4.0+; PSNR 30 as in other tests * GStreamer AVI-demux plugin not installed in Ubuntu test environment? * fix build on very old ffmpeg * fix build on very old ffmpeg * fix build issues * fix build issues (static_cast) * FFMPEG built on Windows without H264 encoder? * fix for write_nothing test on VAAPI * fix warnings * fix cv_writer_get_prop in plugins * use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs * internal function hw_check_device() for device check/logging * two separate tests for HW read and write * image size 640x480 in encode test * WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency) * exception on VP9 SW encoder? * rebase master; refine info message * videoio: fix FFmpeg standalone plugin build * videoio(ffmpeg): eliminate MSVC build warnings * address review comments * videoio(hw): update videocapture_acceleration.read test - remove parallel decoding by SW code path - check PSNR against the original generated image * videoio: minor fixes * videoio(test): disable unsupported MSMF cases (SW and HW) * videoio(test): update PSNR thresholds for HW acceleration read * videoio(test): update debug messages * "hw_acceleration" whitelisting parameter * little optimization in test * D3D11VA supports decoders, doesn't support encoders * videoio(test): adjust PSNR threshold in write_read_position tests * videoio(ffmpeg): fix rejecting on acceleration device name mismatch * videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging * videoio: rework VideoAccelerationType behavior - enum is not a bitset - default value is backend specific - only '_NONE' and '_ANY' may fallback on software processing - specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration. * videoio(test): fix for current FFmpeg wrapper Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2021-03-01 23:51:04 +08:00
static const OpenCV_VideoIO_Writer_Plugin_API writer_plugin_api =
{
{
sizeof(OpenCV_VideoIO_Writer_Plugin_API), WRITER_ABI_VERSION, WRITER_API_VERSION,
CV_VERSION_MAJOR, CV_VERSION_MINOR, CV_VERSION_REVISION, CV_VERSION_STATUS,
"Microsoft Media Foundation OpenCV Video I/O plugin"
},
{
/* 1*/cv::CAP_MSMF,
/* 2*/cv::cv_writer_open,
/* 3*/cv::cv_writer_release,
/* 4*/cv::cv_writer_get_prop,
/* 5*/cv::cv_writer_set_prop,
/* 6*/cv::cv_writer_write
},
{
/* 7*/cv::cv_writer_open_with_params
}
};
const OpenCV_VideoIO_Writer_Plugin_API* opencv_videoio_writer_plugin_init_v1(int requested_abi_version, int requested_api_version, void* /*reserved=NULL*/) CV_NOEXCEPT
{
if (requested_abi_version == WRITER_ABI_VERSION && requested_api_version <= WRITER_API_VERSION)
return &writer_plugin_api;
return NULL;
}
#endif // NEW_PLUGIN
videoio: added plug-in support to MSMF backend
2020-04-08 01:25:10 +08:00
#endif // BUILD_PLUGIN
Reference in New Issue Copy Permalink