Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
// This file is part of OpenCV project.
|
|
|
|
// It is subject to the license terms in the LICENSE file found in the top-level directory
|
|
|
|
// of this distribution and at http://opencv.org/license.html.
|
|
|
|
|
|
|
|
#include "perf_precomp.hpp"
|
|
|
|
|
|
|
|
namespace opencv_test {
|
|
|
|
|
|
|
|
struct EinsumParams {
|
|
|
|
int inputSize;
|
|
|
|
int outputSize;
|
|
|
|
std::string equation;
|
|
|
|
std::vector<MatShape> einsumInpShapes;
|
2023-11-16 21:20:17 +08:00
|
|
|
EinsumParams(std::string equation_, std::vector<MatShape> einsumInpShapes_ = std::vector<MatShape>())
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
{
|
2023-11-16 21:20:17 +08:00
|
|
|
inputSize = einsumInpShapes_.size();
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
equation = equation_;
|
|
|
|
einsumInpShapes = einsumInpShapes_;
|
|
|
|
}
|
|
|
|
};
|
|
|
|
|
|
|
|
static inline void PrintTo(const EinsumParams& params, ::std::ostream* os) {
|
2023-11-16 21:20:17 +08:00
|
|
|
(*os) << "Equation=" << params.equation << " ";
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
|
|
|
(*os) << "InputShape={";
|
|
|
|
for(int i = 0; i < params.einsumInpShapes.size(); i++)
|
|
|
|
{
|
|
|
|
(*os) << "{";
|
|
|
|
for(int j = 0; j < params.einsumInpShapes[i].size(); j++)
|
|
|
|
{
|
|
|
|
(*os) << params.einsumInpShapes[i][j] << ((j < params.einsumInpShapes[i].size() - 1) ? ", " : "");
|
|
|
|
}
|
|
|
|
(*os) << ((i < params.einsumInpShapes.size() - 1) ? "}, " : "}");
|
|
|
|
}
|
|
|
|
(*os) << "}";
|
|
|
|
}
|
|
|
|
|
|
|
|
// test cases
|
|
|
|
static const EinsumParams testEinsumConfigs[] = {
|
|
|
|
// TODO: Add tests with one input after ellips merge
|
2023-11-16 21:20:17 +08:00
|
|
|
{"ij, jk -> ik", {{2, 3}, {3, 2}}},
|
|
|
|
{"ij, jk -> ik", {{20, 30}, {30, 20}}},
|
|
|
|
{"ij, jk -> ik", {{113, 127}, {127, 113}}},
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
2023-11-16 21:20:17 +08:00
|
|
|
{"imkj, injs -> imnks", {{1, 4, 7, 9}, {1, 5, 9, 8}}},
|
|
|
|
{"imkj, injs -> imnks", {{1, 4, 70, 90}, {1, 5, 90, 80}}},
|
|
|
|
{"imkj, injs -> imnks", {{1, 4, 73, 91}, {1, 5, 91, 57}}},
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
2023-11-16 21:20:17 +08:00
|
|
|
{"ij -> i", {{30, 40}}},
|
|
|
|
{"ij -> i", {{113, 374}}},
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
2023-11-16 21:20:17 +08:00
|
|
|
{"...ij -> ...i", {{30, 40}}},
|
|
|
|
{"...ij -> ...i", {{113, 374}}},
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
2023-11-16 21:20:17 +08:00
|
|
|
{"...ij, ...jk -> ...ik", {{40, 50}, {50, 80}}},
|
|
|
|
{"...ij, ...jk -> ...ik", {{47, 51}, {51, 83}}},
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
class Layer_Einsum: public TestBaseWithParam<EinsumParams> {};
|
|
|
|
|
|
|
|
PERF_TEST_P_(Layer_Einsum, einsum) {
|
|
|
|
const EinsumParams& params = GetParam();
|
|
|
|
LayerParams lp;
|
|
|
|
lp.type = "Einsum";
|
|
|
|
lp.name = "testEinsum";
|
|
|
|
lp.set("equation", params.equation);
|
|
|
|
lp.set("inputSize", params.inputSize);
|
2023-11-16 21:20:17 +08:00
|
|
|
lp.set("outputSize", 1);
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
|
|
|
CV_CheckFalse(params.einsumInpShapes.empty(), "ERROR no inputs shapes provided");
|
|
|
|
|
|
|
|
for (int i = 0; i < params.einsumInpShapes.size(); i++) {
|
|
|
|
lp.set("inputShapes" + cv::format("%d", i), DictValue::arrayInt(params.einsumInpShapes[i].begin(), params.einsumInpShapes[i].size()));
|
|
|
|
}
|
|
|
|
|
|
|
|
Net net;
|
|
|
|
std::vector<Mat> inputs;
|
|
|
|
std::vector<std::string> input_names;
|
2023-11-16 21:20:17 +08:00
|
|
|
int id = net.addLayer(lp.name, lp.type, lp);
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
2023-11-16 21:20:17 +08:00
|
|
|
for (int i = 0; i < params.inputSize; ++i) {
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
// create inputs
|
2023-11-16 21:20:17 +08:00
|
|
|
inputs.emplace_back(Mat(params.einsumInpShapes[i].size(), params.einsumInpShapes[i].data(), CV_32FC1));
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
2023-11-16 21:20:17 +08:00
|
|
|
// connect each input to the layer
|
|
|
|
net.connect(0, i, id, i);
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
2023-11-16 21:20:17 +08:00
|
|
|
// create input names dynamically, assuming input naming follows a consistent pattern
|
|
|
|
input_names.emplace_back("input" + std::to_string(i + 1));
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
//warm up
|
2023-11-16 21:20:17 +08:00
|
|
|
std::vector<Mat> outputs;
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
net.setInputsNames(input_names);
|
|
|
|
for (int i = 0; i < input_names.size(); i++){
|
|
|
|
net.setInput(inputs[i], input_names[i]);
|
|
|
|
}
|
2023-11-16 21:20:17 +08:00
|
|
|
net.forward(outputs, "testEinsum");
|
Merge pull request #24445 from Abdurrahheem:ash/dev_einsum_pref
Einsum Layer Performance Test #24445
## This PR adds performance tests for Einsum Layer. See below results of performance test on different inputs
**Notation:**
- WX: windows10_x64
- MX: macos_x64
- MA: macos_arm64
- UX: ubuntu_x64
- UA: ubuntu_arm64
All data in ms (milliseconds).
Gemm is backend for matrix multiplication
---
Benchmarks:
| Equation | Inputs Mat Dims | UX (ms) | UA (ms) | MX (ms) | MA (ms) | WX (ms) |
|-------------------------|-----------------------------------|----------------|---------|---------|---------|---------|
| "ij, jk -> ik" | [2, 3], [3,2] | 0.04 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [20, 30], [30,20] | 0.08 ± 0.00 | - | - | - | - |
| "ij, jk -> ik" | [113, 127], [127,113] | 2.41 ± 0.05 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 7, 9], [1, 5, 9, 8] | 0.11 ± 0.00 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 70, 90], [1, 5, 90, 80] | 15.49 ± 0.46 | - | - | - | - |
| "imkj, injs -> imnks" | [1, 4, 73, 91], [1, 5, 91, 57] | 11.53 ± 0.06 | - | - | - | - |
| "ij -> i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "ij -> i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [30, 40] | 0.03 ± 0.00 | - | - | - | - |
| "...ij -> ...i" | [113, 374] | 0.13 ± 0.00 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [40, 50], [50,80] | 0.37 ± 0.01 | - | - | - | - |
| "...ij, ...jk -> ...ik" | [47, 51], [51, 83] | 0.43 ± 0.01 | - | - | - | - |
-----
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-11-08 16:56:21 +08:00
|
|
|
|
|
|
|
TEST_CYCLE()
|
|
|
|
{
|
|
|
|
net.forward(outputs, "testEinsum");
|
|
|
|
}
|
|
|
|
SANITY_CHECK_NOTHING();
|
|
|
|
}
|
|
|
|
|
|
|
|
INSTANTIATE_TEST_CASE_P(/**/, Layer_Einsum, testing::ValuesIn(testEinsumConfigs));
|
|
|
|
|
|
|
|
}; //namespace
|