vcpkg/ports/tensorflow-cc/fix-dbg-build-errors.patch
Joachim Gehweiler 11b4a16bca
[tensorflow-cc] More or less completely rewritten port, update to 2.3.1 (#13028)
* fix build issues occurring from default bazel config being used because customized bazel config is stored in wrong directory

* [tensorflow-cc] Update CONTROL and ci.baseline.txt

* fix also applies to windows static build

* fix debug and static builds as well as library naming for non-linux
- add patch to fix debug builds
- add patch to fix exports for static linking
- really build debug (instead of cloning release)
- override bazel build options for debug (work around bazel bug)
- bazel doesn't support static libraries: work around by building dynamic library and constructing static linkage commands from build log
- Windows .pdb file can't be >4GB even on x64: work around using reduced debug information
- Windows doesn't support .lib files >4GB even on x64, so split into multiple libs
- vcpkg requires equal amount of libs for debug and release: work around using handcrafted empty dummy libs
- fix naming of libs (.dll on Windows and .dylib on macOS)

* upgrade tensorflow from v1.14 to v2.3
- adapt patch files to tensorflow code changes
- update bazel from v0.25.2 to v3.1
- on Windows use python installed on the host instead of embedded python obtained via vcpkg because embedded python lacks pip, which we need to obtain numpy
- on Windows add MSYS2 to the PATH so that bazel tools can access MSYS2 GIT
- add support for custom CA certificates when using HTTPS_PROXY

* fix execute process macro
The existing implementation totally screwed up commands if the command's arguments contained semicolons (this is the case, e.g., in the FindPython modules of the cmake distribution).

* extend overriden execute_process to more than one COMMAND as there actually are use cases for this

* added another patch required for tensorflow v2.3, fixed path and working directory

* Revert "incorporate changes from microsoft:master"

* Revert "Revert "incorporate changes from microsoft:master""

* final fixes for static build + improving out messages

* enabling linux and osx in CI to see if it works now

* simplified code, fixed version numbers, fixed generated include cmake file

* fix failing postbuild check on handcrafted empty dummy library by spreading the last real libraries contents over the required number of libraries

* remove dead code commit by mistake again

* improvements from code review

* cleaner fix for debug code

* find pip3 in PATH (PYTHON3_DIR apparently not valid for pip3)

* fix error in python helper script

* fix wrong libname in postbuild script

* fix python detection + switch to python on msys2 (instead of embedded python) for Windows as we need numpy

* fix order of arguments

* fix command (it may contain spaces such as C:\Program Files\...)

* revert last commit (root cause for CI failures is something different: there are line breaks in path)

* fix regex comparision
(value needs to be escaped as it may contains regex special characters such as brackets, eg C:/Program Files (x86)/...)

* fix linebreaks in generated file

* fix CRT linkage
(macOS doesn't support static CRT linkage; it's set to dynamic even static target triplets for macOS and linux)

* refactor implemenation to avoid as much code duplication as possible -- algorithmically identical

* fix version numbers in helper scripts

* enable work-around for Windows until bazel fix is available

* install missing python3-pip on linux

* fix linux build by patching

* apply timeout feature now available via merged master branch

* correct linux build patch

* improve debug build patches
(no functional difference because LOG(FATAL, ...) macro internally anyway calls abort(), which the compiler doesn't detect in debug mode...

* improve linux patch

* temporarily add debug to inspect what's going on on macOS CI

* remove temporary debug code and fix static linking scripts for linux and macOS

* fix regex escaping

* fix ambiguous match while grepping for the framework link command

* extend fix of ambiguous match while grepping for the framework link command

* fix what merge of master broke

* fix more what got broken by merging master
(all packages and their dependencies are now maintained manually instead of using pacman...)

* remove "unofficial" from filename

* added switch do distinct classic and manifest mode when generating config.cmake file

* create symlinks for libraries without version number

* fix linux postbuild script

* temporarily disable code making problems

* add note for linking on Linux and macOS

* forget to add README file in previous commit

* add file forgotton in macro fixing patch

* fix python library path

* fix macOS static link command

* update linkage instructions in README

* Update ports/tensorflow-cc/CONTROL

* Update ports/tensorflow-cc/portfile.cmake

* Update scripts/ci.baseline.txt

* use vcpkg_execute_required_process

* pass C_FLAGS and CXX_FLAGS to bazel

* fix INTERFACE_INCLUDE_DIRECTORIES

* fix optional c/cxx arguments

* also add linker opts

* update README

* merge static libs into one
to support force_load (cannot force_load both due to duplicate symbols)

* update README

* quote python path (it might contain spaces that don't get escaped inside outer quotes of bash command)

* fix python path also for static build

* add arm(64) as currently unsupported arch

* bazel 3.7 is available -> remove workaround

* update README, remove necessary c-ares from deps

* update msys package

* add uwp specific options, and minor general improvements

* fix string replace

* fix control file and windows path separator

* revert backslashes-fix -- the root cause was missing .exe extension

* upgrade to tf 2.3.1

* fix hard-coded version

* remove uwp work-in-progress code so that PR can be merged

* update README and print out usage info in portfile

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-windows

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* improve usage hints as discussed in review

* add comment

* apply changes from review

* make additional compiler / linker args space-proof

* Update ports/tensorflow-cc/README-macos

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-macos

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* apply changes from code review

* maybe fix the config files

* rob.maynard CRs

* fix windows static lib naming for first part

* Update ports/tensorflow-cc/generate_static_link_cmd_windows.py

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* apply rob maynards 1st suggestion

* nicole CRs

* format/fix-compile

* fix missing string termination

* prefer IMPORTED_LOCATION over IMPORTED_LOCATION_RELEASE to have default fall-back

* hopefully fix the issue where no libraries are generated

* final stuff

Co-authored-by: Gehweiler <Joachim_Gehweiler@McAfee.com>
Co-authored-by: wangli28 <wangli28@beyondsoft.com>
Co-authored-by: Jack·Boos·Yu <47264268+JackBoosY@users.noreply.github.com>
Co-authored-by: Joachim Gehweiler <joachim@Joachims-iMac.local>
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
2020-11-12 10:15:48 -08:00

112 lines
4.6 KiB
Diff

diff --git a/tensorflow/core/platform/path.cc b/tensorflow/core/platform/path.cc
--- a/tensorflow/core/platform/path.cc
+++ b/tensorflow/core/platform/path.cc
@@ -329,6 +329,7 @@
}
LOG(FATAL) << "No temp directory found.";
#endif
+ std::abort();
}
bool GetTestUndeclaredOutputsDir(string* dir) {
diff --git a/tensorflow/core/framework/device_base.cc b/tensorflow/core/framework/device_base.cc
--- a/tensorflow/core/framework/device_base.cc
+++ b/tensorflow/core/framework/device_base.cc
@@ -65,10 +65,12 @@
const DeviceAttributes& DeviceBase::attributes() const {
LOG(FATAL) << "Device does not implement attributes()";
+ std::abort();
}
const string& DeviceBase::name() const {
LOG(FATAL) << "Device does not implement name()";
+ std::abort();
}
void DeviceBase::set_eigen_cpu_device(Eigen::ThreadPoolDevice* d) {
diff --git a/tensorflow/core/kernels/depthtospace_op.cc b/tensorflow/core/kernels/depthtospace_op.cc
--- a/tensorflow/core/kernels/depthtospace_op.cc
+++ b/tensorflow/core/kernels/depthtospace_op.cc
@@ -117,13 +117,13 @@
// NCHW_VECT_C with 4 x qint8 can be treated as NCHW int32.
auto Tinput_v = input.template reinterpret_last_dimension<int32, 4>();
auto Toutput_v = outputs_tensor->reinterpret_last_dimension<int32, 4>();
- functor::DepthToSpaceOpFunctor<GPUDevice, int32, FORMAT_NCHW> functor;
- functor(context->eigen_device<GPUDevice>(), Tinput_v, block_size_,
+ functor::DepthToSpaceOpFunctor<Device, int32, FORMAT_NCHW> functor;
+ functor(context->eigen_device<Device>(), Tinput_v, block_size_,
Toutput_v);
return;
} else if (data_format_ == FORMAT_NCHW) {
- functor::DepthToSpaceOpFunctor<GPUDevice, T, FORMAT_NCHW> functor;
- functor(context->eigen_device<GPUDevice>(), Tinput, block_size_,
+ functor::DepthToSpaceOpFunctor<Device, T, FORMAT_NCHW> functor;
+ functor(context->eigen_device<Device>(), Tinput, block_size_,
Toutput);
return;
}
@@ -173,6 +173,15 @@
}
}
};
+#ifdef WIN32
+template <typename T>
+struct DepthToSpaceOpFunctor<CPUDevice, T, FORMAT_NCHW> {
+ void operator()(const CPUDevice& d, typename TTypes<T, 4>::ConstTensor input,
+ int block_size, typename TTypes<T, 4>::Tensor output) {
+ LOG(FATAL) << "dummy implementation to make debug build compile";
+ }
+};
+#endif
} // namespace functor
#define REGISTER(type) \
diff --git a/tensorflow/core/kernels/spacetodepth_op.cc b/tensorflow/core/kernels/spacetodepth_op.cc
--- a/tensorflow/core/kernels/spacetodepth_op.cc
+++ b/tensorflow/core/kernels/spacetodepth_op.cc
@@ -132,18 +132,18 @@
// NCHW_VECT_C with 4 x qint8 can be treated as NCHW int32.
auto Tinput_v = input.template reinterpret_last_dimension<int32, 4>();
auto Toutput_v = outputs_tensor->reinterpret_last_dimension<int32, 4>();
- functor::SpaceToDepthOpFunctor<GPUDevice, int32, FORMAT_NCHW> functor;
- functor(context->eigen_device<GPUDevice>(), Tinput_v, block_size_,
+ functor::SpaceToDepthOpFunctor<Device, int32, FORMAT_NCHW> functor;
+ functor(context->eigen_device<Device>(), Tinput_v, block_size_,
Toutput_v);
} else if (data_format_ == FORMAT_NCHW) {
CHECK((std::is_same<T, RT>::value));
- functor::SpaceToDepthOpFunctor<GPUDevice, RT, FORMAT_NCHW> functor;
- functor(context->eigen_device<GPUDevice>(), input.tensor<RT, 4>(),
+ functor::SpaceToDepthOpFunctor<Device, RT, FORMAT_NCHW> functor;
+ functor(context->eigen_device<Device>(), input.tensor<RT, 4>(),
block_size_, outputs_tensor->tensor<RT, 4>());
} else {
CHECK((std::is_same<T, RT>::value));
- functor::SpaceToDepthOpFunctor<GPUDevice, RT, FORMAT_NHWC> functor;
- functor(context->eigen_device<GPUDevice>(), input.tensor<RT, 4>(),
+ functor::SpaceToDepthOpFunctor<Device, RT, FORMAT_NHWC> functor;
+ functor(context->eigen_device<Device>(), input.tensor<RT, 4>(),
block_size_, outputs_tensor->tensor<RT, 4>());
}
} else {
@@ -188,6 +188,15 @@
}
}
};
+#ifdef WIN32
+template <typename T>
+struct SpaceToDepthOpFunctor<CPUDevice, T, FORMAT_NCHW> {
+ void operator()(const CPUDevice& d, typename TTypes<T, 4>::ConstTensor input,
+ int block_size, typename TTypes<T, 4>::Tensor output) {
+ LOG(FATAL) << "dummy implementation to make debug build compile";
+ }
+};
+#endif
} // namespace functor
#define REGISTER(type) \