C++ Library Manager for Windows, Linux, and MacOS
Go to file
Joachim Gehweiler 11b4a16bca
[tensorflow-cc] More or less completely rewritten port, update to 2.3.1 (#13028)
* fix build issues occurring from default bazel config being used because customized bazel config is stored in wrong directory

* [tensorflow-cc] Update CONTROL and ci.baseline.txt

* fix also applies to windows static build

* fix debug and static builds as well as library naming for non-linux
- add patch to fix debug builds
- add patch to fix exports for static linking
- really build debug (instead of cloning release)
- override bazel build options for debug (work around bazel bug)
- bazel doesn't support static libraries: work around by building dynamic library and constructing static linkage commands from build log
- Windows .pdb file can't be >4GB even on x64: work around using reduced debug information
- Windows doesn't support .lib files >4GB even on x64, so split into multiple libs
- vcpkg requires equal amount of libs for debug and release: work around using handcrafted empty dummy libs
- fix naming of libs (.dll on Windows and .dylib on macOS)

* upgrade tensorflow from v1.14 to v2.3
- adapt patch files to tensorflow code changes
- update bazel from v0.25.2 to v3.1
- on Windows use python installed on the host instead of embedded python obtained via vcpkg because embedded python lacks pip, which we need to obtain numpy
- on Windows add MSYS2 to the PATH so that bazel tools can access MSYS2 GIT
- add support for custom CA certificates when using HTTPS_PROXY

* fix execute process macro
The existing implementation totally screwed up commands if the command's arguments contained semicolons (this is the case, e.g., in the FindPython modules of the cmake distribution).

* extend overriden execute_process to more than one COMMAND as there actually are use cases for this

* added another patch required for tensorflow v2.3, fixed path and working directory

* Revert "incorporate changes from microsoft:master"

* Revert "Revert "incorporate changes from microsoft:master""

* final fixes for static build + improving out messages

* enabling linux and osx in CI to see if it works now

* simplified code, fixed version numbers, fixed generated include cmake file

* fix failing postbuild check on handcrafted empty dummy library by spreading the last real libraries contents over the required number of libraries

* remove dead code commit by mistake again

* improvements from code review

* cleaner fix for debug code

* find pip3 in PATH (PYTHON3_DIR apparently not valid for pip3)

* fix error in python helper script

* fix wrong libname in postbuild script

* fix python detection + switch to python on msys2 (instead of embedded python) for Windows as we need numpy

* fix order of arguments

* fix command (it may contain spaces such as C:\Program Files\...)

* revert last commit (root cause for CI failures is something different: there are line breaks in path)

* fix regex comparision
(value needs to be escaped as it may contains regex special characters such as brackets, eg C:/Program Files (x86)/...)

* fix linebreaks in generated file

* fix CRT linkage
(macOS doesn't support static CRT linkage; it's set to dynamic even static target triplets for macOS and linux)

* refactor implemenation to avoid as much code duplication as possible -- algorithmically identical

* fix version numbers in helper scripts

* enable work-around for Windows until bazel fix is available

* install missing python3-pip on linux

* fix linux build by patching

* apply timeout feature now available via merged master branch

* correct linux build patch

* improve debug build patches
(no functional difference because LOG(FATAL, ...) macro internally anyway calls abort(), which the compiler doesn't detect in debug mode...

* improve linux patch

* temporarily add debug to inspect what's going on on macOS CI

* remove temporary debug code and fix static linking scripts for linux and macOS

* fix regex escaping

* fix ambiguous match while grepping for the framework link command

* extend fix of ambiguous match while grepping for the framework link command

* fix what merge of master broke

* fix more what got broken by merging master
(all packages and their dependencies are now maintained manually instead of using pacman...)

* remove "unofficial" from filename

* added switch do distinct classic and manifest mode when generating config.cmake file

* create symlinks for libraries without version number

* fix linux postbuild script

* temporarily disable code making problems

* add note for linking on Linux and macOS

* forget to add README file in previous commit

* add file forgotton in macro fixing patch

* fix python library path

* fix macOS static link command

* update linkage instructions in README

* Update ports/tensorflow-cc/CONTROL

* Update ports/tensorflow-cc/portfile.cmake

* Update scripts/ci.baseline.txt

* use vcpkg_execute_required_process

* pass C_FLAGS and CXX_FLAGS to bazel

* fix INTERFACE_INCLUDE_DIRECTORIES

* fix optional c/cxx arguments

* also add linker opts

* update README

* merge static libs into one
to support force_load (cannot force_load both due to duplicate symbols)

* update README

* quote python path (it might contain spaces that don't get escaped inside outer quotes of bash command)

* fix python path also for static build

* add arm(64) as currently unsupported arch

* bazel 3.7 is available -> remove workaround

* update README, remove necessary c-ares from deps

* update msys package

* add uwp specific options, and minor general improvements

* fix string replace

* fix control file and windows path separator

* revert backslashes-fix -- the root cause was missing .exe extension

* upgrade to tf 2.3.1

* fix hard-coded version

* remove uwp work-in-progress code so that PR can be merged

* update README and print out usage info in portfile

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-windows

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/portfile.cmake

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* improve usage hints as discussed in review

* add comment

* apply changes from review

* make additional compiler / linker args space-proof

* Update ports/tensorflow-cc/README-macos

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-linux

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* Update ports/tensorflow-cc/README-macos

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* apply changes from code review

* maybe fix the config files

* rob.maynard CRs

* fix windows static lib naming for first part

* Update ports/tensorflow-cc/generate_static_link_cmd_windows.py

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>

* apply rob maynards 1st suggestion

* nicole CRs

* format/fix-compile

* fix missing string termination

* prefer IMPORTED_LOCATION over IMPORTED_LOCATION_RELEASE to have default fall-back

* hopefully fix the issue where no libraries are generated

* final stuff

Co-authored-by: Gehweiler <Joachim_Gehweiler@McAfee.com>
Co-authored-by: wangli28 <wangli28@beyondsoft.com>
Co-authored-by: Jack·Boos·Yu <47264268+JackBoosY@users.noreply.github.com>
Co-authored-by: Joachim Gehweiler <joachim@Joachims-iMac.local>
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
2020-11-12 10:15:48 -08:00
.github [vcpkg github] Update pull request template (#11260) 2020-05-11 14:52:07 -07:00
docs [vcpkg] User help: selecting-library-features.md (#14189) 2020-11-02 17:26:25 -08:00
ports [tensorflow-cc] More or less completely rewritten port, update to 2.3.1 (#13028) 2020-11-12 10:15:48 -08:00
scripts [tensorflow-cc] More or less completely rewritten port, update to 2.3.1 (#13028) 2020-11-12 10:15:48 -08:00
toolsrc [tensorflow-cc] More or less completely rewritten port, update to 2.3.1 (#13028) 2020-11-12 10:15:48 -08:00
triplets [szip, openssl] Fix problems with mingw (#14460) 2020-11-09 15:21:03 -08:00
.gitattributes [vcpkg] Make C++ the primary github language (#12810) 2020-08-10 09:28:50 -07:00
.gitignore [vcpkg] Always use version-string in generator (#14262) 2020-10-27 14:36:22 -07:00
.vcpkg-root Add .vcpkg-root file that signifies the root directory 2016-09-19 18:49:38 -07:00
bootstrap-vcpkg.bat bootstrap.bat now forwards args to ps1 2018-06-08 16:26:57 -07:00
bootstrap-vcpkg.sh [vcpkg] Update to CMake 3.14.0 2019-03-20 16:23:02 -07:00
CHANGELOG.md Update CHANGELOG 2020.11.03 (#14377) 2020-11-03 14:58:32 -08:00
CONTRIBUTING.md Moved CONTRIBUTING.md to root. Fixes #54. 2016-09-21 13:35:51 -07:00
LICENSE.txt [vcpkg] remove text from license (#8082) 2019-09-06 14:24:19 -07:00
NOTICE.txt Add third party notices -- copied from chakracore (#7403) 2019-07-24 11:07:55 -07:00
README_es.md [vcpkg/docs] Add Spanish README (#13735) 2020-09-30 09:18:49 -07:00
README_fr.md [vcpkg] translate the README to french (#13971) 2020-10-30 13:21:48 -07:00
README_zh_CN.md [documentation] Fix manifest link (#13685) 2020-09-24 11:36:44 -07:00
README.md [vcpkg] translate the README to french (#13971) 2020-10-30 13:21:48 -07:00

Vcpkg: Overview

中文总览 Español Français

Vcpkg helps you manage C and C++ libraries on Windows, Linux and MacOS. This tool and ecosystem are constantly evolving, and we always appreciate contributions!

If you've never used vcpkg before, or if you're trying to figure out how to use vcpkg, check out our Getting Started section for how to start using vcpkg.

For short description of available commands, once you've installed vcpkg, you can run vcpkg help, or vcpkg help [command] for command-specific help.

Build Status

Table of Contents

Getting Started

First, follow the quick start guide for either Windows, or macOS and Linux, depending on what you're using.

For more information, see Installing and Using Packages. If a library you need is not present in the vcpkg catalog, you can open an issue on the GitHub repo where the vcpkg team and community can see it, and potentially add the port to vcpkg.

After you've gotten vcpkg installed and working, you may wish to add tab completion to your shell.

Finally, if you're interested in the future of vcpkg, check out the manifest guide! This is an experimental feature and will likely have bugs, so try it out and open all the issues!

Quick Start: Windows

Prerequisites:

  • Windows 7 or newer
  • Git
  • Visual Studio 2015 Update 3 or greater with the English language pack

First, download and bootstrap vcpkg itself; it can be installed anywhere, but generally we recommend using vcpkg as a submodule for CMake projects, and installing it globally for Visual Studio projects. We recommend somewhere like C:\src\vcpkg or C:\dev\vcpkg, since otherwise you may run into path issues for some port build systems.

> git clone https://github.com/microsoft/vcpkg
> .\vcpkg\bootstrap-vcpkg.bat

To install the libraries for your project, run:

> .\vcpkg\vcpkg install [packages to install]

You can also search for the libraries you need with the search subcommand:

> .\vcpkg\vcpkg search [search term]

In order to use vcpkg with Visual Studio, run the following command (may require administrator elevation):

> .\vcpkg\vcpkg integrate install

After this, you can now create a New non-CMake Project (or open an existing one). All installed libraries are immediately ready to be #include'd and used in your project without additional configuration.

If you're using CMake with Visual Studio, continue here.

In order to use vcpkg with CMake outside of an IDE, you can use the toolchain file:

> cmake -B [build directory] -S . -DCMAKE_TOOLCHAIN_FILE=[path to vcpkg]/scripts/buildsystems/vcpkg.cmake
> cmake --build [build directory]

With CMake, you will still need to find_package and the like to use the libraries. Check out the CMake section for more information, including on using CMake with an IDE.

For any other tools, including Visual Studio Code, check out the integration guide.

Quick Start: Unix

Prerequisites for Linux:

Prerequisites for macOS:

First, download and bootstrap vcpkg itself; it can be installed anywhere, but generally we recommend using vcpkg as a submodule for CMake projects.

$ git clone https://github.com/microsoft/vcpkg
$ ./vcpkg/bootstrap-vcpkg.sh

To install the libraries for your project, run:

$ ./vcpkg/vcpkg install [packages to install]

You can also search for the libraries you need with the search subcommand:

$ ./vcpkg/vcpkg search [search term]

In order to use vcpkg with CMake, you can use the toolchain file:

$ cmake -B [build directory] -S . -DCMAKE_TOOLCHAIN_FILE=[path to vcpkg]/scripts/buildsystems/vcpkg.cmake
$ cmake --build [build directory]

With CMake, you will still need to find_package and the like to use the libraries. Check out the CMake section for more information on how best to use vcpkg with CMake, and CMake Tools for VSCode.

For any other tools, check out the integration guide.

Installing Linux Developer Tools

Across the different distros of Linux, there are different packages you'll need to install:

  • Debian, Ubuntu, popOS, and other Debian-based distributions:
$ sudo apt-get update
$ sudo apt-get install build-essential tar curl zip unzip
  • CentOS
$ sudo yum install centos-release-scl
$ sudo yum install devtoolset-7
$ scl enable devtoolset-7 bash

For any other distributions, make sure you're installing g++ 6 or above. If you want to add instructions for your specific distro, please open a PR!

Installing macOS Developer Tools

On macOS 10.15, the only thing you should need to do is run the following in your terminal:

$ xcode-select --install

Then follow along with the prompts in the windows that comes up.

On macOS 10.14 and previous, you'll also need to install g++ from homebrew; follow the instructions in the following section.

Installing GCC for macOS before 10.15

This will only be necessary if you're using a macOS version from before 10.15. Installing homebrew should be very easy; check out <brew.sh> for more information, but at its simplest, run the following command:

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Then, in order to grab an up-to-date version of gcc, run the following:

$ brew install gcc

You'll then be able to bootstrap vcpkg along with the quick start guide

Using vcpkg with CMake

If you're using vcpkg with CMake, the following may help!

Visual Studio Code with CMake Tools

Adding the following to your workspace settings.json will make CMake Tools automatically use vcpkg for libraries:

{
  "cmake.configureSettings": {
    "CMAKE_TOOLCHAIN_FILE": "[vcpkg root]/scripts/buildsystems/vcpkg.cmake"
  }
}

Vcpkg with Visual Studio CMake Projects

Open the CMake Settings Editor, and under CMake toolchain file, add the path to the vcpkg toolchain file:

[vcpkg root]/scripts/buildsystems/vcpkg.cmake

Vcpkg with CLion

Open the Toolchains settings (File > Settings on Windows and Linux, CLion > Preferences on macOS), and go to the CMake settings (Build, Execution, Deployment > CMake). Finally, in CMake options, add the following line:

-DCMAKE_TOOLCHAIN_FILE=[vcpkg root]/scripts/buildsystems/vcpkg.cmake

Unfortunately, you'll have to add this to each profile.

Vcpkg as a Submodule

When using vcpkg as a submodule of your project, you can add the following to your CMakeLists.txt before the first project() call, instead of passing CMAKE_TOOLCHAIN_FILE to the cmake invocation.

set(CMAKE_TOOLCHAIN_FILE ${CMAKE_CURRENT_SOURCE_DIR}/vcpkg/scripts/buildsystems/vcpkg.cmake
  CACHE STRING "Vcpkg toolchain file")

This will still allow people to not use vcpkg, by passing the CMAKE_TOOLCHAIN_FILE directly, but it will make the configure-build step slightly easier.

Tab-Completion/Auto-Completion

vcpkg supports auto-completion of commands, package names, and options in both powershell and bash. To enable tab-completion in the shell of your choice, run:

> .\vcpkg integrate powershell

or

$ ./vcpkg integrate bash

depending on the shell you use, then restart your console.

Examples

See the documentation for specific walkthroughs, including installing and using a package, adding a new package from a zipfile, and adding a new package from a GitHub repo.

Our docs are now also available online at ReadTheDocs: https://vcpkg.readthedocs.io/!

See a 4 minute video demo.

Contributing

Vcpkg is an open source project, and is thus built with your contributions. Here are some ways you can contribute:

Please refer to our Contributing Guide for more details.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or email opencode@microsoft.com with any additional questions or comments.

License

The code in this repository is licensed under the MIT License.

Telemetry

vcpkg collects usage data in order to help us improve your experience. The data collected by Microsoft is anonymous. You can opt-out of telemetry by re-running the bootstrap-vcpkg script with -disableMetrics, passing --disable-metrics to vcpkg on the command line, or by setting the VCPKG_DISABLE_METRICS environment variable.

Read more about vcpkg telemetry at docs/about/privacy.md