Commit Graph

817 Commits

Author SHA1 Message Date
ras0219
0b5bbe30d9
[vcpkg] Lift --x-json to a global option, implement experimental x-package-info command (#12845)
* [vcpkg] Improve error reporting in vcpkg::Json

* [vcpkg] Lift --x-json to a common option

* [vcpkg] Address warnings-as-errors in VS2015

* [vcpkg] Remove unused local

* [vcpkg] Extract vcpkg::Install::get_cmake_usage

* [vcpkg] Implement vcpkg::serialize_ipv(ipv, paths)

* [vcpkg] Implement x-package-info to enable tooling

* [vcpkg] Fixup tests to respect new cli mode

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-08-13 18:36:33 -07:00
ras0219
c771e7bd07
[vcpkg] Fix resolution of default features when using Manifest mode (#12829)
* [vcpkg] Fix resolution of default features when using Manifest mode

During manifest mode, the dependencies in the manifest should be treated as explicitly specified -- curl[core] should not install curl's default features.

* [vcpkg] Improve error message when failed to parse manifest file

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-08-12 12:35:26 -07:00
ras0219
481738beae
[vcpkg] Add vcpkg export to E2E tests. Enable E2E tests on all platforms. (#12198)
* [vcpkg] Add `vcpkg export` to E2E tests. Enable E2E tests on MacOS.

* [vcpkg] Fix export --raw --output-dir=/path/ by changing directory to new export root

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2020-08-10 10:22:51 -07:00
nicole mazzuca
40c8aab459
[vcpkg] Fix vs2015 build (#12789)
* move windows bootstrap directory

* fix paths

* actually fix the build

* fix path
2020-08-06 17:04:47 -07:00
Alexander Neumann
4fb846bd44
[vcpkg] add environment variable VCPKG_OVERLAY_PORTS (#12640)
* [vcpkg] add environment variable VCPKG_OVERLAY_PORTS

* add documentation for VCPKG_OVERLAY_PORTS

* append env overlay instead of prepending to give explicitly listed overlays priority

* fix formating (manually)

* manually fixing formating

* improve help text

Co-authored-by: ras0219 <533828+ras0219@users.noreply.github.com>

Co-authored-by: ras0219 <533828+ras0219@users.noreply.github.com>
2020-08-05 11:21:42 -07:00
nicole mazzuca
1c2af99415
[vcpkg format-manifest] Add convert-control flag (#12471)
* [vcpkg format-manifest] initial convert-control attempt

TODO: manifest comments! we should keep $directives

* Finalize x-format-manifest

First, fix Json::parse -- "\c", for any c, was incorrectly parsed.
It would emit the escaped character, and then parse the character, so
that `\b` would give you { '\b', 'b' }.

Second, canonicalize source paragraphs as we're parsing them. This found
an error in qt5 -- The `declarative` feature was listed twice, and we
now catch it, so I removed the second paragraph.

Add PlatformExpression::complexity to allow ordering platform
expressions in a somewhat reasonable way.

Notes:
  - We allow `all_modules` as a feature name for back-compat with
    paraview
  - In order to actually convert CONTROL to vcpkg.json, we'd need to
    rename the qt5 `default` feature.
  - We need to add support for $directives in x-format-manifest

* fix qt5 port

* format

* fix compile

* fix tests for canonicalization

* Clean up code

* add error message for nothing to format

* add extra_info field

* add `const X&` overloads for `Object::insert[_or_replace]`

* fix compile

* simple CRs

* add tests

* format

* Fix mosquitto port file

also unmerge a line

* fail the tests on malformed manifest

* fix format_all

* fix coroutine port-version

* format manifests
2020-08-02 10:08:07 -07:00
nicole mazzuca
f225ba822b
[vcpkg manifest] look at platform directives (#12548)
Fixes #12538
2020-08-01 13:46:26 -07:00
nicole mazzuca
54ec974afe
[vcpkg] Refactor commands 2: Electric Boogaloo (#12641)
* Add BasicCommand and VersionCommand

* Add ContactCommand

* test get_available_commands_type_c

* Change get_available_commands_type_c to return objects

* Add TripletCommand & InstallCommand

* Add SetInstalledCommand

* add linking tests

* Add CICommand

* Add remaining *Command objects

* Add tests for commands_type_a

* Move over to using const TripletCommand* for commands_type_a

* Add PathsCommand

* Add SearchCommand

* add test for commands_type_b

* add *Command for all type b commands

* Switch from function pointers to PathsCommand for everything

* format

* rename get_available_commands

also remove CommandType* types
2020-08-01 13:45:17 -07:00
nicole mazzuca
56fffbe49d
[vcpkg] Refactor Commands: Part 1 (#12585)
* Rename commands.exportifw.cpp -> export.ifw.cpp

* move DryRun from commands.h -> commands.interface.h

* move vcpkgcmdarguments and vcpkgpaths over to commands.interface.h

* move vcpkg::Commands::BuildExternal -> commands.buildexternal.h

* add commands.*.h for the commands.*.cpp

* move vcpkg::Commands::* to commands.*.h

* move vcpkg::Commands::{Hash,Fetch} to their own files

* change include commands.h -> commands.*.h in commands.*.cpp

* remove commands.*.h from commands.h

* join vcpkg::Commands::* into one namespace line

* fix vcxproj build
2020-07-31 11:53:42 -07:00
Andrei Lebedev
077c0746be
[vcpkg] Add initial s390x support (#12534)
Signed-off-by: Andrei Lebedev <lebdron@gmail.com>
2020-07-23 14:16:01 -07:00
Cristian Adam
f4bd64233a
Build fix with MinGW GCC 9.2.0 (#12084)
* Build fix with MinGW GCC 9.2.0

* Fix MinGW tripplets

Fixes #12065

* clang-format

* Fix unused read_symlink_implementation warning marked as error

read_symlink_implementation was used only in copy_symlink_implementation
when #if defined(_WIN32) && !VCPKG_USE_STD_FILESYSTEM  was true.

Removed the warning otherwise.

Co-authored-by: JackBoosY <yuzaiyang@beyondsoft.com>
2020-07-18 08:48:01 -07:00
nicole mazzuca
6bf5adff93
[vcpkg] Clean up command switch code (#12351)
* [vcpkg] Clean up command switch code

Make it more similar to the non-command switch code

* format

* fix the tests

* reformat

* format

* wip

* support x- for command options

* fix autocomplete

* format
2020-07-15 17:29:18 -07:00
nicole mazzuca
d2620cf02b
[vcpkg formatting] Turn off DeriveLineEnding (#12368)
* [vcpkg formatting] Turn off DeriveLineEnding

* format

* Add newlines to the end of files

Since we're reformatting anyways
2020-07-14 08:50:19 -07:00
ras0219
385fb52332
[vcpkg] Track hashes of cmake script helpers in consuming portfiles (#12254)
* [vcpkg] Track hashes of cmake script helpers in consuming portfiles

* [vcpkg] Format

* [vcpkg_find_acquire_program] Use cpkg_from_sourceforge for FLEX and BISON

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-13 13:43:39 -07:00
ras0219
52a9d9a9e4
[vcpkg] Enable binary caching by default (#12370)
* [vcpkg] Enable binary caching by default. Support `x-` migration.

Fix passing multiple copies of single parameter arguments.

* [vcpkg] Handle x- prefixes for general arguments

* [vcpkg] Fix #12285 and improve documentation of default binary cache path

* [vcpkg] Revert x- prefix homogenization for per-command arguments

* [vcpkg] Only use accelerated compiler detection for Windows Desktop + Ninja. Improve breadcrumbs for users encountering issues.

* [vcpkg] Fix compiler tracking not pre-downloading Ninja. Fix compiler tracking not looking in -err.log.

* [vcpkg] Update toolsrc/src/vcpkg/binarycaching.cpp

Co-authored-by: Billy O'Neal <bion@microsoft.com>

* [vcpkg] Format

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Billy O'Neal <bion@microsoft.com>
2020-07-13 12:03:53 -07:00
nicole mazzuca
0523b5eb57
[vcpkg formatting] Fix format regex (#12369)
* [vcpkg formatting] correct the header regexes

* format
2020-07-11 21:07:51 -07:00
nicole mazzuca
8a8d725b43
[vcpkg] Remove unnecessary work queue (#12350)
this work queue implementation was added at some point, and is no longer used anywhere.
Delete it as not used; if we need it again, we can grab it from the history
2020-07-09 16:04:59 -07:00
ras0219
a571c8ecc0
[vcpkg] Move default binary cache from $root/archives to user-wide directory (#12256)
* [vcpkg] Move default binary cache from `$root/archives` to user-wide directory

Teach nuget/nugetconfig binary sources to use read/write.

* [vcpkg] Reformat

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-09 15:14:05 -07:00
nicole mazzuca
3871d73349
[vcpkg manifests] fix some issues (#12227)
Fixes #12190
Fixes #12208
Fixes #12234
Fixes #12286
Fixes #12315
Fixes #12186
Fixes #12331
Fixes googleapis/google-cloud-cpp#4487
2020-07-09 12:32:31 -07:00
ras0219
f9bdf13953
[vcpkg] Implement --editable (#12200)
--editable suppresses binary caching and source re-extraction for packages listed on the command line (similar to --head).

This fundamentally changes the port maintenance loop, so several example documents were updated. To avoid users having substantial changes suddenly destroyed by forgetting to pass --editable, "clean" sources have different extract locations.

The undocumented command `build` implies `--editable`.

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-07 13:53:19 -07:00
nicole mazzuca
6a41626eaf
[vcpkg] Format the C++ in CI (#11655)
* [vcpkg] Format the C++ in the CI

* format the C++

* CR
2020-07-06 16:45:34 -07:00
ras0219
e82e56f27f
[vcpkg] Remove use of std::variant and std::visit to fix VS2015. (#12242)
Fixes #12220

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-06 11:03:38 -07:00
Billy O'Neal
a28bfe7674
[vcpkg] Remove the tombstones and 'ignore' baseline concepts. (#12197)
This changes our PR builds to treat 'fail' in the ci.baseline.txt as 'skip' instead of using tombstones.

We currently have large numbers of spurious failures that get enshrined in PRs through no fault of a PR author, removing the tombstones concept will fix those by allowing the user to retry. This does mean we accept some risk of not detecting when a port is 'fixed', but that failure is reasonable for us to handle after we see it in CI, but that seems worth it given that it lets us get rid of the tombstone concept.

This also helps out the binary caching feature, because we don't have to figure out how to productize tombstones.
2020-07-02 20:20:07 -07:00
ras0219
a218d0e61d
[vcpkg] Remove all uses of Foo::Foo() noexcept = default; to fix #9955 (#12201)
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-01 21:05:18 -07:00
ras0219
5a9d8011f5
[vcpkg] Enable NuGet-based binary caching via mono (#12170)
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-01 12:18:37 -07:00
ras0219
135f91de1e
[vcpkg] Implement --x-write-nuget-packages-config= setting for install and x-set-installed (#12138)
* [vcpkg] Implement --x-write-nuget-packages-config= setting for `install` and `x-set-installed`.

* [vcpkg] Add end-to-end testing suite for install, remove, and binary caching

* [vcpkg] Define `$TestingRoot in end-to-end-tests.ps1

* [vcpkg] Address CR comments

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-01 11:36:09 -07:00
ras0219
3f6342db6d
[vcpkg] Fix vcpkg export --nuget regressions (#12174)
* [vcpkg] Upgrade find_acquire_program(NUGET) to 5.5.1. Add partial SHA to temporary download paths to avoid collisions.

* [vcpkg] Fix regression in `vcpkg export --nuget` due to file locking on .vcpkg-root

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-06-30 14:02:26 -07:00
nicole mazzuca
1d8f0acc9c
[vcpkg manifest] Manifest Implementation (#11757)
==== Changes Related to manifests ====

* Add the `manifests` feature flag
  * This only says whether we look for a `vcpkg.json` in the cwd, not
    whether we support parsing manifests (for ports, for example)
* Changes to the manifests RFC
  * `"authors"` -> `"maintainers"`
  * `--x-classic-mode` -> `-manifests` \in `vcpkg_feature_flags`
  * reserve `"core"` in addition to `"default"`, since that's already
    reserved for features
  * Add a small helper note about what identifiers must look like
  * `<license-string>`: SPDX v3.8 -> v3.9
  * `"feature"."description"` is allowed to be an array of strings as well
  * `"version"` -> `"version-string"` for forward-compat with versions
    RFC
* Add the `--feature-flags` option
* Add the ability to turn off feature flags via passing
  `-<feature-flag>` to `VCPKG_FEATURE_FLAGS` or `--feature-flags`
* Add CMake toolchain support for manifests
  * Requires either:
    * a feature flag of `manifests` in either `Env{VCPKG_FEATURE_FLAGS}`
      or `VCPKG_FEATURE_FLAGS`
    * Passing the `VCPKG_ENABLE_MANIFESTS` option
  * The toolchain will install your packages to
    `${VCPKG_MANIFEST_DIR}/vcpkg_installed`.
* Add MSBuild `vcpkg integrate install` support for manifests
  * Requires `VcpkgEnableManifest` to be true
* `vcpkg create` creates a port that has a `vcpkg.json` instead of a
  `CONTROL`
* argparse, abseil, 3fd, and avisynthplus ports switched to manifest
  from CONTROL
* Add support for `--x-manifest-root`, as well as code for finding it if
  not passed
* Add support for parsing manifests!
* Add a filesystem lock!

==== Important Changes which are somewhat unrelated to manifests ====

* Rename `logicexpression.{h,cpp}` to `platform-expression.{h,cpp}`
* Add `PlatformExpression` type which takes the place of the old logic
  expression
  * Split the parsing of platform expressions from checking whether
    they're true or not
  * Eagerly parse PlatformExpressions as opposed to leaving them as
    strings
* Add checking for feature flag consistency
  * i.e., if `-binarycaching` is passed, you shouldn't be passing
    `--binarysource`
* Add the `Json::Reader` type which, with the help of user-defined
  visitors, converts JSON to your internal type
* VcpkgArgParser: place the switch names into a constant as opposed to
  using magic constants
  * In general update the parsing code so that this ^ works
* Add `Port-Version` fields to CONTROL files
  * This replaces the existing practice of
    `Version: <my-version>-<port-version>`

==== Smaller changes ====
* small drive-by cleanups to some CMake
  * `${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}` ->
    `${CURRENT_INSTALLED_DIR}`
  * Remove `-analyze` when compiling with clang-cl, since that's not a
    supported flag (vcpkg's build system)
  * Add a message about which compiler is detected by vcpkg's build
    system machinery
* Fix `Expected::then`
* Convert `""` to `{}` for `std::string` and `fs::path`, to avoid a
  `strlen` (additionally, `.empty()` instead of `== ""`, and `.clear()`)
* Add `Strings::strto` which converts strings to numeric types
* Support built-in arrays and `StringView` for `Strings::join`
* Add `operator<` and friends to `StringView`
* Add `substr` to `StringView`
* SourceParagraphParser gets some new errors
2020-06-30 10:40:18 -07:00
Billy O'Neal
309f6fc9bc
[vcpkg] Delete unused --purge-tombstones and introduce BufferedPrint class (#12049)
* Introduce buffered_print class to manage buffered write patterns like in the ci command.

* Remove --purge-tombstones option.

* Law of demeter.

* Make buffered_print imobile.

* buffered_print => BufferedPrint

* Fix merge conflict.
2020-06-26 12:16:17 -07:00
ras0219
91e798afd8
[vcpkg] Implementation of --x-binarysource=nuget (and friends) (#12058)
* [vcpkg] Initial implementation of --x-binarysource=nuget

* [vcpkg] Remove double-double quoting of CMake arguments

* [vcpkg] Update nuget.exe to 5.5.1 to support Azure DevOps Artifacts

* [vcpkg] Enable batching of NuGet server calls with prefetch(). Add `interactive` binarysource.

* [vcpkg] Add `nugetconfig` binary source

* [vcpkg] Short circuit querying remote NuGet servers once all refs are found

* [vcpkg] Add experimental help for binary caching

* [vcpkg] Improved NuGet cache package descriptions and version formatting

* [vcpkg] Rename `CmdLineBuilder::build()` to extract()

* [vcpkg-help] Ascii-betize help topics

* [vcpkg] Add tests for cmdlinebuilder. Improve handling of quotes and slashes.

* [vcpkg] Addressing code review comments

* [vcpkg] Add tests for vcpkg::reformat_version()

* [vcpkg] Added test for vcpkg::generate_nuspec()

* [vcpkg] Add tests for vcpkg::XmlSerializer

* [vcpkg] Addressed code review comment

* [vcpkg] Add test for vcpkg::Strings::find_first_of

* [vcpkg] Fix machine-specific paths in test for vcpkg::generate_nuspec()

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-06-26 12:16:02 -07:00
ras0219
4c527e49c4
[vcpkg] Use XDG/LOCALAPPDATA for default binary caching path (#12091)
* [vcpkg] Use XDG Base Directory Specification on non-Windows

* [vcpkg] Move user-wide binary cache on Windows to $LOCALAPPDATA/vcpkg/archives

* [vcpkg] Address code review comments; refactor other uses of LOCALAPPDATA

* [vcpkg] Address code review comments

* [vcpkg] filesystem::path::append() accepts string arguments, not paths.

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-06-26 11:26:38 -07:00
nicole mazzuca
22c8e3a23a
[vcpkg build] fix build command (#12072) 2020-06-25 18:23:30 -07:00
ras0219
aef838536e
[vcpkg] Track compiler information in ABI (#11654)
* [vcpkg] Refactor out abi_tags_from_pre_build_info()

* [vcpkg] Track Windows toolchain file in triplet hash

* [vcpkg] Improve error messages when constructing PreBuildInfo

* [vcpkg] Extract InstallPlanAction::BuildAbiInfo

* [vcpkg] Extract Build::EnvCache and private-impl VcpkgPaths

* [vcpkg] Enable compiler hash detection when binarycaching is enabled

* [vcpkg] Downgrade warning about missing ABI keys

When binarycaching is not enabled, this warning is spurious and provides no user value.

* [vcpkg] Cleanup

* [vcpkg] Refactor compiler tracking into triplet abi computation

Move several static global caches into VcpkgPaths/EnvCache.
Add feature flag 'compilertracking' to enable the new feature.

* [vcpkg] Refactor out PreBuildInfo::using_vcvars()

Move VcpkgTripletVar into build.cpp because it is not used outside that file.

* [vcpkg] Address some code analysis warnings

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-06-22 14:14:36 -07:00
ras0219
4cbbf4ebe6
[vcpkg] Fix issue #9781 by exporting from the installed directory (#11015)
* [vcpkg] Fix issue #9781 by exporting from the installed directory

* [vcpkg] Address code review comments for #11015

* [vcpkg] Remove duplicate triplet in installed path from exports

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-06-17 12:32:05 -07:00
Billy O'Neal
04e214eb0e
[vcpkg] Fix create by extracting common paths settings (#11842)
Resolves #11784
2020-06-16 11:58:11 -07:00
Billy O'Neal
7192d3affa
[vcpkg] Delete g_binary_caching global that should be passed as a parameter. (#11958) 2020-06-15 14:34:26 -07:00
Alexander Neumann
14514508d8
[vcpkg] Add vcpkg item to project settings in Visual Studio (#4361)
Co-authored-by: Curtis J Bezault <curtbezault@gmail.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-06-15 14:24:14 -07:00
Billy O'Neal
4fb2256085
[vcpkg] Allow CI to pass in all relevant directories and remove use of symbolic links (#11483) 2020-06-03 19:31:28 -07:00
nicole mazzuca
09319cd79e
[vcpkg metrics] Allow someone to opt out after build (#11542)
* [vcpkg metrics] start using json library

Additionally, add floats to the JSON library since they're required.

* [vcpkg metrics] allow users to disable metrics after the build

Additionally, as a drive by, fix UUID generation

* fix metrics data

* code review
2020-05-29 14:09:03 -07:00
Billy O'Neal
d9b4acf02d
[vcpkg] Move CI cleaning back out of the 'ci' command into a separate command to restore cross-compilation preinstalls. (#11545) 2020-05-27 18:40:23 -07:00
nicole mazzuca
9d9a50bc98
[vcpkg] fix extern C around ctermid (#11343)
Additionally, move the system_header invocations to their own header file,
<vcpkg/base/system_header.h>
2020-05-20 10:10:26 -07:00
Billy O'Neal
0116b86e38
[vcpkg] Optimize string split slightly. (#11433) 2020-05-19 15:05:53 -07:00
Billy O'Neal
3b0080e3b0
[vcpkg] Harden file removals and clean directory contents in "CI" inside vcpkg itself. (#11432)
`files.h/files.cpp`:
* Add end and else comments to all macros.
* Add "remove_all_inside" function which empties a directory without actually deleting the directory. This is necessary to handle the case where the directory is actually a directory symlink.
* Change remove_all to use std::remove when VCPKG_USE_STD_FILESYSTEM is set; this will engage POSIX delete support available on current Win10.

`commands.ci.cpp`: empty "installed".

`*/initialize_environment.*`: No longer clean the directories outside the tool.

`ci-step.ps1`: Remove unused console output tee-ing.
2020-05-19 13:43:30 -07:00
Billy O'Neal
5504dfa7da
[vcpkg] Turn on tests and PREfast in CI, and fix tests to pass. (#11239)
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
2020-05-14 12:49:31 -07:00
Robert Schumacher
8de3f323dc
[vcpkg] Resolve relative overlay ports to the current working directory. (#11233)
Fixes #10771.
2020-05-08 12:41:44 -07:00
Robert Schumacher
ce8b01a14b
[vcpkg] Add support for VCPKG_BINARY_SOURCES and --x-binarysource=<> (#10476)
* [vcpkg] Add support for VCPKG_BINARY_SOURCES and --binarysource=<>

* [vcpkg] Rename --binarysource to --x-binarysource to denote internal/experimental

* [vcpkg] Address review comments & add tests for BinaryConfigParser

* [vcpkg] Replace do {} while(1); with for(;;)

Avoids conditional expresion is constant warnings

* [vcpkg] Invert if/else

* [vcpkg] Fix warning in export.prefab.cpp

* [vcpkg] Resolve merge regressions
2020-04-29 10:16:40 -07:00
nicole mazzuca
09af1e9b55
[vcpkg] Add initial JSON support (#10521)
* [vcpkg] Add initial JSON support

This adds a JSON parser, as well as the amount of unicode support
required for JSON parsing to work according to the specification. In the
future, I hope to rewrite our existing XML files into JSON.

Additionally, as a drive-by, we've added the following:

* add /wd4800 to pragmas.h -- this is a "performance warning", for when
  you implicitly convert pointers or integers to bool, and shouldn't be
  an issue for us.
* Switched Parse::ParserBase to read unicode (as utf-8), as opposed to
  ASCII
* Building again under VCPKG_DEVELOPMENT_WARNINGS, yay!
2020-04-17 18:16:20 -07:00
nicole mazzuca
556325a1f7
[vcpkg] Add x-set-installed command (#10817)
This command takes a list of ports, and causes the final state of the
installed directory to be as-if one ran the install on an empty
installed directory (removing any unnecessary packages).

This is especially useful with the new `--x-install-root` option, which
allows one to set the `installed` directory for vcpkg to use.

Additionally, as a drive-by, we do some `stdfs` clean-up and add a
`.is_feature()` member function to BinaryParagraph (as opposed to
checking for `.feature().empty()`, which is far less clear to read).

This feature is experimental.
2020-04-17 15:49:59 -07:00
nicole mazzuca
22623e3501
[vcpkg] Clean up CMake build system (#10834)
There are quite a few changes to the CMake build system packaged up into
one set here:
* Added `toolsrc/cmake/utilities.cmake`, which contains the following:
  * `vcpkg_detect_compiler` -- get the name of the C++ compiler, as one
    of {gcc, clang, msvc}
  * `vcpkg_detect_standard_library` -- get the name of the standard
    library we're linking to, as one of {libstdc++, libc++, msvc-stl}
  * `vcpkg_detect_std_filesystem` -- figure out how to link and call
    into C++17's filesystem; whether one needs to link to `stdc++fs` or
    `c++fs`, and whether to use `<filesystem>` or
    `<experimental/filesystem>`.
* Added a `VCPKG_WARNINGS_AS_ERRORS`, split off from
  `VCPKG_DEVELOPMENT_WARNINGS`, which allows one to use the development
  warnings without passing -Werror
* Rename `DEFINE_DISABLE_METRICS` to `VCPKG_DISABLE_METRICS` -- the
  former will now print a deprecation message and set the latter.
* Now, print a deprecation message on `WERROR`; it doesn't do anything
  since the behavior it requested is now the default.
* Pass `-std=c++17` if the compiler allows it, instead of `-std=c++1z`
* Do some code movement
* Pass `USE_STD_FILESYSTEM` if possible, instead of only on minGW
  * Renamed to `VCPKG_USE_STD_FILESYSTEM`

Additionally, we now pass `/W4` in Debug mode on x86 in the Visual
Studio build system; this brings it in line with the CMake build system,
and the x64 Visual Studio build system.

And finally, we make some minor code changes to support compiling in
VCPKG_DEVELOPMENT_WARNINGS mode.
2020-04-14 22:08:50 -07:00
Alexander Neumann
0edffcf125
[vcpkg] New policy: SKIP_ARCHITECTURE_CHECK. (#10398)
* New policy: SKIP_DLL_ARCHITECTURE_CHECK.
The check only works if MS link.exe is used
otherwise the second linker member is missing
(according to the observed errors)

* rename to VCPKG_POLICY_SKIP_ARCHITECTURE_CHECK
since the dll check was ok but the lib check was failing.

* fix indentation

* move the if to a better position.
2020-04-09 17:04:20 -07:00
nicole mazzuca
47a4913834
[vcpkg] Correct UInt128 code 😇 (#10583)
* [vcpkg] Correct UInt128 code 😇

`UInt128::operator<<(x, y)` should clear the bottom 64 bits of `x` if
`y >= 64`; however, we don't do this, and so we duplicate `x`'s bottom
bits into `x.top` instead of moving them. Similarly, we have the
opposite problem for `UInt128::operator>>`. This commit fixes these
latent bugs, which we weren't hitting because the thing we use them for
never actually shifts more than 64 bits.
2020-04-09 14:11:53 -07:00
atkawa7
52b5dfd2ef
Android Support: Exporting to Android Archive (AAR) (#10271)
* added android triplets

* added android support to vcpkg

* added export directories to git ignore

* fix libraries naming

* added vckpg sources to visual studio project files

* rename file location

* issue with std::string fs:path copy initialization

* format path on VStudio

* fix checks format cannot work on fs::path

* support header only libraries

* support using architecture instead of triplets

* added prefab support

* added debug logs and prefab debug flag

* added support for empty packages i.e openssl
2020-04-06 14:36:17 -07:00
Alexander Neumann
aeffa625e5
[vcpkg] Add the possibility to load vcvars.bat with external toolchain. (#10399)
* Add VCPKG_LOAD_VCVARS_ENV
allow loading vcvars.bat if an external toolchain file is used.

* add documentation
2020-03-27 20:19:53 -07:00
Robert Schumacher
42ad12f91d
[vcpkg] Improve common case of ignoring filesystem errors (#10557) 2020-03-25 15:18:10 -07:00
Robert Schumacher
2fc82f6381
[vcpkg-help] Reflow and small cleanups (#10477) 2020-03-25 10:50:27 -07:00
Robert Schumacher
8b201cb43c
Merge pull request #10372 from ras0219-msft/dev/roschuma/jobs
[vcpkg] Introduce Job Objects to improve ctrl-c performance on Windows
2020-03-19 17:15:54 -07:00
Robert Schumacher
2c2e67f35f
Merge pull request #10032 from ras0219-msft/dev/roschuma/compute-all-abis
[vcpkg] Compute ABIs upfront instead of just-in-time
2020-03-19 11:50:15 -07:00
Robert Schumacher
ebf8213945 [vcpkg] Use enum class for RestoreResult 2020-03-18 11:19:25 -07:00
Billy Robert O'Neal III
347d8ef530 Implement a warning when running vcpkg inside a developer command prompt if the set of packages to install all target a different architecture than the prompt. 2020-03-10 23:44:48 -07:00
Robert Schumacher
60cd1202d9 [vcpkg] Introduce Job Objects to improve ctrl-c performance on Windows 2020-03-10 10:27:36 -07:00
Robert Schumacher
6b55c62144 Merge remote-tracking branch 'origin/master' into HEAD 2020-03-09 09:19:05 -07:00
Robert Schumacher
f6d52648ec
[vcpkg] Fix issue #9916; vcpkg upgrade did not load_tag_vars() (#10202) 2020-02-24 14:38:51 -08:00
Victor Romero
ef60f1d9b9
Fix build for VS2015 (#10126)
* Add missing constructors for TexRowCol required by VS2015

* Make TextRowCol() constexpr and noexcept
2020-02-19 16:51:09 -08:00
Robert Schumacher
e21400c7ca [vcpkg] Fix VS2015 compile 2020-02-12 15:50:39 -08:00
Robert Schumacher
3d841c9b6a [vcpkg] Compute all ABIs upfront instead of during build_package() 2020-02-11 23:52:56 -08:00
Robert Schumacher
b9942d0744 [vcpkg] Extract binary caching functionality into separate interface 2020-02-11 18:41:03 -08:00
Robert Schumacher
411b4cc8a0 [vcpkg] Remove superfluous BuildPackageConfig struct (#9997)
BuildPackageConfig held essentially the same information as InstallPlanAction, so deduplicate
2020-02-09 18:55:49 -08:00
Robert Schumacher
a33044c186
[vcpkg] Track parser row/col state in Paragraph (renamed from RawParagraph) (#9987) 2020-02-09 14:50:26 -08:00
Robert Schumacher
f7fb56decd [vcpkg] Use CreateProcess on Windows. Improve EnvVars manipulation and handling. (#9897)
* [vcpkg] Rewrite process spawning on windows to always use CreateProcess, enabling better environment handling

* [vcpkg] Use environment cache to avoid calling vcvars multiple times

* [vcpkg] Increase buffer size while computing hashes

* [vcpkg] Remove unneeded cmd.exe wrapper process on all CreateProcess calls

* [vcpkg] Fix .vcxproj{,.filters}

* [vcpkg] Upgrade Ctrl-C state machine to handle multiple background processes.

* [vcpkg] Fix regression while launching metrics: 'start' can't be passed directly to CreateProcessW

* [vcpkg] Fix typo on non-Windows

* [vcpkg] Fix various uses of >NUL across the code

* [vcpkg] Fix various uses of >NUL across the code
2020-02-08 22:45:36 -08:00
Robert Schumacher
8db6db5dac
[vcpkg] Further parser improvements (#9895)
* [vcpkg] Consolidate several internal parsers together (packagespecs + logicexpression + control) and enhance error messages

* [vcpkg] Migrate Build-Depends parsing to new framework

* [vcpkg] Fix tests. Re-enable underscores in feature names due to libwebp[vwebp_sdl] -- todo: rename this feature and remove underscores.
2020-02-07 11:24:35 -08:00
Alexander Neumann
bd8c07217a [vcpkg] Add skip dumpbin checks
and correct some cmake advices.
Paths should always be handled with "${PATH}" due to possible spaces
2020-02-04 10:58:38 +01:00
Phil Christensen
e62d136128
[vcpkg] Add Supports: field. Use contents of triplets instead of names for dependency resolution. (#8601)
* remove unfinished "supports" tag

* extract "supports" from control files
But do nothing with the value

* Start `Supports` documentation

* Use Supports in a bunch of control files

I only tried matching the already existing logic in the portfile.cmake.

* Cmake var provider (#8)

* Cmake var provider (#9)

* fix windows build (#10)

* Add missing files to build

* Fix test (#11)

* adding hooks for cmake variables in expressions

* Adding hooks for 'supports' in CI test

* Fix test (#12)

* Add overrides to evaluation environment

* use "supported" tag in CI testing

* cleanup comment

* Fix issues with PR

* [var_provider] Get library linkage variables from triplet

* Fix compilation errors in tests

* Add unimplemented functions

* Fix unit tests part 1

* Fix issue when buildtrees dir does not exist

* Change binary output hash

* Fix handling of * feature

* Add core feature when using *

* Do not add Default-Features when installing 'core'

* [vcpkg] WIP. 6 failing tests.

* [vcpkg] WIP. 1 failing tests.

* [vcpkg] WIP. 0 failing tests.

* [vcpkg] Removed 'remove_graph'. 0 failing tests.

* [vcpkg] Removed 'install_graph'. 0 failing tests.

* [vcpkg] Remove AnyAction; replace with ActionPlan

* [vcpkg] Minor cleanup.

* [vcpkg][z3][qt5-connectivity][qt5-purchasing] Improve error messages while parsing. Fix a few trivial port issues.

* [vcpkg] Work around ICE with MSVC v140

* [vcpkg] Add purge on fail to decompress for CI

* [vcpkg] Fix parsing of nested parentheses in qualifiers

* [vcpkg] Fix Linux builds (explicit qualification in declaration)

* [vcpkg] Fix Build-Depends implying default features. Fix qualified dependencies regression.

* [mmx] Add to skip list and full rebuild -- mmx causes problems by installing 'sched.h'

* [libpqxx][mqtt-cpp] Prevent installing include/CMakeLists.txt

* [cppitertools] Fix installed include namespace (should be include/cppitertools)

* [libsoundio] Move headers into soundio/ subdirectory as per original cmake

* [ci.baseline] Temporarily skip charls due to conflict with dcmtk

* [vcpkg] Add restricted include files post build check -- bump global abi version

* [libsoundio] Hotfix stray line in portfile

* [vcpkg] Fix regression: CMake information was not being displayed for build-and-install actions

* [jsonnet] Fix installation of internal headers; use system nlohmann-json

* [grpc][upb] Teach grpc to use packaged upb. Add find_package(upb). Remove inappropriate upb features.

* [zfp] Move problematic 'include/bitstream.h' to 'include/zfp/bitstream.h'

* [x265] Bump control version to trigger rebuild after zfp conflict

* [akali] Disable parallel configure

* [dirent][dlfcn-win32][getopt-win32][pthreads] Grandfather into VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS

* [ci.baseline] Update baseline for improved upb support

* [tgui] Disable parallel configure

* [libiconv] Enable VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS

* [aws-sdk-cpp] Disable parallel configure

* [vcpkg] Implement policy VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS

* [aws-sdk-cpp] Fix amount of escaping semicolons -- Note: I do not know the root cause requiring this change

* [libodb-sqlite] Fix configuring into source directory

* [gettext] Grandfather into VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS

* [libodb] DISABLE_PARALLEL_CONFIGURE

* [vcpkg] Add 'config.h' and 'local.h' to restricted header list

* [mcpp] Remove unused and problematic include 'config.h' from installed files

* [teemo] Move installed headers into subdirectory to prevent conflicts with x265

* [ci.baseline] Update current OSX. Skip libmesh on all platforms due to heavy conflicts.

* [vcpkg] Add 'slice.h' as a restricted header

* [osg] Improve accuracy of dependencies (disable some, add some to Depends)

* [vcpkg] Skip invoking a subprocess for 0 specs in load_tag_vars

* [ci.baseline] Skip mongo-c-driver on osx due to flakiness

* [teemo] Fix incorrect include file read

* [osg] Fix dependency typo: glut -> freeglut

* [vcpkg] Recover some lost performance with the addition of vcpkg_get_tags.

A huge performance cost was loading the triplet files over and over; instead, we splice the sources into a macro and load it once, then just call that macro for each port.
Remove use of hashing because we aren't cross-process-safe anyway (global static will do instead).

* [vcpkg] Change Supports atom 'windows' to include UWP. Improve Supports field documentation.

* [vcpkg] Add docs for VCPKG_ENV_PASSTHROUGH and VCPKG_DEP_INFO_OVERRIDE_VARS

* Fix typo

Co-authored-by: Curtis J Bezault <curtbezault@gmail.com>
Co-authored-by: Victor Romero <romerosanchezv@gmail.com>
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-02-03 14:22:52 -08:00
Cristian Adam
38b959022c Add support for building with MinGW (#9137)
* Add support for building with MinGW

Tested with MSYS2 MinGW 8.3.0, gcc-mcf.lhmouse MinGW 9.2.1,
and StephanTLavavej/mingw-distro!

* Add MinGW toolchain

From your MinGW configured shell you could just use vcpkg to
configure packages.

An x64-mingw triplet would look like:
```
set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE static)
set(VCPKG_ENV_PASSTHROUGH PATH)

set(VCPKG_CMAKE_SYSTEM_NAME MinGW)
```

* Add MinGW community tripplets

x64 tested with https://github.com/StephanTLavavej/mingw-distro
x86, arm64, arm tested with https://github.com/mstorsjo/llvm-mingw
2020-01-17 15:36:08 -08:00
Victor Romero
f56645c3fd
Community Triplets 🤝 (#7976)
* Move untested triplets to Community Triplets

* Document community triplets

* Load community triplets by default and alert when using one

* [triplets] Merge documentation of community triplets

* Bump version
2020-01-17 14:21:19 -08:00
martin-s
299c7c730c Introduce new policy to skip post verification of dll exports (#9642)
* - Introduce new policy to skip post verification of dll exports (see issue #9641).

* - Fixed line endings.
2020-01-13 16:31:58 -08:00
dpoliarush
37265f7a45 [vcpkg] Removed unused template function. (#9287) 2019-12-20 17:26:40 -08:00
Robert Schumacher
45f4b820e5
Merge branch 'master' into vcpkg_nuget 2019-11-22 09:47:40 -08:00
Victor Romero
84ba23ad33
[x-history] Prints CONTROL version history of a port 👻 (#7377)
* [port-history] Print port CONTROL version history

* Add commands.porthistory.cpp to VS project

* Get most recent commit for each version

* Apply clang-format

* Fix output format

* Rename command to x-history
2019-09-06 11:35:56 -07:00
Victor Romero
f5c732b40d
Download Mode (#7950)
* [portfile functions] Override execute_process() to accept ALLOW_IN_DOWNLOAD_MODE option

* [vcpkg install] Set VCPKG_DOWNLOAD_MODE when using --only-downloads option

* [vcpkg_find_acquire_program] Allow in Download Mode

* Don't stop when build fails for a package

* Download sources for all packages in dependency graph

* Improve output messages

* Enable acquiring MSYS packages in download mode

* Documentation

* Update documentation

* execute_process() always fails on Download Mode

* Regenerate docs and fix formatting

* Run clang-format

* Use _execute_process on vcpkg_from_<source> helpers

* Fix calls to _execute_process() when not in Download Mode
2019-08-28 13:49:29 -07:00
Victor Romero
4b404e8cfb
Revert "[vcpkg install] Enable Download Mode (#7797)" (#7949)
This reverts commit 65d4bc146b.
2019-08-28 11:59:30 -07:00
Victor Romero
65d4bc146b
[vcpkg install] Enable Download Mode (#7797)
* [portfile functions] Override execute_process() to accept ALLOW_IN_DOWNLOAD_MODE option

* [vcpkg install] Set VCPKG_DOWNLOAD_MODE when using --only-downloads option

* [vcpkg_find_acquire_program] Allow in Download Mode

* Don't stop when build fails for a package

* Download sources for all packages in dependency graph

* Improve output messages

* Enable acquiring MSYS packages in download mode

* Documentation

* Update documentation

* execute_process() always fails on Download Mode

* Regenerate docs and fix formatting

* Run clang-format

* Use _execute_process on vcpkg_from_<source> helpers
2019-08-28 11:47:17 -07:00
Nicole Mazzuca
7827239593 (#7757) [vcpkg] Switch to internal hash algorithms 📜
On non-Windows platforms, there is no standard way to get the hash of an
item -- before this PR, what we did was check for the existence of a few
common utility names (shasum, sha1, sha256, sha512), and then call that
utility on a file we created containing the contents we wish to hash.
This PR adds internal hashers for sha1, sha256, and sha512, and
standardizes the interface to allow anyone to implement hashers in the
future.

These hashers are not extremely optimized, so it's likely that in the
future we could get more optimized, but for now we just call out to
BCryptHasher on Windows, since it's standard and easy to use (and about
2x faster for sha1 and sha256, and 1.5x faster for sha512). However,
they are reasonably fast for being unoptimized. I attempted a few minor
optimizations, which actually made the code slower! So as of right now,
it's implemented as just a basic conversion of the code on Wikipedia to
C++. I have tested these on the standard NIST test vectors (and those
test vectors are located in vcpkg-test/hash.cpp).
2019-08-26 12:35:22 -07:00
Curtis J Bezault
4da95d667c
[vcpkg]Port toolchains (#7687)
* checkpoint commit

* Only set VCPKG_ENV_OVERRIDES_FILE if it exists

* First pass at working port-toolchain

* Update VERSION.txt

* Return rvalue

* Fix compilation error

* Some fixes are requested by @ubsan

* Fix another compilation error
2019-08-20 08:47:26 -07:00
Nicole Mazzuca
300e21d59e [vcpkg] Major tool CMakeLists.txt updates
- Add the "VCPKG_DEVELOPMENT_WARNINGS" flag
	- setting "WERROR" will also set this flag
	- This flag is set by default
	- on GCC/clang, this will pass '-Wall -Wextra -Wpedantic -Werror'
	- on GCC, this will additionally pass '-Wmissing-declarations'
	- on clang, this will additionally pass '-Wmissing-prototypes'
	- on MSVC, this will pass '-W4 -WX'
- On Visual Studio 2017 and later, pass '-permissive-'
- Change the source for fallout of these changes
- add `format` subcommand
	- formats all C++ source and header files using clang-format
- move `include/vcpkg-test/catch.h` to `include/catch2/catch.hpp`
- pass CONFIGURE_DEPENDS to file(GLOB)
2019-08-16 19:40:53 -07:00
Curtis J Bezault
b47ca1b012
Merge pull request #7305 from cbezault/external_file_abi
[vcpkg] Public ABI override option
2019-08-16 13:51:22 -07:00
Victor Romero
edaf3bf91e
[depend-info] Fix bugs, add --sort, --show-depth and --max-recurse options (#7643)
* [depend-info] Follow same rules as vcpkg install

* [depend-info] Add --max-depth and --sort options

* [depend-info] Improve output readability (a tiny bit)

* [depend-info] Add --show-depth option

* [depend-info] Fix build on VS 2015

* [depend-info] Fix output of --dot and --dgml
2019-08-14 15:38:07 -07:00
nicole mazzuca
875648e10b [vcpkg] Fix the build on VS2015 debug (#7637)
The VS2015 standard library requires, in debug mode, a comparison
operator on `T × U` and `U × T` to also be a comparison operator on
`T × T` and on `U × U`, and so in
vcpkg::Install::install_package::intersection_compare, I've added two
new `operator()` overloads which take those respectively, on VS2015.

Also, `[nodiscard]` was added to somewhere in `vcpkg/base/strings.h`,
which gives a warning in VS2015 -- thus, I added the `vcpkg/pragmas.h`
include, since that fixes the warning.
2019-08-12 14:02:59 -07:00
Nicole Mazzuca
420dbffaa0 clang-format, and fix a leftover 2019-08-10 13:13:21 -07:00
Nicole Mazzuca
52b2e740de [vcpkg] Fix build under /W4
I was building under /W3, because CMake hadn't been set up to build
under /W4 -- therefore, I didn't see some warnings.

We also decided to remove the niebloids and instead break ADL by using
`= delete`, since otherwise we get warnings when we define a local
variable with the same name as a niebloid. I also removed `status` and
`symlink_status` from the `files` header, since it's unnecessary now,
and they're just implementation details of `RealFilesystem`.

I also removed some existing uses of unqualified `status(path)`, since
that no longer compiles. I also added `Filesystem::canonical`, to remove
another use of `fs::stdfs` in a function I was already working in.
2019-08-10 13:13:21 -07:00
Curtis.Bezault
f9561a08db remove unnecessary include, fix variable extraction 2019-08-09 14:47:58 -07:00
Curtis.Bezault
0c7d8f4146 Change purpose of this PR to just overriding the abi 2019-08-09 14:21:58 -07:00
Curtis J Bezault
c4f1a91ef2
Merge branch 'master' into external_file_abi 2019-08-09 11:59:32 -04:00
Nicole Mazzuca
e79f0dc532 [vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).

In addition, I did the following three orthogonal changes:
  - simplified the work queue, basing it on Billy O'Neal's idea
  - Fix warnings on older versions of compilers in tests, by splitting
    the pragmas out of pch.h.
  - Ran clang-format on some files

In fixing up remove_all, the following changes were made:
  - On Windows, regular symlinks and directory symlinks are distinct;
    as an example, to remove directory symlinks (and junctions, for that
    matter), one must use RemoveDirectory. Only on Windows, I added new
    `file_type` and `file_status` types, with `file_type` including a new
    `directory_symlink` enumerator, and `file_status` being exactly the
    same as the old one except using the new `file_type`. On Unix, I
    didn't make that change since they don't make a distinction.
  - I added new `symlink_status` and `status` functions which use the
    new `file_status` on Windows.
  - I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
    to the old version which called `stdfs::exists` directly.
  - Added benchmarks to `vcpkg-test/files.cpp`. They test the
    performance of `remove_all` on small directories (~20 files), with
    symlinks and without, and on large directories (~2000 files), with
    symlinks and without.
2019-08-07 16:51:12 -07:00
Phil Christensen
22e0b9f376
improve logic expression evaluation (#7508)
* better logic expression evaluation

Improve the logic expression evaluation currently used when filtering
dependencies.

Biggest improvements:
+  Allow '|' operator
+  Support nested '()'
+  Allow whitespace
+  Useful error message for malformed expressions

Also changed names of types to RawParagraph when that is what the original author was using.
2019-08-02 21:37:49 -07:00
nicole mazzuca
f990dfaa5b [vcpkg] Fix RealFilesystem::remove_all (#7430)
* fix remove_all

we were attempting to remove READONLY files before this, and so set them to non-READONLY

* fix linux/macos support

* whee fix vs2015
2019-07-26 16:32:33 -07:00
Curtis J Bezault
d60047280d
Merge branch 'master' into external_file_abi 2019-07-24 14:26:34 -07:00
Curtis.Bezault
0c7669d009 store fs::path instead of std::string 2019-07-24 14:24:49 -07:00
Curtis.Bezault
d68b9a08b1 only use filename 2019-07-23 16:28:00 -07:00
Curtis.Bezault
81909e47d1 Remove types from this PR 2019-07-23 15:38:09 -07:00
Curtis.Bezault
829f99b506 remove needs_rebuild from statusparagraphs 2019-07-23 15:36:13 -07:00