Commit Graph

657 Commits

Author SHA1 Message Date
Nicole Mazzuca
e79f0dc532 [vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).

In addition, I did the following three orthogonal changes:
  - simplified the work queue, basing it on Billy O'Neal's idea
  - Fix warnings on older versions of compilers in tests, by splitting
    the pragmas out of pch.h.
  - Ran clang-format on some files

In fixing up remove_all, the following changes were made:
  - On Windows, regular symlinks and directory symlinks are distinct;
    as an example, to remove directory symlinks (and junctions, for that
    matter), one must use RemoveDirectory. Only on Windows, I added new
    `file_type` and `file_status` types, with `file_type` including a new
    `directory_symlink` enumerator, and `file_status` being exactly the
    same as the old one except using the new `file_type`. On Unix, I
    didn't make that change since they don't make a distinction.
  - I added new `symlink_status` and `status` functions which use the
    new `file_status` on Windows.
  - I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
    to the old version which called `stdfs::exists` directly.
  - Added benchmarks to `vcpkg-test/files.cpp`. They test the
    performance of `remove_all` on small directories (~20 files), with
    symlinks and without, and on large directories (~2000 files), with
    symlinks and without.
2019-08-07 16:51:12 -07:00
Phil Christensen
22e0b9f376
improve logic expression evaluation (#7508)
* better logic expression evaluation

Improve the logic expression evaluation currently used when filtering
dependencies.

Biggest improvements:
+  Allow '|' operator
+  Support nested '()'
+  Allow whitespace
+  Useful error message for malformed expressions

Also changed names of types to RawParagraph when that is what the original author was using.
2019-08-02 21:37:49 -07:00
nicole mazzuca
f990dfaa5b [vcpkg] Fix RealFilesystem::remove_all (#7430)
* fix remove_all

we were attempting to remove READONLY files before this, and so set them to non-READONLY

* fix linux/macos support

* whee fix vs2015
2019-07-26 16:32:33 -07:00
Nicole Mazzuca
0d8bba52e4 allow tests to run on older standard libraries 2019-07-19 23:20:28 -07:00
Nicole Mazzuca
c55ea0a0d5 switch to new test framework 2019-07-19 12:56:24 -07:00
Nicole Mazzuca
b3caf67749 Merge branch 'trunk' into parallel-file-ops 2019-07-19 12:17:22 -07:00
Curtis J Bezault
618fa203c1
[vcpkg] Portfile Settings (#7292) 2019-07-19 08:01:38 -07:00
nicole mazzuca
8250553789 Rewrite the tests! now they're cross-platform! (#7315)
* begin exploratory rewriting of tests

* continue working on tests

* more test work! holy butts vcpkg-tests/plan.cpp was a bunch of work

* finish writing new tests

  - [x] write catch2 tests
  - [ ] rewrite/at least delete the VS project files
  - [ ] document running tests

* Fix tests to work on WSL, rewrite test vcxproj

still need to test on macOS
also, delete tests.pch.h

* Condense add_test calls
2019-07-18 19:07:00 -07:00
Nicole Mazzuca
fddebb75da clang-format all the things 2019-07-18 19:03:46 -07:00
Curtis J Bezault
bb3a9ddb6e
[vcpkg] Environment Variable Passthrough (#7290)
* use additional env param

* remove partials

* remove change to linux triplet

* Fix some issues that vicroms pointed out

* whitespace change
2019-07-18 09:02:21 -07:00
Nicole Mazzuca
65d34c5e55 wheeeee more fixes 2019-07-15 18:51:03 -07:00
Nicole Mazzuca
a0fe40ea58 add tests!
Also, fix all the bugs I found when I wrote the tests!
2019-07-11 18:21:25 -07:00
Nicole Mazzuca
510b0c5cc0 fix more comments 2019-07-11 18:21:09 -07:00
Nicole Mazzuca
3190235875 fix some comments from code reviewers 2019-07-11 18:20:36 -07:00
Nicole Mazzuca
bb57907207 make it compile on macos under g++6 2019-07-11 18:20:36 -07:00
Nicole Mazzuca
5b76f24f35 make this compile on macos 2019-07-11 18:20:36 -07:00
Nicole Mazzuca
3b6d6b3465 actually get the code compiling 2019-07-11 18:20:35 -07:00
Nicole Mazzuca
43493b56df delete the random number generator 2019-07-11 18:20:35 -07:00
Nicole Mazzuca
2d6df16849 remove_all parallelized, and fix the issues with symlink 2019-07-11 18:20:35 -07:00
Nicole Mazzuca
5857e2c680 initial remove-in-parallel
doesn't actually do parallel remove yet
2019-07-11 18:20:35 -07:00
gnaggnoyil
7f80c0e2d3 Make handle features (#6797) 2019-07-09 16:02:48 -04:00
Robert Schumacher
2b8e225b2e
[vcpkg] Fix powershell font corruption bug (#7094)
* [vcpkg] Fix font corruption bug on Windows by downloading Powershell Core

* [vcpkg] Rename subtool to powershell-core

* [vcpkg] Add missing includes to project files
2019-07-06 13:29:46 -07:00
martin-s
91da4aab4c Allow redirection of the scripts folder. (#6552)
* Allow redirection of the scripts folder with an environment variable.

* - Updated feature from environment variable to argument.

* Fix crash when no scripts override is given and use --scripts-root=<PATH> format

* Update help messages to use --scripts-root=<PATH> format
2019-07-01 22:51:07 -07:00
Victor Romero
35e985d3cc
Triplets Overlay Implementation (#7053)
* Triplets Overlay Implementation

* Use cache for get_triplet_file_path()

* Code cleanup
2019-06-27 12:20:12 -07:00
Victor Romero
9e565e9867
[--overlay-ports] Show location of overriden ports during install plan (#7002)
* [--overlay-ports] Show source location of overlayed ports during install plan

* Code cleanup

* Code cleanup
2019-06-24 12:09:48 -07:00
Victor Romero
f3db66b403
Ports Overlay partial implementation (#6981)
* Ports Overlay feature spec

* Ports Overlay implementation

* [--overlay-ports] Refactor handling of additional paths

* Code cleanup

* [--overlay-ports] Add help

* [depend-info] Support --overlay-ports

* Add method to load all ports using PathsPortFileProvider

* Make PortFileProvider::load_all_control_files() const

* Remove unused code

* [vcpkg] Avoid double-load of source control file between Build::perform_and_exit and Build::perform_and_exit_ex

* [vcpkg] Clang format

* [vcpkg] Fixup build failure introduced in b069ceb2f2

* Report errors from Paragraphs::try_load_port()
2019-06-21 23:50:05 -07:00
Stephan T. Lavavej
f0902b3537 VS 2019 16.3 deprecates <experimental/filesystem>. (#6968)
VS 2019 16.3 will contain a couple of source-breaking changes:

* <experimental/filesystem> will be deprecated via an
impossible-to-miss preprocessor "#error The <experimental/filesystem>
header providing std::experimental::filesystem is deprecated by
Microsoft and will be REMOVED. It is superseded by the C++17
<filesystem> header providing std::filesystem. You can define
_SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING to acknowledge
that you have received this warning."

* <filesystem> will no longer include <experimental/filesystem>.

In the long term, I believe that vcpkg should detect when it's being
built with VS 2017 15.7 or newer, compile in C++17 mode, include
<filesystem>, and use std::filesystem. (Activating this for VS 2019 16.0
or newer would also be reasonable.) Similarly for other toolsets
supporting std::filesystem.

In the short term, this commit makes vcpkg compatible with the upcoming
deprecation. First, we need to define the silencing macro before
including the appropriate header. I've chosen to define it
unconditionally (without checking for platform or version), since it
has no effect for other platforms or versions. Second, we need to deal
with <filesystem> no longer including <experimental/filesystem>.
I verified that VS 2015 Update 3 contained <experimental/filesystem>
(back then, it simply included the <filesystem> header, where the
experimental implementation was defined; this was later reorganized).
Therefore, all of vcpkg's supported MSVC toolsets have
<experimental/filesystem>, so we can simply always include it.

I've verified that this builds with both VS 2015 Update 3 and
VS 2019 16.1.3 (the current production version).
2019-06-20 11:46:55 -07:00
Robert Schumacher
e5b92a3911
[vcpkg] Improve vcpkg::Files::Filesystem error handling (#6919)
* [vcpkg] Modify Filesystem::remove and Filesystem::rename to not throw.

* [.gitignore] Ignore new VS2019 CMake integration default location

* [.gitignore] Ignore CMakeSettings.json in toolsrc

* [vcpkg] Time external processes called with System::cmd_execute

* [vcpkg] Work around VS2019 CMake bug

* [vcpkg] Fix several unused variable warnings.

* [vcpkg] Improve error handling in vcpkg::Files::Filesystem

Always require either std::error_code or LineInfo to print better errors.

* [vcpkg] Fixup missing return value.

Drive by fix: silence warnings in tests.

* [vcpkg] Fix exiting in error_code overload

Drive by fixes for /analyze with VS2019
2019-06-19 11:49:57 -07:00
Curtis J Bezault
b7d6160b80
[icu] Enable parallel builds (#6695)
* Add VCPKG_NUM_LOGICAL_CORES

* break out logic that retries running a command several times into its own function

* Parallelize icu
2019-06-12 14:18:43 -07:00
Phil Christensen
d962dc7d65 [vcpkg] Add optional 'Homepage' field to CONTROL (#6844)
* [control file] Add optional 'Homepage' tag

This allows a 'Homepage' tag to be added to a port in order to support
changes such as PR #2933.  It currently does not do anything with it.

* [docs]

Add Homepage to the control file documentation

* move urls from descriptions to homepage field.
2019-06-10 16:35:22 -07:00
Robert Schumacher
8045248372
[vcpkg] Apply clang format (#6826) 2019-06-08 18:31:58 -07:00
Sean Warren
17623739df [toolsrc] Optionally allow vcpkg to clean after each build (#6522)
* [toolsrc] Optionally allow vcpkg to clean packages, buildtrees and downloads after each build
Adds switch --clean-after-build

* [toolsrc] Clarify that --clean-after-build deletes downloads

* [toolsrc] Revert changes to ci download caching behaviour
2019-05-23 11:24:02 -07:00
Robert Schumacher
77af264118
[vcpkg] Fix regression in --debug. Remove old features featureflag. (#6507) 2019-05-18 18:15:31 -07:00
grdowns
eeac6187c9 Merge branch 'depend_info_no_recurse' of https://github.com/jediry/vcpkg into dev/grdowns/6055 2019-04-11 19:10:08 -07:00
Curtis J Bezault
8fe6f4bdd8 Print what port installed confliciting files (#6037) 2019-04-11 08:27:44 -07:00
Ryan Saunders
a492caf9d5 Merge branch 'master' into query_deps 2019-04-10 23:07:17 -07:00
Ryan Saunders
80965287d1 Add --no-recurse switch to depend-info command. 2019-04-10 23:01:37 -07:00
Victor Romero
1db72cd0ea Revert "Print what port installed confliciting files"
This reverts commit 79682317c9.
2019-04-09 15:55:20 -07:00
Curtis J Bezault
b1b7ec5c0b
Merge branch 'master' into master 2019-04-09 15:22:49 -07:00
Curtis.Bezault
79682317c9 Print what port installed confliciting files 2019-04-09 15:04:44 -07:00
Robert Schumacher
b39b0899cb [vcpkg] Synchronize vcpkg-base with external repo (#5934) 2019-04-08 23:26:18 -07:00
Griffin Downs
c63e466dd6
Revert "Use feature macro to check for support of filesystem" 2019-03-21 14:10:54 -07:00
Griffin Downs
61922f919f Update check for VS2015 2019-03-12 16:46:40 -07:00
Griffin Downs
1c879a4b92 Use identical check when creating namespace alias 2019-03-12 15:05:49 -07:00
Don
795eb45248 Use feature macro to check for support of filesystem
Rather than defaulting to std::experimental::filesystem in files.h a feature
macro can be used to determine what stdfs uses.
2019-03-12 10:12:18 -07:00
Phil Christensen
8fd34506c3 [vcpkg] improve xunit xml output used in CI tests 2019-02-21 22:24:20 -08:00
Robert Schumacher
39b7876db4 [vcpkg] Randomize topological sort in CI plans to allow concurrent builds to more efficiently interact 2019-01-22 17:11:36 -08:00
Billy O'Neal (VC LIBS)
02839ae4a8 vcpkg was going out of its way to use partition before calling the container erase function which is slower than just using remove_if.
remove_if is already stable, so separate stable and unstable versions are unnecessary.

https://iterator.wordpress.com/2016/01/31/algorithms_0/

Unstable remove_if algorithms are possible that might win, as indicated in that article; but plain remove_if provides the most consistent behavior.
2018-11-15 11:24:24 -08:00
Alexander Karatarakis
f19df646a0 Survey times. Refactor Chrono stuff. All times UTC, unless explicitly mentioned
Survey is set to be every 6 months, but you also get one in the first 10 days.
2018-10-17 19:43:15 -07:00
LRFLEW
3d12e5ca72 Handle symlink when installing or removing a library (#4479) 2018-10-17 11:46:27 -07:00