2017-01-28 04:49:09 +08:00
|
|
|
#pragma once
|
|
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
#include <vcpkg/pragmas.h>
|
2019-04-09 14:26:18 +08:00
|
|
|
|
2017-10-17 02:44:04 +08:00
|
|
|
#if defined(_WIN32)
|
2017-01-28 04:49:09 +08:00
|
|
|
#define NOMINMAX
|
2017-03-28 15:02:33 +08:00
|
|
|
#define WIN32_LEAN_AND_MEAN
|
2017-10-14 09:37:41 +08:00
|
|
|
|
2017-10-03 07:55:06 +08:00
|
|
|
#pragma warning(suppress : 4768)
|
2017-03-30 04:16:23 +08:00
|
|
|
#include <windows.h>
|
|
|
|
|
|
2017-10-14 09:37:41 +08:00
|
|
|
#pragma warning(suppress : 4768)
|
|
|
|
|
#include <Shlobj.h>
|
|
|
|
|
|
2017-10-17 02:44:04 +08:00
|
|
|
#include <process.h>
|
|
|
|
|
#include <shellapi.h>
|
|
|
|
|
#include <winhttp.h>
|
|
|
|
|
#else
|
|
|
|
|
#include <unistd.h>
|
|
|
|
|
#endif
|
|
|
|
|
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <algorithm>
|
2017-01-28 06:33:54 +08:00
|
|
|
#include <array>
|
2017-08-26 07:03:57 +08:00
|
|
|
#include <atomic>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <cassert>
|
|
|
|
|
#include <cctype>
|
|
|
|
|
#include <chrono>
|
|
|
|
|
#include <codecvt>
|
|
|
|
|
#include <cstdarg>
|
2017-06-08 15:36:17 +08:00
|
|
|
#include <cstddef>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <cstdint>
|
VS 2019 16.3 deprecates <experimental/filesystem>. (#6968)
VS 2019 16.3 will contain a couple of source-breaking changes:
* <experimental/filesystem> will be deprecated via an
impossible-to-miss preprocessor "#error The <experimental/filesystem>
header providing std::experimental::filesystem is deprecated by
Microsoft and will be REMOVED. It is superseded by the C++17
<filesystem> header providing std::filesystem. You can define
_SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING to acknowledge
that you have received this warning."
* <filesystem> will no longer include <experimental/filesystem>.
In the long term, I believe that vcpkg should detect when it's being
built with VS 2017 15.7 or newer, compile in C++17 mode, include
<filesystem>, and use std::filesystem. (Activating this for VS 2019 16.0
or newer would also be reasonable.) Similarly for other toolsets
supporting std::filesystem.
In the short term, this commit makes vcpkg compatible with the upcoming
deprecation. First, we need to define the silencing macro before
including the appropriate header. I've chosen to define it
unconditionally (without checking for platform or version), since it
has no effect for other platforms or versions. Second, we need to deal
with <filesystem> no longer including <experimental/filesystem>.
I verified that VS 2015 Update 3 contained <experimental/filesystem>
(back then, it simply included the <filesystem> header, where the
experimental implementation was defined; this was later reorganized).
Therefore, all of vcpkg's supported MSVC toolsets have
<experimental/filesystem>, so we can simply always include it.
I've verified that this builds with both VS 2015 Update 3 and
VS 2019 16.1.3 (the current production version).
2019-06-21 02:46:55 +08:00
|
|
|
#define _SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING
|
2017-10-17 02:44:04 +08:00
|
|
|
#include <cstring>
|
[vcpkg] Clean up CMake build system (#10834)
There are quite a few changes to the CMake build system packaged up into
one set here:
* Added `toolsrc/cmake/utilities.cmake`, which contains the following:
* `vcpkg_detect_compiler` -- get the name of the C++ compiler, as one
of {gcc, clang, msvc}
* `vcpkg_detect_standard_library` -- get the name of the standard
library we're linking to, as one of {libstdc++, libc++, msvc-stl}
* `vcpkg_detect_std_filesystem` -- figure out how to link and call
into C++17's filesystem; whether one needs to link to `stdc++fs` or
`c++fs`, and whether to use `<filesystem>` or
`<experimental/filesystem>`.
* Added a `VCPKG_WARNINGS_AS_ERRORS`, split off from
`VCPKG_DEVELOPMENT_WARNINGS`, which allows one to use the development
warnings without passing -Werror
* Rename `DEFINE_DISABLE_METRICS` to `VCPKG_DISABLE_METRICS` -- the
former will now print a deprecation message and set the latter.
* Now, print a deprecation message on `WERROR`; it doesn't do anything
since the behavior it requested is now the default.
* Pass `-std=c++17` if the compiler allows it, instead of `-std=c++1z`
* Do some code movement
* Pass `USE_STD_FILESYSTEM` if possible, instead of only on minGW
* Renamed to `VCPKG_USE_STD_FILESYSTEM`
Additionally, we now pass `/W4` in Debug mode on x86 in the Visual
Studio build system; this brings it in line with the CMake build system,
and the x64 Visual Studio build system.
And finally, we make some minor code changes to support compiling in
VCPKG_DEVELOPMENT_WARNINGS mode.
2020-04-15 13:08:50 +08:00
|
|
|
#if VCPKG_USE_STD_FILESYSTEM
|
2020-01-18 07:36:08 +08:00
|
|
|
#include <filesystem>
|
|
|
|
|
#else
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
#include <experimental/filesystem>
|
2020-01-18 07:36:08 +08:00
|
|
|
#endif
|
2017-01-28 06:33:54 +08:00
|
|
|
#include <fstream>
|
|
|
|
|
#include <functional>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <iomanip>
|
|
|
|
|
#include <iostream>
|
2017-01-28 04:49:09 +08:00
|
|
|
#include <iterator>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <map>
|
|
|
|
|
#include <memory>
|
2017-08-26 07:03:57 +08:00
|
|
|
#include <mutex>
|
2017-12-02 08:08:09 +08:00
|
|
|
#include <random>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <regex>
|
|
|
|
|
#include <set>
|
|
|
|
|
#include <stdexcept>
|
|
|
|
|
#include <string>
|
2018-03-13 21:25:59 +08:00
|
|
|
#if defined(_WIN32)
|
2017-01-28 06:33:54 +08:00
|
|
|
#include <sys/timeb.h>
|
2018-03-13 21:25:59 +08:00
|
|
|
#else
|
|
|
|
|
#include <sys/time.h>
|
|
|
|
|
#endif
|
[vcpkg] Clean up CMake build system (#10834)
There are quite a few changes to the CMake build system packaged up into
one set here:
* Added `toolsrc/cmake/utilities.cmake`, which contains the following:
* `vcpkg_detect_compiler` -- get the name of the C++ compiler, as one
of {gcc, clang, msvc}
* `vcpkg_detect_standard_library` -- get the name of the standard
library we're linking to, as one of {libstdc++, libc++, msvc-stl}
* `vcpkg_detect_std_filesystem` -- figure out how to link and call
into C++17's filesystem; whether one needs to link to `stdc++fs` or
`c++fs`, and whether to use `<filesystem>` or
`<experimental/filesystem>`.
* Added a `VCPKG_WARNINGS_AS_ERRORS`, split off from
`VCPKG_DEVELOPMENT_WARNINGS`, which allows one to use the development
warnings without passing -Werror
* Rename `DEFINE_DISABLE_METRICS` to `VCPKG_DISABLE_METRICS` -- the
former will now print a deprecation message and set the latter.
* Now, print a deprecation message on `WERROR`; it doesn't do anything
since the behavior it requested is now the default.
* Pass `-std=c++17` if the compiler allows it, instead of `-std=c++1z`
* Do some code movement
* Pass `USE_STD_FILESYSTEM` if possible, instead of only on minGW
* Renamed to `VCPKG_USE_STD_FILESYSTEM`
Additionally, we now pass `/W4` in Debug mode on x86 in the Visual
Studio build system; this brings it in line with the CMake build system,
and the x64 Visual Studio build system.
And finally, we make some minor code changes to support compiling in
VCPKG_DEVELOPMENT_WARNINGS mode.
2020-04-15 13:08:50 +08:00
|
|
|
|
2017-10-17 02:44:04 +08:00
|
|
|
#include <sys/types.h>
|
[vcpkg] Clean up CMake build system (#10834)
There are quite a few changes to the CMake build system packaged up into
one set here:
* Added `toolsrc/cmake/utilities.cmake`, which contains the following:
* `vcpkg_detect_compiler` -- get the name of the C++ compiler, as one
of {gcc, clang, msvc}
* `vcpkg_detect_standard_library` -- get the name of the standard
library we're linking to, as one of {libstdc++, libc++, msvc-stl}
* `vcpkg_detect_std_filesystem` -- figure out how to link and call
into C++17's filesystem; whether one needs to link to `stdc++fs` or
`c++fs`, and whether to use `<filesystem>` or
`<experimental/filesystem>`.
* Added a `VCPKG_WARNINGS_AS_ERRORS`, split off from
`VCPKG_DEVELOPMENT_WARNINGS`, which allows one to use the development
warnings without passing -Werror
* Rename `DEFINE_DISABLE_METRICS` to `VCPKG_DISABLE_METRICS` -- the
former will now print a deprecation message and set the latter.
* Now, print a deprecation message on `WERROR`; it doesn't do anything
since the behavior it requested is now the default.
* Pass `-std=c++17` if the compiler allows it, instead of `-std=c++1z`
* Do some code movement
* Pass `USE_STD_FILESYSTEM` if possible, instead of only on minGW
* Renamed to `VCPKG_USE_STD_FILESYSTEM`
Additionally, we now pass `/W4` in Debug mode on x86 in the Visual
Studio build system; this brings it in line with the CMake build system,
and the x64 Visual Studio build system.
And finally, we make some minor code changes to support compiling in
VCPKG_DEVELOPMENT_WARNINGS mode.
2020-04-15 13:08:50 +08:00
|
|
|
// glibc defines major and minor in sys/types.h, and should not
|
|
|
|
|
#undef major
|
|
|
|
|
#undef minor
|
|
|
|
|
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <system_error>
|
2017-10-17 02:44:04 +08:00
|
|
|
#include <thread>
|
2017-01-28 06:33:54 +08:00
|
|
|
#include <time.h>
|
2017-05-24 15:44:00 +08:00
|
|
|
#include <type_traits>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <unordered_map>
|
|
|
|
|
#include <unordered_set>
|
|
|
|
|
#include <utility>
|
|
|
|
|
#include <vector>
|