2017-01-28 04:49:09 +08:00
|
|
|
#pragma once
|
|
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
#include <vcpkg/pragmas.h>
|
2019-04-09 14:26:18 +08:00
|
|
|
|
2017-10-17 02:44:04 +08:00
|
|
|
#if defined(_WIN32)
|
2017-01-28 04:49:09 +08:00
|
|
|
#define NOMINMAX
|
2017-03-28 15:02:33 +08:00
|
|
|
#define WIN32_LEAN_AND_MEAN
|
2017-10-14 09:37:41 +08:00
|
|
|
|
2017-10-03 07:55:06 +08:00
|
|
|
#pragma warning(suppress : 4768)
|
2017-03-30 04:16:23 +08:00
|
|
|
#include <windows.h>
|
|
|
|
|
|
2017-10-14 09:37:41 +08:00
|
|
|
#pragma warning(suppress : 4768)
|
|
|
|
|
#include <Shlobj.h>
|
|
|
|
|
|
2017-10-17 02:44:04 +08:00
|
|
|
#include <process.h>
|
|
|
|
|
#include <shellapi.h>
|
|
|
|
|
#include <winhttp.h>
|
|
|
|
|
#else
|
|
|
|
|
#include <unistd.h>
|
|
|
|
|
#endif
|
|
|
|
|
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <algorithm>
|
2017-01-28 06:33:54 +08:00
|
|
|
#include <array>
|
2017-08-26 07:03:57 +08:00
|
|
|
#include <atomic>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <cassert>
|
|
|
|
|
#include <cctype>
|
|
|
|
|
#include <chrono>
|
|
|
|
|
#include <codecvt>
|
|
|
|
|
#include <cstdarg>
|
2017-06-08 15:36:17 +08:00
|
|
|
#include <cstddef>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <cstdint>
|
VS 2019 16.3 deprecates <experimental/filesystem>. (#6968)
VS 2019 16.3 will contain a couple of source-breaking changes:
* <experimental/filesystem> will be deprecated via an
impossible-to-miss preprocessor "#error The <experimental/filesystem>
header providing std::experimental::filesystem is deprecated by
Microsoft and will be REMOVED. It is superseded by the C++17
<filesystem> header providing std::filesystem. You can define
_SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING to acknowledge
that you have received this warning."
* <filesystem> will no longer include <experimental/filesystem>.
In the long term, I believe that vcpkg should detect when it's being
built with VS 2017 15.7 or newer, compile in C++17 mode, include
<filesystem>, and use std::filesystem. (Activating this for VS 2019 16.0
or newer would also be reasonable.) Similarly for other toolsets
supporting std::filesystem.
In the short term, this commit makes vcpkg compatible with the upcoming
deprecation. First, we need to define the silencing macro before
including the appropriate header. I've chosen to define it
unconditionally (without checking for platform or version), since it
has no effect for other platforms or versions. Second, we need to deal
with <filesystem> no longer including <experimental/filesystem>.
I verified that VS 2015 Update 3 contained <experimental/filesystem>
(back then, it simply included the <filesystem> header, where the
experimental implementation was defined; this was later reorganized).
Therefore, all of vcpkg's supported MSVC toolsets have
<experimental/filesystem>, so we can simply always include it.
I've verified that this builds with both VS 2015 Update 3 and
VS 2019 16.1.3 (the current production version).
2019-06-21 02:46:55 +08:00
|
|
|
#define _SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING
|
2017-10-17 02:44:04 +08:00
|
|
|
#include <cstring>
|
2020-01-18 07:36:08 +08:00
|
|
|
#if USE_STD_FILESYSTEM
|
|
|
|
|
#include <filesystem>
|
|
|
|
|
#else
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
#include <experimental/filesystem>
|
2020-01-18 07:36:08 +08:00
|
|
|
#endif
|
2017-01-28 06:33:54 +08:00
|
|
|
#include <fstream>
|
|
|
|
|
#include <functional>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <iomanip>
|
|
|
|
|
#include <iostream>
|
2017-01-28 04:49:09 +08:00
|
|
|
#include <iterator>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <map>
|
|
|
|
|
#include <memory>
|
2017-08-26 07:03:57 +08:00
|
|
|
#include <mutex>
|
2017-12-02 08:08:09 +08:00
|
|
|
#include <random>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <regex>
|
|
|
|
|
#include <set>
|
|
|
|
|
#include <stdexcept>
|
|
|
|
|
#include <string>
|
2018-03-13 21:25:59 +08:00
|
|
|
#if defined(_WIN32)
|
2017-01-28 06:33:54 +08:00
|
|
|
#include <sys/timeb.h>
|
2018-03-13 21:25:59 +08:00
|
|
|
#else
|
|
|
|
|
#include <sys/time.h>
|
|
|
|
|
#endif
|
2017-10-17 02:44:04 +08:00
|
|
|
#include <sys/types.h>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <system_error>
|
2017-10-17 02:44:04 +08:00
|
|
|
#include <thread>
|
2017-01-28 06:33:54 +08:00
|
|
|
#include <time.h>
|
2017-05-24 15:44:00 +08:00
|
|
|
#include <type_traits>
|
2017-03-28 15:02:33 +08:00
|
|
|
#include <unordered_map>
|
|
|
|
|
#include <unordered_set>
|
|
|
|
|
#include <utility>
|
|
|
|
|
#include <vector>
|