2019-08-10 02:16:35 +08:00
|
|
|
#include <catch2/catch.hpp>
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
#include <vcpkg/pragmas.h>
|
|
|
|
|
2019-07-20 03:56:24 +08:00
|
|
|
#include <vcpkg/base/files.h>
|
2019-07-19 10:07:00 +08:00
|
|
|
#include <vcpkg/statusparagraph.h>
|
|
|
|
|
|
|
|
#include <memory>
|
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
#define CHECK_EC(ec) \
|
|
|
|
do \
|
|
|
|
{ \
|
|
|
|
if (ec) \
|
|
|
|
{ \
|
|
|
|
FAIL(ec.message()); \
|
|
|
|
} \
|
|
|
|
} while (0)
|
2019-07-19 10:07:00 +08:00
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
namespace vcpkg::Test
|
|
|
|
{
|
2020-02-04 06:22:52 +08:00
|
|
|
std::unique_ptr<SourceControlFile> make_control_file(
|
|
|
|
const char* name,
|
|
|
|
const char* depends,
|
|
|
|
const std::vector<std::pair<const char*, const char*>>& features = {},
|
|
|
|
const std::vector<const char*>& default_features = {});
|
|
|
|
|
2020-02-10 06:50:26 +08:00
|
|
|
inline auto test_parse_control_file(const std::vector<std::unordered_map<std::string, std::string>>& v)
|
|
|
|
{
|
|
|
|
std::vector<vcpkg::Parse::Paragraph> pghs;
|
|
|
|
for (auto&& p : v)
|
|
|
|
{
|
|
|
|
pghs.emplace_back();
|
|
|
|
for (auto&& kv : p)
|
|
|
|
pghs.back().emplace(kv.first, std::make_pair(kv.second, vcpkg::Parse::TextRowCol{}));
|
|
|
|
}
|
|
|
|
return vcpkg::SourceControlFile::parse_control_file("", std::move(pghs));
|
|
|
|
}
|
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
std::unique_ptr<vcpkg::StatusParagraph> make_status_pgh(const char* name,
|
|
|
|
const char* depends = "",
|
|
|
|
const char* default_features = "",
|
|
|
|
const char* triplet = "x86-windows");
|
2019-07-19 10:07:00 +08:00
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
std::unique_ptr<vcpkg::StatusParagraph> make_status_feature_pgh(const char* name,
|
|
|
|
const char* feature,
|
|
|
|
const char* depends = "",
|
|
|
|
const char* triplet = "x86-windows");
|
2019-07-19 10:07:00 +08:00
|
|
|
|
2020-02-04 06:22:52 +08:00
|
|
|
/// <summary>
|
|
|
|
/// Map of source control files by their package name.
|
|
|
|
/// </summary>
|
|
|
|
struct PackageSpecMap
|
|
|
|
{
|
|
|
|
std::unordered_map<std::string, SourceControlFileLocation> map;
|
|
|
|
Triplet triplet;
|
2020-02-08 03:24:35 +08:00
|
|
|
PackageSpecMap(Triplet t = Triplet::X86_WINDOWS) noexcept : triplet(t) {}
|
2020-02-04 06:22:52 +08:00
|
|
|
|
|
|
|
PackageSpec emplace(const char* name,
|
|
|
|
const char* depends = "",
|
|
|
|
const std::vector<std::pair<const char*, const char*>>& features = {},
|
|
|
|
const std::vector<const char*>& default_features = {});
|
|
|
|
|
|
|
|
PackageSpec emplace(vcpkg::SourceControlFileLocation&& scfl);
|
|
|
|
};
|
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
template<class T, class S>
|
|
|
|
T&& unwrap(vcpkg::ExpectedT<T, S>&& p)
|
|
|
|
{
|
|
|
|
REQUIRE(p.has_value());
|
|
|
|
return std::move(*p.get());
|
|
|
|
}
|
2019-07-19 10:07:00 +08:00
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
template<class T>
|
|
|
|
T&& unwrap(vcpkg::Optional<T>&& opt)
|
|
|
|
{
|
|
|
|
REQUIRE(opt.has_value());
|
|
|
|
return std::move(*opt.get());
|
|
|
|
}
|
|
|
|
|
|
|
|
struct AllowSymlinks
|
|
|
|
{
|
|
|
|
enum Tag : bool
|
|
|
|
{
|
|
|
|
No = false,
|
|
|
|
Yes = true,
|
|
|
|
} tag;
|
|
|
|
|
|
|
|
constexpr AllowSymlinks(Tag tag) noexcept : tag(tag) {}
|
|
|
|
|
|
|
|
constexpr explicit AllowSymlinks(bool b) noexcept : tag(b ? Yes : No) {}
|
2019-07-19 10:07:00 +08:00
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
constexpr operator bool() const noexcept { return tag == Yes; }
|
|
|
|
};
|
2019-07-20 03:56:24 +08:00
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
AllowSymlinks can_create_symlinks() noexcept;
|
2019-07-20 03:56:24 +08:00
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
const fs::path& base_temporary_directory() noexcept;
|
2019-07-20 14:20:28 +08:00
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
void create_symlink(const fs::path& file, const fs::path& target, std::error_code& ec);
|
2019-07-20 14:20:28 +08:00
|
|
|
|
[vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
2019-08-03 07:49:45 +08:00
|
|
|
void create_directory_symlink(const fs::path& file, const fs::path& target, std::error_code& ec);
|
2019-07-19 10:07:00 +08:00
|
|
|
}
|