* Add the ability to do full rebuilds with updated copies of vcpkg-tool.
Resurrection of https://github.com/microsoft/vcpkg/pull/23753/
* Extract all scripts out of inline blocks.
* Replicate all changes to the "unstable" testing.
* Fix bootstrap :)
* Fix version files validation path.
* Malformed.
* Paths and permissions.
* Fixed file name typo
* Update scripts/azure-pipelines/bootstrap-from-source.sh
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* Install Ubuntu nasm package rather than building from source; 2.14 which is available in Ubuntu 20.04 is sufficient for intel-ipsec.
* Add cmake and ninja.
* Update CUDA signing key.
* Update pwsh to 7.2.3
* Remove clean downloads step obsoleted since we ripped out pacman.
* Note that haskell-stack is for bond.
* Cherry pick from https://github.com/microsoft/vcpkg/pull/23569 : add cuda-nvml-dev-11-6
* Update pools.
* Baseline updates
PASSING, REMOVE FROM FAIL LIST: mesa:x64-windows-static-md (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
:)
```
REGRESSION: cppgraphqlgen:arm64-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:arm64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows-static failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows-static-md failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x86-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```
This is a compiler behavior change or bug in VS 2022 16.2:
```
C:\PROGRA~1\MICROS~1\2022\ENTERP~1\VC\Tools\MSVC\1432~1.313\bin\Hostx64\x86\cl.exe /TP -DGRAPHQL_DLLEXPORTS -DIMPL_GRAPHQLSERVICE_DLL -Dgraphqlservice_EXPORTS -ID:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\..\include -ID:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\..\PEGTL\include -external:ID:\installed\x86-windows\include -external:W0 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 -MDd /W4 /WX /permissive- -std:c++20 /showIncludes /Fosrc\CMakeFiles\graphqlservice.dir\GraphQLService.cpp.obj /Fdsrc\CMakeFiles\graphqlservice.dir\ /FS -c D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\GraphQLService.cpp
cl : Command line warning D9025 : overriding '/W3' with '/W4'
D:\installed\x86-windows\include\tao\pegtl/demangle.hpp(147): error C2338: static_assert failed: 'internal::dependent_true< T > && ( begin != std::string_view::npos )'
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\include\graphqlservice/internal/SyntaxTree.h(120): note: see reference to function template instantiation 'std::string_view tao::graphqlpeg::demangle<graphql::peg::variable_value>(void) noexcept' being compiled
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\include\graphqlservice/internal/SyntaxTree.h(53): note: see reference to function template instantiation 'std::string_view graphql::peg::ast_node::type_name<graphql::peg::variable_value>(void) noexcept' being compiled
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\GraphQLService.cpp(310): note: see reference to function template instantiation 'bool graphql::peg::ast_node::is_type<graphql::peg::variable_value>(void) noexcept const' being compiled
ninja: build stopped: subcommand failed.
```
This is a compiler behavior change in 17.2. @wravery do you have comments?
```
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```
qtwebengine doesn't report logs, but I'm pretty sure it's this ICE: https://developercommunity.visualstudio.com/t/Visual-Studio-2022-v1720-reports-fata/10039296 @Neumann-A
```
REGRESSION: rsocket:x64-windows failed with BUILD_FAILED. If expected, add rsocket:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: rsocket:x64-windows-static failed with BUILD_FAILED. If expected, add rsocket:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: rsocket:x64-windows-static-md failed with BUILD_FAILED. If expected, add rsocket:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```
ICE :( https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems/edit/1490389
Other changes:
* Removed qt5-webengine skips because they are now skipped by a "supports" clause.
* Removed ctp skips because asset caching exists.
* [stxxl] Guard definition of log2 for current MSVCs.
* Update VM base tools:
* Intel OneAPI updated
* Powershell Updated
* VS updated to install current Windows SDKs, and atlmfc for arm, and CMake/ninja
* Update pools.
* Remove passing-remove-from-fail list port.
* Update Linux VM CUDA to 11.6.
* Update Intel OneAPI
* Update Visual Studio to 2022, and remove deployment of v140 and v141 compilers.
* Update CUDA to 11.6 on Windows.
* Update pools.
* cix ci.baseline.txt
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Some of the Windows runs have started to exceed 24 hours now as well as the MacOS ones as we have added more ports.
* We are no longer trying to constantly "psychic debug" why ports are failing; CI passes somewhat regularly now, and the failures we get seem to be much more deterministic.
* We're still budget constrained on Azure resources (and have been a bit over budget)
To that end, we're going back off nightly CI runs to Monday Wednesday Friday (like osx already is), so we no longer need separate bits for them.
* add moar ram (no PR)
* Provision autoconf-archive on osx (PR #22872)
* [mpfr] Support Linux and OSX (PR #22845)
* VM software versions
* fix minor schema bug
* update pool
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: Kai Pastor <dg0yt@darc.de>
Co-authored-by: JackBoosY <yuzaiyang@beyondsoft.com>
* Revert "incorporate changes from microsoft:master"
* Revert "Revert "incorporate changes from microsoft:master""
* preliminary approach to solve pip issue
* fix python venv
* fix macOS build command again
* bump version
* x-add-version
* Fix spelling of prerequisites.
* Add python3-venv
* Update linux pool.
Co-authored-by: Joachim Gehweiler <joachim@Joachims-iMac.local>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
* [macos ci] move out tensorflow
* rename ExtraPorts and CorePorts
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
* [ci] Move formatting and doc checks into x86-windows to avoid burning a whole job for that 1 run, don't run file lists generation in PRs.
* Deduplicate x86-windows and restore PR file lists, as requested/suggested by @dg0yt
* More quotes
Co-authored-by: nicole mazzuca <83086508+strega-nil-ms@users.noreply.github.com>
Co-authored-by: nicole mazzuca <83086508+strega-nil-ms@users.noreply.github.com>
* update 2021-07-27
macOS -> 11.5.1, XCode -> 12.5.1
* update azure-pipelines
* update sha of macfuse
* change how macos-ci-base works
* fix build errors
* fix itpp:linux
* more fixes
* remove tab
* allow version changes in all the remove/rename-version patches
* fix libunistring for real
* robert CR
Remove no longer used "unstable" scaffolding.
Use ephemeral OS disks for better latency.
Add ability to change whether temp disks are used or explicitly provisioned disks are used; explicitly provisioned disks turned out to be more expensive than expected before.
Disable Windows Updates in the scale set because the machines are very short lived; working on a mechanism of patching the images instead.
* Cherry-pick https://github.com/microsoft/vcpkg/pull/15598
* Hook deploy-inteloneapi into create-vmss.ps1.
* Add script to resolve https://github.com/microsoft/vcpkg/issues/17521
* Move tls settings deployment to the front and respond to script triggering a reboot.
* Go back to provisioning an extra disk to workaround https://github.com/microsoft/vcpkg/issues/18379
* Disallow public access to blob storage and require TLS 1.2.
* Update Pools.
* Update tool to 2021-06-19
* [simage] Skip simage on uwp platforms as it appears broken by 16.10.
* [tensorflow-cc] Skip because changes in our MacOS hardware broke the port.
* start 2021-04-16 process
Major new things:
* update to macOS Big Sur (11.*)
* switch from VirtualBox to Parallels due to ^ (and also ARM)
Minor new things:
* update from xcode CLT 12 to 12.4
This PR includes new stuff for creating the base box for parallels.
Still to do: using the new box type.
* update the vagrantfile stuff
* link the CI to the new version
* modify how macOS boxes are made
the Vagrantfile for the final VM will only download the azure pipeline stuff
this allows us to keep the versions of brew applications stable
* fix cpus and memory
* add vagrant plugins to installables
* Skip cppgraphqlgen ICE.
* [sdformat6] Remove unneeded include(FindBoost) which causes problems when the system cmake version doesn't match the one deployed by vcpkg.
* switch to dmg for installing osxfuse/sshfs
* Set cores to 11 (leaving 1 thread for host)
Co-authored-by: Billy Robert ONeal III <bion@microsoft.com>
* Update to CUDA 11.3 on Windows.
* Update PowerShell to 7.1.3.
* Update CUDA to 11.3 on Linux.
* "Explode" VM provisioning scripts for more consistent feedback during deploy. This resolves the deployment script often hanging with no feedback as to why.
* [libdatachannel] Fix use of deprecated result_type typedef.
Submitted upstream as https://github.com/paullouisageneau/libdatachannel/pull/413
* [libvpx] Get the libvpx outputs from the correct place.
(Perhaps VS2019 version 16.10 moved where these are placed? I've been defensive and left an attempt to find from the old location in place.)
* [chromaprint] Support implementations where lrintf is an intrinsic.
* Add provision-entire-image script.
* [cudnn] Disable download-on-the-fly due to licensing concerns.
* Add libnccl to Linux VMs.
* [wangle] Disable x64-windows due to an ICE.
* [cmake] Update cmake to 3.20.1 to avoid https://gitlab.kitware.com/cmake/cmake/-/issues/21571 race
* [libudis86] Fix passing a bogus working directory which fails on CMake 3.20.x
* [dartsim] Disable unit tests, examples, and tutorials, some of which have CMake authoring errors rejected by 3.20.1.
* Add thrust to the cuda installees.
* [tensorflow] Put .bzl in CURRENT_BUILDTREES_DIR to avoid running out of disk space in CI and to respect --clean-after-build.
* [dimcli] Skip port broken by changes in VS2019 project system.
* [upb] Disable an additional warning.
* [libhv] Fix typo DISABLE_PARALLEL => DISABLE_PARALLEL_CONFIGURE
* Update pools
This reduces bootstrap cost for Windows customers, resolving the issue initially submitted as #12502 .
The `toolsrc` tree was extracted to https://github.com/microsoft/vcpkg-tool. `bootstrap.sh` was changed to download the right source tarball, extract, and build it. This was chosen over the previous attempt, a submodule, over concerns of accidentally destroying people's local modifications.
* Add meson from https://github.com/microsoft/vcpkg/pull/12860/
* Add autoconf-archive from https://github.com/microsoft/vcpkg/pull/13081/
* Add kf5windowsystem libs from https://github.com/microsoft/vcpkg/pull/13467/
* Open the FTP and SFTP ports from https://github.com/microsoft/vcpkg/pull/14412/
* Add libxcb-util0-dev from https://github.com/microsoft/vcpkg/pull/14678/
* Add libasound2-dev from https://github.com/microsoft/vcpkg/pull/14774
* Remove no longer necessary apt-mark calls.
* Update nasm on Linux.
* Fix longstanding bug where Storage was publicly accessible and change to generate SAS token rather than File Share
* Delete no longer necessary azure storage firewall rules.
* Install the newer Windows SDK with the VS installer instead of manually.
* Install the VS2015 and VS2017 compilers.
* Update Powershell-Core to 7.1.0.
* Update source of WDK.
* Update pools.
* [opentracing] Repair arm64-windows failures caused by mojibake in `expected.hpp` and errors in opentracing-cpp's lint for arm64 where it thinks exceptions are disabled when they are enabled.
Fixes:
C:\Dev\vcpkg\buildtrees\opentracing\src\b67575dab0-0250653c81.clean\3rd_party\include\opentracing/expected/expected.hpp(1): warning C4828: The file contains a character starting at offset 0x4a77 that is illegal in the current source character set (codepage 65001).
Fixes:
D:\buildtrees\opentracing\src\b67575dab0-0250653c81.clean\include\opentracing/tracer.h:223:5: error: cannot use 'try' with exceptions disabled [clang-diagnostic-error]
try {
^
* [mmloader] Patch out overrides of CMAKE_C_FLAGS and CMAKE_CXX_FLAGS that inserted /WX.
Note that this port sets /GS-, possibly because it may be for authoring shellcode.
* Bump storage API version to 2020-04-08.
* [vcpkg docs] Change how documenting port functions works
Instead of using `##`, use comment blocks for documentation.
Also, add some minor docs and change RST -> MD
so we actually get docs generated.
* add CI stuff
* regenerate docs
* fix vcpkg_find_acquire_program to not use _execute_process
* [vcpkg osx ci] update to newer bits
these ones should actually work!
* update vagrantfile
* correct scripts, reformat, add docs
* Finish Get-InternalBaseBox, move archives
In order to avoid people maliciously deleting the boxes,
the archives share and file share are in different users now.
* baseline libqcow to be fixed later
* actually check for the mount point
* fix Install-Prerequisites again
* add +x to Get-InternalBaseBox.ps1
additionally, fix some errors
* [vcpkg ci:osx] Remove brew install
* add instructions for creating a new vagrant box
* fix the vagrant scripts for the new box
* finish fixing the setup
* [mecab jxrlib] fix ports for CI
mecab needed to use an actual ref that wasn't master,
and jxrlib needed a patch for xcode 12 CLTs.
Additionally, this fixes the mecab version to be a date, the date of the last commit,
since `1.0` is not the correct version (mecab doesn't have released versions)
* [many ports] fix compile with Xcode 12 CLTs
This mostly means fixing errors on implicit-function-declaration,
and removing some Werrors
* alac-decoder
* apr
* argtable2
* arrow
* hyperscan
* mcpp
* minizip
* mosquitto
* stormlib
* [many ports] even more Xcode 12 CLT fixes
* [jxrlib darknet] fix the last ports! (hopefully)
* CRs, plus minor wip changes to osx scripts
* [vcpkg] Add `vcpkg export` to E2E tests. Enable E2E tests on MacOS.
* [vcpkg] Fix export --raw --output-dir=/path/ by changing directory to new export root
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
* stage checking formatting
this means that the port CI won't run if the formatting failed
* fix invalid names
* add formatting of manifests
* fix dependsOn name
* fix a thing
* CRs
* oops, typo
* [vcpkg] Bump Linux VM memory size and do all operations on the temporary disk.
* [llvm] disable optional dependencies to libxml2 and zlib
* [vcpkg] Reduce quotas on the share to something reasonable.
* [mpir] Skip on Linux because it conflicts with libgmp.
Co-authored-by: yurybura <yurybura@gmail.com>
* Adds scripts to generate scale sets for testing Linux.
* Note workaround for https://github.com/microsoft/azure-pipelines-agent/pull/2929
* Switches Windows validation to 'Spot' VMs.
* Opens the git port 9418.
* Removes provisioning of the no longer used 'logs' file share.
* Changes Azure region to 'westus2', which is cheaper.
* Adds +x to all the scripts in scripts/azure-pipelines.
* Use 'xml-results' for all platforms instead of 'raw xml results' on Windows.
Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first.
Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works.
It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.
