mirror/vcpkg
1
0
Fork 0
mirror of https://github.com/microsoft/vcpkg.git synced 2024-12-20 11:57:50 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
32b7ad04ea
vcpkg/scripts/azure-pipelines/windows/azure-pipelines.yml

80 lines
3.2 KiB
YAML
Raw Normal View History

Change supporting infrastructure to use Azure Virtual Machine Scale Sets for vcpkg's PR builds, which should both improve our PR build times and reduce Azure spending by shutting down machines when they aren't being used. Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first. Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works. It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.
2020-04-22 08:12:21 +08:00
# Copyright (c) Microsoft Corporation.
# SPDX-License-Identifier: MIT
#
jobs:
- job: ${{ parameters.jobName }}
pool:
[vcpkg] Copy macos pipelines into azure-pipelines.yml
2020-04-29 04:02:39 +08:00
name: ${{ parameters.poolName }}
[vcpkg CI] Clean git directory before clone (#11450)
2020-05-20 03:23:39 +08:00
workspace:
clean: resources
Change supporting infrastructure to use Azure Virtual Machine Scale Sets for vcpkg's PR builds, which should both improve our PR build times and reduce Azure spending by shutting down machines when they aren't being used. Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first. Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works. It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.
2020-04-22 08:12:21 +08:00
timeoutInMinutes: 1440 # 1 day
steps:
- task: PowerShell@2
displayName: 'Initialize Environment'
inputs:
filePath: 'scripts/azure-pipelines/windows/initialize-environment.ps1'
[vcpkg] Add disk space report to Windows PR/CI (#12113) * [vcpkg] Add disk space report to PR/CI Example output: ``` Disk Label Size Free Space ---- ----- ---- ---------- C: Sabrent 1907 GiB 1239 GiB D: Dev 447 GiB 383 GiB E: Samsung 960 Pro 1908 GiB 1084 GiB H: Rocket 3815 GiB 863 GiB R: 0 B 0 B S: 0 B 0 B ```
2020-06-26 07:38:13 +08:00
- task: PowerShell@2
displayName: 'Report on Disk Space'
condition: always()
inputs:
filePath: 'scripts/azure-pipelines/windows/disk-space.ps1'
[vcpkg ci] Update formatting CI (#12314) * stage checking formatting this means that the port CI won't run if the formatting failed * fix invalid names * add formatting of manifests * fix dependsOn name * fix a thing * CRs * oops, typo
2020-07-08 05:09:33 +08:00
# Note: D: is the Azure machines' temporary disk.
[vcpkg] Allow CI to pass in all relevant directories and remove use of symbolic links (#11483)
2020-06-04 10:31:28 +08:00
- task: CmdLine@2
[vcpkg] Remove powershell from the 'run vcpkg ci' path to reduce hangs from msys components. (#11512) * [vcpkg] Remove powershell from the 'run vcpkg ci' path to reduce hangs from msys components. * The BatchScript task uses filename rather than filePath * cmd hates BOMs * cmd hates forward slashes
2020-05-22 11:48:05 +08:00
displayName: 'Build vcpkg'
inputs:
[vcpkg] Allow CI to pass in all relevant directories and remove use of symbolic links (#11483)
2020-06-04 10:31:28 +08:00
script: |
set VCPKG_DOWNLOADS=D:\downloads
.\bootstrap-vcpkg.bat
[vcpkg] Merge unit test pass into x86-windows. (#11858)
2020-06-12 08:49:22 +08:00
- task: CmdLine@2
displayName: "Build vcpkg with CMake and Run Tests"
condition: eq('${{ parameters.triplet }}', 'x86-windows')
inputs:
script: |
:: TRANSITION, get these tools on the VMs next time we roll them
set VCPKG_DOWNLOADS=D:\downloads
.\vcpkg.exe fetch cmake
.\vcpkg.exe fetch ninja
set PATH=D:\downloads\tools\cmake-3.17.2-windows\cmake-3.17.2-win32-x86\bin;D:\downloads\tools\ninja-1.10.0-windows;%PATH%
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\Tools\VsDevCmd.bat" -arch=x86 -host_arch=x86
rmdir /s /q build.x86.debug > nul 2> nul
cmake.exe -G Ninja -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=ON -DVCPKG_DEVELOPMENT_WARNINGS=ON -DVCPKG_WARNINGS_AS_ERRORS=ON -DVCPKG_BUILD_FUZZING=ON -B build.x86.debug -S toolsrc
ninja.exe -C build.x86.debug
build.x86.debug\vcpkg-test.exe
[vcpkg] Implement --x-write-nuget-packages-config= setting for `install` and `x-set-installed` (#12138) * [vcpkg] Implement --x-write-nuget-packages-config= setting for `install` and `x-set-installed`. * [vcpkg] Add end-to-end testing suite for install, remove, and binary caching * [vcpkg] Define `$TestingRoot in end-to-end-tests.ps1 * [vcpkg] Address CR comments Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-02 02:36:09 +08:00
powershell.exe -NoProfile -ExecutionPolicy Bypass "scripts\azure-pipelines\end-to-end-tests.ps1 -WorkingRoot \"%cd%\testing\" -triplet x86-windows"
[vcpkg] Merge unit test pass into x86-windows. (#11858)
2020-06-12 08:49:22 +08:00
failOnStderr: true
Change supporting infrastructure to use Azure Virtual Machine Scale Sets for vcpkg's PR builds, which should both improve our PR build times and reduce Azure spending by shutting down machines when they aren't being used. Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first. Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works. It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.
2020-04-22 08:12:21 +08:00
- task: PowerShell@2
[vcpkg] Commonize pipelines build powershell and don't use binary caching in CI (rather than PR) builds (#12082)
2020-06-26 07:36:28 +08:00
displayName: '*** Test Modified Ports and Prepare Test Logs ***'
Change supporting infrastructure to use Azure Virtual Machine Scale Sets for vcpkg's PR builds, which should both improve our PR build times and reduce Azure spending by shutting down machines when they aren't being used. Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first. Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works. It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.
2020-04-22 08:12:21 +08:00
inputs:
failOnStderr: true
[vcpkg] Commonize pipelines build powershell and don't use binary caching in CI (rather than PR) builds (#12082)
2020-06-26 07:36:28 +08:00
filePath: 'scripts/azure-pipelines/test-modified-ports.ps1'
[vcpkg] Remove the tombstones and 'ignore' baseline concepts. (#12197) This changes our PR builds to treat 'fail' in the ci.baseline.txt as 'skip' instead of using tombstones. We currently have large numbers of spurious failures that get enshrined in PRs through no fault of a PR author, removing the tombstones concept will fix those by allowing the user to retry. This does mean we accept some risk of not detecting when a port is 'fixed', but that failure is reasonable for us to handle after we see it in CI, but that seems worth it given that it lets us get rid of the tombstone concept. This also helps out the binary caching feature, because we don't have to figure out how to productize tombstones.
2020-07-03 11:20:07 +08:00
arguments: '-Triplet ${{ parameters.triplet }} -BuildReason $(Build.Reason) -ArchivesRoot W:\ -WorkingRoot D:\ -ArtifactsDirectory $(System.ArtifactsDirectory)'
[vcpkg] Add disk space report to Windows PR/CI (#12113) * [vcpkg] Add disk space report to PR/CI Example output: ``` Disk Label Size Free Space ---- ----- ---- ---------- C: Sabrent 1907 GiB 1239 GiB D: Dev 447 GiB 383 GiB E: Samsung 960 Pro 1908 GiB 1084 GiB H: Rocket 3815 GiB 863 GiB R: 0 B 0 B S: 0 B 0 B ```
2020-06-26 07:38:13 +08:00
- task: PowerShell@2
displayName: 'Report on Disk Space After Build'
condition: always()
inputs:
filePath: 'scripts/azure-pipelines/windows/disk-space.ps1'
Change supporting infrastructure to use Azure Virtual Machine Scale Sets for vcpkg's PR builds, which should both improve our PR build times and reduce Azure spending by shutting down machines when they aren't being used. Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first. Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works. It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.
2020-04-22 08:12:21 +08:00
- task: PublishBuildArtifacts@1
[vcpkg] Onboard Linux to VMSS, open 'git' port, and switch back to Azure Spot * Adds scripts to generate scale sets for testing Linux. * Note workaround for https://github.com/microsoft/azure-pipelines-agent/pull/2929 * Switches Windows validation to 'Spot' VMs. * Opens the git port 9418. * Removes provisioning of the no longer used 'logs' file share. * Changes Azure region to 'westus2', which is cheaper. * Adds +x to all the scripts in scripts/azure-pipelines. * Use 'xml-results' for all platforms instead of 'raw xml results' on Windows.
2020-04-29 13:16:12 +08:00
displayName: 'Publish Artifact: ${{ parameters.triplet }} port build failure logs'
Change supporting infrastructure to use Azure Virtual Machine Scale Sets for vcpkg's PR builds, which should both improve our PR build times and reduce Azure spending by shutting down machines when they aren't being used. Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first. Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works. It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.
2020-04-22 08:12:21 +08:00
inputs:
[vcpkg] Remove the tombstones and 'ignore' baseline concepts. (#12197) This changes our PR builds to treat 'fail' in the ci.baseline.txt as 'skip' instead of using tombstones. We currently have large numbers of spurious failures that get enshrined in PRs through no fault of a PR author, removing the tombstones concept will fix those by allowing the user to retry. This does mean we accept some risk of not detecting when a port is 'fixed', but that failure is reasonable for us to handle after we see it in CI, but that seems worth it given that it lets us get rid of the tombstone concept. This also helps out the binary caching feature, because we don't have to figure out how to productize tombstones.
2020-07-03 11:20:07 +08:00
PathtoPublish: '$(System.ArtifactsDirectory)\failure-logs'
[vcpkg] Onboard Linux to VMSS, open 'git' port, and switch back to Azure Spot * Adds scripts to generate scale sets for testing Linux. * Note workaround for https://github.com/microsoft/azure-pipelines-agent/pull/2929 * Switches Windows validation to 'Spot' VMs. * Opens the git port 9418. * Removes provisioning of the no longer used 'logs' file share. * Changes Azure region to 'westus2', which is cheaper. * Adds +x to all the scripts in scripts/azure-pipelines. * Use 'xml-results' for all platforms instead of 'raw xml results' on Windows.
2020-04-29 13:16:12 +08:00
ArtifactName: '${{ parameters.triplet }} port build failure logs'
Change supporting infrastructure to use Azure Virtual Machine Scale Sets for vcpkg's PR builds, which should both improve our PR build times and reduce Azure spending by shutting down machines when they aren't being used. Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first. Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works. It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.
2020-04-22 08:12:21 +08:00
condition: failed()
[vcpkg] Added python script to generate all packages file list, added to azur… (#12177) * Added python script to generate all packages file list, added to azure pipeline to build at the end of the run * fixed an issue that causes some leading slashes were being removed * Header Database now includes hpp files for C++ header files * Changed Header Database to include all files under \include\ folder * Apply suggestions from code review, changing condition from eq to ne Co-authored-by: Billy O'Neal <bion@microsoft.com> * Update last condition from eq to ne Co-authored-by: Billy O'Neal <bion@microsoft.com>
2020-07-07 04:38:22 +08:00
- task: PowerShell@2
displayName: "Generating all packages files"
[vcpkg-ci] Always publish file lists (#12384) Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-14 03:14:51 +08:00
condition: always()
[vcpkg] Added python script to generate all packages file list, added to azur… (#12177) * Added python script to generate all packages file list, added to azure pipeline to build at the end of the run * fixed an issue that causes some leading slashes were being removed * Header Database now includes hpp files for C++ header files * Changed Header Database to include all files under \include\ folder * Apply suggestions from code review, changing condition from eq to ne Co-authored-by: Billy O'Neal <bion@microsoft.com> * Update last condition from eq to ne Co-authored-by: Billy O'Neal <bion@microsoft.com>
2020-07-07 04:38:22 +08:00
inputs:
targetType: inline
script: |
$env:VCPKG_DOWNLOADS = "D:\downloads"
./vcpkg.exe fetch python3
& $(.\vcpkg fetch python3) .\scripts\file_script.py D:\installed\vcpkg\info\
- task: PublishBuildArtifacts@1
displayName: 'Upload file lists for all packages'
[vcpkg-ci] Always publish file lists (#12384) Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-07-14 03:14:51 +08:00
condition: always()
[vcpkg] Added python script to generate all packages file list, added to azur… (#12177) * Added python script to generate all packages file list, added to azure pipeline to build at the end of the run * fixed an issue that causes some leading slashes were being removed * Header Database now includes hpp files for C++ header files * Changed Header Database to include all files under \include\ folder * Apply suggestions from code review, changing condition from eq to ne Co-authored-by: Billy O'Neal <bion@microsoft.com> * Update last condition from eq to ne Co-authored-by: Billy O'Neal <bion@microsoft.com>
2020-07-07 04:38:22 +08:00
inputs:
PathtoPublish: scripts/list_files
ArtifactName: "${{ parameters.triplet }} package file lists"
Reference in New Issue Copy Permalink