vcpkg/scripts/azure-pipelines/windows/create-vmss.ps1

99 lines
3.0 KiB
PowerShell
Raw Normal View History

# Copyright (c) Microsoft Corporation.
# SPDX-License-Identifier: MIT
#
<#
.SYNOPSIS
Creates a Windows virtual machine scale set, set up for vcpkg's CI.
.DESCRIPTION
create-vmss.ps1 creates an Azure Windows VM scale set, set up for vcpkg's CI
system. See https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/overview
for more information.
This script assumes you have installed Azure tools into PowerShell by following the instructions
at https://docs.microsoft.com/en-us/powershell/azure/install-az-ps?view=azps-3.6.1
or are running from Azure Cloud Shell.
.PARAMETER ImageName
The name of the image to deploy into the scale set.
#>
[CmdLetBinding()]
Param(
Update VMs, CMake to 3.20.1, CUDA to 11.3, and pwsh to 7.1.3 (#17331) * Update to CUDA 11.3 on Windows. * Update PowerShell to 7.1.3. * Update CUDA to 11.3 on Linux. * "Explode" VM provisioning scripts for more consistent feedback during deploy. This resolves the deployment script often hanging with no feedback as to why. * [libdatachannel] Fix use of deprecated result_type typedef. Submitted upstream as https://github.com/paullouisageneau/libdatachannel/pull/413 * [libvpx] Get the libvpx outputs from the correct place. (Perhaps VS2019 version 16.10 moved where these are placed? I've been defensive and left an attempt to find from the old location in place.) * [chromaprint] Support implementations where lrintf is an intrinsic. * Add provision-entire-image script. * [cudnn] Disable download-on-the-fly due to licensing concerns. * Add libnccl to Linux VMs. * [wangle] Disable x64-windows due to an ICE. * [cmake] Update cmake to 3.20.1 to avoid https://gitlab.kitware.com/cmake/cmake/-/issues/21571 race * [libudis86] Fix passing a bogus working directory which fails on CMake 3.20.x * [dartsim] Disable unit tests, examples, and tutorials, some of which have CMake authoring errors rejected by 3.20.1. * Add thrust to the cuda installees. * [tensorflow] Put .bzl in CURRENT_BUILDTREES_DIR to avoid running out of disk space in CI and to respect --clean-after-build. * [dimcli] Skip port broken by changes in VS2019 project system. * [upb] Disable an additional warning. * [libhv] Fix typo DISABLE_PARALLEL => DISABLE_PARALLEL_CONFIGURE * Update pools
2021-04-27 01:27:45 +08:00
[parameter(Mandatory=$true)]
[string]$ImageName
)
$Location = 'eastasia'
$Prefix = 'PrWin-'
$Prefix += (Get-Date -Format 'yyyy-MM-dd')
$VMSize = 'Standard_D32a_v4'
$LiveVMPrefix = 'BUILD'
$ErrorActionPreference = 'Stop'
Import-Module "$PSScriptRoot/../create-vmss-helpers.psm1" -DisableNameChecking
$ResourceGroupName = Find-ResourceGroupName $Prefix
$AdminPW = New-Password
$Image = Get-AzImage -ResourceGroupName 'vcpkg-image-minting' -ImageName $ImageName
New-AzResourceGroup -Name $ResourceGroupName -Location $Location
$VirtualNetwork = Create-LockedDownNetwork -ResourceGroupName $ResourceGroupName -Location $Location
$VmssIpConfigName = $ResourceGroupName + 'VmssIpConfig'
$VmssIpConfig = New-AzVmssIpConfig -SubnetId $VirtualNetwork.Subnets[0].Id -Primary -Name $VmssIpConfigName
$VmssName = $ResourceGroupName + 'Vmss'
$Vmss = New-AzVmssConfig `
-Location $Location `
-SkuCapacity 0 `
-SkuName $VMSize `
-SkuTier 'Standard' `
-Overprovision $false `
-UpgradePolicyMode Manual `
-EvictionPolicy Delete `
-Priority Spot `
-MaxPrice -1
$NicName = $ResourceGroupName + 'NIC'
New-AzNetworkInterface `
-Name $NicName `
-ResourceGroupName $ResourceGroupName `
-Location $Location `
-Subnet $VirtualNetwork.Subnets[0]
$Vmss = Add-AzVmssNetworkInterfaceConfiguration `
-VirtualMachineScaleSet $Vmss `
-Primary $true `
-IpConfiguration $VmssIpConfig `
-NetworkSecurityGroupId $VirtualNetwork.Subnets[0].NetworkSecurityGroup.Id `
-Name $NicName
$Vmss = Set-AzVmssOsProfile `
-VirtualMachineScaleSet $Vmss `
-ComputerNamePrefix $LiveVMPrefix `
-AdminUsername 'AdminUser' `
-AdminPassword $AdminPW `
-WindowsConfigurationProvisionVMAgent $true `
-WindowsConfigurationEnableAutomaticUpdate $false
$Vmss = Set-AzVmssStorageProfile `
-VirtualMachineScaleSet $Vmss `
-OsDiskCreateOption 'FromImage' `
-OsDiskCaching ReadOnly `
-DiffDiskSetting Local `
-ImageReferenceId $Image.Id
Update Windows VMs for Patch Tuesday August 2022 (#26283) * Install haskell-stack with apt instead of piping a sh. * Update pools. * Linux is exploding for some reason. I just found out it is getting created with a 30GB disk I'm guessing may be involved. Did not wait for x64_windows_static_md to finish because enough was already on the floor. PASSING, REMOVE FROM FAIL LIST: angle:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: angle:arm64-windows (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: angle:x64-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: angle:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: angle:x64-windows-static (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: angle:x86-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: rsocket:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). PASSING, REMOVE FROM FAIL LIST: rsocket:x64-windows-static (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). REGRESSION: gazebo:x64-windows failed with BUILD_FAILED. If expected, add gazebo:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. ``` D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2275: 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType': expected an expression instead of a type D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2923: 'std::shared_ptr': 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): note: see declaration of 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list ``` We have been seeing intermittent failures in gazebo:x64-windows and I think this just repro'd that... REGRESSION: graphicsmagick:x64-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: graphicsmagick:x64-windows-static failed with BUILD_FAILED. If expected, add graphicsmagick:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: graphicsmagick:x86-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. ``` D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick/nt_base.h(95): warning C4005: 'HAVE_FT2BUILD_H': macro redefinition D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick/magick_config.h(123): note: see previous definition of 'HAVE_FT2BUILD_H' D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick\effect.c(689) : fatal error C1001: Internal compiler error. (compiler file 'D:\a\_work\1\s\src\vctools\Compiler\Utc\src\p2\main.c', line 224) To work around this problem, try simplifying or changing the program near the locations listed above. If possible please provide a repro here: https://developercommunity.visualstudio.com Please choose the Technical Support command on the Visual C++ Help menu, or open the Technical Support help file for more information cl!RaiseException()+0x6c cl!RaiseException()+0x6c cl!DllGetObjHandler()+0xdb94f cl!DllGetObjHandler()+0x1ac555 ``` 😭 REGRESSION: mathgl:x64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: mathgl:x64-windows-static failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: mathgl:x86-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. ``` -- Performing post-build validation The following files contain an absolute path ('D:\packages\mathgl_x64-windows', 'D:\installed', 'D:\buildtrees\mathgl'): D:\packages\mathgl_x64-windows\include\mgl2\config.h There should be no absolute paths in the installed package, only relative ones. Found 1 post-build check problem(s). To submit these ports to curated catalogs, please first correct the portfile: C:\a\2\s\ports\mathgl\portfile.cmake -- Performing post-build validation done error: building mathgl:x64-windows failed with: POST_BUILD_CHECKS_FAILED ``` @autoantwort REGRESSION: qtwebengine:arm64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add qtwebengine:arm64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt. ``` Building qtwebengine[core,default-features,geolocation,spellchecker,webchannel]:arm64-windows... -- Setting up python virtual environmnent... -- Installing python packages: html5lib -- Setting up python virtual environmnent...finished. CMake Warning at ports/qtwebengine/portfile.cmake:83 (message): Buildtree path 'D:/buildtrees/qtwebengine' is too long. Consider passing --x-buildtrees-root=<shortpath> to vcpkg! Trying to use 'D:/buildtrees/qtwebengine/../tmp' Call Stack (most recent call first): scripts/ports.cmake:147 (include) -- Using cached qtwebengine-everywhere-src-6.3.1.tar.xz. -- Cleaning sources at D:/buildtrees/tmp/src/here-src-6-367c283e50.clean. Use --editable to skip cleaning for the packages you specify. -- Extracting source D:/downloads/qtwebengine-everywhere-src-6.3.1.tar.xz -- Using source at D:/buildtrees/tmp/src/here-src-6-367c283e50.clean -- Found external ninja('1.10.2'). -- Configuring arm64-windows-dbg -- Configuring arm64-windows-rel CMake Warning at D:/installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_configure.cmake:302 (message): The following variables are not used in CMakeLists.txt: FEATURE_webengine_geolocation FEATURE_webengine_spellchecker FEATURE_webengine_v8_snapshot_support FEATURE_webengine_webchannel FEATURE_webengine_webrtc Please recheck them and remove the unnecessary options from the `vcpkg_cmake_configure` call. If these options should still be passed for whatever reason, please use the `MAYBE_UNUSED_VARIABLES` argument. Call Stack (most recent call first): D:/installed/arm64-windows/share/qtbase/qt_install_submodule.cmake:108 (vcpkg_cmake_configure) ports/qtwebengine/portfile.cmake:102 (qt_cmake_configure) scripts/ports.cmake:147 (include) -- Building arm64-windows-dbg -- Building arm64-windows-rel -- Installing: D:/packages/qtwebengine_arm64-windows/share/qtwebengine/copyright -- Performing post-build validation The folder /include is empty or not present. This indicates the library was not correctly installed. There should be no empty directories in D:\packages\qtwebengine_arm64-windows The following empty directories were found: ``` This was previously blocked by qtwebengine:x64-windows=fail. Will investigate... * Remove attempt to install extra stuff for scale set agents. * Turn on boot diagnostics. * Update pools. * Back out most Linux changes. * Analysis of https://dev.azure.com/vcpkg/public/_build/results?buildId=76482&view=results REGRESSION: graphicsmagick:x86-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. Added this skip too. REGRESSION: gazebo:x64-windows failed with BUILD_FAILED. If expected, add gazebo:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. ``` C:\PROGRA~1\MICROS~1\2022\ENTERP~1\VC\Tools\MSVC\1433~1.316\bin\Hostx64\x64\cl.exe /TP -DBOOST_ALL_DYN_LINK -DBOOST_ALL_NO_LIB -DBOOST_TEST_DYN_LINK -DBUILDING_DLL -DBUILDING_DLL_GZ_PHYSICS -DFREEIMAGE_LIB -DH5_BUILT_AS_DYNAMIC_LIB -DIGN_PROFILER_ENABLE=0 -DLIBBULLET_VERSION=3.21 -DLIBBULLET_VERSION_GT_282 -DNOMINMAX -DNOUSER -DPROTOBUF_USE_DLLS -DTINYXML2_DEBUG -DTINYXML2_IMPORT -DWIN32_LEAN_AND_MEAN -D_USE_MATH_DEFINES -DdIDEDOUBLE -Dgazebo_physics_EXPORTS -ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\test\gtest\include -ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean -ID:\buildtrees\gazebo\x64-windows-dbg -ID:\installed\x64-windows\debug\..\include\libusb-1.0 -external:ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\deps\opende\include -external:ID:\installed\x64-windows\include -external:ID:\installed\x64-windows\debug\..\include -external:ID:\installed\x64-windows\include\ignition\msgs5 -external:ID:\installed\x64-windows\include\ignition\math6 -external:ID:\installed\x64-windows\include\ignition\transport8 -external:ID:\installed\x64-windows\include\ignition\common3 -external:ID:\installed\x64-windows\include\ignition\fuel_tools4 -external:ID:\installed\x64-windows\include\OGRE -external:ID:\installed\x64-windows\include\sdformat-9.8 -external:ID:\installed\x64-windows\include\bullet -external:ID:\installed\x64-windows\include\sdformat-9.8\sdf\.. -external:ID:\installed\x64-windows\include\ignition\cmake2 -external:ID:\installed\x64-windows\include\eigen3 -external:ID:\installed\x64-windows\share\urdfdom_headers\..\..\include -external:ID:\installed\x64-windows\include\urdfdom -external:ID:\installed\x64-windows\include\urdfdom_headers -external:W0 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP /wd4005 /wd4068 /wd4244 /wd4251 /wd4267 /wd4275 /wd4996 /EHsc /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /EHsc -I"D:/installed/x64-windows/include" /Zc:__cplusplus /permissive- /Zc:strictStrings- /Zc:externC- -std:c++17 /showIncludes /Fogazebo\physics\CMakeFiles\gazebo_physics.dir\dart\DARTBoxShape.cc.obj /Fdgazebo\physics\CMakeFiles\gazebo_physics.dir\ /FS -c D:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\gazebo\physics\dart\DARTBoxShape.cc D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2275: 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType': expected an expression instead of a type D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2923: 'std::shared_ptr': 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): note: see declaration of 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2275: 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType': expected an expression instead of a type D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2923: 'std::shared_ptr': 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): note: see declaration of 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2955: 'std::shared_ptr': use of class template requires template argument list C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr' D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2955: 'std::shared_ptr': use of class template requires template argument list C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr' ``` PASSING, REMOVE FROM FAIL LIST: luajit:arm64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt). This was https://github.com/microsoft/vcpkg/pull/26360 REGRESSION: qtwebengine:arm64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add qtwebengine:arm64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. This was previously blocked by x64-windows being skipped, so I'm just adding it to the baseline for now.
2022-08-18 07:44:51 +08:00
$Vmss = Set-AzVmssBootDiagnostic `
-VirtualMachineScaleSet $Vmss `
-Enabled $true
New-AzVmss `
-ResourceGroupName $ResourceGroupName `
-Name $VmssName `
-VirtualMachineScaleSet $Vmss
Write-Host "Location: $Location"
Write-Host "Resource group name: $ResourceGroupName"
Write-Host 'Finished!'