Commit Graph

15 Commits

Author SHA1 Message Date
Billy O'Neal
7f52deab66
Update VMs for Patch Tuesday May 2023 (#31358)
* Update PowerShell Core to not vulnerable versions.

* Fix Android create-image relative paths.

* Update pools and add docs.

* Get android create-vmss to setup managed identity.
2023-05-14 01:54:42 -07:00
dan-shaw
5787cfa699
[vcpkg] Android CI support (#29406)
* initial

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix
2023-04-24 21:22:59 -07:00
Billy O'Neal
4217fee510
Update Windows VMs for Patch Tuesday August 2022 (#26283)
* Install haskell-stack with apt instead of piping a sh.

* Update pools.

* Linux is exploding for some reason. I just found out it is getting created with a 30GB disk I'm guessing may be involved.

Did not wait for x64_windows_static_md to finish because enough was already on the floor.

PASSING, REMOVE FROM FAIL LIST: angle:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:arm64-windows (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-windows-static (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x86-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: rsocket:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: rsocket:x64-windows-static (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
REGRESSION: gazebo:x64-windows failed with BUILD_FAILED. If expected, add gazebo:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2275: 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2923: 'std::shared_ptr': 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): note: see declaration of 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
```

We have been seeing intermittent failures in gazebo:x64-windows and I think this just repro'd that...

REGRESSION: graphicsmagick:x64-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: graphicsmagick:x64-windows-static failed with BUILD_FAILED. If expected, add graphicsmagick:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: graphicsmagick:x86-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick/nt_base.h(95): warning C4005: 'HAVE_FT2BUILD_H': macro redefinition
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick/magick_config.h(123): note: see previous definition of 'HAVE_FT2BUILD_H'
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick\effect.c(689) : fatal error C1001: Internal compiler error.
(compiler file 'D:\a\_work\1\s\src\vctools\Compiler\Utc\src\p2\main.c', line 224)
 To work around this problem, try simplifying or changing the program near the locations listed above.
If possible please provide a repro here: https://developercommunity.visualstudio.com
Please choose the Technical Support command on the Visual C++
 Help menu, or open the Technical Support help file for more information
  cl!RaiseException()+0x6c
  cl!RaiseException()+0x6c
  cl!DllGetObjHandler()+0xdb94f
  cl!DllGetObjHandler()+0x1ac555
```

😭

REGRESSION: mathgl:x64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: mathgl:x64-windows-static failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: mathgl:x86-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
-- Performing post-build validation
The following files contain an absolute path ('D:\packages\mathgl_x64-windows', 'D:\installed', 'D:\buildtrees\mathgl'):
    D:\packages\mathgl_x64-windows\include\mgl2\config.h
There should be no absolute paths in the installed package, only relative ones.

Found 1 post-build check problem(s). To submit these ports to curated catalogs, please first correct the portfile:
    C:\a\2\s\ports\mathgl\portfile.cmake
-- Performing post-build validation done
error: building mathgl:x64-windows failed with: POST_BUILD_CHECKS_FAILED
```

@autoantwort

REGRESSION: qtwebengine:arm64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add qtwebengine:arm64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.

```
Building qtwebengine[core,default-features,geolocation,spellchecker,webchannel]:arm64-windows...
-- Setting up python virtual environmnent...
-- Installing python packages: html5lib
-- Setting up python virtual environmnent...finished.
CMake Warning at ports/qtwebengine/portfile.cmake:83 (message):
  Buildtree path 'D:/buildtrees/qtwebengine' is too long.

  Consider passing --x-buildtrees-root=<shortpath> to vcpkg!

  Trying to use 'D:/buildtrees/qtwebengine/../tmp'
Call Stack (most recent call first):
  scripts/ports.cmake:147 (include)

-- Using cached qtwebengine-everywhere-src-6.3.1.tar.xz.
-- Cleaning sources at D:/buildtrees/tmp/src/here-src-6-367c283e50.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source D:/downloads/qtwebengine-everywhere-src-6.3.1.tar.xz
-- Using source at D:/buildtrees/tmp/src/here-src-6-367c283e50.clean
-- Found external ninja('1.10.2').
-- Configuring arm64-windows-dbg
-- Configuring arm64-windows-rel
CMake Warning at D:/installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_configure.cmake:302 (message):
  The following variables are not used in CMakeLists.txt:

      FEATURE_webengine_geolocation
      FEATURE_webengine_spellchecker
      FEATURE_webengine_v8_snapshot_support
      FEATURE_webengine_webchannel
      FEATURE_webengine_webrtc

  Please recheck them and remove the unnecessary options from the
  `vcpkg_cmake_configure` call.

  If these options should still be passed for whatever reason, please use the
  `MAYBE_UNUSED_VARIABLES` argument.
Call Stack (most recent call first):
  D:/installed/arm64-windows/share/qtbase/qt_install_submodule.cmake:108 (vcpkg_cmake_configure)
  ports/qtwebengine/portfile.cmake:102 (qt_cmake_configure)
  scripts/ports.cmake:147 (include)

-- Building arm64-windows-dbg
-- Building arm64-windows-rel
-- Installing: D:/packages/qtwebengine_arm64-windows/share/qtwebengine/copyright
-- Performing post-build validation
The folder /include is empty or not present. This indicates the library was not correctly installed.
There should be no empty directories in D:\packages\qtwebengine_arm64-windows
The following empty directories were found:
```

This was previously blocked by qtwebengine:x64-windows=fail. Will investigate...

* Remove attempt to install extra stuff for scale set agents.

* Turn on boot diagnostics.

* Update pools.

* Back out most Linux changes.

* Analysis of https://dev.azure.com/vcpkg/public/_build/results?buildId=76482&view=results

REGRESSION: graphicsmagick:x86-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

Added this skip too.

REGRESSION: gazebo:x64-windows failed with BUILD_FAILED. If expected, add gazebo:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
C:\PROGRA~1\MICROS~1\2022\ENTERP~1\VC\Tools\MSVC\1433~1.316\bin\Hostx64\x64\cl.exe   /TP -DBOOST_ALL_DYN_LINK -DBOOST_ALL_NO_LIB -DBOOST_TEST_DYN_LINK -DBUILDING_DLL -DBUILDING_DLL_GZ_PHYSICS -DFREEIMAGE_LIB -DH5_BUILT_AS_DYNAMIC_LIB -DIGN_PROFILER_ENABLE=0 -DLIBBULLET_VERSION=3.21 -DLIBBULLET_VERSION_GT_282 -DNOMINMAX -DNOUSER -DPROTOBUF_USE_DLLS -DTINYXML2_DEBUG -DTINYXML2_IMPORT -DWIN32_LEAN_AND_MEAN -D_USE_MATH_DEFINES -DdIDEDOUBLE -Dgazebo_physics_EXPORTS -ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\test\gtest\include -ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean -ID:\buildtrees\gazebo\x64-windows-dbg -ID:\installed\x64-windows\debug\..\include\libusb-1.0 -external:ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\deps\opende\include -external:ID:\installed\x64-windows\include -external:ID:\installed\x64-windows\debug\..\include -external:ID:\installed\x64-windows\include\ignition\msgs5 -external:ID:\installed\x64-windows\include\ignition\math6 -external:ID:\installed\x64-windows\include\ignition\transport8 -external:ID:\installed\x64-windows\include\ignition\common3 -external:ID:\installed\x64-windows\include\ignition\fuel_tools4 -external:ID:\installed\x64-windows\include\OGRE -external:ID:\installed\x64-windows\include\sdformat-9.8 -external:ID:\installed\x64-windows\include\bullet -external:ID:\installed\x64-windows\include\sdformat-9.8\sdf\.. -external:ID:\installed\x64-windows\include\ignition\cmake2 -external:ID:\installed\x64-windows\include\eigen3 -external:ID:\installed\x64-windows\share\urdfdom_headers\..\..\include -external:ID:\installed\x64-windows\include\urdfdom -external:ID:\installed\x64-windows\include\urdfdom_headers -external:W0 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP        /wd4005 /wd4068 /wd4244 /wd4251 /wd4267 /wd4275 /wd4996 /EHsc /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1    /EHsc -I"D:/installed/x64-windows/include" /Zc:__cplusplus /permissive- /Zc:strictStrings- /Zc:externC- -std:c++17 /showIncludes /Fogazebo\physics\CMakeFiles\gazebo_physics.dir\dart\DARTBoxShape.cc.obj /Fdgazebo\physics\CMakeFiles\gazebo_physics.dir\ /FS -c D:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\gazebo\physics\dart\DARTBoxShape.cc
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2275: 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2923: 'std::shared_ptr': 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): note: see declaration of 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2275: 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2923: 'std::shared_ptr': 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): note: see declaration of 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
```

PASSING, REMOVE FROM FAIL LIST: luajit:arm64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).

This was https://github.com/microsoft/vcpkg/pull/26360

REGRESSION: qtwebengine:arm64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add qtwebengine:arm64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This was previously blocked by x64-windows being skipped, so I'm just adding it to the baseline for now.
2022-08-17 16:44:51 -07:00
Billy O'Neal
338c94d342
Move Azure Compute to eastasia (#24821)
* Move to eastasia, and use smaller machine to mint images.

* Update pools.

* Switch storage accounts to eastasia
2022-05-20 13:42:58 -07:00
Billy O'Neal
c26477297c
Update pwsh to 7.1.14, separate image minting from vmss minting, and update VMs. (#20064)
* Update Powershell-Core to 7.1.4.

* Add deployment of .NET 4.7.2 targeting pack, extracted from https://github.com/microsoft/vcpkg/pull/19320

* Separate image minting from vmss minting.

* Update pools.

* Fix image minting typo on Windows.
2021-09-11 13:43:19 -07:00
Billy O'Neal
f698b0e8be
Change VM size to Standard_D32ds_v4. For some reason they are cheaper than D16 right now??? (#18953)
Remove no longer used "unstable" scaffolding.
Use ephemeral OS disks for better latency.
Add ability to change whether temp disks are used or explicitly provisioned disks are used; explicitly provisioned disks turned out to be more expensive than expected before.
Disable Windows Updates in the scale set because the machines are very short lived; working on a mechanism of patching the images instead.
2021-07-15 13:47:58 -07:00
Billy O'Neal
957d2024b8
Update to Ubuntu 20.04. (#18892)
* Update Ubuntu to 20.04.

* [tfhe] Disable -Wall -Werror

* [sophus] Disable -Werror.

* Fix typo in deployment of CUDA.

* Add libxxf86vm-dev for opensubdiv.

* [opensubdiv] Fix OpenCL search on Ubuntu 20.04, and document libxxf86vm-dev dependency

* [msgpack11] Disable -Werror.

* [qtwayland, qt5wayland] Add system library notice.

* Add python-is-python3 to unbreak tensorflow, v8, and others

* [graphqlparser] Remove directive rejected by Bison 3.5.1

* [ogdf] Mark as failing in ci.baseline.txt. Issue filed https://github.com/microsoft/vcpkg/issues/18936

* [bde] Mark failing in ci.baseline.txt, issue filed https://github.com/microsoft/vcpkg/issues/18937

* [soem] Patch out -Werror

* Update pool.
2021-07-14 12:46:05 -07:00
Billy O'Neal
2ed44b4546
Update VMs and pick up VS2019 16.10 (#18233)
* Cherry-pick https://github.com/microsoft/vcpkg/pull/15598

* Hook deploy-inteloneapi into create-vmss.ps1.

* Add script to resolve https://github.com/microsoft/vcpkg/issues/17521

* Move tls settings deployment to the front and respond to script triggering a reboot.

* Go back to provisioning an extra disk to workaround https://github.com/microsoft/vcpkg/issues/18379

* Disallow public access to blob storage and require TLS 1.2.

* Update Pools.

* Update tool to 2021-06-19

* [simage] Skip simage on uwp platforms as it appears broken by 16.10.

* [tensorflow-cc] Skip because changes in our MacOS hardware broke the port.
2021-06-22 19:16:04 -07:00
Billy O'Neal
229f537c94
[opentracing] Update VMs 2020-12 (#15151)
* Add meson from https://github.com/microsoft/vcpkg/pull/12860/

* Add autoconf-archive from https://github.com/microsoft/vcpkg/pull/13081/

* Add kf5windowsystem libs from https://github.com/microsoft/vcpkg/pull/13467/

* Open the FTP and SFTP ports from https://github.com/microsoft/vcpkg/pull/14412/

* Add libxcb-util0-dev from https://github.com/microsoft/vcpkg/pull/14678/

* Add libasound2-dev from https://github.com/microsoft/vcpkg/pull/14774

* Remove no longer necessary apt-mark calls.

* Update nasm on Linux.

* Fix longstanding bug where Storage was publicly accessible and change to generate SAS token rather than File Share

* Delete no longer necessary azure storage firewall rules.

* Install the newer Windows SDK with the VS installer instead of manually.

* Install the VS2015 and VS2017 compilers.

* Update Powershell-Core to 7.1.0.

* Update source of WDK.

* Update pools.

* [opentracing] Repair arm64-windows failures caused by mojibake in `expected.hpp` and errors in opentracing-cpp's lint for arm64 where it thinks exceptions are disabled when they are enabled.

Fixes:
C:\Dev\vcpkg\buildtrees\opentracing\src\b67575dab0-0250653c81.clean\3rd_party\include\opentracing/expected/expected.hpp(1): warning C4828: The file contains a character starting at offset 0x4a77 that is illegal in the current source character set (codepage 65001).

Fixes:
D:\buildtrees\opentracing\src\b67575dab0-0250653c81.clean\include\opentracing/tracer.h:223:5: error: cannot use 'try' with exceptions disabled [clang-diagnostic-error]
    try {
    ^

* [mmloader] Patch out overrides of CMAKE_C_FLAGS and CMAKE_CXX_FLAGS that inserted /WX.

Note that this port sets /GS-, possibly because it may be for authoring shellcode.

* Bump storage API version to 2020-04-08.
2020-12-22 14:52:58 -08:00
Billy O'Neal
8054263f15
[skia, ompl, capnproto] Update VMs for 2020-08-12 (including VS 16.7) (#12884)
Co-authored-by: Nicole Mazzuca <mazzucan@outlook.com>
2020-08-27 12:27:39 -07:00
Billy O'Neal
2bf1f30deb
[vcpkg] Use SSH keys instead of password authentication when minting Linux scale sets (#11999) 2020-06-21 19:13:13 -07:00
Billy O'Neal
55d6bd1b7d
[vcpkg] [cudnn] [msmpi] [openmpi] Update VMSS (#11365)
Co-authored-by: @JackBoosY 
Co-authored-by: Stefano Sinigardi <stesinigardi@hotmail.com>
Co-authored-by: Voskrese <10104740+Voskrese@users.noreply.github.com>
2020-06-11 11:13:09 -07:00
Billy O'Neal
6d36e2a86b
[vcpkg] Use a crypto RNG to generate admin passwords (#11629)
* [vcpkg] Use a crypto RNG to generate admin passwords

* Apply code review comments from Stephan and Casey.

* Extract functions into create-vmss-helpers.psm1.

* Put Wait-Shutdown back.
2020-05-29 22:46:05 -07:00
Billy O'Neal
6920b49047
[vcpkg] [llvm] [mpir] Bump Linux VM memory size and do all operations on the temporary disk. (#11174)
* [vcpkg] Bump Linux VM memory size and do all operations on the temporary disk.

* [llvm] disable optional dependencies to libxml2 and zlib

* [vcpkg] Reduce quotas on the share to something reasonable.

* [mpir] Skip on Linux because it conflicts with libgmp.

Co-authored-by: yurybura <yurybura@gmail.com>
2020-05-10 15:17:27 -07:00
Billy Robert O'Neal III
b8755728ab [vcpkg] Onboard Linux to VMSS, open 'git' port, and switch back to Azure Spot
* Adds scripts to generate scale sets for testing Linux.
    * Note workaround for https://github.com/microsoft/azure-pipelines-agent/pull/2929
* Switches Windows validation to 'Spot' VMs.
* Opens the git port 9418.
* Removes provisioning of the no longer used 'logs' file share.
* Changes Azure region to 'westus2', which is cheaper.
* Adds +x to all the scripts in scripts/azure-pipelines.
* Use 'xml-results' for all platforms instead of 'raw xml results' on Windows.
2020-04-30 21:51:31 -07:00