Commit Graph

75 Commits

Author SHA1 Message Date
Billy O'Neal
7ba0ba7334
Change Azure SKU to Standard_D32ads_v5 and move to West US 3 (#36666) 2024-02-12 15:07:57 -08:00
Billy O'Neal
cf20e29526
Update macOS for January 2024 (#35617)
macOS 14.2.1 on amd64, 14.3 on arm64
XCode Console Tools 15.1

Other changes:

macOS install is done from the installers rather than from the recovery partition
readme.md is now written for a vcpkg-team audience
vagrant is removed as a dependency so that discussion of how to persist the VMs between machines could be included
arm64 is added using https://github.com/s-u/macosvm
2024-01-25 10:50:32 -08:00
Billy O'Neal
91b17dd72a
Patch Tuesday for January 2023 (#36143) 2024-01-16 19:51:14 -08:00
Billy O'Neal
ab00a636c7
Update Windows Fleet for December 2023 Patch Tuesday (#35640) 2023-12-18 10:27:45 -08:00
Billy O'Neal
24d884e1b8
Update Android base image to 2023-12-05. (#35526)
Also tells Docker to invalidate caches so that our apt-get update attempt actually does something.
2023-12-06 12:17:01 -08:00
Billy O'Neal
ba823b1e1f
Patch Tuesday 2023-10-11 (except macOS) (#34431) 2023-10-13 14:57:51 -07:00
Billy O'Neal
b5d4a46b9a
Patch Tuesday September 2023 + arm-uwp=>arm64-uwp (#33759)
Co-authored-by: Leander Schulten <Leander.Schulten@rwth-aachen.de>
2023-09-15 15:01:12 -07:00
Billy O'Neal
d99b6930b9
Update macOS to 13.5 and xcode 14.3.1. (#33722) 2023-09-14 14:02:46 -07:00
Billy O'Neal
5fa92f467e
Patch Tuesday August 2023. (#33088)
* Update PowerShell to 7.3.6

* Add asan install as requested by @Neumann-A

* Use 1ES for Android docker hosts.

* Update pools.

* Update patch tuesday checklist to reflect 1ES.

* [openslide] Fix missing header.

* [libxt] Add missing header.
2023-08-14 13:58:43 -07:00
Billy O'Neal
ad46340bfc
Use 1ES to provision Linux agents. (#32937) 2023-08-07 21:22:34 -07:00
Billy O'Neal
36577b338f
Patch Tuesday July 2023 (#32537) 2023-07-14 17:05:19 -07:00
Billy O'Neal
f6a5d4e8eb
Patch Tuesday June 2023 + Ubuntu 22.04 (#32014) 2023-06-20 12:12:03 -07:00
Billy O'Neal
7f52deab66
Update VMs for Patch Tuesday May 2023 (#31358)
* Update PowerShell Core to not vulnerable versions.

* Fix Android create-image relative paths.

* Update pools and add docs.

* Get android create-vmss to setup managed identity.
2023-05-14 01:54:42 -07:00
dan-shaw
5787cfa699
[vcpkg] Android CI support (#29406)
* initial

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix
2023-04-24 21:22:59 -07:00
Billy O'Neal
cb2167bd7f
Patch Tuesday April 2023 (#30818) 2023-04-14 02:17:56 -07:00
Billy O'Neal
6146b482ec
VM Updates for Patch Tuesday, March 2023 (#30223) 2023-03-21 12:27:26 -07:00
Kai Pastor
328bd79eb8
Add parameter to limit CI to selected triplets (#29738)
* Add parameter to limit CI to selected triplets

Pull requests which analyze the situation in a particular triplet,
arch or platform may temporarily limit the triplets run in CI by
overriding the default value of CI parameter 'tripletPattern'.

* Add special chars for start and end

* Proof of concept

* Revert "Proof of concept"

This reverts commit 326bd2632b.
2023-02-24 11:32:02 -08:00
Billy O'Neal
1cc952582b
Patch Tuesday Feb 2023 (#29669) 2023-02-15 23:11:47 -08:00
Billy O'Neal
d73cbb6a27
VM update January 2023 (#28895)
* Add libxcb-cursor-dev from https://github.com/microsoft/vcpkg/pull/28536/

* Update Pools.
2023-01-13 13:56:26 -08:00
Billy O'Neal
6db74157ac
Update VMs for December 2022 (#28419)
* Update pwsh to 7.3.1.

* Update pools.
2022-12-19 11:17:57 -08:00
Billy O'Neal
62d01b70df
Update VMs for November Patch Tuesday (#27718)
* Cherry pick new KDF5 dependencies from #27454

* Update PowerShell Core to 7.3.0

* PASSING, REMOVE FROM FAIL LIST: cgicc:arm-uwp (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).

PASSING, REMOVE FROM FAIL LIST: cgicc:x64-uwp (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: soci:arm-uwp (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: soci:x64-uwp (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: spirv-tools:arm-uwp (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: spirv-tools:x64-uwp (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).

* Add a Windows 10 SDK that CMake requires for uwp projects.

* Update pools.

* [hunspell] Remove deprecated minimal rebuild switch.

Fixes build break first detected in: https://github.com/microsoft/vcpkg/pull/27718

Submitted upstream as: https://github.com/hunspell/hunspell/pull/890

* [rttr] Add upstream patch for missing #include<string>

Resolves build failure detected in https://github.com/microsoft/vcpkg/pull/27718

* [chakracore] Avoid race condition on generated headers.
2022-11-25 02:00:21 -08:00
Billy O'Neal
6d2d1dc843
VM Update October 2022 (#27212)
* Add libtool-bin as requested in https://github.com/microsoft/vcpkg/pull/26923

* Update pools.
2022-10-13 20:23:14 -07:00
Billy O'Neal
86cab7438f
VM Update September 2022 (#26895)
* Revert Linux to Ubuntu 20.04.

* Cherry-pick intel-mkl from https://github.com/microsoft/vcpkg/pull/15233/

* Update pwsh to 7.2.6.

* Temporarily remove ci.baseline.txt skips for ICEs.

* Fix ICU version for Ubuntu version change.

* Update pools.

* Merge apt installs and fix WSL CUDA bindings.

* Fix WSL detection.

* Remove WSL specific nvidia because it misses nccl and cudnn.

* Add python-yaml for duktape.

* [shogun] Force use of OpenBLAS as the Lapack implementation.

Shogun declares a dependency on blas, which we have set to openblas. Installing Intel MKL on the VMs caused build of Shogun to fail because it tried to use MKL and that build is broken.

* Put colmap, graphicsmagick, and mfl skips back, as they are clearly not fixed.

* PASSING, REMOVE FROM FAIL LIST: jemalloc:x64-osx (/Users/vagrant/Data/work/2/s/scripts/azure-pipelines/../ci.baseline.txt).

* Update pool.

* Revert the intel-mkl changes until the BLAS/LAPACK vendoring problem is fixed.

* Update pool.

* Explicitly add bison, needed by gettext.

* Update pools
2022-09-26 12:36:42 -07:00
Billy O'Neal
4217fee510
Update Windows VMs for Patch Tuesday August 2022 (#26283)
* Install haskell-stack with apt instead of piping a sh.

* Update pools.

* Linux is exploding for some reason. I just found out it is getting created with a 30GB disk I'm guessing may be involved.

Did not wait for x64_windows_static_md to finish because enough was already on the floor.

PASSING, REMOVE FROM FAIL LIST: angle:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:arm64-windows (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-windows-static (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x86-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: rsocket:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: rsocket:x64-windows-static (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
REGRESSION: gazebo:x64-windows failed with BUILD_FAILED. If expected, add gazebo:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2275: 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2923: 'std::shared_ptr': 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): note: see declaration of 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
```

We have been seeing intermittent failures in gazebo:x64-windows and I think this just repro'd that...

REGRESSION: graphicsmagick:x64-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: graphicsmagick:x64-windows-static failed with BUILD_FAILED. If expected, add graphicsmagick:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: graphicsmagick:x86-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick/nt_base.h(95): warning C4005: 'HAVE_FT2BUILD_H': macro redefinition
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick/magick_config.h(123): note: see previous definition of 'HAVE_FT2BUILD_H'
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick\effect.c(689) : fatal error C1001: Internal compiler error.
(compiler file 'D:\a\_work\1\s\src\vctools\Compiler\Utc\src\p2\main.c', line 224)
 To work around this problem, try simplifying or changing the program near the locations listed above.
If possible please provide a repro here: https://developercommunity.visualstudio.com
Please choose the Technical Support command on the Visual C++
 Help menu, or open the Technical Support help file for more information
  cl!RaiseException()+0x6c
  cl!RaiseException()+0x6c
  cl!DllGetObjHandler()+0xdb94f
  cl!DllGetObjHandler()+0x1ac555
```

😭

REGRESSION: mathgl:x64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: mathgl:x64-windows-static failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: mathgl:x86-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
-- Performing post-build validation
The following files contain an absolute path ('D:\packages\mathgl_x64-windows', 'D:\installed', 'D:\buildtrees\mathgl'):
    D:\packages\mathgl_x64-windows\include\mgl2\config.h
There should be no absolute paths in the installed package, only relative ones.

Found 1 post-build check problem(s). To submit these ports to curated catalogs, please first correct the portfile:
    C:\a\2\s\ports\mathgl\portfile.cmake
-- Performing post-build validation done
error: building mathgl:x64-windows failed with: POST_BUILD_CHECKS_FAILED
```

@autoantwort

REGRESSION: qtwebengine:arm64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add qtwebengine:arm64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.

```
Building qtwebengine[core,default-features,geolocation,spellchecker,webchannel]:arm64-windows...
-- Setting up python virtual environmnent...
-- Installing python packages: html5lib
-- Setting up python virtual environmnent...finished.
CMake Warning at ports/qtwebengine/portfile.cmake:83 (message):
  Buildtree path 'D:/buildtrees/qtwebengine' is too long.

  Consider passing --x-buildtrees-root=<shortpath> to vcpkg!

  Trying to use 'D:/buildtrees/qtwebengine/../tmp'
Call Stack (most recent call first):
  scripts/ports.cmake:147 (include)

-- Using cached qtwebengine-everywhere-src-6.3.1.tar.xz.
-- Cleaning sources at D:/buildtrees/tmp/src/here-src-6-367c283e50.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source D:/downloads/qtwebengine-everywhere-src-6.3.1.tar.xz
-- Using source at D:/buildtrees/tmp/src/here-src-6-367c283e50.clean
-- Found external ninja('1.10.2').
-- Configuring arm64-windows-dbg
-- Configuring arm64-windows-rel
CMake Warning at D:/installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_configure.cmake:302 (message):
  The following variables are not used in CMakeLists.txt:

      FEATURE_webengine_geolocation
      FEATURE_webengine_spellchecker
      FEATURE_webengine_v8_snapshot_support
      FEATURE_webengine_webchannel
      FEATURE_webengine_webrtc

  Please recheck them and remove the unnecessary options from the
  `vcpkg_cmake_configure` call.

  If these options should still be passed for whatever reason, please use the
  `MAYBE_UNUSED_VARIABLES` argument.
Call Stack (most recent call first):
  D:/installed/arm64-windows/share/qtbase/qt_install_submodule.cmake:108 (vcpkg_cmake_configure)
  ports/qtwebengine/portfile.cmake:102 (qt_cmake_configure)
  scripts/ports.cmake:147 (include)

-- Building arm64-windows-dbg
-- Building arm64-windows-rel
-- Installing: D:/packages/qtwebengine_arm64-windows/share/qtwebengine/copyright
-- Performing post-build validation
The folder /include is empty or not present. This indicates the library was not correctly installed.
There should be no empty directories in D:\packages\qtwebengine_arm64-windows
The following empty directories were found:
```

This was previously blocked by qtwebengine:x64-windows=fail. Will investigate...

* Remove attempt to install extra stuff for scale set agents.

* Turn on boot diagnostics.

* Update pools.

* Back out most Linux changes.

* Analysis of https://dev.azure.com/vcpkg/public/_build/results?buildId=76482&view=results

REGRESSION: graphicsmagick:x86-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

Added this skip too.

REGRESSION: gazebo:x64-windows failed with BUILD_FAILED. If expected, add gazebo:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
C:\PROGRA~1\MICROS~1\2022\ENTERP~1\VC\Tools\MSVC\1433~1.316\bin\Hostx64\x64\cl.exe   /TP -DBOOST_ALL_DYN_LINK -DBOOST_ALL_NO_LIB -DBOOST_TEST_DYN_LINK -DBUILDING_DLL -DBUILDING_DLL_GZ_PHYSICS -DFREEIMAGE_LIB -DH5_BUILT_AS_DYNAMIC_LIB -DIGN_PROFILER_ENABLE=0 -DLIBBULLET_VERSION=3.21 -DLIBBULLET_VERSION_GT_282 -DNOMINMAX -DNOUSER -DPROTOBUF_USE_DLLS -DTINYXML2_DEBUG -DTINYXML2_IMPORT -DWIN32_LEAN_AND_MEAN -D_USE_MATH_DEFINES -DdIDEDOUBLE -Dgazebo_physics_EXPORTS -ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\test\gtest\include -ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean -ID:\buildtrees\gazebo\x64-windows-dbg -ID:\installed\x64-windows\debug\..\include\libusb-1.0 -external:ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\deps\opende\include -external:ID:\installed\x64-windows\include -external:ID:\installed\x64-windows\debug\..\include -external:ID:\installed\x64-windows\include\ignition\msgs5 -external:ID:\installed\x64-windows\include\ignition\math6 -external:ID:\installed\x64-windows\include\ignition\transport8 -external:ID:\installed\x64-windows\include\ignition\common3 -external:ID:\installed\x64-windows\include\ignition\fuel_tools4 -external:ID:\installed\x64-windows\include\OGRE -external:ID:\installed\x64-windows\include\sdformat-9.8 -external:ID:\installed\x64-windows\include\bullet -external:ID:\installed\x64-windows\include\sdformat-9.8\sdf\.. -external:ID:\installed\x64-windows\include\ignition\cmake2 -external:ID:\installed\x64-windows\include\eigen3 -external:ID:\installed\x64-windows\share\urdfdom_headers\..\..\include -external:ID:\installed\x64-windows\include\urdfdom -external:ID:\installed\x64-windows\include\urdfdom_headers -external:W0 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP        /wd4005 /wd4068 /wd4244 /wd4251 /wd4267 /wd4275 /wd4996 /EHsc /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1    /EHsc -I"D:/installed/x64-windows/include" /Zc:__cplusplus /permissive- /Zc:strictStrings- /Zc:externC- -std:c++17 /showIncludes /Fogazebo\physics\CMakeFiles\gazebo_physics.dir\dart\DARTBoxShape.cc.obj /Fdgazebo\physics\CMakeFiles\gazebo_physics.dir\ /FS -c D:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\gazebo\physics\dart\DARTBoxShape.cc
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2275: 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2923: 'std::shared_ptr': 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): note: see declaration of 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2275: 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2923: 'std::shared_ptr': 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): note: see declaration of 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
```

PASSING, REMOVE FROM FAIL LIST: luajit:arm64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).

This was https://github.com/microsoft/vcpkg/pull/26360

REGRESSION: qtwebengine:arm64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add qtwebengine:arm64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This was previously blocked by x64-windows being skipped, so I'm just adding it to the baseline for now.
2022-08-17 16:44:51 -07:00
Billy O'Neal
68b7fec22e
Update VMs for July 2022 Patch Tuesday, and to Ubuntu 22.04. (#25785)
* Update Ubuntu to 22.04.

* Add libxtst-dev from https://github.com/microsoft/vcpkg/pull/24934

* Add standard prereqs.

* Update some package references for Ubuntu 22.04.

* [cuda] Update to search in /usr/bin as used on Ubuntu 22.04.

* [cudnn] Add where ubuntu 22.04 puts it.

* Add update-nvidia-cudnn calls.

* Update powershell to 7.2.5

* Re-add NCCL installation from CUDA official sources.

* Update pools.
2022-07-15 21:11:16 -07:00
Billy O'Neal
9ba75e6e17
Update VMs for June Patch Tuesday (#25260)
* Update PowerShell to 7.2.4.

* Update Windows SDKs.

* Add libdbus-1-dev. Resolves https://github.com/microsoft/vcpkg/issues/25094

* PASSING, REMOVE FROM FAIL LIST: chmlib:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).

PASSING, REMOVE FROM FAIL LIST: chmlib:x64-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: gl3w:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: glew:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: laszip:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: laszip:x64-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: opengl:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: rttr:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: rttr:x64-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: seal:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: seal:x64-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).

:)

REGRESSION: ftgl:arm-uwp failed with BUILD_FAILED. If expected, add ftgl:arm-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: mathgl:arm-uwp failed with BUILD_FAILED. If expected, add mathgl:arm-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: opencsg:arm-uwp failed with BUILD_FAILED. If expected, add opencsg:arm-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

These were all previously blocked by one of the above passing-remove-from-fail-lists.

REGRESSION: ms-quic:arm-uwp failed with BUILD_FAILED. If expected, add ms-quic:arm-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: ms-quic:x64-uwp failed with BUILD_FAILED. If expected, add ms-quic:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: ms-quic:x64-windows failed with BUILD_FAILED. If expected, add ms-quic:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: ms-quic:x64-windows-static-md failed with BUILD_FAILED. If expected, add ms-quic:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: ms-quic:x86-windows failed with BUILD_FAILED. If expected, add ms-quic:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This port hates the current Windows SDK, and it is a leaf port, so I'm marking it `=fail`:

```
D:\buildtrees\ms-quic\src\v1.2.0-19ce393c24.clean\src\inc\quic_platform_winuser.h(564): error C2375: 'NtQueryTimerResolution': redefinition; different linkage
C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um\winternl.h(713): note: see declaration of 'NtQueryTimerResolution'
D:\buildtrees\ms-quic\src\v1.2.0-19ce393c24.clean\src\inc\quic_platform_winuser.h(765): error C2011: '_THREAD_NAME_INFORMATION': 'struct' type redefinition
C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um\winternl.h(352): note: see declaration of '_THREAD_NAME_INFORMATION'
D:\buildtrees\ms-quic\src\v1.2.0-19ce393c24.clean\src\inc\quic_platform_winuser.h(773): error C2375: 'NtSetInformationThread': redefinition; different linkage
C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um\winternl.h(659): note: see declaration of 'NtSetInformationThread'
```

arm64-windows crashed. Previously opengl failed so we never got here. https://github.com/microsoft/vcpkg-tool/pull/599

```
Building opengl[core]:arm64-windows...
-- Fixing pkgconfig file: D:/packages/opengl_arm64-windows/lib/pkgconfig/glu.pc
-- Fixing pkgconfig file: D:/packages/opengl_arm64-windows/lib/pkgconfig/opengl.pc
-- Using cached msys-mingw-w64-i686-pkg-config-0.29.2-3-any.pkg.tar.zst.
-- Using cached msys-mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst.
-- Using msys root at D:/downloads/tools/msys2/9a1ec3f33446b195
-- Fixing pkgconfig file: D:/packages/opengl_arm64-windows/debug/lib/pkgconfig/glu.pc
-- Fixing pkgconfig file: D:/packages/opengl_arm64-windows/debug/lib/pkgconfig/opengl.pc
-- Performing post-build validation
Found more than 1 architecture in file D:\packages\opengl_arm64-windows\debug\lib\GlU32.Lib
##[error]vcpkg ci failed
At C:\a\2\s\scripts\azure-pipelines\test-modified-ports.ps1:173 char:5
+     throw "vcpkg ci failed"
+     ~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : OperationStopped: (vcpkg ci failed:String) [], RuntimeException
+ FullyQualifiedErrorId : vcpkg ci failed
##[error]PowerShell exited with code '1'.
```
2022-06-23 15:54:56 -07:00
Billy O'Neal
4c82b77b45
Add the ability to do full rebuilds with updated copies of vcpkg-tool. (#24611)
* Add the ability to do full rebuilds with updated copies of vcpkg-tool.

Resurrection of https://github.com/microsoft/vcpkg/pull/23753/

* Extract all scripts out of inline blocks.

* Replicate all changes to the "unstable" testing.

* Fix bootstrap :)

* Fix version files validation path.

* Malformed.

* Paths and permissions.

* Fixed file name typo

* Update scripts/azure-pipelines/bootstrap-from-source.sh

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2022-05-23 13:33:58 -07:00
Billy O'Neal
338c94d342
Move Azure Compute to eastasia (#24821)
* Move to eastasia, and use smaller machine to mint images.

* Update pools.

* Switch storage accounts to eastasia
2022-05-20 13:42:58 -07:00
Billy O'Neal
22b4033861
Update VMs for Patch Tuesday May 2022 (#24695)
* Install Ubuntu nasm package rather than building from source; 2.14 which is available in Ubuntu 20.04 is sufficient for intel-ipsec.

* Add cmake and ninja.

* Update CUDA signing key.

* Update pwsh to 7.2.3

* Remove clean downloads step obsoleted since we ripped out pacman.

* Note that haskell-stack is for bond.

* Cherry pick from https://github.com/microsoft/vcpkg/pull/23569 : add cuda-nvml-dev-11-6

* Update pools.

* Baseline updates

PASSING, REMOVE FROM FAIL LIST: mesa:x64-windows-static-md (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).

:)

```
REGRESSION: cppgraphqlgen:arm64-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:arm64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows-static failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x64-windows-static-md failed with BUILD_FAILED. If expected, add cppgraphqlgen:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: cppgraphqlgen:x86-windows failed with BUILD_FAILED. If expected, add cppgraphqlgen:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```

This is a compiler behavior change or bug in VS 2022 16.2:

```
C:\PROGRA~1\MICROS~1\2022\ENTERP~1\VC\Tools\MSVC\1432~1.313\bin\Hostx64\x86\cl.exe   /TP -DGRAPHQL_DLLEXPORTS -DIMPL_GRAPHQLSERVICE_DLL -Dgraphqlservice_EXPORTS -ID:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\..\include -ID:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\..\PEGTL\include -external:ID:\installed\x86-windows\include -external:W0 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP  /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1  -MDd /W4 /WX /permissive- -std:c++20 /showIncludes /Fosrc\CMakeFiles\graphqlservice.dir\GraphQLService.cpp.obj /Fdsrc\CMakeFiles\graphqlservice.dir\ /FS -c D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\GraphQLService.cpp
cl : Command line warning D9025 : overriding '/W3' with '/W4'
D:\installed\x86-windows\include\tao\pegtl/demangle.hpp(147): error C2338: static_assert failed: 'internal::dependent_true< T > && ( begin != std::string_view::npos )'
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\include\graphqlservice/internal/SyntaxTree.h(120): note: see reference to function template instantiation 'std::string_view tao::graphqlpeg::demangle<graphql::peg::variable_value>(void) noexcept' being compiled
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\include\graphqlservice/internal/SyntaxTree.h(53): note: see reference to function template instantiation 'std::string_view graphql::peg::ast_node::type_name<graphql::peg::variable_value>(void) noexcept' being compiled
D:\buildtrees\cppgraphqlgen\src\v4.3.1-9d04ffd723.clean\src\GraphQLService.cpp(310): note: see reference to function template instantiation 'bool graphql::peg::ast_node::is_type<graphql::peg::variable_value>(void) noexcept const' being compiled
ninja: build stopped: subcommand failed.
```

This is a compiler behavior change in 17.2. @wravery do you have comments?

```
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```

qtwebengine doesn't report logs, but I'm pretty sure it's this ICE: https://developercommunity.visualstudio.com/t/Visual-Studio-2022-v1720-reports-fata/10039296 @Neumann-A

```
REGRESSION: rsocket:x64-windows failed with BUILD_FAILED. If expected, add rsocket:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: rsocket:x64-windows-static failed with BUILD_FAILED. If expected, add rsocket:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: rsocket:x64-windows-static-md failed with BUILD_FAILED. If expected, add rsocket:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
```

ICE :( https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems/edit/1490389

Other changes:
* Removed qt5-webengine skips because they are now skipped by a "supports" clause.
* Removed ctp skips because asset caching exists.

* [stxxl] Guard definition of log2 for current MSVCs.
2022-05-17 13:03:13 -07:00
Billy O'Neal
aebb363eaa
Update VMs for April Patch Tuesday (#24131)
* Update VM base tools:

* Intel OneAPI updated
* Powershell Updated
* VS updated to install current Windows SDKs, and atlmfc for arm, and CMake/ninja

* Update pools.

* Remove passing-remove-from-fail list port.
2022-04-15 15:33:07 -07:00
Billy O'Neal
909334f5a3
Update VMs for VS2022, CUDA 11.6 (#23011)
* Update Linux VM CUDA to 11.6.

* Update Intel OneAPI

* Update Visual Studio to 2022, and remove deployment of v140 and v141 compilers.

* Update CUDA to 11.6 on Windows.

* Update pools.

* cix ci.baseline.txt

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
2022-03-16 16:29:50 -07:00
Billy O'Neal
e6509c997f
Update VMs for March 2022 Patch Tuesday (#23467)
* Add defenses (sleep) to Linux image creation, remove sudo from provisioning scripts, and use apt-get.

* Update pools.
2022-03-10 18:44:08 -08:00
Billy O'Neal
b77f20ec4c
Recombine osx and normal ci. (#23144)
* Some of the Windows runs have started to exceed 24 hours now as well as the MacOS ones as we have added more ports.
* We are no longer trying to constantly "psychic debug" why ports are failing; CI passes somewhat regularly now, and the failures we get seem to be much more deterministic.
* We're still budget constrained on Azure resources (and have been a bit over budget)

To that end, we're going back off nightly CI runs to Monday Wednesday Friday (like osx already is), so we no longer need separate bits for them.
2022-02-16 19:55:46 -08:00
nicole mazzuca
47c5701747
[macos ci] 2022-02-04 update (#22942)
* add moar ram (no PR)

* Provision autoconf-archive on osx (PR #22872)

* [mpfr] Support Linux and OSX (PR #22845)

* VM software versions

* fix minor schema bug

* update pool

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: Kai Pastor <dg0yt@darc.de>
Co-authored-by: JackBoosY <yuzaiyang@beyondsoft.com>
2022-02-09 13:25:57 -08:00
Billy O'Neal
39b17cf84c
Update VMs for Patch Tuesday and fix pwsh CVE. (#22494) 2022-01-13 16:12:55 -08:00
nicole mazzuca
5edd518dc3
[macos ci] update to 2022-01-03 (#22314)
* [macos ci] update to 2022-01-03

* update the instructions

* forgot to update azure-pipelines-osx.yml

* install parallels in Install-Prerequisites.ps1

* oops

* fix ace, libsoundio, and libpqxx

* tensorflow doesn't yet support macOS 12
2022-01-10 16:08:51 -08:00
Billy O'Neal
c6ffd8eeb2
[vcpkg] Update VMs for December 2021 Patch Tuesday (#22013)
* Add Microsoft.VisualStudio.ComponentGroup.UWP.VC.BuildTools to VS as requested by https://github.com/microsoft/vcpkg/issues/19554

* Cherry pick python changes from https://github.com/microsoft/vcpkg/pull/21912

* Update linux pool.

* Update windows pool.
2021-12-14 16:15:33 -08:00
Joachim Gehweiler
a9ff505c5b
[vcpkg baseline][tensorflow] setup python venv and update Linux VMs (#21912)
* Revert "incorporate changes from microsoft:master"

* Revert "Revert "incorporate changes from microsoft:master""

* preliminary approach to solve pip issue

* fix python venv

* fix macOS build command again

* bump version

* x-add-version

* Fix spelling of prerequisites.

* Add python3-venv

* Update linux pool.

Co-authored-by: Joachim Gehweiler <joachim@Joachims-iMac.local>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2021-12-13 17:46:40 -08:00
Billy O'Neal
d70768e374
[vcpkg] Change osx to run every 48 hours rather than every 24. (#21954)
* Change osx to run every 48 hours rather than every 24.

* Back out more former-tensorflow-separation yaml.
2021-12-13 12:54:16 -08:00
Billy O'Neal
e46b12b825
Update VMs for November Patch Tuesday (#21309)
* Apply VM updates for:

* https://github.com/microsoft/vcpkg/pull/13994
* https://github.com/microsoft/vcpkg/pull/20654
* https://github.com/microsoft/vcpkg/pull/20938

* Be more defensive in AzVmRunCommandWithRetries.

* Update Pools.
2021-11-10 14:07:00 -08:00
Billy O'Neal
699c8779f1
[vcpkg] Update VMs for October Patch Tuesday and Server 2022 (#20730)
* Patch tuesday and switch to Server 2022.

* Update pools.
2021-10-16 12:14:53 -07:00
nicole mazzuca
45315bc3aa
[macos ci] move out tensorflow (#20388)
* [macos ci] move out tensorflow

* rename ExtraPorts and CorePorts

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2021-09-29 13:15:25 -07:00
Billy O'Neal
09980780d8
[ci] Move formatting and doc checks into x86-windows (#20402)
* [ci] Move formatting and doc checks into x86-windows to avoid burning a whole job for that 1 run, don't run file lists generation in PRs.

* Deduplicate x86-windows and restore PR file lists, as requested/suggested by @dg0yt

* More quotes

Co-authored-by: nicole mazzuca <83086508+strega-nil-ms@users.noreply.github.com>

Co-authored-by: nicole mazzuca <83086508+strega-nil-ms@users.noreply.github.com>
2021-09-28 13:51:42 -07:00
Billy O'Neal
c26477297c
Update pwsh to 7.1.14, separate image minting from vmss minting, and update VMs. (#20064)
* Update Powershell-Core to 7.1.4.

* Add deployment of .NET 4.7.2 targeting pack, extracted from https://github.com/microsoft/vcpkg/pull/19320

* Separate image minting from vmss minting.

* Update pools.

* Fix image minting typo on Windows.
2021-09-11 13:43:19 -07:00
Billy O'Neal
e22f9d7d70
Update VMs for Patch Tuesday, 2021-08-11 (#19508)
* Deploy SDKs with full features first to ensure we get Debugging Tools for Windows.

* Update VMs for Patch Tuesday, 2021-08-11
2021-08-13 16:12:24 -07:00
nicole mazzuca
ff190a561a
[macos ci] update 2021-07-27 (#19207)
* update 2021-07-27

macOS -> 11.5.1, XCode -> 12.5.1

* update azure-pipelines

* update sha of macfuse

* change how macos-ci-base works

* fix build errors

* fix itpp:linux

* more fixes

* remove tab

* allow version changes in all the remove/rename-version patches

* fix libunistring for real

* robert CR
2021-08-04 13:27:40 -07:00
Billy O'Neal
f698b0e8be
Change VM size to Standard_D32ds_v4. For some reason they are cheaper than D16 right now??? (#18953)
Remove no longer used "unstable" scaffolding.
Use ephemeral OS disks for better latency.
Add ability to change whether temp disks are used or explicitly provisioned disks are used; explicitly provisioned disks turned out to be more expensive than expected before.
Disable Windows Updates in the scale set because the machines are very short lived; working on a mechanism of patching the images instead.
2021-07-15 13:47:58 -07:00
Billy O'Neal
957d2024b8
Update to Ubuntu 20.04. (#18892)
* Update Ubuntu to 20.04.

* [tfhe] Disable -Wall -Werror

* [sophus] Disable -Werror.

* Fix typo in deployment of CUDA.

* Add libxxf86vm-dev for opensubdiv.

* [opensubdiv] Fix OpenCL search on Ubuntu 20.04, and document libxxf86vm-dev dependency

* [msgpack11] Disable -Werror.

* [qtwayland, qt5wayland] Add system library notice.

* Add python-is-python3 to unbreak tensorflow, v8, and others

* [graphqlparser] Remove directive rejected by Bison 3.5.1

* [ogdf] Mark as failing in ci.baseline.txt. Issue filed https://github.com/microsoft/vcpkg/issues/18936

* [bde] Mark failing in ci.baseline.txt, issue filed https://github.com/microsoft/vcpkg/issues/18937

* [soem] Patch out -Werror

* Update pool.
2021-07-14 12:46:05 -07:00
Billy O'Neal
7d169b467c
Update pool to get print spooler CVE fix. (#18891) 2021-07-10 16:47:55 -07:00
Billy O'Neal
2ed44b4546
Update VMs and pick up VS2019 16.10 (#18233)
* Cherry-pick https://github.com/microsoft/vcpkg/pull/15598

* Hook deploy-inteloneapi into create-vmss.ps1.

* Add script to resolve https://github.com/microsoft/vcpkg/issues/17521

* Move tls settings deployment to the front and respond to script triggering a reboot.

* Go back to provisioning an extra disk to workaround https://github.com/microsoft/vcpkg/issues/18379

* Disallow public access to blob storage and require TLS 1.2.

* Update Pools.

* Update tool to 2021-06-19

* [simage] Skip simage on uwp platforms as it appears broken by 16.10.

* [tensorflow-cc] Skip because changes in our MacOS hardware broke the port.
2021-06-22 19:16:04 -07:00