vcpkg/scripts/azure-pipelines/linux/provision-image.sh
Billy O'Neal 4217fee510
Update Windows VMs for Patch Tuesday August 2022 (#26283)
* Install haskell-stack with apt instead of piping a sh.

* Update pools.

* Linux is exploding for some reason. I just found out it is getting created with a 30GB disk I'm guessing may be involved.

Did not wait for x64_windows_static_md to finish because enough was already on the floor.

PASSING, REMOVE FROM FAIL LIST: angle:arm-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:arm64-windows (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-uwp (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x64-windows-static (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: angle:x86-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: qtwebengine:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: rsocket:x64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
PASSING, REMOVE FROM FAIL LIST: rsocket:x64-windows-static (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).
REGRESSION: gazebo:x64-windows failed with BUILD_FAILED. If expected, add gazebo:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2275: 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2923: 'std::shared_ptr': 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): note: see declaration of 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
```

We have been seeing intermittent failures in gazebo:x64-windows and I think this just repro'd that...

REGRESSION: graphicsmagick:x64-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: graphicsmagick:x64-windows-static failed with BUILD_FAILED. If expected, add graphicsmagick:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: graphicsmagick:x86-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick/nt_base.h(95): warning C4005: 'HAVE_FT2BUILD_H': macro redefinition
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick/magick_config.h(123): note: see previous definition of 'HAVE_FT2BUILD_H'
D:\buildtrees\graphicsmagick\src\2e465a2909-adf106de54.clean\magick\effect.c(689) : fatal error C1001: Internal compiler error.
(compiler file 'D:\a\_work\1\s\src\vctools\Compiler\Utc\src\p2\main.c', line 224)
 To work around this problem, try simplifying or changing the program near the locations listed above.
If possible please provide a repro here: https://developercommunity.visualstudio.com
Please choose the Technical Support command on the Visual C++
 Help menu, or open the Technical Support help file for more information
  cl!RaiseException()+0x6c
  cl!RaiseException()+0x6c
  cl!DllGetObjHandler()+0xdb94f
  cl!DllGetObjHandler()+0x1ac555
```

😭

REGRESSION: mathgl:x64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: mathgl:x64-windows-static failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x64-windows-static=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: mathgl:x86-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add mathgl:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
-- Performing post-build validation
The following files contain an absolute path ('D:\packages\mathgl_x64-windows', 'D:\installed', 'D:\buildtrees\mathgl'):
    D:\packages\mathgl_x64-windows\include\mgl2\config.h
There should be no absolute paths in the installed package, only relative ones.

Found 1 post-build check problem(s). To submit these ports to curated catalogs, please first correct the portfile:
    C:\a\2\s\ports\mathgl\portfile.cmake
-- Performing post-build validation done
error: building mathgl:x64-windows failed with: POST_BUILD_CHECKS_FAILED
```

@autoantwort

REGRESSION: qtwebengine:arm64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add qtwebengine:arm64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.

```
Building qtwebengine[core,default-features,geolocation,spellchecker,webchannel]:arm64-windows...
-- Setting up python virtual environmnent...
-- Installing python packages: html5lib
-- Setting up python virtual environmnent...finished.
CMake Warning at ports/qtwebengine/portfile.cmake:83 (message):
  Buildtree path 'D:/buildtrees/qtwebengine' is too long.

  Consider passing --x-buildtrees-root=<shortpath> to vcpkg!

  Trying to use 'D:/buildtrees/qtwebengine/../tmp'
Call Stack (most recent call first):
  scripts/ports.cmake:147 (include)

-- Using cached qtwebengine-everywhere-src-6.3.1.tar.xz.
-- Cleaning sources at D:/buildtrees/tmp/src/here-src-6-367c283e50.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source D:/downloads/qtwebengine-everywhere-src-6.3.1.tar.xz
-- Using source at D:/buildtrees/tmp/src/here-src-6-367c283e50.clean
-- Found external ninja('1.10.2').
-- Configuring arm64-windows-dbg
-- Configuring arm64-windows-rel
CMake Warning at D:/installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_configure.cmake:302 (message):
  The following variables are not used in CMakeLists.txt:

      FEATURE_webengine_geolocation
      FEATURE_webengine_spellchecker
      FEATURE_webengine_v8_snapshot_support
      FEATURE_webengine_webchannel
      FEATURE_webengine_webrtc

  Please recheck them and remove the unnecessary options from the
  `vcpkg_cmake_configure` call.

  If these options should still be passed for whatever reason, please use the
  `MAYBE_UNUSED_VARIABLES` argument.
Call Stack (most recent call first):
  D:/installed/arm64-windows/share/qtbase/qt_install_submodule.cmake:108 (vcpkg_cmake_configure)
  ports/qtwebengine/portfile.cmake:102 (qt_cmake_configure)
  scripts/ports.cmake:147 (include)

-- Building arm64-windows-dbg
-- Building arm64-windows-rel
-- Installing: D:/packages/qtwebengine_arm64-windows/share/qtwebengine/copyright
-- Performing post-build validation
The folder /include is empty or not present. This indicates the library was not correctly installed.
There should be no empty directories in D:\packages\qtwebengine_arm64-windows
The following empty directories were found:
```

This was previously blocked by qtwebengine:x64-windows=fail. Will investigate...

* Remove attempt to install extra stuff for scale set agents.

* Turn on boot diagnostics.

* Update pools.

* Back out most Linux changes.

* Analysis of https://dev.azure.com/vcpkg/public/_build/results?buildId=76482&view=results

REGRESSION: graphicsmagick:x86-windows failed with BUILD_FAILED. If expected, add graphicsmagick:x86-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

Added this skip too.

REGRESSION: gazebo:x64-windows failed with BUILD_FAILED. If expected, add gazebo:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
C:\PROGRA~1\MICROS~1\2022\ENTERP~1\VC\Tools\MSVC\1433~1.316\bin\Hostx64\x64\cl.exe   /TP -DBOOST_ALL_DYN_LINK -DBOOST_ALL_NO_LIB -DBOOST_TEST_DYN_LINK -DBUILDING_DLL -DBUILDING_DLL_GZ_PHYSICS -DFREEIMAGE_LIB -DH5_BUILT_AS_DYNAMIC_LIB -DIGN_PROFILER_ENABLE=0 -DLIBBULLET_VERSION=3.21 -DLIBBULLET_VERSION_GT_282 -DNOMINMAX -DNOUSER -DPROTOBUF_USE_DLLS -DTINYXML2_DEBUG -DTINYXML2_IMPORT -DWIN32_LEAN_AND_MEAN -D_USE_MATH_DEFINES -DdIDEDOUBLE -Dgazebo_physics_EXPORTS -ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\test\gtest\include -ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean -ID:\buildtrees\gazebo\x64-windows-dbg -ID:\installed\x64-windows\debug\..\include\libusb-1.0 -external:ID:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\deps\opende\include -external:ID:\installed\x64-windows\include -external:ID:\installed\x64-windows\debug\..\include -external:ID:\installed\x64-windows\include\ignition\msgs5 -external:ID:\installed\x64-windows\include\ignition\math6 -external:ID:\installed\x64-windows\include\ignition\transport8 -external:ID:\installed\x64-windows\include\ignition\common3 -external:ID:\installed\x64-windows\include\ignition\fuel_tools4 -external:ID:\installed\x64-windows\include\OGRE -external:ID:\installed\x64-windows\include\sdformat-9.8 -external:ID:\installed\x64-windows\include\bullet -external:ID:\installed\x64-windows\include\sdformat-9.8\sdf\.. -external:ID:\installed\x64-windows\include\ignition\cmake2 -external:ID:\installed\x64-windows\include\eigen3 -external:ID:\installed\x64-windows\share\urdfdom_headers\..\..\include -external:ID:\installed\x64-windows\include\urdfdom -external:ID:\installed\x64-windows\include\urdfdom_headers -external:W0 /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP        /wd4005 /wd4068 /wd4244 /wd4251 /wd4267 /wd4275 /wd4996 /EHsc /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1    /EHsc -I"D:/installed/x64-windows/include" /Zc:__cplusplus /permissive- /Zc:strictStrings- /Zc:externC- -std:c++17 /showIncludes /Fogazebo\physics\CMakeFiles\gazebo_physics.dir\dart\DARTBoxShape.cc.obj /Fdgazebo\physics\CMakeFiles\gazebo_physics.dir\ /FS -c D:\buildtrees\gazebo\src\46e867c51d-eba7a8151f.clean\gazebo\physics\dart\DARTBoxShape.cc
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2275: 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2923: 'std::shared_ptr': 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): note: see declaration of 'dart::common::Signal<_Res(_ArgTypes...),Combiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(87): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2275: 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType': expected an expression instead of a type
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2923: 'std::shared_ptr': 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType' is not a valid template type argument for parameter '_Ty'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): note: see declaration of 'dart::common::Signal<void(_ArgTypes...),dart::common::signal::detail::DefaultCombiner>::ConnectionBodyType'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
D:\installed\x64-windows\include\dart/common/detail/Signal.hpp(182): error C2955: 'std::shared_ptr': use of class template requires template argument list
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include\memory(1493): note: see declaration of 'std::shared_ptr'
```

PASSING, REMOVE FROM FAIL LIST: luajit:arm64-windows (C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt).

This was https://github.com/microsoft/vcpkg/pull/26360

REGRESSION: qtwebengine:arm64-windows failed with POST_BUILD_CHECKS_FAILED. If expected, add qtwebengine:arm64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This was previously blocked by x64-windows being skipped, so I'm just adding it to the baseline for now.
2022-08-17 16:44:51 -07:00

96 lines
3.3 KiB
Bash
Executable File

#!/bin/bash
# Copyright (c) Microsoft Corporation.
# SPDX-License-Identifier: MIT
#
export DEBIAN_FRONTEND=noninteractive
apt-get -y update
apt-get -y dist-upgrade
# Install vcpkg prerequisites
APT_PACKAGES="git curl zip unzip tar"
# Install common build dependencies
APT_PACKAGES="$APT_PACKAGES at libxt-dev gperf libxaw7-dev cifs-utils \
build-essential g++ gfortran libx11-dev libxkbcommon-x11-dev libxi-dev \
libgl1-mesa-dev libglu1-mesa-dev mesa-common-dev libxinerama-dev libxxf86vm-dev \
libxcursor-dev yasm libnuma1 libnuma-dev \
flex libbison-dev autoconf libudev-dev libncurses5-dev libtool libxrandr-dev \
xutils-dev dh-autoreconf autoconf-archive libgles2-mesa-dev ruby-full \
pkg-config meson nasm cmake ninja-build"
# CUDA tooling
APT_PACKAGES="$APT_PACKAGES nvidia-cudnn nvidia-cuda-toolkit"
# Additionally required by qt5-base
APT_PACKAGES="$APT_PACKAGES libxext-dev libxfixes-dev libxrender-dev \
libxcb1-dev libx11-xcb-dev libxcb-glx0-dev libxcb-util0-dev"
# Additionally required by qt5-base for qt5-x11extras
APT_PACKAGES="$APT_PACKAGES libxkbcommon-dev libxcb-keysyms1-dev \
libxcb-image0-dev libxcb-shm0-dev libxcb-icccm4-dev libxcb-sync-dev \
libxcb-xfixes0-dev libxcb-shape0-dev libxcb-randr0-dev \
libxcb-render-util0-dev libxcb-xinerama0-dev libxcb-xkb-dev libxcb-xinput-dev"
# Additionally required by libhdfs3
APT_PACKAGES="$APT_PACKAGES libkrb5-dev"
# Additionally required by kf5windowsystem
APT_PACKAGES="$APT_PACKAGES libxcb-res0-dev"
# Additionally required by mesa
APT_PACKAGES="$APT_PACKAGES python3-setuptools python3-mako"
# Additionally required by some packages to install additional python packages
APT_PACKAGES="$APT_PACKAGES python3-pip python3-venv"
# Additionally required by qtwebengine
APT_PACKAGES="$APT_PACKAGES nodejs"
# Additionally required by qtwayland
APT_PACKAGES="$APT_PACKAGES libwayland-dev"
# Additionally required by all GN projects
APT_PACKAGES="$APT_PACKAGES python2 python-is-python3"
# Additionally required by libctl
APT_PACKAGES="$APT_PACKAGES guile-2.2-dev"
# Additionally required by gtk
APT_PACKAGES="$APT_PACKAGES libxdamage-dev"
# Additionally required by gtk3 and at-spi2-atk
APT_PACKAGES="$APT_PACKAGES libdbus-1-dev"
# Additionally required by at-spi2-atk
APT_PACKAGES="$APT_PACKAGES libxtst-dev"
# Additionally required by bond
APT_PACKAGES="$APT_PACKAGES haskell-stack"
# Additionally required/installed by Azure DevOps Scale Set Agents
APT_PACKAGES="$APT_PACKAGES libkrb5-3 zlib1g libicu70"
apt-get -y install $APT_PACKAGES
# Start up cudnn
update-nvidia-cudnn -d
update-nvidia-cudnn -u
# Install nccl
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /"
apt-get -y update
apt-get install --no-install-recommends libnccl2 libnccl-dev
# Install PowerShell
wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
rm -f packages-microsoft-prod.deb
apt-get update
add-apt-repository universe
apt-get install -y powershell