Mostrando entradas con la etiqueta debian-cross. Mostrar todas las entradas
Mostrando entradas con la etiqueta debian-cross. Mostrar todas las entradas

viernes, 2 de noviembre de 2018

Cross compiling CMake-based projects using Ubuntu/Debian's multi arch

As you probably already know Ubuntu (and then Debian) added Multi-Arch support quite some time ago. This means that you can install library packages from multiple architectures on the same machine.

Thanks to the work of many people, in which I would like to specially mention Helmut Grohne, we are now able to cross compile Debian packages using standard sbuild chroots. He even was kind enough to provide me with some numbers:

We have 28790 source packages in Debian unstable.
Of those, 13358 (46.3%) build architecture-dependent binary packages.
Of those, 7301 (54.6%) have satisfiable cross Build-Depends.
Of those, 3696 (50.6% of buildable, 27.6% of sources) were attempted.
Of those, 2695 (72.9% of built, 36.9% of buildable, 20.1% of sources) were successful.
633 bugs affecting 772 packages (7.23% of 10663 unsuccessful) are reported.

Now I asked myself if I could use this to cross compile the code I'm working on without the need of doing a full Debian package build.

My projects uses CMake, so we can cross compile by providing a suitable CMAKE_TOOLCHAIN_FILE.

And so the first question is:

How do we create the necessary file using what Multi-Arch brings to our table?


I asked Helmut and he did not only provide me with lots of tips, he also provided me with the following script, which I modified a little:

Now we can run the script providing it with the desired host arch and voilá, we have our toolchain file.

#!/bin/sh

#set -x

ARCH=$1

DEB_HOST_GNU_TYPE=$(dpkg-architecture -f "-a$1" -qDEB_HOST_GNU_TYPE)
DEB_HOST_GNU_CPU=$(dpkg-architecture -f "-a$1" -qDEB_HOST_GNU_CPU)
case "$(dpkg-architecture -f "-a$1" -qDEB_HOST_ARCH_OS)" in
        linux) system_name=Linux; ;;
        kfreebsd) system_name=kFreeBSD; ;;
        hurd) system_name=GNU; ;;
        *) exit 1; ;;
esac

cat <> cmake_toolchain_$1.cmake
# Use it while calling CMake:
#   mkdir build; cd build
#   cmake -DCMAKE_TOOLCHAIN_FILE="../cmake_toolchain_.cmake" ../
set(CMAKE_SYSTEM_NAME "$system_name")
set(CMAKE_SYSTEM_PROCESSOR "$DEB_HOST_GNU_CPU")
set(CMAKE_C_COMPILER "$DEB_HOST_GNU_TYPE-gcc")
set(CMAKE_CXX_COMPILER "$DEB_HOST_GNU_TYPE-g++")
set(PKG_CONFIG_EXECUTABLE "$DEB_HOST_GNU_TYPE-pkg-config")
set(PKGCONFIG_EXECUTABLE "$DEB_HOST_GNU_TYPE-pkg-config")
EOF


Can we improve this?

Helmut mentioned that meson provides debcrossgen, a script that automates this step. Meson is written in python, so it only needs to know the host architecture to create the necessary definitions.

CMake is not interpreted, but maybe it has a way to know the host arch in advance. If this is true maybe a helper could be added to help in the process. Ideas (or even better, patches/code!) welcomed.

martes, 28 de noviembre de 2017

Experimental cross compiling Qt in Debian packages

Some time ago we the Qt/KDE team were contacted by Helmut Grohne. He was trying to cross compile Debian packages in general thanks to Ubuntu/Debian's multi-arch support, and he was having problems with Qt-based ones.

As far as we understand Qt upstreams only support cross compiling by having a toolchain for each pair of architectures involved. In Debian terms, and only considering current official architectures, that would mean building 90 cross toolchains. It clearly doesn't scale.

So we set up to discuss if somehow we could use multiarch to let debian packages using Qt to cross compile.

In the meantime Enrico Zini had the same idea. He wrote a nice summary of the situation at that time in his blog.

After many thinking some ideas were tested and we've got to the point of solving/hacking the issue. As this is not something directly supported by upstream you should take care, and file bugs whenever necessary.

Dmitry Schachnev from our team's side and Helmut from the debian-cross side worked a lot on it, and I would like to present what they have done. To be fair it's mostly described in our team's gobby qt-cross page, but I would like to give it some publicity in order to let people know about it and why not, find and help solving bugs.

General stuff

The first thing that was done was to move Qt binaries from their (Debian original) multi-arch path to a non multi-arch one, providing symlinks for compatibility. In this way the path of the binaries is the same for any arch (why they were not there is a long story, but nothing to worry now).

This move needed some other touches, like qtchooser being updated with the new paths.

The other changes where related to how we do our packaging:


  • All packages containing binaries are now M-A:foreign.
  • Some packages (qt3d, qtwayland) had binaries split to allow that.
  • qttools5-dev-tools now depends on libqt5sql5-sqlite (not uploaded yet)

qmake related changes

We also needed to address qmake. To begin with we splitted the package containing it into qt5-qmake-bin (M-A:foreign) and qt5-qmake (M-A:same). The first one has the binaries and the second the relevant mkspecs for some arch.

The rest of the "magic" comes from debhelper. It generates a qt.conf file with the right paths for each cross compilation and also passes cross QMAKE_CC and QMAKE_CXX to qmake when needed.


autotools

qt5-qmake will ship /usr/bin/$(DEB_HOST_GNU_TYPE)-qmake executable for use with AC_CHECK_TOOL (not uploaded yet).

There is still work to be done, but so far we have been able to cross compile packages using for example sbuild.

Edit 20171129 11:43 ARST: You should really look at the new Enrico's post.