1

I started developing a dbus based code for pi-zero, and compilation is slow. I would like to cross compile in my PC to speed things up. I have WSL2(Ubuntu) setup on my PC and copied the pi toolchain from https://github.com/raspberrypi/tools. I need to install the armhf libraries used in pi-zero in my WSL2 instance to cross compile. I am not quite sure how to do that, what are my options?

1 Answer 1

3

You need two folders: a sysroot and a staging area.
The sysroot contains all necessary libraries you need for the cross-compilation of your project. The staging area is where you install your project, and you'll eventually install the staging area to the Raspberry Pi to actually run your project.

The toolchain already comes with a sysroot that contains a C library, libgcc, etc., everything you need to compile a basic C or C++ program. It should be read-only, because it's part of the toolchain, and you don't want to mess it up.

If your project has dependencies, such as libdbus, you'll have to install them in your sysroot. Make a copy of the toolchain's sysroot, make it writable (chmod -R u+w your-copy-of-the-sysroot). Once it is writable, you can install the necessary libraries to this sysroot. When cross-compiling, you then point the compiler or build tools to your copy of the sysroot, using GCC's --sysroot option, for example.

To install the necessary libraries to the sysroot, you have multiple options:

  1. You copy the libraries from a Raspbian image or SD card that has them installed.
  2. You download a binary distribution from the developer's website or the Raspbian repositories. For example: http://archive.raspbian.org/raspbian/pool/main/d/dbus/
  3. You cross-compile the libraries yourself.

The first option is pretty cumbersome, because you don't really know which files belong to which package, and which files you do or do not need. It's often easier to just copy the entire image and use that as the sysroot. This requires quite a bit of disk space.

The second one allows you to select some packages, but unlike apt install <pkg>, you have to manage your dependencies manually (if there's a way to use a package manager to handle dependencies in the sysroot, I'm all ears!).

The last option is the most flexible, because you can configure the packages how you like them. For a limited number of small dependencies, this is manageable, but if you have many dependencies, and these dependencies have many dependencies, it can be a lot of work.


The following script downloads and cross-compiles D-Bus and Expat for Raspberry Pi zero:

#!/usr/bin/env bash

set -ex

# Clone the D-bus and Expat libraries
[ -d dbus ] || \
    git clone --branch dbus-1.13.18 --single-branch --depth=1 \
        https://gitlab.freedesktop.org/dbus/dbus.git

[ -d libexpat ] || \
    git clone --branch R_2_2_9 --single-branch --depth=1 \
    https://github.com/libexpat/libexpat.git

# Script for building these libraries:
cat << 'EOF' > build-script-docker.sh
#!/usr/bin/env bash

set -ex
cd "$(dirname "${BASH_SOURCE[0]}")"

# Point pkg-config to the sysroot:
. cross-pkg-config

# Directory to install the packages to:
export RPI_STAGING="$PWD/staging"
rm -rf "${RPI_STAGING}"

# libexpat
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

pushd libexpat/expat
./buildconf.sh
mkdir -p build
pushd build
../configure \
    --prefix="/usr/local" \
    --host="${HOST_TRIPLE}" \
    --with-sysroot="${RPI_SYSROOT}"
make -j$(nproc)
make install DESTDIR="${RPI_SYSROOT}"
make install DESTDIR="${RPI_STAGING}"
popd
popd

# dbus
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

pushd dbus
mkdir -p build
pushd build
cmake .. \
    -DCMAKE_TOOLCHAIN_FILE="$HOME/${HOST_TRIPLE}.cmake" \
    -DCMAKE_BUILD_TYPE="Release" \
    -DCMAKE_INSTALL_PREFIX="/usr/local"
make -j$(nproc)
make install DESTDIR="${RPI_SYSROOT}"
make install DESTDIR="${RPI_STAGING}"
popd
popd
EOF

# Start the Docker container with the toolchain and run the build script:
image="tttapa/rpi-cross:armv6-rpi-linux-gnueabihf-dev"
docker run --rm -it -v "$PWD:/tmp/workdir" $image \
    bash "/tmp/workdir/build-script-docker.sh"

You'll need to have Docker installed. When finished, the libraries will be in the staging folder in the working directory.

The Docker container with the toolchain is one I maintain (https://github.com/tttapa/RPi-Cpp-Toolchain), but the installation process should be similar with the toolchain you're using, you'll just have to install some extra dependencies such as make, autotools, and maybe cross-compile some other dependencies of Expat and D-Bus as well.
I also maintain some notes with instructions of the toolchains and cross-compilation processes, which you might find useful: https://tttapa.github.io/Pages/Raspberry-Pi/C++-Development/index.html

You might want to add some extra options to the configure and cmake steps, but that's outside of the scope of this answer, see the relevant D-Bus documentation.
Also note that installs both libraries to both the sysroot and the staging area, it'll depend on what you want to do with it. You have to install at least libexpat to the ${RPI_SYSROOT} folder, because that's the folder used as the sysroot for compiling dbus which depends on libexpat. The sysroot folder for the compilation of dbus is selected in the CMake Toolchain file, ~/${HOST_TRIPLE}.cmake, it's included with the Docker container. Its contents are:

SET(CMAKE_SYSTEM_NAME Linux)
SET(CMAKE_C_COMPILER armv6-rpi-linux-gnueabihf-gcc)
SET(CMAKE_CXX_COMPILER armv6-rpi-linux-gnueabihf-g++)
SET(CMAKE_SYSTEM_PROCESSOR armv6)

set(CMAKE_SYSROOT $ENV{RPI_SYSROOT})
SET(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT}) 

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

You might also have to point pkg-config to the right sysroot folder. This is handled by the cross-pkg-config script:

export PKG_CONFIG_LIBDIR="${RPI_SYSROOT}/usr/local/lib:${RPI_SYSROOT}/opt/vc/lib"
export PKG_CONFIG_PATH="${RPI_SYSROOT}/usr/local/lib/pkgconfig:${RPI_SYSROOT}/usr/local/share/pkgconfig:${RPI_SYSROOT}/opt/vc/lib/pkgconfig"
export PKG_CONFIG_SYSROOT_DIR="${RPI_SYSROOT}"
8
  • "The first option is pretty cumbersome, because you don't really know which files belong to which package, and which files you do or do not need." -> For compiling (as opposed to executing), you only need the libs themselves, not anything else from the package, so you can use ldd on the executable compiled on the Pi to find this set, remembering to check each .so also. You can then just copy the .so files directly to the staging area, so this is really not a cumbersome task. Great answer.
    – goldilocks
    Commented Jul 28, 2020 at 17:57
  • Yes, you're right, if you just need the libs, it's easy enough. For some libraries you also need the headers, pkg-config files, cmake files, etc. for easy integration with the build system you're using for your project, which may be a bit more work.
    – tttapa
    Commented Jul 28, 2020 at 18:04
  • Yikes forgot about the headers! Although you might as well just throw in all of /usr/include... Most stuff won't use pkg-config but I imagine dbus does.
    – goldilocks
    Commented Jul 28, 2020 at 18:09
  • Wishful thinking on my part, it almost feels like if we can do a fake apt-get, it would make things lot simpler :)
    – kibudude
    Commented Jul 28, 2020 at 19:38
  • I looked at your docker, seems like that route is probably the least complicated for me. Thank you both, great answer and comments, lota insight packed in here.
    – kibudude
    Commented Jul 28, 2020 at 20:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.