Begriffs Buildfarm

[EuAndreh]

Joe Nelson <joe at begriffs.com> writes:

> (I'm CC'ing an email list where we had discussed the idea previously.)

Nice, I'm interested in hearing more from people with similar interest
on this.

> Of the machines for the farm, I wanted ones to exercise 16-, 32-, and
> 64-bit CPUs, different endianness, maybe a Harvard architecture, memory
> alignment restrictions, signed overflow trap, a compiler with weird
> CHAR_BIT size (although this last wouldn't be POSIX conformant). A
> variety of 64-bit memory models like LP64, ILP64, LLP64, ILP32, or LP32
> would be nice too.

+1 for the list of variations. There are some things there I've never
heard about before, such as Harvard architecture.

> Does QEMU require specialized hardware to work efficiently with KVM?

Not that I'm aware of.

> Would you be interested in hosting the buildfarm server and giving us
> network access to run builds?

I am somewhat interested, yes. I took some time to dump what I would
expect to get from such a build farm as a user.

Personally, I would prefer to run the buildfarm myself, so a paid CI
service would simply be a hosted version of what I could run myself,
even though I would probably also pay.

A 'buildfarm-check' top-level script would be the entrypoint utility
that I would run at the root the repository, something like:

    #!/bin/sh
    set -u

    IMAGES='
    debian-glibc-gcc-x86_64-bigendian
    alpine-musl-tinycc-arm32-littleendian
    freebsd-dietlibc-clang-something-something
    '

    SUCCESS="$(mktemp)"
    FAILURE="$(mktemp)"

    for image in $IMAGES; do
      buildfarm-image-run "$image" > "$image-report.log"
      if [ $? = 0 ]; then
        echo "$image" > "$SUCCESS"
      else
        echo "$image" > "$FAILURE"
      fi
      STATUS
    done

    printf 'SUCCESS for images:\n%s\n' "$(cat "$SUCCESS")" >&2

    if [ -s "$FAILURE" ]; then
      printf 'FAILURE for images:\n%s\n' "$(cat "$FAILURE")" >&2
      exit 1
    fi

Then I would run:

    $ buildfarm-check
    ...
    SUCCESS for images:
    debian-glibc-gcc-x86_64-bigendian
    alpine-musl-tinycc-arm32-littleendian
    FAILURE for images:
    freebsd-dietlibc-clang-something-something

This would allow me (and anyone else) to install the buildfarm on their
Git server and run 'buildfarm-check' on the push hook, and have their
independent CI.

The 'buildfarm-image-run' part is probably where most of the logic
lives, downloading the ISOs, provisioning the images, etc. If it was
responsible for the provisioning part, it could simply embed a
'buildfarm-build' script that could look like:

    #!/bin/sh
    set -eu

    if [ -e configure ]; then
      ./configure
    fi

    make

    if [ grep -q '^check:' Makefile ]; then
      make check
    fi

It could also have extra steps depending on the available targets, such
as running install/uninstall targets, installcheck, etc. Since nothing
like that is standard, it could do those as a best-effort.

It could even check for a '.buildfarm.yaml' file, à la builds.sr.ht
manifest, to setup the environment previously:

    packages:
      - pkg-config
      - libsomething
    source: https://git.somewhere.tld/repo.git

Maybe pkgsrc [0] could provide the backbone for the "packages" section,
so that we can use dependencies installed on the system instead of
vendoring them.

[0]: https://pkgsrc.org/

Given this minimal setup, most of the work I see would be on the ongoing
setup of the QEMU images, making sure the base environment is available
with utilities such as c99, pax, yacc, ar, etc., as more variations are
added over time. Creating and configuring a
'debian-glibc-gcc-x86_64-bigendian' image will probably be easy, but a
'minix-dietlibc-tinycc-arm-etc-etc' maybe not so much. But I don't see
why not start with 2 supported images, and incrementally add more
variations with time.

On hosting the buildfarm and providing network access, we could simply
rent a VPS, add a user for each person, and install the buildfarm code
there. QEMU does work with nested VMs, but I never did it on a VPS,
which are mostly VMs themselves. Most of the work would be on making
sure the nested QEMU works.

Having a hosted paid version of this would be phenomenal, but it would
be a much bigger part-time (and maybe even full-time!) project,
involving processing payments, log retention of runs, integrating with
webhooks for code forges, user management, a web interface for the user
to navigate and manage things, etc.

Building this does sounds interesting, though. I myself would be a
paying user.

Other than the code on this email, I have nothing, and its all
vaporware. Have any of you done something similar? Are you planning on
building such paid CI service?