Begriffs Buildfarm

Joe Nelson joe at begriffs.com
Sun Feb 7 18:57:16 UTC 2021


EuAndreh wrote:
> > Would you be interested in hosting the buildfarm server and giving us
> > network access to run builds?
> 
> [...]
> 
> Personally, I would prefer to run the buildfarm myself, so a paid CI
> service would simply be a hosted version of what I could run myself,
> even though I would probably also pay.

That's a great point. I think I was fixated on the idea of setting up a
server somewhere, when really the creation and curation of these QEMU
images is the important part. If we made the images (or scripts to
create them) available for download, then devs could emulate locally.

> The 'buildfarm-image-run' part is probably where most of the logic
> lives, downloading the ISOs, provisioning the images, etc.

Thinking how it might work... I see that QEMU provides a bunch of
binaries named qemu-system-$x, where $x is a processor architecture.
http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/emulators/qemu/pkg/PLIST?rev=1.46&content-type=text/x-cvsweb-markup

Focusing on MIPS, there's the big-endian "qemu-system-mips" and
little-endian "qemu-system-mipsel". We'd then find base ISOs matching
the architectures. Picking, say, Debian, those would be:

https://cdimage.debian.org/debian-cd/current/mips/iso-cd/debian-10.8.0-mips-netinst.iso
https://cdimage.debian.org/debian-cd/current/mipsel/iso-cd/debian-10.8.0-mipsel-netinst.iso

We could run the installer ISOs in QEMU with the -nographic flag, and
hook the Expect tool to the tty to control the installation.
https://core.tcl-lang.org/expect/index
(In fact, we could interact with installer manually, and let
"autoexpect" create the script to automate our actions.)

Once there's an automated way to turn an operating system ISO into a
QEMU disk image with the desired packages installed, then the next step
is using that disk image to test our code.

I was thinking we could somehow bundle the code to be tested into its
own disk image on request, and attach the image to the QEMU guest
machine as if it were a harddrive. Then have our Expect scripter log in
to the guest, mount the drive, and run the build/test steps. That series
of steps would probably need to be customized per OS.

Thus each platform to test would need three things:
1. The base OS ISO
2. The custom installer runner script
3. The custom test runner script

Starting with MIPS on Debian 10 might be a good test case, because aside
from the hardware architecture it would be familiar. I can't get to this
project right away -- still finishing up another thing -- but if you or
another person on the list are able to make progress on it I'd love to
hear about it.


More information about the Friends mailing list