The desired outcome of this project is to claw some more of the Internet back from the corporates by enabling more people to build and run their own internet server—with web sites, e-mail, code repository, wiki, blog, activity pub server, etc.—either on someone elseʼs computer or locally on their own.
This machine that you are accessing is a headless GUIX system running in somebody elseʼs OpenStack cloud, which was put together to replace a Debian system running on Amazon Web Services; this gives more flexibility to move between cloud providers and facilitate a possible hybrid set-up, and to faithfully replicate that to and from a local testing and development facility. More importantly, this system overhaul is to be the last of the tediously drawn out behemoth procedures it normally takes to migrate a system.
It was a painful exercise to get it working, and while all the technical problems which blocked progress in getting it together have been solved, the system is very unsatisfactory: flabbier than it should be with unwanted packages and documentation, and harder to configure to bespoke requirements than it ought to be. It took an age to build, and another age to upload the eventual system image.
And so we find ourselves taking up this project to produce a very specialized fork of the GUIX project: one which doesnʼt have 20,000+ packages, doesnʼt need Grub or an initrd, nor 1,000+ device drivers; it simply runs the modern equivalent of a LAMP stack on a QEMU virtual machine.
The two main pain points are
The two things we want to achieve are
Our use case needs to provide web, GIT and e-mail services in support of the public face of our compute ecosystem. We want to specify the entire thing, including our custom application servers, from the Guix configuration file, and want to be able to build it quickly so that we can test and run locally the entire system, before rapidly deploying to the production machine in the cloud, taking advantage of GUIXʼs transactional updates to make the process safe and atomic.
Of course, we wonʼt be disseminating our private materials as part of the ν² system, but only the baseline: a modified Guix system with a stripped-down Linux kernel, and a small number of packages such as nginx, exim, dovecot, fastcgi; you will have to bring your own middleware to the mix.
At the moment (version 0.1) what we have is an unmodified Guix system (there currently isnʼt an actual fork) with a system configuration which compiles a minimal kernel, i.e. one which does not have thousands of device drivers; the system itself is also minimalist: just enough to be able to secure-shell into it as the admin user. The admin user has full sudo privilege, so can perform whatever actions are necessary to get the rest of the system running as required.
The default Guix linux-libre build uses Linuxʼs own configuration defaults, which provides a modular kernel with the maximum number of device drivers possible, and built for very generic CPU types. The upshot is that a system takes the good part of an hour to build, takes an appreciable time to start up, and occupies memory wastefully. When a system is intended to be used under QEMU, there are no physical devices to deal with, we neednʼt worry about supplying virtualization services to guests, and we can specialize the kernel for a particular CPU architecture (x86-64 in our case).
The result is a kernel (and hence Guix system) which builds in some minutes, boots almost instantly, and takes a minimum amount of resources from the system. Furthermore, we can hope to get away without a boot manager, and use QEMUʼs EFI loader to load the kernel directly.
Currently Guix insists on building a kernel with initrd and booted by Grub, and mandates that some of the kernel configuration options are set a certain way, causing a system build failure with a cryptic message if these are transgressed. These constraints need to change to support our use case better.
The Guix store is typically massive even when a minimal system has been specified, and takes a huge chunk of what may be valuable disk space. To be fair, the store is the entirety of the operating system, but the fact that items are often duplicated or higher-multiplicated is a sore point. Package specifications need to be optimized to make more efficent use of library sharing and minimized output targets. Even a system which is supposed to be headless ends up with X11 libraries (Python→Tk→libx11.so). And then there are often unrequested documentation sets and debug binaries, and generally outputs from packages which are just not needed.
Without a graphical interface specified (using the standard base configuration file), the Guix system creates a QCOW2 image of around 500MiB, which has to be uploaded to the cloud provider. Remember this is supposed to be a minimal headless system providing web service only. Our up-link is not fast, and it takes time to make the transfer, and then to establish a runnable image in the OpenStack system, and then to actually run that image up.
We would also like to standardise the configuration of packages somewhat. This means modelling all configuration files as nested string-named lists of string-named, string-valued pairs. This is a compromise between the two extremes that Guix currently presents: packages like nginx have extensive S-expression syntaxes created for scheme-like configuration within the system configuration file, whereas exim, for example, simply allows for the nomination of a native application configuration file.
The compromise means
The single downside to this approach is that the semantic correctness of configurations cannot be enforced implicitly by the Guile language. The question is, is it better to put effort into the maintenance of configuration structures, or into procedures which validate a configuration? Both are time-consuming and error-prone activities.
The result is a system which can be built from scratch, without substitutes, in under an hour. It runs on a single-core machine with 2GiB memory and 20GiB storage. It runs quickly.
The current state of development is available here.
We considered calling this guix-2, but that would have effectively been a hostile take-over of the existing project. We also considered gnu-2 or even gnu², but that project is these days a very small though important part of open operating systems and we have no reason to be associated with GNU. So we settled on something which is a little of everything and nothing to do with anything: ν².
It is a greek letter, and the name of the project is pronounced “new too”, spelled ν² in UTF, or nu-2 in seven-bit ASCII and in eight-bit ASCII (we know there is a superscript ‘2’ symbol in that latter character set, but we stay with the seven-bit representation for consistency).