Guix shell for programs with batteries included

One of the aspects of Guix that really sets it apart from other package managers is how there is not much difference in power between the user and the repository. Sure, you can build your own .debs just fine and manage them with dpkg, but it takes a lot more infrastructure to have your own small packages play well with apt (and other users). For sharing packages in Guix, most of the time it is sufficient to share a small pkg.scm or similar.

Sometimes you discover these that such features lead to tricks that are just bonkers, equal parts brilliant and terrifying.

Let’s start with the guix shell. Running guix shell pkgs... drops us in a shell where all pkgs... are made available. This is quite useful for one-off tasks such as when building software or testing with different versions of a program. It is also very useful when writing scripts, because you can “bundle” difficult dependencies with the script itself, and have the script fetch them as needed. I like doing it in makefile targets, too, so that when I run make doc.pdf, make knows how to retrieve my document processor as well as running it.

To show it off, we can start by fixing Python. Running Python programs can be painful because libraries, much like shared libraries in the traditional unix sense, share namespace and different programs have a knack for requiring different versions. Over the years there have been multiple tools that try to solve this; virtual environments (venv), pipenv, lately uv is gaining popularity. Let’s solve it with Guix; here’s hello-numpy.py:

#!/usr/bin/env -S guix shell python python-numpy -- python3
import numpy
print("This is numpy", numpy.version.version)

This script loads numpy and prints the version. This is from my Debian 12 system, which has numpy installed:

$ python3 hello-numpy.py
This is numpy 1.24.2

The file is executable with a shebang, so let’s put it in PATH and run it:

$ env PATH=$(pwd):$PATH hello-numpy.py
This is numpy 2.2.5

If you run this on a fresh system, Guix will first download Python and Numpy before running it.

This is already quite useful because we can now reliably run programs with many and even very specific dependencies. But we can take it further, because Guix is not specific to Python environments, it works for all kinds of software. Here’s an example with python, cmake, and coreutils; note how the versions change if run through bash rather than using the shebang.

$ env PATH=$(pwd):$PATH mix
cat (GNU coreutils) 9.1
Python 3.11.2
cmake version 3.25.1

$ bash mix
cat (GNU coreutils) 9.1
Python 3.11.11
cmake version 4.0.3

Guix supports manifest, small Guile programs that guix shell can run to determine what packages to provide. It turns out that with a clever trick we can embed these manifests inside our scripts. There is a nice post on this at futurile, and the trick was originally suggested on the Guix mailing list.

The program in the futurile post is a shell script which spawns a Guix shell in a container with itself as the manifest. This is already spectacular, but since it is really just a shell script we can combine it with other classic tricks. Let’s transform the hello-numpy to use the same trick (plus a new one):

#!/usr/bin/env bash

exec guix shell --manifest="$0" -- python3 <<EOF
import numpy
print("This is numpy", numpy.version.version)
EOF

!#
(specifications->manifest
 (list "python"
       "python-numpy"))

So this is a shell script that embeds a Python script (in a HEREDOC) that knows how to download and configure the Python dependencies by providing itself as a Guile program input to a command. When I run it on my system it prints “This is numpy 2.2.5”, showing that it’s using the Guix one.

Most of the time, manifests are just lists of package names, but since they’re just Guile programs we can put even more interesting things in there. Let’s say we want write a program that prints a nice header and footer and dumps its inputs in bytes and braille. In the examples so far we’ve only looked at very standard pieces of software, but this is more exotic. Here is my program, cowbraille (there’s a demo at the end of this post):

#!/usr/bin/env bash

if [ -z "${BRAILLE_GUIX_ENV+1}" ]; then
    exec guix shell -m "$0" -- env BRAILLE_GUIX_ENV=1 bash "$0" "$@"
fi;

cowsay "They say curiosity killed the cat, but I'm a cow so what do I know"
for f in "$@"; do
    cowsay "This is $f"
    bd "$f"
done
cowsay "That's all Folks!"

exit

!#
(use-modules (guix packages)
             (guix download)
             (guix gap)
             (guix build-system gnu))

(define brailledump
  (package
    (name "brailledump")
    (version "6e6fc38935054db0534d5af4fb99c6193305b946")
    (source
     (origin
       (method url-fetch)
       (uri (string-append
             "https://raw.githubusercontent.com/jart/cosmopolitan/"
             version "/tool/viz/bd.c"))
       (sha256
        (base32
         "1a6l7xym23fscslgxikhfnc91yfx0jscs77djsm08f213mz4sips"))
       (modules '((guix build utils)))
       (snippet
        '(substitute* "bd.c"
           ;; We need to use standard includes, not the cosmopolitan
           ;; internal ones
           (("\"libc/stdio/stdio.h\"")
            "<stdio.h>\n#include <stddef.h>\n#include<locale.h>")
           ;; Call setlocale to print utf8 on glibc
           (("long o;") "long o; setlocale(LC_ALL, \"\");")))))
    (build-system gnu-build-system)
    (arguments
     (list
      #:phases
      #~(modify-phases %standard-phases
          (delete 'configure)
          (delete 'check)
          (replace 'build
            (lambda _ (invoke "gcc" "-O2" "bd.c" "-o" "bd")))
          (replace 'install
            (lambda* (#:key outputs #:allow-other-keys)
              (let ((bin (string-append (assoc-ref outputs "out")
                                        "/bin")))
                (install-file "bd" bin)))))))
    (synopsis "hexdump -C with braille")
    (description
     "braille dump is a drop in replacement for hexdump -C that uses
unicode braille characters to display hex codes 0x81..0xff thereby
improving the readibility of binary.")
    (home-page "https://justine.lol/braille/")
    (license #f)))

(packages->manifest
 (append
  (list brailledump)
  (specifications->packages
   (list "bash" "coreutils" "cowsay"))))

If you have Guix available you should be able to run this yourself, just copy the contents into a file and make it executable!

There’s a lot to unpack here, and maybe a later post will go through the details carefully, but running this program will:

1. Download the source for brailledump
2. Patch it slightly to make it work with glibc
3. Tweak the build rules to call gcc directly, without going through ./configure && make
4. Build brailledump
5. Create a shell where brailledump, cowsay, and bash are available
6. Run brailledump on each argument, with progress reported by cows

All of this neatly packaged up into a single file.

We need to guard the exec guix shell to avoid infinite recursion, and exit so that the script does not try to execute the Guile (which obviously wouldn’t work).

This should work anywhere, but if you can assume bash, zsh, or any other shell that sets the _ env var to the program currently being run you can use this funny-looking shebang and skip the BRAILLE_GUIX_ENV guard.

#!/usr/bin/env -S guix shell -m "${_}" -- bash "${_}"

env (1) will expand named variables, in this case _, before running the interpreter. For cowbraille, it would expand cowbraille files... to:

exec guix shell -m "cowbraille" -- bash "cowbraille"

This fails in dash, and using bash as the interpreter is probably more robust overall, and the guard should feel way more familiar.

So, should we do this for all our programs? Probably not, but it’s a stellar trick that can occasionally be very useful.

Interested in learning more? Patch offers training and workshops on Guix, and can help you manage software and packages with Guix in your organization.

And as promised, here is a demo of running cowbraille: