NixOS <3 Chromebook?

Posted on November 10, 2017 by Linus Heckemann

Note: This is not actually related to my Honours project. I’m just temporarily putting it in this blog for lack of another one. One day I’ll probably factor this all out into a general-purpose personal blog with a category for the GHOTL project. Only time will tell…

Background

This summer, I bought an Asus C201 Chromebook. The reasons I bought this in addition to my main laptop are:

  • It’s small, light and has a great battery life. This makes it very suitable for use while travelling.

  • It’s cheap. While losing it/having it stolen would suck, it wouldn’t suck nearly as much as my main laptop being stolen.

  • It has an ARM CPU, which some nerds (like me) think is cool and which is also a factor in its battery life.

  • It can run on exclusively free software! The stock, coreboot-based firmware is free software, as is the code running on the embedded controller. Libreboot is a distribution of coreboot which has no proprietary components, and works on the C201. This is even cooler than the ARM part!

Of course, I wanted to run NixOS on it. So began my journey.

Mark I: Debian + nix

The first step I took was to install Debian in a chroot which I could access from within Chrome OS. For this, I put my SD card in my main laptop, used Debian’s debootstrap tool to set up a Debian chroot on it. I also wrote a hacky little script for entering the chroot. This isn’t what the script looked like right from the beginning, but is the product of quite a bit of iteration which unfortunately will forever remain undocumented (I didn’t have git available at this stage ☹ ).

Since NixOS’s official binary cache doesn’t provide binaries for ARMv7, I built it from source. This went pretty smoothly, being mostly a matter of following the classic no-package-manager procedure for installing software from source: ./configure && make && make install.

After this, I needed to get nixpkgs (git clone -b nixos-unstable https://github.com/nixos/nixpkgs-channels /nixpkgs, yes, I put it straight in / because nixpkgs is so important 😉 ), stick it in NIX_PATH (export NIX_PATH=nixpkgs=/nixpkgs) and install Nix the proper way: nix-env -f '<nixpkgs>' -iA nix. A bunch of gcc bootstrapping and some environment setup later, I had a proper working Nix installation!

Of course, I wasn’t content with Debian and particularly didn’t want to have Chrome OS running in the background for several reasons:

  • eating memory

  • rebooting whenever the OOM killer was unleashed on the browser I wasn’t using

  • having two very unsatisfactory consoles: the in-browser one, which would always offer to close when I pressed Ctrl-W, and frecon, Chrome OS’s built-in console (more on this later), which only supports a limited variant of the US keyboard layout. I like my British layout!

So I needed to go further.

Mark II: Booting Debian

I tried to get Debian booting with the ChromeOS kernel using the convenient instructions on the Debian wiki. Unfortunately, these instructions are no longer entirely usable since a change in Chrome OS: they removed the kernel’s built-in console, fbcon, and replaced it with a user-space console called frecon. Frecon does have a few advantages, such as supporting 256 colours and using the newer KMS API rather than legacy framebuffers. However, it’s not trivial to get running on Debian! So I think I might have got Debian booting successfully, but I never knew since I had no feedback on the screen. I also didn’t pursue it that much because NixOS was what I really wanted anyway.

Mark III: Booting NixOS

I set up a new chroot on a USB stick using the good old nixos-generate-config and nixos-install tools, which I acquired using Nix magic:

# echo {} > /tmp/empty.nix
# NIX_PATH=nixpkgs=/nixpkgs:nixos-config=/tmp/empty.nix nix-build '<nixpkgs>' -A config.system.build.nixos-generate-config
# ./result/bin/generate-nixos-config --root /mnt
# ... Likewise for nixos-install

Then I took my chroot script and hacked around with it until I could use it for the NixOS chroot…

And then the real fun began. I managed to get NixOS booting with Chrome OS’s kernel based on Debian’s instructions and some valuable help from clever of #nixos on irc.freenode.net. Of course, I was greeted with a black screen since just like with Debian I had no userspace program providing a console… But it responded to the power button by making the USB stick’s light flash a little and then powering off the machine! So it was indeed booting, and checking the journal allowed me to confirm this externally.

Mark IV: Booting NixOS with a visible console

I was stuck at the “booting NixOS without any visual feedback” stage for a while. After trying in vain to get kmscon to show me something on the screen (this involved excruciatingly long builds of LLVM as a dependency of Mesa), I bit the bullet and started trying to build my own kernel (the standard kernel built by nixpkgs wouldn’t work for some reason, still not sure what that reason is). Using nixpkgs for this quickly proved to be unreasonable, as the kernel takes approximately 2-3 hours to build on the little ARM CPU in the Chromebook. So I moved to building it with make myself from a nix-shell.

Many, many make menuconfigs later, and with lots of help from Tuomas Tynkkynen (aka Dezgeg) at the NixCon hackathon, I reached a breakthrough: an honest-to-goodness fbcon Linux console! This was the single biggest piece of progress made along the journey, as it allowed me to cut Chrome OS out of the process (apart from when I screwed up the boot and needed to recover), and make proper use of NixOS and its wonderful declarative config.

I also got some more of the hardware working, particularly the real-time clock (previously the time would be set to the epoch on every boot, which didn’t entirely reflect reality) and WiFi (unfortunately this involved a piece of proprietary/nonfree firmware).

From there I explored various things; w3m worked out of the box, and successfully rendered images to the framebuffer; jfbpdf proved very useful for my first forays into actual productivity with the laptop, allowing me to read materials relevant to university assignments; I got weston running, for the sake of seeing a graphical UI (although I didn’t make much use of this, and it was rather sluggish for lack of graphics acceleration; this is still a TODO); and I wrote a blog post on it! (also this one). I’m still pretty much living on the console, as it provides for most of my needs. The thing I’ve missed most distinctively is a graphical browser, but I wasn’t planning to wait for a build of WebKit or Firefox to complete so that’s still way off in the future.

So, a summary of what works:

  • Console

  • Framebuffer applications (w3m, jfbpdf)

  • WiFi (with nonfree firmware ☹)

  • Webcam (tested using gstreamer’s v4l2src and fbdevsink)

  • Rust! Using Mozilla’s overlay, I was able to get a little Rust program compiling and running. Rust is cool.

What’s still missing (for now!)

  • Audio; the device appears correctly, and I can adjust volumes using alsamixer, but no sound seems to actually come out;

  • Graphical browser. This is mostly an issue of build duration, and I’m hoping to be able to offload this to another machine at some point in the relatively near future.

  • Haskell. Similar issue to getting a browser running: from what I’ve heard, there are multiple painfully long bootstrap stages involved in getting ghc operational. This would be very nice to have though, as it would allow me to use pandoc, hakyll and Agda, which would be extremely helpful for uni work!

  • Graphics acceleration. Probably necessary for a graphical browser to be usable, and would probably make using Weston a lot more fun. Not a high priority though.

  • Nixpkgs-built kernel. I’m still running one built from nix-shell for now.

  • Nixos module for the bootloader. I currently still set up the kernels for booting manually using some scripts. For the extra coolness factor, I’d really like to use linux with a tiny userspace and kexec as a bootloader, in the same vein as petitboot (I can’t seem to find a single project page for it, hence no link). I’ll probably be satisfied with just getting a nixos module to sign and install the kernel for a start though.

  • Wishlist:

    • Networked storage. Since the onboard storage is pretty limited, I’d like to be able to mount a network share. However, I’d also like for it to be cached locally in such a way that I can access most of the stuff I need offline as well. I’m not sure if this has been implemented, or even how technically feasible it is.

    • Offloading builds. They’re slow, so I’d like to be able to run builds on more devices such as my old phone, which has a broken screen but should still be perfectly usable as a headless ARMv7 build server! Nix supports this very well, so it would just be a matter of getting a GNU/Linux system booting on the devices in question.

    • Share builds so others can use them (and maybe contribute too)! Again this should be supported pretty well by Nix and it’s mostly a matter of getting the server in place. I already have a machine available that would do nicely.

This has been a fun adventure so far, and I’m looking forward to making it more and more complete. I’d like to thank Dezgeg and Michael Bishop for their invaluable help, I’d probably be stuck way further back without them!

Coming soon: a hopefully more brief post describing how to reproduce this oneself, as opposed to the journey that led me here.