From: Scott Worley <scottworley@scottworley.com>
Date: Thu, 22 Jun 2023 22:10:39 +0000 (-0700)
Subject: New runTest interface fixes useNixStoreImage test
X-Git-Url: http://git.scottworley.com/nixos-qemu-vm-isolation/commitdiff_plain/47cb654f45831f96a26819d1fcb39770662cd579

New runTest interface fixes useNixStoreImage test

The new runTest interface lets us turn off includeTestScriptReferences,
which lets the useNixStoreImage test pass.

Unfortunately, I then discovered another problem with useNixStoreImage:
It builds a new store image every time the VM starts up!  This is
undesirable for two reasons:

  1. It significantly delays VM start-up.  For network-service-providing
     VMs, this is downtime.  Longer startup time likely won't trouble
     high-availability services with fancy load balancers, health
     checks, and rolling restarts, but for unsophisticated use cases
     that currently just accept some small amount of downtime during VM
     restarts for software updates, using useNixStoreImage means much
     longer periods of interrupted service.

  2. This wastes disk I/O, building the same image over and over.

This useNixStoreImage design choice was made in order to keep the
guest's Nix store image out of the host's Nix store.  This is especially
important for the public Hydra that populates the public binary cache
that's currently hosted on centralized pay-per-byte commercial storage
services -- it doesn't make sense to pay to store and host the Nix store
itself and also many little copies of overlapping pieces of it.

So here in nixos-qemu-vm-isolation, we're happy to pay the cost of
storing the guests' Nix store images in the host's Nix store, because we
really value the faster start-up, and because we don't have the problem
of large storage service bills and no budget to pay them with.  :)
---

diff --git a/README.md b/README.md
index c0aea31..ec9ba70 100644
--- a/README.md
+++ b/README.md
@@ -3,5 +3,6 @@ squashfs for the VM's /nix/store that contains only the VM's dependencies
 (like the installer has) rather than a virtio mount of the host's entire
 /nix/store.
 
-**Update:** `virtualisation.useNixStoreImage` exists now!  But it doesn't
-work!  :(  See the note in `checks/mount-grep.nix`
+**Update:** `virtualisation.useNixStoreImage` exists now!  But it builds
+the store image at runtime, rather that at build-time, drastically
+increasing VM start-up time.
diff --git a/checks/mount-grep.nix b/checks/mount-grep.nix
index 90ae4a7..33478ea 100644
--- a/checks/mount-grep.nix
+++ b/checks/mount-grep.nix
@@ -3,6 +3,8 @@ pkgs: {
 
   hostPkgs = pkgs;
 
+  includeTestScriptReferences = false;
+
   nodes = {
     shared = _: { };
     private = _: { imports = [ ../modules/qemu-vm-isolation.nix ]; };
@@ -26,8 +28,6 @@ pkgs: {
 
     shared.succeed("[[ -e ${pkgs.pv} ]]")
     private.fail("[[ -e ${pkgs.pv} ]]")
-
-    # useNixStoreImage isn't ready until this works:
-    # useNixStoreImage.fail("[[ -e ${pkgs.pv} ]]")
+    useNixStoreImage.fail("[[ -e ${pkgs.pv} ]]")
   '';
 }