I’ve recently found myself needing a new VPS and I heard good things about the generous Oracle Cloud free tier. Oracle does not offer a NixOS image, and I run all my machines under NixOS so I figured I’d document the process of nixifying the machine.

I first tried using the NixOS LUSTRATE approach but that limits you to the partition setup offered in the base image. This is a problem as none of the images offered had a partition layout I liked (and beware: the Ubuntu image’s boot partition is tiny, and NixOS will fill it way too fast). So instead, we’re following the kexec approach.

Create your machine

  1. Register for Oracle Cloud always-free tier. Yes it takes a credit card, but I really don’t think it will charge me. The “always free tier” model apparently requires an explicit opt-in to access paid features, which is honestly really cool.

  2. Create an instance. I’m using image Ubuntu 22.04.

    For the shape, I pick VM.Standard.A1.Flex with 4 OCPU and 24GB of memory. This is the maximum allowed under the very generous free tier. Depending on your region and on machine availability, I’m told this may not be possible, but I had no problems getting a machine allocated (in the Zürich region).

    • Don’t forget to add an SSH key.
    • Keep that in mind while sizing your disk. Oracle gives you up to 2 block storage devices for a total of 200GB at up to 20VPU of performance in the always free tier.

Install Nix on the Ubuntu machine

ssh ubuntu@${ip?}

We’re following https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation and it’s pretty easy:

$ sh <(curl -L https://nixos.org/nix/install) --daemon  # this will prompt you for a few choices, answer n, y, y

You can then logout and SSH again to get a fresh Nix-enabled shell.

Prepare an in-memory NixOS system to perform the installation from

We’re now following https://nixos.wiki/wiki/Install_NixOS_on_a_Server_With_a_Different_Filesystem and it’s a little more advanced:

$ git clone https://github.com/cleverca22/nix-tests.git
$ cd nix-tests/kexec
$ nano myconfig.nix

You’ll want a config like:

{
  imports = [
    ./configuration.nix
  ];

  # Make it use predictable interface names starting with eth0
  boot.kernelParams = [ "net.ifnames=0" ];

  networking.useDHCP = true;

  kexec.autoReboot = false;

  users.users.root.openssh.authorizedKeys.keys = [
    "ssh-rsa AAAAredactedZZZ korfuri@kelyus"
  ];
}

Don’t forget to replace the SSH key in this example with yours.

You can now build an entire kexec-capable NixOS system from this config with:

nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./myconfig.nix

It may be possible to speed up the build process for this system, as it spends quite a bit of time building documentation, support for ZFS, etc.

Untar the resulting tarball and run kexec. This will take a while, but you’ll end up seeing a line that says + kexec -e. Yeah, it will look like it’s stuck for a good 5-10min, but be patient. Note that this is + kexec -e, not + kexec -l which will show up earlier.

$ tar -xf ./result/tarball/nixos-system-aarch64-linux.tar.xz
$ sudo ./kexec_nixos

Once you’ve seen that + kexec -e line, you can terminate your SSH connection (the server won’t reply, just hit enter, ~, . to tell the client to close everything. The server will come back with a different host key, so you’ll want to clean up your known_hosts before sshing again:

$ ssh-keygen -R ${ip?}
$ ssh root@${ip?}

Installing NixOS to the disk

Now we’re root on an im-memory NixOS system. If we rebooted the machine at this point, it would boot back into Ubuntu. We don’t want that, we need to install NixOS on disk, so we’ll immediately start deleting the existing partitions and mold the disk to our liking. You don’t have to follow the same partition layout as I have here, of course.

# parted
(parted) rm 1
(parted) rm 15
(parted) mkpart
Partition name?  []? boot
File system type?  [ext2]? fat32
Start? 2048s
End? 10GB
(parted) print all
Model: ORACLE BlockVolume (scsi)
Disk /dev/sda: 50.0GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  10.0GB  9999MB  fat32        boot  msftdata


(parted) set 1 boot on
(parted) set 1 esp on
(parted) mkpart
Partition name?  []?
File system type?  [ext2]? ext4
Start? 10GB
End? -1s
Warning: You requested a partition from 10.0GB to 50.0GB (sectors 19531250..97677311).
The closest location we can manage is 10.0GB to 50.0GB (sectors 19531776..97677278).
Is this still acceptable to you?
Yes/No? yes
(parted) quit

We can now make filesystems on our brand new partitions, mount them, and configure NixOS to use them:

# mkfs.fat -F 32 /dev/sda1
# mkfs.ext4 /dev/sda2
# mkdir -p /mnt/boot
# mount /dev/sda2 /mnt/
# mount /dev/sda1 /mnt/boot/
# nixos-generate-config --root /mnt
# nano /mnt/etc/nixos/configuration.nix   # You want to at least set openssh.enable = true; and add an ssh key for root, like we did in the temporary system above.

When you’re ready, just run:

# nixos-install

You can now reboot into your NixOS setup. You’ll have to use ssh-keygen -R ${ip?} once again to trust the new host key.

$ nix-shell -p neofetch --run 'neofetch --backend off'`
root@boraha
-----------
OS: NixOS 22.05 (Quokka) aarch64
Host: KVM Virtual Machine virt-4.2
Kernel: 5.15.63
Uptime: 4 mins
Packages: 323 (nix-system) 
Shell: bash 5.1.16 
Resolution: 1024x768 
Terminal: /dev/pts/0 
CPU: (4) 
GPU: Red Hat, Inc. Virtio GPU 
Memory: 500MiB / 23982MiB

Edited 2022-10-01: clarified a few steps and fixed a couple of misleading instructions. Thanks to noah for their corrections!