I love Raspbery Pis - I have done since they first released them, all the way up to the present iteration, the Raspberry Pi 4. They’re phenominal little bits of kit, endlessly hackable and because of their really low price, they open up computing to a huge number of people who otherwise wouldn’t get the opportunity.
One of the irritating things, though, about running Raspberry Pis is that they typically boot from an SD card. SD cards are notorious for having high attrition rates, especially if you’re reading/writing to them regularly. Running a Kubernetes cluster on my Raspberry Pis is especially hard on the SD card because you’re not only running the system’s read and writes, but you’re also running containers and their I/O on the card.
PXE booting the Raspberry Pis will allow me to back the OS and containers with my Synology NAS, and to forget the headache of SD card fatigue! There are three steps to configuring PXE boot for my Pis:
In order for the Raspberry Pi to know where to boot from, we need too configure the DHCP server with some specific options. In my home network DHCP is running from my Ubiquiti EdgeRouter Pro 8. DHCP Option 43 is used to exchange “vendor-specific information” - in this case we need to tell the Raspbery Pi that we have a PXE boot available for it. On my network I’ve configured a DHCP server for RPi called VLAN020.
In the WebUI, it looks like this:
I’ve configured two shared folders on my Synology NAS -
rpi-pxe is shared as an NFS export, with read/write permissions to mount from my Raspbery Pi subnet. Since we’re using this as a Linux boot device, we must not enable root squash (No mapping) and it should be synchronous.
rpi-pxe share I have created a new folder for each of my Pis and used
chmod 777 to enable full permissions on the share. Each Pi will copy it’s root file system into it’s respective folder, and boot from there.
rpi-tftproot is shared as an NFS share in the same way, but also as the TFTP server using the File Services settings on the Synology. The folders and content required will be created later on.
Now that the infrastucture is in place to PXE boot the Pis, it’s time to configure the Pis themselves. I’m assuming you’ve got a Pi already running and configured - this can be a fresh install of Rasbian or one that’s been running a while - it doesn’t much matter. I performed these steps on all of my Pis, from the SD booted install.
Several of the steps need to use the Pi’s serial number, so make a note of the output:
Firstly, we need to configure the EEPROM to boot from the network - by default it will only boot from the SD card.
Once the Pi has rebooted we need to copy the root partition to the NFS server. I created a directory in the
rpi-pxe share for the Pi
rpi4-01 and ensured the write permissions were 777. You might need to install
sudo apt-get install rsync.
Once the copy has completed (grab a coffee, it’ll take a while) edit the
/etc/fstab in the NFS share (so
/nfs/rpi4-01/etc/fstab for me) and remove the existing configuration (only the
proc line should be left). Mount the TFTP boot share as the
/boot partition - the
fstab file should look like this:
To create a bootable TFTP folder, we need to copy the contents of
/boot to the TFTP server. To do this I simply mounted the
rpi-tftpboot folder as an NFS share in my Pi. The Raspberry Pi 4 bootloader will look for bootable files in a folder corresponding to its serial in the TFTP root directory. In the example below I am creating a folder based on the serial and copying the boot files there. This allows a
cmdline.txt for each Pi, which lets me specify where the root of the OS is. It also means that I can update the boot files for each Pi individually.
Next edit the
cmdline.txt file in the new TFTP boot directory and replace the root partition with the NFS partition.
All being well, you can now shutdown the Pi, remove the SD card and boot via PXE!
If you encounter problems booting, double check the steps have been completed correctly:
vcgencmd bootloader_configand make sure the
BOOT_ORDERis set to
tail -f /var/log/opentftp.log