Configuring Home Assistant to run off of a USB drive in a incompatible Raspberry Pi

 A problem with the earlier generations of Raspberry Pi's, especially the Pi 1 and the Pi 2 up to version 1.1, is that these cannot be configured to boot from an external USB storage device.

This is a particular relevant limitation for a number of reasons, including the fact that relying on an SD card for most of the storage needs is a solution that may have limited endurance.

The later versions of the Raspberry Pi (RPi 2 v1.2 and upwards), already offer some form of allowing external USB media to boot the operating system. 

 

This is normally not enabled from the factory, and the user has to first boot the RPi from a MicroSD card with a flag set in the config.txt file (see https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md).

After the flag is passed (program_usb_boot_mode=1), during boot the RPi will burn the flag into the OTP memory that is built inside the Broadcom SoC. Once this is done, it cannot be reverted. This is in most part harmless, but the user should take into account that if a USB storage device is present during boot, the RPi will take longer to start because now it will first initialize and search on the USB drive before booting up from the MicroSD card. This flag also enables the ability to network boot in some devices.

In my case however, the RPi I had available was the RPi 2 v1.1 and as such did not have this feature implemented.

Still I was convinced that there had to be another solution: in a regular computer, we are able to configure a bootloader or some form of boot related routines to start the OS from a different partition or block device. While I struggled to find clear instructions on how to split the boot process (and OS installation) between the MicroSD card and the external USB drive for the older RPi's, I could however find reports of  RPi 4 users (because initially boot from USB was not yet ironed out in the software supporting this version of the RPi), seemed that (at least at a first glance) they were having success with the datactl command of the HassOS. This command would allow transferring the installation from one block device to another.

And so I pursued that route. Instead of using my original Hass.io installation SD card, I decided to:

•  take a full snapshot of  Home Assistant (using the lovelace web UI);
• flash a new image of Hass.io into a new SD card (using balenaEtcher);
• run the RPi 2 with the new SD card;
• once started, choose to restore HA from my snapshot instead of setting up the instance with a new user (this is a new feature of HA - in the past this would be a harder process);
• check the restored installation and make sure everything would be running as before;

And then comes the most interesting part: in order to transfer the installation to the SSD drive, I had to connect it, and restart the RPi with a monitor and keyboard attached to it. Once a login prompt appears, the user enters root, and a password (if it had been previously set). This is the debug mode shell. In order to enter the regular bash shell for the root user, you need to type login and hit enter:

 

 _    _                                         _     _              _   
| |  | |                          /\           (_)   | |            | |  
| |__| | ___  _ __ ___   ___     /  \   ___ ___ _ ___| |_ __ _ _ __ | |_
|  __  |/ _ \| '_ ` _ \ / _ \   / /\ \ / __/ __| / __| __/ _` | '_ \| __|
| |  | | (_) | | | | | |  __/  / ____ \\__ \__ \ \__ \ || (_| | | | | |_
|_|  |_|\___/|_| |_| |_|\___| /_/    \_\___/___/_|___/\__\__,_|_| |_|\__|
                                                                         
Welcome on Home Assistant command line.

For more details use 'help' and 'exit' to close.
If you need access to host system use 'login'.

ha > login

With the SSD drive running - make sure is sufficiently powered - it is possible that the RPi is unable to provide sufficient power to these power hungry USB devices. In my case, I have selected to use a powered USB hub between the SSD and the USB host port. This ensures that there is sufficient power for both the RPi and the SSD drive.

Once the drive is recognized - you can check by running the "lsblk" command, and look for a device named /dev/sda or similar:

 

# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 447.1G  0 disk
`-sda1        8:1    0 447.1G  0 part /mnt/data
sdb           8:16   1   7.5G  0 disk
`-sdb1        8:17   1   7.5G  0 part
mmcblk0     179:0    0  59.5G  0 disk
|-mmcblk0p1 179:1    0    32M  0 part /mnt/boot
|-mmcblk0p2 179:2    0    24M  0 part
|-mmcblk0p3 179:3    0   256M  0 part /
|-mmcblk0p4 179:4    0    24M  0 part
|-mmcblk0p5 179:5    0   256M  0 part
|-mmcblk0p6 179:6    0     8M  0 part
`-mmcblk0p7 179:7    0    96M  0 part /mnt/overlay
zram0       254:0    0   231M  0 disk [SWAP]
zram1       254:1    0    32M  0 disk /var
zram2       254:2    0    16M  0 disk /tmp
 

You should make sure the SSD card has no partitions. If it has any, delete these. You can do it by running the "fdisk /dev/sda" command, and deleting the partitions one by one (by typing "d" followed by the number of the partition). You can list which partitions are present by typing the "p" command:

 

# fdisk /dev/sda

Welcome to fdisk (util-linux 2.35.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): m

Help:

  GPT
   M   enter protective/hybrid MBR

  Generic
   d   delete a partition
   F   list free unpartitioned space
   l   list known partition types
   n   add a new partition
   p   print the partition table
   t   change a partition type
   v   verify the partition table
   i   print information about a partition

  Misc
   m   print this menu
   x   extra functionality (experts only)

  Script
   I   load disk layout from sfdisk script file
   O   dump disk layout to sfdisk script file

  Save & Exit
   w   write table to disk and exit
   q   quit without saving changes

  Create a new label
   g   create a new empty GPT partition table
   G   create a new empty SGI (IRIX) partition table
   o   create a new empty DOS partition table
   s   create a new empty Sun partition table


Command (m for help):
 

Once ready, type:

# datactl move /dev/sda

followed by:

# reboot

The datactl command is fully documented here - https://github.com/home-assistant/operating-system/blob/dev/Documentation/partition.md

Once it restarts and the USB drive is the detected, HassIO will take care of transferring the installation to the SSD drive. In my setup the command took about 45 minutes to complete.

After this process is complete, the /mnt/data (the only r/w volume in HassIO) should be fully transferred to the SSD drive, and the SD card will only be used for booting up the kernel and few more than that. As it will be treated mostly as a read-only device, there should practically be no future wear on the SD card. 

 

Even though in my case, with an old RPi 2, and only USB 2.0 available,  I am unable to take advantage of the full speed of the SSD drive, still the performance would be slightly superior than with an SD card.

 

By performing a test with hdparm I have obtained the following read performance:

# hdparm -tT /dev/sda

/dev/sda:
Timing buffer-cache reads:   312 MB in 0.51 seconds = 623387 kB/s
Timing buffered disk reads:   79 MB in 3.02 seconds = 26718 kB/s

Comparatively, with the SD card I have obtained:

# hdparm -tT /dev/mmcblk0

/dev/mmcblk0:
Timing buffer-cache reads:   308 MB in 0.51 seconds = 615332 kB/s
Timing buffered disk reads:   67 MB in 3.01 seconds = 22759 kB/s

I have repeated the tests and the results were in the same range.

So far I have been using this configuration for a few days and the system have been performing nicely.

In a next post I will explain what I have done regarding the continued operation in case of power outages, and how it all integrates with Home Assistant itself.