Hardware Setup

Allow me to start this article by borrowing from Lemony Snicket’s introduction to A Series of Unfortunate Events:

“If you are interested in stories with happy endings, you would be better off reading some other [article]. In this [article], not only is there no happy ending, there is no happy beginning and very few happy things in the middle. This is because not very many happy things happened in the lives of [IoT hobbyists]. [IoT hobbyists] were intelligent children, and they were charming, and resourceful, and had pleasant facial features, but they were extremely unlucky, and most everything that happened to them was rife with misfortune, misery, and despair. I'm sorry to tell you this, but that is how the story goes.”

That might be slightly over-dramatic, but it very nearly captures the experience of navigating the network challenges required to turn my Home Assistant device from a paper-weight to something more useful.

As mentioned in the last article of this series, I’m working with a Home Assistant Voice Preview Edition and a MINISFORUM UN100P mini PC. Although I’m generally more used to working on Mac’s or on Linux, the mini PC came with Windows 11 Pro pre-installed, so that’s the OS we’ll be working with.

The setup for this splits into two main parts:

1. Install the Home Assistant OS (henceforth called HAOS) onto a VM on the mini PC

2. Make sure the Voice Preview Edition (henceforth called the Voice PE) can communicate with the HAOS

Step 1: Install the Home Assistant OS

The installation of Home Assistant OS on my miniPC was fairly straightforward. The HAOS is a “bare metal OS” - ie it’s designed to run directly on hardware as the only operating system. In order to preserve the ability of the miniPC to do other stuff, like run scripts necessary for the “agent” portion of this project, I had to install the HAOS on a VM instead. Since my miniPC came with Windows 11 Pro already installed, I opted to use Hyper-V.

There were a few steps to setting this up that I’ll spare the gory details of, but at a high level the steps were:

1. I had to set up a Virtual Switch to enable my Voice PE to make calls to the VM. Initially, I configured this as an external connection via my miniPCs WiFi adapter. In other words, my miniPC should be able to receive packets over WiFi that are addressed to either the miniPC or the VM running on the PC (with HAOS on it).

2. After setting up the Virtual Switch, I configured a Hyper-V VM to run HAOS. This basically entailed downloading a .vhdx file from Home Assistant, configuring it as the virtual hard disk, setting my the VMs network adapter to be the Virtual Switch I configured in Step 1, and giving the VM the appropriate resources (4 GB RAM, 2 virtual processors).

With those steps completed, I could spin up my VM, and access either the CLI via a Virtual Machine Connection, or the UI via browser by navigating to http://homeassistant.local:<port_number>.

The one snag with this was that Chrome seemed to have a hard time accessing the VM, but Microsoft Edge worked fine.

Step 2: Ensure HAOS and the Voice PE can Communicate

Setting up the initial Voice PE <> HAOS connection was pretty simple as well. The Home Assistant iOS app makes it straightforward to sign into your Home Assistant account, discover your Voice PE via Bluetooth, and connect it to your WiFi network.

On the Home Assistant OS, you can configure a Voice Assistant and “assign” it to your Voice PE. This is what gives your Voice PE the ability to handle speech-to-text and text-to-speech, in collaboration with the HOAS. You can also configure network settings so that your Voice PE knows where to route it’s calls (seemingly the “automatic” option there doesn’t work very well with VM-hosted setups, so reader be-ware).

So, ta-da, things should be all set up! I can say my wake word (“Okay, Nabu”), my Voice PE gives me a chime to let me know it’s listening, I ask it “What time is it?”, and it gives me the time! Only, it didn’t do this every time. Sometimes it would do it, sometimes it wouldn’t, and there didn’t really seem to be any rhyme or reason as to when it would/wouldn’t work. Hmmm.

Cracking the case

After hours of double checking IP addresses, restarting the miniPC and the Voice PE, and trying to find ways to debug the datastream via tools in HAOS, one suspect in particular came to light. The Virtual Switch! Although it wasn’t really the Virtual Switch’s fault. Seemingly, it’s more of an artifact of how routers and WiFi adapters handle MAC addresses.

Basically, most WiFi adapters are configured to only accept packets over wifi that are addressed to the MAC address of whatever computer that WiFi adapter is sitting on. And, most routers maintain mappings of IP addresses to MAC addresses, relying on consistent presentation of MAC addresses on the network. The issue here is that when we set up our Virtual Switch to leverage the miniPCs WiFi adapter, we’re basically telling the WiFi adapter “hey - instead of just keeping an eye out for packets that are addressed to this miniPC, keep an eye out for all packets sent out on this network, and if you see one addressed to the miniPC or the VM, pick it up.

This approach - having a WiFi Adapter monitor all packets on a network, is called “promiscuous mode”, and evidently it’s not very well handled by standard consumer PCs and routers. As such, when I gave the Voice PE a command, it would try to reach out to HAOS to process that command, but because of routing issues the command would never make it to HAOS - therefore no response.

The solution is to switch from a WiFi connection based Virtual Switch to an ethernet based Virtual Switch. Ethernet adaptors are capable of seeing packets sent to all MAC addresses over the wire, and so they have no trouble picking up data for multiple MAC addresses and routing them appropriately.

I had Claude mock up this nice diagram showing the different approaches:

After I hooked my miniPC up to my router via an ethernet cable, I consistently and reliably got audio output from my Voice PE. Up next - enabling my Voice Assistant in HAOS to make calls to my agent service on my miniPC.