Alexa is the voice assistant that changed the game. Amazon made controlling your smart home with your voice accessible to everyone, but that convenience comes with a cost. Things you say to Alexa get sent to Amazon’s servers and may be listened to by Amazon contractors, but Home Assistant lets you set up truly local voice control.
What local means for a voice assistant
Nothing needs to leave your home
Almost every part of your interaction with Alexa involves the cloud. Wake word detection can happen on-device, but after that, whatever you say gets sent to Amazon’s servers so that the intent can be determined, and the relevant responses are sent from Amazon’s servers back to your home. If you’ve asked Alexa to control a smart device, these commands are often routed through third-party cloud services to make your lights turn on or turn up your thermostat.
It means that almost the entire process takes place outside your home, with speech-to-text (STT) processing that turns your spoken command into text, intent recognition, and text-to-speech (TTS) processing that turns a text response back into a spoken one all happening in the cloud.
A local voice assistant needs to handle all of these steps on your own hardware, so that no information has to leave your local network. The three key steps of STT, intent recognition, and TTS all need to happen at home.
- Brand
-
Seeed Studio
- CPU
-
ESP32-S3R8
The reSpeaker Lite Voice Assistant Kit includes a two-mic array, a pre-soldered XIAO ESP32-S3 controller, and an XMOS XU316 audio processor with onboard natural language understanding, interference cancellation, acoustic echo cancellation, noise suppression, and automatic gain control. Hooked up a 5W speaker, you can create your own local voice assistant that you can connect to Home Assistant via ESPHome.
Assist, Whisper, and Piper have you covered
Everything you need, running on your own hardware
Home Assistant has its own voice assistant called Assist. Assist can handle intent recognition to a limited extent, but the STT and TTS steps still need to be handled separately. Thankfully, there are two components designed to do exactly this.
Whisper is an add-on that runs a local implementation of OpenAI’s open-source speech-to-text model, making it possible to use speech recognition on home servers such as a Raspberry Pi or mini PC. It translates your spoken commands into text that Assist can understand.
You can also use Speech-to-Phrase, which is an alternative STT option that’s limited to a set of pre-trained phrases. This means it can run much faster, but it only supports a limited set of Assist’s commands.
Piper is a fast text-to-speech engine that converts the text generated by Assist and turns it into an audio file of speech that your smart speaker can play. The combination of these three components means that all the jobs that happen in the cloud with Alexa can be performed on your own hardware. How fast the responses are will depend on your hardware; on a Raspberry Pi 4, for example, you might have to wait several seconds for a response, but more powerful hardware or using Speech-to-Phrase can speed this up.
Assist can do the basics by default
Custom intents make it even more useful
Assist doesn’t use natural language processing to understand your intent, unlike AI chatbots. Instead, it uses a set of pre-defined sentences and intents for common smart home commands. You can say things like “turn on the living room light” or “close the kitchen blinds” or “pause the TV” and your smart home should respond accordingly.
You can also do things such as start timers, add things to a shopping list, and query the states of devices in your home. How quickly Assist will respond depends on how many entities you’ve exposed; limiting the number of exposed entities can help keep Assist more responsive.
If you want to use Assist to do something beyond the pre-defined commands, you can create your own. Home Assistant allows you to create your own custom sentences and intents which Assist can use. As long as you create enough custom sentences to capture how you phrase your commands, you should be able to get these to work just like the predefined sentences.
Don’t expect a perfect Alexa replacement
Privacy comes at a cost
If you’re hoping for a completely local version of Alexa that can do everything Amazon’s voice assistant can do, then you’re out of luck. Running everything on your hardware comes with some drawbacks.
On some hardware, response times will be slow. You may not get the instant responses that Alexa normally provides. In addition, you won’t get all the capabilities that Alexa has. You can’t ask for sports scores or what the capital city of Burkina Faso is, without using an LLM to give Assist additional smarts, which may require either a more powerful local machine or an internet-connected cloud provider.
Playing music can also be a challenge. I use Music Assistant, and I had to build my own set of custom sentences and intents to be able to play music just by using my voice, but with a little effort you can get it to work.
Assist is far from perfect
There’s a lot you can do using Assist without the help of an LLM. For the typical voice assistant functions like turning on your lights, setting timers, and powering up a smart plug, Assist can do the job well enough. While it’s not a perfect Alexa replacement, if you value your privacy, it’s definitely worth giving Assist a try.
Stephan is the sports journalist for the Maple Grove Report.
