Far too often, I think of something to add to my to-do list, but then by the time I’ve gotten around to opening the list and adding the tasks, I’ve forgotten what it was. I decided to see if I could quickly add tasks to a to-do list with my voice by using Home Assistant.
Why Home Assistant’s built-in commands aren’t enough
The default adds tasks to the shopping list
Home Assistant has its own voice assistant called Assist. By default, Assist relies on predefined sentence patterns rather than an LLM to interpret what you mean. Instead, it looks for matches between the command that you speak and a file containing predefined sentence templates.
If a match is found, the appropriate action is taken, but if the command doesn’t match one of the predefined sentences, Assist won’t understand the command. It can work well for basic smart home controls that use common patterns, such as “turn on the bedroom light” or “start a five-minute timer.”
The predefined sentences include commands such as “add X to my list.” Unfortunately, the default action for this command is to add something to the native Home Assistant shopping list. While this is really useful for using Assist to add things to your shopping list using your voice, it was of no use for my intended purpose, so I needed to do some customization.
You can hook up Assist to an LLM, which can help it determine your intent using natural language processing. I wanted to see if I could get this working without the help of an LLM.
- Brand
-
Seeed Studio
- CPU
-
ESP32-S3R8
The reSpeaker Lite Voice Assistant Kit includes a two-mic array, a pre-soldered XIAO ESP32-S3 controller, and an XMOS XU316 audio processor with onboard natural language understanding, interference cancellation, acoustic echo cancellation, noise suppression, and automatic gain control. Hooked up a 5W speaker, you can create your own local voice assistant that you can connect to Home Assistant via ESPHome.
Using custom sentences in Home Assistant
Teaching Assist how you actually speak
As well as the predefined sentences that Assist uses to match what you’ve said to the right commands, you can also set up your own custom sentences. This gives Assist additional sentences to match against your spoken commands. For example, I could add a custom sentence such as “add [task] to my important tasks”.
These custom sentences can then be referenced by an intent script, which translates the matched command into a specific action. In this case, the intent would be to create a new to-do item using the [task] data.
This allows me to decide exactly how I want to ask for to-do list items to be added, and exactly how those tasks are then added to the appropriate list. It lets me create my own custom voice commands and define what those commands do.
5 Home Assistant integrations that make my smart home feel like it came from the future
The future is now, old man.
Generating sentence variations is now a lot easier
An AI chatbot can do the hard work
The problem with using custom sentences is that we don’t always ask for things in the same way. I might not remember that I need to use the exact phrase “add [task] to my important tasks,” and I might say something such as “put [task] on my important task list” instead. Since this sentence doesn’t match the pattern of the custom sentence, Assist won’t recognize it, and it won’t add anything to my to-do list.
A simple way to solve this problem is to add variations of how you might say the phrase to your custom sentences. I can add “put [task] on my important task list” to the list of custom sentences and have the intent script perform the same action if that phrase is recognized. I can then say either phrase to add an item to my to-do list.
The more variations I add, the more different phrasings I can cover, but thinking up all the different options, typing them out, and pasting them into Home Assistant can get pretty tedious. Thankfully, these days there’s a much easier way.
This is the kind of text-based task that an LLM is perfect for. I asked Claude to generate alternative ways of asking to add tasks to my important task list, gave it the format required for custom sentences, and it generated a ton of different variations that I then pasted into Home Assistant.
Custom sentences aren’t perfect
They’re good enough for what I wanted
On the whole, the system works really well. I can say, “Hey Jarvis, add ‘write an article about setting up voice control for my to-do list’ to my important tasks,” and remarkably, it will understand that the entire wall of text in the middle of that request is the task that I want to add.
When I use alternative phrasing, it usually works; I don’t have to worry about remembering any specific way of asking. It’s not perfect, however. Occasionally, it will fail if the phrase I used is one that Claude and I both failed to think of, but the vast majority of the time, it does exactly what I need.
Local voice control doesn’t have to rely on AI
Without the help of AI, local voice assistants still aren’t as capable as cloud-based options such as Alexa. If you want them to do specific tasks, however, then they can do the job well. It takes a little more setup, but the results can be solid. Now what was it I was going to add to my to-do list again?
Stephan is the sports journalist for the Maple Grove Report.
