It’s amazing how many of the futuristic features that I saw in sci-fi shows growing up already exist. Video calling and talking watches seemed like the stuff of fantasy, but now they’re so ordinary we barely register them. The same is true of smart home tech; there are plenty of Home Assistant integrations that often make me feel like I’m living in the future.

Cloud conversation agents

Make your smart home artificially intelligent

Credit: Adam Davidson / How-To Geek

When AI chatbots first burst onto the scene, they seemed incredible. You could have natural language conversations with an LLM that were just like talking to a real human. We’ve already reached the point where we’re no longer really impressed by this incredible technology.

Hooking up a cloud-based LLM to Home Assistant using an integration such as OpenAI Conversation can make your smart home feel like you’re living on the bridge of the Starship Enterprise. It gives the Assist voice assistant the brains of a powerful LLM, allowing you to use natural language to control your smart home.

Having built my own Home Assistant-powered smart speaker, I can say “Okay, Nabu, it’s too dark in here,” and the LLM will interpret my intent, figure out what command it needs to send to Home Assistant, and my light will turn on. I didn’t have to ask for the light to be turned on directly; the LLM figured out what I wanted and made it happen. It’s all I can do to stop myself from asking Assist to initiate saucer separation, because I’m afraid of what might happen.

Brand

Seeed Studio

CPU

ESP32-S3R8

The reSpeaker Lite Voice Assistant Kit includes a two-mic array, a pre-soldered XIAO ESP32-S3 controller, and an XMOS XU316 audio processor with onboard natural language understanding, interference cancellation, acoustic echo cancellation, noise suppression, and automatic gain control. Hooked up a 5W speaker, you can create your own local voice assistant that you can connect to Home Assistant via ESPHome.

Ollama

AI smarts that are completely local

Credit: Corbin Davenport / How-To Geek / Ollama

While using OpenAI Conversation is amazing, it does have one major downside. In order to work, your request, along with information about your smart home, has to be sent to the cloud-based LLM that you’re using as the conversation agent. This isn’t really in alignment with the privacy-based focus of Home Assistant.

The good news is that cloud-based LLMs aren’t the only option. You can use an integration such as Ollama to run a local LLM on your own hardware and use that as your conversation agent instead.

The reality is that unless you have suitable hardware, a local LLM isn’t going to match the performance of a cloud-based LLM. It may be slow to respond or unable to handle more complex tasks. If you have decent hardware, however, you can run your own local conversation agent which not only maintains your privacy but also means you won’t have to pay any API fees.

LLM Vision

AI-powered image and video analysis

Smart home security cameras are useful, but a lot of the most useful features are hidden behind paywalls. For example, if you want to go beyond simple motion detection to have your cameras tell you when they detect a person, pet, or package, you often have to pay for the privilege.

With the LLM Vision integration, you can cut out the middleman and analyze snapshots from your smart cameras yourself. LLM Vision sends the snapshots to a vision-capable LLM that can analyze the image and send back a natural language description of what it sees.

For example, when your video doorbell detects a package, that doesn’t tell you where that package has ended up. You can create an automation that sends a snapshot from your doorbell to an AI model which can analyze the image and tell you exactly where the package has ended up. If it’s out in the open rather than hidden from view of the street, you can get an alert telling you that your package is at risk from porch pirates.

Frigate

Local object detection

Once again, a major issue with LLM Vision is that it requires you to send images from your cameras to a third-party AI service. If you’re using cameras inside your home, you may not be completely comfortable with those images ending up on third-party servers. The good news is that there is another option.

Frigate can give you completely local object detection. Frigate is NVR software that can run object detection locally using a neural network, and there’s a useful Home Assistant integration. You can use a device such as a Google Coral TPU for acceleration to make the process faster.

Depending on the model you use, Frigate can detect things such as people, different types of vehicles, different animals, and common street or household items such as potted plants or traffic lights. Recent versions also include native face recognition, allowing you to set up alerts to tell you who is at the door without having to send photos of people’s faces to the cloud.

You can even train custom classification models to recognize object states outside of what it already knows. For example, you can train it to understand when your pet’s water bowl is full, empty, or missing and use Home Assistant to alert you when it needs filling again. If that’s not living in the future I don’t know what is.

Adaptive lighting

Lights that change by themselves

Credit: Nanoleaf

This is something that feels both futuristic and timeless. Using the Adaptive Lighting custom component, you can have the smart lights in your home automatically change brightness and color temperature throughout the day to mimic the natural light of the sun.

In the morning, they’re fairly dim and warm, and gradually become brighter and cooler toward midday, then slowly dim and get warmer toward sunset. This is exactly what happens to the lights on the Enterprise in Star Trek: The Next Generation, with the lights dimming during the night shift to maintain the crew’s circadian rhythm in deep space. Now you can have the same in your own home with a Raspberry Pi and a few smart bulbs. What a time to be alive.

The future is already here

If you ever stop to think about it for a moment, the things we can do with our smart homes are truly insane. It’s easy to take some of the features for granted, so why not take a moment to stop and appreciate just how awesome your smart home really is?