There are plenty of benefits to running local LLMs, not least of which is that you don’t have to share all your personal data with an AI company. The problem is that even fairly small LLMs run slowly on my mini PC or MacBook Air. Despite this, there are plenty of ways that I use local LLMs every day.

Generating morning briefings

Keeping the family informed of the day’s activities

Credit: Adam Davidson / How-To Geek

This is one of my favorite uses of a local LLM. Every morning when we enter the kitchen, a motion sensor is triggered, and an announcement plays on the smart speaker. It gives a morning briefing including what the kids have for lunch at school, which after-school activities they have, what’s on the calendar for the adults, a reminder if the trash needs to be taken out, and a brief weather report.

All of the data is pulled from various sources in Home Assistant, and while Home Assistant can turn that data into a basic templated message, it can’t easily transform it into a natural language briefing. That’s where the local LLM takes over. The key information is passed to Qwen3-4B running in Ollama on my mini PC, which creates a natural language briefing.

Generating the briefing and turning it into a spoken announcement using text-to-speech (TTS) is slow on my hardware. It can take around 10 minutes to complete, depending on how much information the briefing includes. The slow generation isn’t a problem, because the briefing is generated at 5 AM each day, so it’s always ready to play the moment we step into the kitchen.

CPU

Celeron FCBGA1264 3.6GHz

Graphics

Integrated Intel Graphics 24EUs 1000MHz

The Beelink Mini S13 Pro desktop PC is a ultra-compact computer powered by the Intel N150 processor. Shipping with 16GB of DDR4 RAM and a 500GB SSD, this micro desktop is perfect for a variety of workloads. From running simple server programs to replacing your old PC, the Beelink S13 Pro is up to the task. 

I start the day with a summary on my Kindle

I use the same local model for another automation that requires turning information into a specific format. This one takes the top stories from several RSS feeds and turns them into a newsletter that I can read on my Kindle.

The top three stories from each feed are passed to the LLM, which then chooses five of them to turn into a newsletter. It creates a written summary of a few hundred words for each of the stories, as well as a TL;DR covering all five stories. I can then start each day with a summary of news I’m interested in and read the full articles of any stories that grab my interest.

Stripping personal information from documents

No need to send sensitive information to the cloud

Credit: ANDRANIK HAKOBYAN/Shutterstock.com

I tried using a local LLM to analyze my financial data so it could look for potential savings or areas where I’m overspending. With the models that I can run on my hardware, however, the results were fairly useless.

A powerful cloud-based LLM could do a much better job. The problem was that I didn’t want to upload this type of sensitive data to Claude or ChatGPT, so it ended up on a third-party server somewhere.

I finally figured out the ideal solution. I used the local LLM to strip all of the sensitive data from my financial documents, such as names, locations, and account numbers. Once I was satisfied that the sensitive data had been removed, which took several passes and manual verification, I was then comfortable uploading the information to Claude, and it came up with some useful suggestions for how to reduce spending.

Recommending books to read

You don’t always need a chat model

I love reading, but I often struggle to find recommendations for books that really hit the spot. I’ve found Goodreads recommendations are almost always disappointing, so I decided to see if I could build a local recommendation system of my own, based on my Goodreads reading history.

This is one of the more interesting things I’ve done using a local AI model because it doesn’t use a typical chat model at all. Instead, it uses a small embedding model called nomic-embed-text which converts descriptions of books into numerical vectors. These vectors can then be compared so that the system can determine if the vector of a potential book suggestion matches the vectors of other books that I’ve enjoyed.

It’s not perfect, but it usually uncovers at least one or two genuinely good suggestions each time it runs. It takes some time to run, but it’s not something that needs an instant response, so the slow response isn’t an issue.

Reminding me to clear up the kitchen

Fresh messages are harder to ignore

Credit: Chris Hachey / How-To Geek

I’ve used spoken notifications in my smart home for a long time. Having your smart speaker nag you to do things can work well, but if the message is always the same each time, it becomes easier to ignore.

When I looked at the data from my kitchen motion sensor, I discovered that my wife or I enter the kitchen about 50 times a day while the kids are at school. I realized that if we picked up just a single thing each time we were there, then keeping the kitchen clean would become a much easier job.

I use a local LLM to generate multiple different announcements that play through the smart speaker, reminding us to clear up a little whenever we enter the kitchen. These are intended to be humorous and written in different personalities and voices. It works well, as you never know what the speaker is going to say, and this is usually enough for the message to sink in.

Since generating these announcements is very slow on my hardware, the whole list is generated overnight. It means that there are always fresh announcements to play whenever we enter the kitchen.

Slow generation doesn’t always matter

The small local LLMs that run on my hardware are never going to match the capabilities of ChatGPT, Gemini, or Claude. Even fairly simple tasks take a long time to complete. In many instances, however, I don’t need an immediate response; by running the prompts when I’m asleep, the responses are always ready for when I need them.