I used to wish that I had an awesome supercomputer so I could run my own AI, but then I found out that I didn’t have to. Just like with other DIY projects, you can make a server with what you already have; you just need to think outside the box. If you own a spare PC, can find one, or can buy one cheaply, you’re one big step closer to an AI server. What’s even better is that LM Studio was built to make this easier for you.

You can get great performance out of cheap gear

I didn’t spend any money before doing this

Credit: Jorge Aguilar / How To Geek

Setting up a private AI server usually sounds like a project that needs a huge budget, but AI doesn’t have to be expensive. The software can run well on low-cost hardware because it manages system resources very well by letting you offload processing to older CPUs or integrated graphics. When you use the headless daemon called llmster, you run the main language model engine without the graphical user interface.

This frees up system memory and processing cycles for the actual work of generating text. You don’t need to spend thousands of dollars on a new computer to get a private environment. Instead of buying high-end dedicated graphics cards, you can rely on older hardware. My $200 machine can run the Qwen2.5-Coder-3B model since the software doesn’t need much overhead to work. The software works with a lot of different setups because it’s built to run on normal processors and integrated graphics.

I bought my PC years ago, and it was old to begin with, so you could probably beat my rig with any Raspberry Pi. The Qwen2.5-Coder-3B model is a good choice for programming tasks, and its size fits well into machines with limited RAM. With four-bit quantization, the model file shrinks to about two gigabytes, meaning it leaves plenty of room for your context window without crashing the system.

It doesn’t need a dedicated GPU to give you a good response speed for coding tasks. You get steady token generation that easily goes faster than your natural reading speed, making it a good assistant for drafting functions, explaining logic, or catching syntax errors. If you decide to use one of your old PCs, keep in mind that you’re going to give up any use of it for as long as it acts as your server.

This is a pretty serious decision. When I decided to do this, I made sure to wait at least a few days after picking which PC I would use for a server. That way, there are no regrets.

You can run a quiet server in the background

You don’t need to be a hacker to have a server

Setting up your old hardware to work as a dedicated background processor starts with getting the right software. You need to download the correct version of LM Studio for your specific operating system, like Windows, macOS, or Linux. Just make sure the operating system you have is lightweight because you won’t need one taking up precious CPU power.

If you need an older version for your PC, they’re not hard to manage. I actually tried out three versions before sticking to one. Also, make sure you have your connection method ready.

Once you install the application on your target machine, you can open it to find the server controls. These controls are in the Developer tab or the Local Server tab, depending on your application version. This area is where you manage the background processes that will turn the hardware into a dedicated engine for your daily tasks, and it’s also where you can load your preferred models and prepare the system to get outside requests. You get full command over the local environment without paying for expensive cloud subscriptions.

The main goal here is to let the machine work entirely on its own. Turning on headless mode lets the machine run in the background without needing a display or keyboard attached. Don’t worry, because there is always the option to go back in and fix anything needed.

The desktop application has a setting you can toggle to run the server automatically when the machine logs in. When you check this box, if you close the main application, it will minimize to the system tray while the background service keeps working. You get a dedicated server that stays out of your way.

Don’t be afraid to experiment

You need a setup made just for you

There is an alternative that I like to use, which is GPT4All. It has a lot fewer restrictions and not as much bloat. However, the lack of restrictions assumes you know what you’re doing. While it is a good way to feel free, it’s better to do this the right way first and then experiment from there.

There’s also a standalone daemon called llmster that runs completely independent of the graphical interface. You can set this daemon to start automatically when the computer boots up, so it is a good way to manage machines stored in a closet setup or a basement. You can leave the hardware running continuously without using a visual desktop environment.

Basically, you can connect your main laptop to the server and get coding assistance without using your own RAM. Your workhorse computer stays fast since the secondary one does all the heavy computation. This is how I personally have it set up. You can set up a code editor plugin like Continue directly to this new endpoint, letting you receive autocomplete suggestions and chat responses strictly within your own local network. Keeping the model offline means zero source code goes to an external cloud provider, which keeps your projects private and avoids recurring subscription fees.

You just have to be very careful of your system resources. The most important adjustment you can make is setting a smaller context window for your model. The context window determines how much conversation history and code the model can remember at once. As you increase the context length, this cache grows linearly and quickly uses up your available VRAM. If the cache exceeds your graphics card limits and spills over into standard system memory, the whole system gets slow, and generation speeds drop a lot.

By keeping the context window limited to what you need for a specific file or function, you keep memory usage low and stop the generation speed from dropping.

AI isn’t just for big tech companies

You don’t have to spend a fortune when you want a server, even for AI. All you need are some old computers, or maybe you can find one while bargain hunting. I’ve spent nothing on a workhorse PC that can now run an LLM easily because I have a spare machine lying around. Once you’re done, you’ll be grateful your PC doesn’t have to run these programs by itself anymore.

Brand

UGREEN

CPU

Intel 12th Gen N-Series

Memory

8GB (Upgradeable to 16GB)

Drive Bays

2 x 22TB