Follow ZDNET: Add us as a preferred source on Google.
ZDNET’s key takeaways
- Claude can control your computer to complete tasks on its own.
- The feature is limited to paid plans and Mac users for now.
- Permissions and oversight are required to reduce security risks.
Imagine asking an AI to perform a task in which it has to use your computer just as you do. That means it can open files, launch apps, click icons, type text, and browse the web on its own. This type of skill marks the latest evolution of AI agents, and it’s one you can now try with Anthropic’s Claude AI.
Currently in “research preview” mode, Claude’s new computer ability gives it the power to navigate your computer screen and interact with your applications, files, and settings to run your assigned tasks. Just give it a command, such as “enter data into a spreadsheet from the contracts files in my Google Drive, format it, and then save it to a folder,” and the AI should complete each step as long as your computer stays on.
Also: How Claude Code’s new auto mode prevents AI coding disasters – without slowing you down
The only times you’d need to intervene are when Claude requires your permission to perform a specific action, such as opening an app.
The new skill works with Claude Cowork and Claude Code. Cowork is an agentic AI process designed to handle complex tasks independently to save you time and labor. Claude Code uses agentic AI to create computer code and build apps for you.
Sounds intriguing.
Before getting started
There are a few requirements, conditions, and concerns.
First, the computer skill is available only for Claude Pro and Max subscribers. Free users, as well as those with Team or Enterprise plans, are out of the running for now. Second, it’s accessible only on the Mac through the Claude Mac app (though Anthropic says that Windows support is coming soon).
Next, Claude will try to avoid interacting with certain apps or sensitive data. As some examples, it won’t perform stock trading or investment transactions, input sensitive information, or scrape facial images. But since the new skill is still in the preview phase, Claude could still run afoul of its basic programming. For that reason, you may need to manually deny it permission to apps associated with banking, financial, healthcare, or legal matters.
Also: This new Claude Code Review tool uses AI agents to check your pull requests for bugs – here’s how
Further, Claude captures your screen to learn how to navigate it. That means it can see and record whatever information is on the screen. When you issue a command, make sure that any apps or files with confidential or private data aren’t open and visible.
Overall, this seems like a skill with a lot of potential. But it’s also a process that could go haywire, especially given the unpredictability of AI bots. Since you’re essentially letting Claude loose on your computer, the risks multiply. With all that in mind, Anthropic advises you to avoid giving it permission to sensitive apps. You’ll also want to start with simple tasks and be sure to fine-tune your instructions to avoid any type of misunderstanding.
How to use Claude AI to control a computer
If you have a Claude Pro or Max account and a Mac, here’s how to try this.
With the Claude app open, select Settings. In the Desktop app section at the bottom of the Settings screen, select General. Turn on the switch for “Keep computer awake” to make sure your PC doesn’t go to sleep while Claude is off on a mission. Under Computer use, turn on the option for “Let Claude take screenshots and control your keyboard and mouse in apps you allow.”
Also: Want local vibe coding? This AI stack might replace Claude Code and Codex – for free
I wanted to see how Claude’s new skill worked in action, so I sent it on a few different missions. Following Anthropic’s advice, I started by keeping it simple.
For the first request, I asked Claude: “Open Finder, navigate to my Documents folder, and tell me the names of the five most recent files.”
In response, Claude requested permission for mouse and keyboard control, screen recording, and Finder.
As the AI carried out this task, I could see the different windows opening, including the Documents folder. Claude showed me the files sorted by date and eventually displayed the names of the five most recent ones in the chat.
Also: I used Claude Code to vibe code an Apple Watch app in just 12 hours – instead of 2 months
For my third request, I wanted to give it a more interactive and challenging task. I told Claude: “Open the Notes app, create a new note entitled ‘Claude Test,’ and write a short to-do list for a productive morning exercise routine.”
As expected, I had to grant it permission to access the Notes app, after which it created the note and populated it with such items as drinking a glass of water after waking up, doing five minutes of light stretching, and tackling a 20-minute cardio workout of jogging or biking.
Also: I stopped using ChatGPT for everything: These AI models beat it at research, coding, and more
And for my fourth and final request, I asked Claude: “Open my most recently modified document in the Documents folder, summarize its content, then open Mail and draft a new email to a placeholder address with that summary as the body of the email.”
Here, I had to give Claude access to Finder and Mail. The AI then opened a new email with a summary of my latest document. Initially, Claude mistakenly added the subject of the email to the To field. But it caught its own error and corrected it.
Also: 7 AI coding techniques I use to ship real, reliable products – fast
How did Claude fare?
The AI performed each task accurately without any hitches. Even when it ran into roadblocks or made mistakes, it was able to work past them on its own to achieve the right results.
I do have a couple of minor criticisms.
The permission you give to Claude to access a specific app is only good for that session. If you want it to access the same app for a different request, you need to grant permission again. This is a good idea in terms of safety, but I’d like to see an option to give an app permanent permission.
Also: How to switch from ChatGPT to Claude: Transferring your memories and settings is easy
Also, the process itself can be slow, even with a simple task. For example, I could find the five most recent files in my Documents folder much faster than Claude did. But the true strength here lies in the AI’s ability to perform a series of tasks based on a single request, especially if you ask it to generate content.
So far, Claude’s computer skills are off to a promising start. I look forward to seeing how this ability evolves over time, yet still be mindful of your safety and security.
