> **> **TL;DR
Starting with local AI (artificial intelligence processed on your own device) through Hermes Agent and Qwen 3.8 Flash Next offers more privacy than cloud services (cloud-based processing services) and gives you control over tasks on your own machine—but it requires high-end hardware. This model has 125 billion parameters (the values that determine a model’s capabilities), is approximately 105GB in size, and the author uses a Mac Studio with 256GB of unified memory.**
The initial setup was fairly overwhelming, from choosing a model to configuring permissions so the AI could perform real tasks. The morning briefing running through a cron job (a task scheduled to run automatically) was not particularly useful, and it also required keeping macOS awake at 7:30am.
Organizing games in Steam produced clearer results. Hermes categorized more than 400 games by genre within a few minutes, but it first needed a Steam web API key (a key used to authenticate API access), which had to be revoked afterward.
If it were me, I would recommend starting with a small task whose results are easy to measure, such as organizing games, before moving on to daily tasks like a morning briefing.
Getting Started with AI on Our Own Devices
This review began by trying to use AI in everyday life on an M5 Ultra Mac Studio—not merely testing its capabilities on paper. The author installed Hermes Agent and began experimenting with having AI help with small, repetitive tasks.
The interesting thing about local AI is that data stays on your device instead of having to send everything to a cloud service. That can provide greater peace of mind regarding privacy. However, getting started can be confusing because you need to choose the right model, hardware, and permissions for each task.
Having a personal AI is appealing because important data does not need to be sent entirely to the cloud, especially for people who are still hesitant to entrust personal information to services from OpenAI, Google, Microsoft, or Anthropic.
The idea is to have an assistant that works exclusively on your own computer and handles small tasks on command, without requiring a chatbot subscription or giving another company control over all your data. This level of privacy makes local AI more appealing—like having a personal assistant inside your machine, even if getting started is not yet easy.
Where the Mac Studio Fits into This Experiment
The M5 Ultra Mac Studio is the main machine used in this local AI experiment. It has 256GB of unified memory (shared memory between the CPU, GPU, and other processing components), allowing it to run almost any large model. The author installed Hermes Agent on this machine and tested Qwen 3.8 Flash Next, which has 125 billion parameters and is approximately 105GB in size.
This direction aligns with Apple’s push to promote new Macs for local AI work. On the Windows side, RTX Spark is also on the way, with up to 128GB of RAM for agentic AI tasks. This gives the market more options based on model size and the budget users can afford.
Qwen 3.8 Flash Next is the larger model—the big sibling—that has already been installed and used. Smaller Qwen models—the younger siblings—are still part of the testing plan across several devices, though the source has not specified the model names or sizes.
| Factor | Qwen 3.8 Flash Next | Smaller Qwen model |
|---|---|---|
| Status | In use | Planned for testing |
| Model size | 125 billion parameters | Not specified |
| File size | Approximately 105GB | Not specified |
| Devices | M5 Ultra Mac Studio | M6 Mac Mini, M5 MacBook Air, Asus TUF Gaming A14, RTX Spark |
From a Few Installation Steps to Finding Work for AI to Do
Hermes Agent is a self-hosted app (an app installed and operated on your own machine) that works on macOS, Windows, and Linux. It can also be controlled through a Telegram bot from a mobile device, making it suitable for people who want AI to run on their own computers.
The process started with asking it to summarize emails, calendars, and the morning weather before moving on to categorizing more than 400 Steam games through the Steam web API. The latter task took only a few minutes, but it required granting access to the machine and an API key first, followed by revoking those permissions when the task was complete.
How On-Device AI Differs from Cloud Services
Running AI on your own device provides greater privacy and data control because the information stays on your computer. However, you must choose the model yourself and have hardware capable of supporting it. Services from OpenAI, Google, Microsoft, and Anthropic are easier to use, but they require sending data to the cloud and charge usage fees based on tokens.
| Factor | On-device AI | Cloud services |
|---|---|---|
| Privacy | Data stays on the device | Data must be sent to the service |
| Data control | Greater personal control | Depends on the provider |
| Token costs | No token fees | Token-based usage fees |
| Hardware burden | Requires a machine with substantial RAM | No need to maintain hardware for the model |
Strengths and Limitations Found in Real-World Use
Local AI keeps personal data on your device, eliminates token fees, and lets you assign tasks to it directly, such as summarizing emails and calendars or categorizing games in Steam.
The limitations are that the large number of available models can be confusing, setup and device permissions are required, and you still need to decide which tasks are worth delegating to AI.
Pros
- +Personal data stays on the device
- +No token fees, with greater control over tasks
Cons
- −Models and configuration can be confusing
- −Requires granting device access and finding suitable tasks for AI
Costs That Go Beyond the Machine Itself
The Mac Studio used for testing has 256GB of unified memory, but the article points out that building a system like this does not necessarily require a $12,000 computer. The true cost also includes the time spent installing models, troubleshooting problems, and deciding which tasks to assign to AI.
There is also the burden of managing access permissions—for example, giving Hermes a Steam web API key and then remembering to revoke it.