Private AI for a small business: measured on Hugging Face and a MacBook

Under Singapore's Personal Data Protection Act (PDPA), a business stays responsible for its customers' personal data, including when an outside service processes it. For a trade business that runs on WhatsApp, that data is the job itself: names, addresses, phone numbers and photos of a customer's home.

The question came from a free AI audit for a Singapore aircon, plumbing and electrical firm with about ten technicians and roughly 70 enquiries a month, mostly on WhatsApp. Could an AI help sort those messages and plan the day without the messages leaving the business's control, and what would that cost?

Two experiments tested it in October 2026:

1. In the cloud: the same open model on three kinds of hardware rented from Hugging Face Inference Endpoints, sorting 12 sample emails into categories.
2. On a laptop: four open models running on a 2021 MacBook Pro (M1 Max, 32 GB) through Ollama, reading 10 sample messages and proposing CRM updates.

All test messages were made up. No customer data was used. The samples are small, so read the numbers as direction, not precision.

This page covers what was measured. For whether to run a private model and the five setup steps, see Should your business run its own private AI?. For the wider picture of what AI products cost to run, see what AI actually costs a small product.

First, what "private" can mean

"Private AI" covers several setups that protect against different things. Choosing well starts with knowing which risk matters.

Where the model runs	Protects against	Doesn't protect against
A major provider's business API, with a data processing agreement	Your data being used to train their models (by default)	Data processed overseas; a breach at the provider
A big cloud provider's Singapore region	Data leaving Singapore, where that model runs in-region	The cloud provider itself
Your own model on rented hardware (e.g. Hugging Face)	Sharing a service with other customers' traffic	The hosting platform's own security
A machine in your office	Every outside company	Theft, power cuts, and nobody keeping it updated
Remove personal details on your machine, then call an API	Most personal data leaving the building	Mistakes in the removal step

Two newer middle options sit between these. Ollama's cloud service runs open models on hosted hardware; Ollama states that prompts are not logged or used for training and that its hosting partners must follow the same rules. Calling a model maker's own service directly, such as Z.ai for GLM or Moonshot for Kimi, means the data goes to the model maker's own servers under their terms. Read the terms of either before sending anything sensitive.

Experiment 1: the same model on three kinds of cloud hardware

The model was Qwen3.5-9B, compressed to about 5.7 GB. Each setup sorted the same 12 emails into eight categories (lead, client, invoice, scheduling, newsletter, spam, personal, other). Prices are Hugging Face's published hourly rates in October 2026, billed per minute.

Hardware	Price per hour	Correct	Seconds per email (several at once)	Cost per 1,000 emails, sent in a batch
NVIDIA T4 graphics card	US$0.50	11/12	1.1	about US$0.15
8-core processor (CPU)	US$0.27	11/12	5.5	about US$0.40–0.50
4-core CPU, running Laya, a small "decision model" that picks from options instead of writing text	US$0.13	8/12	0.95	about US$0.035

Three findings came out of this table.

The pricier hardware was cheaper per email. The graphics card cost about twice as much per hour as the CPU but was about five times faster, so each email cost roughly a third as much. Compare cost per message, not cost per hour.

The cheapest option was not accurate enough on its own. Laya was the fastest and cheapest per email, but got 8 of 12 right. Its own model card says it saw no email data. Its makers also report that a version trained on their own task set roughly doubled accuracy on those tasks (from 0.36 to 0.77), so training on your own examples is the likely fix. Without that step, it needs a person or a stronger model to check it.

A batch of 1,000 emails is not how a small business receives them. That is the subject of the next section.

The idle-time trap

Rented hardware can switch itself off when nobody is using it ("scale to zero"), so you stop paying. The catch is in how it switches back on. On Hugging Face, an endpoint sleeps after a set idle period (15 minutes in this test). The first message after that wakes it, which took about a minute here, and the endpoint then stays on, and billed, for another 15 minutes in case more messages follow.

A useful way to estimate the real cost:

Cost per message = hourly price × (time working + time starting up + idle time before it sleeps) ÷ messages handled

For the firm in this audit, with about 70 enquiries a month arriving one at a time:

• One enquiry wakes the graphics card: about 1 minute starting + a few seconds working + 15 minutes idle. That is about 16 minutes, or about US$0.13 for that single message.
• The same message inside a batch of 1,000: about US$0.00015.

The same model on the same hardware costs close to a thousand times more per message when messages trickle in, because almost all of the bill is waiting. At this firm's volume, a dedicated graphics card makes sense only if privacy is the reason. On cost alone, a hosted business API handles a short message for well under a cent.

Experiment 2: four models on a laptop

The second experiment ran on a 2021 MacBook Pro used as an always-on office machine. A small program checked a messaging inbox every 60 seconds and passed new messages to a local model, which proposed CRM updates and flagged real buying signals. Nothing was written to the real CRM; every change was only proposed.

Each model was scored on 10 sample messages for three things: picking the right CRM contact, never deleting existing notes, and alerting only on genuine buying intent.

Model (via Ollama)	Seconds per message	Right contact	Notes kept	Right alerts
Qwen3.5 9B	12.9	88%	100%	88%
Qwen3.5 4B	36.6	62%	100%	75%
Qwen3.8 27B	57.1	88%	100%	100%
Gemma 4 31B	149.6	88%	100%	100%

No model deleted anything. Every model had access to a tool that could overwrite a contact's notes, and every model chose the tool that adds a note instead.

Smaller was not cheaper. The 4B model was slower and less accurate than its 9B sibling. On hard messages it produced about 3,500 tokens (roughly 15,000 characters) of internal reasoning, going back and forth on whether to act, and sometimes never answered at all.

Bigger was not better. Gemma 4 31B matched Qwen3.8 on accuracy but took 2.6 times as long. For a narrow, repetitive job, the extra size bought nothing.

The setting that was on by default

Many recent open models can "think" before answering: they write out hidden reasoning first, which helps with hard puzzles. In both experiments this setting was on by default, and nobody had asked for it.

Test	Thinking on	Thinking off
Cloud, Qwen3.5-9B, sorting emails	32 seconds per email, 8/12 correct, 3 emails never answered	2.3 seconds per email, 11/12 correct
Laptop, Qwen3.5 9B, CRM updates	12.9 seconds, 88% right alerts	6.4 seconds, 100% on all three checks
Laptop, Qwen3.5 4B, CRM updates	36.6 seconds, 75% right alerts	17.0 seconds, 38% right alerts

For the 9B model, switching thinking off made it faster and more accurate in both places. For the 4B model it was faster, but it started flagging almost everything, including small talk and an unsubscribe request. Switching thinking off removes delay, and on some models it also removes the judgment. Test both settings on your own messages before choosing.

This matches a distinction Daniel Kahneman made between fast and slow thinking: sorting a message is a fast decision, and paying for slow thinking on it costs time without improving the answer. Two kinds of decisions covers the idea in more depth.

Capacity and speed: what the hardware decides

Two numbers decide what a computer can do with a local model:

• Memory (capacity) decides which models fit at all. The whole model has to sit in memory. The 32 GB laptop could run models up to about 20 GB, because the operating system keeps some memory for itself.
• Memory bandwidth (speed) decides how fast the model answers, because every word it writes requires reading the model from memory. A rough ceiling is bandwidth divided by model size.

The laptop's M1 Max reads memory at 400 GB/s, which is why the 9B model managed about 40 words (tokens) per second. Apple's published specs for the 2024 Mac mini list 120 GB/s for the base M4 chip and 273 GB/s for the M4 Pro. On the same job, that suggests roughly 15–20 seconds per message on a base Mac mini and 9–10 seconds on an M4 Pro. These are estimates, not measurements. That is fine for background work such as updating a CRM, and too slow for replying to a customer while they wait.

A used M1 Max or M2 Max machine is therefore still a strong choice for local AI: on speed, it beats every Mac mini on Apple's 2024 list.

Privacy is the whole system, not just the model

Running the model privately protects the messages only if the system around it is also careful. Both experiments turned up the same kinds of risk:

• A model with an email tool can be talked into using it. On the laptop, the model had no way to send email. Plain code decided whether to send an alert, so a cleverly worded message could not make the AI notify anyone.
• Every connection needs its own key. The program reading the inbox used its own access key, separate from the developer's, so it could be switched off without touching anything else.
• Defaults are not always private. In the cloud test, a code repository was created as public by default and stayed that way for about a minute before being switched to private. It held only code, but the same default would have exposed anything else in it.
• Suggestions, not actions. In both setups, the AI only proposed. A person approved CRM changes and customer replies.

What the firm's design used

For the trade business, the two experiments shaped a split design. The pilot has not run yet, so this describes the plan, not results:

• Customer replies that must be instant go through a hosted business API with a data processing agreement, using only templates the owner approved. Neither the hosted model nor any local model writes customer messages freely.
• Background work, such as reading technicians' job updates and checking them against the calendar, suits an office machine, where messages never leave the premises.
• Which technician to send is decided by simple rules first (skills, location, who is free), then a decision model ranks the remaining options, and the owner approves with one tap.