Why this decision matters more than it looks
People treat hosted vs local as a philosophical question about AI sovereignty or cost efficiency. In OpenClaw, it is an operating decision with concrete consequences.
Your model setup changes whether the assistant feels abundant or expensive, whether you let it run recurring tasks freely or subconsciously ration every prompt, and whether your monthly bill is predictable or creeping.
Hosted models give you the fastest path to quality output — especially for reasoning, writing, and complex tasks. They require no hardware setup and no local inference tuning. But every token costs money, and a setup that routes everything through premium models can get expensive fast.
Local models give you free per-token inference, privacy (your memory files never leave your machine), and always-on background capability without a per-use API cost. But they require Apple Silicon hardware (or equivalent), Ollama setup, and they are weaker than the best hosted models on hard reasoning tasks.
The wrong default makes the whole system feel heavier than it should. Hosted-everything gets expensive. Local-everything gets slow and fiddly. Hybrid with clear routing rules is the answer most operators end up at.
Hosted models: what they win at
Hosted models from Anthropic (Claude), OpenAI (GPT-4o), and Google (Gemini) remain ahead of local models on the tasks that matter most for high-value work:
Complex reasoning: multi-step analysis, strategy sessions, and decision synthesis where the reasoning chain matters. Claude Sonnet 4 is noticeably better than any local 14B model at this.
Important writing: launch copy, strategic documents, and anything where quality directly affects outcomes. Hosted models produce tighter, more nuanced prose.
Research synthesis: gathering information from multiple sources, resolving contradictions, and producing a structured recommendation. Hosted models handle this more reliably.
Code review and architecture: for complex codebases, hosted models like Claude Sonnet 4 understand context and tradeoffs better than current local alternatives.
The main downside: cost accumulates if you route every job through a smart hosted model. A morning brief does not need Claude Sonnet 4. Routing heartbeats and summaries through a $0.003/1K-token model instead of a $0.015/1K-token model saves meaningfully at scale.
Best hosted models for OpenClaw in 2026:
Claude Haiku 3.5: fast, affordable, excellent instruction following. Best for: standard day-to-day work, message drafts, routine summaries, recurring reviews.
Claude Sonnet 4: strong reasoning and writing. Best for: strategy, complex synthesis, important content, code review. The primary workhorse for heavy work.
GPT-4o-mini: good alternative to Haiku for standard work if you prefer OpenAI's ecosystem.
Claude Opus 4: best quality, highest cost. Reserve for genuinely high-stakes decisions where quality clearly matters and mistakes are expensive.
Local models: what they win at
In 2026, local models on Apple Silicon have become genuinely useful for a meaningful slice of OpenClaw work. The key threshold: Apple Silicon unified memory provides GPU-accelerated inference without a discrete GPU, making on-device inference fast and practical.
Speed benchmarks on Mac mini (approximate):
| Model | M2 16GB | M4 16GB | Best for |
|---|---|---|---|
| phi4-mini | 60–90 tok/s | 80–120 tok/s | Heartbeats, briefs, summaries, classification |
| llama3.2 3B | 70–100 tok/s | 90–130 tok/s | Fast general tasks, message drafts |
| llama3.2 8B | 35–55 tok/s | 45–65 tok/s | General Q&A, light research summaries |
| qwen2.5-coder:14b | 20–35 tok/s | 30–45 tok/s | Coding tasks, code review, test writing |
| llama3.3 70B | 5–10 tok/s | 8–15 tok/s | Batch jobs, overnight memory consolidation |
Those speeds mean phi4-mini and llama3.2 3B are fast enough for real-time interactive use, not just batch jobs. A morning brief runs in 15–30 seconds locally. That is well within acceptable latency for a scheduled workflow.
What local models win at specifically:
Heartbeat and brief generation: phi4-mini handles morning briefs, daily summaries, and heartbeat check-ins at zero per-token cost with acceptable quality.
Document classification and routing: simple categorization tasks run well on small local models at 80–120 tok/s.
Memory consolidation: reviewing daily notes and promoting context to MEMORY.md is a low-stakes job well-suited to a local model running overnight.
Coding tasks: qwen2.5-coder:14b is competitive with Claude Haiku 3.5 for many coding jobs while running locally at zero API cost.
Privacy-sensitive work: any job that involves personal data, confidential business context, or system prompts you want to keep off external servers.
Who should start hosted-first
Start with hosted models if any of these describe your situation:
You are still figuring out which workflows are actually useful in your life — hosted models produce better output faster and help you evaluate what is worth automating.
Your highest-value use cases are planning, writing, synthesis, and decision support — these are areas where hosted models lead significantly.
You do not have a Mac mini with Apple Silicon yet — local inference on non-Apple hardware requires more setup and produces slower results.
You want the system useful this week, not after a week of inference tuning — hosted models require zero local setup.
Your API spend is under control and not yet high enough to justify the extra complexity of a local tier.
Hosted-first is not the "beginner" or "less serious" path. It is the fastest path to a system that proves it deserves more investment — including investment in local infrastructure.
When to add local models
Local models become worth adding when one or more of these conditions are true:
Recurring background work is real: you have heartbeat checks running every 30 minutes, daily briefs, weekly reviews, and memory consolidation loops — all of which generate API calls even when you are not actively prompting. At this point, routing background work to local saves meaningfully.
Cost is affecting behavior: if you find yourself hesitating to run a workflow or skimping on task frequency because of API costs, that is the signal. Local models remove the cost psychology from routine work.
Privacy is a real concern: memory files, system prompts, and sensitive business context never leave the machine when running local models. If that matters for your setup, local is the right answer for those jobs.
You have Apple Silicon hardware: if you have a Mac mini M2 or M4, the setup cost for local models via Ollama is low (about 20 minutes) and the results are immediately useful.
The hybrid routing table that works for most operators
The best long-term default is a split-brain system with clear routing rules. Here is the table most operators should run:
| Job | Route to | Why |
|---|---|---|
| Morning brief generation | phi4-mini (local) | Runs daily, no API cost, quality is sufficient |
| Heartbeat checks | phi4-mini (local) | High frequency, no API cost |
| Document summary | llama3.2 or Claude Haiku | Moderate quality need, cost-sensitive |
| Message drafting | Claude Haiku 3.5 | Good quality, low cost, fast |
| Coding tasks | qwen2.5-coder:14b (local) | Best local coding model, zero API cost |
| Research synthesis | Claude Haiku 3.5 | Better than local for multi-source synthesis |
| Memory consolidation | llama3.2 or local 70B overnight | Batch job, low stakes, free locally |
| Strategy and planning | Claude Sonnet 4 | Quality matters, hosted leads significantly |
| Important long-form writing | Claude Sonnet 4 | Best quality output for high-stakes content |
| High-stakes decisions | Claude Opus 4 | Reserve for decisions where quality changes outcomes |
This routing table — phi4-mini for background, Haiku for standard, Sonnet for reasoning — is the hybrid default that keeps costs sustainable without sacrificing quality on work that matters.
Setting up Ollama for local models
Ollama is the standard way to run local models with OpenClaw. It takes about 20 minutes to set up:
Install Ollama:
brew install ollama on macOS, or download from ollama.ai. Start the server with ollama serve.Pull the starter models:
ollama pull phi4-mini for background work, ollama pull llama3.2 for general tasks. Both fit easily on a 16GB Mac mini.Add coding model if needed:
ollama pull qwen2.5-coder:14b (requires ~9GB, works well on 16GB).Configure OpenClaw: add an Ollama provider in your OpenClaw config pointing to
http://localhost:11434. Set specific models to use the ollama-local/phi4-mini path for background workflows.Test before relying on it: run your morning brief manually using the local model. If quality is acceptable, move heartbeats and briefs permanently to local. If not, keep them on a cheap hosted model.
The whole setup adds maybe 30 minutes total — 20 for Ollama install + model pulls, 10 for OpenClaw config updates. The ongoing maintenance cost is near zero once it is running.
Common mistakes
Going local-only before hosted has proven value: running entirely on local models before the setup has demonstrated usefulness is putting the optimization before the problem. Get one useful workflow working with hosted models first, then optimize for cost and privacy.
Using premium hosted models for every recurring task: the cost spiral version of the opposite mistake. Heartbeats, briefs, and summaries do not need Claude Opus. Route them to Haiku or local.
Expecting local models to match hosted on hard tasks: a local 14B model is strong for routine work, but it is not competitive with Claude Sonnet 4 on complex reasoning, multi-step planning, or nuanced writing. Use each where it is strong.
Not setting API spending caps: even with local models handling background work, a runaway automation routing to expensive models can generate a big bill. Set hard monthly caps on every AI provider account.
Choosing based on identity rather than job: "I want to run locally because I care about privacy" is a valid reason for specific jobs. "I want to run locally because it sounds more serious" is not a good routing decision.
The practical decision checklist
Use this to decide what to run where:
Does this job run more than once per day? → Strong candidate for local or cheap cloud model.
Does a wrong or mediocre answer matter? → If yes, use a smart hosted model. If no, local is fine.
Does this job involve sensitive data you want off external servers? → Route to local.
Is this job about reasoning, strategy, or important writing? → Route to Claude Sonnet 4 or equivalent hosted model.
Is this a coding task? → Try qwen2.5-coder:14b locally first; upgrade to hosted if quality is insufficient.
Is this a batch job that can run overnight? → Ideal for local — no latency pressure, zero API cost.
Read this next
Want the opinionated hybrid defaults without figuring them out from scratch?
The RoloClaw starter system includes the model routing table, cost control rules, and local model setup guidance in one place — so you spend the first day building, not researching model comparisons.
Best for: operators who want a working hosted + local hybrid setup with clear routing rules, not weeks of configuration experimentation.
You get: the model routing defaults, cost control rules, Ollama setup guidance, and the specific model recommendations for each job type.
Why buy: the expensive part is not the guide. It is the API bill from routing everything through premium models before figuring out the hybrid rule.
FAQ
Should I use hosted or local models for OpenClaw? Hosted first, then hybrid. Hosted models (Claude, GPT-4o) get you to a useful setup faster with better output. Add local models (phi4-mini, llama3.2 via Ollama) once recurring background work is common enough that cost or privacy starts mattering.
Are local models good enough for OpenClaw in 2026? Yes for many routine jobs. On an M4 Mac mini, llama3.2 runs at 40–60 tokens/second — fast enough for briefs, summaries, and heartbeats. For complex reasoning and important writing, Claude Sonnet 4 still leads significantly.
What local models work best with OpenClaw? phi4-mini for fast background work; llama3.2 for general standard tasks; qwen2.5-coder:14b for coding. All run via Ollama on Apple Silicon Mac mini.
How fast are local models on a Mac mini? On M4 16GB: phi4-mini at 80–120 tok/s, llama3.2 8B at 45–65 tok/s. Fast enough for real-time interactive use on smaller models.
When should I add local models? When recurring background jobs are running frequently enough that API costs are noticeable, when privacy matters for specific workflows, or when you already have Apple Silicon hardware where Ollama setup is quick.
What is the best hybrid model setup? phi4-mini or llama3.2 (local) for background work; Claude Haiku 3.5 for standard daily tasks; Claude Sonnet 4 for reasoning and important outputs. Three models, clear routing rule, 95% of real workflows covered.
Do I need a GPU for local models with OpenClaw? No. Apple Silicon Mac mini provides GPU-accelerated inference through unified memory without a discrete GPU. A Mac mini M4 with 16GB runs 7B and 14B models well enough for daily use.