Advanced

Local models in depth

This page is for readers who want to go further with a local model than the setup steps. You gain privacy and an end to per-use bills, but you give up some intelligence and speed, and how much you give up depends almost entirely on your hardware. Installing Ollama, downloading a model, keeping it local only, and connecting it to each agent are on Run a local model with Ollama, so start there.

Applies to: Agent Zero, Atomic Agent, Hermes Agent, OpenClaw, Goose, QwenPaw, Eigent, Everfern, and OpenManus, whose Ollama example file this guide did not test. NanoClaw appears only in the per-agent notes, because it does not support a local main model through Ollama today.

File sizes, settings, and addresses were checked on 14 November 2026.

What you gain and what you give up

A local model keeps your prompts on your computer and costs nothing per use, because you have already paid for the machine. The prompt still leaves your computer in three cases: when you pick a model whose name ends in :cloud, when you sign in to Ollama so the local server can call Ollama's cloud, and when the agent uses a web tool such as search or a browser.

The cost is intelligence. Researchers measured earlier small models failing once a later tool call depends on an earlier result. This guide still expects an 8 GB or 16 GB computer to stay on single-step jobs. Long chains are hard even for large cloud models, since one paper found GPT-5's success rate fell from 72.5% with 5 core steps to 15.0% with 20.

Speed is the second cost. Without a strong graphics chip, replies come slowly, and Ollama says Intel Macs run models on the processor alone. Pick your model and budget gives published speeds for a 7B model on Apple chips and on a laptop processor. This guide's own research measured none.

Electricity is the third. The chip stays busy for the whole reply, which draws more power than an idle computer, but this guide could not confirm a watt figure for any machine, so it puts no dollar amount on it.

Which computer runs what

On most Windows PCs, the number that matters is RAM. On a Mac with Apple silicon, the processor and graphics share one pool called unified memory, so the whole amount counts. On a Windows PC with a separate graphics card, a model that fits in the card's own memory (VRAM) runs fastest, and Ollama can spill over into ordinary RAM, though responses may be slower.

"Fits" below means the model file is smaller than the memory, with room left for the operating system. It does not promise room for a very long conversation. The judgments in the third column are this guide's, not test scores.

Memory	Models that fit, with file size	What to expect, a judgment	The catch
About 8 GB	Qwen3.5 4B (`qwen3.5:4b`, 3.3–4.0 GB; Q4_K_M is 3.3 GB)	Small local chat. A weak agent once one step depends on the last.	Hardware dependent. The card prints a 256K context, which this computer cannot hold.
About 16 GB	MiMo-V2.6 9B (`mimo-v2.6:9b`, 6.6–8.7 GB)	More comfortable chat, still weak for chained steps.	Hardware dependent. Keep `mimo-v2.6:9b` on this row, and leave the 13.1 GB ByteShape file for a larger machine.
About 24 GB of unified memory or VRAM	Qwen3.8 27B (`qwen3.8:27b`, 18 GB)	The first useful tier for a short loop of tool calls, though slower than a cloud model.	Hardware dependent. Plan on a 32K context with this 18 GB file.
About 32 GB	Qwen3.8 27B (`qwen3.8:27b`, 18 GB)	The same 18 GB file as the row above, with more room left for context.	Hardware dependent. The spare memory is there for context past the 32K planned on 24 GB.
About 48 GB	Qwen3.8 27B (`qwen3.8:27b-q8_0`, 30 GB)	A higher-quality copy of the same 27B model.	Hardware dependent. The 56 GB copy in the next row does not fit here.
About 64 to 80 GB	Qwen3.8 27B (`qwen3.8:27b-bf16`, 56 GB)	The largest copy of this 27B model, for a machine with room to spare.	Hardware dependent. On 64 GB, if the operating system and context do not fit, stay on `qwen3.8:27b-q8_0`.
128 GB and up	Qwen3.5 122B (`qwen3.5:122b`, 81 GB)	A much larger local model for a machine with memory to spare.	Hardware dependent. This is not an ordinary home computer.

Two sizes are easy to get wrong, and both are copies of Qwen3.8 27B. qwen3.8:27b is an 18 GB file, so a 16 GB laptop cannot spare room for the operating system. qwen3.8:27b-bf16 is a 56 GB file, so a 48 GB machine cannot load that copy. The Qwen3.5 4B card prints a 256K context, and an 8 GB computer cannot hold that length. A fit check on 13 November 2026 found qwen3.8:27b using 17 GB at a 32K context. Qwen's models in this table use the Apache 2.0 license, and the MiMo page on Ollama names MIT. Every tag in this table can read images, call tools, and use its thinking mode.

Check how much memory you have

These steps only read information, so nothing on your computer changes.

On Windows 11:

Open Settings, select System, then About, and read Installed RAM under Device specifications.
If you have a graphics card, press Ctrl+Shift+Esc to open Task Manager, select Performance, then GPU, and read its dedicated memory.

On a Mac:

Open the Apple menu and choose About This Mac.
Read Memory, then Chip. If the window says Processor followed by an Intel name instead, you have an Intel Mac, which runs Ollama on the processor only and is not supported by LM Studio.

Smaller machines: Qwen3.8 27B with llama.cpp

ByteShape publishes smaller llama.cpp copies of Qwen3.8 27B than the 18 GB Ollama file. The GPU-5 copy is a 13.1 GB file and keeps 99.63% of the full-precision score. The GPU-4 copy is an 11.0 GB file and keeps 98.72% of that same score. This guide could not confirm Ollama or LM Studio, and the publisher's commands use llama-server. Treat every name in the commands below as a Hugging Face tag for llama.cpp.

The bundled MTP draft is about 250 MB, and that draft still accepts images. DFlash2 needs llama.cpp build b10658 or newer, and that draft path is text only. On a 16 GB computer, keep the Ollama tag mimo-v2.6:9b and leave the 13.1 GB file alone. GPU-1 is 8.8 GB and does not fit an 8 GB computer once the operating system takes its share.

The two commands below are the publisher's, with a backslash continuing each line in a Unix shell. Run the first command when you want the GPU-5 file and its bundled MTP draft.

llama-server \
  -hf byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4_XS-3.84bpw \
  --mmproj-auto \
  --spec-type draft-mtp --spec-draft-n-max 3

Run the second command when you want the DFlash2 draft on that same GPU-5 file.

llama-server \
  -hf byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4_XS-3.84bpw \
  -hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --no-mmproj

Set up Ollama first

Ollama is the simplest model server for most people, and every agent on this page that supports local models can talk to it. Run a local model with Ollama covers the install on Windows 11 and on a Mac, a model for each memory size, the first test, the local-only settings, and the address each agent needs. The sections below assume Ollama already answers in your terminal.

The table above also lists qwen3.8:27b-bf16 and qwen3.5:122b, which the Ollama page leaves out. You download each one the same way, with ollama pull and the tag from the table.

Give the model a longer memory for the conversation

Ollama picks a default context length from your graphics memory: 4,096 tokens with less than 24 GB, 32,768 with 24 to 48 GB, and about 262,000 (shown as 256k) with 48 GB or more. Ollama's own docs say agents and web search need at least 64,000 tokens, and Hermes Agent refuses to start with less. The three ways to raise it, by the slider, by OLLAMA_CONTEXT_LENGTH, or by a Modelfile, are on the Ollama page. This section is about finding the largest length your machine can carry.

Open the Ollama app and go to its settings.
Move the context length slider to 64k.
Run your model again with ollama run <your model> "Hello", then run ollama ps.

What you should see the CONTEXT column in ollama ps showing the new length, and the PROCESSOR column unchanged from before. If more of the model has moved to the CPU, or it fails to load, the longer context does not fit, so move the slider back down one step.

Ollama's docs say a larger context needs more memory, and for the best speed they advise the model's full context length while keeping the model off the CPU. If the model already ran at 100% GPU, you can try the next step up the same way, and stop at the last step where it still shows 100% GPU. On a computer that runs models on the processor only, stop at the first length that loads and replies at a speed you can live with.

Or use LM Studio instead

LM Studio does the same job with a point-and-click window. On a Mac it needs Apple silicon and macOS 14 or newer, and LM Studio recommends 16 GB or more. On Windows it runs on x64 processors with AVX2 support or on ARM, and LM Studio recommends at least 16 GB of RAM and 4 GB of dedicated graphics memory.

Download LM Studio from lmstudio.ai/download and install it.
Open the Discover tab with Ctrl+2 on Windows or ⌘+2 on a Mac, search for a model such as qwen3.5 or qwen3.8, and download a 4-bit (Q4) or higher option that fits your memory.
Open the Developer tab and turn on the Start server switch.
In a web browser on the same computer, open http://localhost:1234/v1/models.

What you should see a short page of text that lists the model you downloaded. Once the files are on disk, LM Studio can run with no internet connection at all.

Connect LM Studio to your agent

LM Studio speaks the OpenAI-compatible format at http://localhost:1234/v1, which you give to an agent installed as a normal app. An agent inside a container cannot use that address, because inside a container localhost means the container itself, so give it http://host.docker.internal:1234/v1 instead. QwenPaw's Docker version, for example, takes that address in Settings → Models as its Base URL.

Most agents in this guide name LM Studio as a choice of its own, and the notes for each agent below say which. For Ollama, the addresses and each agent's exact setting are in Connect it to your agent.

If an agent in Docker still cannot reach the server

Ollama listens only on 127.0.0.1 by default, and Agent Zero's docs say a container may not reach a server that listens there. Their fix is to make the server listen on an address Docker can reach, such as 0.0.0.0, and to keep that port limited to trusted devices. Try the default first, and do this only if the agent reports that it cannot connect.

For Ollama on Windows:

Quit Ollama from its icon in the taskbar.
Open Settings, search for "environment variables," and choose Edit environment variables for your account.
Add a variable named OLLAMA_HOST with the value 0.0.0.0:11434, then click OK.
Start Ollama again from the Start menu.

What you should see Ollama running again, and the agent's next message getting a reply.

For Ollama on a Mac, run this in Terminal, then quit and reopen Ollama:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"

What you should see the agent's next message getting a reply.

For LM Studio, open Developer, then Server Settings, and turn on Serve on Local Network. LM Studio warns that any address other than 127.0.0.1 opens the server beyond your computer and recommends authentication, so also turn on the switch that requires an API token, create a token under Manage Tokens, and give it to the agent as its key. That token feature needs LM Studio 0.4.0 or newer.

What you should see the server address in LM Studio changing from localhost to your computer's network address.

Warning: With 0.0.0.0, every device on the same network can use your model server, and Ollama asks none of them for a key. Do this only on a home network you trust, never forward that port on your router, and on a laptop that travels, remove the setting before you join café or hotel Wi-Fi.

Notes for each agent

Agent Zero: It lists Ollama, LM Studio, llama.cpp, vLLM, and oMLX as local providers, with Docker-ready addresses already filled in. Its advanced page has the steps, and its Utility model can stay on a cloud model while the Main model runs locally. If a small model describes what it would do instead of doing it, try the bundled Tiny Local profile.
Atomic Agent: A local model is its default. It runs llama.cpp's llama-server on your machine with no key, and a first run can start on a small model while a better one downloads. Its terminal wizard has presets for Ollama and LM Studio. In the desktop app, you add Ollama as an OpenAI-compatible API (custom base URL) provider at http://localhost:11434, without /v1, which Atomic Agent adds itself.
Hermes Agent: Run hermes model and choose Custom endpoint (self-hosted / VLLM / etc.), enter http://localhost:11434/v1, skip the API key, and type the model name. LM Studio has its own LM Studio choice in the same menu. Hermes refuses a context under 64,000 tokens, and its docs say models under roughly 30B often fail to call its memory tool. If Hermes runs inside WSL2 while Ollama runs on Windows, localhost points at the Linux side unless Windows mirrored networking is on.
OpenClaw: During onboarding, choose Ollama, then Local only. Local addresses need no real key, and OpenClaw uses the marker ollama-local in its place. Its docs warn that smaller or heavily compressed models are more open to prompt injection, and they tell you to use a large model for any bot that has tools.
Goose: Choose the Ollama provider, which looks at localhost:11434 unless you set OLLAMA_HOST, and the desktop app lists the models you have already pulled. LM Studio is a separate provider at localhost:1234. A model that cannot call tools can only chat in Goose, and you must turn off all its extensions.
NanoClaw: Claude is the default model. Its docs say the /add-ollama-provider skill targets an older setup and is not a supported drop-in on the current version, while /add-ollama-tool only lets another model call a local one as a tool.
QwenPaw: It has a built-in local option called QwenPaw Local, in sizes from 2B to 35B, plus Ollama and LM Studio. Set the address under Settings → Models → Base URL. If you run its Docker image, add --add-host=host.docker.internal:host-gateway to its docker run command and use http://host.docker.internal:11434 for Ollama, or the LM Studio address above.
Eigent: Its docs list Ollama at http://localhost:11434/v1 and LM Studio at http://localhost:1234/v1, plus llama.cpp, vLLM, and SGLang. In v1.0.7 you click the Ollama icon on the Models page, because the Local tab in its docs belongs to an older layout. A task will not start until at least one working model is set.
Everfern: Its README says it can run fully offline with Ollama or LM Studio. In v1.3.0 setup, Local Engine then Ollama shows a Server URL (optional) box, which uses http://localhost:11434 when left empty.
OpenManus: You set the model in config/config.toml with a base_url. The README names no local model server, but the project ships config/config.example-model-ollama.toml, which points both its main model and its picture model at http://localhost:11434/v1. If you copy that file over your settings, add back the two [daytona] lines from your own config.toml. This guide did not test it.

Trade-offs: safety versus capability

A local model or a cloud model. This is the provider choice in each agent's model settings: a local server such as Ollama, or a company's cloud model reached with an API key or a sign-in.

You gain: with a local model, your prompts and files stay on your computer and there is no bill per use. With a cloud model, you get a much stronger agent that finishes long jobs.
You risk: a small local model may loop, give up, or claim it did something it did not do. A cloud model sends your prompts and attached files to the provider.
Makes sense when: a local model fits private material, such as your own notes, on a machine with about 24 GB or more, for short jobs you check afterward. It does not make sense on an 8 GB or 16 GB laptop for multi-step work, where a cloud model with a spend limit will do far better.

A longer context length. The Ollama slider, or OLLAMA_CONTEXT_LENGTH, sets how much of the conversation the model can hold at once.

You gain: the agent keeps track of longer jobs, and agents such as Hermes will start at all.
You risk: more memory use, so a model that fit before may spill onto the processor and slow to a crawl, or fail to load.
Makes sense when: your machine has memory to spare after the model loads, which ollama ps shows. If it starts showing CPU instead of GPU, step back down.

Opening the model server to your network. Setting OLLAMA_HOST to 0.0.0.0:11434, or turning on LM Studio's Serve on Local Network, lets programs beyond localhost reach the server.

You gain: an agent in Docker, or on another computer you own, can use the model.
You risk: anyone on the same network can use your model server, which costs you memory and power and, on Ollama, needs no key.
Makes sense when: the computer stays on a home network you trust, the router does not forward the port, and, for LM Studio, authentication is on. It does not make sense on a laptop you take to public Wi-Fi.

A local model on a chat bot or a schedule. This is the same model choice, applied to an agent that answers chat messages or runs jobs while you are away.

You gain: an always-available agent with no per-use bill.
You risk: OpenClaw's docs say smaller models are more open to prompt injection, and nobody is watching when a scheduled job reads a booby-trapped page.
Makes sense when: the job only reads public pages and writes its own notes. Keep any job that sends, posts, or spends on a strong model and behind approvals.

For the three questions behind these choices, see Safety versus capability.