For every agent

Run a local model with Ollama

This page puts a local model on your own computer with Ollama and connects it to any agent in this guide. Your prompts stay on your machine and there is no bill per use, but the model is only as strong as your memory allows. On an 8 GB or 16 GB laptop, expect single-step jobs, not multi-step agent work. For jobs with many steps, use the free cloud key in Pick your model and budget instead, or keep it for the hard jobs and use the local model for the rest.

File sizes, settings, and addresses on this page were checked on 14 November 2026.

What you need

You do not need an agent yet to follow the first half of this page, since Ollama can answer in the terminal on its own.

Install Ollama

Windows

  1. Open ollama.com/download and download the Windows installer, OllamaSetup.exe.
  2. Run it. It installs in your own account without administrator rights.
  3. Open PowerShell, not as an administrator, and run:
ollama --version

What you should see a version number in PowerShell, and an Ollama icon in the taskbar's notification area.

Mac

  1. Open ollama.com/download, download the Mac app, open ollama.dmg, and drag Ollama into the Applications folder.
  2. Open Ollama. On first launch it offers to link the ollama command into /usr/local/bin, so accept that.
  3. Open Terminal and run:
ollama --version

What you should see a version number in Terminal, and an Ollama icon in the menu bar.

Note: On both systems, Ollama starts by itself when you sign in, and it downloads its own updates. To stop it starting at sign-in, find Ollama under Startup apps in Task Manager on Windows, or under Login Items in System Settings on a Mac.

Pick a model for your memory

Find the row that matches your memory and download the one model it names. "Fits" means the file is smaller than your memory with room left for the operating system, and it does not promise room for a long conversation. The judgments in the last column are this guide's, not test scores.

Your memory Model name to download File size What to expect
About 8 GB qwen3.5:4b 3.3–4.0 GB Small local chat and single-step jobs. Weak once one step depends on the last.
About 16 GB mimo-v2.6:9b 6.6–8.7 GB More comfortable chat, still weak for chained steps.
About 24 GB of unified memory or graphics memory qwen3.8:27b 18 GB The first useful tier for a short loop of tool calls, though slower than a cloud model.
About 32 GB qwen3.8:27b 18 GB The same file as the 24 GB row, with more room left for context.
About 48 GB qwen3.8:27b-q8_0 30 GB A higher-quality copy of the same 27B model.

The Qwen3.5 4B card prints a 256K context, which an 8 GB computer cannot hold. On a 24 GB computer, plan on a 32K context for the qwen3.8:27b tag. Every tag in this table can read images, call tools, and use its thinking mode.

ByteShape publishes a smaller Qwen3.8 27B file for llama.cpp, and Local models in depth has those steps. A 16 GB computer should keep the Ollama tag mimo-v2.6:9b shown in the table above.

Why small models struggle, and what bigger machines can run, is in Local models in depth. That page also names the tags for 64 GB, 80 GB, and 128 GB machines.

Download and test it

The commands are the same in PowerShell on Windows and in Terminal on a Mac. This example uses the 24 GB tier, so swap in the name from your own row:

ollama pull qwen3.8:27b

The download shows a progress bar and can take a long time. When it finishes, give the model a short job and then check where it loaded:

ollama run qwen3.8:27b "Reply with one short sentence about the weather."
ollama list
ollama ps

ollama ps has a PROCESSOR column. 100% GPU means the model fits in graphics memory, 100% CPU means it runs in ordinary memory and will be slow, and a split such as 48%/52% CPU/GPU means it sits partly in each.

What you should see a one-sentence reply, the model's name in the ollama list output, and a PROCESSOR column from ollama ps.

Give it enough context for agent work

The context length is how many tokens of the conversation the model can hold at once. An agent fills it fast, because every step resends its instructions, its tools, and the chat so far.

Ollama's context-length page says the default depends on your graphics memory: 4k tokens under 24 GiB, 32k from 24 to 48 GiB, and 256k at 48 GiB or more. Its FAQ still gives a flat 4096-token default, which matches the first tier, so most laptops start at about 4,000 tokens. The context-length page also says agents, coding tools, and web search should have at least 64000 tokens. Some agents insist, since Hermes Agent refuses to start below 64,000 and QwenPaw asks for at least 32k.

A longer context needs more memory, so raise it in one of these three ways and then check that the model still fits.

In the Ollama app. Open Ollama's settings and move the context length slider to 64k. This is the simplest route.

For the whole server, with a setting. On Windows, quit Ollama, open Settings, search for "environment variables," choose Edit environment variables for your account, and add a variable named OLLAMA_CONTEXT_LENGTH with the value 64000, then start Ollama again. On a Mac, use the slider in the Ollama app instead. A variable set in Terminal with launchctl setenv is forgotten when the Mac restarts. If you run the command below and then restart Ollama, the setting is lost at the next Mac restart and the context drops back to its default:

launchctl setenv OLLAMA_CONTEXT_LENGTH 64000

Ollama's docs show the same setting on a command line as OLLAMA_CONTEXT_LENGTH=64000 ollama serve, and the steps above are its documented way to set any Ollama variable for the app.

For one model only, with a Modelfile. Ollama's OpenAI-compatible address has no way to set the context, so its docs suggest building a copy of the model with the length built in. Save a plain text file named Modelfile, with no extension, containing these two lines and your own model name. In Notepad, set Save as type to All files so it is not saved as Modelfile.txt.

FROM qwen3.8:27b
PARAMETER num_ctx 64000

Then, in the same folder, run ollama create qwen38-64k -f Modelfile, and give your agent the name qwen38-64k.

Whichever way you choose, run your model once more with ollama run, then run ollama ps.

What you should see the CONTEXT column showing the new length, with PROCESSOR unchanged. If the model now loads partly on the CPU or fails to load, the longer context does not fit, so step back down to 32k.

Keep it on your computer only

Ollama listens on 127.0.0.1 port 11434 by default, which means only programs on this computer can reach it. Leave it that way. Ollama's local server asks for no key, so anyone who can reach the port can use your model.

The prompt also leaves your computer whenever the agent uses a web tool such as search or a browser, because those reach the internet themselves.

If you want a hard stop, Ollama can switch off its cloud features entirely, which also removes its web search.

  1. Open Ollama's settings and turn off Cloud. This works on Windows and on a Mac, and it stays off after a restart.
  2. To lock it off on Windows as well, quit Ollama, open Settings, search for "environment variables," choose Edit environment variables for your account, add a variable named OLLAMA_NO_CLOUD with the value 1, and start Ollama again. On a Mac, do not rely on launchctl setenv for this, because a restart forgets it and cloud features come back on without warning. The command below, followed by a restart of Ollama, is lost when the Mac restarts:
launchctl setenv OLLAMA_NO_CLOUD 1

What you should see :cloud models no longer run, and Ollama's server log shows Ollama cloud disabled: true. On Windows, that log is server.log in the folder %LOCALAPPDATA%\Ollama, and on a Mac it is in ~/.ollama/logs.

Connect it to your agent

Each agent asks for three things: the provider, the address, and the model name. Type the model name exactly as ollama list shows it, such as qwen3.8:27b.

The address depends on where the agent runs and which kind of address it wants:

Ollama's local server does not check API keys. If an agent will not save the setting without one, Ollama's own example uses the word ollama as a placeholder.

Agent Where it runs Ollama setting
Agent Zero Docker container In first-run setup, pick Local model, choose Ollama for both the Main and Utility models, type each name, and click Finish setup. Keep its default address, http://host.docker.internal:11434. Its Tiny Local profile helps small models that describe a step instead of doing it.
Atomic Agent Normal app Its default local model runs without Ollama. To use Ollama in the desktop app, open Settings › Models › Cloud providers, click Add provider, and choose Ollama (local). It fills in http://localhost:11434 and asks for no key. The terminal version has the same preset.
Hermes Agent Normal app Run hermes model, choose "Custom endpoint (self-hosted / VLLM / etc.)", enter http://localhost:11434/v1, skip the key, and type the model name. It needs a context of at least 64,000 tokens.
OpenClaw Normal app Run openclaw onboard, choose Ollama, then Local only, and give http://127.0.0.1:11434, its default, as the base URL. Then run openclaw models set with ollama/ and the model name. It looks for a model that supports tools and has at least 16K of context. Ollama's OpenClaw page recommends 64k, so raise it as shown above.
Goose Normal app Open Settings, then Models, then Configure Providers, and choose Ollama. It uses localhost:11434 unless you set OLLAMA_HOST, and it lists the models you have downloaded.
NanoClaw Docker container Not supported as the main model through Ollama, because its Ollama provider add-on is out of date. /add-ollama-tool only lets the main model call a local one as a tool. Setup's OpenCode choice accepts a local OpenAI-compatible server, but this guide could not confirm the values or the address that work from inside its container.
QwenPaw Normal app, or a Docker image In Settings → Models, open the Ollama provider and set its Base URL to http://localhost:11434. Click Update model list, which its docs call Discover Models, then Test Connection, and pick the model under Default LLM. With the Docker image, add --add-host=host.docker.internal:host-gateway to its docker run command and use http://host.docker.internal:11434. It wants a context of at least 32k.
Eigent Normal app Open Settings, and under Global Settings click Models, then click the Ollama icon. Leave the key empty, check that API host setting shows http://localhost:11434/v1, pick your model, click Save, and choose it under Default model.
Everfern Normal app In setup, choose Local Engine, then Ollama. Leave Server URL (optional) empty to use its default, http://localhost:11434, then pick your model under Installed Models.
OpenManus Program in a terminal In config/config.toml, set the [llm] values to your model name, base_url = "http://localhost:11434/v1", and api_key = "ollama", as in its example file config/config.example-model-ollama.toml. Keep the two [daytona] lines. This guide did not test it.

What you should see a reply to a short test message in your agent, and your model listed by ollama ps while it answers.

Common fixes

The agent says it cannot connect. Ollama may not be running, so look for its icon in the taskbar or menu bar, and start it from the Start menu or the Applications folder if it is missing. Next, check the address against the list above, because a Docker agent needs host.docker.internal rather than localhost. Finally, compare the model name letter for letter with ollama list.

Replies are very slow. Run ollama ps while the agent works. If PROCESSOR shows 100% CPU or a large CPU share, the model or its context does not fit in graphics memory, so pick a smaller model from the table or step the context length down.

The agent forgets its instructions, ignores its tools, or loops. The context is probably still at the 4k default, so the start of the conversation falls out of view, which Goose's docs describe for extensions and its hints file. Raise the length as shown above. If it still loops, the model may be too small for the job, so break the job into single steps as Prompting strong and weak models shows.

Stop and uninstall

To stop Ollama for now, click its icon in the taskbar or menu bar and quit it. To free memory without quitting, unload one model with ollama stop and the model's name, such as ollama stop qwen3.8:27b. Ollama also unloads an idle model by itself after 5 minutes.

To delete one model and get its disk space back, run ollama rm and the name, such as ollama rm qwen3.8:27b.

To remove Ollama from Windows, open Settings, find Ollama under Add or remove programs, and uninstall it. If you moved your models with OLLAMA_MODELS, the uninstaller leaves them where you put them, so delete that folder yourself.

To remove Ollama from a Mac, quit it, then run these lines in Terminal. Ollama's docs list the same files, but they quote two of the paths in a way that stops ~ from meaning your home folder, so the quotes here are moved.

Warning: These lines delete files for good, including every model in ~/.ollama. The first two ask for your Mac password.

sudo rm -rf /Applications/Ollama.app
sudo rm /usr/local/bin/ollama
rm -rf ~/Library/"Application Support"/Ollama
rm -rf ~/Library/"Saved Application State"/com.electron.ollama.savedState
rm -rf ~/Library/Caches/com.electron.ollama/
rm -rf ~/Library/Caches/ollama
rm -rf ~/Library/WebKit/com.electron.ollama
rm -rf ~/.ollama

If you set OLLAMA_NO_CLOUD or OLLAMA_CONTEXT_LENGTH on Windows, remove those variables too, from Edit environment variables for your account.

What you should see ollama --version in a new terminal window now reports that the command is not found.

Trade-offs: safety versus capability

A local model or the free cloud key. This is the provider choice in your agent's model settings: Ollama on your computer, or a cloud model such as Gemini reached with a key.

A longer context length. The Ollama slider, OLLAMA_CONTEXT_LENGTH, or num_ctx in a Modelfile sets how much of the conversation the model can hold at once.

Ollama's cloud features on or off. The Cloud switch in Ollama's settings, or OLLAMA_NO_CLOUD on Windows, decides whether this copy of Ollama can reach Ollama's cloud models and web search at all.

For the three questions behind these choices, see Safety versus capability.

Sources

Next

Checked 14 November 2026 · prices, limits, and versions change