Raw research notes

Local models as an agent brain

Checked against project docs and model cards on 2026-11-14. Weight figures below are the download sizes printed on Ollama tag pages or the checkpoint sizes in the OpenAI gpt-oss model card. A weight file that fits in RAM still needs extra memory for the operating system and for the conversation (the key-value cache). Ollama’s own quickstart says larger context windows need more memory. No tokens-per-second numbers and no watt figures are given here, because none were measured for this note.

1. Install Ollama, and the local OpenAI-style URL

Ollama is the straightforward way to run an open-weight model and expose it as a local server.

Windows 11. The Windows doc requires “Windows 10 22H2 or newer, Home or Pro,” which covers Windows 11. It says: “The easiest way to install Ollama on Windows is to use the OllamaSetup.exe installer. It installs in your account without requiring Administrator rights.” After that, “Ollama will run in the background and the ollama command line is available in cmd, powershell or your favorite terminal application. As usual the Ollama API will be served on http://localhost:11434.” The installer wants about 4 GB of disk for the program, then more disk for the models (“tens to hundreds of GB”). Docs: docs.ollama.com/windows.

macOS. The macOS doc requires “MacOS Sonoma (v14) or newer” and “Apple M series (CPU and GPU support) or x86 (CPU only).” It says: “The preferred method of installation is to mount the ollama.dmg and drag-and-drop the Ollama application to the system-wide Applications folder.” On first launch the app offers to link the ollama command into /usr/local/bin. Docs: docs.ollama.com/macos. The macOS download page also prints a terminal installer: curl -fsSL https://ollama.com/install.sh | sh (ollama.com/download/mac).

OpenAI-compatible URL. Ollama’s API introduction table lists the local OpenAI-compatibility base URL as http://localhost:11434/v1 (the cloud one is https://ollama.com/v1). The same page says: “Cloud requests need an API key. Local requests do not.” The OpenAI-compatibility page’s local example is:

client = OpenAI(
    base_url='http://localhost:11434/v1/',
    api_key='ollama',  # required but ignored
)

That example calls model='gpt-oss:20b'. The quickstart’s local curl is http://localhost:11434/v1/chat/completions. Native Ollama chat (not the OpenAI shape) is http://localhost:11434/api/chat.

A model name that ends in :cloud, or a call sent to https://ollama.com, is Ollama’s hosted service. That is a different machine from yours.

2. The other path: llama.cpp and LM Studio

These are the usual alternatives when someone does not want the Ollama app. This note does not walk through their build flags.

3. What fits, and whether sources say it can chain tools

“Fits” here means the published weight file is smaller than the tier, with some room left over. It does not mean a 128K context window will fit. Tool-calling sentences on a vendor page are the vendor’s claim. Where an outside writeup measured multi-step calling, that is labeled.

Tier Models that fit Weights Multi-step tool calling, as sources actually put it
About 8 GB RAM llama3.2:3b 2.0 GB, Q4_K_M, 3.21B parameters, text only Ollama’s Llama 3.2 readme lists “Tool use” among tasks where the 3B “outperforms” smaller models, and says the text models are “optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.” That is a vendor claim about tool use, not a measured long chain. A separate paper and a personal benchmark (quoted below) say models this small fail once the call depends on an earlier tool result.
About 8 GB RAM qwen3:4b 2.5 GB, Q4_K_M, 4.02B parameters, text only The Qwen3 family readme on Ollama says the family has “Expertise in agent capabilities, enabling precise integration with external tools” and “leading performance among open-source models in complex agent-based tasks.” That sentence is the family’s own claim, and the same page gives no method. The same page also says “even a tiny model like Qwen3-4B can rival” Qwen2.5-72B-Instruct. That is a vendor sentence, not a result reproduced here. Independent multi-turn numbers for Qwen3-4B are quoted below, and they are weak.
About 16 GB RAM qwen3.5:9b Library range 6.6–7.6 GB. The tag page shows a llama.cpp file at 6.6 GB: language weights Q4_K_M, 5.6 GB, 8.95B parameters, plus a BF16 vision projector of 922 MB. Another file on the same tag is 7.6 GB. Text and image. The Ollama qwen3.5 page tags the family, including 9b, with “tools.” The Hugging Face card for Qwen3.5-4B says “Qwen3.5 excels in tool calling capabilities.” That is the publisher’s sentence. A personal gist (method stated, not a journal paper) reported one 9B run doing better on a 3-tool chain than older small models, and also reported the other test machine timing out. Treat that as the author’s result.
About 16 GB RAM qwen3:14b 9.3 GB, Q4_K_M, 14.8B parameters, text only Same Qwen3 family claim as the 4B tag. This note did not find a separate measured long tool chain for the 14B tag.
About 24 GB unified memory or VRAM gpt-oss:20b 14 GB on Ollama, quantization MXFP4, 20.9B parameters on the tag page. The OpenAI model card says 20.91B total and 3.61B active, checkpoint 12.8 GiB. Text only. This is the clearest “yes, it was trained to call tools” in this set. The model card’s agentic section says the models were taught a browsing tool, a Python tool, and “arbitrary developer functions,” and that the model “can interleave CoT, function calls, function responses, intermediate messages … and final answers.” Ollama’s page repeats “function calling” and “python tool calls.” OpenAI’s Ollama cookbook says tool calls happen inside the chain of thought, so “it’s important for you to return the reasoning returned by the API back into a subsequent call.” That is a real integration requirement, not a claim that long chains are reliable.
About 24 GB unified memory or VRAM gemma4:26b Library range 16–19 GB. Ollama describes it as a mixture-of-experts model with 25.2B total parameters and 3.8B active. Text and image. The quantization name was not printed in the library summary the way Q4_K_M is printed on the Qwen tags. The Gemma 4 readme says the family has “native function-calling support, powering highly capable autonomous agents.” That is Google/Ollama’s claim. The same page prints its own benchmark table, including a Tau2 column. Those cells are the publisher’s table, not an independent run for this note. gemma4:31b is listed at 19–20 GB (the tag page shows a 19 GB file). That file can sit in 24 GB only with little memory left for context, so 26B is the one named here.
48 GB and up llama3.3:70b 43 GB, Q4_K_M, 70.6B parameters, text only, 128K context on the library listing The Ollama Llama 3.3 page read for this note describes a dialogue model and does not claim multi-step tool calling. 43 GB of weights on a 48 GB machine leaves little spare for a long context.
48 GB and up, really the 80 GB class gpt-oss:120b 65 GB on Ollama, MXFP4. Model card: 116.83B total, 5.13B active, checkpoint 60.8 GiB. Text only. Same agentic tool section as the 20B model (“the models”). It does not fit in 48 GB. Ollama and the model card say the MXFP4 checkpoint is meant for a single 80 GB GPU.

gemma4:e2b is the model in Ollama’s own local quickstart, and it does not belong in the 8 GB tier. The quickstart says: “The model download is about 7.2 GB. We recommend 8 GB of available VRAM, or unified memory on a Mac.” That is 8 GB free for the model, on top of the operating system. The library lists gemma4:e2b at 4.6–7.5 GB.

gpt-oss:20b is often described as a 16 GB model. Ollama’s gpt-oss page says MXFP4 “enables the smaller model to run on systems with as little as 16GB memory.” The OpenAI cookbook says it is “Best with ≥16GB VRAM or unified memory.” The Ollama file is 14 GB. On a laptop whose total RAM is 16 GB, the operating system also needs memory, so this note places that model in the 24 GB tier, where the 14 GB file has room.

What sources say about small models and long tool chains

Haque et al., “TinyLLM” (arXiv:2511.22138, 27 Nov 2025), evaluate small models on the Berkeley Function Calling Leaderboard. Their Table IV, the table that is specifically multi-turn, says: “The Qwen3-4B (Prompt) and Qwen3-1.7B (Prompt) models achieve 16.88% and 8.38% respectively, reflecting limited ability to manage long-context reasoning when constrained to prompt-based function calling.” The same paragraph says Qwen3-0.6B “is almost entirely unable to sustain multi-turn performance, reaching only 1.38%,” and that TinyLlama-1.1B and TinyAgent-1.1B “record zero performance across all subcategories.” The paper then says “smaller SLMs [are] incapable of retaining sufficient context for accurate function calling across turns.”

That paper disagrees with itself on the exact percent. Table I’s multi-turn column lists Qwen3-4B at 35.25% and Qwen3-1.7B at 16.88%, while Table IV lists 16.88% and 8.38% for those two names. The direction is the same in both tables: multi-turn is the weak column, and the sub-1B models are near zero. The exact Qwen3-4B multi-turn score is not stable inside the PDF, so this note does not pick one of those two figures as the true score.

A personal GitHub gist, “Local agentic AI benchmark for 2026” (author’s own test: five queries that each need three tools in order, repeated on a Mac mini and a Lunar Lake PC), states the failure mode directly: “They fail when tool B's argument depends on tool A's output. That's the on-device cliff.” The gist’s own table gives Llama 3.2 3B a plan-and-execute rate of 33% on the Mac mini and 47% on Lunar Lake, against 8/8 on that author’s easier single-tool test. Those percentages are the gist author’s, from a small hand-built set. They are not a published leaderboard.

Long chains are hard even for models that are not small. Patil et al. (arXiv:2509.26553) write that “success rates fall sharply as sequence length grows: GPT-5 drops from 72.5% (5 core nodes) to 15.0% (20 core nodes).” That paper’s method is their FuncBenchGen framework. It is evidence about chain length, not a reason to expect an 8 GB local model to do better.

4. Families confirmed on a model card or Ollama library page

These names were opened on Ollama library pages, Hugging Face cards, or the OpenAI model card. Sizes below are the ones those pages listed. No extra version numbers are added.

Qwen2.5 is still on the Ollama library (for example qwen2.5:72b at 47 GB, Q4_K_M, 72.7B). That series is older than Qwen3 and Qwen3.5. It is listed so a 47 GB file is not mistaken for a 2026 model.

5. Privacy, and the practical costs

Privacy. A model you have pulled, called at http://localhost:11434 with no API key, runs on that computer. Ollama’s API introduction says local requests do not need an API key, and the quickstart’s local tab says “No API key required.” LM Studio’s docs say the app “can operate entirely offline” once the model files are already on disk.

The prompt stays on the machine only while the agent stays local. It leaves if any of these happen:

Speed. On a machine without a strong GPU, generation is the slow part. Ollama’s macOS doc says Intel Macs are “x86 (CPU only).” The macOS download page says: “Large models are slow on a computer without a strong GPU.” The quickstart says: “With less VRAM, Ollama can use system RAM, but responses may be slower.” OpenAI’s cookbook says you can offload gpt-oss to CPU “but expect it to run slower.”

Vision. Text-only cards cannot take an image. On the pages opened here, that is Llama 3.2, Llama 3.3, Qwen3, and gpt-oss (all marked Text). Gemma 3 (4B and up), Gemma 4, Qwen3.5, and Llama 4 are marked Text and Image. Gemma 4 E2B and E4B also list audio on the model-information table; the 31B row does not.

Electricity. No vendor watt or kilowatt-hour figure was found for this note, so none is stated. The practical cost is that the CPU or GPU stays busy for the whole reply, which draws more power than an idle computer, and a slow CPU run stays busy longer.

6. Plain recommendation

An 8 GB computer can run Llama 3.2 3B (2.0 GB) or Qwen3 4B (2.5 GB). Expect a weak agent. Those models are the ones sources actually measure failing at multi-turn or chained tool calls. A 16 GB computer can run Qwen3.5 9B or Qwen3 14B with the weight file fitting more comfortably. That is a better chat model, and still a weak agent for a long tool chain, because the published small-model results fall off once one call depends on the last one. The 14 GB gpt-oss 20B file is what vendors call a 16 GB model; on a 16 GB laptop it is at the edge.

About 24 GB of unified memory or VRAM is the first ordinary tier where a documented tool-calling model fits with room: gpt-oss 20B at 14 GB. Gemma 4 26B (16–19 GB) is the second file in that tier. This is the tier that can feel like a local agent for a short tool loop. “Feel the AGI” is an impression, not a benchmark. It will be slower than a hosted frontier model, it will not see images unless you pick a vision model, and long chains stay fragile even for much larger models (the FuncBenchGen paper’s own GPT-5 figure drops from 72.5% at 5 steps’ worth of core nodes to 15.0% at 20).

A 48 GB machine is enough for Llama 3.3 70B Q4_K_M (43 GB), with little memory left over, and that Ollama page does not claim multi-step tools. gpt-oss 120B is a 65 GB file aimed at an 80 GB GPU. That is the open model in this set whose publisher describes the same agent training as the 20B, at a size that needs a workstation well past 48 GB.

Sources

Addendum, 14 November 2026

The 14 November 2026 check replaces the local picks in the tables above. An 8 GB computer uses qwen3.5:4b (3.3–4.0 GB, and Q4_K_M is 3.3 GB), with vision, tools, thinking, and Apache 2.0, and the printed 256K context is not a promise on 8 GB. The publisher's card claims LiveCodeBench v6 55.8, TAU2-Bench 79.9, and BFCL-V4 50.3 (https://huggingface.co/Qwen/Qwen3.5-4B). A 16 GB computer uses mimo-v2.6:9b (6.6–8.7 GB), with vision, tools, and thinking, and the Ollama page states MIT. MiMo's own table, the publisher's claim and not an independent paper, lists SWE Verified 61.1 against Qwen3.5-9B at 60.0, SWE Pro 44.6 against 32.0, and Terminal Bench 2.1 37.1 against 27.0 (https://ollama.com/library/mimo-v2.6). About 24 GB for Ollama uses qwen3.8:27b at 18 GB, with vision, tools, thinking, and Apache 2.0, and 32K fits because Local Agent Bench, which is not a paper, measured 17 GB at a 32K context (https://localagentbench.com/results/). Do not promise 256K on that tag, and do not cite that bench to claim a 4B model matches Qwen3.8. The publisher's Qwen3.8-27B card claims Terminal Bench 2.1 73.0, SWE-bench Pro 61.7, and LiveCodeBench v6 90.3, while Muse Glimmer-30B on that same table is 51.7 and 51.2, so Glimmer is not the pick even though an 18 GB file fits the same machines, and no open Muse exists under 30B (https://huggingface.co/Qwen/Qwen3.8-27B). The llama.cpp file is ByteShape GPU-5 at 13.1 GB, which the publisher says keeps 99.63% of the full-precision score, and GPU-4 is 11.0 GB and keeps 98.72%; it is not an Ollama tag, a 16 GB computer stays on mimo-v2.6:9b, and GPU-1 at 8.8 GB does not fit an 8 GB computer once the operating system takes its share. At 32 GB the same qwen3.8:27b tag has more room for context, at 48 GB the tag is qwen3.8:27b-q8_0 (30 GB), and at 64–80 GB the tag is qwen3.8:27b-bf16 (56 GB), falling back to the 30 GB quant on 64 GB if the operating system and context do not fit. At 128 GB and above the tag is qwen3.5:122b (81 GB), with vision, tools, thinking, and Apache 2.0, and Muse Spark Open is not a local pull. The publisher's Qwen3.5-122B-A10B card claims SWE-bench Verified 72.0 against 62.0 and Terminal Bench 2 49.4 against 18.7, and the lower figure in each pair is Qwen's column for GPT-OSS-120B, not an OpenAI number (https://huggingface.co/Qwen/Qwen3.5-122B-A10B). Cloud-only names stay glm-5.3:cloud, glm-5.3-flash:cloud, deepseek-v4.1-flash:cloud, and muse-spark-open:cloud. https://ollama.com/library/deepseek-v4 was a 404 on this date. An older generation is not the pick at any of these sizes.

Checked 14 November 2026 · prices, limits, and versions change