Running LLMs Locally: LM Studio vs Ollama (2026)

The pitch for running a large language model on your own machine is genuinely appealing. No subscription, no rate limits, no company logging your prompts, and it works on a plane. The pitch is also, for most people, oversold. This guide covers the two tools almost everyone starts with — LM Studio and Ollama — and then does the less popular part: telling you what a model that fits on your laptop can and cannot do, using file sizes we actually checked rather than round numbers copied from a vendor’s blog.

Quick answer

What each tool actually is

LM Studio

LM Studio is a desktop application for macOS, Windows and Linux, built by Element Labs. You install it, browse a catalogue of models inside the app, click download, and chat. It handles the parts that are tedious to do by hand: picking a quantisation that fits your machine, setting a sensible context length, and wiring up the prompt template a given model expects.

Since July 2026 the flagship app has been LM Studio Bionic, which repositions the product from “a chat window for local models” to an agent that can use tools — it reads and writes project files, runs sandboxed Python, browses, produces documents and spreadsheets, and as of Bionic 1.0.8 (17 August 2026) supports installable Skills. It is also an MCP host, so the same connectors you use elsewhere work here. If you have read our guide to MCP servers, everything in it applies.

Underneath the GUI there is a command-line tool, lms, SDKs for Python and TypeScript, and both OpenAI- and Anthropic-compatible API endpoints — the latter meaning you can point Claude Code at a model running on your own machine.

Ollama

Ollama is the command-line-first option, and the more widely deployed one: the project sits at roughly 179,000 GitHub stars as of August 2026. You install it, type ollama run gemma4, and it pulls the model and drops you into a prompt. More importantly it runs a background server on localhost:11434 with an OpenAI-compatible API, which is why it has become the default way to give a script, a home-lab service, or a self-hosted app access to a model.

It has grown well past that. Recent releases added an interactive terminal UI, a Responses API with web search, and an ollama launch command that starts third-party agent harnesses — including Claude Code, Codex and Pi — against a local model. There is a desktop app too, so the old “Ollama has no interface” line is out of date.

llama.cpp — the thing underneath both

Neither tool wrote its own inference engine from scratch. Both build on llama.cpp, the MIT-licensed C/C++ project (around 125,000 GitHub stars) that made CPU and consumer-GPU inference practical, and on Apple’s MLX framework for Apple Silicon. This matters for one reason: token generation speed is a property of the engine, the model file and your hardware, not of the wrapper you chose. If someone tells you one of these apps is dramatically faster than the other, they compared different context lengths or different quantisations.

You can also skip both and use llama.cpp directly — llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF gives you an API and a web UI. It is more fiddly and considerably more configurable.

The differences that actually matter

One thing worth knowing before you commit: both companies now sell hosted inference alongside the free local product. Ollama charges $20/month for Pro and $100/month for Max, with new Max sign-ups paused in August 2026 while capacity catches up. LM Studio sells pay-as-you-go cloud credits, priced per million tokens. Local use remains free and unlimited on both — but the existence of these tiers says something honest about the state of local AI, which we come back to below.

Quantisation, explained plainly

A model’s weights are numbers. Trained at full precision, each number takes 16 bits. Quantisation stores them in fewer bits — 8, 4, sometimes fewer — which shrinks the file and the memory it occupies, at some cost to output quality.

The naming is unhelpful but the pattern is simple. Q4_K_M is a 4-bit mixed scheme and is the near-universal default: roughly a quarter the size of full precision, with quality loss that most people cannot detect in ordinary use. Q8_0 is 8-bit, about half size, effectively indistinguishable from full precision. BF16 is the unquantised original. Below 4 bits, degradation becomes obvious — repetition, dropped instructions, worse code.

There is also QAT (quantisation-aware training), where the model is trained with quantisation in mind rather than compressed afterwards. Google ships QAT builds of Gemma 4, and they are meaningfully better than a naive 4-bit compression of the same weights at a similar size. If a QAT build exists for a model you want, take it.

Here is what that looks like in real downloads, taken from the Ollama library on 19 August 2026:

Two things to notice. First, the jump from 4-bit to 8-bit roughly doubles the footprint for a quality difference most users will not notice — which is why 4-bit is the sensible default and 8-bit is a luxury for people with memory to spare. Second, “27B” and “30B” models land in almost the same place, so parameter count is a reasonable proxy for what will fit.

What you can actually run

The model file is not the whole story. You also need headroom for the KV cache, which grows with context length. Several current models advertise 256,000-token context windows; filling one costs real memory on top of the weights. Budget a few gigabytes above the file size, more if you use long contexts.

On Apple Silicon, RAM and VRAM are the same pool — unified memory — which is why a 64GB MacBook Pro runs models that would need a serious desktop GPU otherwise. Not all of it is available to the GPU: macOS reserves a portion for the system. [Insert live content] confirm the default fraction of unified memory macOS makes available to the GPU on current Apple Silicon

LM Studio’s published requirements are macOS 14 or newer on Apple Silicon with 16GB recommended (Intel Macs are not supported), and on Windows an AVX2-capable CPU with at least 4GB of dedicated VRAM. Those are floors, not targets. [Insert live content] LM Studio’s requirements page lists M1–M4 only; confirm official M5 support

If a model does not fit entirely in GPU memory, llama.cpp will split it between GPU and CPU rather than refuse. It works. It is also dramatically slower, and the experience of watching a model type at reading speed is what puts most people off local AI permanently.

Which models are worth your disk space

Gemma 4 (Google) is the easiest first model. The range runs from an E2B build that fits in 4.3GB up to 31B, all with vision and tool use, and the QAT variants punch above their file size. The 12B at 4-bit is the best fit for a 16GB machine we are aware of.

Qwen3.8 27B (Alibaba — see our Qwen profile) is the strongest general-purpose model in the size class that fits on a 32GB machine, with a 256K context window and vision input, released under Apache 2.0 in August 2026.

Muse Glimmer 30B (Meta) is Apache 2.0 and tuned for agent workloads — tool use, long tasks, recovering from failure. Try it if you want a local model driving a coding agent rather than answering questions.

gpt-oss 20B (OpenAI) remains a well-behaved general model and one of the more forgiving choices on modest hardware.

On the DeepSeek and Qwen question, two facts are worth separating. Technically, both families are competitive with anything else in their weight class, and both ship under permissive licences — Qwen3.8 27B under Apache 2.0, DeepSeek V4 under MIT. On governance: weights you download and run locally transmit nothing to anyone, which is the entire point of running them locally, and the country of origin of the training run does not change that. The data question arises only when you use a hosted endpoint, where jurisdiction, logging and retention policy differ by provider — LM Studio, for instance, serves DeepSeek V4 from US infrastructure with zero data retention, which is a different arrangement from DeepSeek’s own API. If your organisation has a policy here, it should be written against the endpoint, not the model. Our DeepSeek vs ChatGPT comparison goes into the hosted side in more detail.

The very large open models you will see in the news — DeepSeek V4 Pro, Kimi K3, GLM-5.2, MiniMax M3 — are open-weight but not locally runnable on consumer hardware at any quantisation. Both Ollama and LM Studio serve them from the cloud. That is not a workaround; it is an admission.

The quality gap, stated honestly

Epoch AI tracks capability across open and closed models. Since January 2026, the most capable open-weight models have lagged frontier closed models by an average of four months, or about 8 points on their Capabilities Index — a gap they compare to the distance between GPT-5 and GPT-5.5. That sounds narrow, and it is genuinely encouraging.

It is also not the number that applies to you. That comparison uses the best open-weight models, which are hundreds of billions of parameters and require datacentre hardware. The model on your laptop is an order of magnitude smaller. A 27B model at 4-bit is a capable assistant; it is not a frontier model with the branding filed off. It will hold a conversation, summarise a document, answer routine questions and write serviceable code. It will also fail more often on multi-step reasoning, lose the thread in long contexts, and invent facts with more confidence than it should. Anyone telling you a 7B quantised model rivals a frontier system is either benchmarking on tasks that do not discriminate, or selling something.

So should you bother?

For most casual users: no. If you want a good assistant and you are comparing against the free tiers of the major hosted products, local loses on quality, speed and convenience, and the electricity is not free either. The honest recommendation is to use the hosted tool and not feel bad about it.

The reasons that hold up:

• Privacy that is structural rather than promised. If you handle legal, medical, financial or otherwise confidential material, a model running on your machine with the network off cannot leak it. No policy document is required, because there is no transmission. This is the strongest reason by a distance.
• Offline operation. Aeroplanes, field work, air-gapped networks, unreliable connections. It works or it doesn’t; there is no service to be down.
• Volume. If you are classifying a hundred thousand documents or generating embeddings at scale, per-token API pricing adds up and a machine you already own does not meter you.
• Learning and control. Quantisation, context windows, sampling parameters and prompt templates stop being abstractions once you have to set them yourself. If you build with this technology, that understanding is worth the weekend.

If one of those applies, install Ollama if you are comfortable in a terminal and LM Studio if you are not, pull Gemma 4 at whichever size your memory allows, and give it an hour. You will know quickly whether it is for you — and if it isn’t, you have lost a download and learned something concrete about where this technology actually is.