Ollama, LM Studio and MLX each pick a different part of your chip, none of them asks you, and one of them was picking wrong on the newest Macs until September 1.

Read the article for free here.

In June someone with a new MacBook Pro found that a part of their chip had never been used. Apple's newest chips put a small matrix engine inside every graphics core, and those engines do one job: the heavy math a model does while it reads your prompt. This laptop had 40 of them, and the app running its models had switched off every one.

Prompts took twice as long to read as they needed to, while replies came out at exactly the same speed, because those engines only touch the reading. The fix landed in early September, ten months after the chip went on sale.

Three Engines on One Piece of Silicon

Everything in a Mac that can run a language model sits on one chip and draws from one pool of memory. What sits on that chip is three different kinds of processor, built years apart for different purposes, and only two of them are on every Mac. The matrix units arrived with the M5. An M1 through M4 has graphics cores and a Neural Engine.

The graphics cores are the generalists, thousands of small units that will run whatever math you hand them. That flexibility is why they became the default. People were porting language models to Apple hardware years before any Mac had a matrix unit, so the general cores were the only target available, and every local LLM tool grew up pointed at them.

Each matrix unit does one operation: multiplying grids of numbers together, far faster than a general core manages. They only wake up when a program asks for them through Apple's Metal 4 tensor API (in a specific format). Get that wrong and nothing breaks. The general cores take the work and grind through it the slow way.

The Neural Engine is the odd one out. Apple has shipped it for years and every M5 has 16 of these cores. It was built for image and audio work, where the shape of the math is fixed and known in advance. Text generation is neither of those things, so it never fit well.

All three share the same memory, so nothing has to be copied between them. That is why a Mac can run big models at all.

Sharing memory does not mean the work goes to the right engine. Something has to choose. That something is the runtime, the software underneath whatever app you opened. It picks when the model loads, and it does not ask you.

Which Engine Is Running Your Model Right Now

A model answers you in two phases, and they stress the machine in opposite ways.

The first phase is reading. You hand it a prompt and it takes the whole thing in one pass, multiplying numbers until it has the context. This is called prefill, and it is limited by how fast your chip can multiply. Matrix units are built for exactly that, which is why prefill runs 3.33x to 4.06x faster on the M5 than on the M4.

The second phase is writing. The model produces one word, then reads its entire set of weights to produce the next one, and it does that again for every word in the reply. This is called decode, and it is limited by how fast memory can move rather than how fast the chip can multiply. On the same comparison, decode gained only 1.19x to 1.27x.

Those weights have a size. Qwen-14B at 4-bit is 9.16 GB, so that is what has to move for every word. 4-bit is a shrunk copy of the weights, smaller and slightly less precise.

Sparse models get around it. A 30B-Qwen holds 17.31 GB but fires only a slice of its network per word. That is how a bigger sparse model can write faster than a smaller dense one.

How fast that data moves depends on which Mac you bought.

A 614 GB/s machine writes text faster than a 153 GB/s machine, no matter how many matrix units either one has. Reading works the other way round. Even the base M5 at 153 GB/s returns the first token of a dense 14B model in under 10 seconds, and under 3 for a 30B sparse model.

This is why a tokens-per-second number on its own tells you nothing. Ask which phase it came from.

Which Engine Is Running Your Model Right Now

Every time you load a model, your Mac decides whether to use those matrix units. It writes the decision into a log you have probably never opened.

LM Studio and Ollama are front ends. The program running your model underneath is llama.cpp, unless Ollama has switched that model to MLX. When llama.cpp loads a model it checks whether it can reach the matrix units, and writes the result to that log.

If it cannot reach them, you get this line:

ggml_metal_device_init: the tensor API is not supported in this environment — disabling

If it can, the line is simply absent. A build compiled from source prints has tensor = true instead.

Where the log lives depends on the app. In LM Studio, turn on Developer mode in App Settings, then read ~/.lmstudio/server-logs or run lms log stream — source server. Ollama prints it to the terminal when you run ollama serve. mlx-lm prints it at startup. Search for tensor.

An open bug report filed in June 2026 tracked this on an M5 Max with 128 GB. The user loaded a 2,048-token prompt on two different models.

The 1,833 figure came from running the same model on an upstream llama.cpp build that passed the check. That is more than a second of dead time before the model starts typing, and it is what the matrix units buy you. The writing speed did not move in either case, as the bandwidth limit predicts.

The cause was a build setting. The app's graphics code had been compiled against an older version of Metal, and the tensor API does not exist in that version. So the finished build had no way to see it. At startup the check asked whether the tensor API was there, got no for an answer, and switched the matrix units off. Compiled from source with the newer setting, the same code passed. The repair merged on September 1, 2026.

MLX got there first. It is Apple's own machine learning framework, it added tensor API support before llama.cpp did, and that is why it was reaching the matrix units while llama.cpp was not.

Ollama moved onto MLX in version 0.19 in March 2026. In preview, for some models, and only on Macs with more than 32 GB. Their own before-and-after looks like this.

Read that second column carefully. Decode nearly doubled, and no compute engine can do that to a phase limited by memory. That half of the gain is the compression format they changed in the same release, not the engine.

A cleaner test came from one independent run, one machine, one set of weights. On an M3 Max with 64 GB running Qwen3.6–35B at 4-bit, mlx-lm produced 163 tokens per second and Ollama on llama.cpp produced 47. An M3 Max has no matrix units at all, so that gap is the runtime alone.

Where the Fast Engine Loses

MLX wins the reading phase, and it wins short chats. Past a certain length of conversation it starts losing, and the reason has nothing to do with how fast it can multiply.

Everything you have said so far gets sent back to the model with every new message. That whole pile is the context, and the model has to work out how each word in it relates to every other word. That step is called attention, and the memory it needs grows with every token. Past a point the machine spends more time moving that data around than doing the math.

llama.cpp has a fix for this called flash attention. It breaks the work into blocks so the processor touches memory far less often. MLX has no equivalent. Someone proposed adding one in December 2025, as mlx issue 2955, and that issue is now closed with nothing shipped. The proposal pointed at community-built Metal kernels that raised throughput about 77% on a 30B 4-bit model, so the technique works. It is just not in MLX.

Here is where the two cross over, measured on an M3 Ultra with 256 GB running Minimax-m2.1 at 4-bit

Those are writing speeds. MLX still wins the read at both lengths. At 146,000 tokens it writes at half llama.cpp's rate.

Note the machine. An M3 Ultra has no matrix units, so none of this is about the M5. It is a software gap, and a newer Mac does not close it. If your prompts run to whole documents or whole codebases, this is what decides your runtime, not the matrix units.

The Engine Nobody's Model Runs On

Apple ships a dedicated neural processor in every device it makes and markets it as the AI hardware. Over 2 billion devices have one and none of them run a language model on it.

Part of the reason is fit. It was built for work with fixed, known shapes, and writing text one word at a time is the opposite, so it spends more time waiting than computing. The bigger reason is that you cannot insist on it. Core ML, the public framework, treats hardware choice as a suggestion. Ask for the Neural Engine and you may get the CPU or the GPU instead, with no way to tell which one ran. Two research teams had to go around Core ML entirely (through private APIs) just to measure it.

Forced to run a language model, it is slow. ANEMLL, the main open project doing this, gets about 9 tokens per second on an 8B model where MLX on the graphics cores gets 93 or more. The trade is power and memory. ANEMLL reports about 2 W against the graphics path's 20 W, on a footprint of 98 to 230 MB rather than 390 to 4,376 MB.

Use the graphics cores for an answer you are waiting on, and the Neural Engine for something small that runs all day on battery, if you can get to it.

Choosing, by What You Own and What You Do

There is no universal winner, and the decision only has three inputs.

Start with your chip, because the matrix units only exist on the M5. Then your prompt length, because that decides whether MLX helps or hurts. Then your memory, because the model has to fit alongside the conversation.

Prompt length is the one people get wrong, since nobody counts it. As an anchor, 30,000 tokens is a long design document, or a few thousand lines of code.

The 32 GB floor in the table comes from what has to fit at once. Qwen-14B at 4-bit is 9.16 GB of weights, and a 30B sparse model is 17.31 GB. The conversation needs its own memory on top of that, and it grows with every word. On an 8 GB Mac the 14B model does not fit at all. On a 24 GB Mac it fits, and then a long conversation pushes the machine into swapping to disk, which is slower than anything else in this article. Ollama's floor is the margin that keeps that from happening.

That crossover was measured on an M3 Ultra before the tensor-API fix landed, and nobody has re-run it on an M5 since. If your prompts routinely run past 30,000 tokens, llama.cpp is still the safer bet until someone does.

Where the Choice Goes Next

That decision exists because three pieces of hardware sit behind three separate sets of tooling. Apple is trying to end it.

In June 2026 Apple announced Core AI, which replaces Core ML after 9 years. The pitch is one interface that spreads work across all three engines and picks for you, covering everything from 3B vision models up to 70B reasoning models. If it works, the question stops being which engine and becomes which model.

The hardware keeps moving too. The Mac Studio ships on September 22 with the M5 Ultra at 1.2 TB/s, roughly double the fastest laptop, and a 512 GB configuration is reported for late October.

Core ML treated hardware selection as a hint rather than a promise, and that is what made the neural processor unusable for serious work. Nothing announced so far says Core AI changed that. Until the documentation shows a developer can pin execution to one engine, you still cannot tell where the work went.

Until then it stays your call, and the answer is already sitting in a log on your machine. Open it and search for tensor.

if this helped, clap 👏 so others can find it too — and if you want the shorter, sharper cuts of stuff like this, I also post notes on substack. Further reading:

Everyone Told You to Run Ollama. 2026 Made That a Harder Call.: Which local tool to actually run now, by use case.

Qwen 3.8 27B Runs Faster on a 3090 Than a 4090. The GPU Isn't Why.: Same argument on NVIDIA: settings and runtime beat the card.

Running Local LLMs in 2026: 5 Things That Actually Matter: Five ordinary decisions that outrank your GPU and your model.

Mac Mini M4 vs RTX 5090 vs Cloud GPUs for Local AI in 2026: What to buy once you know how the hardware behaves.

What Is the Best Local LLM for Coding in 2026?: Now pick the model to point your chosen runtime at.