Don't Buy a Mac Mini For Local AI (Do This)
local-ailocal-llmapple-siliconmac-minihardwarememory-bandwidthmixture-of-experts
Verdict: For most people Kai recommends the M6 Mac mini at 32 GB ($1,299) over the $2,699 64 GB M5 Pro, since even 64 GB can't hold gpt-oss-120b with context; first change the inference server, try sparse models and rent the model you want for a month
TL;DR
- Kai (Sept 2026) almost bought a $2,699 Mac mini (M5 Pro, 64 GB) for local AI and walks through why he didn't. For most people his pick is the M6 Mac mini at 32 GB for $1,299, plus two free steps taken before buying anything.
- Apple charges exactly $25 per GB of memory across the M6, M5 Pro, M5 Max and M5 Ultra. The gap between the $899 base and the $2,699 model is memory. Even 64 GB is about 1 GB short of gpt-oss-120b before macOS and context, and the next step, 128 GB, means a Mac Studio M5 Max at $2,400 more.
- Token generation is limited by memory bandwidth: the M5 Pro's 307 GB/s against the M6's 153 GB/s roughly doubles speed. Memory prices aren't Apple's doing. AI data centers took the high-bandwidth memory supply, and the alternatives went up more (Framework Desktop $1,999 to $3,449, DGX Spark $4,699, RTX 5090 above $4,000 street).
- Free fix 1: measure time to first token. One user's 57 tok/s was 3 tok/s end to end at 8,000 tokens of context, because prefill took 94% of the time, and a different inference server cut prefill from 49s to 1.7s.
- Free fix 2: run sparse mixture-of-experts models. A 35B model with about 3B active ran at 78.8 tok/s against 14.7 for a dense 27B on the same M4 Pro. Then rent the model you want for a month before deciding local is worth it.
Full summary
The pitch, and what he nearly bought
Kai opens on the late-night version of the decision: the Mac mini product page in one tab, a calculator in another, and the hope of running AI privately instead of paying OpenAI $20 a month. A month earlier he had an M5 Pro configured in his cart, alongside a whole spreadsheet and tabs comparing it to a Mac Studio. He says he almost spent $2,700 without understanding what he was buying. He frames the video as the mistake he almost made, and says the answer may not be a machine at all.
The real price is memory
The Mac mini starts at $899, and the base machine doesn't matter for this use. Many models worth running won't load in 16 GB at all; he calls it "a hard wall where the model does not fit and so the model does not run." Moving to 24 and then 32 GB brings the price to $1,299. Buyers then read that 64 GB is where the big open-weights models fit, and the M5 Pro maxed out is $2,699. What upset him is that Apple charges exactly $25 per GB, a rate he checked across the M6 Mac mini, the M5 Pro Mac mini, the Mac Studio M5 Max and the Mac Studio M5 Ultra: four chips, two product lines, seven upgrade tiers. The $1,800 gap between the base model and the recommended one is soldered memory.
The names are confusing too. There is no M5 Mac mini, and reviews of one are about a MacBook. The two models on sale use the M6 and the M5 Pro, so the more expensive chip is a generation older. The older M5 Pro has up to 18 CPU and 20 GPU cores, 307 GB/s of memory bandwidth, Thunderbolt 5 and up to 64 GB, against the base M6's 12 CPU cores, 12 GPU cores, 153 GB/s and Thunderbolt 4.
Why bandwidth decides speed
Generation, the part where the model writes its answer, is limited almost entirely by how fast the chip reads memory; he cites Apple's own machine learning research for this. Each token means reading billions of parameters, so twice the bandwidth gives roughly twice the speed, and the Neural Engine or extra CPU cores don't change that. That is why buying guides point to the M5 Pro over the newer M6.
The 64 GB ceiling
Even the top Mac mini runs out quickly. gpt-oss-120b, a large open-weights model, needs about 66 GB in its native 4-bit format, and owners of 128 GB machines report 64 to 65 GB resident. A 64 GB Mac mini is about 1 GB short before macOS takes its share and before any context. Context is the other cost: for a 70B model the key-value cache, which holds the conversation, runs about 10 GB at 32,000 tokens and about 40 GB at 128,000. You don't run out of memory on day one, he says, but six weeks in, during a long conversation. Going from 64 to 128 GB means a different computer, a Mac Studio M5 Max, $2,400 more. He asked himself whether he wanted the machine he needed or the one that sounds impressive in the comments.
The memory market
Apple charged the same per-GB rate on the M4 Mac mini in 2024, when people called it the Apple memory tax because street DRAM cost a few dollars a gigabyte. Since then DRAM spot prices rose about 700% year over year. AI data centers bought up high-bandwidth memory, fabs moved wafer capacity from ordinary DRAM to HBM, and data centers now take about 70% of high-end production; SK Hynix has warned the squeeze may last past 2030. Apple's rate stayed put and the market rose to meet it. The data centers people want to escape by running AI locally have made local memory cost 50 times more per gigabyte than storage. The base Mac mini went from $599 at launch in late 2024 to $899, which he puts on the fabs. Other options rose more:
- Framework Desktop with Ryzen AI Max: $1,999 to $3,449 in 19 months
- Nvidia DGX Spark: $4,699
- RTX 5090: $1,999 list, above $4,000 street, for 32 GB
Two free fixes before buying
The first is the inference server. One user's software reported 57 tokens per second, but measured end to end, including the wait for the first word, he was getting 3 tokens per second at 8,000 tokens of context, because prefill (the model reading the prompt) took 94% of the response time. Benchmark videos show generation speed after the wait. Switching inference servers, with the same machine, model and quantization, cut prefill from 49 seconds to 1.7 seconds while generation stayed the same. Another user running agents on an older Ultra went from a minute or two of prefill to under 10 seconds. He notes that "download this for free" carries no affiliate link, which is why buying guides skip it.
The second is sparse models. A dense 27B model reads all its parameters for every token; a sparse 35B mixture-of-experts model might use about 3B. On the same M4 Pro, the 35B MoE ran at 78.8 tokens per second and the dense 27B at 14.7, so the bigger model was about five times faster. At 4-bit the 35B sparse model is about 17 GB, so it fits in 24 GB with room for context. Because only the active part goes through the memory bus, the bandwidth case against the M6 gets much weaker.
What he would buy
If you already depend on a model that needs 50 to 60 GB, buy the M5 Pro with 64 GB; $2,699 is a fair price in this market. The power draw helps (about 2.6 W idle against roughly 870 W for a two-GPU rig under load), and so does data that legally can't leave the building. Size it for the model you run today, since everyone he saw regret the purchase sized it for a future that changed. For most people he recommends the M6 at 32 GB for $1,299: 24 GB leaves about 19 GB usable, just above the 17 GB sparse models. Start from the 24 GB M6 configuration rather than the 16 GB one upgraded to 32, because the 24 GB version runs at 170 GB/s against 153 for the same memory and price. Then rent the model you want from a host for a month. One M5 Pro owner with 64 GB went back to a subscription for coding, citing heat, slowness and weaker models. If you truly need 128 GB, you were never buying a Mac mini. He closes by telling viewers to buy the small one, run sparse models "and leave the memory slider alone."
Key moments
- [00:02] 1 a.m. on the product page โ the $899 page with a calculator opened over it.
- [02:31] $25 per GB โ the same rate across four chips and seven upgrade tiers.
- [03:45] There is no M5 Mac mini โ M6 (153 GB/s) against the older but faster M5 Pro (307 GB/s).
- [04:41] Bandwidth sets the speed โ generation reads every parameter for each token.
- [05:28] 1 GB short โ gpt-oss-120b at about 66 GB on a 64 GB machine; the KV cache adds 10 to 40 GB. frame_0005.jpg
- [07:12] The memory shortage โ DRAM spot up about 700%; data centers take about 70% of high-end production; alternatives priced higher. frame_0007.jpg
- [09:28] Time to first token โ 57 tok/s reported, 3 tok/s measured; a server swap cut prefill from 49s to 1.7s.
- [11:01] Sparse models โ 35B MoE (about 3B active) at 78.8 tok/s against a dense 27B at 14.7. frame_0009.jpg
- [13:29] The recommendation โ M6 with 32 GB for $1,299, upgraded from the 24 GB configuration. frame_0011.jpg
- [15:19] Spend $0 first โ change the server, try a sparse model, rent for a month.
Hook microscope (0-10s)
-
Frames: 20 at 2 fps
Read these hook frames with the Read tool:
hook_frames/frame_0001.jpg(t=0.0s)hook_frames/frame_0002.jpg(t=0.5s)hook_frames/frame_0003.jpg(t=1.0s)hook_frames/frame_0004.jpg(t=1.5s)hook_frames/frame_0005.jpg(t=2.0s)hook_frames/frame_0006.jpg(t=2.5s)hook_frames/frame_0007.jpg(t=3.0s)hook_frames/frame_0008.jpg(t=3.5s)hook_frames/frame_0009.jpg(t=4.0s)hook_frames/frame_0010.jpg(t=4.5s)hook_frames/frame_0011.jpg(t=5.0s)hook_frames/frame_0012.jpg(t=5.5s)hook_frames/frame_0013.jpg(t=6.0s)hook_frames/frame_0014.jpg(t=6.5s)hook_frames/frame_0015.jpg(t=7.0s)hook_frames/frame_0016.jpg(t=7.5s)hook_frames/frame_0017.jpg(t=8.0s)hook_frames/frame_0018.jpg(t=8.5s)hook_frames/frame_0019.jpg(t=9.0s)hook_frames/frame_0020.jpg(t=9.5s)
-
Word-level transcript (29 words):
[ 0.13s] Okay,
[ 0.64s] be
[ 0.73s] honest
[ 1.12s] with
[ 1.44s] me
[ 1.64s] for
[ 1.84s] a
[ 1.92s] second.
[ 2.44s] You've
[ 2.76s] been
[ 3.17s] staring
[ 3.34s] at
[ 3.44s] that
[ 3.66s] Mac
[ 3.96s] mini
[ 4.34s] product
[ 4.98s] page,
[ 5.14s] haven't
[ 5.56s] you?
[ 5.84s] Maybe
[ 6.23s] it's
[ 6.57s] 1am
[ 6.91s] with
[ 7.22s] a
[ 7.29s] calculator
[ 8.08s] open
[ 8.44s] in
[ 8.70s] another
[ 9.60s] tab,
- 0.0-4.0s: Dark background, a minimal outline animation. A browser window draws in showing apple.com's Mac mini page at $899 with an "Add to bag" button while he says "be honest with me... staring at that Mac Mini product page".
- 4.5s: The price and product image are the whole frame as he says "haven't you?".
- 5.0-9.5s: A calculator window slides over the page and a "1:00 AM" clock appears in the corner, matching "Maybe it's 1:00 a.m. with a calculator open in another tab".
Pattern: a direct-address, second-person scenario ("be honest with me") that puts the viewer in the buying moment, then a confession ("I was that person") to set up the warning. The motion graphics act out each phrase as it's spoken.
Editorial profile
No scene-change data โ likely a static/screen-recorded source.
Faceless voiceover explainer with minimal motion graphics on a dark background (outline icons, memory-cell grids, labelled boxes), each animation built to the line being said; no scene cuts detected, and the numbers carry the video.
Quotable moments
- [02:02] "There is no version of this where it's just a little slow and it's a hard wall where the model does not fit and so the model does not run."
- [03:03] "Apple isn't really selling us a Mac Mini. It is running a flat commodity meter on DRAM and bolting a computer to it."
- [10:07] "The waiting is the part they don't show us."
- [10:53] "It's the step the buying guide skip over because there is no affiliate link in download this for free."
- [15:50] "Leave the memory slider alone because once that order goes through, the only part of the machine anyone can still change is their mind."
Entities mentioned
- People: Kai (channel), an unnamed benchmark poster, an M5 Pro owner who returned to a subscription
- Companies: Apple, OpenAI, Nvidia, Framework, AMD, SK Hynix
- Tools / products: Mac mini (M6, M5 Pro), Mac Studio (M5 Max, M5 Ultra), M4 Mac mini, gpt-oss-120b, Framework Desktop (Ryzen AI Max), Nvidia DGX Spark, RTX 5090
Concepts surfaced
- memory-bandwidth-bound-generation: generating each token means reading the active weights from memory, so bandwidth (GB/s) sets local speed more than core count does.
- time-to-first-token: prefill (reading the prompt) can dominate the wait at long context; tokens-per-second figures leave it out, and the inference server changes it a lot.
- mixture-of-experts: sparse models activate a few billion parameters per token, so a bigger MoE model can run faster than a smaller dense one on the same memory bus.
- KV cache cost: context needs its own memory (about 10 GB at 32k and 40 GB at 128k tokens for a 70B model), so a model that barely loads won't hold a long conversation.
- Size for today's model: buy for the model you run now and rent the one you want before buying.
Frame timeline
What's said while each frame is on screen.
[00:00]

Okay, be honest with me for a second. You've been staring at that Mac Mini product page, haven't you? Maybe it's 1:00 a.m. with a calculator open in another tab, and you're telling yourself that if you skip eating out for 3 months, you can swing the 64 gig M5 Pro because you've watched every review and read every Reddit thread. And somewhere deep down, you've convinced yourself that this is the machine that finally lets you run AI locally and privately on your own hardware...
[01:20]

the honeymoon is over. But this isn't a Mac Mini bad video, which would be too easy and also kind of wrong. It's really more of a let me save you from the mistake I almost made video because the answer here is not what any of the buying guides said, and it might not even be a machine at all. So, let's start with the number that genuinely annoyed me when I finally did the math properly. The Mac Mini starts at $899. And that is the figure Apple puts in the headline and the...
[02:41]

we are staring at $2,699. Now, what genuinely upset me a little when it finally clicked is that Apple charges exactly $25 per GB of memory. And that is exactly, not approximately. I went and checked it across the entire lineup, which is the M6 Mac Mini, the M5 Pro Mac Mini, the Mac Studio M5 Max, and the Mac Studio M5 Ultra. That's four chips, two product lines, and seven upgrade tiers. And every single one prices out at exactly $25 a gigabyte. So, Apple isn't really...
[04:02]

an M6 and the Mac Mini with an M5 Pro. So, the more expensive chip is a generation older than the cheaper one, and I know how that sounds, so just sit with it for a second. The M6 is Apple's latest chip, but it is the base chip with 12 CPU cores, 12 GPU cores, 153 GB per second of memory bandwidth, and Thunderbolt 4. The M5 Pro, meanwhile, is last generation's professional chip with up to 18 CPU cores, 20 GPU cores, 307 GB per second, and Thunderbolt 5. It moves memory...
[05:23]

reason every buying guide this week points at it over the shiny new M6. But this is where I want to slow down because this is the part that actually stopped me from buying that machine. Even that maxed out M5 Pro has a ceiling and we hit it a lot faster than you would think. Take GPT O 120B, which is one of the big open weights models people are actually excited about right now. It's the kind of model that gives local AI its real appeal and it needs about 66 GB in its...
[06:44]

because it means buying a completely different computer. And Apple's cheapest 128 GB machine is a Mac Studio M5 Max at $5,99. So, the gap between being comfortable at 64 and being comfortable at 128 is $2,400 in a different product line. Now, the point of this isn't to depress anyone, but when it was all laid out in front of me, I had to ask myself honestly whether I was buying the machine I need or the machine that sounds impressive in the comments. And the part that was...
[08:05]

has warned this squeeze might not ease until past 2030. Apple's rate didn't move and the market moved on to Apple's rate instead. And that is a bit dark when we think about it. We are trying to run AI locally to get away from renting compute from data centers. And those same data centers have made the memory in our local machine cost 50 times more per gigabyte than storage. And the base Mac Mini was $599 when it launched in late 2024 and it is $899 today. So the cost went...
[09:26]

everything is expensive. Goodbye. There are two things and one of them costs nothing while the other is much cheaper than the machine on that product page. The one that costs nothing comes from a benchmark someone posted that broke my brain a little the first time around. His software interface was reporting 57 tokens per second. And he thought his machine was flying. Then he measured it properly end to end counting the time before the first word appeared. And he was...
[10:46]

leaving performance on the table and most people never find out because they are too busy looking at the tokens per second stat rather than the time to first token. So before we buy anything, we should change the server. It's the step the buying guide skip over because there is no affiliate link in download this for free. Now the bigger insight is the model shift because the models have moved sideways in a way that actually helps us. mixture of experts architectures which...
[12:07]

could run the sparse model on 24 GB and still have memory left over for context. And it will be faster, smarter, and more enjoyable to talk to than the dense model on a machine with twice the memory. This also completely reframes the base model question because the 16 gig M6 can't run a 17 GB model, so 24 is the floor. But once we are at 24 or 32, sparse models open up and the bandwidth argument against the M6 gets a lot weaker. Now we aren't feeding the whole model through...
[13:28]

NDA because local means local and no token price competes with that. But size it for the model you run today and not the model you hope to run because every person in every thread who regrets the purchase sized it for a future that moved. For everyone else, and I think that is most of us, the buy is the M6 at 32 GB. That's a $400 step up from the base and it lands at $1,299. The reason it is 32 specifically is that a 24 GB machine has about 19 GB usable once the OS takes...
[14:49]

than the benchmark video, the Reddit thread, or even this video. One M5 Pro owner with 64 GB put it plainly, saying that between the heat, the slowness, and the intelligence trade-offs, local wasn't worth it. So, he went back to a subscription for coding even though he had bought the exact machine this whole video is about. And if after all of that the answer comes back that we genuinely need 128 GB, then we were never buying a Mac Mini in the first place. The $1,000 we...
Transcript
Source: captions.
[00:03] Okay, be honest with me for a second. You've been staring at that Mac Mini
[00:05] You've been staring at that Mac Mini product page, haven't you? Maybe it's
[00:07] product page, haven't you? Maybe it's 1:00 a.m. with a calculator open in
[00:09] 1:00 a.m. with a calculator open in another tab, and you're telling yourself
[00:11] another tab, and you're telling yourself that if you skip eating out for 3
[00:13] that if you skip eating out for 3 months, you can swing the 64 gig M5 Pro
[00:16] months, you can swing the 64 gig M5 Pro because you've watched every review and
[00:18] because you've watched every review and read every Reddit thread. And somewhere
[00:20] read every Reddit thread. And somewhere deep down, you've convinced yourself
[00:22] deep down, you've convinced yourself that this is the machine that finally
[00:24] that this is the machine that finally lets you run AI locally and privately on
[00:27] lets you run AI locally and privately on your own hardware without paying OpenAI
[00:29] your own hardware without paying OpenAI $20 a month forever. And I know this
[00:32] $20 a month forever. And I know this because I was that person completely. A
[00:35] because I was that person completely. A month ago, I was embarrassingly obsessed
[00:38] month ago, I was embarrassingly obsessed with buying a Mac Mini for Local AI.
[00:40] with buying a Mac Mini for Local AI. There was a whole spreadsheet. The M5
[00:42] There was a whole spreadsheet. The M5 Pro was configured on Apple's website
[00:44] Pro was configured on Apple's website sitting in my cart and there were tabs
[00:46] sitting in my cart and there were tabs open comparing it to a Mac Studio. And
[00:48] open comparing it to a Mac Studio. And I'm not exaggerating when I say I almost
[00:50] I'm not exaggerating when I say I almost pulled the trigger on a $2,700
[00:53] pulled the trigger on a $2,700 machine without fully understanding what
[00:55] machine without fully understanding what it actually was. Now, the part nobody is
[00:58] it actually was. Now, the part nobody is saying out loud in these buying guides
[01:00] saying out loud in these buying guides is that the headline price is not the
[01:02] is that the headline price is not the real price. The machine Apple is
[01:04] real price. The machine Apple is advertising at $899 is not the machine
[01:07] advertising at $899 is not the machine we would be buying for Local AI. And the
[01:10] we would be buying for Local AI. And the machine we would actually need starts at
[01:12] machine we would actually need starts at $2,699.
[01:14] $2,699. And even then, there is a ceiling that
[01:16] And even then, there is a ceiling that will frustrate us about six weeks after
[01:18] will frustrate us about six weeks after the honeymoon is over. But this isn't a
[01:20] the honeymoon is over. But this isn't a Mac Mini bad video, which would be too
[01:22] Mac Mini bad video, which would be too easy and also kind of wrong. It's really
[01:25] easy and also kind of wrong. It's really more of a let me save you from the
[01:27] more of a let me save you from the mistake I almost made video because the
[01:29] mistake I almost made video because the answer here is not what any of the
[01:31] answer here is not what any of the buying guides said, and it might not
[01:32] buying guides said, and it might not even be a machine at all. So, let's
[01:34] even be a machine at all. So, let's start with the number that genuinely
[01:36] start with the number that genuinely annoyed me when I finally did the math
[01:38] annoyed me when I finally did the math properly. The Mac Mini starts at $899.
[01:41] properly. The Mac Mini starts at $899. And that is the figure Apple puts in the
[01:43] And that is the figure Apple puts in the headline and the figure that gets people
[01:45] headline and the figure that gets people excited. And for basic use, it is a
[01:47] excited. And for basic use, it is a reasonable computer, but we are not here
[01:49] reasonable computer, but we are not here for a reasonable computer. We are here
[01:51] for a reasonable computer. We are here to run large language models locally.
[01:54] to run large language models locally. And the moment that becomes the goal,
[01:56] And the moment that becomes the goal, the base machine is essentially
[01:57] the base machine is essentially irrelevant. So a lot of the models worth
[02:00] irrelevant. So a lot of the models worth running won't even load on the 16 GB
[02:02] running won't even load on the 16 GB base model. And that doesn't mean they
[02:04] base model. And that doesn't mean they run slowly. it means they don't run at
[02:06] run slowly. it means they don't run at all since there is no version of this
[02:08] all since there is no version of this where it's just a little slow and it's a
[02:09] where it's just a little slow and it's a hard wall where the model does not fit
[02:11] hard wall where the model does not fit and so the model does not run. So we
[02:13] and so the model does not run. So we step up to 24 gigs and then 32 and
[02:16] step up to 24 gigs and then 32 and suddenly we are at $1,299.
[02:23] And then we start reading about how 64 GB is really where we want to be for
[02:25] GB is really where we want to be for serious use because that is where the
[02:27] serious use because that is where the big open weights models actually live
[02:29] big open weights models actually live comfortably. And now we've done it
[02:32] comfortably. And now we've done it because we've slid that memory slider
[02:33] because we've slid that memory slider all the way to the top on the M5 Pro and
[02:36] all the way to the top on the M5 Pro and we are staring at $2,699.
[02:40] we are staring at $2,699. Now, what genuinely upset me a little
[02:42] Now, what genuinely upset me a little when it finally clicked is that Apple
[02:44] when it finally clicked is that Apple charges exactly $25 per GB of memory.
[02:48] charges exactly $25 per GB of memory. And that is exactly, not approximately.
[02:50] And that is exactly, not approximately. I went and checked it across the entire
[02:52] I went and checked it across the entire lineup, which is the M6 Mac Mini, the M5
[02:56] lineup, which is the M6 Mac Mini, the M5 Pro Mac Mini, the Mac Studio M5 Max, and
[02:58] Pro Mac Mini, the Mac Studio M5 Max, and the Mac Studio M5 Ultra. That's four
[03:01] the Mac Studio M5 Ultra. That's four chips, two product lines, and seven
[03:03] chips, two product lines, and seven upgrade tiers. And every single one
[03:06] upgrade tiers. And every single one prices out at exactly $25 a gigabyte.
[03:09] prices out at exactly $25 a gigabyte. So, Apple isn't really selling us a Mac
[03:12] So, Apple isn't really selling us a Mac Mini. It is running a flat commodity
[03:14] Mini. It is running a flat commodity meter on DRAM and bolting a computer to
[03:16] meter on DRAM and bolting a computer to it. And none of this is to say Apple is
[03:18] it. And none of this is to say Apple is uniquely evil here. And we will get to
[03:20] uniquely evil here. And we will get to why in a minute, but it's worth feeling
[03:22] why in a minute, but it's worth feeling what that number felt like, which was
[03:24] what that number felt like, which was the slight vertigo of realizing that the
[03:26] the slight vertigo of realizing that the $1,800 gap between the base model and
[03:29] $1,800 gap between the base model and the model people are actually
[03:31] the model people are actually recommending is just memory, and it's
[03:33] recommending is just memory, and it's soldered, non-upgradable, buy it now or
[03:36] soldered, non-upgradable, buy it now or forever, hold your peace memory at that.
[03:39] forever, hold your peace memory at that. Now, the naming legitimately confused me
[03:42] Now, the naming legitimately confused me for an embarrassing amount of time, so
[03:44] for an embarrassing amount of time, so it's worth clearing up right now. There
[03:46] it's worth clearing up right now. There is no M5 Mac Mini, and there never was
[03:48] is no M5 Mac Mini, and there never was one. So, if you searched for an M5 Mac
[03:50] one. So, if you searched for an M5 Mac Mini and found reviews, those reviews
[03:53] Mini and found reviews, those reviews are about a MacBook. The two machines
[03:55] are about a MacBook. The two machines that went on sale are the Mac Mini with
[03:57] that went on sale are the Mac Mini with an M6 and the Mac Mini with an M5 Pro.
[04:00] an M6 and the Mac Mini with an M5 Pro. So, the more expensive chip is a
[04:02] So, the more expensive chip is a generation older than the cheaper one,
[04:04] generation older than the cheaper one, and I know how that sounds, so just sit
[04:06] and I know how that sounds, so just sit with it for a second. The M6 is Apple's
[04:09] with it for a second. The M6 is Apple's latest chip, but it is the base chip
[04:12] latest chip, but it is the base chip with 12 CPU cores, 12 GPU cores, 153 GB
[04:17] with 12 CPU cores, 12 GPU cores, 153 GB per second of memory bandwidth, and
[04:19] per second of memory bandwidth, and Thunderbolt 4. The M5 Pro, meanwhile, is
[04:22] Thunderbolt 4. The M5 Pro, meanwhile, is last generation's professional chip with
[04:24] last generation's professional chip with up to 18 CPU cores, 20 GPU cores, 307 GB
[04:30] up to 18 CPU cores, 20 GPU cores, 307 GB per second, and Thunderbolt 5. It moves
[04:32] per second, and Thunderbolt 5. It moves memory almost exactly twice as fast and
[04:35] memory almost exactly twice as fast and that matters enormously for local AI.
[04:38] that matters enormously for local AI. And the Pro can be configured all the
[04:39] And the Pro can be configured all the way to 64 GB. So why does bandwidth
[04:43] way to 64 GB. So why does bandwidth matter so much? Token generation, which
[04:45] matter so much? Token generation, which is the part where the model is actually
[04:47] is the part where the model is actually talking to us and producing words, is
[04:49] talking to us and producing words, is bottlenecked almost entirely by how fast
[04:52] bottlenecked almost entirely by how fast the chip can read from memory. And that
[04:54] the chip can read from memory. And that isn't just my opinion. Apple's own
[04:56] isn't just my opinion. Apple's own machine learning research post says
[04:58] machine learning research post says exactly that. Every time the model
[05:00] exactly that. Every time the model generates a token, it reads through
[05:02] generates a token, it reads through billions of parameters in memory. So if
[05:04] billions of parameters in memory. So if the chip can read memory twice as fast,
[05:07] the chip can read memory twice as fast, the model talks to us roughly twice as
[05:09] the model talks to us roughly twice as fast. And neither the neural engine nor
[05:11] fast. And neither the neural engine nor the extra CPU cores save us here. It is
[05:14] the extra CPU cores save us here. It is a memory bandwidth race. And the prochip
[05:16] a memory bandwidth race. And the prochip wins it comfortably. That's the real
[05:18] wins it comfortably. That's the real reason every buying guide this week
[05:20] reason every buying guide this week points at it over the shiny new M6. But
[05:23] points at it over the shiny new M6. But this is where I want to slow down
[05:25] this is where I want to slow down because this is the part that actually
[05:26] because this is the part that actually stopped me from buying that machine.
[05:29] stopped me from buying that machine. Even that maxed out M5 Pro has a ceiling
[05:31] Even that maxed out M5 Pro has a ceiling and we hit it a lot faster than you
[05:33] and we hit it a lot faster than you would think. Take GPT O 120B, which is
[05:37] would think. Take GPT O 120B, which is one of the big open weights models
[05:39] one of the big open weights models people are actually excited about right
[05:41] people are actually excited about right now. It's the kind of model that gives
[05:43] now. It's the kind of model that gives local AI its real appeal and it needs
[05:45] local AI its real appeal and it needs about 66 GB in its native 4-bit format.
[05:49] about 66 GB in its native 4-bit format. And owners on 128 GB machines reported
[05:52] And owners on 128 GB machines reported sitting at 64 to 65 GB resident. So, the
[05:56] sitting at 64 to 65 GB resident. So, the top Mac Mini with 64 gigs is, and I'm
[05:59] top Mac Mini with 64 gigs is, and I'm not joking, roughly 1 GBTE short before
[06:02] not joking, roughly 1 GBTE short before Mac OS takes its share, before we load
[06:04] Mac OS takes its share, before we load any context, and before we do anything
[06:06] any context, and before we do anything at all. And context is the other half of
[06:09] at all. And context is the other half of this problem that nobody talks about
[06:10] this problem that nobody talks about until after they've bought the machine.
[06:12] until after they've bought the machine. When we look at a 70 billion parameter
[06:14] When we look at a 70 billion parameter model, its key value cache is the thing
[06:16] model, its key value cache is the thing that holds our conversation history. So,
[06:18] that holds our conversation history. So, the model doesn't forget what we said
[06:20] the model doesn't forget what we said three messages ago. That cache runs
[06:23] three messages ago. That cache runs about 10 GB at 32,000 tokens and about
[06:26] about 10 GB at 32,000 tokens and about 40 GB at 128,000. So we don't run out of
[06:30] 40 GB at 128,000. So we don't run out of memory on day 1. We run out 6 weeks in
[06:32] memory on day 1. We run out 6 weeks in on a long conversation in the middle of
[06:35] on a long conversation in the middle of something important. And the step from
[06:37] something important. And the step from 64 GB to 128 is not a slider upgrade
[06:41] 64 GB to 128 is not a slider upgrade because it means buying a completely
[06:43] because it means buying a completely different computer. And Apple's cheapest
[06:45] different computer. And Apple's cheapest 128 GB machine is a Mac Studio M5 Max at
[06:49] 128 GB machine is a Mac Studio M5 Max at $5,99.
[06:51] $5,99. So, the gap between being comfortable at
[06:53] So, the gap between being comfortable at 64 and being comfortable at 128 is
[06:57] 64 and being comfortable at 128 is $2,400
[06:58] $2,400 in a different product line. Now, the
[07:01] in a different product line. Now, the point of this isn't to depress anyone,
[07:03] point of this isn't to depress anyone, but when it was all laid out in front of
[07:05] but when it was all laid out in front of me, I had to ask myself honestly whether
[07:07] me, I had to ask myself honestly whether I was buying the machine I need or the
[07:09] I was buying the machine I need or the machine that sounds impressive in the
[07:11] machine that sounds impressive in the comments. And the part that was
[07:13] comments. And the part that was strangely comforting in a misery loves
[07:15] strangely comforting in a misery loves company kind of way is that Apple's rate
[07:18] company kind of way is that Apple's rate per gigabyte isn't new. And Apple
[07:21] per gigabyte isn't new. And Apple charged exactly the same rate back in
[07:23] charged exactly the same rate back in 2024 on the M4 Mac Mini and people
[07:26] 2024 on the M4 Mac Mini and people called it the Apple memory tax back then
[07:29] called it the Apple memory tax back then because Street Dram was a few dollars a
[07:31] because Street Dram was a few dollars a gigabyte and everyone dunked on it
[07:33] gigabyte and everyone dunked on it rightfully. But then DM spot prices shot
[07:36] rightfully. But then DM spot prices shot up roughly 700% year-over-year. That
[07:40] up roughly 700% year-over-year. That happened because the AI data centers
[07:42] happened because the AI data centers which are the very same infrastructure
[07:44] which are the very same infrastructure we are trying to escape by buying a
[07:45] we are trying to escape by buying a local machine bought up basically all of
[07:48] local machine bought up basically all of the high bandwidth memory production.
[07:50] the high bandwidth memory production. The fabs shifted wafer capacity away
[07:53] The fabs shifted wafer capacity away from ordinary DAM toward HBM and the
[07:56] from ordinary DAM toward HBM and the data centers now consume around 70% of
[07:59] data centers now consume around 70% of high-end production. While SKH Highix
[08:01] high-end production. While SKH Highix has warned this squeeze might not ease
[08:03] has warned this squeeze might not ease until past 2030. Apple's rate didn't
[08:06] until past 2030. Apple's rate didn't move and the market moved on to Apple's
[08:08] move and the market moved on to Apple's rate instead. And that is a bit dark
[08:10] rate instead. And that is a bit dark when we think about it. We are trying to
[08:12] when we think about it. We are trying to run AI locally to get away from renting
[08:14] run AI locally to get away from renting compute from data centers. And those
[08:16] compute from data centers. And those same data centers have made the memory
[08:18] same data centers have made the memory in our local machine cost 50 times more
[08:21] in our local machine cost 50 times more per gigabyte than storage. And the base
[08:24] per gigabyte than storage. And the base Mac Mini was $599 when it launched in
[08:26] Mac Mini was $599 when it launched in late 2024 and it is $899 today. So the
[08:30] late 2024 and it is $899 today. So the cost went up 50%. And that increase is
[08:33] cost went up 50%. And that increase is the fabs, not Apple. And before anyone
[08:36] the fabs, not Apple. And before anyone pivots to just getting a Windows machine
[08:37] pivots to just getting a Windows machine with an RTX card, the alternatives
[08:40] with an RTX card, the alternatives actually got hit harder. The Framework
[08:42] actually got hit harder. The Framework Desktop with the Ryzen AI Max that
[08:44] Desktop with the Ryzen AI Max that people were recommending went from
[08:46] people were recommending went from $1,999
[08:48] $1,999 to $3,449
[08:51] to $3,449 in 19 months. And Nvidia's DJX Spark is
[08:54] in 19 months. And Nvidia's DJX Spark is $4,699.
[08:57] $4,699. And an RTX5090 has a $1,999
[09:01] And an RTX5090 has a $1,999 sticker. and a street price above 4,000
[09:04] sticker. and a street price above 4,000 for 32 GB. So, there is no secret clever
[09:07] for 32 GB. So, there is no secret clever machine hiding around the corner from
[09:09] machine hiding around the corner from the Mac Mini. There is a memory shortage
[09:11] the Mac Mini. There is a memory shortage and every box in this category is priced
[09:14] and every box in this category is priced inside it. Okay, so after all of that,
[09:17] inside it. Okay, so after all of that, what do we actually do? An answer was
[09:19] what do we actually do? An answer was promised and it's not going to be
[09:22] promised and it's not going to be everything is expensive. Goodbye. There
[09:24] everything is expensive. Goodbye. There are two things and one of them costs
[09:26] are two things and one of them costs nothing while the other is much cheaper
[09:28] nothing while the other is much cheaper than the machine on that product page.
[09:30] than the machine on that product page. The one that costs nothing comes from a
[09:32] The one that costs nothing comes from a benchmark someone posted that broke my
[09:34] benchmark someone posted that broke my brain a little the first time around.
[09:36] brain a little the first time around. His software interface was reporting 57
[09:39] His software interface was reporting 57 tokens per second. And he thought his
[09:41] tokens per second. And he thought his machine was flying. Then he measured it
[09:43] machine was flying. Then he measured it properly end to end counting the time
[09:46] properly end to end counting the time before the first word appeared. And he
[09:48] before the first word appeared. And he was actually getting three tokens per
[09:50] was actually getting three tokens per second at 8,000 tokens of context
[09:53] second at 8,000 tokens of context because prefill, which is the part where
[09:55] because prefill, which is the part where the model reads the prompt before it
[09:57] the model reads the prompt before it starts responding, was eating 94% of his
[10:01] starts responding, was eating 94% of his total response time. And every tokens
[10:03] total response time. And every tokens per second number floating around in the
[10:05] per second number floating around in the benchmark videos we have all seen is the
[10:07] benchmark videos we have all seen is the generation speed measured after the
[10:10] generation speed measured after the waiting is over. The waiting is the part
[10:12] waiting is over. The waiting is the part they don't show us. Then he switched his
[10:14] they don't show us. Then he switched his inference server on the same machine
[10:16] inference server on the same machine with the same model and the same
[10:17] with the same model and the same quantization. Prefill at that context
[10:20] quantization. Prefill at that context length went from 49 seconds to 1.7
[10:23] length went from 49 seconds to 1.7 seconds while generation didn't move
[10:25] seconds while generation didn't move because generation was never what was
[10:27] because generation was never what was broken. So he got a completely
[10:29] broken. So he got a completely transformed user experience without
[10:31] transformed user experience without spending a single dollar. And another
[10:33] spending a single dollar. And another user running agents on an older Ultra
[10:35] user running agents on an older Ultra reported that the same swap took prefill
[10:38] reported that the same swap took prefill from a minute or two down to under 10
[10:40] from a minute or two down to under 10 seconds. The default inference server is
[10:42] seconds. The default inference server is leaving performance on the table and
[10:44] leaving performance on the table and most people never find out because they
[10:46] most people never find out because they are too busy looking at the tokens per
[10:48] are too busy looking at the tokens per second stat rather than the time to
[10:50] second stat rather than the time to first token. So before we buy anything,
[10:52] first token. So before we buy anything, we should change the server. It's the
[10:54] we should change the server. It's the step the buying guide skip over because
[10:56] step the buying guide skip over because there is no affiliate link in download
[10:58] there is no affiliate link in download this for free. Now the bigger insight is
[11:01] this for free. Now the bigger insight is the model shift because the models have
[11:03] the model shift because the models have moved sideways in a way that actually
[11:05] moved sideways in a way that actually helps us. mixture of experts
[11:08] helps us. mixture of experts architectures which are the sparse
[11:09] architectures which are the sparse models are now serious. In practice, a
[11:12] models are now serious. In practice, a dense 27 billion parameter model reads
[11:16] dense 27 billion parameter model reads all 27 billion parameters for every
[11:18] all 27 billion parameters for every single token it generates. But a sparse
[11:21] single token it generates. But a sparse 35 billion parameter model might only
[11:23] 35 billion parameter model might only have 3 billion parameters active per
[11:25] have 3 billion parameters active per token because the rest are dormant
[11:27] token because the rest are dormant experts that aren't needed right now. On
[11:30] experts that aren't needed right now. On an M4 Pro machine on the same afternoon
[11:32] an M4 Pro machine on the same afternoon and on the same box, a 35 billion
[11:35] and on the same box, a 35 billion parameter mixture of experts model ran
[11:37] parameter mixture of experts model ran at 78.8 tokens per second and a dense 27
[11:41] at 78.8 tokens per second and a dense 27 billion ran at 14.7. So the bigger model
[11:44] billion ran at 14.7. So the bigger model was five times faster to talk to. And
[11:47] was five times faster to talk to. And the reason is that the bandwidth penalty
[11:49] the reason is that the bandwidth penalty for the sparse model is only a fraction
[11:51] for the sparse model is only a fraction since we are reading 3 billion
[11:53] since we are reading 3 billion parameters worth per token rather than
[11:55] parameters worth per token rather than 35 billion. A 35 billion sparse model in
[11:58] 35 billion. A 35 billion sparse model in 4-bit quantization sits at about 17 GB
[12:01] 4-bit quantization sits at about 17 GB and a dense 27B sits at around 9. So we
[12:05] and a dense 27B sits at around 9. So we could run the sparse model on 24 GB and
[12:07] could run the sparse model on 24 GB and still have memory left over for context.
[12:10] still have memory left over for context. And it will be faster, smarter, and more
[12:12] And it will be faster, smarter, and more enjoyable to talk to than the dense
[12:14] enjoyable to talk to than the dense model on a machine with twice the
[12:16] model on a machine with twice the memory. This also completely reframes
[12:18] memory. This also completely reframes the base model question because the 16
[12:21] the base model question because the 16 gig M6 can't run a 17 GB model, so 24 is
[12:24] gig M6 can't run a 17 GB model, so 24 is the floor. But once we are at 24 or 32,
[12:28] the floor. But once we are at 24 or 32, sparse models open up and the bandwidth
[12:31] sparse models open up and the bandwidth argument against the M6 gets a lot
[12:33] argument against the M6 gets a lot weaker. Now we aren't feeding the whole
[12:35] weaker. Now we aren't feeding the whole model through that memory bus for every
[12:37] model through that memory bus for every token. We are only feeding the small
[12:39] token. We are only feeding the small active part of it. So this is how it
[12:41] active part of it. So this is how it shakes out and it is the honest version
[12:44] shakes out and it is the honest version rather than the buy the most expensive
[12:46] rather than the buy the most expensive thing version. Maybe you already know
[12:48] thing version. Maybe you already know the model you run and you've tested it
[12:50] the model you run and you've tested it and you depend on it and it needs 50 to
[12:52] and you depend on it and it needs 50 to 60 GB. So, in that case, buy the M5 Pro
[12:55] 60 GB. So, in that case, buy the M5 Pro at 64 gigs because $2,699
[12:59] at 64 gigs because $2,699 is a real price in this market, and the
[13:01] is a real price in this market, and the machine is genuinely good. The power
[13:04] machine is genuinely good. The power argument alone is serious. Since a Mac
[13:06] argument alone is serious. Since a Mac Mini idles at about 2.6 W, while a
[13:08] Mini idles at about 2.6 W, while a twocar GPU rig pulls near 870 W under
[13:11] twocar GPU rig pulls near 870 W under load, if that box is on all day in a
[13:14] load, if that box is on all day in a room you sit in, the electricity bill
[13:15] room you sit in, the electricity bill difference is the product. And it is the
[13:18] difference is the product. And it is the same story if the data can't leave the
[13:20] same story if the data can't leave the building for legal reasons like
[13:21] building for legal reasons like regulated records or client code under
[13:24] regulated records or client code under NDA because local means local and no
[13:27] NDA because local means local and no token price competes with that. But size
[13:30] token price competes with that. But size it for the model you run today and not
[13:32] it for the model you run today and not the model you hope to run because every
[13:34] the model you hope to run because every person in every thread who regrets the
[13:37] person in every thread who regrets the purchase sized it for a future that
[13:39] purchase sized it for a future that moved. For everyone else, and I think
[13:41] moved. For everyone else, and I think that is most of us, the buy is the M6 at
[13:44] that is most of us, the buy is the M6 at 32 GB. That's a $400 step up from the
[13:48] 32 GB. That's a $400 step up from the base and it lands at $1,299.
[13:52] base and it lands at $1,299. The reason it is 32 specifically is that
[13:54] The reason it is 32 specifically is that a 24 GB machine has about 19 GB usable
[13:58] a 24 GB machine has about 19 GB usable once the OS takes its share and the
[14:00] once the OS takes its share and the sparse models worth running sit at 17.
[14:03] sparse models worth running sit at 17. So we clear the floor with no headroom
[14:05] So we clear the floor with no headroom whereas 32 gives us room for the model
[14:07] whereas 32 gives us room for the model and for real context. And there is one
[14:10] and for real context. And there is one more thing to keep in mind if we are
[14:11] more thing to keep in mind if we are choosing between upgrading the 16 gig to
[14:13] choosing between upgrading the 16 gig to 32 or buying the 24 gig and upgrading
[14:16] 32 or buying the 24 gig and upgrading that to 32. The 24 gig base is the one
[14:19] that to 32. The 24 gig base is the one to buy because the upgraded 16 gig model
[14:22] to buy because the upgraded 16 gig model runs at 153 GB per second while the 24
[14:25] runs at 153 GB per second while the 24 gig base chip runs at 170. That is the
[14:28] gig base chip runs at 170. That is the same amount of memory with 11% more
[14:31] same amount of memory with 11% more bandwidth at the same price point. So,
[14:33] bandwidth at the same price point. So, it's simply the better machine. Then
[14:35] it's simply the better machine. Then rent the model you think you want from a
[14:36] rent the model you think you want from a host for one month, which costs about as
[14:38] host for one month, which costs about as much as a Thunderbolt cable and let the
[14:41] much as a Thunderbolt cable and let the actual usage decide whether local AI is
[14:43] actual usage decide whether local AI is worth it for the way you work rather
[14:45] worth it for the way you work rather than the benchmark video, the Reddit
[14:47] than the benchmark video, the Reddit thread, or even this video. One M5 Pro
[14:50] thread, or even this video. One M5 Pro owner with 64 GB put it plainly, saying
[14:53] owner with 64 GB put it plainly, saying that between the heat, the slowness, and
[14:55] that between the heat, the slowness, and the intelligence trade-offs, local
[14:57] the intelligence trade-offs, local wasn't worth it. So, he went back to a
[14:59] wasn't worth it. So, he went back to a subscription for coding even though he
[15:01] subscription for coding even though he had bought the exact machine this whole
[15:03] had bought the exact machine this whole video is about. And if after all of that
[15:05] video is about. And if after all of that the answer comes back that we genuinely
[15:07] the answer comes back that we genuinely need 128 GB, then we were never buying a
[15:10] need 128 GB, then we were never buying a Mac Mini in the first place. The $1,000
[15:13] Mac Mini in the first place. The $1,000 we didn't spend on the memory slider
[15:15] we didn't spend on the memory slider becomes the deposit on the Mac Studio we
[15:17] becomes the deposit on the Mac Studio we actually needed. So, to wrap this up,
[15:20] actually needed. So, to wrap this up, here is what I wish someone had told me
[15:21] here is what I wish someone had told me 3 months ago back when that product page
[15:24] 3 months ago back when that product page was open at 1:00 a.m. The machine is
[15:26] was open at 1:00 a.m. The machine is real and the performance is real and the
[15:28] real and the performance is real and the price is real, too. But it is the market
[15:30] price is real, too. But it is the market that made it that way and not just
[15:32] that made it that way and not just Apple. And the smartest move before
[15:34] Apple. And the smartest move before spending that kind of money is to spend
[15:36] spending that kind of money is to spend $0 first by switching the inference
[15:39] $0 first by switching the inference server, trying a sparse model, and
[15:41] server, trying a sparse model, and renting the model you want for a month.
[15:43] renting the model you want for a month. That way, the experience tells us what
[15:45] That way, the experience tells us what we need rather than the spec sheet
[15:47] we need rather than the spec sheet telling us what we want. So, buy the
[15:49] telling us what we want. So, buy the small one, run the sparse models, and
[15:51] small one, run the sparse models, and leave the memory slider alone because
[15:53] leave the memory slider alone because once that order goes through, the only
[15:55] once that order goes through, the only part of the machine anyone can still
[15:57] part of the machine anyone can still change is their mind. If this helped,
[16:00] change is their mind. If this helped, drop it a like because it genuinely
[16:02] drop it a like because it genuinely helps the channel. And if you're in the
[16:03] helps the channel. And if you're in the middle of this decision right now, drop
[16:05] middle of this decision right now, drop your use case in the comments where I
[16:07] your use case in the comments where I read them and we'll try to give you an
[16:08] read them and we'll try to give you an honest answer. and I'll see you in the
[16:10] honest answer. and I'll see you in the next one.
All frames
Total: 12. Hero frames flagged with star.
frames/frame_0001.jpg(t=00:00)frames/frame_0002.jpg(t=01:21)frames/frame_0003.jpg(t=02:42)frames/frame_0004.jpg(t=04:03)frames/frame_0005.jpg(t=05:23)frames/frame_0006.jpg(t=06:44)frames/frame_0007.jpg(t=08:05)frames/frame_0008.jpg(t=09:26)frames/frame_0009.jpg(t=10:47)frames/frame_0010.jpg(t=12:08)frames/frame_0011.jpg(t=13:29)frames/frame_0012.jpg(t=14:49)