Hardware is the exact same as what used to be available for $2K last year (and is still $1K cheaper from Chinese OEMs).
LTT Lab's LLM testing is getting more sophisticated, which is great - I think it's worth noting that ROCm/Vulkan versions and llama.cpp build versions are going to have some big differences for numbers.
For those wanting to get the most out of their Strix Halos, there's both kernel tweaks and utilities like ryzenadj that can help you get the most out of it. ( http://strixhalo.wiki/ has most of that documented). Also, if you're running for coding or agentic work, if you model supports MTP, that's mature and should give you a decent (30%?) decode boost.
it's worth noting that AMD's software is universally weak and not worth any degree of reliance. and it's not just ROCm, every few months they merge a serious regression into amdgpu and sometimes even backport it into stable. they are amateurs.
just a few weeks ago they backported a kernel oops amdgpu null dereference into stable, it's still not fixed.
The "ROCm" situation with Strix Halo was pretty bad for a while. I think it finally stabilized late last year. You needed the right combo of ROCm, Linux kernel, and kernel firmware for it to work reliably.
Whenever I rebuild llama.cpp, I wind up using the Vulkan build anyway.
And thats a pretty big annoyance. You need to perfectly line up all the holes in the swiss cheese for the AMD stack to work, then their dev team kicks you in the nuts anyway.
They have made a few attempts at investing in the hardware, but the software side is letting them down hard, and that part is almost entirely their own fault. They have underinvestment in their own stack, and into popular standard and Community libraries that would make it easy to use their gear.
Yep, it definitely wasn't an "out-of-box" experience. This was on Ubuntu 25.10. Maybe other distros are better?
Also, when you do Python work, you have to remember to install dependencies like pytorch from an alternate repo, otherwise you wind up with the CUDA versions that only use the CPU and not the GPU through ROCm.
I started a project w/ the assumption that fully tuned for HIP kernels could get closer to roofline and beat llama.cpp's performance (and vs their ROCm engine it does!) but after thousands of iterations/experiments my suspicion is that RADV/ACO is actually just much better than LLVM/HIP when it comes to a lower-level scheduling etc for RDNA3.
I haven't finished comprehensive tests but I found:
- Vulkan is up to 2.25X faster for most coalesced, strided and interleave variants for memory-side scheduling/access shapes
- 3.3X faster on specific dot-path sweeps, including for scalar-dequant
- For matched LDS, Vulkan can be 8-14X+ faster (!!!) than matched HIP LDS
HIP doesn't always win against RADV/ACO, but on dispatch/runtime, it does appear to be quite a bit faster than HIP/LLVM on gfx1151 (Strix Halo). I'll be publishing sharing full data once I also run vs gfx1100...
biggest mistake was buying an amd laptop. this thing had gpu crashes from day one, and it got worse as time went on. now just playing a video and closing the lid will crash the kernel 100% of the time. amd - never again
Without further info that’s a lot to pin on the bottom layer of the stack. I have an Asus ProArt Ryzen AI HX 390 with 64GB RAM. Running CachyOS and Pop! the thing kept falling over at random times. Running Fedora workstation, it’s rock solid. YMMV.
In case it saves anyone some time (from the article):
"The AMD Ryzen AI Max+ 395(Strix Halo) processor has been available since Spring 2025 and the Halo doesn’t offer anything new on that front."
It has the same 256 GB/s memory bandwidth limit as every board previously, not sure why this is even being released right now as if it's some new fangled thing - you can go get a Framework Desktop for roughly the same price or a GMKtec EVO-X2 for a bit cheaper.
It's being released right now because it's massively profitable and in high demand and has actually gone up in price over the past year so obviously AMD wants to cash in on that instead of selling these units to PC manufacturers at a lower price.
I also think this is forward looking and helps reanchor expectations for RAM and memory bandwidth, which historically have been areas where companies do value engineering.
The memory shortages won't last forever; when companies start adding capacity I wouldn't be surprised to see massive sticks of RAM being sold for consumers.
Before AI I kinda thought more ram and faster ram was kind of useless because I had written off increased speeds as sort of a luxury whose only application is bloated unoptimized software. Boy was I wrong about that. But then why stop there?
I mean, there's a difference between "my IDE opens 0.1s faster" and "my desktop can now write my code for me" and only one of those justifies increasing RAM production.
I got the EVO-X2 for $1,599! In 44+ years of buying computers, I've seen some appreciation, but nothing like this. Going from $1,599 to ~$3,500 in a year is just insane.
And GMKtec just released an EVO-X3 [0] based on the same board but with Oculink. When AMD launched this I thought it was the new chipset that was going to have 192GB of unified memory, but they look to be capitalizing on the inflated premium these "dev" focused desktop solutions.
Regretting not buying the Framework back when it was around $2k.
128GB of the most in demand consumer chip when all the manufacturers are switching to HBM manufacturing because the margins are far better there will do that to you. It sucks.
I was looking at buying hardware for me and as testing ground to learn. With those prices I just nipped out and will just rent in demand at runpod or so.
I had a pre-order in for the 128GB DIY motherboard only model last Oct 2025, but cancelled it and just upgraded my current 5900x rig to 64GB instead and replaced the x370 board with a used x470. Really with I had pulled the trigger, but I honestly am just fine with the performance as is on my current system for now. The Framework should have lasted me just as long, but who knows. Hoping my current rig lasts at last another 4 years..
Framework, weirdly, overcharges considerably for their SSDs. You can currently get a Samsung 990 Pro 2TB on Amazon for $390; Framework charges $625 for the Sandisk 850x 2TB, which has similar performance (and is being sold on Amazon for $530).
If you DIY your own SSD, you can spec a Framework Desktop for below $4k; but not much below. Roughly the same price.
The extremely charitable answer is that this is AMD's chance to control the built-in software, so they can ship a bunch of built-in AI stuff and a controlled environment similar to what NVIDIA does with their DGX OS, rather than e.g. Framework which is going to give you a stock Linux distro.
(I agree that you almost certainly don't need this)
It’s tragic that memory is so expensive, and yet, we don’t have chipsets to exploit the bandwidth possibilities of the memory being burnt on these devices.
Absolutely no reason that these need to be capped at 256 GB/s other than shortsighted design from three years ago.
I really want a 128gb+ machine but it's brutal to be at only 256 GB/s for $4k (especially with the drawbacks of both ARM and AMD).
I fear that by the time the RTX Spark comes out it'd have to be $6k, and by the time a 128gb or more machine with 700+
GB/s comes out it'd be at $10k, way out of most consumers' hands.
A Mac Studio is a much better buy in terms of memory bandwidth, but impossible to buy in a 128 GB configuration. Honestly there aren’t great options right now and it’s probably better to wait for the market to be less insane.
I looked for one and it's impossible to find, let alone at a reasonable price + it does suffer from being harder to train/use less common models and workflows (e.g. arbitrary comfyui ones). Spark at least doesnt have that drawback, while AMD has both drawbacks.
Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.
No, equities naturally trend up with economic growth. RAM is up because of a supply shock, as new capacity comes on prices will drop, it’s a commodity.
Yes, in the very long term, but in the medium term where the listed capacities even matter we are not close to that. Really, the cost per gb of vram on flagships hasnt went down since 1080ti and thats not accouting for the recent increases which will likely last for years.
Asus rog 13 128gb is still sub $3k last I checked and ticks the boxes. If you get tired of AI it's a kick ass tablet except the weight and can run AAA games for now decently enough.
RAM is more like agricultural products (with short shelf-life) than commodities like fossil fuels, mineral ores, etc. You can manage an inventory or speculate on production, but you cannot really hold a "portfolio" of it in any sensible way.
So, you should get into RAM futures if you believe this is more than a transient arbitrage sort of situation. All extant RAM will become obsolete as the demand shifts to newer, fancier versions.
Yes but on what timescale? Replacing the > 5 year old sticks in my laptop would currently cost well over half what the machine ran me when it was brand new.
The Mac Studio M3 Ultra with 96GB of RAM with 1Tb SSD costs $6799, but has triple the memory bandwidth [1]. It's almost twice the price of Strix Halo and when the M5 Ultras come out it will be interesting to see how much they go for. I expect a lot of demand for them!
There was a post recently, that showed the DGX spark is twice as fast as the M3 Ultra at prompt processing, but half as fast at output tokens [2]. They used gps-oss-120 for that test with a small context.
I’ve got a 128gb m5 max mbp and two sparks. For my real-world use cases, a single spark running DS4 Flash will have fully responded by the time my Mac has even started generating tokens. I figured I’d have more generation heavy work when I also bought the Mac, but it has done very little work running LLMs since I got the first Spark.
I mostly run them clustered for DS4 and am quite happy with the performance, and the cost isn’t that much more for two than the MBP while giving me double the unified memory.
I’ll probably pick up a third to run multiple smaller models. I don’t understand why people would buy a halo over a spark at comparable prices, particularly because if you want to cluster, the cx7 be beats the shit out of them when it comes to latency and throughput
Apple rumor mill is suggesting that we may see M5 Mac Studio announced in September at that event. Apple just increased their pricing across the board, so I'm not holding my breath that they will be reasonably priced. They also cancelled development of their M6 in favor of the future M7 which I suspect will be a major, AI-focused upgrade compared to the M4->M5 upgrade
Yeah, folks should be aware that if you're filling up the memory on a Strix Halo for an inference workload, you're going to be getting uncomfortably slow token rates. Like, DS4 (a 1-bit quantization of DeepSeek V4 Flash) runs at something like 9-13 tokens/second, with a loooong time to first token. It is not a realistic interactive coding model for agentic use.
I like my Strix Halo and keep it chewing on stuff, mostly non-interactive workloads (security audits of software mostly, training experiments, etc.), I get a lot of use out of it. If you want to experiment with AI, it is a good platform for that, though at $4k you can get an Nvidia-based Asus Ascend GX10, which is probably better. But, if you want a local model for interactive agentic use, you're going to be running either Qwen 3.6 or Gemma 4, which will fit comfortably on 2x64GB GPUs (even old GPUs will run them faster than the Strix Halo...I have dual Radeon Pro V620s which are faster, and they're six years old), or snugly on 32GB. A 48GB or 64GB Mac would run them well. Two Radeon AI Pro R9700 GPUs is probably the sweet spot, right now for GPUs. Not the cost of a good used car, like a 5090 or 4090, but plenty of memory and performance for local inference. Also, not finicky and weird and needing custom 3D printed fan shrouds like the old server GPUs on eBay.
At the moment, there just isn't a model that works better on a 128GB inference machine like this that don't also work fine on 64GB machines, which may be faster (very few 32GB GPUs will be slower, though I wouldn't recommend buying any GPU that isn't currently actively supported by the vendor drivers and CUDA or ROCm...so probably don't buy an MI50 or V100 or whatever).
I said, "Like, DS4 (a 1-bit quantization of DeepSeek V4 Flash) runs at something like 9-13 tokens/second, with a loooong time to first token."
Which almost exactly matches the benchmark you just linked. Looks like it's possible to goose it to 15 tokens per second with a tiny context, but why would I want a giant model with a 2k context? DeepSeek is too big to be fast enough on a Strix Halo.
To be clear, if you think that's comfortable for interactive use, more power to you. But, I'm not waiting for that. I'll pay DeepSeek to host it for me. Their token prices are quite cheap and their cached tokens are even cheaper...and they have the most effective caching in the business, as far as I can tell. Even naively using the API you get 80-90% cached token rate. If you use Reasonix, you get ~98% cached token rate. I just built a feature for an app I'm working on for $0.10 for 20 minutes of work. Not bad at all.
For anyone considering these devices, the only reason I would recommend against them is if you plan on getting multiple to link together - the DGX Spark has a much, much faster interconnect bandwidth ceiling than the AMD devices do.
Microcenter has them marked at $4500 right now (that's with the 4TB SSD) [1]. I suspect it comes down to what you're using it for; if you're looking for a general purpose computer that's also solid at AI, the AMD machine is better. But if you want the best possible AI machine at below $5k... actually you should probably just buy an RTX 3090 or 5090. But if the 128gb of memory is critical, then yeah DGX Spark is it.
The CPU of Ryzen is better than that of DGX Spark, especially for modern programs that have been updated to use AVX-512 (i.e. it has a significantly higher multithreaded performance).
Only for GPU applications the NVIDIA system is likely to be better.
For inference, the prefill on the DGX Spark is like 5x the speed. Decode about the same (both bandwidth limited to about the same memory speed, unfortunately).
Difference is on the Spark you can use and learn CUDA, vLLM, SGlang etc which is the industry standard.
I bought mine (ASUS version) back in December with the intent to learn that stack of stuff. I've been on and off with it but it seems to have paid off, looks like I might be getting work in the inference serving space.
Yep, the only reason I bought mine (in late 2025, before hardware prices went totally crazy) was because it was half the price of a Spark. I spent a while fiddling around with the right Linux kernel, kernel firmware, ROCm installs, etc.
I got a Beelink GTR 9 Pro for $1980. These Strix Halo systems were a good deal at $2k when the alternative was a DGX Spark (which is similarly memory-constrained, but has about twice the iGPU processing power of the Radeon 8060S, having as many CUDA cores as an RTX 5070) for $4k. The pitch was basically "half the GPU compute (negative), x86 instead of ARM (positive), but no CUDA (negative), for half the price (positive), but you also don't get the ConnectX-7 NIC (negative)". These more or less balanced out to being worth it if you wanted a single-node system that could also double as a generic x86 homelab server once it was obsolete for LLM workloads.
These days, you can get a DGX Spark for $4.7k, so yes, the price has risen, but Strix Halo (with a few exceptions like the Bosgame and Corsair systems) $4k (or more!) is simply not a very good deal. If I were buying new right now, I'd 100% go DGX Spark without even thinking about it.
Gorgon Halo (releasing this fall/winter) is allegedly coming with 3GB memory chips, enabling a 192GB maximum unified memory SKU, alongside minor (100MHz) clock bumps in the iGPU and CPU, plus memory bumping from 8000 MT/s to 8533 MT/s (matching the MBW of the DGX Spark), and is otherwise unchanged. I fear these will be $5000+. At $3000, these would be awesome. At $5000, not so much.
I have a Strix Halo device, and like it, but at this price, might as well buy the Nvidia-based ASUS GX 10, if you're buying it for AI. CUDA remains the stronger ecosystem. The AMD is a better desktop machine, as it has a better CPU, but the Nvidia will be a little faster and a little better supported for inference and training workloads. You can almost always do the same things with ROCm, but you're going to work a little more.
Though, I will say that Nvidia ships a dogshit custom Ubuntu on their hardware that's hard to deal with. Nvidia is not good at software. I keep thinking they'll get better at it, but I've been dealing with their Jetson line for a couple of years now, and it still sucks. Still a clumsy custom Ubuntu, and it's not as easy as simply installing a different Linux version as it's a complicated image-based thing and no UEFI. At least, I assume they ship Ubuntu on the big devices; I've only dealt with the little embedded Jetson machines. The AMD stuff, being a regular x86_64 PC, you can install pretty much any Linux. I immediately put Fedora on mine.
I ... don't find the Ubuntu on my Spark to be dogshit? It's ... fine? It's just Ubuntu. Hasn't given me any grief and it's so far the only vendor I've seen that actually ships a properly supported Linux on an ARM64 device for Linux, so there's that. I use my ASUS GX10 as my daily driver, my primary workstation. Only thing that doesn't work for me on it is Spotify (probably some DRM thing). Oh, and there's no Signal ARM64 client, it seems.
The big advantage of the DGX Spark over the Strix Halo is much faster prefill. Like 5x the speed. Also the networking hardware on it is insanely powerful, though I and 99% of other Spark users, are unlikely to use it to its full capacity.
The unusual bootloader and custom hardware makes modifying and upgrading the OS a challenge. I work on robots that need a custom OS image loaded on the machine. The Nvidia Ubuntu version makes that a huge pain in the ass. They've got binary-only drivers that have to be there, the install/upgrade process is finicky and prone to failure (always recoverable, so far, but all the techs in production I work with have a hard time with it and have to be walked through it, as it's just so alien to folks who work with regular PCs most of the time).
Let's just say if I had my druthers, I would not choose Ubuntu, and I really wouldn't choose the Nvidia/ARM spin of Ubuntu. The Strix Halo has the benefit of being entirely a normal x86_64 PC that happens to also have a big chunk of unified memory. You can put pretty much anything on it. Any Linux, regular Windows, probably even a BSD (though good luck getting AI stuff working there).
But, as I said, if you're spending $4000 exclusively for inference and AI workloads, you might as well get the Nvidia-based unit. It is better for that.
Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.
Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)
I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways?
I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and...
Yeah, the stuff they preinstall is crazy. It's a ~16GB OS image, which is a huge pain in the ass (in addition to all the other things that are a pain in the ass about Nvidia's OS situation); I can't ship their 16GB pile of crap plus our custom 3GB of stuff on my robots, there's no reasonable way to do OTA upgrades with a pile of shit like that. It's just an unpleasant experience all around. What I always want is a nice clean basic install, and then my custom packages of everything I need goes in. Especially for embedded cases, like the Jetson line is allegedly intended for, their whole approach is just stupid, and they don't provide any documentation for how to not use their stupid thing.
Still, I'd rather have the blazing prefill speed of the Nvidia over my Strix Halo, but I wasn't willing to spend nearly twice as much for it (when I bought, the Strix Halo was $2100 and the GX10 was going for $3700, I think). Now that the difference between the two is much smaller, there's no reason to get the AMD.
The $2100 vs $3700 price difference really came down to the ConnectX hardware in the thing, I guess. But the price gap narrowed over time, now they both suck.
I am not sure why NVIDIA couldn't have released a cheaper version of the thing with just standard Ethernet on it and leave it at that. Hardly anybody can afford two or more of them to cluster via ConnectX anyways.
The Spark does a system update on first boot, so they would have updated had they been installed
I'm running vllm + GoModel on k3s, works like a charm. Even wrote some CUE last night to generate the values file for my Helm chart. It calculates the GPU percentage for me from the memGB I assign to a model.
Wow the prices on these have really come up.. Got my Framework desktop mainboard (Just the motherboard + CPU + soldered 128gb RAM) in Dec 2025 for ~1900 EUR
Are you sure about that? High memory speed is great for dense models, or when serving at high concurrency.
However for local single-user setups, it's often better to have access to more capable/bigger MoE models at reasonable speeds and lower concurrences, which is enabled by these platforms.
If you're using a MoE model, then why do you care about the larger RAM offered by these devices? That's the main problem with low bandwidth devices: they limit the effective ram you can make use.
I do (and have historically done) quite a work with both local LLMs and local diffusion models. I have an M3 Max MBP at 400 GB/s and also a desktop with a RTX 4090 with 1,008 GB/s
While the M3 Max MBP can serve up MoE reasonably fast (~60 token/sec)the RTX 4090 is an entirely different experience (~170 token/sec). I also do a fair bit of experimentation and am currently running a custom decoder that requires expensive look-ahead, but I'm still able to get a usable 25 token/s on the RTX.
The raison d'etre for the DGX spark is not practical home inference, but rather offering the same fundamental architecture as data center cards for a affordable CUDA prototyping. If you want to build software to run on H100s, you probably can't justify buying (and running) a single card. The DGX spark solves this by having the same fundamental setup as what those cards have.
That makes these non-NVIDIA DGX-like devices confusing to me. The entire benefit of the DGX series is the NVIDIA architecture itself.
Anyone interested in home LLMs should decide whether a Mac or a dedicated GPU is the more sensible path based on their budget and other computer use. Each has their own benefits.
I run DSv4 Flash at home on 2 DGX Sparks and am pretty sure there is no more cost effective way for me to do so. I'm not interested in running smaller models.
Using regular DeepSeek-V4-Flash, I see 2000tok/s prompt processing and from 40 to 50 tok/s generation. Performance doesn't drop much at long contexts, DSv4 is really nice for that.
With DeepSeek-V4-Flash-DSpark (Deepseek's new speculative decoding scheme), which is still barely supported anywhere, we're seeing a more steady 45-55 tok/s with bursts into the 60s.
What quant size are you using? Just picked up my second spark and a cable and wanted to try this one out for fun. I generally run a bunch of models for different purposes (easy / bulk tasks) and use APIs for harder tasks.
Any performance gains caused by the internal bandwidth of the card will evaporate once you spill into system RAM, because now your bottleneck is probably a slow PCI lane.
And if your jobs do fit onto a 24GB card, then you are not the target user for the "AI mini PC" niche that these guys are trying to carve out
what matters is how much memory it has; with the new MTP models, Qwen3.6 with 35B MOE, it's pumping out tokens up to ~80k context with little slow down.
It's great to get lots of tokens, but being able to handle and extent context is why it'll continue to be a great machine compared to any of the small graphics cards.
i wish there was a system like strix halo, but with enough lanes for a dedicated PCIe 5.0 x16 slot so you can have the best of both worlds: large sparse models on CPU with unified memory, dense models on GPU with real tensors and higher bandwidth memory.
> A shame, really, as the Ryzen 7640U, 7840U, 7840HS, and 7940HS all support 256GB of RAM.
To be fair, those platforms support dual dimms per channel, which Strix Halo would not, at least not at it's high speeds.
But reciprocally Gorgon Halo 400 just launched and it supports... 192GB. And is the exact same APU.
Memory chips did finally have their first big doubling per chip semi recently (available last February), with 48 & 64GB dimms becoming available. There is some reasonable lag here, that Strix Halo & Gorgon Halonuse lpddr5x, which perhaps had some lag, that 32GB (x4) was the best available. But now with Gorgon Halo being 192GB capable but not 256GB, it sure feels looks & seems like this is just bad spirited fuckery from AMD.
https://forum.level1techs.com/t/where-are-the-ddr5-unbuffere...
I assume they validated certain DRAM chips when the 395 first came out and they're just not going to validate any more. So newer DRAM is validated for the 495. We can't compare DDR5 and LPDDR5 since they are completely different; if 256 GB DDR5 is possible that doesn't mean anything.
But 192GB lpddr5x on Goron Halo 495 does strongly strongly suggest 256GB would be too. It took some big work to go from 32GB to 64GB dimms, was a long long long time coming. 48GB came latter.
Its very unlikely that with 192GB Gorgon Halo that anything really blocks 256GB Gorgon Halo, or Strix Halo. Higher than 32GB chips are possible and exist.
All the gpu makers make all their profit selling datacenter products. They don’t want consumer/home lab stuff with lower margins to replace their data center products so they handicap the vram in those products to make them less enticing for datacenter use.
32 Gb DDR4 RAM module has a bandwidth of 25 Gb/s and costs $160. If you buy 8 of these, you get 256 Gb RAM with 200 Gb/s bandwidth at $1280. And if you buy 16 x 16 Gb modules (each at $60) then you can get 400 Gb/s of bandwidth for $960.
The only problem, you need 8 or 16 memory controllers. Memory controllers are not that expensive: Intel Core i3-14100F has 2 channel controller and costs $110, so we can estimate that 16-channel controller should cost not more than $880, and 8-channel controller should cost $440.
So isn't it better to make a cheap CPU with 16 DRAM controllers instead of this $4K gear having only 128 Gb? Or maybe 2 CPUs each having 8 RAM channels?
DDR5 costs 2 times more ($360 for 32 Gb) while not even having 2 times the bandwidth so it is not worth buying. It is more reasonable to make more RAM channels and stuff them with DDR4.
So what I am trying to say, industry took a wrong turn. Instead of moving to over-priced DDR5, they should just make even cheapest CPUs support 8/16 DDR4 channels. Because a 32Gb DDR5-4800 module costs $360, and two 32Gb DDR4-3200 modules cost $320, so you get twice more size, more bandwidth and it costs you less. DDR5 is just a rip off.
Each memory controller interface is a not-insignificant number of PCB traces. Increasing the number of memory controllers may dramatically increase the number of PCB layers (or may not, it really depends on the CPU pinout) but it definitely will increase the number of pins on the CPU socket.
This is one of the main reasons (the other is the number of PCIe lanes) why high end desktop and server CPUs have like double the number of pins and so much bigger sockets as compared to consumer desktop CPUs.
To get memory mapping across systems you'd want to look into infiniband (https://en.wikipedia.org/wiki/InfiniBand) but the adapters generally aren't cheap. Common to find in super computer clusters.
If you have a desktop CPU with 2 memory channels and 2 DIMMs per channel, then the 2 DIMMs on each single channel share all of the address and data lines, there's just a chip select difference to pick which DIMM you're addressing. There is some extra loading on the lines, since you have 2 DIMMs, hence the supported speeds are usually slightly lower, but you only add 1-2 more actual PCB traces to have 2 DIMMs per channel vs 1 DIMM per channel. In contrast, adding another memory controller would add upwards of almost 100 additional PCB traces from the CPU.
In the alternate reality where this happened, wouldn't the price of DDR4 still be sky high? We'll ignore any costs for CPU, chip set, and motherboard redesign. You're just pushing the demand somewhere else.
The problem is if DDR4 was viable here it would also be absurdly expensive. There’s no world in which we survive this demand without a corresponding increase in supply. It’s not like there are zettabytes of spare cheap DDR4 sitting around.
In addition to cost and possibly market segmentation, presumably the additional power consumption wouldn't be worth the tradeoff for laptop parts. If you want high channel count you've always been able to purchase power hungry datacenter gear. You can also pick up surplus 10 GbE or even 100 GbE fiber NICs to link up your beowulf cluster. You'll probably have to run a few new electrical circuits and a small bit of HVAC if you're installing this in a residential home but it's entirely doable to cram 10 kW in a closet if you feel like it.
Alternatively you could just rent cloud instances by the hour to avoid the hassle.
Well to be honest, there are a lot of NOOP pins on CPUs, but using them basically means fabbing a new die altogether, which is basically making a whole new CPU altogether.
This is an odd comment. Pins are seriously expensive. Companies don't just throw them in for fun. Usually they're necessary for signal integrity or packaging constraints.
I think this is about semiconductor design and/or die packaging related difficulties/costs. idk exactly why, but "pin count" in this exact expression is not rare at all on chip marketing materials.
Was “only” $2k in its previous form but even in this updated box the mem bandwidth is woefully inadequate.
There’s a few models with space for a dedicated GPU for hybrid inference but imo not worth it.
Save your money for a Xeon or EPYC build
this product really deserves the "halo" in the name. very difficult to gauge its fit in the current market.
if you want inference, go mac with much higher memory bandwidth. given the price premium here, or the little there is, you might as well.
if you want to finetune and experiment, cuda still has the moat and the kit is not much cheaper, if at all, than dgx spark.
from personal experience, i had access to amd developer cloud with a fair bit of credits. however, even doing inference outside of their supported use cases (which are often dated btw) using vllm was a pain. in the end, despite great computing potential on paper, i decided to not spend more time than its worth on it. if their enterprise cloud continue to have these grievances, i am not optimistic about this kit.
it might be down to skill issue on my end. perhaps if these sell and it gives amd enough motivation to add more software staff in-house, more power to them.
otherwise, good article from labs as usual. nice to know that other kits based on this soc are more or less the same (unsurprisingly).
This is just a little under the price of NVidia's DGX Spark with CUDA or a Mac with 128GB and twice the memory bandwidth. The point of Strix Halo used to be that it was half the price of those way more capable machines. You'd be crazy to buy the AMD chip at this price. But the hardware market is generally crazy right now, so I'm sure this will sell as well, unfortunately.
Only the top variants of M5 Max have more than twice the memory bandwidth of AMD, but those are much more expensive, i.e. over $10,000.
M5 Pro has as slightly higher memory bandwidth, but currently it is available only with up to 64 GB of DRAM. Even with this small memory, it is more expensive than either AMD or NVIDIA, especially when you do not want a puny SSD, but a normal-sized SSD, i.e. big enough to put there an LLM if you want to compute a quantization yourself (i.e. more than $5600 with a 4 TB SSD & 64 GB DRAM).
If you want to do LLM inference, I do not see Apple as a competitive solution, as their price is much, much higher, while also having limited expansion for SSDs, where you need a lot of space if you want to store a few LLMs.
The only thing that is correct is that the AMD Strix Halo system used to be much cheaper than NVIDIA, but now it has the same price. The CPU of Strix Halo is better than that of NVIDIA, but the NVIDIA GPU is likely to be better than the AMD GPU and CUDA is guaranteed to work fine.
But ideally they would be competitive, right? If your goal is LLM or Diffusion inference or - god forbid - training, you're going to get way better performance on DGX Spark. The difference is more stark than 250 vs 273 GB/s bandwidth delta would suggest.
Now I think it's totally fine to have a less capable offering, and the Strix Halo is still a mighty capable machine for inference on mid-size MoEs. At 2k it was a tinkerer's dream. But the performance difference should be reflected in the price. This is roughly a doubling of the price compared to less than a year ago without adding any notable features, it's appalling.
The M4 Max with 128GB RAM has 546GB/s memory bandwidth [0], compared to Strix Halo's 250 (on the label, I've yet to see a benchmark that tops 220). It's not available at 128GB RAM anymore, at least in my shop, but when it was not so long ago it was about 4,7k, or a little over twice the price of a cheaper Strix Halo PC (around 2,2k a few months ago).
You're the one making things up. An M3 Ultra with 128GB RAM doesn't exist, the M3 Max has 410GB/s bandwidth [0]. I was of course talking about the M4 Max with 546GB/s, which was closer to twice the price of a Strix Halo mini PC in a typical configuration when it was still available. And memory bandwidth isn't everything, NVidia's lead in software is substantial, look up any tests comparing them side-by-side.
I have another strix halo that I got for half the price (before this price increase world wide). AMD making lemonade is one of the best reasons to get a strix halo. Lemonade + qwen3.6 35B MTP @ Q8_0 + anythingLLM (in docker) replaced 90%+ of my AI usage. And it’s fully local! Setting everything up took less than 3 hours total, including installing the OS
15 square cm box? Wow. Are there similar size, but less powered (and cheaper) workstations? I need a box that can build chromium reasonably fast and I would rather have something portable like this than a PC tower, but this is an overkill at $4k.
It's 195 x 190 x 60mm and takes a standard ITX board. You'll need to relocate the hole for the fan depending on your motherboard, but CAD files are available and you only need to change 2 parameters (X,Y of the hole center).
BTW mine was upgraded to 64GB RAM and a 5700G (zen 3 APU) but it died and I'm still trying to bring up a newer board - still socket AM4.
BTW to get the center coordinates of the hole, measure from the edges of the board to the top and bottom of the metal plate under the CPU socket and take their average distance. That plate is symmetric and centered under the hole.
How much are we going to pay for "AI kits" once the DRAM shortage is over? Will we be able to run a local model equivalent to the current AI frontier in sub $1000 hardware, even if dedicated, in 5 years?
Yeeeeep. There is no moat at the moment. AI companies are trying to dig one as fast as they possibly can. Either through passing laws to prevent local inference ("It's too dangerous! We need to control it") or by creating/limiting possible integrations (locking down OS/hardware, APIs/MCPs that only work with Claude/ChatGPT, etc).
Good luck trying to enforce those laws outside of the USA. And in the future China will be happy to sell local inference hardware at competitive prices.
No it won't because Chinese DRAM manufacturers have relatively low capacity and it's already being used. And in an auction, prices from different suppliers converge.
When this hardware was announced, it was expected to be in the $1200-1400 range new... so, maybe. The real question is will the powers that be let this bubble burst, and how painful will the fallout be... I have a feeling it will be worse than 2001-2002.
I recently bought a few sparks from Micro Center for the exact same price and it comes with ConnectX-7 200Gbps inter-connectivity. Not sure how AMD feels it can charge exactly the same for less.
The differences are basically, sparks require ARM and sparks allow interconnects; so if you do have dreams of electric sheep to chain them together, you're not gonna get the AMD halo units.
But if you just want to putz around with a dev machine and do other things, not sure you'd want a spark.
They had it on sale last week for $3999, it will likely happen again. Also if you are willing to buy ASUS/Acer/MSI you can get them cheaper, in the same range as well. Those units are identical (mainboard/ram/chipset/connectivity), they only tend to differ in SSD being offered.
Only Asus from what I have seen. Acer/Dell units seem to run cooler than the Nvidia unit, by a noticeable margin. That said I bought the OEM units for the Gen 5 4TB SSD and the fact that it was on sale.
Same here, no issue at all with my ASUS Spark-thing. And back when I was selecting this, I watched a whole video that broke them all down by thermal mgmt and there wasn't anything wrong with the ASUS.
I do wish didn't buy the one that only had the 1TB nVME in it though. It's one of those expensive half-sized dealies, which makes upgrading expensive. Esp now.
I bought the 1T and then did it again. I don't see the need for more sad on a vllm server, I have dozens of models and quant in that space. If I need more for data, I intend to plug in an old 4T HDD
The repeated claim that all these different forms are not directly comparable is a very strange aspect.
Only thing that separates them is the build quality and the extra 20W of boost the framework desktop and this variant support.
They have a note on the thermals but no measurement of noise. Doesn't matter if it's stricly a whoosh or a whine, only if they bother people in the same room. And the small ones like Bosgame get a consistent complaint about the noise in in-depth youtube videos.
How come the same hardware that was $2000 is now packed under a different name and costs double - $4000?! It's literally the same PC that was twice as cheaper 6 months ago.
Seems like having a big and clunky external power supply enables a smaller profile for the rest of the unit while making installation a bit more complex. How exactly is this thing going to be installed for use? Wouldn't it be easier to just have a bigger box with more shielding and heat dissipation?
External power supplies make UL certification cheaper; that's the reason. I hate power bricks but I have so many on my desk that one more makes no difference. The Beelink version has an internal PSU BTW.
Seems not really worth it? About the same cost as DGX, same amount of memory and yet the bandwidth is actually slightly lower. And also the DGX is CUDA being an Nvidia device which is a big compatibility advantage
For this to be compelling it would need to be eg 256GB minimum or something
I imagine there may be users who can’t use macOS, or maybe they want the ability to upgrade storage.
The framework desktop even has a usable PCIe 4x slot available if you put the board in a different case. They sell the 128GB board on its own for $3150.
People complain about the cost of RAM for data centers, yet happily buy devices with 128 GB of RAM that will see an order of magnitude less use than servers in those same data centers.
So... I dont want to ruin gaming more but why not get a gaming PC? Figured this out 15 years ago if its good for gaming, put some more RAM in and boom you have a workstation...
With a desktop your system memory is slow and your fast graphics memory is limited in size.
To me it seems like the best bang for your buck in the BYO desktop PC space is to get a board with dual PCIe slots then find some old generation 24GB GPUs like RTX 3090.
But you’re not getting access to more than 48GB of fast memory without something similar to this or a Mac Studio.
The biggest problem is that if you want to run large (continuous memory) models, gaming graphics cards aren't sufficient, and if you manage to get graphics cards that you can chain it becomes a lot more expensive (and better performance) than these machines and GB10 machines.
It would be really nice if they included clustering support like a blueprint on how to buy several of these and cluster them to run the really large models in the best way possible.
i always thought Ryzen AI Halo, together with DGS Spark, has mismatched compute capacity with memory size. Given 128GB VRAM, people would want to run large models, but the GPU compute is constraint in these types of use case. If the box runs models that don't need high compute, then there is no need of 128GB VRAM.
On the high end side, it is too slow. On the low end size, it waste money on VRAM.
I bought one of this system back when you could get one for $1800 (the GMKTek Ryzen AI Halo 128gb machine). It's a very good dev machine, the 16 core CPU is quite good for development work. I find this is a useful configuration for local LLMs with plenty of RAM left over for doing actual work on the system (split something like 64 system, 64 dedicated to LLMs).
I don't think I'd pay $4k for it today though, 2 years ago and less than half the price feels like a good machine. I'd be very disappointed in it today for $4k.
When this was half the price of the DGX Spark, it made sense. But same price is a ridiculous premium for inferior performance but the ability to run Windows.
I was in Gray Scott School for HPC last week, and even in scientific usage, CPU-only cases, AMD is still a pain point. Many tools and libraries don't have first class AMD support or any support at all.
It loses to Intel in CPU, and NVIDIA in GPU, in case of scientific libraries and HPC-worthy libs, tools.
I think people who want an "AI Dev Kit" will lean towards Intel + NVIDIA setup.
I am not a fan of Intel, but their MKL, MPI, etc. are not paralleled. Same goes for CUDA with NVIDIA.
I had hoped this was about Medusa Halo, but unfortunately, it's about 2025 technology. It's the same as Framework Desktop was at the end of last summer, which would have been a slightly silly but fun buy at $2k... I'd hope Mark Cerny / Sony launch PS6 sooner rather than later, as together with the upcoming LPDDR6 standard, it should trickle down to us in the local LLM mud eventually?
True, this is the new reality though. My main gripe with Strix Halo is memory bandwidth and compute performance. Gaming performance sits squarely in base PS5 territory just as is the case with Steam Machine AFAIR; yet due to economies of scale "cheap" 2020 era PS5 still has higher memory bandwidth by quite a bit last time I checked.
Perhaps if less spending went towards their private aviation interests LTT labs could review a piece of hardware that was released _this_ year, or maybe extend their narrow testing process to cover real-world use metrics like TTFT. Not to mention the lack of real value-perf comparison to CUDA
$4k is pretty darn spendy. I recently purchased a refurbished Corsair AI Workstation with almost the same hardware (same chip, same 128GB RAM, but only 1TB storage) for $2160. Pretty good deal! Codex and I wrote a Linux driver to report the power mode of the device:
>Depends how long this market can remain utterly loony though.
Probably way, way longer than we'd ideally like. Every now and then, you hear that this is the new normal, that it'll last until 2032 or something, and I can practically smell the paid advertising or cult-like messaging behind attempts to destroy any speck of hope consumers have so they'll just pay the toll. But man, it really could be exactly that. There's just too much money to be made for all the involved parties.
> There's just too much money to be made for all the involved parties.
Even setting that aside, depending on the pace of datacenter buildout even a round of new fab capacity coming online might not be enough to bring prices down. I think there's a decent chance we have to wait for investors to get tired of building new datacenters which could easily be 5+ years.
The Mac beats it in all benchmarks, is probably more energy efficient, can add more ram, and is more cost efficient (?)… plus you get a Mac. This doesn’t even give you cuda. I’m not sure who this is for.
The one thing that's new/worth pointing out are the https://developer.amd.com/playbooks/ (https://github.com/amd/playbooks) - this is AMD's answer to Nvidia's playbooks (https://build.nvidia.com/spark / https://github.com/NVIDIA/dgx-spark-playbooks ) - I think it's great that they're actually taking this more seriously.
Hardware is the exact same as what used to be available for $2K last year (and is still $1K cheaper from Chinese OEMs).
LTT Lab's LLM testing is getting more sophisticated, which is great - I think it's worth noting that ROCm/Vulkan versions and llama.cpp build versions are going to have some big differences for numbers.
For those wanting to get the most out of their Strix Halos, there's both kernel tweaks and utilities like ryzenadj that can help you get the most out of it. ( http://strixhalo.wiki/ has most of that documented). Also, if you're running for coding or agentic work, if you model supports MTP, that's mature and should give you a decent (30%?) decode boost.
it's worth noting that AMD's software is universally weak and not worth any degree of reliance. and it's not just ROCm, every few months they merge a serious regression into amdgpu and sometimes even backport it into stable. they are amateurs.
just a few weeks ago they backported a kernel oops amdgpu null dereference into stable, it's still not fixed.
The "ROCm" situation with Strix Halo was pretty bad for a while. I think it finally stabilized late last year. You needed the right combo of ROCm, Linux kernel, and kernel firmware for it to work reliably.
Whenever I rebuild llama.cpp, I wind up using the Vulkan build anyway.
And thats a pretty big annoyance. You need to perfectly line up all the holes in the swiss cheese for the AMD stack to work, then their dev team kicks you in the nuts anyway.
They have made a few attempts at investing in the hardware, but the software side is letting them down hard, and that part is almost entirely their own fault. They have underinvestment in their own stack, and into popular standard and Community libraries that would make it easy to use their gear.
Yep, it definitely wasn't an "out-of-box" experience. This was on Ubuntu 25.10. Maybe other distros are better?
Also, when you do Python work, you have to remember to install dependencies like pytorch from an alternate repo, otherwise you wind up with the CUDA versions that only use the CPU and not the GPU through ROCm.
I started a project w/ the assumption that fully tuned for HIP kernels could get closer to roofline and beat llama.cpp's performance (and vs their ROCm engine it does!) but after thousands of iterations/experiments my suspicion is that RADV/ACO is actually just much better than LLVM/HIP when it comes to a lower-level scheduling etc for RDNA3.
I haven't finished comprehensive tests but I found:
- Vulkan is up to 2.25X faster for most coalesced, strided and interleave variants for memory-side scheduling/access shapes
- 3.3X faster on specific dot-path sweeps, including for scalar-dequant
- For matched LDS, Vulkan can be 8-14X+ faster (!!!) than matched HIP LDS
HIP doesn't always win against RADV/ACO, but on dispatch/runtime, it does appear to be quite a bit faster than HIP/LLVM on gfx1151 (Strix Halo). I'll be publishing sharing full data once I also run vs gfx1100...
biggest mistake was buying an amd laptop. this thing had gpu crashes from day one, and it got worse as time went on. now just playing a video and closing the lid will crash the kernel 100% of the time. amd - never again
Without further info that’s a lot to pin on the bottom layer of the stack. I have an Asus ProArt Ryzen AI HX 390 with 64GB RAM. Running CachyOS and Pop! the thing kept falling over at random times. Running Fedora workstation, it’s rock solid. YMMV.
Same, I have an all AMD framework laptop 16 with the gaming GPU, and running fedora, and it has been rock solid. I really couldn't ask for more
Just to add another data point, my AMD laptop was as the poster above described, unreliable drivers, glitches and crashes.
Personally I’ll never buy AMD again.
In case it saves anyone some time (from the article): "The AMD Ryzen AI Max+ 395(Strix Halo) processor has been available since Spring 2025 and the Halo doesn’t offer anything new on that front."
It has the same 256 GB/s memory bandwidth limit as every board previously, not sure why this is even being released right now as if it's some new fangled thing - you can go get a Framework Desktop for roughly the same price or a GMKtec EVO-X2 for a bit cheaper.
It's being released right now because it's massively profitable and in high demand and has actually gone up in price over the past year so obviously AMD wants to cash in on that instead of selling these units to PC manufacturers at a lower price.
I also think this is forward looking and helps reanchor expectations for RAM and memory bandwidth, which historically have been areas where companies do value engineering.
The memory shortages won't last forever; when companies start adding capacity I wouldn't be surprised to see massive sticks of RAM being sold for consumers.
Before AI I kinda thought more ram and faster ram was kind of useless because I had written off increased speeds as sort of a luxury whose only application is bloated unoptimized software. Boy was I wrong about that. But then why stop there?
I mean, there's a difference between "my IDE opens 0.1s faster" and "my desktop can now write my code for me" and only one of those justifies increasing RAM production.
This. I bought Framework Desktop in November 2025 with almost these exact specs for ~$2.5k
I got the EVO-X2 for $1,599! In 44+ years of buying computers, I've seen some appreciation, but nothing like this. Going from $1,599 to ~$3,500 in a year is just insane.
And GMKtec just released an EVO-X3 [0] based on the same board but with Oculink. When AMD launched this I thought it was the new chipset that was going to have 192GB of unified memory, but they look to be capitalizing on the inflated premium these "dev" focused desktop solutions.
Regretting not buying the Framework back when it was around $2k.
[0] https://www.gmktec.com/products/gmktec-evo-x3-ai-mini-pc-amd...
I've never seen appreciation in buying PCs. They were always like cars, once you drive them off the lot...
Well, GPUs sorta went up in the past 5 years.. but nothing like now.
Right. But they shouldn't have :) These are wild times. In the past memory and GPU prices always went down heavily.
[dead]
128GB of the most in demand chip on the planet will do that to you.
128GB of the most in demand consumer chip when all the manufacturers are switching to HBM manufacturing because the margins are far better there will do that to you. It sucks.
I was looking at buying hardware for me and as testing ground to learn. With those prices I just nipped out and will just rent in demand at runpod or so.
Absolute insanity
I had a pre-order in for the 128GB DIY motherboard only model last Oct 2025, but cancelled it and just upgraded my current 5900x rig to 64GB instead and replaced the x370 board with a used x470. Really with I had pulled the trigger, but I honestly am just fine with the performance as is on my current system for now. The Framework should have lasted me just as long, but who knows. Hoping my current rig lasts at last another 4 years..
Same spec Framework costs $4k+ today, this is actually cheaper (albeit not as upgradable as framework).
Framework, weirdly, overcharges considerably for their SSDs. You can currently get a Samsung 990 Pro 2TB on Amazon for $390; Framework charges $625 for the Sandisk 850x 2TB, which has similar performance (and is being sold on Amazon for $530).
If you DIY your own SSD, you can spec a Framework Desktop for below $4k; but not much below. Roughly the same price.
If you can supply your own SSD, case, and 120mm fan the mainboard is "only" $3,149
And it was "only" $1700 last year!
A year ago, you could have bought this computer for $2000. Basically double the price now.
I bought a bosgame version for $2199 back in February.
The extremely charitable answer is that this is AMD's chance to control the built-in software, so they can ship a bunch of built-in AI stuff and a controlled environment similar to what NVIDIA does with their DGX OS, rather than e.g. Framework which is going to give you a stock Linux distro.
(I agree that you almost certainly don't need this)
It’s tragic that memory is so expensive, and yet, we don’t have chipsets to exploit the bandwidth possibilities of the memory being burnt on these devices.
Absolutely no reason that these need to be capped at 256 GB/s other than shortsighted design from three years ago.
If you're not space-constrained, why go with a Framework Desktop? Just get a normal tower or a server rack computer.
One of the big benefits I’ve heard about for the framework, especially with the Noctua cooler, is that it’s virtually silent.
You can put Noctua fans in server rack computers. I even modified my NAS to use them: https://pics.ealex.net/s/noctua-fans-ftw
https://community.frame.work/t/noisy-psu-fan/74751
Well, almost. Maybe. Not all units. But probably.
Yeah the 256GB/s bandwidth is really very limiting
I really want a 128gb+ machine but it's brutal to be at only 256 GB/s for $4k (especially with the drawbacks of both ARM and AMD).
I fear that by the time the RTX Spark comes out it'd have to be $6k, and by the time a 128gb or more machine with 700+ GB/s comes out it'd be at $10k, way out of most consumers' hands.
Edit: capitalized gb/s to GB/s.
A Mac Studio is a much better buy in terms of memory bandwidth, but impossible to buy in a 128 GB configuration. Honestly there aren’t great options right now and it’s probably better to wait for the market to be less insane.
I looked for one and it's impossible to find, let alone at a reasonable price + it does suffer from being harder to train/use less common models and workflows (e.g. arbitrary comfyui ones). Spark at least doesnt have that drawback, while AMD has both drawbacks.
Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.
No, equities naturally trend up with economic growth. RAM is up because of a supply shock, as new capacity comes on prices will drop, it’s a commodity.
Yes, in the very long term, but in the medium term where the listed capacities even matter we are not close to that. Really, the cost per gb of vram on flagships hasnt went down since 1080ti and thats not accouting for the recent increases which will likely last for years.
There are some minor exceptions but agreed otherwise. Manufacturers have been excessively using VRAM limitations as a price point distinguisher.
But it could get worse for two years yet.
If it flys, floats or FLOPs it is better to rent.
> Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.
I said less insane, not sane. If prices go down 20% but are still up 150% over a few years ago, that would still be an improvement from now.
Or it could work the other way: if new hardware comes out in 2027 such that the tokens/$ ratio works out better, that would also be less insane.
Asus rog 13 128gb is still sub $3k last I checked and ticks the boxes. If you get tired of AI it's a kick ass tablet except the weight and can run AAA games for now decently enough.
Is that a notebook? As the thermals don't really compare for a notebook and these high performance minipcs.
> Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.
lol this is so wrong it's funny - equities go up in price, commodity goods go down in price. the two markets are literally diametrically opposed.
I'd have a better portfolio right now if I invested in RAM instead of equities.
RAM is more like agricultural products (with short shelf-life) than commodities like fossil fuels, mineral ores, etc. You can manage an inventory or speculate on production, but you cannot really hold a "portfolio" of it in any sensible way.
So, you should get into RAM futures if you believe this is more than a transient arbitrage sort of situation. All extant RAM will become obsolete as the demand shifts to newer, fancier versions.
>but you cannot really hold a "portfolio" of it in any sensible way.
xAI effectively did and lucked out to cover their losses and more with it.
> All extant RAM will become obsolete
Yes but on what timescale? Replacing the > 5 year old sticks in my laptop would currently cost well over half what the machine ran me when it was brand new.
The Mac Studio M3 Ultra with 96GB of RAM with 1Tb SSD costs $6799, but has triple the memory bandwidth [1]. It's almost twice the price of Strix Halo and when the M5 Ultras come out it will be interesting to see how much they go for. I expect a lot of demand for them!
There was a post recently, that showed the DGX spark is twice as fast as the M3 Ultra at prompt processing, but half as fast at output tokens [2]. They used gps-oss-120 for that test with a small context.
[1] https://gpuquicklist.com/apus?models=GB10%20Grace%20Blackwel...
[2] https://aimultiple.com/dgx-spark-alternatives , https://news.ycombinator.com/item?id=48732679
I’ve got a 128gb m5 max mbp and two sparks. For my real-world use cases, a single spark running DS4 Flash will have fully responded by the time my Mac has even started generating tokens. I figured I’d have more generation heavy work when I also bought the Mac, but it has done very little work running LLMs since I got the first Spark.
I mostly run them clustered for DS4 and am quite happy with the performance, and the cost isn’t that much more for two than the MBP while giving me double the unified memory.
I’ll probably pick up a third to run multiple smaller models. I don’t understand why people would buy a halo over a spark at comparable prices, particularly because if you want to cluster, the cx7 be beats the shit out of them when it comes to latency and throughput
Apple already announced that will increase prices.. doubt that Mac studio will stay cheap
MacBook Pro can come configured with 128GB M5 Max.
Apple rumor mill is suggesting that we may see M5 Mac Studio announced in September at that event. Apple just increased their pricing across the board, so I'm not holding my breath that they will be reasonably priced. They also cancelled development of their M6 in favor of the future M7 which I suspect will be a major, AI-focused upgrade compared to the M4->M5 upgrade
They cancelled the higher-end M6 models (including the Pro, this time).
There will be a basic M6.
Yeah, folks should be aware that if you're filling up the memory on a Strix Halo for an inference workload, you're going to be getting uncomfortably slow token rates. Like, DS4 (a 1-bit quantization of DeepSeek V4 Flash) runs at something like 9-13 tokens/second, with a loooong time to first token. It is not a realistic interactive coding model for agentic use.
I like my Strix Halo and keep it chewing on stuff, mostly non-interactive workloads (security audits of software mostly, training experiments, etc.), I get a lot of use out of it. If you want to experiment with AI, it is a good platform for that, though at $4k you can get an Nvidia-based Asus Ascend GX10, which is probably better. But, if you want a local model for interactive agentic use, you're going to be running either Qwen 3.6 or Gemma 4, which will fit comfortably on 2x64GB GPUs (even old GPUs will run them faster than the Strix Halo...I have dual Radeon Pro V620s which are faster, and they're six years old), or snugly on 32GB. A 48GB or 64GB Mac would run them well. Two Radeon AI Pro R9700 GPUs is probably the sweet spot, right now for GPUs. Not the cost of a good used car, like a 5090 or 4090, but plenty of memory and performance for local inference. Also, not finicky and weird and needing custom 3D printed fan shrouds like the old server GPUs on eBay.
At the moment, there just isn't a model that works better on a 128GB inference machine like this that don't also work fine on 64GB machines, which may be faster (very few 32GB GPUs will be slower, though I wouldn't recommend buying any GPU that isn't currently actively supported by the vendor drivers and CUDA or ROCm...so probably don't buy an MI50 or V100 or whatever).
this is not true. q2 Deepseek flash works on AMD Strix halo with pretty good results. Benchmark except:
ctx_tokens,prefill_tokens,prefill_tps,gen_tokens,gen_tps,kvcache_bytes 2048,2048,202.02,128,15.31,52184460 4096,2048,211.03,128,14.64,80373132 6144,2048,208.04,128,14.59,108561804 8192,2048,200.78,128,14.43,136750476 10240,2048,203.04,128,14.37,164939148 12288,2048,200.82,128,14.27,193127820 14336,2048,198.62,128,14.22,221316492 16384,2048,196.14,128,14.20,249505164 18432,2048,189.48,128,14.13,277693836 20480,2048,186.59,128,14.06,305882508 22528,2048,183.88,128,13.99,334071180 24576,2048,183.38,128,13.92,362259852 26624,2048,181.57,128,13.87,390448524 28672,2048,183.46,128,13.80,418637196 30720,2048,181.80,128,13.73,446825868 32768,2048,175.93,128,13.55,475014540 34816,2048,175.42,128,13.46,503203212
https://kyuz0.github.io/strix-halo-ds4-toolbox/
I said, "Like, DS4 (a 1-bit quantization of DeepSeek V4 Flash) runs at something like 9-13 tokens/second, with a loooong time to first token."
Which almost exactly matches the benchmark you just linked. Looks like it's possible to goose it to 15 tokens per second with a tiny context, but why would I want a giant model with a 2k context? DeepSeek is too big to be fast enough on a Strix Halo.
To be clear, if you think that's comfortable for interactive use, more power to you. But, I'm not waiting for that. I'll pay DeepSeek to host it for me. Their token prices are quite cheap and their cached tokens are even cheaper...and they have the most effective caching in the business, as far as I can tell. Even naively using the API you get 80-90% cached token rate. If you use Reasonix, you get ~98% cached token rate. I just built a feature for an app I'm working on for $0.10 for 20 minutes of work. Not bad at all.
At the moment, for around $4000-5000 you can either have speed (a GPU + 32GB VRAM), or you can have capacity - a DGX Spark/Halo, but not both.
I think once someone comes up with a machine which has both it will easily sell for $10000 and people will be queueing to buy it.
It's about $75k https://tinycorp.myshopify.com/products/tinybox-green-v2-wit...
That’s about the price of a workstation-class Nvidia GPU nowadays and if you splurge double, you can get a PCIe version of a data center card.
Sure but where do you stop spending? :)
https://github.com/jamesob/local-llm
Until your agents have their own PCs and a little army of agents. :)
To be clear though that's GB/s. Which is 2 terabits/sec
And 4000 USD is over 1.2 million Hungarian forints.
yea but 4000 USD dont get confused for Hungarian forints just because one says 4000 usd instead.
[dead]
These devices were great when they were cheaper than the DGX Spark.
But when they cost the same price (unless the Spark has shot up too), there's no reason to buy this over a Spark.
The Spark is literally a faster version of this, with better software support.
Edit: And I say that as an owner of a Ryzen AI Max 395 device.
Ability to run any OS is a pretty nice benefit versus the spark.
As far as I know, you can use other OSes once the Spark's firmware is updated with LVFS.
You'll need a custom-built distro image, but that goes for like 90% of ARM hardware on Linux.
> that goes for like 90% of ARM hardware on Linux.
And that's where AMD CPU shines vs ARM.
Nice benefit is NVPF4 from the Spark (GB10)
I dropped a Framework mainboard in rack mount case and use it as a speedy low power x86 homelab as well as an inference server.
Cheapest I've been able to get a DGX Spark FE is now around $4700 just FYI. This is from multiple vendors in higher-ed.
Ah ya then that's a bit of a gap.
For anyone considering these devices, the only reason I would recommend against them is if you plan on getting multiple to link together - the DGX Spark has a much, much faster interconnect bandwidth ceiling than the AMD devices do.
Otherwise, they're great!
Microcenter has them marked at $4500 right now (that's with the 4TB SSD) [1]. I suspect it comes down to what you're using it for; if you're looking for a general purpose computer that's also solid at AI, the AMD machine is better. But if you want the best possible AI machine at below $5k... actually you should probably just buy an RTX 3090 or 5090. But if the 128gb of memory is critical, then yeah DGX Spark is it.
[1] https://www.microcenter.com/product/699008/nvidia-dgx-spark
I'm looking at Amazon and I see $4k GB10 devices right now (not an affiliate link) https://www.amazon.com/ASUS-Supercomputer-Superchip-Supports...
Yea I mentioned the FE version which is 4TB NVME. The one you linked is 1TB NVME.
I snagged my two 4tb acer variants for under 4k each. CDW still has them listed for close to what I paid.
I can't wait for the memory price crisis to end so these things drop in price so I can get a second one.
Depends on what you use it for.
The CPU of Ryzen is better than that of DGX Spark, especially for modern programs that have been updated to use AVX-512 (i.e. it has a significantly higher multithreaded performance).
Only for GPU applications the NVIDIA system is likely to be better.
For inference, the prefill on the DGX Spark is like 5x the speed. Decode about the same (both bandwidth limited to about the same memory speed, unfortunately).
Difference is on the Spark you can use and learn CUDA, vLLM, SGlang etc which is the industry standard.
I bought mine (ASUS version) back in December with the intent to learn that stack of stuff. I've been on and off with it but it seems to have paid off, looks like I might be getting work in the inference serving space.
Yep, the only reason I bought mine (in late 2025, before hardware prices went totally crazy) was because it was half the price of a Spark. I spent a while fiddling around with the right Linux kernel, kernel firmware, ROCm installs, etc.
yeah, if only there wasn't some global hegemony that immediately drove up the price of all memory everywhere...
That's the rub... this platform was supposed to be around $1200-1400 maxed out, but now it's $4k, which is insane.
Spark has also gone up in price, my second cost me $500 more than my first a couple months earlier (both this spring)
I got a Beelink GTR 9 Pro for $1980. These Strix Halo systems were a good deal at $2k when the alternative was a DGX Spark (which is similarly memory-constrained, but has about twice the iGPU processing power of the Radeon 8060S, having as many CUDA cores as an RTX 5070) for $4k. The pitch was basically "half the GPU compute (negative), x86 instead of ARM (positive), but no CUDA (negative), for half the price (positive), but you also don't get the ConnectX-7 NIC (negative)". These more or less balanced out to being worth it if you wanted a single-node system that could also double as a generic x86 homelab server once it was obsolete for LLM workloads.
These days, you can get a DGX Spark for $4.7k, so yes, the price has risen, but Strix Halo (with a few exceptions like the Bosgame and Corsair systems) $4k (or more!) is simply not a very good deal. If I were buying new right now, I'd 100% go DGX Spark without even thinking about it.
Gorgon Halo (releasing this fall/winter) is allegedly coming with 3GB memory chips, enabling a 192GB maximum unified memory SKU, alongside minor (100MHz) clock bumps in the iGPU and CPU, plus memory bumping from 8000 MT/s to 8533 MT/s (matching the MBW of the DGX Spark), and is otherwise unchanged. I fear these will be $5000+. At $3000, these would be awesome. At $5000, not so much.
I have a Strix Halo device, and like it, but at this price, might as well buy the Nvidia-based ASUS GX 10, if you're buying it for AI. CUDA remains the stronger ecosystem. The AMD is a better desktop machine, as it has a better CPU, but the Nvidia will be a little faster and a little better supported for inference and training workloads. You can almost always do the same things with ROCm, but you're going to work a little more.
Though, I will say that Nvidia ships a dogshit custom Ubuntu on their hardware that's hard to deal with. Nvidia is not good at software. I keep thinking they'll get better at it, but I've been dealing with their Jetson line for a couple of years now, and it still sucks. Still a clumsy custom Ubuntu, and it's not as easy as simply installing a different Linux version as it's a complicated image-based thing and no UEFI. At least, I assume they ship Ubuntu on the big devices; I've only dealt with the little embedded Jetson machines. The AMD stuff, being a regular x86_64 PC, you can install pretty much any Linux. I immediately put Fedora on mine.
I ... don't find the Ubuntu on my Spark to be dogshit? It's ... fine? It's just Ubuntu. Hasn't given me any grief and it's so far the only vendor I've seen that actually ships a properly supported Linux on an ARM64 device for Linux, so there's that. I use my ASUS GX10 as my daily driver, my primary workstation. Only thing that doesn't work for me on it is Spotify (probably some DRM thing). Oh, and there's no Signal ARM64 client, it seems.
The big advantage of the DGX Spark over the Strix Halo is much faster prefill. Like 5x the speed. Also the networking hardware on it is insanely powerful, though I and 99% of other Spark users, are unlikely to use it to its full capacity.
The unusual bootloader and custom hardware makes modifying and upgrading the OS a challenge. I work on robots that need a custom OS image loaded on the machine. The Nvidia Ubuntu version makes that a huge pain in the ass. They've got binary-only drivers that have to be there, the install/upgrade process is finicky and prone to failure (always recoverable, so far, but all the techs in production I work with have a hard time with it and have to be walked through it, as it's just so alien to folks who work with regular PCs most of the time).
Let's just say if I had my druthers, I would not choose Ubuntu, and I really wouldn't choose the Nvidia/ARM spin of Ubuntu. The Strix Halo has the benefit of being entirely a normal x86_64 PC that happens to also have a big chunk of unified memory. You can put pretty much anything on it. Any Linux, regular Windows, probably even a BSD (though good luck getting AI stuff working there).
But, as I said, if you're spending $4000 exclusively for inference and AI workloads, you might as well get the Nvidia-based unit. It is better for that.
No the Spark uses UEFI + ACPI
Regular Ubuntu or Fedora ISOs do work out of the box
Just needs the Nvidia GPU driver install afterwards
(And the realtek 10gbe module oot, or blocklist if not used)
And for the Jetsons btw still uses device tree with custom kernels but at least it's UEFI from Orin onwards.
For Jetson Orin and later:
You can download an ISO from https://developer.nvidia.com/embedded/jetpack/downloads and that'll work. Other distributions are still a bit of a mess though but Yocto is supported now.
Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.
Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)
I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways?
I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and...
... that was more the disappointing part.
Yeah, the stuff they preinstall is crazy. It's a ~16GB OS image, which is a huge pain in the ass (in addition to all the other things that are a pain in the ass about Nvidia's OS situation); I can't ship their 16GB pile of crap plus our custom 3GB of stuff on my robots, there's no reasonable way to do OTA upgrades with a pile of shit like that. It's just an unpleasant experience all around. What I always want is a nice clean basic install, and then my custom packages of everything I need goes in. Especially for embedded cases, like the Jetson line is allegedly intended for, their whole approach is just stupid, and they don't provide any documentation for how to not use their stupid thing.
Still, I'd rather have the blazing prefill speed of the Nvidia over my Strix Halo, but I wasn't willing to spend nearly twice as much for it (when I bought, the Strix Halo was $2100 and the GX10 was going for $3700, I think). Now that the difference between the two is much smaller, there's no reason to get the AMD.
The $2100 vs $3700 price difference really came down to the ConnectX hardware in the thing, I guess. But the price gap narrowed over time, now they both suck.
I am not sure why NVIDIA couldn't have released a cheaper version of the thing with just standard Ethernet on it and leave it at that. Hardly anybody can afford two or more of them to cluster via ConnectX anyways.
The Spark does a system update on first boot, so they would have updated had they been installed
I'm running vllm + GoModel on k3s, works like a charm. Even wrote some CUE last night to generate the values file for my Helm chart. It calculates the GPU percentage for me from the memGB I assign to a model.
Wow the prices on these have really come up.. Got my Framework desktop mainboard (Just the motherboard + CPU + soldered 128gb RAM) in Dec 2025 for ~1900 EUR
Indeed December 2025 was the best time to buy.
256gbs memory bandwidth is about 1/4 that of a 3090. It would be a better buy with half the memory at 4x the speed.
Are you sure about that? High memory speed is great for dense models, or when serving at high concurrency.
However for local single-user setups, it's often better to have access to more capable/bigger MoE models at reasonable speeds and lower concurrences, which is enabled by these platforms.
If you're using a MoE model, then why do you care about the larger RAM offered by these devices? That's the main problem with low bandwidth devices: they limit the effective ram you can make use.
I do (and have historically done) quite a work with both local LLMs and local diffusion models. I have an M3 Max MBP at 400 GB/s and also a desktop with a RTX 4090 with 1,008 GB/s
While the M3 Max MBP can serve up MoE reasonably fast (~60 token/sec)the RTX 4090 is an entirely different experience (~170 token/sec). I also do a fair bit of experimentation and am currently running a custom decoder that requires expensive look-ahead, but I'm still able to get a usable 25 token/s on the RTX.
The raison d'etre for the DGX spark is not practical home inference, but rather offering the same fundamental architecture as data center cards for a affordable CUDA prototyping. If you want to build software to run on H100s, you probably can't justify buying (and running) a single card. The DGX spark solves this by having the same fundamental setup as what those cards have.
That makes these non-NVIDIA DGX-like devices confusing to me. The entire benefit of the DGX series is the NVIDIA architecture itself.
Anyone interested in home LLMs should decide whether a Mac or a dedicated GPU is the more sensible path based on their budget and other computer use. Each has their own benefits.
I run DSv4 Flash at home on 2 DGX Sparks and am pretty sure there is no more cost effective way for me to do so. I'm not interested in running smaller models.
What's the tokens/sec you're getting on that setup (genuinely curious because it's a setup I haven't actually run myself)?
Using regular DeepSeek-V4-Flash, I see 2000tok/s prompt processing and from 40 to 50 tok/s generation. Performance doesn't drop much at long contexts, DSv4 is really nice for that.
With DeepSeek-V4-Flash-DSpark (Deepseek's new speculative decoding scheme), which is still barely supported anywhere, we're seeing a more steady 45-55 tok/s with bursts into the 60s.
> from 40 to 50 tok/s generation
That's actually much better than I would have thought!
Thanks for the answer and it does make this approach make more sense as a budget solution to running larger models locally.
What quant size are you using? Just picked up my second spark and a cable and wanted to try this one out for fun. I generally run a bunch of models for different purposes (easy / bulk tasks) and use APIs for harder tasks.
It's the native DeepSeek V4 Flash quant which is released in MXFP4. You have enough RAM left over with 2 sparks for ~2M tokens of working KV cache.
Any performance gains caused by the internal bandwidth of the card will evaporate once you spill into system RAM, because now your bottleneck is probably a slow PCI lane.
And if your jobs do fit onto a 24GB card, then you are not the target user for the "AI mini PC" niche that these guys are trying to carve out
what matters is how much memory it has; with the new MTP models, Qwen3.6 with 35B MOE, it's pumping out tokens up to ~80k context with little slow down.
It's great to get lots of tokens, but being able to handle and extent context is why it'll continue to be a great machine compared to any of the small graphics cards.
it depends
it allows you to run smaller models much better
imo 3090s make the most sense if you can buy at least 2x ideally 4x but of course we're talking about a completely different budget at that point
i wish there was a system like strix halo, but with enough lanes for a dedicated PCIe 5.0 x16 slot so you can have the best of both worlds: large sparse models on CPU with unified memory, dense models on GPU with real tensors and higher bandwidth memory.
Maybe this is what Medusa Halo will turn out to be?
AMEN! 100% agree.
Plus a reasonably inexpensive super low-latency interconnect.
Why do all similar products have a hard limit on the 128 GB VRAM part? For that price, I hoped to get at least 224 GB VRAM
Because it’s a limit of the platform
https://community.frame.work/t/was-there-no-possible-way-to-...
From the replies,
> A shame, really, as the Ryzen 7640U, 7840U, 7840HS, and 7940HS all support 256GB of RAM.
To be fair, those platforms support dual dimms per channel, which Strix Halo would not, at least not at it's high speeds.
But reciprocally Gorgon Halo 400 just launched and it supports... 192GB. And is the exact same APU.
Memory chips did finally have their first big doubling per chip semi recently (available last February), with 48 & 64GB dimms becoming available. There is some reasonable lag here, that Strix Halo & Gorgon Halonuse lpddr5x, which perhaps had some lag, that 32GB (x4) was the best available. But now with Gorgon Halo being 192GB capable but not 256GB, it sure feels looks & seems like this is just bad spirited fuckery from AMD. https://forum.level1techs.com/t/where-are-the-ddr5-unbuffere...
I assume they validated certain DRAM chips when the 395 first came out and they're just not going to validate any more. So newer DRAM is validated for the 495. We can't compare DDR5 and LPDDR5 since they are completely different; if 256 GB DDR5 is possible that doesn't mean anything.
But 192GB lpddr5x on Goron Halo 495 does strongly strongly suggest 256GB would be too. It took some big work to go from 32GB to 64GB dimms, was a long long long time coming. 48GB came latter.
Its very unlikely that with 192GB Gorgon Halo that anything really blocks 256GB Gorgon Halo, or Strix Halo. Higher than 32GB chips are possible and exist.
The 495 is going to support 192 GB. It depends on the memory bus.
128 bit: 96 GB?
256 bit: 192 GB
512 bit: 384 GB?
1024 bit: 768 GB?
All the gpu makers make all their profit selling datacenter products. They don’t want consumer/home lab stuff with lower margins to replace their data center products so they handicap the vram in those products to make them less enticing for datacenter use.
32 Gb DDR4 RAM module has a bandwidth of 25 Gb/s and costs $160. If you buy 8 of these, you get 256 Gb RAM with 200 Gb/s bandwidth at $1280. And if you buy 16 x 16 Gb modules (each at $60) then you can get 400 Gb/s of bandwidth for $960.
The only problem, you need 8 or 16 memory controllers. Memory controllers are not that expensive: Intel Core i3-14100F has 2 channel controller and costs $110, so we can estimate that 16-channel controller should cost not more than $880, and 8-channel controller should cost $440.
So isn't it better to make a cheap CPU with 16 DRAM controllers instead of this $4K gear having only 128 Gb? Or maybe 2 CPUs each having 8 RAM channels?
DDR5 costs 2 times more ($360 for 32 Gb) while not even having 2 times the bandwidth so it is not worth buying. It is more reasonable to make more RAM channels and stuff them with DDR4.
If you want Epyc go for it. The motherboards can be quite expensive though.
So what I am trying to say, industry took a wrong turn. Instead of moving to over-priced DDR5, they should just make even cheapest CPUs support 8/16 DDR4 channels. Because a 32Gb DDR5-4800 module costs $360, and two 32Gb DDR4-3200 modules cost $320, so you get twice more size, more bandwidth and it costs you less. DDR5 is just a rip off.
Each memory controller interface is a not-insignificant number of PCB traces. Increasing the number of memory controllers may dramatically increase the number of PCB layers (or may not, it really depends on the CPU pinout) but it definitely will increase the number of pins on the CPU socket.
This is one of the main reasons (the other is the number of PCIe lanes) why high end desktop and server CPUs have like double the number of pins and so much bigger sockets as compared to consumer desktop CPUs.
Then what's about using 4-8 cheapest motherboards with 64Gb DDR4 and a cheap CPU, and connecting them via PCIE x16 sockets?
And as for DRAM channels, typical cheap motherboard has 2 channels and 4 slots, it should not be super difficult to add 2 more channels.
To get memory mapping across systems you'd want to look into infiniband (https://en.wikipedia.org/wiki/InfiniBand) but the adapters generally aren't cheap. Common to find in super computer clusters.
If you have a desktop CPU with 2 memory channels and 2 DIMMs per channel, then the 2 DIMMs on each single channel share all of the address and data lines, there's just a chip select difference to pick which DIMM you're addressing. There is some extra loading on the lines, since you have 2 DIMMs, hence the supported speeds are usually slightly lower, but you only add 1-2 more actual PCB traces to have 2 DIMMs per channel vs 1 DIMM per channel. In contrast, adding another memory controller would add upwards of almost 100 additional PCB traces from the CPU.
Don't you need a new CPU to support more channels? It doesn't sound like a cheap, trivial change to upgrade my Ryzen to a Threadripper.
In the alternate reality where this happened, wouldn't the price of DDR4 still be sky high? We'll ignore any costs for CPU, chip set, and motherboard redesign. You're just pushing the demand somewhere else.
> any costs for CPU, chip set, and motherboard redesign.
Yes they spent those costs to switch from DDR4 to over-priced DDR5 and I suggested the cost could be spent on adding more DDR4 channels instead.
The problem is if DDR4 was viable here it would also be absurdly expensive. There’s no world in which we survive this demand without a corresponding increase in supply. It’s not like there are zettabytes of spare cheap DDR4 sitting around.
> even cheapest CPUs support 8/16 DDR4 channels
In addition to cost and possibly market segmentation, presumably the additional power consumption wouldn't be worth the tradeoff for laptop parts. If you want high channel count you've always been able to purchase power hungry datacenter gear. You can also pick up surplus 10 GbE or even 100 GbE fiber NICs to link up your beowulf cluster. You'll probably have to run a few new electrical circuits and a small bit of HVAC if you're installing this in a residential home but it's entirely doable to cram 10 kW in a closet if you feel like it.
Alternatively you could just rent cloud instances by the hour to avoid the hassle.
> just make even cheapest CPUs support 8/16 DDR4 channels
Isn't adding pins kind of expensive?
Well to be honest, there are a lot of NOOP pins on CPUs, but using them basically means fabbing a new die altogether, which is basically making a whole new CPU altogether.
This is an odd comment. Pins are seriously expensive. Companies don't just throw them in for fun. Usually they're necessary for signal integrity or packaging constraints.
Yes but not $1000 level expensive, right? It is just a tiny piece of metal, and on laptops there are no pins at all.
I think this is about semiconductor design and/or die packaging related difficulties/costs. idk exactly why, but "pin count" in this exact expression is not rare at all on chip marketing materials.
Was “only” $2k in its previous form but even in this updated box the mem bandwidth is woefully inadequate. There’s a few models with space for a dedicated GPU for hybrid inference but imo not worth it. Save your money for a Xeon or EPYC build
this product really deserves the "halo" in the name. very difficult to gauge its fit in the current market.
if you want inference, go mac with much higher memory bandwidth. given the price premium here, or the little there is, you might as well.
if you want to finetune and experiment, cuda still has the moat and the kit is not much cheaper, if at all, than dgx spark.
from personal experience, i had access to amd developer cloud with a fair bit of credits. however, even doing inference outside of their supported use cases (which are often dated btw) using vllm was a pain. in the end, despite great computing potential on paper, i decided to not spend more time than its worth on it. if their enterprise cloud continue to have these grievances, i am not optimistic about this kit.
it might be down to skill issue on my end. perhaps if these sell and it gives amd enough motivation to add more software staff in-house, more power to them.
otherwise, good article from labs as usual. nice to know that other kits based on this soc are more or less the same (unsurprisingly).
Does this have the same memory bandwidth problems as the spark?
Yes. And the same not-enough-memory problems too
What's the Spark's memory bandwidth?
273 GB/s. Same ballpark as M4 Pro and Strix Halo.
Ah yeah, bummer. It's fine for building something that you know needs to run faster in the next generation.
This is just a little under the price of NVidia's DGX Spark with CUDA or a Mac with 128GB and twice the memory bandwidth. The point of Strix Halo used to be that it was half the price of those way more capable machines. You'd be crazy to buy the AMD chip at this price. But the hardware market is generally crazy right now, so I'm sure this will sell as well, unfortunately.
Only the top variants of M5 Max have more than twice the memory bandwidth of AMD, but those are much more expensive, i.e. over $10,000.
M5 Pro has as slightly higher memory bandwidth, but currently it is available only with up to 64 GB of DRAM. Even with this small memory, it is more expensive than either AMD or NVIDIA, especially when you do not want a puny SSD, but a normal-sized SSD, i.e. big enough to put there an LLM if you want to compute a quantization yourself (i.e. more than $5600 with a 4 TB SSD & 64 GB DRAM).
If you want to do LLM inference, I do not see Apple as a competitive solution, as their price is much, much higher, while also having limited expansion for SSDs, where you need a lot of space if you want to store a few LLMs.
The only thing that is correct is that the AMD Strix Halo system used to be much cheaper than NVIDIA, but now it has the same price. The CPU of Strix Halo is better than that of NVIDIA, but the NVIDIA GPU is likely to be better than the AMD GPU and CUDA is guaranteed to work fine.
Personally, I'm totally ok to have a competitor to Nvidia, regardless of whether they are under the price or not.
But ideally they would be competitive, right? If your goal is LLM or Diffusion inference or - god forbid - training, you're going to get way better performance on DGX Spark. The difference is more stark than 250 vs 273 GB/s bandwidth delta would suggest.
Now I think it's totally fine to have a less capable offering, and the Strix Halo is still a mighty capable machine for inference on mid-size MoEs. At 2k it was a tinkerer's dream. But the performance difference should be reflected in the price. This is roughly a doubling of the price compared to less than a year ago without adding any notable features, it's appalling.
The Nvidia DGX are sitting on the shelf unsold at $4.5k.
where do you see "twice the memory bandwidth"?
The M4 Max with 128GB RAM has 546GB/s memory bandwidth [0], compared to Strix Halo's 250 (on the label, I've yet to see a benchmark that tops 220). It's not available at 128GB RAM anymore, at least in my shop, but when it was not so long ago it was about 4,7k, or a little over twice the price of a cheaper Strix Halo PC (around 2,2k a few months ago).
[0] https://en.wikipedia.org/wiki/Apple_M4
He is making things up. It is the same bandwidth as DGX Spark (256 GB/s vs 273 GB/s) and far behind M3 Ultra (~819 GB/s)
You're the one making things up. An M3 Ultra with 128GB RAM doesn't exist, the M3 Max has 410GB/s bandwidth [0]. I was of course talking about the M4 Max with 546GB/s, which was closer to twice the price of a Strix Halo mini PC in a typical configuration when it was still available. And memory bandwidth isn't everything, NVidia's lead in software is substantial, look up any tests comparing them side-by-side.
[0] https://en.wikipedia.org/wiki/Apple_M3 [1] https://en.wikipedia.org/wiki/Apple_M4
Google is your friend: https://www.apple.com/mac-studio/specs/
Apple M3 Ultra chip 819GB/s memory bandwidth
They have 96, 128, 256, and 512GB variants my friend.
They stopped selling anything greater than the 96 gig variant a while back. If you want more RAM, be prepared to bid high on ebay.
Well, some exist, but I guess they don't make any new ones for now.
I have another strix halo that I got for half the price (before this price increase world wide). AMD making lemonade is one of the best reasons to get a strix halo. Lemonade + qwen3.6 35B MTP @ Q8_0 + anythingLLM (in docker) replaced 90%+ of my AI usage. And it’s fully local! Setting everything up took less than 3 hours total, including installing the OS
https://lemonade-server.ai/
15 square cm box? Wow. Are there similar size, but less powered (and cheaper) workstations? I need a box that can build chromium reasonably fast and I would rather have something portable like this than a PC tower, but this is an overkill at $4k.
>> Are there similar size, but less powered (and cheaper) workstations?
I designed this:
https://github.com/phkahler/mellori_ITX
It's 195 x 190 x 60mm and takes a standard ITX board. You'll need to relocate the hole for the fan depending on your motherboard, but CAD files are available and you only need to change 2 parameters (X,Y of the hole center).
BTW mine was upgraded to 64GB RAM and a 5700G (zen 3 APU) but it died and I'm still trying to bring up a newer board - still socket AM4.
BTW to get the center coordinates of the hole, measure from the edges of the board to the top and bottom of the metal plate under the CPU socket and take their average distance. That plate is symmetric and centered under the hole.
Look at Minisforum; they have a bunch of different models.
How much are we going to pay for "AI kits" once the DRAM shortage is over? Will we be able to run a local model equivalent to the current AI frontier in sub $1000 hardware, even if dedicated, in 5 years?
Yeeeeep. There is no moat at the moment. AI companies are trying to dig one as fast as they possibly can. Either through passing laws to prevent local inference ("It's too dangerous! We need to control it") or by creating/limiting possible integrations (locking down OS/hardware, APIs/MCPs that only work with Claude/ChatGPT, etc).
Good luck trying to enforce those laws outside of the USA. And in the future China will be happy to sell local inference hardware at competitive prices.
Open, cheap & good enough will win the race.
frontier to laptop runnable open weight so far seems to be ~2 years latency, so maybe there's some hope
For that matter, just getting Chinese DRAM into the market could cool pricing down a lot to oppose the cartel.
No it won't because Chinese DRAM manufacturers have relatively low capacity and it's already being used. And in an auction, prices from different suppliers converge.
Because China can't ramp up production at all... no means to do so... they aren't even working on it at all.
Not really. I imagine demand for AI just from China would eat up their entire production let alone demand from the west.
And after all, that demand from consumers.
When this hardware was announced, it was expected to be in the $1200-1400 range new... so, maybe. The real question is will the powers that be let this bubble burst, and how painful will the fallout be... I have a feeling it will be worse than 2001-2002.
I recently bought a few sparks from Micro Center for the exact same price and it comes with ConnectX-7 200Gbps inter-connectivity. Not sure how AMD feels it can charge exactly the same for less.
it's 2026.07 and 128 GB of VRAM costs a firstborn.
The spark also has 128GB VRAM (same type) and by recently I mean I bought them last week for $3999 each.
yeah, so $500 spread https://www.microcenter.com/product/699008/nvidia-dgx-spark is what the current price appears to be.
The differences are basically, sparks require ARM and sparks allow interconnects; so if you do have dreams of electric sheep to chain them together, you're not gonna get the AMD halo units.
But if you just want to putz around with a dev machine and do other things, not sure you'd want a spark.
They had it on sale last week for $3999, it will likely happen again. Also if you are willing to buy ASUS/Acer/MSI you can get them cheaper, in the same range as well. Those units are identical (mainboard/ram/chipset/connectivity), they only tend to differ in SSD being offered.
The more beneficial difference between DGX and OEM Spark is in cooling. DGX has had cooling issue based on user reports.
Only Asus from what I have seen. Acer/Dell units seem to run cooler than the Nvidia unit, by a noticeable margin. That said I bought the OEM units for the Gen 5 4TB SSD and the fact that it was on sale.
I have ASUS and no thermal issues, not sure if that is what you meant, ambiguity
Same here, no issue at all with my ASUS Spark-thing. And back when I was selecting this, I watched a whole video that broke them all down by thermal mgmt and there wasn't anything wrong with the ASUS.
I do wish didn't buy the one that only had the 1TB nVME in it though. It's one of those expensive half-sized dealies, which makes upgrading expensive. Esp now.
I bought the 1T and then did it again. I don't see the need for more sad on a vllm server, I have dozens of models and quant in that space. If I need more for data, I intend to plug in an old 4T HDD
Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.
> the only problem you need 8 DRAM controllers.
Which is a huge problem? Even using 2x memory controllers in typical consumer motherboard can make system very unstable.
Why is that so?
Running large MOE models won't saturate the EPYC memory.
You'll never see that 200 Gb/s running inference.
Because it'll bottleneck at the CPU? Or because 200 bB/s is overkill when running a model (that fits in memory)?
Bosgame is $2799 does the same thing if you plan to run only 1 of them
The repeated claim that all these different forms are not directly comparable is a very strange aspect.
Only thing that separates them is the build quality and the extra 20W of boost the framework desktop and this variant support.
They have a note on the thermals but no measurement of noise. Doesn't matter if it's stricly a whoosh or a whine, only if they bother people in the same room. And the small ones like Bosgame get a consistent complaint about the noise in in-depth youtube videos.
The framework has a noisy PSU , the bosgame is noisy However you can fix both - the bosgame leaves you a large margin to do so
Are you talking on particular about the M5? Is there room for a huge fan replacement?
How come the same hardware that was $2000 is now packed under a different name and costs double - $4000?! It's literally the same PC that was twice as cheaper 6 months ago.
demand >>> supply
Why does this lab test only have marketing materials? They are not even installing linux on a supposedly LLM dev machine
I seriously wonder how this is going to disrupt the market. At this rate, a new semiconductor, GPU, CPU is releasing every quarter.
Seems like having a big and clunky external power supply enables a smaller profile for the rest of the unit while making installation a bit more complex. How exactly is this thing going to be installed for use? Wouldn't it be easier to just have a bigger box with more shielding and heat dissipation?
External power supplies make UL certification cheaper; that's the reason. I hate power bricks but I have so many on my desk that one more makes no difference. The Beelink version has an internal PSU BTW.
Seems not really worth it? About the same cost as DGX, same amount of memory and yet the bandwidth is actually slightly lower. And also the DGX is CUDA being an Nvidia device which is a big compatibility advantage
For this to be compelling it would need to be eg 256GB minimum or something
Even a two-year-old Mac Studio outperforms this kit. A used unit with sufficient memory currently seems to offer the best price-to-performance ratio
"The Apple Silicon Mac Studios outperform the AMD Ryzen AI Max+ 395 machines"
Comically, the 512 GB M3 Ultra Mac Studio that we tested isn't even available for purchase any more. The highest you can purchase from Apple is 96 GB.
I imagine there may be users who can’t use macOS, or maybe they want the ability to upgrade storage.
The framework desktop even has a usable PCIe 4x slot available if you put the board in a different case. They sell the 128GB board on its own for $3150.
And how much can you buy a 128GB Mac Studio for now? Go look. I think you'll be shocked.
A 2-year-old Mac Studio 128GB also sells for more.
People complain about the cost of RAM for data centers, yet happily buy devices with 128 GB of RAM that will see an order of magnitude less use than servers in those same data centers.
So... I dont want to ruin gaming more but why not get a gaming PC? Figured this out 15 years ago if its good for gaming, put some more RAM in and boom you have a workstation...
The shortcoming is the memory speed/bandwisth.
With a desktop your system memory is slow and your fast graphics memory is limited in size.
To me it seems like the best bang for your buck in the BYO desktop PC space is to get a board with dual PCIe slots then find some old generation 24GB GPUs like RTX 3090.
But you’re not getting access to more than 48GB of fast memory without something similar to this or a Mac Studio.
Interesting thank you, almost reminds me of a PS2 then, super fast memory but not very good "traditional" GPU
The biggest problem is that if you want to run large (continuous memory) models, gaming graphics cards aren't sufficient, and if you manage to get graphics cards that you can chain it becomes a lot more expensive (and better performance) than these machines and GB10 machines.
It would be really nice if they included clustering support like a blueprint on how to buy several of these and cluster them to run the really large models in the best way possible.
Is this really worth $4000? I'd be curious how it compares to a couple of used 3090s in terms of the models it can run and inference speed.
i always thought Ryzen AI Halo, together with DGS Spark, has mismatched compute capacity with memory size. Given 128GB VRAM, people would want to run large models, but the GPU compute is constraint in these types of use case. If the box runs models that don't need high compute, then there is no need of 128GB VRAM.
On the high end side, it is too slow. On the low end size, it waste money on VRAM.
I bought one of this system back when you could get one for $1800 (the GMKTek Ryzen AI Halo 128gb machine). It's a very good dev machine, the 16 core CPU is quite good for development work. I find this is a useful configuration for local LLMs with plenty of RAM left over for doing actual work on the system (split something like 64 system, 64 dedicated to LLMs).
I don't think I'd pay $4k for it today though, 2 years ago and less than half the price feels like a good machine. I'd be very disappointed in it today for $4k.
When this was half the price of the DGX Spark, it made sense. But same price is a ridiculous premium for inferior performance but the ability to run Windows.
I was considering getting an AI Max+ machine last year when the price was around $2K. Crazy to see the same specs now going for double the price.
I was in Gray Scott School for HPC last week, and even in scientific usage, CPU-only cases, AMD is still a pain point. Many tools and libraries don't have first class AMD support or any support at all.
It loses to Intel in CPU, and NVIDIA in GPU, in case of scientific libraries and HPC-worthy libs, tools.
I think people who want an "AI Dev Kit" will lean towards Intel + NVIDIA setup.
I am not a fan of Intel, but their MKL, MPI, etc. are not paralleled. Same goes for CUDA with NVIDIA.
> It loses to Intel in CPU
Not really, their CPU market share has dropped in the last decade.
I spoke in the context of supercomputer clusters, and usage in scientific HPC.
Are the likes of Dell and Lenovo not going to be annoyed that AMD are cutting them out?
As traditionally AMD was a supplier of parts.
Both Dell and Lenovo have tended to favor Intel first... even more recently on business laptops.
Do they care that Microsoft is selling the Surface, or that Intel used to sell the NUC?
Intel = Fair as they sell both Intel and AMD so no loyalty on either side.
Microsoft = yes, they care enormously, as Surface has taken away many sales. Albeit they sold some ChromeBooks
Dell is an Intel shop. They dgaf about AMD.
Pretty sure they sell lots of EPYC servers
I had hoped this was about Medusa Halo, but unfortunately, it's about 2025 technology. It's the same as Framework Desktop was at the end of last summer, which would have been a slightly silly but fun buy at $2k... I'd hope Mark Cerny / Sony launch PS6 sooner rather than later, as together with the upcoming LPDDR6 standard, it should trickle down to us in the local LLM mud eventually?
> I hope Mark Cerny launches his PS6 sooner rather than later
With the current RAM and SSD prices... I rather a bit later.
True, this is the new reality though. My main gripe with Strix Halo is memory bandwidth and compute performance. Gaming performance sits squarely in base PS5 territory just as is the case with Steam Machine AFAIR; yet due to economies of scale "cheap" 2020 era PS5 still has higher memory bandwidth by quite a bit last time I checked.
PS6 "undertaker of physical media" will supposedly be priced >$1k: https://youtu.be/-F1JS-4Abjo
I want to play with openclaw for continuous workflows without burning my cloud credits. Do I want this?
no, you openclaw is too vibecoded, use Hermes
Why buy this over a framework desktop?
No practical reasons. Just cool looks.
Drastically slower than Macs and NVIDIA unified memory boxes while not being any cheaper.
The $4K is awful lot of heavy API use.
Perhaps if less spending went towards their private aviation interests LTT labs could review a piece of hardware that was released _this_ year, or maybe extend their narrow testing process to cover real-world use metrics like TTFT. Not to mention the lack of real value-perf comparison to CUDA
This hardware was just released this week? The CPU is from last year, but this dev kit is brand new.
Can you explain which benchmarking differences you'd expect to see from this hardware, considering?
None evidently. Why does that matter? I was just pointing out that this hardware was released _this_ year.
Can you articulate which new hardware you would have preferred them to look at and why?
what can you realistically do with this ? $4k is a lot of money to spend on something like this without really being sure what models can reliably run
How well can it run GLM 5.2?
Tough sell. High price, uncompetitive mem throughput
wow only $4k for an unupgradeable computer that will never be able to run anything that uses CUDA
This site is pretty awesome
Ok, but can it run crysis?
$4k is pretty darn spendy. I recently purchased a refurbished Corsair AI Workstation with almost the same hardware (same chip, same 128GB RAM, but only 1TB storage) for $2160. Pretty good deal! Codex and I wrote a Linux driver to report the power mode of the device:
https://github.com/pettijohn/corsair-ai-workstation-performa...
Is that price still available?
I think I got lucky -- look for "revival" series on their web site. I happened to see it one time and pulled the trigger.
Anybody knows when will it be possible to buy the newer 192gb part?
this reminds me of 'the box' from silicon valley show
why is everything a BOX ?????????
why not some other platonic solid
I think I might be tempted to wait for the Butlerian market crash and pick up stuff from the firesale.
Depends how long this market can remain utterly loony though.
>Depends how long this market can remain utterly loony though.
Probably way, way longer than we'd ideally like. Every now and then, you hear that this is the new normal, that it'll last until 2032 or something, and I can practically smell the paid advertising or cult-like messaging behind attempts to destroy any speck of hope consumers have so they'll just pay the toll. But man, it really could be exactly that. There's just too much money to be made for all the involved parties.
> There's just too much money to be made for all the involved parties.
Even setting that aside, depending on the pace of datacenter buildout even a round of new fab capacity coming online might not be enough to bring prices down. I think there's a decent chance we have to wait for investors to get tired of building new datacenters which could easily be 5+ years.
Nice, but I understand that ROCm, the software part is terrible.
Another Framework Desktop clone.
[flagged]
[dead]
I love(!) how these dev!kits are for devs in silicon valley making 300k+ a year and not any other dev in any other part of the world.
Satire if you can’t tell…
Eww, LTT.
The Mac beats it in all benchmarks, is probably more energy efficient, can add more ram, and is more cost efficient (?)… plus you get a Mac. This doesn’t even give you cuda. I’m not sure who this is for.
The 128 GB Mac Studio is no longer sold and when they bring it back I predict it will be more expensive.
This is for people that want an x86 machine instead of an ARM machine stranded on macOS.
Let me guess, the power brick is twice as large as the actual machine and it sounds like a vacuum cleaner because the fans are way too small.
So it's usable only in data center conditions but then why make it in this form factor?