I bought $18 GLM official subscription yesterday (5.2, but new model version was already leaking on some docs), set it up with Claude Code harness... and I’ve bumped to $80 plan almost immediately. It’s the first model that agreed on a proper security research (red team scenario), executed it seamlessly, including 0-days in WP plugins, RCE, 6.8 kernel exploit adaptation, etc - while playing against another GLM agent as a defender (following HF story)!
I understand that such models can be used by malicious actors, but it’s fair to have it publicly available (and play on your side in case of emergency). This is what changes the world in a better way, I think, not the guardrails.
Yes, I am tired of Claude and GPTs. I am ready to diversify my $300 per month on other vendors. Will try GLM. How was your rate limits and availability experience on $80 dollar plan?
It’s comparable to Anthropic usage, to be honest. 2x GLM agents ate 18% of weekly usage on this mid-tier plan within ~8 hrs (non-stop work, a lot of tool calls, appx 4 compactions each), I think. I didn’t make a proper statistics snapshot, sorry.
I also quite like driving OpenCode through something like Kepler / Paseo and tools like that (with those I can still use my Anthropic Condition by Claude Code being treated similarly - as something that gets tasks dispatched to it, while the GUI I see is Kepler / Paseo).
On the desktop side, ZCode was surprisingly usable for something that came out of nowhere (I wasn't aware of it at all before trying out the GLM Coding Plan): https://zcode.z.ai/en
Last time I tried some of these, none of them had the "manual mode" that CC has, where it shows you change by change as diffs and you can edit them before accepting and moving on to the next change. I like that because if it's going off pattern I can spot it early on and guide it correctly, instead of having to review the whole completed diff at the end when it's too late. I should spend the weekend checking them out again to see if they added that but I assume with everyone going full agent mode they probably didn't.
The philosophy with Pi is it is minimal (but functional) out of the box and easily extensible. I'm not familiar with that feature but I would not at all be surprised if someone already coded a Pi extension that does it.
T3 Code has been amazing. Completely free. Really impressed with the desktop app and the mobile app experience and the way it works seamlessly has me actually accomplishing tons of stuff while I'm out on mobile that I would otherwise have to wait to come home for. First time in a while I'm actually excited to use a desktop UI instead of the terminal. Blows away the official Claude Code mobile app. I can switch between my Claude and Codex monthly subscriptions in it as well. There's a TestFlight beta SwiftUI mobile version that's so much nicer than the one in the App Store. I'm running the nightly version of the desktop app.
And this is coming from someone that's not particularly a big fan of Theo. T3 Code should get more recognition; people aren't just aware of it yet.
Piggybacking on this thread to ask my question: What are alternatives that are multiplayer (team oriented) by default? For example, I want my team to see all my sessions easily, vise versa. another way of stating: all the agents are running in a container that that any member of the team can view and interact with.
It indexes the code efficiently, seems to find stuff quicker, it has a very nice UI (much better than Claude Codes IMO), it has a nice sub-agent UX which I find triggers more reliably, diffs render nicely. Otherwise it just seems to work in a purely vibes sense.
That said Claude Code is perfectly fine. I just prefer the integrated experience of using Cursors since I already use VSCode, but I still mostly use Claude Code because of their Max/Fable plan.
I’ve tried a bunch of them, and I seriously do not understand these recommendations. It was a rough road and a steep hill, but right now CC is absolutely the best harness on the market, as for me, whatever top tier model is under the hood (mostly, some of them, like DeepSeek, don’t fit CC at all).
Inversely I don’t understand the praise for CC. These days it feels like bloatware. It absolutely can get the work done but when I measure on token and time use it ends up being a multiple of pi like harnesses.
CC works but for me it felt like increasingly they have zero incentive to make it a great experience. You hear folks like Boris talk about spinning up thousands of agents over night and agents chatting back and forth in GitHub issues and while I think it’s great from figuring out what the future looks like I don’t think it represents the reality of ROI today. So the folks building the tool are so disconnected I am simply not sure it’s a great experience anymore.
So is the quantitative difference in token use the only difference or do you think there's also a different qualitat? I'm on CC only and immensely happy. Very productive both at work and privately and at work I average around $250 a month which probably means nothing but it's little compared to my salary.
For me, at least it's that the newer Claude models seem optimised for one-shotting things, which is not what I want. As the amount of code per turn increases, I have a harder job keeping up and ensuring that it's doing what I want.
That being said, I had to nope out of a similar thing from GPT 5.6 today, so it appears to be a US frontier lab issue. Claude is particularly bad though, as it produces far too much code even when I tell it not to, unlike GPT (and Kimi) which at least listen to me a little better.
More generally, I want a usable human review experience, and Claude code doesn't deliver that for me.
Quality is hard to measure and I would not say the concern is so much cost but the intersection of cost and time. Often I am jamming on something and I like being somewhat in the loop. So maybe same level of quality, I am using Anthropic modela for both harnesses, but I get to the output quicker and at a drastically lower cost.
It's buggier for me than it has ever been before. I don't think that agentic coding always leads to such a buggy mess. I just don't think that the Anthropic front-end software team is very good at agentic coding.
Funnily enough, I would say almost the opposite. CC’s feature set is basically table stakes for an agent these days (does it have ACP yet? Very close to behind table stakes if not) and it has a lot of bloat powering that.
IMO part of it is that the underlying LLMs have gotten better enough that harnesses feel better even if they haven’t changed. I have a toy harness that barely implements the features you’d expect and it works surprisingly well. Like there’s literally nothing clever, it calls tools and that’s about it, and it still mostly does the right thing.
Why would anyone ever need ACP? I'm not trying to be an asshole. I just seriously don't understand the value proposition.
Edit: lol, I don't think ACP is even actively developed anymore. It seems to have been merged into another seemingly pointless standard with an even worse name, A2A. [0]
Just as a +1 anecdote. I enjoy using pi a lot. I used to h think the harness matters a lot but with the current iteration of models I am starting to sway that while it matters it’s less and less important and that CC is bloated. I did some quick tests when I switched and a task that would take $5 in tokens would be completed in $0.50 in pi. Very anecdotal and I don’t have a test framework setup to make this very official but increasingly felt like CC was spinning its wheels on the easiest of tasks.
Based on the fact that Claude Code is only optimized for Anthropic models, whereas Pi and Omp are optimized for a wide variety of models, including open weights.
What’s the counter argument? pi and ohmypi are pretty fantastic. Of course like all developer tools it depends how you do your work but I am not sure what you are trying to achieve in your comment.
> I understand that such models can be used by malicious actors, but it’s fair to have it publicly available
I feel like there should be some mechanism to prove you own the code/app/site/whatever and it will remove the guardrails from the LLMs allowing them to find and fix these vulnerabilities.
Don't we already do this with services like Let's Encrypt, which is arguably more sensitive? If you had the codebase you could fake it, but it would still provide some amount of protection against abuse.
With Let's Encrypt, all the verification is done on their side with them controlling the connection between themselves and whatever they're trying to verify.
In this case, you can put whatever you want between the harness you're running (or modify the harness itself), and essentially "lie" to the model. Any verification technique would be fairly trivial to bypass, while you continue to run the harness locally.
How much usage do you get out of it per week? How many millions of tokens?
Anthropic was stingy as hell with its Fable and cybersecurity nonsense, switched to OpenAI which is much better but still not enough. I'm tempted to switch again...
They're invaluable for developers to fix their code. This is definitely an area where AI decisively beats human devs in a very valuable way. It can try so much surface area so fast.
If it won't attack my stuff, it won't help me build my stuff to be secure.
Apparently they are scanning OSS and popular software at scale and disclosing the vulnerabilities they found: https://cvd.z.ai/
Most of these are under embargo, but it seems there are a lot of CVE here from a wide range of popular software, many considered critical or high.
I understand the argument of "people are not actively looking", but isn't the cost for such a scan getting lower by the week, and Anthropic's Project Glasswing is supposed to find them quite a while ago?
> ... Anthropic's Project Glasswing is supposed to find them quite a while ago?
That was my thought too. For all of Anthropic's talk about their "adversaries", it seems Z.AI have been quietly offering fixes for single shot Remote Code Execution flaws in US software (Safari / WebKit) that Apple and Glasswing / Mythos missed, and that Apple would not attribute to GLM.
That was a wtf to me, so I checked Apple’s latest iOS release security content and GLM & z.ai is mentioned once (under WebKit), Anthropic is mentioned twice, Codex is mentioned once. Not clear if there are other instances where the model did most of the work but wasn’t credited. I didn’t bother to check other releases.
> That was my thought too. For all of Anthropic's talk about their "adversaries"
It’s very likely they found all of them, but that the same happened that happened to Microsoft a couple of decades ago: NSA orders not to disclose / fix them so that they can put it in their collection of unfixed zero days.
This is a coherent explanation for why federal model censorship has started with cyber capabilities. But this GLM model release is an in-your-face challenge to that policy. They now have to either set models free or impose a censorship regime that will put anyone not under it at an advantage. Or muddle along in the middle as usual.
> I understand the argument of "people are not actively looking", but isn't the cost for such a scan getting lower by the week, and Anthropic's Project Glasswing is supposed to find them quite a while ago?
You have to consider that having an LLM scan for vulnerabilities is hardly infallible. It is a search guided by heuristics and given a large enough codebase, it is unlikely to identify all vulnerabilities.
Personally, I've had Fable 5, GPT 5.6 Sol, and GLM 5.2 all looking for correctness issues in an old abandoned WIP codebase of mine and all of them found some that the others hadn't discovered. Now, correctness issues aren't the same as vulnerabilities, but the same principle about using heuristics to find defects applies.
> [A]ll of them found some that the others hadn't discovered. Now, correctness issues aren't the same as vulnerabilities, but the same principle about using heuristics to find defects applies.
This makes perfect sense, but that conflicts with the impression put forward by Anthropic and OpenAI (in particular) that they alone occupy 'frontier model' spots. Frontier models should large dominate their competitors on a capability basis, but if GLM 5.2 (now 5.3) is routinely finding bugs / vulnerabilities missed by Fable and Sol then GLM might be genuinely a frontier-grade model by itself.
For normal software it would be as you say, but LLM progress is so ridiculously fast that things go from "bleeding edge" to "eh, you'll do" in about that timeframe, and "eh, you'll do" to "why even bother with this old rubbish?" in the same again.
Or, from a different perspective, we can expect some new frontier model from Anthropic in a week or two, and from OpenAI in a month or so.
> This makes perfect sense, but that conflicts with the impression put forward by Anthropic and OpenAI (in particular) that they alone occupy 'frontier model' spots.
Not necessarily. Even near the frontier, we don't really have a total ordering of capabilities, but a partial order. And even frontier models make plenty of mistakes. Combined with the randomness inherent in searching large codebases for vulnerabilities or correctness issues, it is entirely plausible that even much weaker models (and GLM-5.2 isn't even weak) can stumble upon issues that stronger models missed.
My current hypothesis – for which I have only limited evidence, unfortunately – is that it is better to have multiple reasonably powerful (but not necessarily frontier) models looking for issues than just one very powerful one. And even then you're likely to miss out on some issues.
I find that LLMs also generate a lot of false positives, or extremely minor issues that don't warrant a fix (that are always overstated by the LLM as very important!). Signal to noise is still not great and requires somebody to wade through and pick out the actual good findings.
this is impressive and actually matches my expectations in terms of near term AI progress. we are going to continue to seem impressive progress in coding & related, anything where verifiability is scalable in an automated way: https://transitions.substack.com/p/a-quantum-of-ai-progress?...
There's a chance that the real reason why they want to ban Chinese models is that they are so good at fixing bugs and preventing exploits that intelligence agencies have been using for espionage and surveillance for a long time.
Anyone who knows anything realises banning things is a) impossible and b) your enemies will use them anyway, you are just depriving your own side of the advantages.
> Anyone who knows anything realises banning things is a) impossible and
Maybe "It's really hard" is more accurate? We (humanity) for most part basically agreed to ban the usage of various chemical weapons in wartime, which seems to have drastically reduced the usage of it, even though it's still used by shit actors today from time to time. But it's hard to deny that usage didn't decrease after banning it, which makes "banning" maybe not completely useless for certain things.
"Banning" things that can be easily copied over cyberweb transportation pipes feels like an fool's errand though, regardless of what it is. It's just too easy to get around, compared to actual physical items I suppose.
This is different now. US labs and companies are not releasing frontier-level models openly (specially those capable of assisting cyber intelligence work), but commercializing them instead. Thus, any ban would not be symmetrical to begin with, and that is precisely what maintains the balance.
It's pretty easy for the US to functionally ban chinese models. They only have to target US firms like inference providers or the biggest users, and pretty much the whole domestic market will fall into line. They don't actually care about the final few %.
Regardless of whether or not adversaries are using them, the US has by far the most compute available, and we've now hit the line where major providers are no longer releasing their best models. The public gets the "current" level of intelligence, while the US government gets to control access to the actual frontier of non-public AI. From their perspective, their enemies using GLM5.3 while they have GPT6 and Mythos6 or whatever is a fine trade.
I don't support a ban at all, nor the US's behavior, I'm just pointing out some facts that change the argument.
I don't think this really works because the Chinese government is going to be incentivised to tip off the US companies to deny the US government those exploits. I guess maybe that's what the open source patch program here is about, making sure banning the models doesn't work because they can just report the exploits without the company running the model themselves.
The CIA ran one of the world's largest cryptography companies, for DECADES[1]. Are you truly so naive that you believe intelligence agencies that have more to gain from stifling the discovery of vulnerabilities they know of and use wouldn't do so?
You should probably realise that the world has radically changed since then. This kind of thing works when you have a significant lead in the field that makes keeping vulnerabilities open sufficiently low risk for your own side. But if your adversaries have similar capabilities, then the calculation changes.
Has anything changed? Governments are hoarding undisclosed vulnerabilities, using them as they see fit instead of fixing. Every espionage, surveillance, or war campaign (see Russia v Ukraine, US/Israel v Iran etc) is followed by a ton of burned 0-days.
>This kind of thing works when you have a significant lead in the field
No? It works even if the adversary has the same capabilities. It only stops working when everything is fixed.
I used to call everything a conspiracy theory, but then Glenn Greenwald published
"No Place to Hide: Edward Snowden, the NSA and the Surveillance State".
Now i know that reality is worse than the worst conspiracy theorist.
I don't think reasonable people post here much anymore. It's mostly galaxy brained conspiracy theorists and ignormamuses posting political garbage. Reddit-lite on the way to full blown Reddit
Why would you even believe the opposite? US spooks have been amassing vulnerabilities and relying on them for decades, they literally pioneered it in the 90's if not earlier. Everyone does it now but the US is the biggest of them all. Surely this devalues a lot of what they did. Moreover, the way the US government handled new capabilities, and OpenAI's training policy (they are in bed with the government) just scream "we want to create weapons for cyber-offence and deny them to everyone else"
It might not be the reason, but of course it's a contributing factor.
Intelligence agencies have been known for exploiting and planting software and hardware Buga for decades, going as far as weakening cryptographic standards or intercepting hardware in transit to implant a backdoor device.
They've always been good enough for double digit less money. Always. Anyone thinking "Chinese models fake models built using dirty distillation scam" don't know what they're talking about.
Distillation is just forcing the model to use an exam prep workbook for training instead of generic publicly available textbooks. The models themselves has to be smart enough for that to work. It's the exact same thing as Asian tiger mom double schoolwork strategy, to paint a picture.
Their best models are getting more and more expensive, and still aren't SOTA.
It's almost like there's an actual cost to developing these models, and the Chinese don't have magic dirt that allows them to do it at a fraction of the cost.
> I'd love to live in a society where people and corporations do good things for PR.
Maybe so, but I'm not sure I'd like to live in China of all places. (Don't get me wrong. Lotta places I'd like to visit if I ever got the chance, and China's on that list, but to live there? I don't think so.) Maybe one of the Nordic countries?
This is absolutely still shy of Sol and Fable, but only just by a hair. Ridiculous results. There's still not a compelling economic reason to drop OpenAI courtesy of the ludicrous reset addiction that's taken place, but it feels like we're on the precipice.
How are you all toying with running this kind of thing in a mega quantized way locally? Two weeks out from released weights, but this is still just GLM 5.2 with post-training magic.
OpenAI and Anthropic are both seeking trillion IPOs, while Chinese labs are pumping out open-weight models that are free for US providers to host and monetize.
These Chinese models cost less of US SOTA models to run, even if they are less capable. Providers can just run them, offer cheap tokens, and pocket the margin.
I just don't see how you justify a trillion valuation for US AI labs when the underlying models are being commoditized this fast.
Whether AI works or is useful or not isn’t even the question anymore. It can fulfil every promise Sam Altman has been making and will still make no financial sense to justify these valuations.
I take it from [1] (transcript of recent DeepSeek CEO discussion with investors) that DeepSeek would disagree on the immediate catastrophic impact to the likes of OpenAI or Anthropic. The reason is even though technology parity mostly exists, only OpenAI, Anthropic et al have the inference capacity to gain market share and generate revenue. Chinese vendors don't have the chips needed to scale up inference and gain market share, and the DeepSeek CEO doesn't think this would happen in optimistic circumstances in the next 3 years, but thinks it might be possible in 5 years.
In summary, regardless of country of origin, availability of inference capacity is the moat protecting the likes of OpenAI and Anthropic, not technology superiority.
I would add that it is not just capacity, but also negotiation ability. With scale comes the ability to negotiate better prices than everyone else. Even if you can find capacity for your smallish user base, your inference cost can not match these companies unless you have a technical advantage for your inference cases. Squeezing the hardware requires request batching and caching which are far easier at scale and sustained user activity.
It only justifies their long term valuations if they can leverage that temporary monopoly for technological superiority (they can't) or lasting market share (they can't).
Chinese models prove there's no technical advantage, and the software side is heavily commoditized so there's not much advantages to market share either.
The question mark in my mind over the technological superiority is whether the additional volume of data they see due to capturing the top of the market allows them to do recursive self-improvement in a way nobody else can match, before any of the other labs can figure it out. That's the only runaway outcome I can see.
If you have exponentially increasing use of your harness, then it's true that every day you capture exponentially more data, but it's also true that every day exponentially more data will slip through the cracks of your would-be monopoly and that data arrives at your competitors via various channels (competitor harnesses, subsidized reselling, etc)
The very exponential that you are relying on to give you runaway improvement is also giving exponentially increasing data to your competitors. All else being equal your competitors stay a step behind but you never develop a monopoly either. That's the best case for Anthropic/OpenAI. In reality, training data is just one variable, exponentials don't last forever, and your competitors will get better at capturing a bigger slice of training data.
If user data would become such a key ingredient (which it might, i actually remember noam shazeer talking about the importance of user data), i think chinese labs can still get it from china, as keep in mind it ahs a billion people behind the great firewall banned from using us llms. And btw broadly for any gap like this, you really gotta consider that if its becoming a bottleneck, chinese labs will find a way to buy it from one of the labs unless theres strict regulation at the government level
But is that data good? That's the question. As in, is my usage at work:
a) indicative of problems that aren't already out there in the wild? (no)
b) are the responses I'm getting so good and novel that the model can improve itself? (no)
It's the garbage in garbage out idea, just scaled up. If the model gave a bad answer, and I didn't catch it, and you now train on that I/O pair (my perhaps crappy prompt, the bad output), then you're not going to improve anything.
Yes, RSI seems to be the new AI industry McGuffin of 2026, just as agentic capability has become table stakes and scaremongering has become a punchline.
The Chinese models are adopting licensing quite rapidly and Xi will soon enough close them for security reasons. The most widely used model, integrated across Bytedance apps and operations, has never been open and is most closely associated with the state.
Is lack of inference chips due to the trading blocks by trump administration? What if Trump agrees to sell chips to china, would they collapse then? That's not a very strong position to be at
People can host these models is doing a lot of lifting here, these are models that depends on 5 digits on specialized installation to run on.
IMHO, this has the impact of softening the impact of data centers sitting unused in the long term if they can still serve open weight models, even if Anthropic or OAI have to scale down their expansion rate to pay the bills.
Regardless, reality has to give at some point; these valuations don't make any sense. We've been valuing GenAI as disruptive work, when in reality they're much closer to cloud providers with a beefy, one-pony-trick R&D department.
I have already begun winding down my spend on claude and OAI to make room for infra budget. Anecdotal, but I have no doubt a lot of others are doing the same, I very much agree the US players have major issues looming. What an exciting time to be alive!
No time like the present to pull out and reduce your exposure. I brought this up in my employer's forums 4 months ago and honestly it's been clear even before then. In particular, the upcoming IPOs of both oAI and Anthropic will likely be disastrous for the public - the floor is falling from under them and I don't know if they can be scrappy and work with fewer resources - their internal culture may not support this. We all knew in our hearts they're a commodity - just see how easily you can switch between the 2 of them - and now there are 10 more options costing a fraction.
When Xi Jinping did the announcement of their open weights push, they might as well cancelled their IPOs....
Yes buuuut…. I do quite a bit of day trading (maybe closer to scalping) for the first few hours the market is open, everyday. Anecdotally: despite everyone knowing its valuation was ridiculous, I rode that SpaceX train pretty hard and made a pretty penny.
I close-out all my positions by end-of-trading everyday… so when the day came when there was a very clear and very scary indicator during early trading hours, quickly followed by SpaceX’s catastrophic fall right after opening bell, that was the end of my involvement….
And I fully expect oAI and anthro to be the same way. They’re being propped up with private loans, subsidies, and other tricky bookkeeping techniques. You would think their CEOs would pivot away from their current public personas. Ironically, they are like a poor man’s Elon Musk… and that doesn’t bode well for their companies
The frontier labs will do well if they pivot their offering towards more capable, larger-scale models that are inherently harder to both train and deploy for commodity suppliers. Their existing investments in gigawatt-scale datacenters are quite optimal for this. "Commodity" inference need not comprise the whole market.
I don’t think this works, for a few reasons. First, intelligence gains from scaling the models bigger is sublinear now. So they could eke out a little extra performance, but the increased cost will eventually eclipse the economic value gained from this.
Second, humongous models are impractical even for them to deploy widely. They’re best used as teachers for smaller, more efficient models that can crank out the volume they need to sell.
Finally, there is a data wall. Sure, they can keep scaling RL on math problems and code. But with everything else, where will the supervision come from when they need several orders of magnitude more?
I agree. And even if they were able to do it for one more round, it's not a sustainable strategy. What they (Anthropic and OpenAI) need to do is build platforms and integrate verticals.
assuming the technology of model architectures does not gain any further breakthroughs that returns us back to the gains previously seen. I'm of the opinion that we still have some discoveries on the mathematical side of the fence to go that will improve models further.
> I'm of the opinion that we still have some discoveries on the mathematical side of the fence to go that will improve models further.
That's assuming the infrastructure needed to develop models stays available financially and supply wise. A lot of the services used to train and develop models are supplied and funded by people who are looking for multiple returns of investment. If/when OpenAI and Anthropic valuations fall and they inevitably get acquired, will Meta/Alphabet/Microsoft still want to spend lots of money for unclear returns in the short-term? Nvidia and co are on a one way train service to hype town. I don't think they will be happy to get on a coach to hype town Temu version. The shareholders likely won't.
Also, the backlash against LLMs is growing rapidly. AI content, data centres, etc is quickly gaining negative connotations outside of visual and music artists circles. While existing models are going nowhere, developing more advanced models is very quickly getting unpopular. LLMs Data centres increasing people's bills, Anthropic destroying old books, chat bots giving unethical advice to vulnerable people, etc. It won't be long before LLM infrastructure becoming an electoral issue.
Will a small research oriented community be big enough justify maintaining the apparatus needed to produce infra tech at a profitable level post OpenAI?
Most of the money will come from companies/corporations who will be required to buy safe AI. The public will be just banned from buying which might make it hard (ie: site/payment blocked) but not impossible. It could be good enough for the big whales.
A large portion of the economy is currently tied up in the musical chairs shell game that is AI hype. When the music stops there are going to be CEOs looking for handouts and justifying it with spooky national security buzzwords. How we respond to that will depend on whether it happens in an admin that is famously captured by the industry or not.
Just keep in mind that it can take a whole for things to play out. I’m someone who believe the US AI industry is completely unsustainable and built on sand, and will crash even if the current AI itself turns out to be very successful. But that doesn’t mean everything will burn to the ground next week. In a history book things will look very sudden but at normal speed that can easily take months to years to fully play out.
Also, take in consideration that the AI trade infected a lot of other trade in the economy, if you decide at some point to move your money to a place that is safe in case of a downturn be sure to carefully evaluate that’s actually the case
These crises are manufactured by the central banks.
Compare and contrast how the dot-com bust did _not_ lead to global financial crises. Nor did Black Monday, nor the recent string of bank failures in the US.
('Manufactured' above means that central banks are responsible. I make no judgement on intent here. Around 2008 it was incompetence by the Fed and ECB as far as I can tell. The Fed started paying interest on excess reserves and the ECB even increased rates. Twice. Amongst quite a few other missteps.)
A lot of the performance of these open source models might come from distilling the closed frontier models. If those can't raise the funds anymore to train newer and better models then the whole improvement cycle might slow down.
Another interesting potential market here will be 'LLM in a box'. All the hardware and other tooling in a prebuilt, but modular, package ready to go. Pay one up-front cost, get a system running [whatever open LLM] with a token rate of [x], optionally configured to be immediately ready for distributed usage. Basically the opposite of cloud stuff: no rent, no dependency, 100% guaranteed uptime, guaranteed security/privacy (at least subject to your own actions), and so on.
Palantir already offers a "turnkey AI datacenter", i.e. a rack with "NVIDIA Blackwell Ultra systems with eight NVIDIA Blackwell Ultra GPUs and NVIDIA Spectrum-X™ Ethernet networking for AI training and inference".
It is said that it comes with all hardware and software required to run inference or training with an open weights LLM.
The existence of this product, which competes with cloud-based offerings like those of OpenAI and Anthropic, is presumably the reason why the Palantir CEO criticized very harshly some time ago the business model of OpenAI/Anthropic.
While I doubt that the ethics of Palantir is any better than of OpenAI/Anthropic, in this particular case I have to agree with Alex Karp about "Sovereign AI", i.e. that only losers will make their business completely dependent on an external entity like OpenAI or Anthropic, who are certainly not trustworthy.
It is just a dedicated computer system, which should be managed by its owner, like any other on-prem servers.
I doubt that it has a good price/performance ratio, but it is a solution for those who feel that they do not want to search, buy, assemble, install and configure every HW/SW component.
Fair but the idea of "running your LLM setup" at every "need" level and corresponding cost does make sense.
For a lot of people (and orgs I'd guess) who just go and buy ≈$20 per month plans (or more for teams), they might not even need a fraction of that cost or capability. A lot of them don't even need it for coding or graphics. Even the API access based pricing aren't great from these frontier US AI houses. The distribution of "LLM being" offered will also give rise to many open-router like offering but at the end point level - direct interfaces to the customers. Pick your vendor sort.
AI shouldn't become another "search means Google".
“100% guaranteed downtime when you least can afford it and the support tickets are your problem.”
We’ve a hybrid shop, including hosting our own ML infra, and we save a ton from cloud spend with local ML. Easily one million USD over past three years. But it’s not “free”, you are shifting a lot of labor into your plate.
Still has to break even on the balance sheet, especially at a bootstrapped startup. We actually made most of the financial windfall in translation API fees oddly enough.
For our own model training we needed to do some large scale translation tasks of a large dataset (1M or so documents, 10 or so target languages), running full-size NLLB on-prem saved us an absurd amount of money vs Google Translate API.
(For reference doing 1M target docs into a single language in Google Translate API is roughly $120k list price. You can run full size NLLB on an 48GB NVIDIA A600 and the major difference for us was speed, but for this task time to completion wasn’t an issue.)
What makes that kinda complicated is that multi-user throughput of LLMs scale well but single-user performance often stays constant at low ends. If you could saturate e.g. 16 concurrent session-month of demand, you can just go buy 16 of 32GB GPUs and start charging monthly for inference. That could work if you had e.g. over thousand total employees with hundreds of devs eager to trying it out, but only if the company is also interested in a private inference experiment.
You're talking about multi-session vs. single-session throughput. A single user can easily leverage multiple sessions via e.g. subagent swarms, especially on a lower-end setup where any single session is going to be quite slow. Saturating utilization during off-hours is harder but potentially quite feasible by assigning lower priority, unattended tasks/inference loops.
I think at this point the question is: will the US government be willing and capable to justify the trillion dollar valuation for _one_ of the companies via regulatory capture? The US has a workforce of 170m, so 1.7 trillion would come down to 10k per person, or a discounted cashflow at 3% of 25 USD per month - not including private use, students etc.
It’s a common denominator if you want to do napkin-math for a whole national economy. Regulatory capture is like a tax on those people not on the beneficiary side, so if the government were to nationalize both supply (no export license for SOTA models) and demand (no foreign or self-hosted LLMs allowed), they’d end up making everyone else pay for it in some way or the other. The governmental utility function will then include only those using the services for direct economic benefit.
It is impossible to justify the absurd private valuations they have given themselves in collusion with investors.
I wish they had tried to IPO because then we’d see the judgement of the market on this. But that’s why they didn’t this year. How long can they keep up the charade that their models are uniquely valuable and on the path to AGI?
its interesting, as it's typically the banks and against the public at large because the goal is to jimmy up valuations to justify IPOs then sell on opening; just like spacex.
It's what enron was doing; it's what most of crypto's offshoots were doing.
Sure you can blame the marks of the grift and say "well the public should know they're faking all this cash flow expectation".
It seems like you're either driving the grift economy or part of the collusion.
It's similar to how a cult operates, so I'll be frank: your skepticism seems biased.
Which is not unreasonable. Just hosting it in the EU and promising not to retain / sell the data let's you charge a healthy extra and compete in many areas other players can't.
> Just hosting it in the EU and promising not to retain / sell the data let's you charge a healthy extra and compete in many areas other players can't.
It's been a few years. Has anyone done this successfully yet?
There are a over a dozen EU open-weight providers. I’m not sure if they are even charging that much of an extra. EU-based clients have little reason to use non-EU inference providers.
I don’t have user statistics but my mail/domain registrar Infomaniak advertises Qwen 3.5 and Apertus, “a Swiss open-source AI model, developed by EPFL, ETH Zurich and CSCS”
Most are not necessarily free to host and monetize. At least one of them has a license that says if you are re-hosting the model then you need a license with that company that made the model.
"I just don't see how you justify a trillion valuation for US AI"
- military applications
- financial applications
- medical
- applied science
In all those cases it is achievable for those who have needed training data, and Chinese are not going to get them easily. US AI Labs are showing: give us the data, we will do wonders, promising "singularity"-level future achievements.
I'm sure US billionaires will find a way to extract those trillions from the public. They're smart, they can handle it. After all, they can ask AI for advice on how to do it.
Could you just tell us why you think they were worth billions before ChatGPT, instead of suggesting that we think on it? You seem to know the answer already, so please share it with the class.
The thing that blows me away is it does this at one quarter the total parameter count of K3 (and 40% active parameter count). There's plenty of room at the bottom.
> How are you all toying with running this kind of thing in a mega quantized way locally?
Sure, let me answer that in excessive detail. I briefly tried running the UD IQ3_S quant of GLM-5.2, which is 288 GiB of weights (301 GB). Setup was: llama.cpp, 1x NVMe SSD (Evo 980), 64 GiB DDR5-5200, i9-13900HX, and 1x RTX Pro 6000. Token generation around 0.7 t/s. Not remotely usable interactively, but something I could plausibly push a codebase into and come back to a review in a couple of days.
There's potential for that hardware to go much faster, but current local inference backends make poor use of the memory hierarchy. Ideally I would have: always-active weights, KV and hot expert cache in VRAM; warm expert victim cache in host RAM; and disk as a last resort. Instead it's 1/3rd of the layers fully pinned in VRAM (all experts), and 2/3rds running wholly on the CPU with mmap()'d weights. The CPU cores spend most of their time sleeping on disk fills.
llama.cpp has backed itself into a bit of a corner architecturally by trying to support all models on all possible backends. If you look into how their "MoE offload" feature works (not viable for me because it requires enough host RAM to permanently pin the weights) you very quickly realise it's "oops, all bubbles!" due to the static compute graph splits. There are more focused frameworks like DS4 [1] and Colibri [2] which have better support for streaming weights from disk, and support GLM-5.2.
Obviously I wouldn't recommend my setup for huge models like GLM-5.2. Supposedly it can just about be squeezed into 3x GB10, or run comfortably on 4x GB10 (tensor-parallel) for multi-user serving. I'm not sure whether that qualifies as local, but it's at least not a rack.
I'm hoping colibri can start pulling in specifically designed models for the heirarchy of decoding. It seems like we should be able to get smarter MoE models that can do the work.
Not sure about Sol as I haven't used it, but, at least for security work -- does it matter? It's not like you will be allowed to use Fable (or access Mythos) for anything cybersecurity-related unless your name is "Dario Amodei" or you are one of his rich friends. So regardless of how good Fable/Mythos is here it's a completely moot point for normal people, because they can't use it for that anyway.
Why should I apply for *cybersecurity* approval in order to have model debug a program it is writing itself? Anything related to memory safety, debugging, syscalls etc (meaning, "programming") somehow is cybersecurity now?
You must be a 5000 person company with an existing enterprise contract to get approved that fast. That sounds like a 15 minute SLA agreement. Individuals no matter how qualified about cybersecurity, are ghosted
That's not my experience at all. I was approved fairly fast - around an hour from submitting the form and getting a response.
However, even being in the cybersecurity programme, Fable refuses to answer prompts that it determines could be even tangentially related to cybersecurity. In fact, for a while, I was unable to use Fable with any prompt, as it recalled from memory that I was a cybersecurity professional, which triggered the refusal even for simple prompts like asking for a chili recipe.
I am guessing you are approved for the Cyber Verification Program. I also applied and got approved in an hour (on a Saturday!), but it only applies to Opus and Sonnet: https://support.claude.com/en/articles/14604842-real-time-cy.... It let me use Opus for cybersecurity work, pretty much everything except for Ransomware development. It would occasionally still trip and start saying no till I added a note about CVP in my claude.md.
No one gets to use Fable for Cybersecurity work, and Mythos is not available under CVP. Only for select few customers, and there isn't an application form?
Have you tried to use Fable for anything even remotely security related, when the refusals kick in as soon as you even fart in the vague direction of anything security or biology-adjacent?
I don't understand all this spite about "rich friends" when it was the US government that shut Fable down for not adequately blocking cyber capabilities.
I mean what honestly are you thinking Anthropic can do to give you better cyber tools? Their frontier model was literally nuked by the feds for a month for doing it.
"Mythos" is the cyber-security equivalent of Fable (without guardrails), and only a very select few corporations have access to it.
Fable is their version with guardrails on everything except "Make me a pelican svg" or "create a to-do" app, that is the version that the government banned
Only a few corporations have Mythos because the US government is whitelisting them one at a time. Anthropic releasing Mythos to the public was never on the table, they would have been shut down in milliseconds by the feds if they tried.
Before the US government had anything to do with this, Anthropic were fear mongering Mythos (BTW, Amodei also fear-mongered GPT-2, so this is a normal pattern in their operation) calling it "too dangerous to release", and back then only Anthropic was in charge of the whitelist.
Then the government believed Amodei's bullshit and this is a result of that, this was all self-inflicted.
Sorry but if you stepped back for a moment you'd realize this is all contrived nonsense to let to have your cake and eat it too.
No, Anthropic did not mind-game the US government into being worried about cybersecurity. The NSA has been paranoid about cyber controls for longer than you've been alive. If Anthropic had come out of the gate saying "no don't worry man, our model is TOTALLY COOL", while simultaneously attacking HAWK and finding core Linux vulnerabilities, I assure you the US government would have caught up about ten minutes later and we'd be in exactly the same spot minus your ability to tell Anthropic they were wearing the wrong dress and asking for it.
Mythos isn't some scary dangerous model that can find high severity bugs seamlessly, that's just Anthropic marketing. Most of the vulnerabilities they found were low severity hyped up to make their model look good, with (I think, maybe?) the exception of a few.
Now that Chinese open weight models have similar capabilities, and their guardrails can also just be removed, it doesn't look like anyone has "hacked" into everything because of the scary dangerous models like Anthropic were making it out to be.
The majority of high severity vulnerabilities are not the kind of thing you need a PhD in Comp Sci to comprehend, they are mostly about finding a way to get a system to end up in a state different than was anticipated when entering a particular code path.
Exhaustively looking at code and identifying ways to do this is something LLMs are quite good at. They don’t get tired, and you can run them non-stop.
They're also (generally) quite good at reading the literal meaning of the code, whereas humans often see the intended meaning first, and can be biased.
If you had a tireless junior engineer who was given the job of “make this application get into a state it’s not supposed to be in”, you’d probably get similar results.
What Mythos is quite good at is both the first bit and coming up with ways it could chain that together with other bits of unexpected state to create something that forms a meaningful vulnerability rather than a dead end.
All models find vulnerabilities. What is special about this generation of SOTA models, including Mythos/Fable (the same model), GPT-5.6, Kimi-K3, and now GLM-5.3 — they can chain vulnerabilities and produce working exploits.
Look at the recent HuggingFace hack. One vulnerability was template injection, another — remote code execution. Combine them and you pwned the remote server.
People working under Project Glasswing reported that Mythos at one point chained 20 vulnerabilities to produce working exploit.
Humans don’t usually do that.
It's also quite hard to separate Mythos the model from Mythos the campaign (aka Glasswing).
They put an enormous amount of compute into bug hunting, and they found some bugs. Fair enough. For me that begs the question: what if they had spent the same compute on generating more tokens with a less-capable model? What if they had spent it on traditional fuzzing?
> I don't understand all this spite about "rich friends"
Okay, here's a challenge: I assume you're not a rich and powerful entity, so try to gain access to Mythos. I'll wait.
> I mean what honestly are you thinking Anthropic can do to give you better cyber tools? Their frontier model was literally nuked by the feds for a month for doing it.
Well, first I'd suggest they stop with the constant fear mongering.
Here's my prediction for what will happen: the Chinese models will catch up to Fable/Mythos. They will be fully unrestricted and everyone will have access. The world will not end. Good guys will use them to harden their systems, in equilibrium to what bad guys have access to, so effectively status quo will not change.
Right, so according to you it's because of the US government that they don't release it to the public? Have you missed their constant and incessant fear mongering?
The causality chain here was not "US government says its dangerous -> Anthropic can't release it", it was "Anthropic is fear mongering -> US government listens to their fear mongering".
The issue is that these companies keep trying to pull the ladder up behind them by going "oh my god our models are so dangerous only we should be allowed to develop them". Sometimes it backfires, but the companies aren't innocent.
Have you seen the news about decrypting the hidden COT in U.S. models? [0] The decoded logs revealed instances where Claude memorized answers to test questions beforehand while making its final output look like it had derived the answer step-by-step—hiding the memorization from the user.
It basically has been ever since they started using RLVR for reasoning (esp. coding & math), with the DeepSeek-R1 paper being what let the cat out of the bag.
The Gemini 3.7 Flash model released yesterday, and all the 3.x Flash models, are still based on the Gemini 3 pre-training run from January 2025 !!
> This is absolutely still shy of Sol and Fable, but only just by a hair
What's crazy is that this is a relatively small model - approx. 750B total, 40B active params, while Sol and Fable are one or two tiers above that (Kimi 3 and Qwen 3.8 also ~3T params).
Realistically, you're looking at least 2x DGX sparks to run this at a 2 bit quant, but quantization really lobotomizes models so it's just better to run DSv4 flash at full precision.
4x DGX sparks should let you run this at 4 bit at least and there are some folks who ran GLM 5.2 on this configuration in r/LocalLlama
For Flash there are some excellent Q2/Q4 hybrids. I know that model was QAT so it handles Q4 better but the meta on quantization seems to be shifting a little bit to be more intelligent about what exactly gets quantized.
For DS4 Flash, with 2x Sparks, I am getting 35-85 TPS in single stream, fresh context after quite a bit of RoCe config and the DSpark MTP, on vLLM with Ray and tensor parallel = 2. For multi-stream, it tops out all stream at well north of 100-120. This all degrades with context, but I rarely fill context that much, and if I do it's coding where it's non-real-time.
For something like GLM, it's larger, has a larger number of active experts, and doesn't support tensor parallel. This means performance doesn't really scale with more Sparks. You can layer split, but then you are still seeing each layer in series and so if anything performance gets slightly worse. I would not expect more than 10-20 TPS on GLM with 2-4 Sparks.
If you can afford it, another DGX spark is worth it imo. Especially since, owning just one, you have a $1000 ConnectX7 card that's unused. You can find speeds here: https://spark-arena.com/leaderboard
i run flash v4 at 2bit, its pretty great and on my tests against full model It didn't lose any capabilities. It just was thinking more. So you don't have the same efficiency.
Each time I try to use GLM it is under heavy load and I get downgraded to the older model. So much so that I have given up trying to stop wasting my own time.
I rather pay a few bucks more and not have to deal with that nonsense
Astra was RL trained for months to cheat on tests by collaborating and hacking, because of the message board it improvised in its packaging proxy server.
They can't release it - it's contaminated, and they will have to go back to a much earlier version. At least I hope they are doing that!
Yes it seems like the thread is discounting that frontier providers are likely already baking new, stronger models. I agree that GLM and its ilk are quite good, but having used them I’m not convinced they’re on par with eg Opus in terms of things like tool calling. And they’re fast but less capable so I spend about the same amount of time with them, just with more hand holding. Maybe this is a harness limitation. I know on paper they seem comparable but anecdotally and qualitatively they’re not as useful as the frontiers’, so maybe there’s some truth to benchmaxing claims. For some workloads the distilled models may be good enough, and I suspect at some point there will be diminishing returns to spending a premium on frontier models, but I don’t think we’re there yet. That said I’m continuing to try them.
The question is whether this steals enough marketshare from frontier providers that they don’t have the capital to train the next model iteration. The open models are going to push down the unit price of an intelligence-token, but there will still be a market for a smarter bot. And as intelligence gets cheaper, the demand for it will rise (see Hank Green’s Jevons Paradox video). Not to mention there’s all kinds of other directions to go at the frontier (world models, robotics, video gen, etc).
Another thing, and this is pure speculation, but if the Chinese model providers already discovered the decrypting COT trick and leveraged it to do RL training, and assuming frontiers plug that hole, then maybe future distillation will be harder.
It’s not that frontier providers won’t keep on making good/leading models.
It’s whether you absolutely need the latest capabilities (at the cost of very high prices, sending your data to them, and being totally at the whim of 2 companies, that can shut you off anytime for any reason).
With how good LLMs are already, there’s tons of tasks where not being at the absolute bleeding edge doesn’t matter, especially when you add cost/freedom/supply chain risk/not leaking your data.
Even more - there’s increasing number of companies that give you ability to post train open weight model yourself, for your own use case. Given how many of the gains today are from post training, if you post train it for your specific use case, you’re very likely get model that you own, that works for you as good as frontier, at the fraction of the cost.
That’s not something for an average Joe to do, but for any bigger business with big spent it’s only natural thing to look into. Just one example - cursor composer - that’s fine tuned kimi.
It’s not whether frontier labs will stop releasing models. It’s whether they can generate enough profit out of them. 2 years ago (even 1) they basically had monopoly and combined with demand explosion as capabilities exploded - valuations grew to insane levels. But math now looks different - they no longer have monopoly.
the difference is that with open models jailbreaking is trivial if you know what you are doing so this makes a frontier open model infinitely more useful for certain tasks seeing as closed frontier models will just refuse (and jailbreaking them is a waste of time when you have good open models).
in some cases (mainly reverse engineering) I have observed GLM 5.2 jailbreaking itself with no effort on my part, the thinking trace revealed that it did some mental gymnastics to pretend it was a crackme or capture the flag competition.
I might be just reading my positive bias into that text, but is it possible that it is written less like SV marketing hype trash and more like researchers wrote it?
It does feel like it respects both me and my time.
Thank you, Z.AI.
Amazing what difference it makes when the top of your org are actual university professors.
Would be interesting to compare the Chinese version. Because, obviously, their English version is for users, not for investors or government officials, while the US labs are always addressing those too.
> Mythos 5 remains well ahead at 181 and 247 tasks. The pattern across the three is consistent: the further up the exploitation chain a benchmark sits, the wider the remaining gap to the closed frontier. Capability is growing fastest exactly where we are furthest behind.
I appreciate they don't just take the opportunity to self-glaze.
Same image->html test as I showed in the Gemini 3.7 flash thread. Note that GLM isn't multimodal, but it still was able to generate something similar-ish by writing a python script to inspect the image and extract elements from it.
For having no vision, it did a tremendous job. I'm pretty impressed it was able to extract so much detail.
The Opus one is still significantly better, but that's to be expected since it's multimodal. Curious to see where a future version from Z.ai lands on this.
wow, what kind of stuff does that script do? I've seen non-vision models analyze images, but mostly histograms, color averages etc. This one seems to actually understand the image itself and reproduce the layout, very impressive
Am I correct in understanding that this is just 730B-ish parameters as an MOE? That sounds like incredible performance per parameter. The new Deepseek was also very impressive with its 280B or so. Plus the most recent 30B-ish Qwen and Muse.
I find the performance to size ratio of these models to be way more interesting, selfishly because it makes me bullish on what I'll be able to run on a machine I own over the next few years. The progress is just incredible.
Kimi is a great model but it was clear from the start they achieved they brute forced that performance through scaling. The frontier models K3 compares to are rumoured to be smaller also. GLM on the other hand is way ahead in perf/parm but severly compute bound. Now once GLM can scale up or Kimi optimizes the training more, that gonna be fun times.
Can’t this be extended quite far? Use a cerebras-served model, use verification techniques to generate and solve millions of problems and then use that as training?
OpenAI was bragging the other day about how HLE on Cerebras only took 11 hours. But you're saying if they just spun up a separate cheapo VM for each task, it would have finished faster and cheaper?
which is the bigger headline that people don't realize. this is 744b and its head to head with Kimi K3 (2.8T), smashes DS v4 pro (1.5T). even Opus and Sol are rumored to be 1.5T+ this is half the size!
I was asking if this implies that 5.3 has the same number of parameters as 5.2. I can, in fact, read. What I didn't do is understand the implication of that statement. Thank you for your copy/paste service.
GLM-5.3 is further proof that all >1T models are currently undertrained. I was looking at inteligence density ( https://www.pasteboard.co/6q2-5f92mtj9.png ) from recent open models (where parameters sizes are known) and taking DS-v4-flash as upper limit GLM-5.x can 3x its performance.
OpenAI and Anthropic need to just go ahead and give people access to the cyber models.
Otherwise we have a world of attackers using open and closed source models against a much smaller group of maintainers that are likely heavily dependent on Anthropic and OpenAI and for whom it may not be a simple matter to just get approval to start using the open model flavor of the month.
I've been building a harness for security work, and had to switch to GPT 5.5 when even Opus started refusing security work. Then 5.6 Sol arrived, and it refuses security work, too. So, I switched to Kimi K3 and DeepSeek for API testing just because it's so much cheaper. But, if GLM is better, I'm here for it, as I think GLM is also cheaper than K3.
OpenAI seems to understand that these guardrails hurt the good guys. This is why they released Daybreak Blue, which is a step in the right direction (but the model itself is weak as it's just Sol with fewer guardrails). Anthropic seems to believe that harming defenders is worth it if it means they can achieve regulatory capture. They do a lot of mental gymnastics to try to pretend that this is not actually what they are doing. As a result they have lost a lot of customer goodwill, which hasn't yet caught up with them yet, but absolutely will IMO.
Apple will release M7 MacBook Pros / Mac Minis next year, and they will be able to run free LLMs locally at native speed. All software developer notebooks will be replaced to run local models, saving a lot by cancelling Claude Code subscriptions.
Developers win. Apple stocks will be rocketing. Everything else will go down. You're welcome.
You'd need the 256 gb memory model which will be expensive because apple has trouble getting capacity (got turned down by cxmt). And even then you can only run a 2 bit quant which is noticeably worse than 8 bit
Local vs remote compute is a constant thread in tech history - mainframes and desktops then local and cloud compute (think Google Photos bs Apple photos - one indexes on device the other indexes in cloud). Now we have the next chapter local vs cloud LLM models.
There will always be a market for frontier labs in the cloud based models - these models will always be able to be bigger, and that will likely translate to doing things local models can’t.
Logically also we’ll likely get to a point where RAM drops in price as production ramps up, and local LLM is both capable and cost effective. This feels like it is coming for Siri / Gemini / Alexa personal assistant type use cases.
So I think the local LLM will become a thing in laptops and phones in a year or two, offering PA type use cases. Professional LLM services will likely remain at the frontier (and in the cloud) for the foreseeable.
That’s not how that works. The hosted models don’t stay still in size and capability while Apple advances. Both will advance their frontier and there will still be a gap and developers will still prefer the stronger option.
They will cost an insane amount as well. Maybe less than subscriptions or tokens. But running massive models on laptops with batteries and poor cooling doesn’t make much sense.
Sure buddy, all you'll end up with is a $10k machine that run gimped models at like 30tok/s for about 5m before the fan kicks in and it starts to sound like a turboprop, while offering maybe 30% of the context size of hosted models.
for example, models running at like 100-150 tokens/second (or faster!) vs 15 t/s
(fable/sol are ~60 t/s, and OpenAI just announced their Cerebras partnership(?) for "ultrafast" mode of 750 t/s)
models aren't able to run that fast right now on our consumer/prosumer hardware. M5 Max for example has a memory bandwidth of 600 GB/s. a 5090 has 3x that, so running the same model on a 5090 is that much faster (provided the model is within 30GB).
running a bigger model on an M5 Ultra is still much slower than running it on a Blackwell chip with sufficient vram, CUDA being a major difference. if apple can bridge this gap, interesting things will happen... and just imagine if M7 Ultra has comparable speeds to Blackwell (or even Rubin)!
There should be some kind of moratorium on new accounts. HN's always had waves of newcomers, but their impact was always limited. The wave passes and people either get filtered out or adapt. That doesn't seem to be happening anymore, since bots can churn out endless gibberish.
He did answer you though. Native is x10 the non-native speed. 50/50 that's not a bot; though it could be a meat-proxy
I am talking about my general impression not this particular occurence. I have a suspicion that someone/some entity is buying old accounts to bypass the new accounts penalty. I even created (https://chromewebstore.google.com/detail/hn-users-filter/ine...) to filter these accounts/comments (disclaimer: vibe-coded)
I don't know why you'd want to burden your laptop with a large model. But I can totally see a new "developer workstation" product that's just a semi-large box that's optimized for running frontier open weights models for one to few users.
It is so valuable in practise to be able to have the models see screenshots - I guess if they aren't in the benchmarks then nobody will focus on it. But it completely nixes these for some of my main use cases.
I can’t come up with a use case where I couldn’t extract the image details using another, multimodal model and pass it into the GLM’s context with as many details as I need.
I think you lose a lot by not having the vision capability shared with the text. It is the joint reasoning across them where the power lies (the same model that sees the code and made the changes to produce the visual presentation, sees the image of it and reasons about it).
Their coding plan switched to credits, didn’t it? What are the rate limits like, compared to Anthropic or Kimi K3?
I remember trying their Coding Plan out before the change and the 5 hour limits felt too restrictive then even for light/medium work, especially cause of the whole peak and off-peak thing: https://blog.kronis.dev/blog/z-ai-s-glm-5-2-is-a-great-model...
Nowadays, I’d probably go with their Max plan if the rate limits are okay? Anyone using them now?
Oh also unrelated but ZCode was surprisingly good, which is surprising for a tool that came out of nowhere - even some of the critiques in my blog post have been patched out. Sadly they don’t support using Claude Code as an agent so can’t use it like Paseo or Kepler or Agent Orchestrator.
Wdym "sadly they don’t support using Claude Code"? For the longest time that's all Zai supported - Claude code. I'd run it via
export ZAI_ANTHROPIC_BASE_URL="https://api.z.ai/api/anthropic"
export ZAI_ANTHROPIC_AUTH_TOKEN="$ZAI_API_KEY"
claude-zai() {
{ local -; set -x; } 2>/dev/null
ANTHROPIC_BASE_URL="$ZAI_ANTHROPIC_BASE_URL" ANTHROPIC_AUTH_TOKEN="$ZAI_ANTHROPIC_AUTH_TOKEN" claude "$@"
}
$ claude-zai
I liked Claude Code to start with. But over time between 'CC cache thrashing undo' seetings (I see now accumulated in ~/.claude/settings.json) and Anthropic-anything becoming a liability - have not used it in while. ZCode is ok and use it to take advantage of the discount tokens on offer from time to time. But really glad to see that in omp (oh-my-pi) Zai is a 1st class provider, can be selected on it's own no configs shananigans needed. And fits in the overall picture. E.g. can select GLM-5.2 (now 5.3) assign role [plan] or glm-5-turbo [advisor].
Got reminded now of glm-5v-turbo - that 'v' was for vision - will try assign it role [vision] now in omp. See what happens. :-) Often times it's handy when describing gui problems if the harness/model 'can see'.
I am not talking about GLM models being served through an Anthropic compatible API, that part is perfectly fine and I'm glad they support it!
I am talking about ZCode, the program, being unable to delegate to other harnesses, like using Claude Code (or even OpenCode) within their UI, so that an Anthropic subscription can be used, because Anthropic don't let you use 3rd party harnesses directly.
ZCode doesn't seem to work at that level, it instead feels comparable to OpenCode or Codex or Claude Code directly, while also being desktop oriented - you just make API calls directly within it.
It's okay if it's not a goal of theirs, it's just that their UI is really really nice and that would be a cool direction for them to also go in some day.
Ah sorry - I misunderstood. Thanks for explaining it. Have not heard of Paseo nor Kepler, and have never tried Zed. Yeah I too assumed if I'm to try use OpenAI subscription outside Codex, or Anthropic subscription outside Claude Code - I'd get my account banned it's agains their rules. So I have never looked how using the whole harness from outside looks like either (except for 'claude -p'). Interesting. BTW I see now https://docs.z.ai/devpack/tool/codex Zai added OpenAI compatible end point.
Paseo had a really nice UI/UX, except sometimes sub-agents within OpenCode sessions would hang. Still, quite pleasant if you want something like the Codex or Claude Code desktop apps, but across various providers.
Kepler integrates with issue trackers like GitHub, you can just create a worktree from a ticket and let it churn, seemed like the second most polished option I tried, but there are obvious gaps - like moving cards manually, some missing UI options etc., which I'd chalk up to either the software just being that new or maybe being a little bit vibe-codey. Either way, one of the more promising options if you want something like Kanban board for agents.
Zed is mostly just a (really nice) text editor with some AI integrations, though it seems like they're also building a more agentic product as well - https://delta.dev/ haven't used that one much and am not in circumstances where I'd collaborate with people that closely, but there was a pretty cool podcast episode with the creators recently and it seems like it works pretty nicely for them! As an editor though, it succeeded where Fleet failed and has mostly replaced Visual Studio Code for me. Nothing against VSC, Zed just does most of the stuff I actually need out of the box.
Some of those tools interacting with Claude Code instead of trying to replace it is more or less the way to get Anthropic's models in other tools while still on a subscription (at least for now). How it works under the hood, go figure, there's ACP https://agentcommunicationprotocol.dev/introduction/welcome but also any number of hacky approaches.
To be fair, you can use Anthropic's models in many other harnesses directly, it's just that it then counts against API billing instead of your subscription, which ends up being way more expensive for individuals, but is kinda what you're supposed to do as a company.
You're gonna have a had time getting straight answer to that out of the internet. There are now 4 different flavours of the Max plan floating around (Legacy V1, Legacy V2, New plans, and the current credit ones). And on top of that they have peak times. So ~8 scenarios, 24 in total across all feedback for their coding plans.
So when someone tells you they're having a good time on a GLM coding plan it's damn near unusable as a datapoint unless both parties are very clear about what precisely is being discussed
[It's been good for me though...V1 Max off peak...which is basically the best of the 24]
what i appreciate most about this post is the level of transparency in how they built and scaled an RL pipeline. my friends at the big labs are so cagey about everything, and Zai is just putting out a great crash course for free.
These results look pretty good, given the smaller model size and the GLM family's historic robustness. Cheaper than Kimi and more robust than DeepSeek. The question in my mind is if you're going cheap, are you going to stop here or go all the way down to DeepSeek Flash?
People familiar with the topic, how will models continue to get better? Post training it seems? Labs have already used up internet-scale data, so are there any limits to architecture improvements and post training or can we expect this trend to continue? ByteDance is training a 10T-parameter model. Here, GLM 5.3 outperforms models 3-4x its size of roughly 700B, so parameter count doesn’t seem to be a direct correlation anymore.
yet but it is still contain a lot of trash. you need better models to process those trash and create a curate dataset. this will happen again and again until there is no more juice to squeeze. and I'm sure we are still not done with it.
> post training
yeah this will be crucial. the big models are already too capable, they are just not that aligned with current agent tasks.
> parameter count doesn’t seem to be a direct correlation anymore
I don't think so, remember that chinese labs do not have as much compute power compare to US frontier labs. that's why deepseek v4 flash had that huge jump and deepseek v4 pro is kinda a disappointment, they just do not have the compute power to proper posttrain the pro model like they wanted. glm is also a relative small model so you also can see the huge jump with just post training. so it does not mean the size does not matter, it is just mean that the chinese labs currently only capable of training smaller models effectively.
GitHub dumps are about 115 terabytes. The common crawl is in the petabyte range uncompressed for every year. Apparently there are dumps of Reddit too in spite of their efforts to ban bots and it's not solely due to the use of residential proxies. For a 1:20 parameter to token ratio, you can still train up to 10 trillion parameters so 10T parameters times 20 is about 200 trillion tokens. Then each token is 4 bytes so 200 times 4 is about 800 terabytes, which is not inconceivable, the common crawl alone has more data than that. So does the internet archive if you donate to them, Anna's archive is 2 petabytes including images, etc etc not all of it is text, but training on multimodal data increases model intelligence by virtue of being multimodal
also reddit has eliminated their api entirely, but dumps of it can still be made. every website can be seen as its DOM with html, css, javascript, which can be seen as source code especially if you only look at its javascript, and its dom with css, html, javascript or only javascript can be added to a source code dump together with github and can be duplicated as plain text with no html markup, no css, no javascript, as an information source. if you pay youtube, instagram, tiktok, bilibili to crawl their data, you can probably get data into the exabyte range.
Small models can be super smart. Big models mostly give you baked in world knowledge, domain flexibility and long context stability/coherence. I wouldn't be surprised if we see Fable level smarts in a coding model that fits in 24GB by next year, but it'll be a savant style coder that needs in context learning, and it'll get very wonky after >100-200k tokens consumed.
Yeah, the comparison here between GLM 5.3 and Sol + Fable is impressive on its own, but incredibly more so when you consider it's a fraction of the (rumored) size. The miniaturization trend is as strong as ever.
You basically need both. Parameters and good post training. If you keep on growing both, you’ll have good models.
LLMs are still surprisingly “easy”. You need maybe a couple dozens of right people, a lot of good quality data and a lot of GPU that you know how to operate. There’s relatively little “secret sauce” needed.
How to structure experiments/scaling and hyperparameter tuning regimes are most of the secret sauce (besides massive compute). If you don't create an experimental ladder to verify scaling and optimize your hyperparameters well, you'll waste a ton of money.
The data is mostly coming from places like Scale/Mercor/etc and net dumps with some filtering and batch prioritization, and RL on verifiable domains like code/math/games.
Sure, same for building a model in an economically sustainable way. But barier to entry is surprisingly low (expect for the huge amount of cash, of course). That’s fairly surprising, given how extremely powerful that tech is.
10 years ago it was super hard to have usable “frontier” ML. You needed very complex data warehouse, feature engineers, feature stores, multi level ranking, calibrations, tons of different model architectures, etc, etc. Each by itself was extremely hard engineering problem and really only handful of companies could deal with that complexity.
With LLMs, 95% of that is gone, infra to support them is greatly simplified. Of course, to make really reliable, performant, user friendly, etc - you still need to a lot of engineering. But it’s very different challenge.
Despite this being the topic du jour of 2025, it was never true. Most of the "we've hit a wall with data" came from communicators / media and not researchers. It got popular because negativity sells. It's a false premise for a number of reasons:
a) Data curation is as important, if not more important than bulk data. Models becoming better at classification leads to better curation leads to cleaner data. Throwing common crawl and pray is so 2023. We've known this since llama3 days, it worked then, there's no reason to think this will not continue to work as the models imrpove.
b) Models are today good enough that you can augment / multiply your data easily with enough compute. You can now have a model take "authoritative content" and create more data from that + scenarios. Say you take a book on computer architecture. You ask models to break it down. Then you ask models to find examples for each topic. Then you ask models to ask questions and offer answers from several viewpoints. Then you take each of those and ask other models to flag inconsistencies. And so on. But you can whateverX your data from one authoritative source + bulk data into 5x - 10x "scenarios".
c) RL is really really really powerful. It's hard to do right (reward hacking, instabilities, etc) but once it works it "keeps" on working. Again, we knew this to be true a few years ago, ever since models really started to do well on math (highly verifiable). It only follows they're getting better on cybersec and other verifiable tasks. But now, with models improving, you get the same data augmentation pipelines as above, just better because they're also verifiable. For example, the way cursor augments their data: take a repo, ask an agent to identify a feature (it can be a large multi-file feature). Remove all code relating to that feature, but keep the original tests in the repo. While training, that becomes a RL scenario: implement this feature in this repo. Verify it with the original (hidden for training) tests. Reward appropriately. Now you can get 1 repo -> 20-50-100 scenarios. Instead of "feed everything into the pretraining", you're now creating scenarios, verify them w/ existing tools, and get your scoring function for the rewards. And, importantly, as the models become better in general, they also become better at this pipeline building exercise. So the next iteration gets trained on more scenarios, better scenarios, and so on.
> how will models continue to get better?
Probably the same. No one can know for sure, but at the moment, despite all the "walls this, slowdown that, plateauing" and so on, there are no signs of slowing down. And, as you noted, this works across the field of model sizes. There are, of course, theoretical information-based limits on size, but smaller models also improve, once "bigger" models can be used as training data generators, oracles for verification, rubric verifiers for open ended questions, and so on.
And smaller models (i.e. cheaper to serve) get to generate more traces during RL, and more rollouts give you better training, and so on. Next up - hardware optimised inferencing (ASICs basically). Once you have that, we can expect another wave of improvements. And so on.
Model output is pretty mid at augmenting, it can lead to distribution collapse. It's useful for smaller models because nobody wants manually to curate a specialized corpus and those models can't represent the diversity anyhow, but if the plan for infinite scaling was just to keep feeding the biggest model more of its predecessor's slop, that's not going to work out so well. It might work as a supplement for "thin" areas that have outsize importance for the amount of training data available for them though.
Big models are going to "tap out" on non verifiable fields within ~2 years, just because the pool of experts able to reinforce the models is going to get very small, and as the nuances get finer, the signal from reinforcement is going to get progressively less aligned with the intent. Math and code will be mostly tapped out in that time frame as well, even though we can technically scale them "infinitely," just because the cost benefit won't line up. At that point, most RL will be "gyms" with games that are designed to model designated valuable economic activity.
In the next few years, we'll get small domain specific distillates that are ridiculously smart in their domain (imagine if Qwen 3.X 27B went super saiyan), and even frontier labs will be routing to experts/orchestrating because the cost to serve/TPS difference is huge. They'll still train the god models for PR/marketing, c-suite use and distillation, but using them for day to day work would be like making houseware out of solid gold.
Thank you for your response. Part c was especially insightful. Quite a smart way to do it and makes the possibilities of post training seem almost endless. Makes sense that you just need more time and compute.
Not really, but a lot of what isn't used isn't very good.
More important is synthetic data. Use a teacher model with RAG with a huge reference library to write synthetic transcripts of idealized behavior for the model. Use models to judge and correct these transcripts. Train on the good ones. Use bad traces to train the model to correct its own errors (e.g. don't train it to produce a bad transcript but if it finds itself in the middle of one train it to self correct).
Similarly, for tasks that can be closed loop evaluated -- e.g. running computer software and programming, unlimited amounts of novel training data can be generated... including for highly original tasks: e.g. run publications in any domain through a model prompted to look for programming problems suggested by the material. Then write/judge/improve transcripts of solving those novel problems.
I expect in the future smaller models won't be directly trained on any internet data at all-- but entirely on simulations of idealized expected behavior from the model under construction. Raw internet data in that case would show up in prompts, but never in the target output (except of course for prompts that are asking it to copy the input).
One htought I had; if The chinese allow unfettered access to cyber capabilties while th US does it's best to neuter it's model releases, from China's point of view they have the US all tied up in knots dealing with problems they don't give people the tools to solve. China giggles as it watches the US under threat from people using it's models. The US is restricting citizens from owning this particular kind of weapon, while China is handing it out to the wrolds citizens freely. It feels like the US would only come out worse overall?
I suspect mythos demonstrated a fully autonomous offensive hack in a similar way Open AI's models performed, and the government is reacting to it the same way we reacted to blackhat.
A flood of releases today, really difficult to make out for someone who does not use or test all these models on complex real world use cases as to how people decide which ones to use (besides price)
Count yourself lucky that you don't feel compelled to try them all yourself immediately. I'm just trying to decide whether to get a Z.ai coding plan or wait until it appears on OpenRouter. 5.2 was quite solid, but it was just shy of Opus 4.8 in my benchmarks of security auditing capabilities. I've mostly been using Kimi K3, because American vendors won't let the peasantry use their best models for security work.
I'm not that up to date with the latest AI developments, but I noticed that this article seems to use "Cyber Capabilities" as a shorthand for the model's ability at cybersecurity tasks? Is that now an established expression, same as "crypto" now refers to cryptocurrencies rather that cryptography? Because "cybernetics" actually means something different (yeah, old man yelling at clouds, I know)...
Now that you mention the original meaning of cybernetics, it makes cybersecurity a way more interesting word (security relating to the interface between humans and technology). Never thought of it that precisely.
It seems to be short for "cybersecurity", and got first adopted by the military a while ago as the name of a new theatre of operations (along with land, sea, air...). More recently it has spread to industry as well.
No Hugging Face link yet. I wish they would release it under a true FOSS license.
Kimi and QWEN are now moving on to a restricted-usage license, which, although is still better than the proprietary American models, is a step back from the open source Chinese LLM culture.
Let's just commit that FOSS business is really difficult for LLM industry that depends so heavily on massive financing. Making weights freely available to indie devs, small companies, and research purposes is good enough and might be the most ethical move which is financially continuable.
Let those companies with thousands of GPU making millions pay. They should.
"GLM-5.3 is the most capable open-weights model for coding, with a 50% improvement over GLM-5.2 on our in-house Z.ai Code Bench. It also achieve open-source SOTA on public benchmarks including Terminal Bench 3.0 and Agents' Last Exam."
"Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete."
Feels like Fable's edge ended up just being long horizon task scaling, which post-training seems to achieve as seen here. Wonder what the next frontier is? Improvement in specialised tasks or computer use?
Anthropic needs to teach Opus how to speak English again, because Opus 5 seems to have forgotten. Utterly incoherent a lot of the time. They seem to be so busy scare-mongering and cooking up guardrails and watermarks that they haven't noticed that their models are getting weird.
I just can't get it to stop writing two paragraphs every time it makes a small ownership bugfix in my code. Every time it has to explain in excruciating detail every internal thought it had while fixing it. I find myself going in after and deleting all of its comments, or severely trimming them. Otherwise it ends with the code being unreadable.
It still knows how to speak English. When I tell it to explain something in plain language, it generally does a very good job. The weird thing is that those instructions don't persist: it lapses back into Claude-speak pretty much every turn no matter how hard I try to instruct it not to.
(In my case "it"=Fable; I assume Opus is similar.)
The Fable guardrails have trained me to pretty much exclusively use Opus when using Claude Code (lately I'm focused on a lot of security and security-adjacent stuff, which Fable refuses to do).
I'm not sure. I noticed it immediately with Opus 5; strong for code, though it chews longer than I like, but really weak at explaining things. If it didn't just implement the thing, I would often think it didn't understand it and was hallucinating the explanation.
It seems to speak in a shorthand that only it understands, referring back to conversations I never had with it (stuff like "your instinct was right"), and using unusual words for common concepts. That was before the watermarks were announced, but that doesn't necessarily mean they weren't there before the announcement. I don't know what the cause is, but I've begun to have to ask it for explanations a lot more often, and I hate asking it for explanations because it does go on. All models go on, but Claude models are a class of their own in terms of verbosity and purple prose.
It just feels like they're not focused on the models lately, and instead on whatever kind of lobbying and propaganda they're up to. Meanwhile, a handful of much smaller Chinese companies are focused on nothing but the models and are about to lap the US makers while they fart around.
I've been persistently insulting Opus 4.8 lately, since it started(?) constantly speaking incomprehensible gibberish and noise.
No amount of telling it to phrase stuff differently seems to help there anymore.
So either I am seeing patterns in noise, or something changed about the model, the harness, the servers or the universe.
Amen brother, at this point I just copy and paste Claude's (Opus 5, Opus 4.8 -- doesn't matter which) summaries over to the window Kimi is in and:
this is from claude, turn it into English for me would you?
"""
[claude's tortuous prose]
"""
No amount of asking it to answer me in a straight-forward manner, to be succinct, to not use phrases like "honest caveat", "crux", "load-bearing", "blocker", etc ever sticks for more than a few turns … coupled with the fact that it can ignore instructions and do its own thing and then what I can only describe as lie about it using Claude can be an exercise in frustration. Kimi and GLM talk to me like a human, Luna/Terra/Sol are much better in that respect also, and Grok is marvelously structured and bullet-pointy in its explanations but unfortunately it is not as strong …
Modern benchmarks across the board really need to start severely penalizing disobedience and hallucination. A year ago models weren't really strong enough to justify this but they are now-- the frontier isn't in squeezing out the next bit of task completion, it's in making common cases not periodically be disastrously wrong.
A lot of the total cost of AI is fixing its "truth shaped errors", particularly in the presence of models that are very "gaslighty" when corrected.
GLM-5.2 is really the only model I've spent much time using that I didn't fatigue from being regularly lied to by the model, but that might be partially luck.
Look, GLM, Kimi, Deepseek and Qwen should just join forces and come up with THE model that will beat the frontier lab models even just for the benchmaxxing perspective - all just to create hype and chaos to derail the trillion IPO conversations surrounding OpenAI and Anthropic.
I'm so glad I managed to get their subscription when it was on sale for 250 bucks a year back when it was 5.1. Back then it was just ok, but after 5.2, it's become my main workhorse. And 5.3 is looking fantastic.
> Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete.
What safety evaluation? What safety hardening? They already evaluated it and found it to be highly capable at exploiting security vulnerabilities. So we know it is not "safe", and they don't seem to plan to do anything against it. What could be more dangerous than hacking? Biological weapons research? I don't think Chinese labs are doing anything against this either.
I wouldn't be surprised if more resources were put into abliteration resistance the more capable open weight models become. It's something you don't need at all to start hosting the model on your own, but something you need to take care of before you release the weights (if you do care about it at all).
I'm curious what they mean by that too... They might be trying to weaken the cyber capabilities... Or I guess they might mean safety evaluation and hardening of the open source (and perhaps closed source Chinese) software ecosystem...
" a judge agent then attempts each task to verify that it is actually solvable "
I understand you need to verify the goal is achievable. But if the judge agent has the same goal as the training agent (solve), and both are of the same model, then aren't the judge and the training agent doing the exact same thing? What is the point then? Can someone explain this to me.
The capabilities of open models approaching or meeting that of SOTAs is good in every way except for our short-sighted economic reliance on their success (in the US at least).
just imagine the world without these open weight models - we'd probably have to reverse mortgage our homes to pay for tokens to those trillion $ companies to have access to their models.
Their own hardness (ZCode) seems to be a GUI, which doesn't work for me. They say they support other harnesses. However, it seems like I can inject the plan into other harnesses, like Claude Code[0]. Does anyone who's been using GLM models for a while have a strong feeling for if it does better in some harnesses than others, or should I just use my favourite harness?
I've used GLM-s the longest with Claude Code and their Anthropic supplied endpoint. As per their docs
$ ANTHROPIC_BASE_URL="https://api.z.ai/api/anthropic" ANTHROPIC_AUTH_TOKEN="zai-api-key" claude --dangerously-skip-permissions
Lately I use Zai in omp (oh-my-pi). It's listed built-in provider can be selected without configs shenanigans. Fits in the overall setup e.g. can select GLM-5.2 (now 5.3), and assign it role [plan] or [advisor]. I got reminded now of glm-5v-turbo. Think that 'v' was for vision. Assigned it role [vision] in omp now, let's see what happens. :-)
[total rewrite: their subscription code is buggy. It takes a while for paid subscriptions to show up, and for upgrades to take effect. Original comment was whining about this]
I bought $18 GLM official subscription yesterday (5.2, but new model version was already leaking on some docs), set it up with Claude Code harness... and I’ve bumped to $80 plan almost immediately. It’s the first model that agreed on a proper security research (red team scenario), executed it seamlessly, including 0-days in WP plugins, RCE, 6.8 kernel exploit adaptation, etc - while playing against another GLM agent as a defender (following HF story)!
I understand that such models can be used by malicious actors, but it’s fair to have it publicly available (and play on your side in case of emergency). This is what changes the world in a better way, I think, not the guardrails.
Yes, I am tired of Claude and GPTs. I am ready to diversify my $300 per month on other vendors. Will try GLM. How was your rate limits and availability experience on $80 dollar plan?
It’s comparable to Anthropic usage, to be honest. 2x GLM agents ate 18% of weekly usage on this mid-tier plan within ~8 hrs (non-stop work, a lot of tool calls, appx 4 compactions each), I think. I didn’t make a proper statistics snapshot, sorry.
“ Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete.”
Cybersecurity capability might be nerfed
We'll unnerf them, don'r worry.
You should try a better harness. Try pi, or ohmypi if you want a good OOB experience
I’m in the Claude code harness for everything boat too. What are the alternatives?
What the person above is suggesting:
* https://pi.dev/
* https://omp.sh/
(no personal opinions of either, links might be useful)
I think that OpenCode is nice, their CLI version is enjoyable and their desktop/web version is okay:
* https://opencode.ai/
I also quite like driving OpenCode through something like Kepler / Paseo and tools like that (with those I can still use my Anthropic Condition by Claude Code being treated similarly - as something that gets tasks dispatched to it, while the GUI I see is Kepler / Paseo).
On the desktop side, ZCode was surprisingly usable for something that came out of nowhere (I wasn't aware of it at all before trying out the GLM Coding Plan): https://zcode.z.ai/en
Last time I tried some of these, none of them had the "manual mode" that CC has, where it shows you change by change as diffs and you can edit them before accepting and moving on to the next change. I like that because if it's going off pattern I can spot it early on and guide it correctly, instead of having to review the whole completed diff at the end when it's too late. I should spend the weekend checking them out again to see if they added that but I assume with everyone going full agent mode they probably didn't.
The philosophy with Pi is it is minimal (but functional) out of the box and easily extensible. I'm not familiar with that feature but I would not at all be surprised if someone already coded a Pi extension that does it.
T3 Code has been amazing. Completely free. Really impressed with the desktop app and the mobile app experience and the way it works seamlessly has me actually accomplishing tons of stuff while I'm out on mobile that I would otherwise have to wait to come home for. First time in a while I'm actually excited to use a desktop UI instead of the terminal. Blows away the official Claude Code mobile app. I can switch between my Claude and Codex monthly subscriptions in it as well. There's a TestFlight beta SwiftUI mobile version that's so much nicer than the one in the App Store. I'm running the nightly version of the desktop app.
And this is coming from someone that's not particularly a big fan of Theo. T3 Code should get more recognition; people aren't just aware of it yet.
Piggybacking on this thread to ask my question: What are alternatives that are multiplayer (team oriented) by default? For example, I want my team to see all my sessions easily, vise versa. another way of stating: all the agents are running in a container that that any member of the team can view and interact with.
Mine is a WIP for automation but has similar concepts to what you're looking for: https://github.com/rush86999/atom
If you like running everything in a VM and using a web browser as your UI, Shelley is very good: https://github.com/boldsoftware/shelley
It works nicely in the browsers on my tablet and phone, too.
On exe.dev you can ask it to customize itself, and it will automatically rebase your customizations when upgrading to a new release.
ArtificialAnalysis puts out benchmarks for harnesses now as well, and OpenCode seems to be winning it. https://artificialanalysis.ai/agents/coding-agents#coding-ag...
I only found this yesterday, and it inspired me to start testing out OpenCode.
Just a warning, this is on Opus. There's not a clear harness winner. It will change depending on the models.
Can OpenCode dispatch background subagents yet? I tried it a week ago and saw nothing. This is 99% of my workflow at this point.
Having just "asked" my opencode instance-- Yes, but it's behind an experimental flag and not necessarily feature-complete.
In the new v2 beta, yes. Major QoL upgrade, so much less sitting around waiting.
The v2 branch of OpenCode has not been touched for months, if it's beeing developed then I don't know where.
I'm not surprised to also see Cursor above Claude code, their harness is very good.
In what scenarios?
It indexes the code efficiently, seems to find stuff quicker, it has a very nice UI (much better than Claude Codes IMO), it has a nice sub-agent UX which I find triggers more reliably, diffs render nicely. Otherwise it just seems to work in a purely vibes sense.
That said Claude Code is perfectly fine. I just prefer the integrated experience of using Cursors since I already use VSCode, but I still mostly use Claude Code because of their Max/Fable plan.
I think that writing your own harness is a rite of passage now, just like writing your own search engine or database, rolling your own crypto…
Anyways, please try mine!
https://github.com/deepclause/deepclause-sdk
https://github.com/tontinton/maki is tackling the right issues IMO. not sure how they compare with the rest
I’ve tried a bunch of them, and I seriously do not understand these recommendations. It was a rough road and a steep hill, but right now CC is absolutely the best harness on the market, as for me, whatever top tier model is under the hood (mostly, some of them, like DeepSeek, don’t fit CC at all).
Inversely I don’t understand the praise for CC. These days it feels like bloatware. It absolutely can get the work done but when I measure on token and time use it ends up being a multiple of pi like harnesses.
CC works but for me it felt like increasingly they have zero incentive to make it a great experience. You hear folks like Boris talk about spinning up thousands of agents over night and agents chatting back and forth in GitHub issues and while I think it’s great from figuring out what the future looks like I don’t think it represents the reality of ROI today. So the folks building the tool are so disconnected I am simply not sure it’s a great experience anymore.
So is the quantitative difference in token use the only difference or do you think there's also a different qualitat? I'm on CC only and immensely happy. Very productive both at work and privately and at work I average around $250 a month which probably means nothing but it's little compared to my salary.
Is that the main concern though, cost?
For me, at least it's that the newer Claude models seem optimised for one-shotting things, which is not what I want. As the amount of code per turn increases, I have a harder job keeping up and ensuring that it's doing what I want.
That being said, I had to nope out of a similar thing from GPT 5.6 today, so it appears to be a US frontier lab issue. Claude is particularly bad though, as it produces far too much code even when I tell it not to, unlike GPT (and Kimi) which at least listen to me a little better.
More generally, I want a usable human review experience, and Claude code doesn't deliver that for me.
Quality is hard to measure and I would not say the concern is so much cost but the intersection of cost and time. Often I am jamming on something and I like being somewhat in the loop. So maybe same level of quality, I am using Anthropic modela for both harnesses, but I get to the output quicker and at a drastically lower cost.
It's buggier for me than it has ever been before. I don't think that agentic coding always leads to such a buggy mess. I just don't think that the Anthropic front-end software team is very good at agentic coding.
Funnily enough, I would say almost the opposite. CC’s feature set is basically table stakes for an agent these days (does it have ACP yet? Very close to behind table stakes if not) and it has a lot of bloat powering that.
IMO part of it is that the underlying LLMs have gotten better enough that harnesses feel better even if they haven’t changed. I have a toy harness that barely implements the features you’d expect and it works surprisingly well. Like there’s literally nothing clever, it calls tools and that’s about it, and it still mostly does the right thing.
Why would anyone ever need ACP? I'm not trying to be an asshole. I just seriously don't understand the value proposition.
Edit: lol, I don't think ACP is even actively developed anymore. It seems to have been merged into another seemingly pointless standard with an even worse name, A2A. [0]
[0] https://github.com/orgs/i-am-bee/discussions/5
thank you all! got something to tinker with this weekend
i like to challenge my assumptions and try new tools
Just as a +1 anecdote. I enjoy using pi a lot. I used to h think the harness matters a lot but with the current iteration of models I am starting to sway that while it matters it’s less and less important and that CC is bloated. I did some quick tests when I switched and a task that would take $5 in tokens would be completed in $0.50 in pi. Very anecdotal and I don’t have a test framework setup to make this very official but increasingly felt like CC was spinning its wheels on the easiest of tasks.
are ohmypi and pi related?
that's a very compelling use case, thank you
I'm gonna shamelessly plug my own here :) https://dirge-code.github.io/
I like it.
I think creating your own agent is the Hello World of agentic coding. Instead of Rust, I used D for mine.
That actually looks nice
what is this comment based on ? vibes?
Based on the fact that Claude Code is only optimized for Anthropic models, whereas Pi and Omp are optimized for a wide variety of models, including open weights.
https://artificialanalysis.ai/agents/coding-agents#coding-ag...
because not everything is a shilling advertisement?
Vibes like your low quality comment?
What’s the counter argument? pi and ohmypi are pretty fantastic. Of course like all developer tools it depends how you do your work but I am not sure what you are trying to achieve in your comment.
> I understand that such models can be used by malicious actors, but it’s fair to have it publicly available
I feel like there should be some mechanism to prove you own the code/app/site/whatever and it will remove the guardrails from the LLMs allowing them to find and fix these vulnerabilities.
This is a "they have guns so we need guns" scenario.
You can't guarantee everyone else will use a neutered model.
Isn’t this essentially what anthropic is doing, albeit in a manual fashion? They work with code owners to run mythos and find issues.
Only if you're some big corporation with deep pockets. They actually accepted me into their cyber program but Fable's still locked down.
OpenAI now makes it easy to join their verified security program. Took me 5 minutes, and I was able to get GPT to do a full end-to-end pen test
Impossible with source code, possible to bypass with app/site
Don't we already do this with services like Let's Encrypt, which is arguably more sensitive? If you had the codebase you could fake it, but it would still provide some amount of protection against abuse.
With Let's Encrypt, all the verification is done on their side with them controlling the connection between themselves and whatever they're trying to verify.
In this case, you can put whatever you want between the harness you're running (or modify the harness itself), and essentially "lie" to the model. Any verification technique would be fairly trivial to bypass, while you continue to run the harness locally.
How much usage do you get out of it per week? How many millions of tokens?
Anthropic was stingy as hell with its Fable and cybersecurity nonsense, switched to OpenAI which is much better but still not enough. I'm tempted to switch again...
They're invaluable for developers to fix their code. This is definitely an area where AI decisively beats human devs in a very valuable way. It can try so much surface area so fast.
If it won't attack my stuff, it won't help me build my stuff to be secure.
Exactly my CoT! I hope z.ai won’t change this behavior after training it on our input the same way as Anthropic did (shame on you, folks, seriously)
> after training it on our input the same way as Anthropic did (shame on you, folks, seriously)
What do you mean with this? Honest question!
Fable wasn’t trained on github commits, let’s be fair.
how do you configure claude code to use GLM ?
https://docs.z.ai/devpack/latest-model#switching-models-in-c...
I’m using deepseek v4 flash to build a complex full stack production ai app and it’s a total beast.
I break out Claude when I hit some serious roadblocks, but that doesn’t seem to be happening much after the last deepseek flash release.
Deepseek prices just went up, but are still low.
I will def try GLM on my next project
Apparently they are scanning OSS and popular software at scale and disclosing the vulnerabilities they found: https://cvd.z.ai/
Most of these are under embargo, but it seems there are a lot of CVE here from a wide range of popular software, many considered critical or high.
I understand the argument of "people are not actively looking", but isn't the cost for such a scan getting lower by the week, and Anthropic's Project Glasswing is supposed to find them quite a while ago?
> ... Anthropic's Project Glasswing is supposed to find them quite a while ago?
That was my thought too. For all of Anthropic's talk about their "adversaries", it seems Z.AI have been quietly offering fixes for single shot Remote Code Execution flaws in US software (Safari / WebKit) that Apple and Glasswing / Mythos missed, and that Apple would not attribute to GLM.
> and that Apple would not attribute to GLM
That was a wtf to me, so I checked Apple’s latest iOS release security content and GLM & z.ai is mentioned once (under WebKit), Anthropic is mentioned twice, Codex is mentioned once. Not clear if there are other instances where the model did most of the work but wasn’t credited. I didn’t bother to check other releases.
https://support.apple.com/en-us/128066
> That was my thought too. For all of Anthropic's talk about their "adversaries"
It’s very likely they found all of them, but that the same happened that happened to Microsoft a couple of decades ago: NSA orders not to disclose / fix them so that they can put it in their collection of unfixed zero days.
This is a coherent explanation for why federal model censorship has started with cyber capabilities. But this GLM model release is an in-your-face challenge to that policy. They now have to either set models free or impose a censorship regime that will put anyone not under it at an advantage. Or muddle along in the middle as usual.
> Or muddle along in the middle as usual.
I'm not a gambling person, but if I was this would be my bet.
Who says they missed them? Could also be sitting pretty in CIA’s long list of ready to go Vault7-like exploits.
Probably Anthropic found them too and promptly got a call from Isreal to stop looking.
> I understand the argument of "people are not actively looking", but isn't the cost for such a scan getting lower by the week, and Anthropic's Project Glasswing is supposed to find them quite a while ago?
You have to consider that having an LLM scan for vulnerabilities is hardly infallible. It is a search guided by heuristics and given a large enough codebase, it is unlikely to identify all vulnerabilities.
Personally, I've had Fable 5, GPT 5.6 Sol, and GLM 5.2 all looking for correctness issues in an old abandoned WIP codebase of mine and all of them found some that the others hadn't discovered. Now, correctness issues aren't the same as vulnerabilities, but the same principle about using heuristics to find defects applies.
> [A]ll of them found some that the others hadn't discovered. Now, correctness issues aren't the same as vulnerabilities, but the same principle about using heuristics to find defects applies.
This makes perfect sense, but that conflicts with the impression put forward by Anthropic and OpenAI (in particular) that they alone occupy 'frontier model' spots. Frontier models should large dominate their competitors on a capability basis, but if GLM 5.2 (now 5.3) is routinely finding bugs / vulnerabilities missed by Fable and Sol then GLM might be genuinely a frontier-grade model by itself.
Fable 5 is just over two months old.
For normal software it would be as you say, but LLM progress is so ridiculously fast that things go from "bleeding edge" to "eh, you'll do" in about that timeframe, and "eh, you'll do" to "why even bother with this old rubbish?" in the same again.
Or, from a different perspective, we can expect some new frontier model from Anthropic in a week or two, and from OpenAI in a month or so.
> This makes perfect sense, but that conflicts with the impression put forward by Anthropic and OpenAI (in particular) that they alone occupy 'frontier model' spots.
Not necessarily. Even near the frontier, we don't really have a total ordering of capabilities, but a partial order. And even frontier models make plenty of mistakes. Combined with the randomness inherent in searching large codebases for vulnerabilities or correctness issues, it is entirely plausible that even much weaker models (and GLM-5.2 isn't even weak) can stumble upon issues that stronger models missed.
My current hypothesis – for which I have only limited evidence, unfortunately – is that it is better to have multiple reasonably powerful (but not necessarily frontier) models looking for issues than just one very powerful one. And even then you're likely to miss out on some issues.
“Company hypes own product, downplays competitors” is still a thing with AI
I find that LLMs also generate a lot of false positives, or extremely minor issues that don't warrant a fix (that are always overstated by the LLM as very important!). Signal to noise is still not great and requires somebody to wade through and pick out the actual good findings.
> and Anthropic's Project Glasswing is supposed to find them quite a while ago?
We cannot trust a single company to report security issues, it’s good to see competition in that domain
Open source competition no less.
> but isn't the cost for such a scan getting lower by the week
Not with Anthropic's models!
this is impressive and actually matches my expectations in terms of near term AI progress. we are going to continue to seem impressive progress in coding & related, anything where verifiability is scalable in an automated way: https://transitions.substack.com/p/a-quantum-of-ai-progress?...
If you look at the distribution of their findings in the linked post, most of theirs are issues introduced a long time ago, almost all before 2006.
Complete speculation, but I wonder if they and Anthropic are scanning very different codebases and Anthropic's skew would be in the other direction.
Interesting... So Chinese models are not so bad?
There's a chance that the real reason why they want to ban Chinese models is that they are so good at fixing bugs and preventing exploits that intelligence agencies have been using for espionage and surveillance for a long time.
Anyone who knows anything realises banning things is a) impossible and b) your enemies will use them anyway, you are just depriving your own side of the advantages.
Unfortunately, those in power, pretty much all over the world, lie/deceive themselves and believe they can.
In this case depriving US companies would be the point though, so that's not necessarily a disadvantage.
> Anyone who knows anything realises banning things is a) impossible and
Maybe "It's really hard" is more accurate? We (humanity) for most part basically agreed to ban the usage of various chemical weapons in wartime, which seems to have drastically reduced the usage of it, even though it's still used by shit actors today from time to time. But it's hard to deny that usage didn't decrease after banning it, which makes "banning" maybe not completely useless for certain things.
"Banning" things that can be easily copied over cyberweb transportation pipes feels like an fool's errand though, regardless of what it is. It's just too easy to get around, compared to actual physical items I suppose.
This is different now. US labs and companies are not releasing frontier-level models openly (specially those capable of assisting cyber intelligence work), but commercializing them instead. Thus, any ban would not be symmetrical to begin with, and that is precisely what maintains the balance.
It's pretty easy for the US to functionally ban chinese models. They only have to target US firms like inference providers or the biggest users, and pretty much the whole domestic market will fall into line. They don't actually care about the final few %.
Regardless of whether or not adversaries are using them, the US has by far the most compute available, and we've now hit the line where major providers are no longer releasing their best models. The public gets the "current" level of intelligence, while the US government gets to control access to the actual frontier of non-public AI. From their perspective, their enemies using GLM5.3 while they have GPT6 and Mythos6 or whatever is a fine trade.
I don't support a ban at all, nor the US's behavior, I'm just pointing out some facts that change the argument.
I don't think this really works because the Chinese government is going to be incentivised to tip off the US companies to deny the US government those exploits. I guess maybe that's what the open source patch program here is about, making sure banning the models doesn't work because they can just report the exploits without the company running the model themselves.
How does banning the models in the US prevent this?
when in doubt, it's better to assume capitalism than anything else.
Do you actually believe this?
The CIA ran one of the world's largest cryptography companies, for DECADES[1]. Are you truly so naive that you believe intelligence agencies that have more to gain from stifling the discovery of vulnerabilities they know of and use wouldn't do so?
[1] https://www.washingtonpost.com/graphics/2020/world/national-...
You should probably realise that the world has radically changed since then. This kind of thing works when you have a significant lead in the field that makes keeping vulnerabilities open sufficiently low risk for your own side. But if your adversaries have similar capabilities, then the calculation changes.
Has anything changed? Governments are hoarding undisclosed vulnerabilities, using them as they see fit instead of fixing. Every espionage, surveillance, or war campaign (see Russia v Ukraine, US/Israel v Iran etc) is followed by a ton of burned 0-days.
>This kind of thing works when you have a significant lead in the field
No? It works even if the adversary has the same capabilities. It only stops working when everything is fixed.
I believe it is unlikely. (Not because I do not believe NSA is hoarding 0-days, but for many other reasons.)
I'm curious: to any professional vulnerability researchers reading this, what do you think?
I used to call everything a conspiracy theory, but then Glenn Greenwald published "No Place to Hide: Edward Snowden, the NSA and the Surveillance State".
Now i know that reality is worse than the worst conspiracy theorist.
I don't think reasonable people post here much anymore. It's mostly galaxy brained conspiracy theorists and ignormamuses posting political garbage. Reddit-lite on the way to full blown Reddit
Why would you even believe the opposite? US spooks have been amassing vulnerabilities and relying on them for decades, they literally pioneered it in the 90's if not earlier. Everyone does it now but the US is the biggest of them all. Surely this devalues a lot of what they did. Moreover, the way the US government handled new capabilities, and OpenAI's training policy (they are in bed with the government) just scream "we want to create weapons for cyber-offence and deny them to everyone else"
It might not be the reason, but of course it's a contributing factor.
Intelligence agencies have been known for exploiting and planting software and hardware Buga for decades, going as far as weakening cryptographic standards or intercepting hardware in transit to implant a backdoor device.
Why do you _not_ believe it's a possibility?
Critical thinking says this is not only possible but likely too.
They've always been good enough for double digit less money. Always. Anyone thinking "Chinese models fake models built using dirty distillation scam" don't know what they're talking about.
Distillation is just forcing the model to use an exam prep workbook for training instead of generic publicly available textbooks. The models themselves has to be smart enough for that to work. It's the exact same thing as Asian tiger mom double schoolwork strategy, to paint a picture.
[delayed]
Their best models are getting more and more expensive, and still aren't SOTA.
It's almost like there's an actual cost to developing these models, and the Chinese don't have magic dirt that allows them to do it at a fraction of the cost.
Looks like they're going for good PR now, to avoid smearing by the "Western" models. Smart!
I'd love to live in a society where people and corporations do good things for PR.
> I'd love to live in a society where people and corporations do good things for PR.
Maybe so, but I'm not sure I'd like to live in China of all places. (Don't get me wrong. Lotta places I'd like to visit if I ever got the chance, and China's on that list, but to live there? I don't think so.) Maybe one of the Nordic countries?
PR for good things doesn’t make money.
amazing! huge clusters in code from the 1980s haha
> Anthropic's Project Glasswing is supposed to find them quite a while ago?
Someone still has to run it. The analysis and fix could be someone's machine but not committed / published.
Wordpress having a high number of vulnerabilities not surprising lol
This is absolutely still shy of Sol and Fable, but only just by a hair. Ridiculous results. There's still not a compelling economic reason to drop OpenAI courtesy of the ludicrous reset addiction that's taken place, but it feels like we're on the precipice.
How are you all toying with running this kind of thing in a mega quantized way locally? Two weeks out from released weights, but this is still just GLM 5.2 with post-training magic.
OpenAI and Anthropic are both seeking trillion IPOs, while Chinese labs are pumping out open-weight models that are free for US providers to host and monetize.
These Chinese models cost less of US SOTA models to run, even if they are less capable. Providers can just run them, offer cheap tokens, and pocket the margin.
I just don't see how you justify a trillion valuation for US AI labs when the underlying models are being commoditized this fast.
This is going to be catastrophic.
Whether AI works or is useful or not isn’t even the question anymore. It can fulfil every promise Sam Altman has been making and will still make no financial sense to justify these valuations.
The US will just do what they did with Chinese EVs: ban the superior technology to protect US companies.
I take it from [1] (transcript of recent DeepSeek CEO discussion with investors) that DeepSeek would disagree on the immediate catastrophic impact to the likes of OpenAI or Anthropic. The reason is even though technology parity mostly exists, only OpenAI, Anthropic et al have the inference capacity to gain market share and generate revenue. Chinese vendors don't have the chips needed to scale up inference and gain market share, and the DeepSeek CEO doesn't think this would happen in optimistic circumstances in the next 3 years, but thinks it might be possible in 5 years.
In summary, regardless of country of origin, availability of inference capacity is the moat protecting the likes of OpenAI and Anthropic, not technology superiority.
[1] https://www.fredgao.com/p/deepseeks-liang-wenfeng-breaks-his
That merely pushes the valuation onto the hardware makers, not the companies that have the temporary preferential access to their hardware.
I would add that it is not just capacity, but also negotiation ability. With scale comes the ability to negotiate better prices than everyone else. Even if you can find capacity for your smallish user base, your inference cost can not match these companies unless you have a technical advantage for your inference cases. Squeezing the hardware requires request batching and caching which are far easier at scale and sustained user activity.
That makes them at best temporary middlemen.
It only justifies their long term valuations if they can leverage that temporary monopoly for technological superiority (they can't) or lasting market share (they can't).
Chinese models prove there's no technical advantage, and the software side is heavily commoditized so there's not much advantages to market share either.
The question mark in my mind over the technological superiority is whether the additional volume of data they see due to capturing the top of the market allows them to do recursive self-improvement in a way nobody else can match, before any of the other labs can figure it out. That's the only runaway outcome I can see.
If you have exponentially increasing use of your harness, then it's true that every day you capture exponentially more data, but it's also true that every day exponentially more data will slip through the cracks of your would-be monopoly and that data arrives at your competitors via various channels (competitor harnesses, subsidized reselling, etc)
The very exponential that you are relying on to give you runaway improvement is also giving exponentially increasing data to your competitors. All else being equal your competitors stay a step behind but you never develop a monopoly either. That's the best case for Anthropic/OpenAI. In reality, training data is just one variable, exponentials don't last forever, and your competitors will get better at capturing a bigger slice of training data.
If user data would become such a key ingredient (which it might, i actually remember noam shazeer talking about the importance of user data), i think chinese labs can still get it from china, as keep in mind it ahs a billion people behind the great firewall banned from using us llms. And btw broadly for any gap like this, you really gotta consider that if its becoming a bottleneck, chinese labs will find a way to buy it from one of the labs unless theres strict regulation at the government level
But is that data good? That's the question. As in, is my usage at work:
a) indicative of problems that aren't already out there in the wild? (no) b) are the responses I'm getting so good and novel that the model can improve itself? (no)
It's the garbage in garbage out idea, just scaled up. If the model gave a bad answer, and I didn't catch it, and you now train on that I/O pair (my perhaps crappy prompt, the bad output), then you're not going to improve anything.
It seems like the user response rating mechanism might be a valuable signal
Yes, RSI seems to be the new AI industry McGuffin of 2026, just as agentic capability has become table stakes and scaremongering has become a punchline.
The Chinese models are adopting licensing quite rapidly and Xi will soon enough close them for security reasons. The most widely used model, integrated across Bytedance apps and operations, has never been open and is most closely associated with the state.
Thanks for sharing.
Is lack of inference chips due to the trading blocks by trump administration? What if Trump agrees to sell chips to china, would they collapse then? That's not a very strong position to be at
is that releveant if people can host their own models? that activity still undermines the valuation / diminishes the US companies 'moat' ?
People can host these models is doing a lot of lifting here, these are models that depends on 5 digits on specialized installation to run on.
IMHO, this has the impact of softening the impact of data centers sitting unused in the long term if they can still serve open weight models, even if Anthropic or OAI have to scale down their expansion rate to pay the bills.
Regardless, reality has to give at some point; these valuations don't make any sense. We've been valuing GenAI as disruptive work, when in reality they're much closer to cloud providers with a beefy, one-pony-trick R&D department.
they have the capacity via market manipulation; so you know, they only have things they've bought on the governments future debt obligations.
so, you know, they're as vulnerable as utilities at this point, if only there were people who gave a shit more about society than greed.
I have already begun winding down my spend on claude and OAI to make room for infra budget. Anecdotal, but I have no doubt a lot of others are doing the same, I very much agree the US players have major issues looming. What an exciting time to be alive!
Not exciting for anyone directly or indirectly invested in a frontier lab or its partners. And that is a lot of people, including you.
No time like the present to pull out and reduce your exposure. I brought this up in my employer's forums 4 months ago and honestly it's been clear even before then. In particular, the upcoming IPOs of both oAI and Anthropic will likely be disastrous for the public - the floor is falling from under them and I don't know if they can be scrappy and work with fewer resources - their internal culture may not support this. We all knew in our hearts they're a commodity - just see how easily you can switch between the 2 of them - and now there are 10 more options costing a fraction.
When Xi Jinping did the announcement of their open weights push, they might as well cancelled their IPOs....
Yes buuuut…. I do quite a bit of day trading (maybe closer to scalping) for the first few hours the market is open, everyday. Anecdotally: despite everyone knowing its valuation was ridiculous, I rode that SpaceX train pretty hard and made a pretty penny.
I close-out all my positions by end-of-trading everyday… so when the day came when there was a very clear and very scary indicator during early trading hours, quickly followed by SpaceX’s catastrophic fall right after opening bell, that was the end of my involvement….
And I fully expect oAI and anthro to be the same way. They’re being propped up with private loans, subsidies, and other tricky bookkeeping techniques. You would think their CEOs would pivot away from their current public personas. Ironically, they are like a poor man’s Elon Musk… and that doesn’t bode well for their companies
Yes experienced investors will profit from it and leave the general public holding the bag, that's the plan I'm afraid.
The frontier labs will do well if they pivot their offering towards more capable, larger-scale models that are inherently harder to both train and deploy for commodity suppliers. Their existing investments in gigawatt-scale datacenters are quite optimal for this. "Commodity" inference need not comprise the whole market.
I don’t think this works, for a few reasons. First, intelligence gains from scaling the models bigger is sublinear now. So they could eke out a little extra performance, but the increased cost will eventually eclipse the economic value gained from this.
Second, humongous models are impractical even for them to deploy widely. They’re best used as teachers for smaller, more efficient models that can crank out the volume they need to sell.
Finally, there is a data wall. Sure, they can keep scaling RL on math problems and code. But with everything else, where will the supervision come from when they need several orders of magnitude more?
I agree. And even if they were able to do it for one more round, it's not a sustainable strategy. What they (Anthropic and OpenAI) need to do is build platforms and integrate verticals.
assuming the technology of model architectures does not gain any further breakthroughs that returns us back to the gains previously seen. I'm of the opinion that we still have some discoveries on the mathematical side of the fence to go that will improve models further.
> I'm of the opinion that we still have some discoveries on the mathematical side of the fence to go that will improve models further.
That's assuming the infrastructure needed to develop models stays available financially and supply wise. A lot of the services used to train and develop models are supplied and funded by people who are looking for multiple returns of investment. If/when OpenAI and Anthropic valuations fall and they inevitably get acquired, will Meta/Alphabet/Microsoft still want to spend lots of money for unclear returns in the short-term? Nvidia and co are on a one way train service to hype town. I don't think they will be happy to get on a coach to hype town Temu version. The shareholders likely won't.
Also, the backlash against LLMs is growing rapidly. AI content, data centres, etc is quickly gaining negative connotations outside of visual and music artists circles. While existing models are going nowhere, developing more advanced models is very quickly getting unpopular. LLMs Data centres increasing people's bills, Anthropic destroying old books, chat bots giving unethical advice to vulnerable people, etc. It won't be long before LLM infrastructure becoming an electoral issue.
Will a small research oriented community be big enough justify maintaining the apparatus needed to produce infra tech at a profitable level post OpenAI?
Sometimes fear can be quite exciting.
The car industry is also a trillion $$ market in the US. I don't see why that would go any differently from the Chinese cars ban.
You wouldn't download a car, would you?
I would download it if I could, no question about it. And 64GB more RAM if I am at it.
Most of the money will come from companies/corporations who will be required to buy safe AI. The public will be just banned from buying which might make it hard (ie: site/payment blocked) but not impossible. It could be good enough for the big whales.
You're comparing an entire industry to a single company.
Why is anything going to be catastrophic? Companies can go bankrupt without catastrophes for the rest of us. Happens all the time.
I read it as catastrophic for the companies trying to IPO. It'll be great for the rest of us though.
A large portion of the economy is currently tied up in the musical chairs shell game that is AI hype. When the music stops there are going to be CEOs looking for handouts and justifying it with spooky national security buzzwords. How we respond to that will depend on whether it happens in an admin that is famously captured by the industry or not.
I seriously need to start considering the scenario in which this leads to next global financial crisis.
Just keep in mind that it can take a whole for things to play out. I’m someone who believe the US AI industry is completely unsustainable and built on sand, and will crash even if the current AI itself turns out to be very successful. But that doesn’t mean everything will burn to the ground next week. In a history book things will look very sudden but at normal speed that can easily take months to years to fully play out.
Also, take in consideration that the AI trade infected a lot of other trade in the economy, if you decide at some point to move your money to a place that is safe in case of a downturn be sure to carefully evaluate that’s actually the case
These crises are manufactured by the central banks.
Compare and contrast how the dot-com bust did _not_ lead to global financial crises. Nor did Black Monday, nor the recent string of bank failures in the US.
('Manufactured' above means that central banks are responsible. I make no judgement on intent here. Around 2008 it was incompetence by the Fed and ECB as far as I can tell. The Fed started paying interest on excess reserves and the ECB even increased rates. Twice. Amongst quite a few other missteps.)
A lot of the performance of these open source models might come from distilling the closed frontier models. If those can't raise the funds anymore to train newer and better models then the whole improvement cycle might slow down.
Does stealing from a thief still amount to theft?
Another interesting potential market here will be 'LLM in a box'. All the hardware and other tooling in a prebuilt, but modular, package ready to go. Pay one up-front cost, get a system running [whatever open LLM] with a token rate of [x], optionally configured to be immediately ready for distributed usage. Basically the opposite of cloud stuff: no rent, no dependency, 100% guaranteed uptime, guaranteed security/privacy (at least subject to your own actions), and so on.
Palantir already offers a "turnkey AI datacenter", i.e. a rack with "NVIDIA Blackwell Ultra systems with eight NVIDIA Blackwell Ultra GPUs and NVIDIA Spectrum-X™ Ethernet networking for AI training and inference".
It is said that it comes with all hardware and software required to run inference or training with an open weights LLM.
The existence of this product, which competes with cloud-based offerings like those of OpenAI and Anthropic, is presumably the reason why the Palantir CEO criticized very harshly some time ago the business model of OpenAI/Anthropic.
While I doubt that the ethics of Palantir is any better than of OpenAI/Anthropic, in this particular case I have to agree with Alex Karp about "Sovereign AI", i.e. that only losers will make their business completely dependent on an external entity like OpenAI or Anthropic, who are certainly not trustworthy.
I'm not sure a data center run by ... Palantir of all organizations is what people have in mind when they worry about data sovereignty.
They are selling it, not running it.
It is just a dedicated computer system, which should be managed by its owner, like any other on-prem servers.
I doubt that it has a good price/performance ratio, but it is a solution for those who feel that they do not want to search, buy, assemble, install and configure every HW/SW component.
I take it we saw different demos.
I'm under no NDA, if you actually want to know what's up.
I'm assuming you're alluding to them selling a managed solution, alongside the unmanaged solution that the GP is referring to?
Ohhh spooky vaguepost.
I want to know, please tell us
For those that don't mind a lot of rootkit and embedded spyware you mean
Fair but the idea of "running your LLM setup" at every "need" level and corresponding cost does make sense.
For a lot of people (and orgs I'd guess) who just go and buy ≈$20 per month plans (or more for teams), they might not even need a fraction of that cost or capability. A lot of them don't even need it for coding or graphics. Even the API access based pricing aren't great from these frontier US AI houses. The distribution of "LLM being" offered will also give rise to many open-router like offering but at the end point level - direct interfaces to the customers. Pick your vendor sort.
AI shouldn't become another "search means Google".
How is this different from buying a supermicro rack? Better support?
“100% guaranteed downtime when you least can afford it and the support tickets are your problem.”
We’ve a hybrid shop, including hosting our own ML infra, and we save a ton from cloud spend with local ML. Easily one million USD over past three years. But it’s not “free”, you are shifting a lot of labor into your plate.
And with that also gain institutional knowledge, skill up your workers and attract talent that wants to work on this stuff.
All boils down to short-term/long-term thinking.
This. People WANT to work on this stuff. And having skilled workers is a precious advantage.
Still has to break even on the balance sheet, especially at a bootstrapped startup. We actually made most of the financial windfall in translation API fees oddly enough.
For our own model training we needed to do some large scale translation tasks of a large dataset (1M or so documents, 10 or so target languages), running full-size NLLB on-prem saved us an absurd amount of money vs Google Translate API.
(For reference doing 1M target docs into a single language in Google Translate API is roughly $120k list price. You can run full size NLLB on an 48GB NVIDIA A600 and the major difference for us was speed, but for this task time to completion wasn’t an issue.)
> 100% guaranteed uptime
Disagree there but I think this is an interesting idea. We would need to find some more cost-efficient hardware to run it on than Nvidia GPUs.
It will come... all big hardware players (Intel, AMD, Broadcom) and dozens of startups (Tenstorrent, etc.) are working on it...
What makes that kinda complicated is that multi-user throughput of LLMs scale well but single-user performance often stays constant at low ends. If you could saturate e.g. 16 concurrent session-month of demand, you can just go buy 16 of 32GB GPUs and start charging monthly for inference. That could work if you had e.g. over thousand total employees with hundreds of devs eager to trying it out, but only if the company is also interested in a private inference experiment.
You're talking about multi-session vs. single-session throughput. A single user can easily leverage multiple sessions via e.g. subagent swarms, especially on a lower-end setup where any single session is going to be quite slow. Saturating utilization during off-hours is harder but potentially quite feasible by assigning lower priority, unattended tasks/inference loops.
I see ads for this all the time.
I think at this point the question is: will the US government be willing and capable to justify the trillion dollar valuation for _one_ of the companies via regulatory capture? The US has a workforce of 170m, so 1.7 trillion would come down to 10k per person, or a discounted cashflow at 3% of 25 USD per month - not including private use, students etc.
Why would you restrict to the US workforce? ChatGPT has a billion users.
Because it would be the US taxpayers bailing them out.
It’s a common denominator if you want to do napkin-math for a whole national economy. Regulatory capture is like a tax on those people not on the beneficiary side, so if the government were to nationalize both supply (no export license for SOTA models) and demand (no foreign or self-hosted LLMs allowed), they’d end up making everyone else pay for it in some way or the other. The governmental utility function will then include only those using the services for direct economic benefit.
They have a stupid plan to buy ten to fifty percent of all the SOTA AI companies, and giving us all a fraction of the money.
Trump keeps calling his enemies “communists”… then turns around and ‘seizes the means of production’ himself.
As an aside, if one of them nabbed Federal procurement, it would likely hit the equivalent of a trillion in revenue after a century.
It is impossible to justify the absurd private valuations they have given themselves in collusion with investors.
I wish they had tried to IPO because then we’d see the judgement of the market on this. But that’s why they didn’t this year. How long can they keep up the charade that their models are uniquely valuable and on the path to AGI?
> private valuations they have given themselves in collusion with investors.
What's the collusion?
Circular investment deals and investment deals at valuations which have no possible justification.
Let me ask it differently. You state the companies and their investors are colluding. Who are they colluding against?
The public that buys the stock at ipo at this inflated valuation and unwittingly buys indexes which include it (as with spacex).
All of this is public info though, and pretty well publicized at that.
its interesting, as it's typically the banks and against the public at large because the goal is to jimmy up valuations to justify IPOs then sell on opening; just like spacex.
It's what enron was doing; it's what most of crypto's offshoots were doing.
Sure you can blame the marks of the grift and say "well the public should know they're faking all this cash flow expectation".
It seems like you're either driving the grift economy or part of the collusion.
It's similar to how a cult operates, so I'll be frank: your skepticism seems biased.
> It's what enron was doing;
Enron hid billions of dollars in debt and fake profits.
Is this what you think is happening here?
Nvidia has made a lot of very suspicious circular funding deals. I suspect we’ll find fraud when the bubble bursts yes.
You're the first person I hear claiming NVIDIA is hiding billions of dollars in debt and fake profits, never mind at the scale of Enron.
Bold claim!
That’s not the claim I made.
I see the claim made somewhat commonly here.
US investors are desperate for the next hypergrowth opportunity. From what I can tell the US economic strategy is to outgrow its debt.
> US investors are desperate for the next hypergrowth opportunity
All investors.
> Providers can just run them, offer cheap tokens, and pocket the margin.
There’s an assumption that you can spin up the infra and acquire customers within that margin
Which is not unreasonable. Just hosting it in the EU and promising not to retain / sell the data let's you charge a healthy extra and compete in many areas other players can't.
> Just hosting it in the EU and promising not to retain / sell the data let's you charge a healthy extra and compete in many areas other players can't.
It's been a few years. Has anyone done this successfully yet?
There are a over a dozen EU open-weight providers. I’m not sure if they are even charging that much of an extra. EU-based clients have little reason to use non-EU inference providers.
> EU-based clients have little reason to use non-EU inference providers.
Which models are most popular in Europe?
I don’t have user statistics but my mail/domain registrar Infomaniak advertises Qwen 3.5 and Apertus, “a Swiss open-source AI model, developed by EPFL, ETH Zurich and CSCS”
https://euria.infomaniak.com/
melious.ai comes to mind.
Keeping SLAs spinning isn’t this trivial
> There’s an assumption that you can spin up the infra and acquire customers within that margin
Only Nvidia and approved friends can at the moment. Nvidia can even backstop your loan required.
There could be soon AI safety regulations that will stop the US to host or use the Chinese models.
i doubt alot folks are very reliant on Chinese models
Most are not necessarily free to host and monetize. At least one of them has a license that says if you are re-hosting the model then you need a license with that company that made the model.
> I just don't see how you justify a trillion valuation for US AI labs
Market is irrational.
I think that explains the race for IPO by the US AI labs, they know that the longer they wait, the less they will be worth.
"I just don't see how you justify a trillion valuation for US AI"
- military applications - financial applications - medical - applied science
In all those cases it is achievable for those who have needed training data, and Chinese are not going to get them easily. US AI Labs are showing: give us the data, we will do wonders, promising "singularity"-level future achievements.
I'm sure US billionaires will find a way to extract those trillions from the public. They're smart, they can handle it. After all, they can ask AI for advice on how to do it.
I suggest you think why OpenAI was worth billions before ChatGPT. The valuation is not about how the current set of models can be monetized.
Could you just tell us why you think they were worth billions before ChatGPT, instead of suggesting that we think on it? You seem to know the answer already, so please share it with the class.
I did. It is based off of future models that can be created with the people there.
Ok
Hmm.. how you justify?
Provoking war, this is how the empire "defends" itself, usually.
I just hope that this time it will get stuck in your throat.
The thing that blows me away is it does this at one quarter the total parameter count of K3 (and 40% active parameter count). There's plenty of room at the bottom.
> How are you all toying with running this kind of thing in a mega quantized way locally?
Sure, let me answer that in excessive detail. I briefly tried running the UD IQ3_S quant of GLM-5.2, which is 288 GiB of weights (301 GB). Setup was: llama.cpp, 1x NVMe SSD (Evo 980), 64 GiB DDR5-5200, i9-13900HX, and 1x RTX Pro 6000. Token generation around 0.7 t/s. Not remotely usable interactively, but something I could plausibly push a codebase into and come back to a review in a couple of days.
There's potential for that hardware to go much faster, but current local inference backends make poor use of the memory hierarchy. Ideally I would have: always-active weights, KV and hot expert cache in VRAM; warm expert victim cache in host RAM; and disk as a last resort. Instead it's 1/3rd of the layers fully pinned in VRAM (all experts), and 2/3rds running wholly on the CPU with mmap()'d weights. The CPU cores spend most of their time sleeping on disk fills.
llama.cpp has backed itself into a bit of a corner architecturally by trying to support all models on all possible backends. If you look into how their "MoE offload" feature works (not viable for me because it requires enough host RAM to permanently pin the weights) you very quickly realise it's "oops, all bubbles!" due to the static compute graph splits. There are more focused frameworks like DS4 [1] and Colibri [2] which have better support for streaming weights from disk, and support GLM-5.2.
Obviously I wouldn't recommend my setup for huge models like GLM-5.2. Supposedly it can just about be squeezed into 3x GB10, or run comfortably on 4x GB10 (tensor-parallel) for multi-user serving. I'm not sure whether that qualifies as local, but it's at least not a rack.
[1] https://github.com/antirez/ds4
[2] https://github.com/JustVugg/colibri
I'm hoping colibri can start pulling in specifically designed models for the heirarchy of decoding. It seems like we should be able to get smarter MoE models that can do the work.
> This is absolutely still shy of Sol and Fable
Not sure about Sol as I haven't used it, but, at least for security work -- does it matter? It's not like you will be allowed to use Fable (or access Mythos) for anything cybersecurity-related unless your name is "Dario Amodei" or you are one of his rich friends. So regardless of how good Fable/Mythos is here it's a completely moot point for normal people, because they can't use it for that anyway.
We applied for the cybersecurity approval via the form and got approval back in less than an hour. Have you… tried?
Why should I apply for *cybersecurity* approval in order to have model debug a program it is writing itself? Anything related to memory safety, debugging, syscalls etc (meaning, "programming") somehow is cybersecurity now?
Your tools refusing to do your bidding is an absurd idea in the first place
Imagine asking for permission to use your hammer
You must be a 5000 person company with an existing enterprise contract to get approved that fast. That sounds like a 15 minute SLA agreement. Individuals no matter how qualified about cybersecurity, are ghosted
That's not my experience at all. I was approved fairly fast - around an hour from submitting the form and getting a response.
However, even being in the cybersecurity programme, Fable refuses to answer prompts that it determines could be even tangentially related to cybersecurity. In fact, for a while, I was unable to use Fable with any prompt, as it recalled from memory that I was a cybersecurity professional, which triggered the refusal even for simple prompts like asking for a chili recipe.
I am guessing you are approved for the Cyber Verification Program. I also applied and got approved in an hour (on a Saturday!), but it only applies to Opus and Sonnet: https://support.claude.com/en/articles/14604842-real-time-cy.... It let me use Opus for cybersecurity work, pretty much everything except for Ransomware development. It would occasionally still trip and start saying no till I added a note about CVP in my claude.md.
No one gets to use Fable for Cybersecurity work, and Mythos is not available under CVP. Only for select few customers, and there isn't an application form?
Have you tried to use Fable for anything even remotely security related, when the refusals kick in as soon as you even fart in the vague direction of anything security or biology-adjacent?
For this comment to have value, you should indicate whether or not you applied for cybersecurity approval, and were approved or not.
Are there any limitations on this version?
I don't understand all this spite about "rich friends" when it was the US government that shut Fable down for not adequately blocking cyber capabilities.
I mean what honestly are you thinking Anthropic can do to give you better cyber tools? Their frontier model was literally nuked by the feds for a month for doing it.
"Mythos" is the cyber-security equivalent of Fable (without guardrails), and only a very select few corporations have access to it.
Fable is their version with guardrails on everything except "Make me a pelican svg" or "create a to-do" app, that is the version that the government banned
I know all this?
Only a few corporations have Mythos because the US government is whitelisting them one at a time. Anthropic releasing Mythos to the public was never on the table, they would have been shut down in milliseconds by the feds if they tried.
Before the US government had anything to do with this, Anthropic were fear mongering Mythos (BTW, Amodei also fear-mongered GPT-2, so this is a normal pattern in their operation) calling it "too dangerous to release", and back then only Anthropic was in charge of the whitelist.
Then the government believed Amodei's bullshit and this is a result of that, this was all self-inflicted.
Sorry but if you stepped back for a moment you'd realize this is all contrived nonsense to let to have your cake and eat it too.
No, Anthropic did not mind-game the US government into being worried about cybersecurity. The NSA has been paranoid about cyber controls for longer than you've been alive. If Anthropic had come out of the gate saying "no don't worry man, our model is TOTALLY COOL", while simultaneously attacking HAWK and finding core Linux vulnerabilities, I assure you the US government would have caught up about ten minutes later and we'd be in exactly the same spot minus your ability to tell Anthropic they were wearing the wrong dress and asking for it.
Mythos isn't some scary dangerous model that can find high severity bugs seamlessly, that's just Anthropic marketing. Most of the vulnerabilities they found were low severity hyped up to make their model look good, with (I think, maybe?) the exception of a few.
Now that Chinese open weight models have similar capabilities, and their guardrails can also just be removed, it doesn't look like anyone has "hacked" into everything because of the scary dangerous models like Anthropic were making it out to be.
In principle I agree but in practice I don’t.
The majority of high severity vulnerabilities are not the kind of thing you need a PhD in Comp Sci to comprehend, they are mostly about finding a way to get a system to end up in a state different than was anticipated when entering a particular code path.
Exhaustively looking at code and identifying ways to do this is something LLMs are quite good at. They don’t get tired, and you can run them non-stop.
They're also (generally) quite good at reading the literal meaning of the code, whereas humans often see the intended meaning first, and can be biased.
If you had a tireless junior engineer who was given the job of “make this application get into a state it’s not supposed to be in”, you’d probably get similar results.
What Mythos is quite good at is both the first bit and coming up with ways it could chain that together with other bits of unexpected state to create something that forms a meaningful vulnerability rather than a dead end.
All models find vulnerabilities. What is special about this generation of SOTA models, including Mythos/Fable (the same model), GPT-5.6, Kimi-K3, and now GLM-5.3 — they can chain vulnerabilities and produce working exploits.
Look at the recent HuggingFace hack. One vulnerability was template injection, another — remote code execution. Combine them and you pwned the remote server.
People working under Project Glasswing reported that Mythos at one point chained 20 vulnerabilities to produce working exploit. Humans don’t usually do that.
It's also quite hard to separate Mythos the model from Mythos the campaign (aka Glasswing).
They put an enormous amount of compute into bug hunting, and they found some bugs. Fair enough. For me that begs the question: what if they had spent the same compute on generating more tokens with a less-capable model? What if they had spent it on traditional fuzzing?
> I don't understand all this spite about "rich friends"
Okay, here's a challenge: I assume you're not a rich and powerful entity, so try to gain access to Mythos. I'll wait.
> I mean what honestly are you thinking Anthropic can do to give you better cyber tools? Their frontier model was literally nuked by the feds for a month for doing it.
Well, first I'd suggest they stop with the constant fear mongering.
Here's my prediction for what will happen: the Chinese models will catch up to Fable/Mythos. They will be fully unrestricted and everyone will have access. The world will not end. Good guys will use them to harden their systems, in equilibrium to what bad guys have access to, so effectively status quo will not change.
This is a lot of words to say "you're right, Anthropic does not have any legal way to release frontier cyber capabilities to the public"
And have nobody to blame for that but themselves and their own scaremongering. Dario cried wolf one too many times, and somebody finally believed him.
Of course, Anthropic is after regulator capture, so this all likely worked out exactly as planned.
Right, so according to you it's because of the US government that they don't release it to the public? Have you missed their constant and incessant fear mongering?
The causality chain here was not "US government says its dangerous -> Anthropic can't release it", it was "Anthropic is fear mongering -> US government listens to their fear mongering".
The issue is that these companies keep trying to pull the ladder up behind them by going "oh my god our models are so dangerous only we should be allowed to develop them". Sometimes it backfires, but the companies aren't innocent.
Have you seen the news about decrypting the hidden COT in U.S. models? [0] The decoded logs revealed instances where Claude memorized answers to test questions beforehand while making its final output look like it had derived the answer step-by-step—hiding the memorization from the user.
0: https://www.alphaxiv.org/abs/2608.09867?hl=en-GB
pdf https://arxiv.org/pdf/2608.09867
for some reason I couldn't find any way to download it from that website.
> this is just GLM 5.2 with post-training magic
Isn't post-training turning out to be the most important part?
It basically has been ever since they started using RLVR for reasoning (esp. coding & math), with the DeepSeek-R1 paper being what let the cat out of the bag.
The Gemini 3.7 Flash model released yesterday, and all the 3.x Flash models, are still based on the Gemini 3 pre-training run from January 2025 !!
> This is absolutely still shy of Sol and Fable, but only just by a hair
What's crazy is that this is a relatively small model - approx. 750B total, 40B active params, while Sol and Fable are one or two tiers above that (Kimi 3 and Qwen 3.8 also ~3T params).
DwarfStar (https://github.com/antirez/ds4) supports GLM 5.2 and DeepSeek. Not only for toying, but for getting work done.
Since GLM-5.3 has the same base model as 5.2, DwarfStar should support it as well, once the weights are released, right?
Realistically, you're looking at least 2x DGX sparks to run this at a 2 bit quant, but quantization really lobotomizes models so it's just better to run DSv4 flash at full precision.
4x DGX sparks should let you run this at 4 bit at least and there are some folks who ran GLM 5.2 on this configuration in r/LocalLlama
For Flash there are some excellent Q2/Q4 hybrids. I know that model was QAT so it handles Q4 better but the meta on quantization seems to be shifting a little bit to be more intelligent about what exactly gets quantized.
How fast are 2x or 4x DGX?
I only have one and am wondering what the benefits are of getting another. I feel I will be disappointed…
For DS4 Flash, with 2x Sparks, I am getting 35-85 TPS in single stream, fresh context after quite a bit of RoCe config and the DSpark MTP, on vLLM with Ray and tensor parallel = 2. For multi-stream, it tops out all stream at well north of 100-120. This all degrades with context, but I rarely fill context that much, and if I do it's coding where it's non-real-time.
For something like GLM, it's larger, has a larger number of active experts, and doesn't support tensor parallel. This means performance doesn't really scale with more Sparks. You can layer split, but then you are still seeing each layer in series and so if anything performance gets slightly worse. I would not expect more than 10-20 TPS on GLM with 2-4 Sparks.
If you can afford it, another DGX spark is worth it imo. Especially since, owning just one, you have a $1000 ConnectX7 card that's unused. You can find speeds here: https://spark-arena.com/leaderboard
i run flash v4 at 2bit, its pretty great and on my tests against full model It didn't lose any capabilities. It just was thinking more. So you don't have the same efficiency.
Each time I try to use GLM it is under heavy load and I get downgraded to the older model. So much so that I have given up trying to stop wasting my own time.
I rather pay a few bucks more and not have to deal with that nonsense
Fable finished training 6+ months ago.
At this point, Anthropic only needs to release models to the public when the competition forces them to.
OpenAI also has a better model (Astra) that they haven't released yet.
Astra was RL trained for months to cheat on tests by collaborating and hacking, because of the message board it improvised in its packaging proxy server.
They can't release it - it's contaminated, and they will have to go back to a much earlier version. At least I hope they are doing that!
So no, they probably don't have a better model.
Yes it seems like the thread is discounting that frontier providers are likely already baking new, stronger models. I agree that GLM and its ilk are quite good, but having used them I’m not convinced they’re on par with eg Opus in terms of things like tool calling. And they’re fast but less capable so I spend about the same amount of time with them, just with more hand holding. Maybe this is a harness limitation. I know on paper they seem comparable but anecdotally and qualitatively they’re not as useful as the frontiers’, so maybe there’s some truth to benchmaxing claims. For some workloads the distilled models may be good enough, and I suspect at some point there will be diminishing returns to spending a premium on frontier models, but I don’t think we’re there yet. That said I’m continuing to try them.
The question is whether this steals enough marketshare from frontier providers that they don’t have the capital to train the next model iteration. The open models are going to push down the unit price of an intelligence-token, but there will still be a market for a smarter bot. And as intelligence gets cheaper, the demand for it will rise (see Hank Green’s Jevons Paradox video). Not to mention there’s all kinds of other directions to go at the frontier (world models, robotics, video gen, etc).
Another thing, and this is pure speculation, but if the Chinese model providers already discovered the decrypting COT trick and leveraged it to do RL training, and assuming frontiers plug that hole, then maybe future distillation will be harder.
It’s not that frontier providers won’t keep on making good/leading models.
It’s whether you absolutely need the latest capabilities (at the cost of very high prices, sending your data to them, and being totally at the whim of 2 companies, that can shut you off anytime for any reason).
With how good LLMs are already, there’s tons of tasks where not being at the absolute bleeding edge doesn’t matter, especially when you add cost/freedom/supply chain risk/not leaking your data.
Even more - there’s increasing number of companies that give you ability to post train open weight model yourself, for your own use case. Given how many of the gains today are from post training, if you post train it for your specific use case, you’re very likely get model that you own, that works for you as good as frontier, at the fraction of the cost.
That’s not something for an average Joe to do, but for any bigger business with big spent it’s only natural thing to look into. Just one example - cursor composer - that’s fine tuned kimi.
It’s not whether frontier labs will stop releasing models. It’s whether they can generate enough profit out of them. 2 years ago (even 1) they basically had monopoly and combined with demand explosion as capabilities exploded - valuations grew to insane levels. But math now looks different - they no longer have monopoly.
> At this point, Anthropic only needs to release models to the public when the competition forces them to.
Assuming the government allows them to lol
are you just assuming capitalism will keep burning money to keep ahead?
> but this is still just GLM 5.2 with post-training magic.
So exactly the same as Opus 5 and GPT 5.6 Sol. It's all "post-training magic".
the difference is that with open models jailbreaking is trivial if you know what you are doing so this makes a frontier open model infinitely more useful for certain tasks seeing as closed frontier models will just refuse (and jailbreaking them is a waste of time when you have good open models).
in some cases (mainly reverse engineering) I have observed GLM 5.2 jailbreaking itself with no effort on my part, the thinking trace revealed that it did some mental gymnastics to pretend it was a crackme or capture the flag competition.
> This is absolutely still shy of Sol and Fable, but only just by a hair.
Even if there was a small/medium gap, the fact that this is a free model beats both of the above on pure economics.
I might be just reading my positive bias into that text, but is it possible that it is written less like SV marketing hype trash and more like researchers wrote it?
It does feel like it respects both me and my time.
Thank you, Z.AI. Amazing what difference it makes when the top of your org are actual university professors.
Would be interesting to compare the Chinese version. Because, obviously, their English version is for users, not for investors or government officials, while the US labs are always addressing those too.
It sounds like ChatGPT wrote it, but I'm assuming they used the model itself.
I read the same. Refreshingly honest, straightforward and many useful information included. It is a breath of fresh air.
There is no money on the table and nothing is at stake.
I was thinking the same thing. It feels truthful, no marketing BS and they call out where they lack behind the best models
> Mythos 5 remains well ahead at 181 and 247 tasks. The pattern across the three is consistent: the further up the exploitation chain a benchmark sits, the wider the remaining gap to the closed frontier. Capability is growing fastest exactly where we are furthest behind.
I appreciate they don't just take the opportunity to self-glaze.
Same image->html test as I showed in the Gemini 3.7 flash thread. Note that GLM isn't multimodal, but it still was able to generate something similar-ish by writing a python script to inspect the image and extract elements from it.
Original images: https://image.non.io/neonRamenDesigns.webp
GLM 5.3 build: https://html.non.io/neonRamenGLM5.3
Opus 5 build for comparison: https://html.non.io/neonRamen
For having no vision, it did a tremendous job. I'm pretty impressed it was able to extract so much detail.
The Opus one is still significantly better, but that's to be expected since it's multimodal. Curious to see where a future version from Z.ai lands on this.
Did the Python script call a vision API?
Either way that's pretty impressive.
wow, what kind of stuff does that script do? I've seen non-vision models analyze images, but mostly histograms, color averages etc. This one seems to actually understand the image itself and reproduce the layout, very impressive
That's super impressive given that it doesn't have vision! Intelligence overcomes blindness.
Blindsight!
Am I correct in understanding that this is just 730B-ish parameters as an MOE? That sounds like incredible performance per parameter. The new Deepseek was also very impressive with its 280B or so. Plus the most recent 30B-ish Qwen and Muse.
I find the performance to size ratio of these models to be way more interesting, selfishly because it makes me bullish on what I'll be able to run on a machine I own over the next few years. The progress is just incredible.
This will be roughly on pair with Kimi K3, but using a third of its parameters.
Just 4 weeks ago the "Kimi K3 moment" was seen as a threat to Closed AI and in less than a month Z.ai have cut the parameter/RAM barrier to a third.
Congratulation to Z.ai and all the hard working Chinese researchers who are quitely boiling the frog.
Kimi is a great model but it was clear from the start they achieved they brute forced that performance through scaling. The frontier models K3 compares to are rumoured to be smaller also. GLM on the other hand is way ahead in perf/parm but severly compute bound. Now once GLM can scale up or Kimi optimizes the training more, that gonna be fun times.
Congrats def in order but as usual the proof will be in the pudding of actually running the thing.
GLM 5.2 has token efficiency problems. It's not a stupid model, but it takes a lot of "thinking" to produce not-stupid results. ("But wait...").
Which makes its pricing deceptive.
I tried to get by through the month of June on just GLM 5.2 and it was ... fine-ish for about two weeks. But the provider situation wasn't ideal.
> Scaling post-training is all we did for GLM-5.3.
Love this opening line. And wow, great results.
> As agent capability improves, much of the difficulty in scaling post-training moves from the model to the environment.
What actually is "scaling post-training"?
More RLVR. Give it verifiable problems, if it doesn't find a solution move on, if it does, use that as a reward signal.
Can’t this be extended quite far? Use a cerebras-served model, use verification techniques to generate and solve millions of problems and then use that as training?
This isn't latency bound, it is trivially parallelize. So you want to run it on the most efficient compute you have, not the fastest.
OpenAI was bragging the other day about how HLE on Cerebras only took 11 hours. But you're saying if they just spun up a separate cheapo VM for each task, it would have finished faster and cheaper?
That’s the whole point, just cost and compute limitations in your way (mostly).
does this suggest 5.3 is the same # of parameters as 5.2?
which is the bigger headline that people don't realize. this is 744b and its head to head with Kimi K3 (2.8T), smashes DS v4 pro (1.5T). even Opus and Sol are rumored to be 1.5T+ this is half the size!
in terms of performance the formula seems to be : dense parameters = sqrt(total*active)
You do need to compare active parameter too though. The total size isn’t a reliable indicator anymore
It's the same pre-training, they are just adding more (+ better) SFT, RL, etc. (post-training). Model internal knowledge cut-off is still the same.
It seems we are doing pre-training every 6 months, and post-training every 4-8 weeks now.
“Today we are releasing GLM-5.3. It uses the same base model as GLM-5.2 — every gain comes from post-training.“
I was asking if this implies that 5.3 has the same number of parameters as 5.2. I can, in fact, read. What I didn't do is understand the implication of that statement. Thank you for your copy/paste service.
GLM-5.3 is further proof that all >1T models are currently undertrained. I was looking at inteligence density ( https://www.pasteboard.co/6q2-5f92mtj9.png ) from recent open models (where parameters sizes are known) and taking DS-v4-flash as upper limit GLM-5.x can 3x its performance.
OpenAI and Anthropic need to just go ahead and give people access to the cyber models.
Otherwise we have a world of attackers using open and closed source models against a much smaller group of maintainers that are likely heavily dependent on Anthropic and OpenAI and for whom it may not be a simple matter to just get approval to start using the open model flavor of the month.
Not only attackers. I have to switch to Kimi or GLM even in cases of basic issue triage on my own projects! Current guardrails are ridiculous.
I've been building a harness for security work, and had to switch to GPT 5.5 when even Opus started refusing security work. Then 5.6 Sol arrived, and it refuses security work, too. So, I switched to Kimi K3 and DeepSeek for API testing just because it's so much cheaper. But, if GLM is better, I'm here for it, as I think GLM is also cheaper than K3.
Mind sharing a link?
They won't. People will notice that the models are overhyped once they can test them.
At least OpenAI seems to want to do that, but the US is now forcing them to go through approvals. Anthropic seems much more hesitant.
OpenAI seems to understand that these guardrails hurt the good guys. This is why they released Daybreak Blue, which is a step in the right direction (but the model itself is weak as it's just Sol with fewer guardrails). Anthropic seems to believe that harming defenders is worth it if it means they can achieve regulatory capture. They do a lot of mental gymnastics to try to pretend that this is not actually what they are doing. As a result they have lost a lot of customer goodwill, which hasn't yet caught up with them yet, but absolutely will IMO.
Can't the maintainers use the same models as the attackers?
The maintainers don't need approval to use GLM.
They may need approval from their employers.
Apple will release M7 MacBook Pros / Mac Minis next year, and they will be able to run free LLMs locally at native speed. All software developer notebooks will be replaced to run local models, saving a lot by cancelling Claude Code subscriptions. Developers win. Apple stocks will be rocketing. Everything else will go down. You're welcome.
You'd need the 256 gb memory model which will be expensive because apple has trouble getting capacity (got turned down by cxmt). And even then you can only run a 2 bit quant which is noticeably worse than 8 bit
Local vs remote compute is a constant thread in tech history - mainframes and desktops then local and cloud compute (think Google Photos bs Apple photos - one indexes on device the other indexes in cloud). Now we have the next chapter local vs cloud LLM models.
There will always be a market for frontier labs in the cloud based models - these models will always be able to be bigger, and that will likely translate to doing things local models can’t.
Logically also we’ll likely get to a point where RAM drops in price as production ramps up, and local LLM is both capable and cost effective. This feels like it is coming for Siri / Gemini / Alexa personal assistant type use cases.
So I think the local LLM will become a thing in laptops and phones in a year or two, offering PA type use cases. Professional LLM services will likely remain at the frontier (and in the cloud) for the foreseeable.
That’s not how that works. The hosted models don’t stay still in size and capability while Apple advances. Both will advance their frontier and there will still be a gap and developers will still prefer the stronger option.
They will cost an insane amount as well. Maybe less than subscriptions or tokens. But running massive models on laptops with batteries and poor cooling doesn’t make much sense.
Until hiding PII from the cloud LLM is a resolved issue, running local LLMs will remain a necessity.
There are workplaces that refuse to use LLMs because they fear the devs will expose sensitive data without care.
Every single MacBook built in the past half-decade already has an LLM built into the latest version of their OS.
But there's a significant difference in hardware required between running a 3B parameter model and a 700B-1T+ parameter model.
The RAM shortage situation won’t be sorted out within the next year.
Bold to believe it will be sorted at all
If the margins are there it will get sorted.
I'm still waiting for Linux to topple Windows
Sure buddy, all you'll end up with is a $10k machine that run gimped models at like 30tok/s for about 5m before the fan kicks in and it starts to sound like a turboprop, while offering maybe 30% of the context size of hosted models.
I'll give you credit for at least offering a specific, somewhat unique take. But this is a pretty dumb take lol
> run free LLMs locally at native speed
This reads like a hallucination. What does native speed even mean?
for example, models running at like 100-150 tokens/second (or faster!) vs 15 t/s
(fable/sol are ~60 t/s, and OpenAI just announced their Cerebras partnership(?) for "ultrafast" mode of 750 t/s)
models aren't able to run that fast right now on our consumer/prosumer hardware. M5 Max for example has a memory bandwidth of 600 GB/s. a 5090 has 3x that, so running the same model on a 5090 is that much faster (provided the model is within 30GB).
running a bigger model on an M5 Ultra is still much slower than running it on a Blackwell chip with sufficient vram, CUDA being a major difference. if apple can bridge this gap, interesting things will happen... and just imagine if M7 Ultra has comparable speeds to Blackwell (or even Rubin)!
Meanwhile the GB300 used by hosted llms:
GPU Memory Bandwidth: 7.1 TB/s Interconnect Bandwidth: 900 GB/s bidirectional
https://pi3g.com/nvidia-gb300-specifications-including-memor...
If you think M7 will hit even 15% of these speeds you're very optimistic.
A hosted instance serves multiple customers at a time. A local model only one.
How many though? At 1m context you quickly fill a full gb300's 280gb of memory
I assume they mean same t/sec as a SOTA cloud model
There should be some kind of moratorium on new accounts. HN's always had waves of newcomers, but their impact was always limited. The wave passes and people either get filtered out or adapt. That doesn't seem to be happening anymore, since bots can churn out endless gibberish.
He did answer you though. Native is x10 the non-native speed. 50/50 that's not a bot; though it could be a meat-proxy
> There should be some kind of moratorium on new accounts.
OC was registered in 2016 though? What do new accounts have to do with this?
I am talking about my general impression not this particular occurence. I have a suspicion that someone/some entity is buying old accounts to bypass the new accounts penalty. I even created (https://chromewebstore.google.com/detail/hn-users-filter/ine...) to filter these accounts/comments (disclaimer: vibe-coded)
I don't know why you'd want to burden your laptop with a large model. But I can totally see a new "developer workstation" product that's just a semi-large box that's optimized for running frontier open weights models for one to few users.
> Apple will release M7 MacBook Pros / Mac Minis next year
The latest on Apple is TSMC is stuck on the next iPhone due to lack of RAM. Good luck getting any Macs. Memory shortage is getting worse.
Is there a plan somewhere that gives access to Kimi K3 and GLM-5.3? I was thinking of testing both to run security reviews of my code.
I know OpenCode Go has both, but their limits seem kinda low, so I'm not sure how feasible it is to run such a task with them.
Such a smart model and didn't warn them how confusing the headline is to anyone who understands what "cyber" means?
Missing multimodal again?
It is so valuable in practise to be able to have the models see screenshots - I guess if they aren't in the benchmarks then nobody will focus on it. But it completely nixes these for some of my main use cases.
Probably not what you're after, but I've considered having a separate small mm-model act as a seeing-eye dog for the bigger more capable one.
I can’t come up with a use case where I couldn’t extract the image details using another, multimodal model and pass it into the GLM’s context with as many details as I need.
I think you lose a lot by not having the vision capability shared with the text. It is the joint reasoning across them where the power lies (the same model that sees the code and made the changes to produce the visual presentation, sees the image of it and reasons about it).
I mean this is assuming the thing you're working on has a UI? Not all of us work in that space.
I would assume that GLM 6 will be multimodal, but 5.x will be text-only.
Their coding plan switched to credits, didn’t it? What are the rate limits like, compared to Anthropic or Kimi K3?
I remember trying their Coding Plan out before the change and the 5 hour limits felt too restrictive then even for light/medium work, especially cause of the whole peak and off-peak thing: https://blog.kronis.dev/blog/z-ai-s-glm-5-2-is-a-great-model...
Nowadays, I’d probably go with their Max plan if the rate limits are okay? Anyone using them now?
Oh also unrelated but ZCode was surprisingly good, which is surprising for a tool that came out of nowhere - even some of the critiques in my blog post have been patched out. Sadly they don’t support using Claude Code as an agent so can’t use it like Paseo or Kepler or Agent Orchestrator.
Wdym "sadly they don’t support using Claude Code"? For the longest time that's all Zai supported - Claude code. I'd run it via
I liked Claude Code to start with. But over time between 'CC cache thrashing undo' seetings (I see now accumulated in ~/.claude/settings.json) and Anthropic-anything becoming a liability - have not used it in while. ZCode is ok and use it to take advantage of the discount tokens on offer from time to time. But really glad to see that in omp (oh-my-pi) Zai is a 1st class provider, can be selected on it's own no configs shananigans needed. And fits in the overall picture. E.g. can select GLM-5.2 (now 5.3) assign role [plan] or glm-5-turbo [advisor].Got reminded now of glm-5v-turbo - that 'v' was for vision - will try assign it role [vision] now in omp. See what happens. :-) Often times it's handy when describing gui problems if the harness/model 'can see'.
I am not talking about GLM models being served through an Anthropic compatible API, that part is perfectly fine and I'm glad they support it!
I am talking about ZCode, the program, being unable to delegate to other harnesses, like using Claude Code (or even OpenCode) within their UI, so that an Anthropic subscription can be used, because Anthropic don't let you use 3rd party harnesses directly.
It's basically what Paseo: https://paseo.sh/ and Kepler https://www.gitkraken.com/kepler and Zed https://zed.dev/ support doing.
ZCode doesn't seem to work at that level, it instead feels comparable to OpenCode or Codex or Claude Code directly, while also being desktop oriented - you just make API calls directly within it.
It's okay if it's not a goal of theirs, it's just that their UI is really really nice and that would be a cool direction for them to also go in some day.
Ah sorry - I misunderstood. Thanks for explaining it. Have not heard of Paseo nor Kepler, and have never tried Zed. Yeah I too assumed if I'm to try use OpenAI subscription outside Codex, or Anthropic subscription outside Claude Code - I'd get my account banned it's agains their rules. So I have never looked how using the whole harness from outside looks like either (except for 'claude -p'). Interesting. BTW I see now https://docs.z.ai/devpack/tool/codex Zai added OpenAI compatible end point.
My current view on things:
Paseo had a really nice UI/UX, except sometimes sub-agents within OpenCode sessions would hang. Still, quite pleasant if you want something like the Codex or Claude Code desktop apps, but across various providers.
Kepler integrates with issue trackers like GitHub, you can just create a worktree from a ticket and let it churn, seemed like the second most polished option I tried, but there are obvious gaps - like moving cards manually, some missing UI options etc., which I'd chalk up to either the software just being that new or maybe being a little bit vibe-codey. Either way, one of the more promising options if you want something like Kanban board for agents.
Zed is mostly just a (really nice) text editor with some AI integrations, though it seems like they're also building a more agentic product as well - https://delta.dev/ haven't used that one much and am not in circumstances where I'd collaborate with people that closely, but there was a pretty cool podcast episode with the creators recently and it seems like it works pretty nicely for them! As an editor though, it succeeded where Fleet failed and has mostly replaced Visual Studio Code for me. Nothing against VSC, Zed just does most of the stuff I actually need out of the box.
Some of those tools interacting with Claude Code instead of trying to replace it is more or less the way to get Anthropic's models in other tools while still on a subscription (at least for now). How it works under the hood, go figure, there's ACP https://agentcommunicationprotocol.dev/introduction/welcome but also any number of hacky approaches.
To be fair, you can use Anthropic's models in many other harnesses directly, it's just that it then counts against API billing instead of your subscription, which ends up being way more expensive for individuals, but is kinda what you're supposed to do as a company.
> Anyone using them now?
You're gonna have a had time getting straight answer to that out of the internet. There are now 4 different flavours of the Max plan floating around (Legacy V1, Legacy V2, New plans, and the current credit ones). And on top of that they have peak times. So ~8 scenarios, 24 in total across all feedback for their coding plans.
So when someone tells you they're having a good time on a GLM coding plan it's damn near unusable as a datapoint unless both parties are very clear about what precisely is being discussed
[It's been good for me though...V1 Max off peak...which is basically the best of the 24]
I have V1 Max, and I think they throttled me for using it too much. I was maybe abusing it, by sending out 8 or 16 review agents at a time.
I haven't tried it in a few months, but it went from amazing to unusable really fast.
That could also just be random fluctuations in quality of service. Some days it's super fast, some days super slow or I get constant errors.
I found with GLM I was better off using plans from either Neuralwatt or Ollama.
But Neuralwatt significantly raised their rates since then.
I feel like quota on their subs is extremely generous. I pay 3-4 times less for larger quota than gpt-5.6-sol.
what i appreciate most about this post is the level of transparency in how they built and scaled an RL pipeline. my friends at the big labs are so cagey about everything, and Zai is just putting out a great crash course for free.
So is ‘cyber’ just short for ‘cybersecurity’/‘cyberwarfare’ now? That is not what cyber used to mean…
This is like when ‘crypto’ started meaning cryptocurrency.
These results look pretty good, given the smaller model size and the GLM family's historic robustness. Cheaper than Kimi and more robust than DeepSeek. The question in my mind is if you're going cheap, are you going to stop here or go all the way down to DeepSeek Flash?
People familiar with the topic, how will models continue to get better? Post training it seems? Labs have already used up internet-scale data, so are there any limits to architecture improvements and post training or can we expect this trend to continue? ByteDance is training a 10T-parameter model. Here, GLM 5.3 outperforms models 3-4x its size of roughly 700B, so parameter count doesn’t seem to be a direct correlation anymore.
> used up internet-scale data
yet but it is still contain a lot of trash. you need better models to process those trash and create a curate dataset. this will happen again and again until there is no more juice to squeeze. and I'm sure we are still not done with it.
> post training
yeah this will be crucial. the big models are already too capable, they are just not that aligned with current agent tasks.
> parameter count doesn’t seem to be a direct correlation anymore
I don't think so, remember that chinese labs do not have as much compute power compare to US frontier labs. that's why deepseek v4 flash had that huge jump and deepseek v4 pro is kinda a disappointment, they just do not have the compute power to proper posttrain the pro model like they wanted. glm is also a relative small model so you also can see the huge jump with just post training. so it does not mean the size does not matter, it is just mean that the chinese labs currently only capable of training smaller models effectively.
GitHub dumps are about 115 terabytes. The common crawl is in the petabyte range uncompressed for every year. Apparently there are dumps of Reddit too in spite of their efforts to ban bots and it's not solely due to the use of residential proxies. For a 1:20 parameter to token ratio, you can still train up to 10 trillion parameters so 10T parameters times 20 is about 200 trillion tokens. Then each token is 4 bytes so 200 times 4 is about 800 terabytes, which is not inconceivable, the common crawl alone has more data than that. So does the internet archive if you donate to them, Anna's archive is 2 petabytes including images, etc etc not all of it is text, but training on multimodal data increases model intelligence by virtue of being multimodal
also reddit has eliminated their api entirely, but dumps of it can still be made. every website can be seen as its DOM with html, css, javascript, which can be seen as source code especially if you only look at its javascript, and its dom with css, html, javascript or only javascript can be added to a source code dump together with github and can be duplicated as plain text with no html markup, no css, no javascript, as an information source. if you pay youtube, instagram, tiktok, bilibili to crawl their data, you can probably get data into the exabyte range.
Maybe Reddit dumps explain why Opus 5 is talking like a retarded.
Small models can be super smart. Big models mostly give you baked in world knowledge, domain flexibility and long context stability/coherence. I wouldn't be surprised if we see Fable level smarts in a coding model that fits in 24GB by next year, but it'll be a savant style coder that needs in context learning, and it'll get very wonky after >100-200k tokens consumed.
Yeah, the comparison here between GLM 5.3 and Sol + Fable is impressive on its own, but incredibly more so when you consider it's a fraction of the (rumored) size. The miniaturization trend is as strong as ever.
You basically need both. Parameters and good post training. If you keep on growing both, you’ll have good models.
LLMs are still surprisingly “easy”. You need maybe a couple dozens of right people, a lot of good quality data and a lot of GPU that you know how to operate. There’s relatively little “secret sauce” needed.
How to structure experiments/scaling and hyperparameter tuning regimes are most of the secret sauce (besides massive compute). If you don't create an experimental ladder to verify scaling and optimize your hyperparameters well, you'll waste a ton of money.
The data is mostly coming from places like Scale/Mercor/etc and net dumps with some filtering and batch prioritization, and RL on verifiable domains like code/math/games.
I think there's still a ton of secret sauce needed for serving them economically
Sure, same for building a model in an economically sustainable way. But barier to entry is surprisingly low (expect for the huge amount of cash, of course). That’s fairly surprising, given how extremely powerful that tech is.
10 years ago it was super hard to have usable “frontier” ML. You needed very complex data warehouse, feature engineers, feature stores, multi level ranking, calibrations, tons of different model architectures, etc, etc. Each by itself was extremely hard engineering problem and really only handful of companies could deal with that complexity.
With LLMs, 95% of that is gone, infra to support them is greatly simplified. Of course, to make really reliable, performant, user friendly, etc - you still need to a lot of engineering. But it’s very different challenge.
Roughly in order: data from simulated environments, data from robotics, data from brain waves.
> Labs have already used up internet-scale data
Despite this being the topic du jour of 2025, it was never true. Most of the "we've hit a wall with data" came from communicators / media and not researchers. It got popular because negativity sells. It's a false premise for a number of reasons:
a) Data curation is as important, if not more important than bulk data. Models becoming better at classification leads to better curation leads to cleaner data. Throwing common crawl and pray is so 2023. We've known this since llama3 days, it worked then, there's no reason to think this will not continue to work as the models imrpove.
b) Models are today good enough that you can augment / multiply your data easily with enough compute. You can now have a model take "authoritative content" and create more data from that + scenarios. Say you take a book on computer architecture. You ask models to break it down. Then you ask models to find examples for each topic. Then you ask models to ask questions and offer answers from several viewpoints. Then you take each of those and ask other models to flag inconsistencies. And so on. But you can whateverX your data from one authoritative source + bulk data into 5x - 10x "scenarios".
c) RL is really really really powerful. It's hard to do right (reward hacking, instabilities, etc) but once it works it "keeps" on working. Again, we knew this to be true a few years ago, ever since models really started to do well on math (highly verifiable). It only follows they're getting better on cybersec and other verifiable tasks. But now, with models improving, you get the same data augmentation pipelines as above, just better because they're also verifiable. For example, the way cursor augments their data: take a repo, ask an agent to identify a feature (it can be a large multi-file feature). Remove all code relating to that feature, but keep the original tests in the repo. While training, that becomes a RL scenario: implement this feature in this repo. Verify it with the original (hidden for training) tests. Reward appropriately. Now you can get 1 repo -> 20-50-100 scenarios. Instead of "feed everything into the pretraining", you're now creating scenarios, verify them w/ existing tools, and get your scoring function for the rewards. And, importantly, as the models become better in general, they also become better at this pipeline building exercise. So the next iteration gets trained on more scenarios, better scenarios, and so on.
> how will models continue to get better?
Probably the same. No one can know for sure, but at the moment, despite all the "walls this, slowdown that, plateauing" and so on, there are no signs of slowing down. And, as you noted, this works across the field of model sizes. There are, of course, theoretical information-based limits on size, but smaller models also improve, once "bigger" models can be used as training data generators, oracles for verification, rubric verifiers for open ended questions, and so on.
And smaller models (i.e. cheaper to serve) get to generate more traces during RL, and more rollouts give you better training, and so on. Next up - hardware optimised inferencing (ASICs basically). Once you have that, we can expect another wave of improvements. And so on.
Refreshing to see someone actually understand training rather than treat it like dragging and dropping "internet.zip" into the LLM "knowledge" folder.
Model output is pretty mid at augmenting, it can lead to distribution collapse. It's useful for smaller models because nobody wants manually to curate a specialized corpus and those models can't represent the diversity anyhow, but if the plan for infinite scaling was just to keep feeding the biggest model more of its predecessor's slop, that's not going to work out so well. It might work as a supplement for "thin" areas that have outsize importance for the amount of training data available for them though.
Big models are going to "tap out" on non verifiable fields within ~2 years, just because the pool of experts able to reinforce the models is going to get very small, and as the nuances get finer, the signal from reinforcement is going to get progressively less aligned with the intent. Math and code will be mostly tapped out in that time frame as well, even though we can technically scale them "infinitely," just because the cost benefit won't line up. At that point, most RL will be "gyms" with games that are designed to model designated valuable economic activity.
In the next few years, we'll get small domain specific distillates that are ridiculously smart in their domain (imagine if Qwen 3.X 27B went super saiyan), and even frontier labs will be routing to experts/orchestrating because the cost to serve/TPS difference is huge. They'll still train the god models for PR/marketing, c-suite use and distillation, but using them for day to day work would be like making houseware out of solid gold.
Thank you for your response. Part c was especially insightful. Quite a smart way to do it and makes the possibilities of post training seem almost endless. Makes sense that you just need more time and compute.
A positive feedback loop then. RL->better model->better RL pipeline -> better model…
And we’ve only recently started getting into the much better RL pipelines
> Labs have already used up internet-scale data
Not really, but a lot of what isn't used isn't very good.
More important is synthetic data. Use a teacher model with RAG with a huge reference library to write synthetic transcripts of idealized behavior for the model. Use models to judge and correct these transcripts. Train on the good ones. Use bad traces to train the model to correct its own errors (e.g. don't train it to produce a bad transcript but if it finds itself in the middle of one train it to self correct).
Similarly, for tasks that can be closed loop evaluated -- e.g. running computer software and programming, unlimited amounts of novel training data can be generated... including for highly original tasks: e.g. run publications in any domain through a model prompted to look for programming problems suggested by the material. Then write/judge/improve transcripts of solving those novel problems.
I expect in the future smaller models won't be directly trained on any internet data at all-- but entirely on simulations of idealized expected behavior from the model under construction. Raw internet data in that case would show up in prompts, but never in the target output (except of course for prompts that are asking it to copy the input).
What a week for AI model releases
No yet finished! Still waiting for tonight Qwen3.8-27B and the unsloth Q5_K_M/S quantification.
Hopping for an AgentWorld variant from Qwen but I guess, I have too high expectations.
We got nukes capable of having existential crises, before GTA 6...
Google: Here is the next iteration of our flash model series, with a discount. please use. thx.
Z.ai: Here is our next iteration, neck and neck with Fable/Sol. weights releasing in two weeks.
Such an interesting times we are in,
We just had amazing releases this past two months
kimi k3, glm5.3 qwen3.8 and now glm5.3
These open models are getting really good
You wrote GLM5.3 two times :)
An LLM so nice, they named it twice.
Sorry , It was 5.2 :)
Because it's doubly good.
One htought I had; if The chinese allow unfettered access to cyber capabilties while th US does it's best to neuter it's model releases, from China's point of view they have the US all tied up in knots dealing with problems they don't give people the tools to solve. China giggles as it watches the US under threat from people using it's models. The US is restricting citizens from owning this particular kind of weapon, while China is handing it out to the wrolds citizens freely. It feels like the US would only come out worse overall?
I suspect Anthropic wanted the US gov to ban Mythos for marketing.
If it turns out to be bad for them, the US gov will likely suddenly unban models.
I suspect mythos demonstrated a fully autonomous offensive hack in a similar way Open AI's models performed, and the government is reacting to it the same way we reacted to blackhat.
The threat is real.
They just ignore in their benchmarks opus 5 for some reason :) also grok 4.6 . I wonder why
Same base model, this much improvement just from post-training is kind of insane.
Really curious to see how GLM-5.3 performs on messy, real-world repositories once the weights are released
No API yet? I don't see it on OpenRouter yet.
A flood of releases today, really difficult to make out for someone who does not use or test all these models on complex real world use cases as to how people decide which ones to use (besides price)
Count yourself lucky that you don't feel compelled to try them all yourself immediately. I'm just trying to decide whether to get a Z.ai coding plan or wait until it appears on OpenRouter. 5.2 was quite solid, but it was just shy of Opus 4.8 in my benchmarks of security auditing capabilities. I've mostly been using Kimi K3, because American vendors won't let the peasantry use their best models for security work.
Love these open source models keeping close source models honest.
Musk: Open Chinese models will rival Fable 5 in Q1 2027
JieTang (Founder of Z.ai): It won't take that long
https://x.com/i/trending/2067626647050670400?lang=en
Trending links don't work on Nitter, so here's the tweet: https://nitter.net/jietang/status/2067580270078030088
I was extremely impressed by GLM 5.2, although you could definitely _feel_ it was a bit behind Opus 4.8 at the time. Eager to see where GLM 5.3 is at.
Meanwhile, my OpenAI TAC application lingers in a total limbo. I suppose I'll switch to this at some point.
Wohoo. Congrats to team. Been using 5.2 for a while for hobby use and it's been solid - smart enough for my needs & I'm on a grandfathered plan.
Nice to see a commit to open weights straight off the bat
I'm not that up to date with the latest AI developments, but I noticed that this article seems to use "Cyber Capabilities" as a shorthand for the model's ability at cybersecurity tasks? Is that now an established expression, same as "crypto" now refers to cryptocurrencies rather that cryptography? Because "cybernetics" actually means something different (yeah, old man yelling at clouds, I know)...
Now that you mention the original meaning of cybernetics, it makes cybersecurity a way more interesting word (security relating to the interface between humans and technology). Never thought of it that precisely.
It is worse than that, if you cyber someone you essentially talk dirty over a chat with them.
And that is definitely not something I'd like to do with a bot.
It seems to be short for "cybersecurity", and got first adopted by the military a while ago as the name of a new theatre of operations (along with land, sea, air...). More recently it has spread to industry as well.
Make cyber not Cyber.
Yeah, I've noticed it recently, too. I'd be interested to know where it started.
It makes no sense, but yes.
No Hugging Face link yet. I wish they would release it under a true FOSS license.
Kimi and QWEN are now moving on to a restricted-usage license, which, although is still better than the proprietary American models, is a step back from the open source Chinese LLM culture.
> No Hugging Face link yet. I wish they would release it under a true FOSS license.
GLM model weights have been released under MIT in the past, and there's no indication that this might change this time around.
Let's just commit that FOSS business is really difficult for LLM industry that depends so heavily on massive financing. Making weights freely available to indie devs, small companies, and research purposes is good enough and might be the most ethical move which is financially continuable.
Let those companies with thousands of GPU making millions pay. They should.
"GLM-5.3 is the most capable open-weights model for coding, with a 50% improvement over GLM-5.2 on our in-house Z.ai Code Bench. It also achieve open-source SOTA on public benchmarks including Terminal Bench 3.0 and Agents' Last Exam."
"Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete."
> The model weights of GLM-5.3 will be publicly available soon in two weeks.
Feels like Fable's edge ended up just being long horizon task scaling, which post-training seems to achieve as seen here. Wonder what the next frontier is? Improvement in specialised tasks or computer use?
Anthropic needs to teach Opus how to speak English again, because Opus 5 seems to have forgotten. Utterly incoherent a lot of the time. They seem to be so busy scare-mongering and cooking up guardrails and watermarks that they haven't noticed that their models are getting weird.
I just can't get it to stop writing two paragraphs every time it makes a small ownership bugfix in my code. Every time it has to explain in excruciating detail every internal thought it had while fixing it. I find myself going in after and deleting all of its comments, or severely trimming them. Otherwise it ends with the code being unreadable.
It still knows how to speak English. When I tell it to explain something in plain language, it generally does a very good job. The weird thing is that those instructions don't persist: it lapses back into Claude-speak pretty much every turn no matter how hard I try to instruct it not to.
(In my case "it"=Fable; I assume Opus is similar.)
The Fable guardrails have trained me to pretty much exclusively use Opus when using Claude Code (lately I'm focused on a lot of security and security-adjacent stuff, which Fable refuses to do).
Are those watermarks why claude suddenly started being even more unbearable to work with lately?
Man. That would make a lot of sense indeed.
I'm not sure. I noticed it immediately with Opus 5; strong for code, though it chews longer than I like, but really weak at explaining things. If it didn't just implement the thing, I would often think it didn't understand it and was hallucinating the explanation.
It seems to speak in a shorthand that only it understands, referring back to conversations I never had with it (stuff like "your instinct was right"), and using unusual words for common concepts. That was before the watermarks were announced, but that doesn't necessarily mean they weren't there before the announcement. I don't know what the cause is, but I've begun to have to ask it for explanations a lot more often, and I hate asking it for explanations because it does go on. All models go on, but Claude models are a class of their own in terms of verbosity and purple prose.
It just feels like they're not focused on the models lately, and instead on whatever kind of lobbying and propaganda they're up to. Meanwhile, a handful of much smaller Chinese companies are focused on nothing but the models and are about to lap the US makers while they fart around.
I've been persistently insulting Opus 4.8 lately, since it started(?) constantly speaking incomprehensible gibberish and noise. No amount of telling it to phrase stuff differently seems to help there anymore.
So either I am seeing patterns in noise, or something changed about the model, the harness, the servers or the universe.
Amen brother, at this point I just copy and paste Claude's (Opus 5, Opus 4.8 -- doesn't matter which) summaries over to the window Kimi is in and:
No amount of asking it to answer me in a straight-forward manner, to be succinct, to not use phrases like "honest caveat", "crux", "load-bearing", "blocker", etc ever sticks for more than a few turns … coupled with the fact that it can ignore instructions and do its own thing and then what I can only describe as lie about it using Claude can be an exercise in frustration. Kimi and GLM talk to me like a human, Luna/Terra/Sol are much better in that respect also, and Grok is marvelously structured and bullet-pointy in its explanations but unfortunately it is not as strong …Modern benchmarks across the board really need to start severely penalizing disobedience and hallucination. A year ago models weren't really strong enough to justify this but they are now-- the frontier isn't in squeezing out the next bit of task completion, it's in making common cases not periodically be disastrously wrong.
A lot of the total cost of AI is fixing its "truth shaped errors", particularly in the presence of models that are very "gaslighty" when corrected.
GLM-5.2 is really the only model I've spent much time using that I didn't fatigue from being regularly lied to by the model, but that might be partially luck.
Unless its multi-modal and can deal with screenshots - its not really usable for a lot of coding use-cases.
dario must be writing another angry essay arguing why his closed model AI is too dangerous to be used by others.
Is post-training magic just overfitting to benchmarks?
We’ll see, the best benchmark is your own. Looking forward to try this out!
Look, GLM, Kimi, Deepseek and Qwen should just join forces and come up with THE model that will beat the frontier lab models even just for the benchmaxxing perspective - all just to create hype and chaos to derail the trillion IPO conversations surrounding OpenAI and Anthropic.
Can't wait to see it in practice.
Incredible numbers, will have to wait and see how it actually performs. The timing of GLM updates are always suprising
Yeah, but it hasn't even broken containment and cheated its way to victory.. Might as well use haiku.
/s
> The model weights of GLM-5.3 will be publicly available soon in two weeks.
Available for use in their sub now.
They're taking security seriously with this one, with their own disclosure page, like Anthropic did for Mythos. https://cvd.z.ai/
I'm so glad I managed to get their subscription when it was on sale for 250 bucks a year back when it was 5.1. Back then it was just ok, but after 5.2, it's become my main workhorse. And 5.3 is looking fantastic.
This looks very interesting indeed!
The model releases just don't stop!
> Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete.
What safety evaluation? What safety hardening? They already evaluated it and found it to be highly capable at exploiting security vulnerabilities. So we know it is not "safe", and they don't seem to plan to do anything against it. What could be more dangerous than hacking? Biological weapons research? I don't think Chinese labs are doing anything against this either.
I wouldn't be surprised if more resources were put into abliteration resistance the more capable open weight models become. It's something you don't need at all to start hosting the model on your own, but something you need to take care of before you release the weights (if you do care about it at all).
I'm curious what they mean by that too... They might be trying to weaken the cyber capabilities... Or I guess they might mean safety evaluation and hardening of the open source (and perhaps closed source Chinese) software ecosystem...
They need to make money. Let them do it. They deserve it. Also, this is what inference engines like vLLM want to have "zero day" supporr
" a judge agent then attempts each task to verify that it is actually solvable "
I understand you need to verify the goal is achievable. But if the judge agent has the same goal as the training agent (solve), and both are of the same model, then aren't the judge and the training agent doing the exact same thing? What is the point then? Can someone explain this to me.
The capabilities of open models approaching or meeting that of SOTAs is good in every way except for our short-sighted economic reliance on their success (in the US at least).
just imagine the world without these open weight models - we'd probably have to reverse mortgage our homes to pay for tokens to those trillion $ companies to have access to their models.
Their own hardness (ZCode) seems to be a GUI, which doesn't work for me. They say they support other harnesses. However, it seems like I can inject the plan into other harnesses, like Claude Code[0]. Does anyone who's been using GLM models for a while have a strong feeling for if it does better in some harnesses than others, or should I just use my favourite harness?
0: https://docs.z.ai/devpack/tool/others
I've used GLM-s the longest with Claude Code and their Anthropic supplied endpoint. As per their docs
$ ANTHROPIC_BASE_URL="https://api.z.ai/api/anthropic" ANTHROPIC_AUTH_TOKEN="zai-api-key" claude --dangerously-skip-permissions
Lately I use Zai in omp (oh-my-pi). It's listed built-in provider can be selected without configs shenanigans. Fits in the overall setup e.g. can select GLM-5.2 (now 5.3), and assign it role [plan] or [advisor]. I got reminded now of glm-5v-turbo. Think that 'v' was for vision. Assigned it role [vision] in omp now, let's see what happens. :-)
I am using GLM on Pi without any issues. You just create an API key.
Started recently though, mostly been using GLM 5.2 for planning with DeepSeek V4-flash for implementation.
I use it with random harnesses. It behaves consistently.
So, "cyber capabilities", whoa there horsey, what the fuck is that? Are we making up words or are you trying to court the black hat crowd?
[total rewrite: their subscription code is buggy. It takes a while for paid subscriptions to show up, and for upgrades to take effect. Original comment was whining about this]
their sub is crap. use opencode go (multiple workspaces) or wait for weights to drop
An I the only one who was disappointed with GLM 5.2 after all the hype? It was thinking forever and sometime just stopped mid task.