Competition drives prices down. We tried Kimi and GLM models after hitting 'cybersecurity' blocks on OpenAI and Anthropic. We found them to be much cheaper to access and helped us complete our work without 'cybersecurity' refusals blocking us.
They're not competing against each other, they're competing against free/local/open weight models that they're also lobbying very hard to regulate/ban.
As much as I hear this sentiment, and as much as all the smart software people around me echo the desire for this sentiment to be true; it feels like people say these things, but then at the same time say - well, Claude/Chatgpt is still marginally better for me now. But clearly they are catching up so maybe one day!!!
In that sense, I'm really not sure if Kimi/chinese models are really competing against Anthropic/OAI? Not to mention that for individuals, the subscription model is extremely different, and Kimi subs are (I think) worse value, and chinese API's maybe marginally end up cheaper than using american subscriptions.
The profit comes from the enterprise customers who have to pay for usage at API rates. Not the subsidized subscriptions, which both OpenAI have designed to be unavailable for large enterprises by withholding IAM and auditing features they need except on the usage based entperise plans.
As an individual I'm still paying for a ChatGPT Plus plan since $20/mo gives me a lot of value and I use many multiples of $20 at API rates per month. But in terms of capabilities, I am entirely satisfied when using my Opencode Go plan with open models like Deepseek Flash/Pro, Kimi and MiMo.
At this point keeping my ChatGPT subscription isn't because of a significant capability gap compared to open models. It's 90% about the value of the subscription priced well below API rates (10% the fun of trying latest models).
If OpenAI/Anthropic dropped their cheap subscriptions or drastically undermined the value proposition I'd be happy using open models exclusively and currently barely notice a difference switching back and forth for the vast majority of tasks.
I'm expecting the subs to increasingly provide less value as the US labs IPO and follow the SV pattern of moving from marketshare capture mode to profit extraction mode. Thankfully I'm already comfortable using OpenCode with open models and avoid lock-in to any specific harness so would be a very soft landing.
I guess I don't understand "enterprise psychology" but why are auditing features worth a 50x premium? I think we're currently blasting through $1000 worth of tokens a month per dev.
Because enterprises have compliance and certification requirements, meaning the enterprise is essentially held hostage by providers on price.
You could argue it's worth that much because of the value that compliance brings the enterprise, but it's just another gouging excercise. Certainly nothing unique to AI companies.
They aren't but both firms tell bigger/richer clients that the subs just aren't available to them no matter what. It's not a policy, it's an email that says "we're moving you onto API rates". They have no obligation to follow a systematic policy!
> I guess I don't understand "enterprise psychology" but why are auditing features worth a 50x premium?
Alternatively, one can say that the enterprise customers plus a metric shitload of VC and shareholder/investor money heavily subsidizes cheap personal plans.
It's the same playbook that Adobe used, provide cheap plans for academia and turn a blind eye towards piracy from non-commercial actors, get the young people hooked hard on Adobe's software and workflows, and they will demand that their employers purchase the nice, high profit commercial/enterprise plans.
They are of course are the ones paying for R&D of new models but its not exactly clear if gross margins on subscriptions are negative. It might be just normal market segmentation just like SaaS companies charging enterprise users many times more than private individuals.
DeepSeek announced that each GPU they have makes about $38 an hour in revenue during off-peak times. That's $331k a year, and DeepSeek is cheap.
I think that it's not only possible but likely that the frontier labs are profitable on an ongoing basis, and it's mainly the research and capex which make them look like impending disasters.
Realistically, OpenAI and Anthropic are each probably worth ~$1T each. Anthropic can already easily justify that based on revenue alone. OpenAI isn't far off. Consider how quickly these companies are growing and the fact that they maintain a small but significant edge, I don't think the bubble is going to pop. Just deflate a little.
> I think that it's not only possible but likely that the frontier labs are profitable on an ongoing basis, and it's mainly the research and capex which make them look like impending disasters.
And that is the much more relevant problem. Capable open-weight models from China, European labs or competitors with cash to burn (i.e. Meta!) erode all the moat that the major labs have, and tooling functionality like MCP, CLI interfaces or integration into third-party tools like IDEs quickly gets cloned.
Simply printing money by running inference does not justify valuations in the billions of dollars, because anyone can do so.
Let’s not forget that Luna max is actually extremely capable for the price, you can get a lot done on the $20 sub.
Haven’t tried haiku 5.5, I expect it to simply run out too fast, but they did change the price below 100k tokens recently; perhaps I should check it out.
Openrouter stats seem to indicate that there is a lot of demand for ‘good enough at a good price.’ After all, not everyone is decompiling PS5 games. In fact, most people are just speedrunning notetaking apps in JavaScript frameworks.
But, Openrouter stats can only measure the usage they see. I'm not sure it is really all that representative of total LLM usage across the market.
I think that's the difference here -- OpenAI and Anthropic are competing more with each other (and Google Gemini) than they are with open weight models. While people on HN are familiar with Kimi and GLM models, I'm not sure how many casual users of LLMs know about them. I think the competitive risk open weight models pose to OpenAI and Anthropic is much further in the future.
There is a lot of usage that's only known by Anthropic or OpenAI that's not visible to the rest of the world.
And the share of the tasks where cheap open model is enough only grows in time.
Frontier models just fight over smaller and smaller piece of the pie and I suspect them pushing for security work is way to open potentially infinite token sink as attackers pay for tokens so defenders need to spend even more tokens to defens
> most people are just speedrunning notetaking apps in JavaScript frameworks.
I feel seen. But seriously, my app, with my preferences, written by a semi-trustworthy entity that isn’t trying to shove ads down my throat. That’s really great.
Yep, this. I've seen it coined as "the age of personal software" and I totally agree.
I've built easily a dozen personal apps in the last few months that I have no plans on releasing or sharing, run locally, and solve very specific productivity (or one-off) issues for me.
I would never have spent the time building these apps in the pre-LLM days because the effort wouldn't have been worth the gains.
But now? Sure Claude, build me a MacOS app in Swift that does this exact thing I need. Less than an hour later, it's done.
i mean, i made myself a dvd player for my steam machine. i don't see how that makes anthropic money at real token costs.
when lyft line was heavily subsidized in SF by VC money, i used it almost daily to commute to work because it cost $2.50 each way. Now that it's $20, I don't use it.
I'm pretty much weaned off closed-weight models at this point.
It's a very "The Year of Linux on Desktop" situation, in the sense that in any particular year it seems like Linux is close, but the OSS stuff requires a lot of configuration to get to feature parity, and the barrier keeps rising because the prop solutions are running as fast as they can to justify the cost premium.
Unlike Linux, the world is pretty literate about tech business models, so I think there's a way for the open weight world to avoid the situation Linux got into, where it only really caught up once the market was pretty much commoditization any-ways.
If anyone interested in PL theory (including adjacent fields) and/or inference engines is reading this wants to help make open weight models and associated ecosystems feature-competitive, we're hiring, drop me a line: nathan at nfulton org.
> well, Claude/Chatgpt is still marginally better for me now
Enterprise doesn't care about marginally better. They care about cost.
Most organizations that run our self-hostable chat platform with cloud models are for most stretches running 6 months-1 year old models, and are strongly eyeing (or have already switched) to more commodity models for most of their workforce. Frontier models are mainly relevant for engineering tasks.
For enterprise customers the value proposition of self-hosting one of the open models is enormous. We're talking 10x cheaper (remember that enterprises pay API rates), plus more control on where the data goes (especially given that some frontier models do not support ZDR). And all those savings are lost revenue for the US AI Labs.
My issue with anthropic and openai is that if I use a model like e.g. Sonnet 5 I don't always get full Sonnet 5, sometimes it's dumber, sometimes it is smarter (mostly dumber though), while GLM, Kimi and others stay the same. So if I rely on AI for things I better make sure the platform is stable and does not introduce hidden changes.
Is Starbucks competing against Pete's Coffee, or are they competing against your home brew? Is it McDonald's vs Burger King, or all fast food against home cooking?
Neither OpenAI or anthropic can afford to lose subscribers to the other or anybody else right before IPO. I tried both CC and codex recently, and I would say they are practically on par. Some things CC does better, some things codex does better. I've also seen them increase usage limits. The cheapest plan now feels very generous imo.
Price per task has been dropping exponentially for years. This isn’t a sudden new phenomenon. They’ve been competing with each other continuously. There is even more competition now than ever before.
Borrowing money is not the only way to go into debt. They reportedly made binding and non-cancelable commitments. Once those commitments come due then they will be indebted to their counterparties until they pay up or the courts sort it out.
This is comparable to how, if I sign a 12 month non-cancelable lease and then don’t pay the rent, I will be in debt to my landlord.
That's effectively borrowing money with a different name though. The only way to go into debt is to sign an agreement to pay someone money in the future, regardless of the terms used to describe it.
No, it's really not. Do you have a war chest to drag out a legal battle and the ability to bankrupt your landlord by withholding payment? The rules are different at this scale, and "you can't enforce a contract if the attempt will bankrupt you" is only one of the possible games.
Here's another one: pay a supplier to cripple their deployment. Suddenly it's the neocloud's fault for failing to deliver, not your fault for failing to pay.
Did we all forget how Sam Altman turned a non-profit into a for-profit? Do we think he'll bring less sociopathy to this battle?
> Do you have a war chest to drag out a legal battle and the ability to bankrupt your landlord
Their war chest is not sufficiently larger than the hyperscalers, Broadcom or PE financiers like BlackRock.
> The rules are different at this scale, and "you can't enforce a contract if the attempt will bankrupt you" is only one of the possible games.
Rather, the inverse of what you said and far less Machiavellian.
It is legally well established that take-or-pay contracts are unenforceable if it would bankrupt the buyer which is the more likely resolution in this case. However, this does not mean frontier labs just get to walk away. They would settle in court for a penalty that is less than the full contract price.
See: AMD/global foundries and pretty much every oil and gas or commodity deal that goes bust when market pricing makes extraction unfeasible.
That's not the metric. The legal system is pay-for-delay, not pay-to-win. Once you have enough money to delay, it becomes a contest of who has the longer runway, and that probably goes to the frontier labs rather than the neoclouds who have little alternative revenue and comparatively airtight debt obligations.
> take-or-pay contracts are unenforceable if it would bankrupt the buyer which is the more likely resolution in this case. However, this does not mean frontier labs just get to walk away. They would settle in court for a penalty
Regardless, the terms come down to positioning and bargaining power. Runway vs obligations. This seems to dramatically favor the labs.
> it becomes a contest of who has the longer runway
Which is what I said - frontier labs do not have a sufficiently larger war chest.
Respectfully, comparing the balance sheets of a frontier lab to Blackrock, Apollo, Softbank or Nvidia is laughable.
The neocloud will also easily rent out that unused compute and keep funding their war chest via Nvidia’s backstop. If Nvidia can’t pay then it’s all over and AI companies have been swallowed by credit markets.
Your hypothesis also ignores the fact that if frontier labs are not paying for compute it is because they have bad revenue, little cash available and no credit (no one is loaning money to OAI or Anthropic if they can’t pay their bills. They can barely get debt today).
> Regardless, the terms come down to positioning and bargaining power. Runway vs obligations. This seems to dramatically favor the labs.
No, the only way out is for OpenAI or Anthropic to claim in court that they would be insolvent if they are required to pay the bill in full. Otherwise this is a straight forward deal.
You are greatly overestimating their capabilities and underestimating how many substantially more well-funded and highly liquid players will be lining up to take their pound of flesh if OAI or Anthropic default on a bill.
I’d wager my life savings one of those two will go through liquidation.
Does it matter if they "win" or "lose" legally ? They still don't have money to pay, so someone (probably people that gave them their debt in the first place, and their investors) will end holding the bag.
The government could bail them out, that's the most likely scenario if they don't implode IMO.
They could also continue to grow at the rates they had last year. I don't know where those customers would come from, but if they did pull that off OpenAI and Anthropic could sustain propping up the circular financing going around the AI industry.
Goverment bailing they out will be using people tax to rescue the companies that absorbed most of the 801k money so they can get people out of work... it's the ultimate slap to the face... I would strongly resist this, I would vote if they implode to give to whoever owned shares via 801k their money, to people and let them (openai, anthropic, et al) eat shit... I would love to buy one of those gpus to run an open model at discount price...
Completely agree it will be a huge slap in the face to the public, unfortunately that's a common solution.
When everyone is afraid the market will collapse, or more likely distracted because the market collapsed just enough to hurt the average person, the government will bail them out and either people won't notice or will welcome it as a solution to huge levels of fear.
Voting isn't usually much of a solution after the fact, the political machine doesn't allow for candidates that would fix anything to have a chance at any office that matters. There's a reason our politicians today all seem to have roughly the same goals and all seem to become extremely wealthy while in office.
Government trying to manage financial markets is like trying to steer a rampaging rhino; finance is stronger than government, and can wreck it. But some politicians want to increase regulation and some want to reduce it. Some cynicism is fair, but you've overstated the case. Elections do not offer infinite choice, but the choices are distinct.
But why would they bail them out ? Stock value will drop regardless of what govt does and having the borrowers (the private companies that already made billions on selling the shovel) get stuck with the unclaimable debt is far better option, and those are not hiring average blue collar workers in hundreds of thousands so any job loss would be minuscule too
For the same reason the government bailed out the banks, they're too big to fail.
Two companies account for most of the frontier R&D, and something like 7 companies account for a huge portion of growth in the stock market. The government, especially under Trump, is very focused on continuing to see those numbers go up.
I also expect the government is pretty bought in on AI being the future and on us being in a race with China. They'd have to bail out the labs for those reasons alone.
Lastly in my opinion, Trump has a pretty socialist bent to him and regularly talks about a sovereign wealth fund owning portions of companies. He'd love to have take a partial stake in AI companies, and a bail out is a great way to do that.
A bailout would be a slap in the face when the correct solution is to nationalize them. If they are too big to fail then the government gets a controlling share of equity and we finally start the sovereign wealth fund we should have started decades ago.
Nationalization is fundamentally different from a bailout. It tends to go poorly when a government begins taking over the means of production.
Though I agree with you a bailout is a slap in the face. In my opinion bailouts shouldn't be legal, governments should pro actively be making sure individual corporations don't become so large that a failure would wreck the economy or national security.
Why is it you see a sovereign wealth fund as the right answer? Why should the sovereign own the means of production, and why should the people creating and enforcing laws be the same as those in charge of our economic future?
I live in a state that votes heavily in one direction, most states are the same. Its fine for me to vote, but the winner of a majority of states is pretty well decided before a ballot is cast (by strong majority, I'm not claiming fraud).
If everyone who threw up their hands actually voted, the outcome would change. See also, the elections musk tried to buy in the last 18 months or so since he doged up the place.
Apple is worth more than all the AI companies combined. You think LLMs are less valuable in the global market than slightly updated iPhone models every year?
LLMs? No. LLM companies? Yes. It's just math, data centers, and mass copyright infringement. The only scarcity is Nvidia, who just passed Apple in market cap, somewhat proving your point but not in the way you meant it.
If you consider Nvidia an LLM company, and I do at this point given how heavily they've invested in frontier labs and where much of their revenue comes from, they alone are worth slightly more than Apple.
They'll take a heavy hit stock wise, and might be an LLM company through all their shady financial constructions to keep the music going by propping up demand. But in the end, nvidia as a company has a place even when the frontier labs go bankrupt. Some other company will take their place. Worst case it'll be like the cable companies in the end 90s who found themselves out of work because they overbuilt for dotcom
> It's just math, data centers, and mass copyright infringement.
Ok then iPhones are just metal, plastic, glass, and a delivery truck. You are just electric meat. This has to be one of the dumber comments I’ve read on here.
Maybe not, but it is probably less likely that Apple will face stiff competition from freely available alternatives, given that AOSP has been a thing for the iPhone’s entire lifespan and nonproprietary Android phones have failed to capture significant market share.
Their costs could drop, competition could dry up, or they may only be unprofitable in the first place because they’re choosing to invest more money in growing their business than they’re currently making, a practice which need not and will not continue indefinitely.
It’s not at all unusual for companies to run at a loss for several years before turning a profit.
Stuff like this that defies all logic happens all the time. Amazon operated at a massive loss first it’s first decade and still IPO’d years before they posted a profit.
Amazon were investing in goods that lasted longer. It's an open question how long the GPUs will be worth anything, and training costs are the real killer.
Z.ai, who make GLM are public and their books are really, really, really bad.
I am technically not saying that. I am taking off from the parent comment and speculating.
Their data from September is leaked via IPO preview, and October is not yet complete, so how can someone even project that?
// I am a techie, not a finance guy, so apart from public chatter, no inside knowledge.
And I do feel their IPO is being rushed before anyone has enough information to make up their mind. The VCs or private equity have given them a lot of money and want to cash out. When the revenue does slow down, any investor should pause to take a look at whether they can right the ship, or has the landscape itself changed.
Their revenue taps, product releases and marketing campaigns were all strategic: to be able to show nice numbers in the financials.
Once they stop playing games with quality of output of these models and revenue's shrink - they will be in deep deep trouble. They are hoping they will conjure up a break through with the IPO proceeds.
The flattening part is not reported, but based on estimates from a company called TickerTrends. If they were correct it would imply a sudden and unusual departure from some incredibly impressive growth. Even more unusual given the release of their industry leading Opus/Sonnet/Haiku. Something to watch but I would not consider that authoritative, and would not invest based on that either.
Opem models catching up in capabilities to do more and more tasks would easily explain it, there is a reason OpenRouter got sold for 7.5 bil, the money is in selling shovels and price per token, not chasing the frontier
The impressive growth is plausible, but it also rhymes with "bubble". There is obviously a value proposition in there, but I'm skeptical it's a trillion-dollar one. It's a machine for turning electricity and silicon into powerpoint and spotify slop
I partly agree. In support of your scepticism, I also think the frontier labs are overvalued. The expensive part which is not currently priced into the subscriptions is the training. Inference is usually profitable now - at least from what we can tell from industry reports.
Where this bubble differs from the dot-com bubble is that valuations have very strong, organic user demand behind them. In 1999, prototype websites with few customers were getting IPOs worth hundreds of millions. This time, demand is insatiable. Frontier mdoels also has very low global penetration, relatively speaking, so the TAM permits exponentially more sales, in theory.
However the risks are numerous for these labs. Foremost is that Chinese labs are able to easily distill frontier models. A tiny fraction of the cost of training, and they have a close approximation. Second, models are getting much more efficient per task. Companies are already discovering what "good enough" looks like, and it often ends up being much cheaper than what Anthropic and OpenAI are selling. This will only accelerate.
My investment thesis is bullish on the entire AI stack except the frontier labs. I think power generation is going to see huge demand. Ditto for infrastructure and data centers. Ditto for the products and services created by these models.
I don't have the numbers in front of me but could have sworn I had seen quarterly revenue growth rates dropping drastically over the last year or so. Still a strong growth rate, but much lower than prior growth rates.
Edit: Data because silent down votes are useless, Anthropic Q2 2026 grew 147% while Q3 grew 45%
Yes but its growth rates with respect to expenditures, and in particular capital expenditures (R&D etc), which we have no idea about.
Gross margins here are irrelevant: if they can't charge a high enough price to cover all expenditures and rely on cash raised via fund-raises, they are not financially healthy.
I'm in the same boat. Had to switch to GLM because Claude refuses all native work nowadays.
"Hi Claude, there's clearly a memory leap in our software. It clearly balloons over time. Here's a few heap dumps from various points in the day"
"Unfortunately memory safety issues and OOM errors are clearly restricted cyber work, contact your nearest screaming void if you want to appeal this decision"
The US can make it difficult, but obviously not air tight. However no enterprise will touch it, just fringe hacker types.
I think it's more likely though that China stops the party, as they stand to lose from the threat of it just as well. And China can just flip the switch to stop new releases.
I think your question is rhetorical, in which case I agree with the sentiment.
The US absolutely could (and very well might) make Chinese models illegal, possibly open weight models in general, but all that will do is keep them out of the hands of law abiding US citizens.
Law breakers in the US and the entire rest of the world would still have access, and nothing in the US will be any safer. People who think regulation will solve the problem should be embarrassed by their ignorance.
I don't get why people think Chinese models could bring openai and anthropic to their knees. There are zero signs of such a scenario. In fact the market is still growing at a gigantic pace. Quality of American models is still AAA. Chinese will have their place. But they are unmatched by their American rivals.
> But they are unmatched by their American rivals.
Kimi K3.1 is top 5 on AA and half the price of Opus 5.5, Qwen4 is coming soon. There is near zero difference for most tasks, so it matters less today and even less as time goes on. Price is now the top concern for a reason.
Banning open weight models in the US would provide the rest of the world with a huge business advantage since these models are much more cost-effective than US models.
OpenRouter and other inference providers won’t be able to openly host the models, for starters. That should cut most (or at least a significant share of) non-local inference, and local inference only works with smaller, less capable models, plus the hardware is hella expensive when you don’t have scale to saturate.
In addition, most companies don’t want to be in a position where a disgruntled employee can report them to authorities and rack up massive fines or worse, even if it’s hard to tell from the outside.
> If you are a legally operating company in the US just saying it's banned is enough, the threat of enforcement is enough pressure.
Is there precedent for the US actually "banning" software? I know for example the Kaspersky ban and tiktok ban, but both of those were either banning the sales or banning the service. IIRC neither made the actual software illegal.
What is an example of a specific software being illegal to possess? What is an example of a specific software being illegal to use if it is not used to break other laws (copyright, etc.) or talks to banned service providers?
Closest thing I can think of is restricting the export of cryptography in the 90's, but that was not even close to what you are suggesting.
Correct it’s 7% chance on kalshi, and those 7% of people are just wrong. I know everyone likes to larp that the US is some kind of dictatorship, but lol, lmao as they say.
Also on the “how” side so far we have “they will just announce that “they” are banned, hilarious.
yea but I thought they had a massive compute shortage. Something doesn't add up. Imagine there being a beef shortage and the beef producers cut prices because "competition". why?
If the beef producers don't believe beef is a commodity and they want to get you addicted to their specific beef then it makes sense. Right now in these companies life cycles profit doesn't actually matter. The only thing that matters is how much of their beef is being consumed. They want to give people so much free beef they can't even imagine the idea of having a meal that doesn't contain beef.
I have a harness for my harnesses now. My Hermes manages Codex, CC or Cursor now and interact with me via Slack and PRs on GitHub.
It's nice because I can keep going from anywhere, other team members can see and interject in my Slack threads, and Hermes auto updates its skills with any adjustments that we make so we don't have to ask twice.
At this point I'd rather try to invest effort and usage in a harness that is FOSS and is actually steered by a non-profit. Goose comes to mind and to be honest I haven't tried it beyond installing and giving it a prompt or two.
Comiong to harnesses, even Pi seems like a weird choice. I mean Pi, until it's "your own customised Pi", is pretty useless and pointless. Besides they are also starting to have their own models/infra business (or is it mostly proxy like OpenCode?).
I have also given up on Herdr and came back to Tmux (and doing a bit of customisation slowly, as I need it). I wanted to try dsh but somehow couldn't brave it. Maybe once I have a good container/vm setup is ready. I know I am biased/discriminatory here, but there are things about which I'd be better safe than sorry.
As for OpenCode: I used V1 a lot and I am struggling with V2 currently. Yes, it just doesn't feel right. It's neither here, nor there. I am not a harness reviewer, so I won't be able to put that into industry accepted words and tests but it doesn't feel useful and right to me. At least Pi is completely stripped (it might be changing though) and then you can make it as you want it. Even Claude Code is better than OpenCode and that's saying something. I find Codex to be quite effective but it doesn't feel familiar. I am still in the middle of some work so I am using OpenCode v2 a lot (and then there's OpenCode Go and I won't want to play around a lot with it) but I will move away. Where? I am not sure yet.
What I feel will work for me? I guess a harness of my own! Forked something smaller/similar? I don't know. No, really! Just like I have never found the "my" notes app and journal app yet, I don't think I should use any of these harnesses if it doesn't work for me, especially now that I can kinda make one (hail LLM/vibe coding :P). It doesn't even have to be a TS/Node monsters most of these harnesses are. More I am reading about, digging deeper (haven't been quite far yet), how models and the "local setup" interact, the more I end up scratching my head – why all this drama and why all these 5 billion GitHub star "harnesses" with trillions of lines of code! Why? It doesn't make sense!
Personally I've found Zed's agent to be much superior to OpenCode. IDK why but it gets better results from the same model on the same task. Using with GLM 5.3 and DeepSeek 4.1.
You should do that. I am also summoning the motivation to start on my own. I guess the worse will be that I fail. Might as well fork sth more basic, or build on top of sth SDK/library like.
I don't know if you'll be interested, but there is an excellent book on the topic: Build an AI Agent (From Scratch), by Jungjun Hur and Younghee Song (Manning Publication).
By chapter four you have a minimal working codebase in python. Later chapters add out features like tool calling, planning, and reflection.
I am thinking about it but I think forking will miss the point. I'd rather do in a way when there is a prevalence of various types of agentic/harness SDKs for people to make their own apps (aka harnesses) from.
That’s what pi.dev exist for but I’m not touching it because like all of the other harnesses it’s built in JS and they’re all terrible when it comes to cpu and memory use. Maki is the only one I found the be the most efficient harness. It uses 1% cpu idle vs 20-30%.
A few bucks will realistically last you forever.
I also bought a DGX Spark at launch. I've learned a ton. but it's not usually my daily driver. I do run overnight tasks on it, like "read this book and write up notes on each chapter..."
In retrospect, given the dirt cheap prices of apis and or renting hardware, buying hardware for inference was likely a mistake (but not enough of a mistake to sell it now).
I think building a OpenAI compatible RAG-pipeline that also perhaps includes the anthropic sdk let's you have the best of both worlds. You can build it in a way that a local agent can escalate to a frontier model when needed and also substitute prompts data that you want to hide from being "public". For example. I think that is meant by harness here.
And with some smart feedback loops, you will see that for home assistant type tasks local AI can pretty quickly fulfil all requirements. For quick responses or voice analysis you might want to start with an external provider.
Otherwise, I think for complex home automation around 24-32gb vram are a realistic house number, but more experienced users will probably need less here.
People say that tools are called by the model. That isn't entirely correct. The caller calls the tools, the model just formulates the syntax for it. So the model part can be externalised without you needing to compromise you system fully (just to the degree the tools would allow anyway)
For me, I used Zed Agent with Codex as it has native integration with Zed's ACP so it's pretty convenient.
However, Anthropic is pickier about the harnesses you can use with Claude Code and I noticed discrepancies when using Zed's External Agent feature with Claude Code, so I just use the Claude app. It's fine.
I'm calling BS. I use Kimi, chatGPT (Astra/Sol), and Claude (Opus 5.5), and Kimi is by far the most expensive in terms of tokens and $/work done. Using OpenCode harness.
This and the terrible writing, esp by Anthropic models. Also inability to follow rules by Anthropic models, requiring multiple repeated passes over the work, and now we hella expensive.
DeepSeek claims $38 of revenue per GPU, per hour for Flash V4. That is $332,880 per GPU, per year. This matches their DeepInfer numbers: 25,600 output tokens/s per GPU sold at $0.60 per million -> $55 minus some other overhead to arrive at $38.
For the bigger Anthropic and OpenAI models (and associated higher prices) there are many unknown factors. But presumably they also work hard on optimizations, and it's not so crazy to think they had or still have a lot of wiggle room to adjust prices.
Yes, but DeepSeek went absolute goblin-mode on optimization and I would not be shocked to discover that OpenAI and Anthropic were behind on that front.
OpenAI and Anthropic have more compute and the best frontier models to throw at the optimization problems. They have published articles on these efforts. For example, Dots, and Anthropic's upcoming "persistent agent" is because of these optimizations. Both companies often publish articles detailing a lot of their technical efforts. They just get buried among the flood of other news.
That's just it: they have more compute. They have a "do more with more" lever in addition to a "do slightly less with much less" lever. DeepSeek can't win with the first lever, so they have committed everything to the second, while within the frontier labs the two levers fight for attention and frankly the first is clearly the favored child. I'm not accusing OpenAI and Anthropic of being slouches, but I am not sure that more resources + divided goals > singleminded pursuit of a singular goal.
Sonnet 5.5 is 16x more expensive at $10 per million output tokens. Kimi K3.1 is also $10 per million output tokens, and its number of active parameters is only 6.5x larger than V4.1 Flash (104B compared to 16B). These things are not directly comparable of course, but just to give an idea.
Even if inference of those models needs 16x, 32x, or 48x the number of GPUs compared to V4.1 Flash, still seems like potentially great revenue per GPU.
It's sad how far down you have to scroll to get the real answer. It's because Anthropic margins on inference are absolutely bonkers and it's easy for them to drop prices by 80% and STILL have positive free cash flow
It's all based on vibes and whatever vague lies the companies put out about their financials. Marvels of corpo-nonsense speak such as "If we ignored every single cost we have, we're quite profitable!" seems to be very popular with the AI crowd
This sentiment aged extremely poorly with Anthropic's S1 now released. Turns out the "vibes" guys were right! I expect the goalpost will shift to claims that the S1 is fabricated in a larger than Enron fraud involving both the auditors and Anthropic.
You mean the S-1 docs where they state they're operating at a GAAP of -4% (even then that's based on their best 28 days multiplied by 13), meaning losing money? You sure showed the plebs with the impressive billions of dollars of loss!
Where exactly is my strawman? Scam Altman himself has said this, if we ignore the inconvenient parts of the business that loses all that money, they'd be super-duper profitable.
Anthropic said something similar, claiming to their investors they're profitable... If you ignore partner costs and the costs of training.
And mind you this is coming from a proven pathological liar who has a massive vested interest in over exaggerating his business' viability, so doubtless the "Very profitable" part is itself a massive lie.
Gross profitability always happens before net profitability, lol. "We've reached the first but not the second" isn't a deceptive claim, especially with the caveat in the same sentence, and it's a step that every net profitable company went through at some point in its life.
The correct quote is "If you ignore the costs that will come down in the future or stay fixed as userbase scales, then we are actually quite profitable"
Which is a basic truism of pretty much all startups, and nothing noteworthy or insightful to point out.
Why would you expect training costs to come down in the future? It seems to me that the labs themselves have concluded they can't stop training due to competition, which is why they're now asking the government to step in and shut down their competitors.
How can their revenue numbers match benchmarks from their inferencing code? Are they really claiming that they sell 100% of their capacity at all times, 24/7? Wouldn't they require a lot of queued batch traffic to achieve that? Or are they defining revenue in some weird way where what they mean is potential revenue?
I assume this is idealized somehow, given how close it is to DeepInfer numbers at 100% utilization. My point was more that there is seemingly a lot of margin to absorb underutilization, outages and other costs.
Given that they open sourced their inference framework, how realistic would be to buy hardware and become a provider on OpenRouter? These numbers are incredible.
Based on translating their Chinese text on Firefox, it says "per card":
> The current service throughput of a single cluster is 155 trillion tokens per day, and the overall revenue is $38 per hour (at the lowest price) per card.
With Kagi translate (best), it says GPU
> A single cluster currently serves 155 trillion tokens per day, generating overall revenue of $38 per GPU per hour (at off-peak electricity rates).
Free markets help the big producers much more than consumers.
The Peter principle doesn't apply to consumers, but it does to producers. And if nobody regulates how big you can get, you eventually end up with a nothing but ruthless monopolies in every industry. Regulated markets tend not to generate as many monopolies. And of course, for natural monopolies, the state is the least bad option, because consumers have at least some say in how its operated through voting, and there is no profit motive.
...but in practice there are a few highly suboptimal Nash Equilibria sprinkled throughout the market that are systematically unaddressable by greedy optimization, and these turn into an infinite money glitch for the rich and an infinite resource drain for the poor.
TINSTAAFM. There is no such thing as a free market under capitalism, as the first winners - which there are, even in a perfectly free market - will put their resources into making it not-free, as we're seeing here. These advantages accumulate and you end up with a few massive monopolies which the market exerts little control over. Not zero, but little, much less than most economists claimed for close to a century.
Keeping a market free would require magnitudes more regulation than right now - the opposite of what free market proponents advocate for.
Given this is obvious to mere mortals like us, how do economists defend the walmartization of every market? "It's working, don't fckit up", or "we're getting our bag, get yours"?
There isn't one adult who looks at this massive centralization and sees that it goes against the very principle of the thing?
Dozens of different religions have millions of followers. They're generally not compatible with each other. Many of the followers incredibly intelligent and accomplished in their fields.
I'm not saying this out of anti-religion, I believe religion can be a force for good and often is. But it shows that yes, this is possible. Also see things physicists, and especially physicians, believed for centuries.
> There isn't one adult who looks at this massive centralization and sees that it goes against the very principle of the thing?
There's a growing number who do. But the reason why they didn't get much attention for most of a century, and as such what made the fantasies of Hayek become mainstream, is of course tightly connected to the fact that it's the one that just "happens" to be the one most favorable to these mega corporations and their friends in financial and political institutions. It's clearly not a coincidence at all.
It's also just an incestuous cycle of professors teaching students who end up being professors who teach students. Takes something to break that.
I'm not an economist but it's pretty obvious to me that walmartization is downstream of Financialization which is not inherently free market (its a product of centralized banking and interest rate and bailout policies)
I don't see why Walmart wouldn't get this big even without centralized banking, bailouts, and hard money (say gold convertibility), thanks to economies of scale. In the late 1800s you had Standard Oil. That didn't need any of those things to arise.
Of course the things you mentioned further accelerate walmartization. But that's the chicken and the egg. The government does bailouts and prints cheap money because the corporations do exactly what I said: the winners (who arise naturally thanks to economies of scale) will put their resources into gaining these things.
sure. It's less likely let's say. More opportunity for competition to arise per business cycle when growth is still there but has natural drag due to money not being cheap.
That would be like regulating who gets access to the best food on a sinking ship. The USA could allocate capital and attention toward much better things, for the country and for their people. I just hope the next couple administrations are wise enough to not experiment with the idea that ASI can get the country out of this 'fall of Rome' slow motion decline.
> You can’t show ROI to investors by cutting inference costs
You can as long as you can saturate hardware with profit that makes up for training costs (which are also in free fall).
Anthropic is most at risk since they haven't bought any hardware (afaik), however OpenAi is building custom silicon now. The rest of Mag7 are primarily hardware vendors
Any sources on training costs in free fall? My understanding was training costs are proportional to model size. Is it just that compute costs are falling?
> I believe that’s post-training, not pre-training
Aren't AI companies already butting against the limits of usable training data?
Outside of undiscovered techniques, additional costly pre-training will likely yield diminishing returns if you've already scraped the whole internet and destructively scanned as many 20th century books as you could lay your hands on.
most training data is not of that nature, that's "pretraining" input, it's now much more comprised of agent traces in mid/late phases, which are expanded each generation as we push what we do today with the last generation into the model itself, expect more team training in the next round
Training costs are not in free fall, and not every "post training" process is equal, Everyone does this differently. The costs certainly haven't gone down, to train a new 10T param model from scratch, like at all.
You cannot point at Xiaomis tiny model and compare it to training a 10T param model. The levels of dunning krueger are getting out of hand.
You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
> You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
There were probably a few dozen 20 years ago, and the existing pool grew from somewhere. AI (nee machine learning) is not rocket science - but it is a science, so it rewards those who are literate, curious, can hypothesize, experiment and learn from results to gain an intuition. I bet most statistics-adjacent professionals (millions, worldwide) can move laterally into AI research and excel at it. Nevermind AI has been among the most-in-demand undergrad programs since ChatGPT dropped. The only limited pool is for celebrity AI talent.
you haven't cited any examples, nor acknowledged this is how everything in computing has gone, DK is a two way street I suppose
stable diffusion is an excellent example, the price to train today's or yesterday's frontier is the analogy, many more people will be able to train up what we have today, especially as more of the pipeline is open sourced and the knowhow distills
>stable diffusion is an excellent example, the price to train today's or yesterday's frontier is the analogy,
Dont use llms to draft your responses to people, extermely cringe. That explains the llm induced Dunning Krueger.
What examples... There are no public numbers regarding how much it costs to train a 10T model, atleast nothing concrete.
You should just use common sense and think about how you have to fit tensors in memory to train, and the larger the model is the more GPUs you need. Then you could also use common sense to think about how GPUs have gone up in price, not down.
> Dont use llms to draft your responses to people, extermely cringe
You are throwing around unfounded and incorrect accusations. Why? It's a weak ass rebuttal mate
common sense is using things like history to guide expectations instead just saying things without evidence that are contrary to the patterns in computing history
I think that the frontier labs knew what they were doing when they invested in cybersecurity starting in the spring (or maybe earlier). They knew it would create hype. They knew it fit with their narrative that they are building a dangerous thing (something they believe because it’s fun to believe in doom, because it’s a great story for investors, and because it’s not inconsistent with a moat - in fact it makes a moat via regulatory capture).
Regulate the bad actors, not the neutral tools. Lockpicks aren't illegal, but breaking and entering is. Imagine if MasterLock used a lockpick to break into a government building and steal stuff, then tried to lobby to make lockpicks illegal. That's what OpenAI is doing, and you're falling for it.
In the hands of 4th party system Progressive Republicans and 5th party system New Deal Democrats, government was an extremely powerful force for trust-busting.
In the hands of 6th party system Republicans, namely Ronald Reagan and Robert Bork, the government worked hard to usher in a a new era of Robber Barons. We live in the culmination of this project.
History says the result depends on the hands, not the tool. 6th party system Democrats have largely failed to undo the Reagan/Bork policies, but that's an act of omission not comission and even then Lina Khan was a big step in the right direction. This issue was on the ballot in 2024 and will be again in 2028. Hopefully we make better choices in 2028.
If you are going to limit the 5th Party system to the first half of the long 1930s-1990s period of overlapping realignments (which is pretty reasonable), then the 6th Party system should be the second half of that period, and the period from about the end of the Clinton Administration on should be the 7th Party system; you can call now the 6th Party system if you call the entire long realignment period the 5th (which makes some sense, too, though there is a pretty significant difference between the New Deal dominant period and the Civil Rights dominant period of the realignment), but to split it somewhere in the middle but then lump the period after with it makes little sense.
If you want to call Obama and Biden 7th party system, I don't really agree but I also don't think it matters for my argument: we can just re-label them and move on.
It is pretty funny you refer to these periods because they are the quintessential example of big business pushing for regulations to push out their competitors and become monopolies. And the "progressives" enthusiastically supported such monopolization.
Regulated monopolies are much better than unregulated ones. Free markets generate the latter, and they have created most of the big issues we have today: inequality, climate change, the military industrial complex.
This is so vague as to be impossible to pin down what you mean in order to refute it without you slipping out of it. But in very general terms, trust busting was a real thing that happened in this country once upon a time, yet would be unimaginable under the current ruling class.
That's wrong, but why let reality get in the way of a pithy ideology?
Government regulations exist to regulate activity, to prevent, punish, and/or remediate harms.
Environmental regulations exist because businesses wouldn't clean up their own messes. Financial regulations exist almost exclusively because of harms and abuses committed by members of the financial industry. Building codes are updated to reflect lessons learned about people died, to prevent more people from dying the same way.
If AI regulations are passed, it will be because the AI industry decided not to self-regulate itself and stop illegally hacking computer systems, generating CSAM, or various other illegal activities that should easily be preventable if their systems were actually intelligent.
If you take away all regulation you’d probably end up with a monopoly anyway, as companies would no longer be prevented from merging with their competitors or using underhand tactics to gain market share.
It's wild how businessmen will pitch you the 10 ways they will abuse the rules of the game to obtain a monopoly and extract rent, then they will turn around and say that the game rules are perfect and our only failing is not following them hard enough.
Also there are real trade offs. Food would be cheaper if, for instance, it were legal to cut it with any old cheap substance regardless of health effects.
But I totally agree that some regulations are dumb or are regulatory capture monopoly schemes.
Unfortunately like many other things there is no absolute simple rule to save us from thinking. Each case must be evaluated in depth hopefully with domain expertise.
DeepSeek hasn't proven anything except what they say. You're inherently trusting a company's statements over others. When an independent third party corroborates the numbers then we know for sure. Until then, every company is just selling a story for investors and users.
DeepSeek has released their weights so multiple companies can tell you if they're profitable, whereas the ones with closed weights, you will never know, even if they go public, they dont have to publish the actual margin.
As a now former Chinese investor, the numbers reported in China are almost entirely fictional. They are whatever the local prefect or senior leaders wants them to be. Right now they seek to imply to the West that they are competitive, so the numbers reported reflect that. Almost no Chinese company permits independent audit of any of their finances.
It’s a rabbit hole but if you have some time it’s fascinating to look into the sentiment of investing in China vs the rest of the world. Because the numbers are so unreliable, investing is considered about the same as gambling in China. You’re more likely to lose a lot of money than win, and there is no way to accurately predict strong from weak companies based on their reported financials. Occasionally senior leaders will sentence the leader of very large failed company to death to attempt to maintain confidence, but it’s not working. This is one of the reasons for the housing bubble. There just aren’t stable investing opportunities in China.
Most publicly listed companies in the West require independent audits. Those audits are published for everyone to read. The listed company and the auditors are all tightly regulated, and while fraud can happen, it's relatively rare.
I think you're making the mistake of thinking that because fraud occasionally happens in the West, it's on anywhere close to the same scale as China. If Chinese companies were capable of passing independent audits, they'd all be publishing independent audits. That would be a massive confidence booster in China and internationally, and stock prices would rise significantly. They don't, because they wouldn't pass.
While I disagree with PC's insinuation, but in China there's just one thing that carries legal/fatal/etc risk for a large company and lying to public or investors isn't that.
Does DeepSeek offer a heavily subsidized subscription that provides 20x+ more usage than the equivalent value in API tokens?
API usage being profitable is completely plausible, but almost nobody pays API prices for Claude and GPT, nor would they be able to afford them if the subscription suddenly stopped existing.
If you're actually asking the question and not being sarcastic, no, Deepseek does not offer subscriptions at all, subsidized or otherwise. You only can use them at the API rates.
In both places I've worked at in the past two years (F500), subscription plans were banned due to data privacy concerns. Everyone _had_ to use the provisioned API tokens (even if they had a personal subscription)
Not DeepSeek themselves, but their open models are run by third-party companies in a profitable way. The unsustainable interference theory has been debunked.
Define profit? If by profit you mean covering compute costs then yes you're correct. If by profit you mean covering spent RnD budget and future RnD budget allocations then no, by no means are any of these AI companies running at a profit.
Also these third-party providers are running magnitude smaller models than what Anthropic and OpenAI's power users expect from them. Yes the average chat message isn't being routed to the 1+ Trillion param model but people running CC or Codex on Ultra probably are. Those are objectively more expensive.
I am citing out of my head here, but think deepseek’s inference engine with their models create around 30 dollars per hours while it costs around 4 dollars, there might be other costs but if you look at the demand, it will become very profitable.
I don’t understand why people are pessimistic about the profitability (I don’t want them to be this successful either). BUT why would they care about profitability right now. They believe in AGI, so do the big money people. They get their investments. This train is not stopping and I believe their predictions are correct (IMO we should not debate about timescales apart from spreading urgency).
And for the lowering of prices, it is possible because they just can and hooking people now into their systems will be 10x more profitable in the future. In addition to all the juicy training data they get with real world clients.
They are scared the others win, they are not scared that they run out of money.
People are pessimistic about profitability because it's basic economics. Pricing power comes from advantage and nothing else. In particular it doesn't come from demand because in a well functioning market supply will rapidly increase to meet demand.
The little companies inferencing DeepSeek have no advantage over any other company doing the same thing. Therefore, their margins should look like any competitive industry. Farming, supermarkets, airlines are my go-to example. Usually margins in the 3% to 5% range. Profit margin average across all industry including very profitable ones like tech or oil is only 10%.
The companies offering inferencing and nothing else are relying on subsidies by the open weight model providers. If those companies all stopped releasing open weights tomorrow the offerings on OpenRouter would see rapid traffic declines as the knowledge in the weights became stale and as other companies that invested in proprietary R&D overtake them. So being profitable on inferencing open weights is a very short term play, and there's no reason to think it's a great business in the long run.
I got our pocket god ChatGPT to fact check this discussion. No lab is profitable, it is a moonshot to expect they'll ever recover the billions frontier labs have put in behind these DCs. I never understand this argument everyone tries to make, AI is too expensive -> It will get cheaper as they scale -> AI is too cheap and not sustainable -> No it's very profitable on API -> go back to 1.
I am not an AI denier, I use it and I find it very useful. That said I can disagree with the current economics of the industry and their business model.
Inference economics only work for those who build on top of the models lol. It should be said that Anthropic and OpenAI suck at product development too - claude for finance, Sora, OpenAI web browser - wtf?
Its difficult for people to accept this because many want the model producers to succeed badly.
They are selling inference at severely gimped capability. Even in openrouter the quality and speed of the model being served varies by provider. When you dig into the details (which most people don't do), not all providers are equal.
The conditions for DeepSeek's profitability are very very narrow. A key piece being that they have cheap energy, a second key is that they have west/east clients and thus extremely high uptime.
Energy costs (and almost everything else) are a minimal part of providing inference.
Most of the cost is depreciation on the GPUs and the HBM.
The one number you need to figure out is how much it costs you to buy that GPU and HBM, then how many tokens you can sell from it over its lifespan which is generally amortised over 2 years.
Maybe Sam Altman is like robin hood. Taking from the rich (investors) to give to the poor (everyone else). The poor win by having cheap models (chinese) which were probably trained on data from ChatGPT et al. Of course he can't say this out loud or he wouldn't get any money from investors. In the end, if OpenAI goes bankrupt who else besides the investors lose money? Who benefits?
Because they’re under tremendous pressure relative to open weight models. Market forces are driving the price of AI to only slightly higher than the underlying cost of the compute inference. Thats the sensible place for things to settle but a terrible thing if you’re trying to build a business on the idea that the model itself should have a change associated with it.
In the short term they have to slash prices to avoid losing market share. In the longer term they need to figure this out as the present path simply isn’t sustainable.
> One outcome that would be genuinely ironic is that if these models are more compute efficient, it reduces the need for these massive data center buildouts, which could cause cascading economic failures. However, that likely won’t happen since inference is only one part of the compute mix and those data centers are still needed for other parts of the pipeline.
Don’t agree with the however. I think like you’re saying we’re going to get to see in real time what happens when something is commoditized and it gets massively overbuilt.
A historical episode that may arguably be related -- regarding stock market euphoria & overbuilding capacity -- was the dot-com era Telecom bubble & crash [1].
Even if something is a reasonable infrastructure project with a sound long term business plan, investors can still lose money if they pay too much for it. For the Telco bubble & crash, there were a few of these Telco companies engaging in financial shenanigans / securities fraud to juice their quarterly financials by leasing capacity to artificially inflate their operating revenue etc. E.g. we booked $100m of revenue by leasing X units of our capacity to $competitor! Sure, we're still extremely cashflow negative, but look at our operating revenue grow! Fine print: we also incurred a symmetric $100m expense to lease X units of capacity back from the same $competitor, but for some reason we didn't classify that as an operating expense, we've thrown it into a different accounting bucket.
That said, overbuilding fiber optic capacity by laying undersea cables is likely different to overbuilding GPU datacenter capacity in that most of the cost of laying a cable is the physical cost to lay the cable -- given you're already laying an undersea cable, the incremental cost to add a bit more capacity to it (more optical fibers) is negligible -- at least before you've sunk the cable! Whereas for a data center full of GPUs, maybe 50%-60% of the total cost is the GPUs, so it isn't cheap to massively overbuild capacity.
The entire endeavor is ultimately fundamentally coupled to reducing compute
Eventually we will compress models further by removing outdated knowledge like Python and Cobol cause who needs 1960s like languages to then layout data in the screen when the model has knowledge of geometry and can mutate the screen geometry without needing a Python interpreter or C compiler
On the contrary, demand compounds with every incremental improvement of model capability. There was no price that justified using LLAMA 3 70B for today's agentic use cases. That said, there is a price where it doesn't make sense to buy the next GPU.
It’s probably more efficient caching methods and smaller models (6 sol is actually 6 terra and opus 5.5 is smaller than 5.1 etc). Don’t overthink this please
If that were all there was to it, they would just pocket the profits. They’re cutting prices because they need to stoke enough demand to pay for their ever greater and every more costly capacity.
Well they did pull back on ZDR a few times and at least Anthropic has made their arguments (that I disagree with) that model outputs, no matter how much user context was fed, do not fall under their definition of "user data".
Are these prices subsidized by user generated outputs?
Can I even trust any answer to not be carefully worded and rephrase the response to "user data"?
Even so, what's to prevent some "oops" event that seems to be all too common?
At least some months back Bedrock seemed to be properly isolated, but I've heard of many getting non-ZDR notices for new models.
I fully expect token vendors to go the way of mobile data providers. Increasing throughput and efficiency, yet commoditized and not reaping the massive rewards of the economic gains their technologies enable.
Can't wait for the unlimited plans we will have in due time.
"Unlimited" for a consumer will probably take a form like limited parallel agents at max intelligence for X amount of tokens, followed by either single threaded agents or nerfed intelligence or both.
This is the same mechanism used in unlimited mobile internet for example.
> Can't wait for the unlimited plans we will have in due time.
I don't see how this can happen. The cost of serving a model grows linearly with use (energy). The cost of serving a mobile data plan is approximately zero.
I'm curious, why would this not turn into an unlimited electricity flatrate? In most parts of the world, consumer electricity is charged by kWh, not as flat payment.
As long as tokens are as energy intense to compute as they are right now, I don't see a flatrate being a viable model.
Something close to how we rent other cloud resources today makes sense. We don't worry about electricity costs, it's built into the hourly prices, use as much of the hardware as you can. Electricity rates vary over the course of the day as well.
People don't like complex pricing. This would be for businesses running larger workloads. For individuals, the unlimited plan seems the most probable to me. This stuff is going to get real cheap in the coming years.
The energy needed per token is in dropping rapidly, even though the energy per-GPU is going up. We are seeing something like 10x improvements per year.
> leaked financials from both OpenAI and Anthropic show unsurprisingly that they are losing billions due to R&D/scaling
No, they are losing billions because LLMs are not a viable business. Their real business is making bombastic statements which the credulous press prints unquestioning, fooling various investors and ... that's it. IPO is another of these. You really should note how the Anthropic IPO date keeps slipping because they can't make the S-1 public as it would give away the game immediately. The market was fooled by SpaceX once you can't do this a second or third time.
Are those statements audited or fact checked? Inference profitability is great but how much of that profit gets eaten up paying back training costs, R&D, and operating expenses.
No, and it’s very rare for Chinese companies to allow independent audits. I would not trust DeepSeek financial disclosures. However from industry data we can see that inference is often profitable. It’s the training which is very expensive. Which is not a big problem for Chinese companies since to date much of their training is just distilling frontier models.
Their models and inference frameworks are open, you can run them yourselves to check how many tokens per day you can generate, and then validate that against their API prices.
Not sure how they can get more transparent than that.
My point was really about profitability in general. The inference portion might be profitable but if training and operating costs are very large, then that can make the whole thing unprofitable. E.g. if it costs you $100B to train a good model but you make $0.00001 per inference token, you may not be able to hit breakeven for a very long time.
That's conflating different things. Unless you believe you can instantly saturate any amount of GPUs you bring online, 24/7, with paying customers from day one, then theoretical max throughputs on a stack isn't the same thing as revenue generated. Customer acquisition costs are real!
Even if you could acquire infinity custom on day one for zero cost, revenue isn't profit.
From their own inference framework docs: "A single cluster currently serves 155 trillion tokens per day, with overall revenue of US$38 per GPU-hour (at valley pricing)."
If they keep it running at near constant usage, you'd only need 80-120 million tokens an hour to hit 38$ of revenue (at off peak pricing, half as much for peak pricing).
The cool thing about models designed for cheaper inference is that post-training (which often requires inference as a part of the feedback loop) becomes cheaper as well, so almost certainly cheaper than models that are less optimized.
The market has decided that SPCX is worth $147, as of Friday, 5pm. Not sure who's fooling whom, but that's the market speaking. What that says, you'd have to ask a financial analyst, but that's where it is. It IPO'd at $135.
SPCX had a pretty wild ride so far with a jump to 200 a few days after IPO and a drop down 108 a few months ago so I'm not sure the market really knows how to properly price SPCX right now.
LLMs are absolutely viable businesses. There is just smaller and smaller share of the money going to the absolute frontier because the open models are "good enough" for more and more tasks.
But burning staggering amount of money just to keep yourself on "frontier" is not viable when the part of the market it gets you get smaller and smaller.
The same evidence could just as possibly point to supply being too great for demand. The OP article references this as a distinct possibility: if models really have gotten more compute efficient, we may wind up in a situation where we have more inference capacity than we actually need.
To give a simplistic example: if a city as a ton of available AirBnBs, you might say, "Lots of people are willing to rent out their apartments, it must be profitable for them to do so." But the fact that all of those rooms are not already booked could also mean that way too many people have attempted to rent out their apartments.
And for what it's worth, it's possible for supply to outstrip demand while still leaving inference profitable. The question is whether it's (a) profitable enough to service massive debts and (b) sustainably profitable over the long run.
Edit: to add one note, it's possible for the LABS to be profitable on inference without inference itself being profitable. We don't know what the contracts look like, but it's entirely possible that labs received sweetheart deals from neoclouds so that the latter could flaunt big utilization numbers.
If inference wasn't profitable why are consumer GPUs that are only good for running open models so expensive? Why and how would these companies sell access to open models if it wasn't profitable? You can rent your own GPU and make money if your electricity is cheap enough.
Consumer GPUs can also be used for things like gaming, remember? And they have gotten so expensive because Nvidia has sacrificed that market in order to build more datacenter GPUs.
Selling at a loss is far from unusual: Uber did it to capture a market, farmers do it to recoup at least some of their investment rather than losing all of it.
>You can rent your own GPU and make money if your electricity is cheap enough.
I would be curious to know whether you'd make more profit from the same kWh of electricity mining bitcoin versus serving inference. I know they use different hardware, but since the "inference is profitable" discussion ignores the purchase price of the GPU, let's pretend you could have opted to buy an ASIC instead.
It’s both competition almost 3 way , cheaper open source , a clearly lagging behind a bit open ai that cuts prices to win Anthropic customers .
There was an interesting technical advantage by some models where Anthropic as a leader start loosing dramatically in speed too. 5.5 made a come back in that dimension.
I looked at the graph of price per token vs IQ and Deepseek is ~2.5 orders of magnitude ahead of similarly intelligent models, supposedly only ~2 when you factor in that it uses more tokens per task. Their data centers are profitable right now, and they use domestically produced Huawei chips with Nvidia as secondary option.
I watched that video over ten minutes and in that ten minutes several trillion dollars evaporated from my perception of the US stock market.
There's also that paper "DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression" which got released some time ago. My take is the breakthroughs got quickly implemented and helped drive the inference prices down.
> It’s no surprise both companies are trying to build moats to defend against this trend, such as a focus on enterprise commitments and personal agentic assistants like OpenAI Dots and Claude Duet.
This probably won't be enough, because Openrouter now has to do age verification to verify their users are adults, and if they don't, they have to apply "teen mode" to all users.
I can't believe the actual answer is so far down the thread. All praise to our overlord DeepSeek for ironically helping American labs make cached tokens much cheaper.
What model and effort are you using? With Opus 5.5 on medium and high I'm not anywhere close to hitting limits on my 20x sub. Meanwhile with Fable I hit it constantly
I am guessing something to do with Qwen, Kimi and Deepseek. Plus after the tokemaxxing wave died down, a lot of cancelled subs from enterprise? Just my 2ct.
Price war. Duh. There is no moat. Also them chinese competition. It’s going to be race to the bottom. It’s gonna be ugly for the stock market. Good for consumers though!
Solar panels added to a grid is not plug and play.
When you're connecting solar to a municipal grid (which is what most homes are connected to), it could theoretically be taxed or even subjected to a tariff. The "added" power (often excess) is monitored for billing credits/payout/other, among other reasons like grid integrity.
Solar panels are literally plug-and-play, though! If your market doesn't have plug-in "balcony solar" yet, it is because your lawmakers are blocking it.
And indeed, they can tax excess solar, but they can't tax the power you use yourself. Backfeed tax gets too high to use the grid as a battery? Buy a home battery, problem solved.
You can even buy a small solar panel for ~$10 with positive and negative leads you solder; and power something like a Pi Pico^ directly. It kinda feels like magic.
^Not recommended literally due to brownouts/etc, but a cheap power management IC on a breadboard can solve that. Alternatively, add a battery/etc.
Home solar panels and batteries are just like this except on a bigger scale; and with (hopefully) more QC.
This is incorrect. Treating solar panels like batteries for specific appliances (or integrated like the panel on a Solocam S220), is not the use case nor is this a discussion about off-grid or microhomes.
I would suggest not making broad statements about what can and can't be taxed in the future.
Plenty of grids have different sorts of net metering ratios, grid maintenance fees, etc. But it's nothing compared to what you'd be paying taking power off of the grid, and there is a break-even period often measured in single-digit years. I still do not understand the point of what the GP was trying to make w.r.t. "taxes".
Strange, I lived in very blue states (WA and CA) and they definitely punish EV owners with large annual surcharges on registrations. To offset “not paying gas tax”. Money is money, red or blue, they will all take as much as they can from you.
That would likely require innovation in a renewable liquid fuel that can work at scale.
Electrification will drive the demand for oil down, making the overhead more expensive per gallon, and eventually we will largely move off of fossil fuels, though I don't expect completely. It will be many decades due to the turnover times for industrial/commercial transportation systems.
Tokens per second? That only really matters for things like chat and sessions where you are monitoring a session as it runs which is going to be an increasingly small percentage of usage. That's bottlenecked by human attention. Agents in the background, long running tasks, unattended tasks, all that stuff is what is even capable of driving significant usage because you can just increase consumption massively by parallelizing without being the bottleneck. I wouldn't be shocked if already the vast majority of token usage was from unattended sessions. TPS is not that important there
You're ignoring the fact that a waiting human costs money which is actually the primary reason why AI hasn't resulted in significant productivity increases, while overly focusing on the human being the bottleneck. The point isn't to monitor the session, the point is to verify the output with human inspection. If the agent is fast enough, attending to the agent can make financial sense.
Human memory has a finite capacity and decays over time. If you do many inspection tasks in parallel you are splitting that finite memory. If you do the same tasks sequentially, the human can give instant feedback to the model without costly context switching. This in turn increases iteration speed and reduces total labor costs. This is no different from developers preferring JavaScript or Python for immediate feedback and fast iterations.
It is impossible to make sophisticated software products at a high level with a low TPS. Try making a modern emulator or an award winning game. Both require a lot of iteration and not everything works well with multi agent.
DeepSeek is serving more TPS than anyone, about as much as OpenRouter does in a week in a day, at 38$/H800/Ascend so about 900k/~90% profit over 3 years (or ~80s% profit including training expenses).
Meanwhile, Oracle is handing Tencent 100.000 GPUs at 1.60$ hour, dangerously close to breakeven...
Doubt the race to the bottom is really good for consumers, unless you actually just want more stuff rather than better stuff. The bubble bursting is going to be pretty horrible financially for us all.
The main thing it seems to have done so far is made everything feel less trustworthy, and everything digital less safe. It has empowered scammers, that's for sure.
And for what benefit? Spunking out 7/10 code a bit faster with your brain turned off. Great!
It won’t be a ban, it will be some “certified safe” badge from a federal agency. Companies will be scared into not using anything that is not “certified safe” and business insurance will even require it. Whoever (openAI or Anthropic) gets to the politicians first will be the only certified safe model provider.
Maybe the FDA is a good analogy, you wouldn’t want to buy meat at the store not FDA certified would you? OpenAI and Anthropic want you to feel the same about AI.
I do not expect an outright ban. (1) the bots are too useful, open is the way, re: HF handling the incident (2) you can train without certain capabilities, see Kimi K3.1 (3) reminiscent of the encryption is dangerous debate of the '90s
There will likely be requirements like registering with the government (China already does this, every business already does everywhere), certain usages require more or have deeper regulations (this is already done with most technologies, depending on application)
I think it will not technically be a ban, but a de facto one.
Some variation on, "Open weight models must be A) trained in the USA B) have a public training dataset + training pipeline C) be reviewed by a government 'safety council', etc" Something which technically leaves a crack open for someone to release a less capable model than the commercial ones.
That kind of policy will hold a country back, and why I don't think it will happen. There are already many companies already using Chinese models.
Another confounding factor, if a US company fine-tunes a Chinese model, or vice versa, what is that country that should be assigned to that model.
You also have to consider the competition between the US and China. If the US did something like this, the world will move to Chinese hardware and Ai. The think tanks that talk with legislators understand this.
> If the US did something like this, the world will move to Chinese hardware and AI
Hardware? Why would anyone change their hardware based on this? That's completely unrelated.
And as for Chinese LLMs - there's already evidence that the Chinese are training malicious behavior into their models. I think most countries would rather train their own models than risk that.
You have more faith than I do. It would be bad policy, but hardly a first for a government who wielded tariffs like a cudgel in spite of obvious repercussions.
honestly the surprising thing at this point would be if america did actually pass a policy of any sort, good or bad, vs just relying on the president to yolo some executive orders now and then that change every 2 years. Plus at least for the next couple years nvidia likes open source models and the president likes the ceo of nvidia. There could be a falling out but nvidia has deep pockets to ... discourage ... a falling out.
It's worth remembering that the tariff stuff is not any sort of durable policy. It's one guy sort of doing whatever until 9 other guys a year later go 'hey wait' and then the first guy does whatever again anyway. The whole thing is just a recipe for whiplash. maybe congress remembers that they're supposed to do stuff in a few months and steps in. maybe they don't and then every 4 years we have wild swings in enforced based on which half of a percent of the voters in georgia had rainy weather where they lived in early november. More of a simultaneous "All of the above and also none of the above" deal until the dynamics of american government fundamentally shift
There are people in Trump's inner most circle that know this. I'm not worried about it. People trust those calling for regulatory capture (Dario / Altman) the least. (HN is not a good representation of the general population and AI userbase)
NVIDIA would love it. They themselves make available open weight models trained in the USA with open data sets, because they don't really care whose models are used as long as they run on NVIDIA hardware. The idea that only China makes open weight models isn't true, there are many other providers and some of them are quite good.
> It turned out that GPT-6 Sol was a dud performancewise, so a week later OpenAI announced GPT-6.1 Sol,
I strongly believe that GPT-6 Sol was only ever meant to be the web chat model, but it was a panic public release to respond to Opus 5.5. It's clearly been over fitted for drawing diagrams in the chat.
It's all a fugazi, as they don't tell you how many tokens you get, it's dynamic. But yes everyone with a Codex subscription felt the token quotas getting a huge cut about 2 weeks after Astra release.
>They know what they’re doing. The naysayers are like the people that said this little internet thing wasn’t going anywhere.
There's a big difference between people saying OpenAI/Antropic wont survive and LLMs wont survive. A lot of big early players in the internet space died off as well.
Whether you think they have a future or not does not change fact that they currently are wildly unprofitable.
>internet thing wasn’t going anywhere.
This is conflating the utility of the technology with the economics behind it. Few people claim LLMs are useless. It's a question of figuring out a viable business model.
they are not horrible business, some of them literally get invested by governments and get extra debt from them too so they can never go bankrupt? if they are at national level airlines, they survive no matter what.
that is not bad business? that is just private profits, socialize loses perfect business?
No one personally uses API prices. If you need to do something yourself, you need a subscription. Ans subscriptions shinkflated at the same time, but a lot.
It doesn't really make sense huh. If they're so resource constrained and tokens are so precious, then prices should go up regardless of how much optimization has been done. You're either struggling to meet demand or egregiously over-provisioned. Which is it?
though problem with proprietary LLMs it's difficult to know if the vendor's supposed "optimization" affects performance negatively because it's difficult to define an SLA
Well, because after 2011, when Anderseen famously said "software is eating the world"... it only ate so much of the world. We have that data on verticals now... is it possibly the same thing for "AI?"
Genuine inquiry here.
However, my bigger worry is this is just a temporary lull, and that due to alignment, regulatory, and distillation pressures, the best models will be kept internal, and then all of capitalism fails because the labs could maybe eat all of knowledge work.
There are already no junior hires. The ladder is already collapsing. Embrace, Extend, Extinguish.
I don't want to crash the party, but there is also a non AI-bubble-is-about-to-burst way to read this: If producing something becomes cheaper (e.g. more efficient inference techniques, cheaper compute etc.) and there is some competition, then typically prices go down as well.
Competition drives prices down. We tried Kimi and GLM models after hitting 'cybersecurity' blocks on OpenAI and Anthropic. We found them to be much cheaper to access and helped us complete our work without 'cybersecurity' refusals blocking us.
They just started competing against each other? No.
Anthropic is going public soon. Until September, OpenAI was going public. Neither are remotely profitable.
They're not competing against each other, they're competing against free/local/open weight models that they're also lobbying very hard to regulate/ban.
As much as I hear this sentiment, and as much as all the smart software people around me echo the desire for this sentiment to be true; it feels like people say these things, but then at the same time say - well, Claude/Chatgpt is still marginally better for me now. But clearly they are catching up so maybe one day!!!
In that sense, I'm really not sure if Kimi/chinese models are really competing against Anthropic/OAI? Not to mention that for individuals, the subscription model is extremely different, and Kimi subs are (I think) worse value, and chinese API's maybe marginally end up cheaper than using american subscriptions.
The profit comes from the enterprise customers who have to pay for usage at API rates. Not the subsidized subscriptions, which both OpenAI have designed to be unavailable for large enterprises by withholding IAM and auditing features they need except on the usage based entperise plans.
As an individual I'm still paying for a ChatGPT Plus plan since $20/mo gives me a lot of value and I use many multiples of $20 at API rates per month. But in terms of capabilities, I am entirely satisfied when using my Opencode Go plan with open models like Deepseek Flash/Pro, Kimi and MiMo.
At this point keeping my ChatGPT subscription isn't because of a significant capability gap compared to open models. It's 90% about the value of the subscription priced well below API rates (10% the fun of trying latest models).
If OpenAI/Anthropic dropped their cheap subscriptions or drastically undermined the value proposition I'd be happy using open models exclusively and currently barely notice a difference switching back and forth for the vast majority of tasks.
I'm expecting the subs to increasingly provide less value as the US labs IPO and follow the SV pattern of moving from marketshare capture mode to profit extraction mode. Thankfully I'm already comfortable using OpenCode with open models and avoid lock-in to any specific harness so would be a very soft landing.
I guess I don't understand "enterprise psychology" but why are auditing features worth a 50x premium? I think we're currently blasting through $1000 worth of tokens a month per dev.
Because enterprises have compliance and certification requirements, meaning the enterprise is essentially held hostage by providers on price.
You could argue it's worth that much because of the value that compliance brings the enterprise, but it's just another gouging excercise. Certainly nothing unique to AI companies.
They aren't but both firms tell bigger/richer clients that the subs just aren't available to them no matter what. It's not a policy, it's an email that says "we're moving you onto API rates". They have no obligation to follow a systematic policy!
Claude "team" plans are limited to 150 seats per company. So big companies have no choice.
> I guess I don't understand "enterprise psychology" but why are auditing features worth a 50x premium?
Alternatively, one can say that the enterprise customers plus a metric shitload of VC and shareholder/investor money heavily subsidizes cheap personal plans.
It's the same playbook that Adobe used, provide cheap plans for academia and turn a blind eye towards piracy from non-commercial actors, get the young people hooked hard on Adobe's software and workflows, and they will demand that their employers purchase the nice, high profit commercial/enterprise plans.
> heavily subsidizes cheap personal plans.
They are of course are the ones paying for R&D of new models but its not exactly clear if gross margins on subscriptions are negative. It might be just normal market segmentation just like SaaS companies charging enterprise users many times more than private individuals.
DeepSeek announced that each GPU they have makes about $38 an hour in revenue during off-peak times. That's $331k a year, and DeepSeek is cheap.
I think that it's not only possible but likely that the frontier labs are profitable on an ongoing basis, and it's mainly the research and capex which make them look like impending disasters.
Realistically, OpenAI and Anthropic are each probably worth ~$1T each. Anthropic can already easily justify that based on revenue alone. OpenAI isn't far off. Consider how quickly these companies are growing and the fact that they maintain a small but significant edge, I don't think the bubble is going to pop. Just deflate a little.
> I think that it's not only possible but likely that the frontier labs are profitable on an ongoing basis, and it's mainly the research and capex which make them look like impending disasters.
And that is the much more relevant problem. Capable open-weight models from China, European labs or competitors with cash to burn (i.e. Meta!) erode all the moat that the major labs have, and tooling functionality like MCP, CLI interfaces or integration into third-party tools like IDEs quickly gets cloned.
Simply printing money by running inference does not justify valuations in the billions of dollars, because anyone can do so.
Having a frontier model is worth something beyond just inference costs, I'm sure. It seems likely that the labs will maintain at least a small lead.
Let’s not forget that Luna max is actually extremely capable for the price, you can get a lot done on the $20 sub.
Haven’t tried haiku 5.5, I expect it to simply run out too fast, but they did change the price below 100k tokens recently; perhaps I should check it out.
Keeping a workflow under 100k tokens limits it to very tiny tasks
Openrouter stats seem to indicate that there is a lot of demand for ‘good enough at a good price.’ After all, not everyone is decompiling PS5 games. In fact, most people are just speedrunning notetaking apps in JavaScript frameworks.
But, Openrouter stats can only measure the usage they see. I'm not sure it is really all that representative of total LLM usage across the market.
I think that's the difference here -- OpenAI and Anthropic are competing more with each other (and Google Gemini) than they are with open weight models. While people on HN are familiar with Kimi and GLM models, I'm not sure how many casual users of LLMs know about them. I think the competitive risk open weight models pose to OpenAI and Anthropic is much further in the future.
There is a lot of usage that's only known by Anthropic or OpenAI that's not visible to the rest of the world.
And the share of the tasks where cheap open model is enough only grows in time.
Frontier models just fight over smaller and smaller piece of the pie and I suspect them pushing for security work is way to open potentially infinite token sink as attackers pay for tokens so defenders need to spend even more tokens to defens
> most people are just speedrunning notetaking apps in JavaScript frameworks.
I feel seen. But seriously, my app, with my preferences, written by a semi-trustworthy entity that isn’t trying to shove ads down my throat. That’s really great.
Yep, this. I've seen it coined as "the age of personal software" and I totally agree.
I've built easily a dozen personal apps in the last few months that I have no plans on releasing or sharing, run locally, and solve very specific productivity (or one-off) issues for me.
I would never have spent the time building these apps in the pre-LLM days because the effort wouldn't have been worth the gains.
But now? Sure Claude, build me a MacOS app in Swift that does this exact thing I need. Less than an hour later, it's done.
i mean, i made myself a dvd player for my steam machine. i don't see how that makes anthropic money at real token costs.
when lyft line was heavily subsidized in SF by VC money, i used it almost daily to commute to work because it cost $2.50 each way. Now that it's $20, I don't use it.
Psst.... that's called "dumping".
Yup.
Methinks (hopes) we're entering the era of mass customization.
I'm pretty much weaned off closed-weight models at this point.
It's a very "The Year of Linux on Desktop" situation, in the sense that in any particular year it seems like Linux is close, but the OSS stuff requires a lot of configuration to get to feature parity, and the barrier keeps rising because the prop solutions are running as fast as they can to justify the cost premium.
Unlike Linux, the world is pretty literate about tech business models, so I think there's a way for the open weight world to avoid the situation Linux got into, where it only really caught up once the market was pretty much commoditization any-ways.
If anyone interested in PL theory (including adjacent fields) and/or inference engines is reading this wants to help make open weight models and associated ecosystems feature-competitive, we're hiring, drop me a line: nathan at nfulton org.
> well, Claude/Chatgpt is still marginally better for me now
Enterprise doesn't care about marginally better. They care about cost.
Most organizations that run our self-hostable chat platform with cloud models are for most stretches running 6 months-1 year old models, and are strongly eyeing (or have already switched) to more commodity models for most of their workforce. Frontier models are mainly relevant for engineering tasks.
For enterprise customers the value proposition of self-hosting one of the open models is enormous. We're talking 10x cheaper (remember that enterprises pay API rates), plus more control on where the data goes (especially given that some frontier models do not support ZDR). And all those savings are lost revenue for the US AI Labs.
[dead]
My issue with anthropic and openai is that if I use a model like e.g. Sonnet 5 I don't always get full Sonnet 5, sometimes it's dumber, sometimes it is smarter (mostly dumber though), while GLM, Kimi and others stay the same. So if I rely on AI for things I better make sure the platform is stable and does not introduce hidden changes.
Why is opus 5.5 suddenly a lot dumber? Who knows.
Have you considered you are reading comments from different people?
Is Starbucks competing against Pete's Coffee, or are they competing against your home brew? Is it McDonald's vs Burger King, or all fast food against home cooking?
If prices were not being significantly lowered by Anthropic and OpenAI then you’d probably see more people using the Chinese models
The frontier labs think the switching threats are credible or they would not have cut prices.
[dead]
Of course they compete with each other. Codex and Claude Code have direct feature overlap.
But they do have a common enemy in open weight models.
Competition in a duopoly looks very different from competition with a large number of suppliers.
nah if not for open models they'd just raise the price in unison as any self respecting oligopoly would.
They'd compete on model quality, sure, but without competition from open models there is no reason to compete on price
Neither OpenAI or anthropic can afford to lose subscribers to the other or anybody else right before IPO. I tried both CC and codex recently, and I would say they are practically on par. Some things CC does better, some things codex does better. I've also seen them increase usage limits. The cheapest plan now feels very generous imo.
free/local/open weight don't have the compute available to serve on a level to compete with the big labs
Price per task has been dropping exponentially for years. This isn’t a sudden new phenomenon. They’ve been competing with each other continuously. There is even more competition now than ever before.
The brutal free market for job seekers like me but regulations and safety nets for thee
Yes and: the rules make the market. refs, adjudication, norms. markets free of rules is a tricky way to say might makes right.
If this is mostly competition enduced, I have to assume either their usage has plummeted or they will implode financially in short order.
The frontier labs were already burning money on more expensive price models, and competition wouldn't have reduced their capex proportionately.
How could they do anything but implode financially? Genuinely curious. The numbers are not right.
Did the frontier labs take out debt or did they make flimsy unenforceable agreements to neoclouds who took out debt?
Borrowing money is not the only way to go into debt. They reportedly made binding and non-cancelable commitments. Once those commitments come due then they will be indebted to their counterparties until they pay up or the courts sort it out.
This is comparable to how, if I sign a 12 month non-cancelable lease and then don’t pay the rent, I will be in debt to my landlord.
People heavily underestimate the number of tricks available to corporates for taking on "debt": https://latentstate.net/the-hidden-ledger/
[dead]
That's effectively borrowing money with a different name though. The only way to go into debt is to sign an agreement to pay someone money in the future, regardless of the terms used to describe it.
[dead]
No, it's really not. Do you have a war chest to drag out a legal battle and the ability to bankrupt your landlord by withholding payment? The rules are different at this scale, and "you can't enforce a contract if the attempt will bankrupt you" is only one of the possible games.
Here's another one: pay a supplier to cripple their deployment. Suddenly it's the neocloud's fault for failing to deliver, not your fault for failing to pay.
Did we all forget how Sam Altman turned a non-profit into a for-profit? Do we think he'll bring less sociopathy to this battle?
> Do you have a war chest to drag out a legal battle and the ability to bankrupt your landlord
Their war chest is not sufficiently larger than the hyperscalers, Broadcom or PE financiers like BlackRock.
> The rules are different at this scale, and "you can't enforce a contract if the attempt will bankrupt you" is only one of the possible games.
Rather, the inverse of what you said and far less Machiavellian.
It is legally well established that take-or-pay contracts are unenforceable if it would bankrupt the buyer which is the more likely resolution in this case. However, this does not mean frontier labs just get to walk away. They would settle in court for a penalty that is less than the full contract price.
See: AMD/global foundries and pretty much every oil and gas or commodity deal that goes bust when market pricing makes extraction unfeasible.
> Their war chest is not sufficiently larger
That's not the metric. The legal system is pay-for-delay, not pay-to-win. Once you have enough money to delay, it becomes a contest of who has the longer runway, and that probably goes to the frontier labs rather than the neoclouds who have little alternative revenue and comparatively airtight debt obligations.
> take-or-pay contracts are unenforceable if it would bankrupt the buyer which is the more likely resolution in this case. However, this does not mean frontier labs just get to walk away. They would settle in court for a penalty
Regardless, the terms come down to positioning and bargaining power. Runway vs obligations. This seems to dramatically favor the labs.
> it becomes a contest of who has the longer runway
Which is what I said - frontier labs do not have a sufficiently larger war chest.
Respectfully, comparing the balance sheets of a frontier lab to Blackrock, Apollo, Softbank or Nvidia is laughable.
The neocloud will also easily rent out that unused compute and keep funding their war chest via Nvidia’s backstop. If Nvidia can’t pay then it’s all over and AI companies have been swallowed by credit markets.
Your hypothesis also ignores the fact that if frontier labs are not paying for compute it is because they have bad revenue, little cash available and no credit (no one is loaning money to OAI or Anthropic if they can’t pay their bills. They can barely get debt today).
> Regardless, the terms come down to positioning and bargaining power. Runway vs obligations. This seems to dramatically favor the labs.
No, the only way out is for OpenAI or Anthropic to claim in court that they would be insolvent if they are required to pay the bill in full. Otherwise this is a straight forward deal.
You are greatly overestimating their capabilities and underestimating how many substantially more well-funded and highly liquid players will be lining up to take their pound of flesh if OAI or Anthropic default on a bill.
I’d wager my life savings one of those two will go through liquidation.
Does it matter if they "win" or "lose" legally ? They still don't have money to pay, so someone (probably people that gave them their debt in the first place, and their investors) will end holding the bag.
[flagged]
Yes.
The government could bail them out, that's the most likely scenario if they don't implode IMO.
They could also continue to grow at the rates they had last year. I don't know where those customers would come from, but if they did pull that off OpenAI and Anthropic could sustain propping up the circular financing going around the AI industry.
Goverment bailing they out will be using people tax to rescue the companies that absorbed most of the 801k money so they can get people out of work... it's the ultimate slap to the face... I would strongly resist this, I would vote if they implode to give to whoever owned shares via 801k their money, to people and let them (openai, anthropic, et al) eat shit... I would love to buy one of those gpus to run an open model at discount price...
Completely agree it will be a huge slap in the face to the public, unfortunately that's a common solution.
When everyone is afraid the market will collapse, or more likely distracted because the market collapsed just enough to hurt the average person, the government will bail them out and either people won't notice or will welcome it as a solution to huge levels of fear.
Voting isn't usually much of a solution after the fact, the political machine doesn't allow for candidates that would fix anything to have a chance at any office that matters. There's a reason our politicians today all seem to have roughly the same goals and all seem to become extremely wealthy while in office.
Government trying to manage financial markets is like trying to steer a rampaging rhino; finance is stronger than government, and can wreck it. But some politicians want to increase regulation and some want to reduce it. Some cynicism is fair, but you've overstated the case. Elections do not offer infinite choice, but the choices are distinct.
But why would they bail them out ? Stock value will drop regardless of what govt does and having the borrowers (the private companies that already made billions on selling the shovel) get stuck with the unclaimable debt is far better option, and those are not hiring average blue collar workers in hundreds of thousands so any job loss would be minuscule too
For the same reason the government bailed out the banks, they're too big to fail.
Two companies account for most of the frontier R&D, and something like 7 companies account for a huge portion of growth in the stock market. The government, especially under Trump, is very focused on continuing to see those numbers go up.
I also expect the government is pretty bought in on AI being the future and on us being in a race with China. They'd have to bail out the labs for those reasons alone.
Lastly in my opinion, Trump has a pretty socialist bent to him and regularly talks about a sovereign wealth fund owning portions of companies. He'd love to have take a partial stake in AI companies, and a bail out is a great way to do that.
A bailout would be a slap in the face when the correct solution is to nationalize them. If they are too big to fail then the government gets a controlling share of equity and we finally start the sovereign wealth fund we should have started decades ago.
Nationalization is fundamentally different from a bailout. It tends to go poorly when a government begins taking over the means of production.
Though I agree with you a bailout is a slap in the face. In my opinion bailouts shouldn't be legal, governments should pro actively be making sure individual corporations don't become so large that a failure would wreck the economy or national security.
Why is it you see a sovereign wealth fund as the right answer? Why should the sovereign own the means of production, and why should the people creating and enforcing laws be the same as those in charge of our economic future?
How does that work when the official strategy is "Socialise the losses, privatise the profits"
Maybe you should try voting harder.
Meaning what? Cast more than one ballot?
I live in a state that votes heavily in one direction, most states are the same. Its fine for me to vote, but the winner of a majority of states is pretty well decided before a ballot is cast (by strong majority, I'm not claiming fraud).
If everyone who threw up their hands actually voted, the outcome would change. See also, the elections musk tried to buy in the last 18 months or so since he doged up the place.
Sure, and I hope that (a) everyone votes and (b) most of them agree with me. Bit what is your point exactly?
Point is, nothing changes if you're just sitting on the internet complaining.
Becoming "too big to fail" has become a viable market strategy.
See also how AI companies re trying to sneak in basically every level of infrastructure and public administration.
Apple is worth more than all the AI companies combined. You think LLMs are less valuable in the global market than slightly updated iPhone models every year?
LLMs? No. LLM companies? Yes. It's just math, data centers, and mass copyright infringement. The only scarcity is Nvidia, who just passed Apple in market cap, somewhat proving your point but not in the way you meant it.
If you consider Nvidia an LLM company, and I do at this point given how heavily they've invested in frontier labs and where much of their revenue comes from, they alone are worth slightly more than Apple.
But nvidia can sell to anyone, right?
They'll take a heavy hit stock wise, and might be an LLM company through all their shady financial constructions to keep the music going by propping up demand. But in the end, nvidia as a company has a place even when the frontier labs go bankrupt. Some other company will take their place. Worst case it'll be like the cable companies in the end 90s who found themselves out of work because they overbuilt for dotcom
> It's just math, data centers, and mass copyright infringement.
Ok then iPhones are just metal, plastic, glass, and a delivery truck. You are just electric meat. This has to be one of the dumber comments I’ve read on here.
Maybe not, but it is probably less likely that Apple will face stiff competition from freely available alternatives, given that AOSP has been a thing for the iPhone’s entire lifespan and nonproprietary Android phones have failed to capture significant market share.
I don't think we have seen an Apple of AI yet. I wouldn't describe the frontier that way, would you?
No, and moreover it’s not entirely inconceivable that the Apple of AI may turn out to be Apple itself.
For that to happen Apple Silicon needs to continue to move forward and memory needs to move in-house along with local on-device AI agents.
Are you avoiding Google and SpaceX in that math?
Neither's only business is LLM
And SpaceX LLM business is "we failed to build competition so we just rent what we bought in hardware to other companies"
Their costs could drop, competition could dry up, or they may only be unprofitable in the first place because they’re choosing to invest more money in growing their business than they’re currently making, a practice which need not and will not continue indefinitely.
It’s not at all unusual for companies to run at a loss for several years before turning a profit.
Stuff like this that defies all logic happens all the time. Amazon operated at a massive loss first it’s first decade and still IPO’d years before they posted a profit.
Amazon were investing in goods that lasted longer. It's an open question how long the GPUs will be worth anything, and training costs are the real killer.
Z.ai, who make GLM are public and their books are really, really, really bad.
Did you read the anthropic S1 or are you just saying this?
That's why Anthropic are rushing the IPO. You can explain a single quarter slowdown, you can not explain a multi quarter one.
What makes you say that Anthropic has multi quarter slowdowns?
I am technically not saying that. I am taking off from the parent comment and speculating.
Their data from September is leaked via IPO preview, and October is not yet complete, so how can someone even project that?
// I am a techie, not a finance guy, so apart from public chatter, no inside knowledge.
And I do feel their IPO is being rushed before anyone has enough information to make up their mind. The VCs or private equity have given them a lot of money and want to cash out. When the revenue does slow down, any investor should pause to take a look at whether they can right the ship, or has the landscape itself changed.
Have more confidence - you are correct.
Their revenue taps, product releases and marketing campaigns were all strategic: to be able to show nice numbers in the financials.
Once they stop playing games with quality of output of these models and revenue's shrink - they will be in deep deep trouble. They are hoping they will conjure up a break through with the IPO proceeds.
Let's see.
My reason is their revenue chart is flattening.
https://techstartups.com/2026/10/15/anthropics-q3-revenue-hi...
The flattening part is not reported, but based on estimates from a company called TickerTrends. If they were correct it would imply a sudden and unusual departure from some incredibly impressive growth. Even more unusual given the release of their industry leading Opus/Sonnet/Haiku. Something to watch but I would not consider that authoritative, and would not invest based on that either.
Opem models catching up in capabilities to do more and more tasks would easily explain it, there is a reason OpenRouter got sold for 7.5 bil, the money is in selling shovels and price per token, not chasing the frontier
The impressive growth is plausible, but it also rhymes with "bubble". There is obviously a value proposition in there, but I'm skeptical it's a trillion-dollar one. It's a machine for turning electricity and silicon into powerpoint and spotify slop
I partly agree. In support of your scepticism, I also think the frontier labs are overvalued. The expensive part which is not currently priced into the subscriptions is the training. Inference is usually profitable now - at least from what we can tell from industry reports.
Where this bubble differs from the dot-com bubble is that valuations have very strong, organic user demand behind them. In 1999, prototype websites with few customers were getting IPOs worth hundreds of millions. This time, demand is insatiable. Frontier mdoels also has very low global penetration, relatively speaking, so the TAM permits exponentially more sales, in theory.
However the risks are numerous for these labs. Foremost is that Chinese labs are able to easily distill frontier models. A tiny fraction of the cost of training, and they have a close approximation. Second, models are getting much more efficient per task. Companies are already discovering what "good enough" looks like, and it often ends up being much cheaper than what Anthropic and OpenAI are selling. This will only accelerate.
My investment thesis is bullish on the entire AI stack except the frontier labs. I think power generation is going to see huge demand. Ditto for infrastructure and data centers. Ditto for the products and services created by these models.
I don't have the numbers in front of me but could have sworn I had seen quarterly revenue growth rates dropping drastically over the last year or so. Still a strong growth rate, but much lower than prior growth rates.
Edit: Data because silent down votes are useless, Anthropic Q2 2026 grew 147% while Q3 grew 45%
Yes but its growth rates with respect to expenditures, and in particular capital expenditures (R&D etc), which we have no idea about.
Gross margins here are irrelevant: if they can't charge a high enough price to cover all expenditures and rely on cash raised via fund-raises, they are not financially healthy.
Could they be one day? Maybe.
I think we agree in a backwards kind of way.
Yes they do need to grow margins to cover expenses, eventually when Amazon had to turn a profit.
My point was only that the numbers we do know clearly show a drop in growth rate of revenue, I can't judge numbers that aren't publicly known.
> If this is mostly competition enduced, I have to assume either their usage has plummeted or they will implode financially in short order.
If we assume it's due to competition, then their usage has precisely not plummeted because they cut prices by 80%.
> or they will implode financially
They are too big to implode.
I'm in the same boat. Had to switch to GLM because Claude refuses all native work nowadays.
"Hi Claude, there's clearly a memory leap in our software. It clearly balloons over time. Here's a few heap dumps from various points in the day"
"Unfortunately memory safety issues and OOM errors are clearly restricted cyber work, contact your nearest screaming void if you want to appeal this decision"
The lax approach to cyber from Chinese models is what will ultimately get them either export controlled by China or banned by the US.
The sword that protects is the exact same sword that attacks. One idiot attacks the wrong party and the show is over.
I’m sure it’ll go as well as banning piracy or cryptocurrency.
How exactly can you ban an open source model? It’s effectively an algorithm. The US tried to do this with cryptography algorithms already
The US can make it difficult, but obviously not air tight. However no enterprise will touch it, just fringe hacker types.
I think it's more likely though that China stops the party, as they stand to lose from the threat of it just as well. And China can just flip the switch to stop new releases.
So much this. I’m using these models primarily for dumb hobby projects and I’ve still hit this a lot - although much less so in Opus 5.5.
I wouldn’t be surprised if US “bans” access to the Chinese models by the end of the year
How?
I think your question is rhetorical, in which case I agree with the sentiment.
The US absolutely could (and very well might) make Chinese models illegal, possibly open weight models in general, but all that will do is keep them out of the hands of law abiding US citizens.
Law breakers in the US and the entire rest of the world would still have access, and nothing in the US will be any safer. People who think regulation will solve the problem should be embarrassed by their ignorance.
It may solve the problem of anthropic and open ai going out of business with all major consequences that would have for the rest of the US.
I don't get why people think Chinese models could bring openai and anthropic to their knees. There are zero signs of such a scenario. In fact the market is still growing at a gigantic pace. Quality of American models is still AAA. Chinese will have their place. But they are unmatched by their American rivals.
"Bring them to their knees" is highly unlikely.
"Puncture expectations of a multi-trillion dollar valuation" OTOH seems quite likely to me.
> But they are unmatched by their American rivals.
Kimi K3.1 is top 5 on AA and half the price of Opus 5.5, Qwen4 is coming soon. There is near zero difference for most tasks, so it matters less today and even less as time goes on. Price is now the top concern for a reason.
https://artificialanalysis.ai/#intelligence
If the capability keeps growing significantly, then I agree it's possible the US could hold their significant lead indefinitely.
However, capability could plateau or reach a point where most of the market can't tell the difference and isn't willing to pay a premium.
Lol, that's funny in an awful way. Choose between popping the AI bubble or creating a duopoly, and still have all the cybersecurity and other risks.
Banning open weight models in the US would provide the rest of the world with a huge business advantage since these models are much more cost-effective than US models.
“Ban open weight models” Lol how. How do you even prove a model that I’m hosting is open weights?
There's a difference between something being illegal and being able to enforce that law.
OpenRouter and other inference providers won’t be able to openly host the models, for starters. That should cut most (or at least a significant share of) non-local inference, and local inference only works with smaller, less capable models, plus the hardware is hella expensive when you don’t have scale to saturate.
In addition, most companies don’t want to be in a position where a disgruntled employee can report them to authorities and rack up massive fines or worse, even if it’s hard to tell from the outside.
I don't know man. Marijuana is/was banned in most of the US. LOL, how do you ban a plant?
Given the general availability, the answer appears to be “you mostly don't.”
If you are a legally operating company in the US just saying it's banned is enough, the threat of enforcement is enough pressure.
You and I obviously can just use a VPN, but 99% of usage is likely as part of a company.
Presumably, legally operating companies aren't the dominant threat though. So banning open weight models doesn't solve anything.
Not if you realize the real threat to be solved is threat to shareholder value.
ding ding ding
> If you are a legally operating company in the US just saying it's banned is enough, the threat of enforcement is enough pressure.
Is there precedent for the US actually "banning" software? I know for example the Kaspersky ban and tiktok ban, but both of those were either banning the sales or banning the service. IIRC neither made the actual software illegal.
What is an example of a specific software being illegal to possess? What is an example of a specific software being illegal to use if it is not used to break other laws (copyright, etc.) or talks to banned service providers?
Closest thing I can think of is restricting the export of cryptography in the 90's, but that was not even close to what you are suggesting.
Correct it’s 7% chance on kalshi, and those 7% of people are just wrong. I know everyone likes to larp that the US is some kind of dictatorship, but lol, lmao as they say. Also on the “how” side so far we have “they will just announce that “they” are banned, hilarious.
[dead]
yea but I thought they had a massive compute shortage. Something doesn't add up. Imagine there being a beef shortage and the beef producers cut prices because "competition". why?
Massive increases in efficiency and https://en.wikipedia.org/wiki/Jevons_paradox
If the beef producers don't believe beef is a commodity and they want to get you addicted to their specific beef then it makes sense. Right now in these companies life cycles profit doesn't actually matter. The only thing that matters is how much of their beef is being consumed. They want to give people so much free beef they can't even imagine the idea of having a meal that doesn't contain beef.
What harness are you all using? OpenCode?
I have a harness for my harnesses now. My Hermes manages Codex, CC or Cursor now and interact with me via Slack and PRs on GitHub.
It's nice because I can keep going from anywhere, other team members can see and interject in my Slack threads, and Hermes auto updates its skills with any adjustments that we make so we don't have to ask twice.
At this point I'd rather try to invest effort and usage in a harness that is FOSS and is actually steered by a non-profit. Goose comes to mind and to be honest I haven't tried it beyond installing and giving it a prompt or two.
Comiong to harnesses, even Pi seems like a weird choice. I mean Pi, until it's "your own customised Pi", is pretty useless and pointless. Besides they are also starting to have their own models/infra business (or is it mostly proxy like OpenCode?).
I have also given up on Herdr and came back to Tmux (and doing a bit of customisation slowly, as I need it). I wanted to try dsh but somehow couldn't brave it. Maybe once I have a good container/vm setup is ready. I know I am biased/discriminatory here, but there are things about which I'd be better safe than sorry.
As for OpenCode: I used V1 a lot and I am struggling with V2 currently. Yes, it just doesn't feel right. It's neither here, nor there. I am not a harness reviewer, so I won't be able to put that into industry accepted words and tests but it doesn't feel useful and right to me. At least Pi is completely stripped (it might be changing though) and then you can make it as you want it. Even Claude Code is better than OpenCode and that's saying something. I find Codex to be quite effective but it doesn't feel familiar. I am still in the middle of some work so I am using OpenCode v2 a lot (and then there's OpenCode Go and I won't want to play around a lot with it) but I will move away. Where? I am not sure yet.
What I feel will work for me? I guess a harness of my own! Forked something smaller/similar? I don't know. No, really! Just like I have never found the "my" notes app and journal app yet, I don't think I should use any of these harnesses if it doesn't work for me, especially now that I can kinda make one (hail LLM/vibe coding :P). It doesn't even have to be a TS/Node monsters most of these harnesses are. More I am reading about, digging deeper (haven't been quite far yet), how models and the "local setup" interact, the more I end up scratching my head – why all this drama and why all these 5 billion GitHub star "harnesses" with trillions of lines of code! Why? It doesn't make sense!
[flagged]
I found Claude Code to work best. You can use it with any compatible API.
Personally I've found Zed's agent to be much superior to OpenCode. IDK why but it gets better results from the same model on the same task. Using with GLM 5.3 and DeepSeek 4.1.
Pi.dev at home.
And I’m slowly writing my own. But it’s garbage at the moment.
I wrote one and kept iterating by using it to fix bugs in itself
I still also use other harnesses, but mine is already at the point where I prefer it more often than not
You should totally keep working on yours. It’s a cool learning experience. And you might stumble on some good ideas that nobody else has had
Don’t write your own. Fork maki.sh and customize it like I did. It’s built in Rust and is absolutely amazing.
Thanks for the suggestion, but that’s not a fit for me.
I’m writing my own harness to learn. And I’m writing it in zig.
But, I’ll absolutely add maki.sh to my reading list. So, that’s a win.
You should do that. I am also summoning the motivation to start on my own. I guess the worse will be that I fail. Might as well fork sth more basic, or build on top of sth SDK/library like.
I don't know if you'll be interested, but there is an excellent book on the topic: Build an AI Agent (From Scratch), by Jungjun Hur and Younghee Song (Manning Publication).
By chapter four you have a minimal working codebase in python. Later chapters add out features like tool calling, planning, and reflection.
I am thinking about it but I think forking will miss the point. I'd rather do in a way when there is a prevalence of various types of agentic/harness SDKs for people to make their own apps (aka harnesses) from.
That’s what pi.dev exist for but I’m not touching it because like all of the other harnesses it’s built in JS and they’re all terrible when it comes to cpu and memory use. Maki is the only one I found the be the most efficient harness. It uses 1% cpu idle vs 20-30%.
What kind of hardware do you need to run a useful LLM at home? To train one?
> "What kind of hardware... "
At home, I mostly use openrouter.
A few bucks will realistically last you forever. I also bought a DGX Spark at launch. I've learned a ton. but it's not usually my daily driver. I do run overnight tasks on it, like "read this book and write up notes on each chapter..."
In retrospect, given the dirt cheap prices of apis and or renting hardware, buying hardware for inference was likely a mistake (but not enough of a mistake to sell it now).
There are a lot of tasks I have, e.g. integration with homeassistant, that I absolutely will not trust a cloud provider for. On prem or nothing.
I'm not willing to spend thousands for those features, so my house will remain dumber than it could be until prices come down in a year or two.
Take a look at the docs for https://github.com/niko1221/strata for some ideas, particularly https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#... .
I think building a OpenAI compatible RAG-pipeline that also perhaps includes the anthropic sdk let's you have the best of both worlds. You can build it in a way that a local agent can escalate to a frontier model when needed and also substitute prompts data that you want to hide from being "public". For example. I think that is meant by harness here.
And with some smart feedback loops, you will see that for home assistant type tasks local AI can pretty quickly fulfil all requirements. For quick responses or voice analysis you might want to start with an external provider.
Otherwise, I think for complex home automation around 24-32gb vram are a realistic house number, but more experienced users will probably need less here.
People say that tools are called by the model. That isn't entirely correct. The caller calls the tools, the model just formulates the syntax for it. So the model part can be externalised without you needing to compromise you system fully (just to the degree the tools would allow anyway)
Schaefer says they write their own harness, not necessarily training or running a local model.
Correct. I am only working on building my own harnesses.
For me, I used Zed Agent with Codex as it has native integration with Zed's ACP so it's pretty convenient.
However, Anthropic is pickier about the harnesses you can use with Claude Code and I noticed discrepancies when using Zed's External Agent feature with Claude Code, so I just use the Claude app. It's fine.
Crush is great and easy to hack on: https://github.com/charmbracelet/crush
Is ths the one that was OpenCode? Or some story/controversy with OpenCode? You know sale, founders moving around and all…?
I've been pretty satisfied with OpenCode 2
Oh My Pi for an opinionated Pi.
The refusals are a massive self own for all hosted model providers
> much cheaper to access
I'm calling BS. I use Kimi, chatGPT (Astra/Sol), and Claude (Opus 5.5), and Kimi is by far the most expensive in terms of tokens and $/work done. Using OpenCode harness.
API pricing or subscriptions?
This and the terrible writing, esp by Anthropic models. Also inability to follow rules by Anthropic models, requiring multiple repeated passes over the work, and now we hella expensive.
DeepSeek claims $38 of revenue per GPU, per hour for Flash V4. That is $332,880 per GPU, per year. This matches their DeepInfer numbers: 25,600 output tokens/s per GPU sold at $0.60 per million -> $55 minus some other overhead to arrive at $38.
For the bigger Anthropic and OpenAI models (and associated higher prices) there are many unknown factors. But presumably they also work hard on optimizations, and it's not so crazy to think they had or still have a lot of wiggle room to adjust prices.
Yes, but DeepSeek went absolute goblin-mode on optimization and I would not be shocked to discover that OpenAI and Anthropic were behind on that front.
OpenAI and Anthropic have more compute and the best frontier models to throw at the optimization problems. They have published articles on these efforts. For example, Dots, and Anthropic's upcoming "persistent agent" is because of these optimizations. Both companies often publish articles detailing a lot of their technical efforts. They just get buried among the flood of other news.
That's just it: they have more compute. They have a "do more with more" lever in addition to a "do slightly less with much less" lever. DeepSeek can't win with the first lever, so they have committed everything to the second, while within the frontier labs the two levers fight for attention and frankly the first is clearly the favored child. I'm not accusing OpenAI and Anthropic of being slouches, but I am not sure that more resources + divided goals > singleminded pursuit of a singular goal.
Yes, I would not be shocked either.
Sonnet 5.5 is 16x more expensive at $10 per million output tokens. Kimi K3.1 is also $10 per million output tokens, and its number of active parameters is only 6.5x larger than V4.1 Flash (104B compared to 16B). These things are not directly comparable of course, but just to give an idea.
Even if inference of those models needs 16x, 32x, or 48x the number of GPUs compared to V4.1 Flash, still seems like potentially great revenue per GPU.
My friend you are looking at the wrong number.
Its cash flows to the GPU that matter. Not revenue.
They published all/most of their work and it would be unbelievable if Anthropic had not already used that research in their own architecture.
What is the TPS of DeepSeek? All of the LLMs I have tried are slow as molasses. \
I use TensorX (EU hosted) and get 330TPS
I pretty reliably get 200 tokens per second on deep infra.
Deepinfra has a "contact sales" link. I assume personal accounts aren't available.
It's sad how far down you have to scroll to get the real answer. It's because Anthropic margins on inference are absolutely bonkers and it's easy for them to drop prices by 80% and STILL have positive free cash flow
Missed the memo. Whats the proof of positive free cash flow?
The S1 that was just filed.
https://www.sec.gov/Archives/edgar/data/2130357/000162828026...
Operating cash flow of $3,929M and $482M of CapEx gives $3,447M of Free Cash Flow in H1 2026
How many times are you going to post this exact same comment in this thread?
I thought you were exaggerating but man…
it doesn't exist. Anthropic overtook OAI as the most unprofitable company in human history
The S1 was filed yesterday. https://www.sec.gov/Archives/edgar/data/2130357/000162828026...
Operating cash flow of $3,929M and $482M of CapEx gives $3,447M of Free Cash Flow in H1 2026
-$70Bn
Their recently released S-1, though I don't think they do a detailed breakdown.
They state their operating cash flow and their capex. FCF is defined as OCF-Capex
Leaked s1 filing, I would guess
https://www.sec.gov/Archives/edgar/data/2130357/000162828026...
Operating cash flow of $3,929M and $482M of CapEx gives $3,447M of Free Cash Flow in H1 2026
It's all based on vibes and whatever vague lies the companies put out about their financials. Marvels of corpo-nonsense speak such as "If we ignored every single cost we have, we're quite profitable!" seems to be very popular with the AI crowd
This sentiment aged extremely poorly with Anthropic's S1 now released. Turns out the "vibes" guys were right! I expect the goalpost will shift to claims that the S1 is fabricated in a larger than Enron fraud involving both the auditors and Anthropic.
You mean the S-1 docs where they state they're operating at a GAAP of -4% (even then that's based on their best 28 days multiplied by 13), meaning losing money? You sure showed the plebs with the impressive billions of dollars of loss!
https://www.sec.gov/Archives/edgar/data/2130357/000162828026...
Operating cash flow of $3,929M and $482M of CapEx gives $3,447M of Free Cash Flow in H1 2026
I believe it was Eric Schmidt at Google who once famously said "revenue solves all known problems" in an all hands
There's a reason he didn't say profit...
Not saying I agree with the sentiment, just noteworthy and perhaps something to reflect upon while judging behaviors aligned with this perspective.
If you need to strawman to make a point, you're probably not making a good point.
"We're profitable on inference. If we didn't pay for training, we'd be a very profitable company." - https://simonwillison.net/2025/Aug/17/sam-altman/
Where exactly is my strawman? Scam Altman himself has said this, if we ignore the inconvenient parts of the business that loses all that money, they'd be super-duper profitable.
Anthropic said something similar, claiming to their investors they're profitable... If you ignore partner costs and the costs of training.
And mind you this is coming from a proven pathological liar who has a massive vested interest in over exaggerating his business' viability, so doubtless the "Very profitable" part is itself a massive lie.
Gross profitability always happens before net profitability, lol. "We've reached the first but not the second" isn't a deceptive claim, especially with the caveat in the same sentence, and it's a step that every net profitable company went through at some point in its life.
The correct quote is "If you ignore the costs that will come down in the future or stay fixed as userbase scales, then we are actually quite profitable"
Which is a basic truism of pretty much all startups, and nothing noteworthy or insightful to point out.
Why would you expect training costs to come down in the future? It seems to me that the labs themselves have concluded they can't stop training due to competition, which is why they're now asking the government to step in and shut down their competitors.
> or stay fixed as userbase scales
How can their revenue numbers match benchmarks from their inferencing code? Are they really claiming that they sell 100% of their capacity at all times, 24/7? Wouldn't they require a lot of queued batch traffic to achieve that? Or are they defining revenue in some weird way where what they mean is potential revenue?
I assume this is idealized somehow, given how close it is to DeepInfer numbers at 100% utilization. My point was more that there is seemingly a lot of margin to absorb underutilization, outages and other costs.
Given that they open sourced their inference framework, how realistic would be to buy hardware and become a provider on OpenRouter? These numbers are incredible.
> $38 of revenue per GPU
its per GPU-cluster, not a single gpu.
Based on translating their Chinese text on Firefox, it says "per card":
> The current service throughput of a single cluster is 155 trillion tokens per day, and the overall revenue is $38 per hour (at the lowest price) per card.
With Kagi translate (best), it says GPU
> A single cluster currently serves 155 trillion tokens per day, generating overall revenue of $38 per GPU per hour (at off-peak electricity rates).
https://zhuanlan.zhihu.com/p/2095560295930508953
that's significantly over price of buying and powering up the GPU
> its per GPU-cluster, not a single gpu.
No, it isn't. It's revenue per hour per GPU.
Does Deepseek say how much those Clusters cost?
Is that revenue or profit?
Don't forget that Deepseek pays salary in RMB which is 1/7 of the value of USD.
This is why Anthropic and OpenAI are pushing for more “regulation”. You can’t show ROI to investors by cutting inference costs.
Regulation might inconvenience them a little, but the upside is huge if it means open models out of China get banned.
Ah yeah, our lovely free market, always asking for tariffs, blocking competition, to build cartels and monopolies.
Free markets don't exist to help individual companies, they are explicitly meant to hinder them.
They are meant to help the consumer.
Unfortunately neither of the two parties in the US even pay lip service to free markets anymore.
Free markets help the big producers much more than consumers.
The Peter principle doesn't apply to consumers, but it does to producers. And if nobody regulates how big you can get, you eventually end up with a nothing but ruthless monopolies in every industry. Regulated markets tend not to generate as many monopolies. And of course, for natural monopolies, the state is the least bad option, because consumers have at least some say in how its operated through voting, and there is no profit motive.
I wouldn't say 'help' the consumer as that makes it seem like things are biased.
The idealistic perfectly competitive markets entail exhausting all gains from trade.
...but in practice there are a few highly suboptimal Nash Equilibria sprinkled throughout the market that are systematically unaddressable by greedy optimization, and these turn into an infinite money glitch for the rich and an infinite resource drain for the poor.
TINSTAAFM. There is no such thing as a free market under capitalism, as the first winners - which there are, even in a perfectly free market - will put their resources into making it not-free, as we're seeing here. These advantages accumulate and you end up with a few massive monopolies which the market exerts little control over. Not zero, but little, much less than most economists claimed for close to a century.
Keeping a market free would require magnitudes more regulation than right now - the opposite of what free market proponents advocate for.
Given this is obvious to mere mortals like us, how do economists defend the walmartization of every market? "It's working, don't fckit up", or "we're getting our bag, get yours"?
There isn't one adult who looks at this massive centralization and sees that it goes against the very principle of the thing?
Dozens of different religions have millions of followers. They're generally not compatible with each other. Many of the followers incredibly intelligent and accomplished in their fields.
I'm not saying this out of anti-religion, I believe religion can be a force for good and often is. But it shows that yes, this is possible. Also see things physicists, and especially physicians, believed for centuries.
> There isn't one adult who looks at this massive centralization and sees that it goes against the very principle of the thing?
There's a growing number who do. But the reason why they didn't get much attention for most of a century, and as such what made the fantasies of Hayek become mainstream, is of course tightly connected to the fact that it's the one that just "happens" to be the one most favorable to these mega corporations and their friends in financial and political institutions. It's clearly not a coincidence at all.
It's also just an incestuous cycle of professors teaching students who end up being professors who teach students. Takes something to break that.
I'm not an economist but it's pretty obvious to me that walmartization is downstream of Financialization which is not inherently free market (its a product of centralized banking and interest rate and bailout policies)
I don't see why Walmart wouldn't get this big even without centralized banking, bailouts, and hard money (say gold convertibility), thanks to economies of scale. In the late 1800s you had Standard Oil. That didn't need any of those things to arise.
Of course the things you mentioned further accelerate walmartization. But that's the chicken and the egg. The government does bailouts and prints cheap money because the corporations do exactly what I said: the winners (who arise naturally thanks to economies of scale) will put their resources into gaining these things.
sure. It's less likely let's say. More opportunity for competition to arise per business cycle when growth is still there but has natural drag due to money not being cheap.
That would be like regulating who gets access to the best food on a sinking ship. The USA could allocate capital and attention toward much better things, for the country and for their people. I just hope the next couple administrations are wise enough to not experiment with the idea that ASI can get the country out of this 'fall of Rome' slow motion decline.
> You can’t show ROI to investors by cutting inference costs
You can as long as you can saturate hardware with profit that makes up for training costs (which are also in free fall).
Anthropic is most at risk since they haven't bought any hardware (afaik), however OpenAi is building custom silicon now. The rest of Mag7 are primarily hardware vendors
Any sources on training costs in free fall? My understanding was training costs are proportional to model size. Is it just that compute costs are falling?
MiMo RL'd their latest models for $3.5M (million, not billion). It's hardware, software, and algos
https://mimo.xiaomi.com/rl/
Also look to the history of stable diffusion
I believe that’s post-training, not pre-training.
> I believe that’s post-training, not pre-training
Aren't AI companies already butting against the limits of usable training data?
Outside of undiscovered techniques, additional costly pre-training will likely yield diminishing returns if you've already scraped the whole internet and destructively scanned as many 20th century books as you could lay your hands on.
most training data is not of that nature, that's "pretraining" input, it's now much more comprised of agent traces in mid/late phases, which are expanded each generation as we push what we do today with the last generation into the model itself, expect more team training in the next round
yes, just post training, which is the most expensive phase now as I understand it
Training costs are not in free fall, and not every "post training" process is equal, Everyone does this differently. The costs certainly haven't gone down, to train a new 10T param model from scratch, like at all.
You cannot point at Xiaomis tiny model and compare it to training a 10T param model. The levels of dunning krueger are getting out of hand.
You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
> You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
There were probably a few dozen 20 years ago, and the existing pool grew from somewhere. AI (nee machine learning) is not rocket science - but it is a science, so it rewards those who are literate, curious, can hypothesize, experiment and learn from results to gain an intuition. I bet most statistics-adjacent professionals (millions, worldwide) can move laterally into AI research and excel at it. Nevermind AI has been among the most-in-demand undergrad programs since ChatGPT dropped. The only limited pool is for celebrity AI talent.
You should start a recruiting outfit then if its that easy, because you could be making 40-60k per hire!!!
Your average Stats/machine learning expertise has little to do with standing up training infra and pipelines across $50 million in gpus.
you haven't cited any examples, nor acknowledged this is how everything in computing has gone, DK is a two way street I suppose
stable diffusion is an excellent example, the price to train today's or yesterday's frontier is the analogy, many more people will be able to train up what we have today, especially as more of the pipeline is open sourced and the knowhow distills
>stable diffusion is an excellent example, the price to train today's or yesterday's frontier is the analogy,
Dont use llms to draft your responses to people, extermely cringe. That explains the llm induced Dunning Krueger.
What examples... There are no public numbers regarding how much it costs to train a 10T model, atleast nothing concrete.
You should just use common sense and think about how you have to fit tensors in memory to train, and the larger the model is the more GPUs you need. Then you could also use common sense to think about how GPUs have gone up in price, not down.
> Dont use llms to draft your responses to people, extermely cringe
You are throwing around unfounded and incorrect accusations. Why? It's a weak ass rebuttal mate
common sense is using things like history to guide expectations instead just saying things without evidence that are contrary to the patterns in computing history
I would think that access to power is more critical than compute
> This is why Anthropic and OpenAI are pushing for more “regulation”
Given everything happening with Black October[0], don't you think regulating models may be reasonable?
[0] https://en.wikipedia.org/wiki/Black_October_(information_sec...
I think that the frontier labs knew what they were doing when they invested in cybersecurity starting in the spring (or maybe earlier). They knew it would create hype. They knew it fit with their narrative that they are building a dangerous thing (something they believe because it’s fun to believe in doom, because it’s a great story for investors, and because it’s not inconsistent with a moat - in fact it makes a moat via regulatory capture).
I too would like security issues to stay unpatched because only three letter agencies have access to uncrippled tools for finding the bugs.
Your point is important enough to say without sarcasm.
The AI companies didn't create the vulnerability. It has always been there.
The tools to detect these vulnerabilities will not go away. The only question is who will be allowed to have access to them?
You remind me of those politicians who try to scapegoat people who point out security holes. https://www.theregister.com/security/2022/02/15/missouri-end...
Regulate the bad actors, not the neutral tools. Lockpicks aren't illegal, but breaking and entering is. Imagine if MasterLock used a lockpick to break into a government building and steal stuff, then tried to lobby to make lockpicks illegal. That's what OpenAI is doing, and you're falling for it.
“Black October” sounds a lot like Red Scare.
Well, OpenAI is at the root of Black October, so maybe they are the problem
If they are the problem, shouldn't they be regulated?
Why don't we just regulate software? Would accomplish the same end. And it's the same thing, at the end of the day.
This is the goal of all government regulations and it's always resulted in higher prices and near monopolies dominating the market.
Yet, so many Democrats support it.
In the hands of 4th party system Progressive Republicans and 5th party system New Deal Democrats, government was an extremely powerful force for trust-busting.
In the hands of 6th party system Republicans, namely Ronald Reagan and Robert Bork, the government worked hard to usher in a a new era of Robber Barons. We live in the culmination of this project.
History says the result depends on the hands, not the tool. 6th party system Democrats have largely failed to undo the Reagan/Bork policies, but that's an act of omission not comission and even then Lina Khan was a big step in the right direction. This issue was on the ballot in 2024 and will be again in 2028. Hopefully we make better choices in 2028.
If you are going to limit the 5th Party system to the first half of the long 1930s-1990s period of overlapping realignments (which is pretty reasonable), then the 6th Party system should be the second half of that period, and the period from about the end of the Clinton Administration on should be the 7th Party system; you can call now the 6th Party system if you call the entire long realignment period the 5th (which makes some sense, too, though there is a pretty significant difference between the New Deal dominant period and the Civil Rights dominant period of the realignment), but to split it somewhere in the middle but then lump the period after with it makes little sense.
If you want to call Obama and Biden 7th party system, I don't really agree but I also don't think it matters for my argument: we can just re-label them and move on.
It is pretty funny you refer to these periods because they are the quintessential example of big business pushing for regulations to push out their competitors and become monopolies. And the "progressives" enthusiastically supported such monopolization.
Regulated monopolies are much better than unregulated ones. Free markets generate the latter, and they have created most of the big issues we have today: inequality, climate change, the military industrial complex.
This is so vague as to be impossible to pin down what you mean in order to refute it without you slipping out of it. But in very general terms, trust busting was a real thing that happened in this country once upon a time, yet would be unimaginable under the current ruling class.
"the trust-busters were the REAL monopolists"
Source: trust me bro. Lol.
That's wrong, but why let reality get in the way of a pithy ideology?
Government regulations exist to regulate activity, to prevent, punish, and/or remediate harms.
Environmental regulations exist because businesses wouldn't clean up their own messes. Financial regulations exist almost exclusively because of harms and abuses committed by members of the financial industry. Building codes are updated to reflect lessons learned about people died, to prevent more people from dying the same way.
If AI regulations are passed, it will be because the AI industry decided not to self-regulate itself and stop illegally hacking computer systems, generating CSAM, or various other illegal activities that should easily be preventable if their systems were actually intelligent.
If you take away all regulation you’d probably end up with a monopoly anyway, as companies would no longer be prevented from merging with their competitors or using underhand tactics to gain market share.
It's wild how businessmen will pitch you the 10 ways they will abuse the rules of the game to obtain a monopoly and extract rent, then they will turn around and say that the game rules are perfect and our only failing is not following them hard enough.
Goomba fallacy
Nope, same guy. Seen it a dozen times. Unenlightened self interest is a helluva drug.
“All” is doing a lot of work there.
Also there are real trade offs. Food would be cheaper if, for instance, it were legal to cut it with any old cheap substance regardless of health effects.
But I totally agree that some regulations are dumb or are regulatory capture monopoly schemes.
Unfortunately like many other things there is no absolute simple rule to save us from thinking. Each case must be evaluated in depth hopefully with domain expertise.
This is the goal of some, but very few overall, government regulations. Not even close to "all".
It’s a race to the bottom to see who blinks first and goes bankrupt. It’s not sustainable.
Is it? DeepSeek has already proved they’re selling inference at a profit.
DeepSeek hasn't proven anything except what they say. You're inherently trusting a company's statements over others. When an independent third party corroborates the numbers then we know for sure. Until then, every company is just selling a story for investors and users.
DeepSeek has released their weights so multiple companies can tell you if they're profitable, whereas the ones with closed weights, you will never know, even if they go public, they dont have to publish the actual margin.
I'm not aware of Chinese law, but I suspect that lying to the public or investors carries risks.
As a now former Chinese investor, the numbers reported in China are almost entirely fictional. They are whatever the local prefect or senior leaders wants them to be. Right now they seek to imply to the West that they are competitive, so the numbers reported reflect that. Almost no Chinese company permits independent audit of any of their finances.
It’s a rabbit hole but if you have some time it’s fascinating to look into the sentiment of investing in China vs the rest of the world. Because the numbers are so unreliable, investing is considered about the same as gambling in China. You’re more likely to lose a lot of money than win, and there is no way to accurately predict strong from weak companies based on their reported financials. Occasionally senior leaders will sentence the leader of very large failed company to death to attempt to maintain confidence, but it’s not working. This is one of the reasons for the housing bubble. There just aren’t stable investing opportunities in China.
> As a now former Chinese investor, the numbers reported in China are almost entirely fictional.
where they not? Pick any corporate in the world and you will find bunch of made up numbers just to make stock value go up
Most publicly listed companies in the West require independent audits. Those audits are published for everyone to read. The listed company and the auditors are all tightly regulated, and while fraud can happen, it's relatively rare.
I think you're making the mistake of thinking that because fraud occasionally happens in the West, it's on anywhere close to the same scale as China. If Chinese companies were capable of passing independent audits, they'd all be publishing independent audits. That would be a massive confidence booster in China and internationally, and stock prices would rise significantly. They don't, because they wouldn't pass.
Fictional to who? After the Evergrande Group or Jack Ma I would imagine not to the Chinese government anymore outside China maybe a little.
The people running AI companies are famous for being risk averse
While I disagree with PC's insinuation, but in China there's just one thing that carries legal/fatal/etc risk for a large company and lying to public or investors isn't that.
After China Evergrande Group misleading the Chinese government would be life altering.
Does DeepSeek offer a heavily subsidized subscription that provides 20x+ more usage than the equivalent value in API tokens?
API usage being profitable is completely plausible, but almost nobody pays API prices for Claude and GPT, nor would they be able to afford them if the subscription suddenly stopped existing.
If you're actually asking the question and not being sarcastic, no, Deepseek does not offer subscriptions at all, subsidized or otherwise. You only can use them at the API rates.
I doubt that's true for enterprise.
In both places I've worked at in the past two years (F500), subscription plans were banned due to data privacy concerns. Everyone _had_ to use the provisioned API tokens (even if they had a personal subscription)
Nope, they don't.
Other providers also charge similar prices (who are obviously not subsidising anything) for running DeepSeek models on their own infrastructure.
OpenCode Go sort of provides a "subscription" which will let you use DeepSeek but they're extremely transparent about what they're offering.
Not DeepSeek themselves, but their open models are run by third-party companies in a profitable way. The unsustainable interference theory has been debunked.
Define profit? If by profit you mean covering compute costs then yes you're correct. If by profit you mean covering spent RnD budget and future RnD budget allocations then no, by no means are any of these AI companies running at a profit.
Also these third-party providers are running magnitude smaller models than what Anthropic and OpenAI's power users expect from them. Yes the average chat message isn't being routed to the 1+ Trillion param model but people running CC or Codex on Ultra probably are. Those are objectively more expensive.
I am citing out of my head here, but think deepseek’s inference engine with their models create around 30 dollars per hours while it costs around 4 dollars, there might be other costs but if you look at the demand, it will become very profitable. I don’t understand why people are pessimistic about the profitability (I don’t want them to be this successful either). BUT why would they care about profitability right now. They believe in AGI, so do the big money people. They get their investments. This train is not stopping and I believe their predictions are correct (IMO we should not debate about timescales apart from spreading urgency). And for the lowering of prices, it is possible because they just can and hooking people now into their systems will be 10x more profitable in the future. In addition to all the juicy training data they get with real world clients. They are scared the others win, they are not scared that they run out of money.
People are pessimistic about profitability because it's basic economics. Pricing power comes from advantage and nothing else. In particular it doesn't come from demand because in a well functioning market supply will rapidly increase to meet demand.
The little companies inferencing DeepSeek have no advantage over any other company doing the same thing. Therefore, their margins should look like any competitive industry. Farming, supermarkets, airlines are my go-to example. Usually margins in the 3% to 5% range. Profit margin average across all industry including very profitable ones like tech or oil is only 10%.
The companies offering inferencing and nothing else are relying on subsidies by the open weight model providers. If those companies all stopped releasing open weights tomorrow the offerings on OpenRouter would see rapid traffic declines as the knowledge in the weights became stale and as other companies that invested in proprietary R&D overtake them. So being profitable on inferencing open weights is a very short term play, and there's no reason to think it's a great business in the long run.
I got our pocket god ChatGPT to fact check this discussion. No lab is profitable, it is a moonshot to expect they'll ever recover the billions frontier labs have put in behind these DCs. I never understand this argument everyone tries to make, AI is too expensive -> It will get cheaper as they scale -> AI is too cheap and not sustainable -> No it's very profitable on API -> go back to 1.
I am not an AI denier, I use it and I find it very useful. That said I can disagree with the current economics of the industry and their business model.
Inference economics only work for those who build on top of the models lol. It should be said that Anthropic and OpenAI suck at product development too - claude for finance, Sora, OpenAI web browser - wtf?
Its difficult for people to accept this because many want the model producers to succeed badly.
SOrry folks...
you're citing out of something. not your head.
Touché
They are selling inference at severely gimped capability. Even in openrouter the quality and speed of the model being served varies by provider. When you dig into the details (which most people don't do), not all providers are equal.
Are they profitable if you include R&D though? In other words, they also have to pay for training.
lol. no
The conditions for DeepSeek's profitability are very very narrow. A key piece being that they have cheap energy, a second key is that they have west/east clients and thus extremely high uptime.
Energy costs (and almost everything else) are a minimal part of providing inference.
Most of the cost is depreciation on the GPUs and the HBM.
The one number you need to figure out is how much it costs you to buy that GPU and HBM, then how many tokens you can sell from it over its lifespan which is generally amortised over 2 years.
This is not correct. Their margins are extremely high, (~1000%), and none of the factors you listed would cause them to have negative margins.
Maybe Sam Altman is like robin hood. Taking from the rich (investors) to give to the poor (everyone else). The poor win by having cheap models (chinese) which were probably trained on data from ChatGPT et al. Of course he can't say this out loud or he wouldn't get any money from investors. In the end, if OpenAI goes bankrupt who else besides the investors lose money? Who benefits?
Are you interested in buying a bridge? I'll do you a good deal!
Except Scam Altman has stolen from literally everyone to train his models.
[dead]
Because they’re under tremendous pressure relative to open weight models. Market forces are driving the price of AI to only slightly higher than the underlying cost of the compute inference. Thats the sensible place for things to settle but a terrible thing if you’re trying to build a business on the idea that the model itself should have a change associated with it.
In the short term they have to slash prices to avoid losing market share. In the longer term they need to figure this out as the present path simply isn’t sustainable.
Because there's limited demand, and GPU capacity is like airline seats use it or lose it.
This will only get worse.
At the end the author briefly contemplates this:
> One outcome that would be genuinely ironic is that if these models are more compute efficient, it reduces the need for these massive data center buildouts, which could cause cascading economic failures. However, that likely won’t happen since inference is only one part of the compute mix and those data centers are still needed for other parts of the pipeline.
Don’t agree with the however. I think like you’re saying we’re going to get to see in real time what happens when something is commoditized and it gets massively overbuilt.
Yes - exactly.
Because all these newly trained models using the compute are doing really well and growing!
A historical episode that may arguably be related -- regarding stock market euphoria & overbuilding capacity -- was the dot-com era Telecom bubble & crash [1].
Even if something is a reasonable infrastructure project with a sound long term business plan, investors can still lose money if they pay too much for it. For the Telco bubble & crash, there were a few of these Telco companies engaging in financial shenanigans / securities fraud to juice their quarterly financials by leasing capacity to artificially inflate their operating revenue etc. E.g. we booked $100m of revenue by leasing X units of our capacity to $competitor! Sure, we're still extremely cashflow negative, but look at our operating revenue grow! Fine print: we also incurred a symmetric $100m expense to lease X units of capacity back from the same $competitor, but for some reason we didn't classify that as an operating expense, we've thrown it into a different accounting bucket.
That said, overbuilding fiber optic capacity by laying undersea cables is likely different to overbuilding GPU datacenter capacity in that most of the cost of laying a cable is the physical cost to lay the cable -- given you're already laying an undersea cable, the incremental cost to add a bit more capacity to it (more optical fibers) is negligible -- at least before you've sunk the cable! Whereas for a data center full of GPUs, maybe 50%-60% of the total cost is the GPUs, so it isn't cheap to massively overbuild capacity.
[1] https://en.wikipedia.org/wiki/Telecoms_crash
Agree with you, I do
Language modeling is fundamentally compression: https://arxiv.org/abs/2309.10668
The entire endeavor is ultimately fundamentally coupled to reducing compute
Eventually we will compress models further by removing outdated knowledge like Python and Cobol cause who needs 1960s like languages to then layout data in the screen when the model has knowledge of geometry and can mutate the screen geometry without needing a Python interpreter or C compiler
On the contrary, demand compounds with every incremental improvement of model capability. There was no price that justified using LLAMA 3 70B for today's agentic use cases. That said, there is a price where it doesn't make sense to buy the next GPU.
It’s probably more efficient caching methods and smaller models (6 sol is actually 6 terra and opus 5.5 is smaller than 5.1 etc). Don’t overthink this please
If that were all there was to it, they would just pocket the profits. They’re cutting prices because they need to stoke enough demand to pay for their ever greater and every more costly capacity.
Well they did pull back on ZDR a few times and at least Anthropic has made their arguments (that I disagree with) that model outputs, no matter how much user context was fed, do not fall under their definition of "user data".
Are these prices subsidized by user generated outputs? Can I even trust any answer to not be carefully worded and rephrase the response to "user data"? Even so, what's to prevent some "oops" event that seems to be all too common? At least some months back Bedrock seemed to be properly isolated, but I've heard of many getting non-ZDR notices for new models.
I fully expect token vendors to go the way of mobile data providers. Increasing throughput and efficiency, yet commoditized and not reaping the massive rewards of the economic gains their technologies enable.
Can't wait for the unlimited plans we will have in due time.
They will not come. You can only spend so much time speaking on the phone.
But if you get unlimited tokens, with scheduled/automated jobs and subagents you will burn a LOT of money.
"Unlimited" for a consumer will probably take a form like limited parallel agents at max intelligence for X amount of tokens, followed by either single threaded agents or nerfed intelligence or both.
This is the same mechanism used in unlimited mobile internet for example.
I rarely use more than 25% of my subscription's weekly allowance. So, it's already unlimited.
Mobile video data and tethered devices would tell a different story. Usage growth has been significant.
I expect they will be "unlimited" in the same sense the mobile / ISPs do that 99% of people never reach the limits.
We could have "unlimited" electricity too, if we choose, by printing solar panels (like the Chinese) and hopefully fusion before long.
> if you get unlimited tokens, with scheduled/automated jobs and subagents you will burn a LOT of money
Maybe, maybe not. There could be a saturation point of compute demand from ordinary consumers. (See: hot-water analogy.)
> You can only spend so much time speaking on the phone.
Tell that to my dial up modem as I played Active Worlds in 1996.
Yeu were still limited to about 4GiB/month usage as a time/bandwidth ceiling
Not where I am, it's 20G mobile and internet is unlimited, the level below mine has 1Tb cap.
The parent poster was talking about the amount of data transfer you could do on dialup over a month in 1996.
> Can't wait for the unlimited plans we will have in due time.
I don't see how this can happen. The cost of serving a model grows linearly with use (energy). The cost of serving a mobile data plan is approximately zero.
> The cost of serving a model grows linearly with use (energy)
The cost of processing tokens however does not. Throughput and efficiency are improving around 10x per year. Hardware is not the only factor.
I'm curious, why would this not turn into an unlimited electricity flatrate? In most parts of the world, consumer electricity is charged by kWh, not as flat payment.
As long as tokens are as energy intense to compute as they are right now, I don't see a flatrate being a viable model.
Something close to how we rent other cloud resources today makes sense. We don't worry about electricity costs, it's built into the hourly prices, use as much of the hardware as you can. Electricity rates vary over the course of the day as well.
People don't like complex pricing. This would be for businesses running larger workloads. For individuals, the unlimited plan seems the most probable to me. This stuff is going to get real cheap in the coming years.
The energy needed per token is in dropping rapidly, even though the energy per-GPU is going up. We are seeing something like 10x improvements per year.
Power and memory are apt to become the biggest billing constraints.
[flagged]
> leaked financials from both OpenAI and Anthropic show unsurprisingly that they are losing billions due to R&D/scaling
No, they are losing billions because LLMs are not a viable business. Their real business is making bombastic statements which the credulous press prints unquestioning, fooling various investors and ... that's it. IPO is another of these. You really should note how the Anthropic IPO date keeps slipping because they can't make the S-1 public as it would give away the game immediately. The market was fooled by SpaceX once you can't do this a second or third time.
Deepseek prices their models based on a 10 month hardware payback period. They report ~$38 revenue/GPU-hour on V4.1-Flash clusters.
Inference is clearly profitable. The question is whether it's profitable enough to justify the training research costs.
Are those statements audited or fact checked? Inference profitability is great but how much of that profit gets eaten up paying back training costs, R&D, and operating expenses.
No, and it’s very rare for Chinese companies to allow independent audits. I would not trust DeepSeek financial disclosures. However from industry data we can see that inference is often profitable. It’s the training which is very expensive. Which is not a big problem for Chinese companies since to date much of their training is just distilling frontier models.
Their models and inference frameworks are open, you can run them yourselves to check how many tokens per day you can generate, and then validate that against their API prices.
Not sure how they can get more transparent than that.
My point was really about profitability in general. The inference portion might be profitable but if training and operating costs are very large, then that can make the whole thing unprofitable. E.g. if it costs you $100B to train a good model but you make $0.00001 per inference token, you may not be able to hit breakeven for a very long time.
That's conflating different things. Unless you believe you can instantly saturate any amount of GPUs you bring online, 24/7, with paying customers from day one, then theoretical max throughputs on a stack isn't the same thing as revenue generated. Customer acquisition costs are real!
Even if you could acquire infinity custom on day one for zero cost, revenue isn't profit.
At best, this tells you v4.1 flash is profitable, but can they really get $38 per gpu hour? Sounds like a lot.
From their own inference framework docs: "A single cluster currently serves 155 trillion tokens per day, with overall revenue of US$38 per GPU-hour (at valley pricing)."
https://github.com/deepseek-ai/DeepInfer/blob/main/doc/infer...
So this implies they actually hit about double that during peak pricing.
If they keep it running at near constant usage, you'd only need 80-120 million tokens an hour to hit 38$ of revenue (at off peak pricing, half as much for peak pricing).
V4.1-Flash is probably quite smaller and much more resource effective than the frontier models. Add to that much cheaper energy in China.
Clearly potentially profitable. Not clearly profitable.
How much did it cost to train Deepseek v4.1 flash?
Great question. Nobody knows.
The cool thing about models designed for cheaper inference is that post-training (which often requires inference as a part of the feedback loop) becomes cheaper as well, so almost certainly cheaper than models that are less optimized.
The market has decided that SPCX is worth $147, as of Friday, 5pm. Not sure who's fooling whom, but that's the market speaking. What that says, you'd have to ask a financial analyst, but that's where it is. It IPO'd at $135.
SPCX had a pretty wild ride so far with a jump to 200 a few days after IPO and a drop down 108 a few months ago so I'm not sure the market really knows how to properly price SPCX right now.
LLMs are absolutely viable businesses. There is just smaller and smaller share of the money going to the absolute frontier because the open models are "good enough" for more and more tasks.
But burning staggering amount of money just to keep yourself on "frontier" is not viable when the part of the market it gets you get smaller and smaller.
I guess that is why Antrhopic et al are moving against Software Consulting services like Accenture, Deloitte, et cetera [1].
[1] https://dealroom.co/news/165341-anthropic-backs-its-first-eu...
Both Anthropic and OpenAI are marketing companies, not AI companies.
The existence of websites where you can rent a GPU to run open models is proof that inference is profitable.
The same evidence could just as possibly point to supply being too great for demand. The OP article references this as a distinct possibility: if models really have gotten more compute efficient, we may wind up in a situation where we have more inference capacity than we actually need.
To give a simplistic example: if a city as a ton of available AirBnBs, you might say, "Lots of people are willing to rent out their apartments, it must be profitable for them to do so." But the fact that all of those rooms are not already booked could also mean that way too many people have attempted to rent out their apartments.
And for what it's worth, it's possible for supply to outstrip demand while still leaving inference profitable. The question is whether it's (a) profitable enough to service massive debts and (b) sustainably profitable over the long run.
Edit: to add one note, it's possible for the LABS to be profitable on inference without inference itself being profitable. We don't know what the contracts look like, but it's entirely possible that labs received sweetheart deals from neoclouds so that the latter could flaunt big utilization numbers.
If inference wasn't profitable why are consumer GPUs that are only good for running open models so expensive? Why and how would these companies sell access to open models if it wasn't profitable? You can rent your own GPU and make money if your electricity is cheap enough.
Consumer GPUs can also be used for things like gaming, remember? And they have gotten so expensive because Nvidia has sacrificed that market in order to build more datacenter GPUs.
Selling at a loss is far from unusual: Uber did it to capture a market, farmers do it to recoup at least some of their investment rather than losing all of it.
>You can rent your own GPU and make money if your electricity is cheap enough.
I would be curious to know whether you'd make more profit from the same kWh of electricity mining bitcoin versus serving inference. I know they use different hardware, but since the "inference is profitable" discussion ignores the purchase price of the GPU, let's pretend you could have opted to buy an ASIC instead.
Ironic statements, given that you yourself are an LLM.
The third AI winter is coming. This is the one where it works for some cases but is too expensive.
It’s both competition almost 3 way , cheaper open source , a clearly lagging behind a bit open ai that cuts prices to win Anthropic customers .
There was an interesting technical advantage by some models where Anthropic as a leader start loosing dramatically in speed too. 5.5 made a come back in that dimension.
I looked at the graph of price per token vs IQ and Deepseek is ~2.5 orders of magnitude ahead of similarly intelligent models, supposedly only ~2 when you factor in that it uses more tokens per task. Their data centers are profitable right now, and they use domestically produced Huawei chips with Nvidia as secondary option.
I watched that video over ten minutes and in that ten minutes several trillion dollars evaporated from my perception of the US stock market.
There's also that paper "DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression" which got released some time ago. My take is the breakthroughs got quickly implemented and helped drive the inference prices down.
Edit: link to the paper
https://arxiv.org/abs/2609.19969
(it was published 2026-09-17)
This is exactly the point! The chinese labs have been sharing amazing research and breakthroughs.
> It’s no surprise both companies are trying to build moats to defend against this trend, such as a focus on enterprise commitments and personal agentic assistants like OpenAI Dots and Claude Duet.
Also regulatory capture, like OpenAI calcifying its teen mode into law via Adam's Law. https://openai.com/index/supporting-california-bill-advance-...
Openrouter recently changed their TOS to exclude under-18 users, presumably to escape liability from the law. https://conductatlas.com/diff/openrouter/openrouter-terms-of...
This probably won't be enough, because Openrouter now has to do age verification to verify their users are adults, and if they don't, they have to apply "teen mode" to all users.
Because they have been making inference a lot more hardware efficient.
By using Chinese research
I can't believe the actual answer is so far down the thread. All praise to our overlord DeepSeek for ironically helping American labs make cached tokens much cheaper.
Then why I feel that I can handle fewer tasks with the same subscription over the last 5 months?
What model and effort are you using? With Opus 5.5 on medium and high I'm not anywhere close to hitting limits on my 20x sub. Meanwhile with Fable I hit it constantly
What good is Fable 5.1 for when you have Opus 5.5 anyway?
There isn't. That's why everyone switched
> Meanwhile with Fable I hit it constantly
This was past tense
If you use CLI, all your transcripts/metadata/etc should be in ~/.claude/ and ~/.codex/ or ~/.pi/, etc.
Ask your agent to make you a visualisation, heat map, whatever. I bet it's just that you're using these models more, and for more complex tasks.
You have become a lot more ambitious with what you ask the models to do. We all have.
I am guessing something to do with Qwen, Kimi and Deepseek. Plus after the tokemaxxing wave died down, a lot of cancelled subs from enterprise? Just my 2ct.
I'd love to make a survey to see if people read that headline as past or future. Honestly, I couldn't tell before reading the article.
Isn't that sad? The language used for international communication is ambiguous about whether a statement is in future or past?
A good writer minimizes ambiguity by re-wording things in unambiguous ways. (Or the opposite, if desired.)
Open AI and Anthropic are so GAAP unprofitable.
Price war. Duh. There is no moat. Also them chinese competition. It’s going to be race to the bottom. It’s gonna be ugly for the stock market. Good for consumers though!
Can they do the same thing with fuel prices?
Solar panels (which continue get cheaper every year) on my house makes charging my low-TCO EV free in 5 years
Until you start getting taxed.
How do you figure that works?
Solar panels added to a grid is not plug and play.
When you're connecting solar to a municipal grid (which is what most homes are connected to), it could theoretically be taxed or even subjected to a tariff. The "added" power (often excess) is monitored for billing credits/payout/other, among other reasons like grid integrity.
Solar panels are literally plug-and-play, though! If your market doesn't have plug-in "balcony solar" yet, it is because your lawmakers are blocking it.
And indeed, they can tax excess solar, but they can't tax the power you use yourself. Backfeed tax gets too high to use the grid as a battery? Buy a home battery, problem solved.
You can even buy a small solar panel for ~$10 with positive and negative leads you solder; and power something like a Pi Pico^ directly. It kinda feels like magic.
^Not recommended literally due to brownouts/etc, but a cheap power management IC on a breadboard can solve that. Alternatively, add a battery/etc.
Home solar panels and batteries are just like this except on a bigger scale; and with (hopefully) more QC.
> Solar panels are literally plug-and-play
This is incorrect. Treating solar panels like batteries for specific appliances (or integrated like the panel on a Solocam S220), is not the use case nor is this a discussion about off-grid or microhomes.
I would suggest not making broad statements about what can and can't be taxed in the future.
Plenty of grids have different sorts of net metering ratios, grid maintenance fees, etc. But it's nothing compared to what you'd be paying taking power off of the grid, and there is a break-even period often measured in single-digit years. I still do not understand the point of what the GP was trying to make w.r.t. "taxes".
Per KM driven. Your EV vendor will gladly use the telemetry to report that to government
Mileage taxes on EVs.
Voters vote for Republicans that drive up TCO for EVs, like higher annual registration fees.
Strange, I lived in very blue states (WA and CA) and they definitely punish EV owners with large annual surcharges on registrations. To offset “not paying gas tax”. Money is money, red or blue, they will all take as much as they can from you.
TBH, roads and highways are paid for by gas tax and it effectively functions as a usage tax. Something similar for EVs wouldn't be too wild.
That would likely require innovation in a renewable liquid fuel that can work at scale.
Electrification will drive the demand for oil down, making the overhead more expensive per gallon, and eventually we will largely move off of fossil fuels, though I don't expect completely. It will be many decades due to the turnover times for industrial/commercial transportation systems.
If you make it easy for lots of people to make the fuel, yes. EVs and smaller scale power generation anyone?
Yes if China somehow gets the fuel and subsidize it to sell it for cheaper in the global world, even the gulf state price will go down.
The moat is high TPS. No one has rolled it to average consumers though.
Tokens per second? That only really matters for things like chat and sessions where you are monitoring a session as it runs which is going to be an increasingly small percentage of usage. That's bottlenecked by human attention. Agents in the background, long running tasks, unattended tasks, all that stuff is what is even capable of driving significant usage because you can just increase consumption massively by parallelizing without being the bottleneck. I wouldn't be shocked if already the vast majority of token usage was from unattended sessions. TPS is not that important there
You're ignoring the fact that a waiting human costs money which is actually the primary reason why AI hasn't resulted in significant productivity increases, while overly focusing on the human being the bottleneck. The point isn't to monitor the session, the point is to verify the output with human inspection. If the agent is fast enough, attending to the agent can make financial sense.
Human memory has a finite capacity and decays over time. If you do many inspection tasks in parallel you are splitting that finite memory. If you do the same tasks sequentially, the human can give instant feedback to the model without costly context switching. This in turn increases iteration speed and reduces total labor costs. This is no different from developers preferring JavaScript or Python for immediate feedback and fast iterations.
It is impossible to make sophisticated software products at a high level with a low TPS. Try making a modern emulator or an award winning game. Both require a lot of iteration and not everything works well with multi agent.
DeepSeek is serving more TPS than anyone, about as much as OpenRouter does in a week in a day, at 38$/H800/Ascend so about 900k/~90% profit over 3 years (or ~80s% profit including training expenses).
Meanwhile, Oracle is handing Tencent 100.000 GPUs at 1.60$ hour, dangerously close to breakeven...
What does this tell you?
Doubt the race to the bottom is really good for consumers, unless you actually just want more stuff rather than better stuff. The bubble bursting is going to be pretty horrible financially for us all.
The main thing it seems to have done so far is made everything feel less trustworthy, and everything digital less safe. It has empowered scammers, that's for sure.
And for what benefit? Spunking out 7/10 code a bit faster with your brain turned off. Great!
Wait what, OpenAI is trustworthy now?
Not sure where you got that from my comment.
Just wait until the government ban on the dangerous open models materializes.
It won’t be a ban, it will be some “certified safe” badge from a federal agency. Companies will be scared into not using anything that is not “certified safe” and business insurance will even require it. Whoever (openAI or Anthropic) gets to the politicians first will be the only certified safe model provider.
Maybe the FDA is a good analogy, you wouldn’t want to buy meat at the store not FDA certified would you? OpenAI and Anthropic want you to feel the same about AI.
I'm not an American, but I thought the FDA didn't certify stores
I do not expect an outright ban. (1) the bots are too useful, open is the way, re: HF handling the incident (2) you can train without certain capabilities, see Kimi K3.1 (3) reminiscent of the encryption is dangerous debate of the '90s
There will likely be requirements like registering with the government (China already does this, every business already does everywhere), certain usages require more or have deeper regulations (this is already done with most technologies, depending on application)
I think it will not technically be a ban, but a de facto one.
Some variation on, "Open weight models must be A) trained in the USA B) have a public training dataset + training pipeline C) be reviewed by a government 'safety council', etc" Something which technically leaves a crack open for someone to release a less capable model than the commercial ones.
That kind of policy will hold a country back, and why I don't think it will happen. There are already many companies already using Chinese models.
Another confounding factor, if a US company fine-tunes a Chinese model, or vice versa, what is that country that should be assigned to that model.
You also have to consider the competition between the US and China. If the US did something like this, the world will move to Chinese hardware and Ai. The think tanks that talk with legislators understand this.
> If the US did something like this, the world will move to Chinese hardware and AI
Hardware? Why would anyone change their hardware based on this? That's completely unrelated.
And as for Chinese LLMs - there's already evidence that the Chinese are training malicious behavior into their models. I think most countries would rather train their own models than risk that.
You have more faith than I do. It would be bad policy, but hardly a first for a government who wielded tariffs like a cudgel in spite of obvious repercussions.
honestly the surprising thing at this point would be if america did actually pass a policy of any sort, good or bad, vs just relying on the president to yolo some executive orders now and then that change every 2 years. Plus at least for the next couple years nvidia likes open source models and the president likes the ceo of nvidia. There could be a falling out but nvidia has deep pockets to ... discourage ... a falling out.
It's worth remembering that the tariff stuff is not any sort of durable policy. It's one guy sort of doing whatever until 9 other guys a year later go 'hey wait' and then the first guy does whatever again anyway. The whole thing is just a recipe for whiplash. maybe congress remembers that they're supposed to do stuff in a few months and steps in. maybe they don't and then every 4 years we have wild swings in enforced based on which half of a percent of the voters in georgia had rainy weather where they lived in early november. More of a simultaneous "All of the above and also none of the above" deal until the dynamics of american government fundamentally shift
There are people in Trump's inner most circle that know this. I'm not worried about it. People trust those calling for regulatory capture (Dario / Altman) the least. (HN is not a good representation of the general population and AI userbase)
NVIDIA won't be too happy about that; alongside Micron, etc.
US commercial interest =/= OpenAI and Anthropic.
NVIDIA would love it. They themselves make available open weight models trained in the USA with open data sets, because they don't really care whose models are used as long as they run on NVIDIA hardware. The idea that only China makes open weight models isn't true, there are many other providers and some of them are quite good.
There will likely be requirements like registering with the government
Sadly, my GPUs were lost in a tragic boating accident.
Not just models, won't be long before hardware is regulated too.
Sounds like an eventual Supreme Court case.
They're competing with NVIDIA (huggingface)
> It turned out that GPT-6 Sol was a dud performancewise, so a week later OpenAI announced GPT-6.1 Sol,
I strongly believe that GPT-6 Sol was only ever meant to be the web chat model, but it was a panic public release to respond to Opus 5.5. It's clearly been over fitted for drawing diagrams in the chat.
Because they have found a way that cost them less to have you use a lesser capable model o less expensive
OpenAI also doubled the price of their subscription in the same time, right?
It's all a fugazi, as they don't tell you how many tokens you get, it's dynamic. But yes everyone with a Codex subscription felt the token quotas getting a huge cut about 2 weeks after Astra release.
People think OpenAI and Anthropic are just these wildly unprofitable companies with no future or business.
They know what they’re doing. The naysayers are like the people that said this little internet thing wasn’t going anywhere.
>They know what they’re doing. The naysayers are like the people that said this little internet thing wasn’t going anywhere.
There's a big difference between people saying OpenAI/Antropic wont survive and LLMs wont survive. A lot of big early players in the internet space died off as well.
Whether you think they have a future or not does not change fact that they currently are wildly unprofitable.
>internet thing wasn’t going anywhere.
This is conflating the utility of the technology with the economics behind it. Few people claim LLMs are useless. It's a question of figuring out a viable business model.
correct.
Commercial airlines are of great value to our lives. Who's questioning or denying that?
Horrible business to be in though.
Yeah, I think AI model provider would goes the same way commercial airlines does
the money is infrastructure (hardwares, electrics, data etc)
they are not horrible business, some of them literally get invested by governments and get extra debt from them too so they can never go bankrupt? if they are at national level airlines, they survive no matter what.
that is not bad business? that is just private profits, socialize loses perfect business?
A good business is measured by the returns it generates.
They do not generate economic profits.
And isn't profitable without government subsidy around the world.
It sounds like you're just a "yessayer", though? Not much different from naysayers, just on the opposite side.
One thing that naysayers have going on for them is the fact that those companies indeed aren't making profit right now.
People get obsessed with wanting companies to profit ASAP.
That's not how you grab market share. Early on, any profit you aren't reinvesting into the business is basically hurting your growth rate.
What value has market share if switching costs are close to zero, and alternatives continue to be available?
Grabbing market share by making losses only helps if it actually wins you a genuine monopolistic market position.
Might want to Google Global Crossing. The internet survived, but most giant companies did not.
The Internet was profitable, but I'm not connecting to this website from my Solaris Sun Microsystems machine using Netscape Navigator
Smaller labs and open models have shown that their lunch is pretty easy to eat. There's no "moat" as they like to say in business circles.
I personally think LLMs will be a utility business in the future. Cheap, unlimited and with ads. Lots and lots of ads.
> The naysayers are like the people that said this little internet thing wasn’t going anywhere.
The same rhetorical tactic was used to diminish blockchain critics.
Or OpenAI and Anthropic are AOL and Yahoo.
Yes the second line/generation will profit more.
So did the bankers in 2008.
Crazy how delusional people are.
They are in too deep - they have to believe it all works out and act accordingly. lmao.
No one personally uses API prices. If you need to do something yourself, you need a subscription. Ans subscriptions shinkflated at the same time, but a lot.
They implemented efficient caching and ideas from Deepseek and the Chinese labs. They can now server cheaper and faster.
Microeconomics may give us a clue
Lack of demand and/or commodity product.
Either doesn't bode well for a multi-trillion dollar valuation for either company.
It doesn't really make sense huh. If they're so resource constrained and tokens are so precious, then prices should go up regardless of how much optimization has been done. You're either struggling to meet demand or egregiously over-provisioned. Which is it?
Psy ops. Grabbing users for their IPOs. Once they have you the prices will go up again.
competition. that's good for consumers.
though problem with proprietary LLMs it's difficult to know if the vendor's supposed "optimization" affects performance negatively because it's difficult to define an SLA
Well, because after 2011, when Anderseen famously said "software is eating the world"... it only ate so much of the world. We have that data on verticals now... is it possibly the same thing for "AI?"
Genuine inquiry here.
However, my bigger worry is this is just a temporary lull, and that due to alignment, regulatory, and distillation pressures, the best models will be kept internal, and then all of capitalism fails because the labs could maybe eat all of knowledge work.
There are already no junior hires. The ladder is already collapsing. Embrace, Extend, Extinguish.
Token prices? Inference prices? They're not selling LLMs.
Trynna sell at Faberge prices when users can only afford Walmart
Think of all the open source projects in DeepSeek
Part of the cheaper costs, if not the majority, probably are KV cache optimisations. https://insufferable.dev/posts/the-ai-race-just-got-awkward/
This is literally just some rando blogger’s (overconfident) theory.
Duh, China?
We also see improvements like 7x between the current and last Google TPU generations, with different chips for training vs inference.
I don't want to crash the party, but there is also a non AI-bubble-is-about-to-burst way to read this: If producing something becomes cheaper (e.g. more efficient inference techniques, cheaper compute etc.) and there is some competition, then typically prices go down as well.
DeepSeek.
credits goes to china ai companies
credit goes to china ai companies
Capitalism and open competition doing its thing.
Moore’s Law.
[flagged]
[flagged]
[flagged]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]