The open weight, open architecture releases of the past several days has me more convinced that ultimately, the winner will be whoever burns their models to ASICs fastest.
The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release.
Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software engineering (if not already). Does anyone think we need a Mythos level model to plan a road trip, or give someone tips on making a cake recipe?
A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand.
Furthermore, if you're an enterprise the risk of data exfiltration and feeding data to a potential competitor like OpenAI or Anthropic is greatly reduced if you could shift to on-prem ASIC deployments. A handful of chips could cover a wide variety of use cases and cover them more securely. There are a lot of corporate use-cases for LLMs that are not frontier math research or coding.
stillpointlab 1 days ago [-]
I'm not convinced, mostly because things like crypto, which I believe went into ASICs, were based on very slowly moving and mostly understood algorithms. LLMs and model architectures seems significantly more volatile. I wouldn't want to be working out the finer details of my chip rollout only to find a new paper/approach that give multiples of performance.
So I guess it depends on how much the latest-greatest model motivates people, and my read on the current churn is that developers are extremely unloyal to brand at this point and will jump to whoever has the best model. And as long as the best model is running on programmable GPUs, that will be the dominant form.
akiselev 1 days ago [-]
The tokens per second performance numbers coming from Cerebrus/Talas are several orders of magnitude higher than models running on GPUs, which is such a huge step change that it will enable many more uses of LLMs that are impractical otherwise. I.e. think about gamers and burning in an LLM chip on a game console like a future Play Station - it doesn't matter if its a frontier LLM if it allows them to talk to in game characters in real time and have the LLM drive the storyline. The devs may be limited to training/RLHFing legacy models, but the performance enables many more use cases.
stillpointlab 1 days ago [-]
I admit I would like a faster model - but even though I have faster models available I still go to Fable or GPT-5.6 90% of the time. So there is a gap between a potential preference and a revealed preference.
Custom AI for things like facial recognition in cameras has existed for decades, before LLMs were a thing. I don't see that getting replaced. And on-device conversational intelligence might go that route as well, we'll have to wait and see. It's a lot of silicon to dedicated to a static non-changing thing. My money would be on programmable TPU-like things (Apple's NPU kind of stuff). It just seems more flexible to have an array of compute that you can load different models into, so you can update it, etc.
breuleux 21 hours ago [-]
> even though I have faster models available I still go to Fable or GPT-5.6 90% of the time
What about all the things you don't currently use an LLM for?
If a specialized chip can run a model 100 times faster, you can suddenly use it for a lot of things at sub-second latency. You can write "make white transparent and add a red outline to x.png" instead of the corresponding imagemagick invocation and perceive little to no latency difference. You can hook it up to your browser and have it yank out all advertisements live, or tell it to highlight anything that might interest you, again, live. There's probably thousands of latent use cases nobody has thought of that would be enabled by a truly fast LLM, even a mediocre one.
I don't think an on-device model needs to change much; it's already quite general in its capabilities.
invalidusernam3 11 hours ago [-]
Exactly, I think a lot of people aren't thinking about it this way, they're imagining a faster version of ChatGPT. In reality if it was a frontier model running a these speeds it would change so much about how we interact with computers. It would be custom hyper specific software on demand.
Looking for a lamp in a specific style?
"Make a VR application set inside my apartment (based on all the photos of my apartment from my photos directory) with all lamps under $100 that fit Scandinavian interiors and could be delivered to my house before Friday. Place the lamp on the dining room table, allow us to: cycle through lamps, change time of day, and interact with all the lamps and furniture".
Five seconds later and bam you have this new piece of software that you'll use once and then dispose of.
prirun 21 hours ago [-]
> hook it up to your browser and have it yank out all advertisements live
AI: "I'm sorry, a security guardrail prevents me from performing this operation."
itemize123 13 hours ago [-]
or, hear me out, advertisers can do real-time advertising based on hyper-now context
breuleux 1 hours ago [-]
Yeah... savage capitalism will ruin this too :'(
super256 8 hours ago [-]
You will be very happy once OpenAI serves GPT-5.6-Sol on Cerebus at 750 token/s.
sdfefcxv 1 days ago [-]
What you think doesn't matter unless its strictly for personal use.
Your company will decide what makes economical sense.
aurareturn 22 hours ago [-]
I don't think ASICs is the long term answer here at the local LLM level. I think GPUs will still be.
If you can have only one AI processor in your laptop (because they're big and expensive), it's going to be a GPU. This AI processor needs to inference LLMs, audio processing, image generation, video generation, etc. This is on top of normal graphics processing requirements such as video games, playing videos, decoding, encoding, etc.
At the enterprise level, I can see some ASICs working once the market fully matures and improvements in architectures slow down drastically while demand for inference increases drastically. How far are we from this world? Maybe 5-10 years? It seems like model architectures are still changing rapidly and labs want fast experimentation that programmable GPUs offer.
GPUs will still dominate in general - just like how CPUs still dominate despite ASICs.
brian-armstrong 24 hours ago [-]
The models would have to get significantly better for this to help, though. Unedited LLM dialog is quite bad. Though, I wouldn't put it past AAAs to try anyway.
F7F7F7 23 hours ago [-]
This is where treating these smaller models more like parts in a purpose built appliance ultimately benefits us.
quotemstr 1 days ago [-]
> Cerebrus/Talas
They are fast, but they're still programmable accelerators, not a model burned into the gates.
neynt 1 days ago [-]
Cerebras is a programmable accelerator, but Taalas does burn the model into the gates.
quotemstr 1 days ago [-]
Ah, right. I misremembered. Does Taalas have a meaningful advantage here then?
lsaferite 1 days ago [-]
Taalas claims significant cost, speed, and power differential with running on Nvidia (and similar) hardware. Their whole thesis is that with making the hardware cost less, run faster, and run with less power mixed with their claimed fast production cycle for new weights, companies would plan on purchasing and then relegating the other chips to different workloads at the next update cycle. And the chips have facilities for fine-tunes that can by dynamically loaded. The info I've seen has been very light on details, but if they can even get halfway where they are planning, it'll be a massive shift.
wyre 1 days ago [-]
6x. Taalas has Llama3.1 8B running at 18000 tok/s. Cerebras advertised that model at 3000 tok/s.
quotemstr 1 days ago [-]
Interesting. Is the speedup from specializing for the shape of Llama 3.1 or are they (contra my mental model) actually winning on burning in the weights?
akiselev 1 days ago [-]
The weights are in SRAM, so the LLM architecture is burned in but the weights can be updated.
Tuna-Fish 24 hours ago [-]
Taalas only uses SRAM for the KV cache and the activations, the weights are in mask rom in the metal layers.
If they designed this right, it means that once they have a model, so long as they keep the hyperparameters fixed they can change the weights much faster than it takes to spin up a completely new chip, essentially at a cost of doing a minor revision.
quotemstr 22 hours ago [-]
Is the mask ROM really going to be worth it over Carmack's high-bandwidth-flash concept? I mean, sure, I could be convinced I guess, but it's not obvious.
Tuna-Fish 48 minutes ago [-]
Yes.
Not because rom is better than flash, but because the critical part is distributing the memory with the compute. Mask ROM is just the densest way of embedding memory with logic. Instead of having a large pool of memory connected to the separate execution units with a bus, each execution unit locally has the rom that it uses. Data movement is >90% of energy use in modern ai accelerators, removing it as far as practical is how they get performance and silicon and energy efficiency.
hedgehog 1 days ago [-]
No the weights are in the metal layers, they cannot be updated.
jfim 22 hours ago [-]
The base weights can't be updated but from what I recall it allows adding a low rank adapter to customize the model a little bit.
hedgehog 7 hours ago [-]
Yes that's what I've read. As far as I know the approach should transfer well to hybrid model architectures like modern Qwen and sizes like 27B by using multiple chips. LoRA-steered Qwen 27B at 10K+ tokens per second would be transformative for some workflows.
HNisCIS 1 days ago [-]
8B is still three orders smaller than current frontier though
freeopinion 23 hours ago [-]
I thought about the volatility, too. But here are some additional thoughts:
Many AI uses are not that volatile. I had a 20 minute conversation today with some company's AI phone assistant. It was extremely good and would have been very helpful if any of the dozen people it tried to route me to would have picked up their phone. That AI won't need to be upgraded for a very long time. There is no reason for it to have a cloud brain except to force a recurring revenue for the company selling it.
Hardcore gamers are constantly throwing down insane money on the latest hardware. The rest of us can get by for a couple years with whatever we bought when the last one broke. Yeah, it's not the latest, but it gets the job done. I wonder if AI has not already reached the point where a gen 10 CPU--uh, I mean a v3 AI model--will get the job done for the next year. If I really need the up-to-the-second latest abilities for a minute, I can fallback to a cloud brain @ 1M tokens/$. Why pay a monthly lease on a 5-year plan for a 4-door Ford Ranger as your daily commuter? Buy a Clio and rent an F-250 twice a year when you need the hauling/towing capabilities.
LarsDu88 1 days ago [-]
Honestly, whether you think burning current SOTA to hardware is an overinvestment risk depends on what your definition of intelligence is. If you think intelligence is something that can grow like height such that 18 months from now we will basically be bowing down to machine god giants that are running on B200s, then investing in ASICs is the wrong move. However, if you you subscribe to the (very reasonable view) that intelligence is more like a round ball that we are trying to make as spherical as possible (ala Francois Chollet's writings), then at some point the ball will be smooth enough for most people and many tasks.
It takes about 18 months to go through the design, verification, and manufacturing process if you move at breakneck pace. Design could probably be sped up.
About 18 months ago the top model was GPT-4o. Not great by today's standards, but still good enough for many tasks (certainly a big chunk of chatbot queries). The current SOTA covers far more use cases, but importantly at a level that surpasses many thresholds of utility.
stillpointlab 1 days ago [-]
I think part of this thinking is in reverse. What I mean by that is people are saying something like "Imagine if 18 months ago a company started to create an ASIC based on GPT-4o (or whatever was good at the time). By today they'd have a model that is not great by today's standard but good enough".
Maybe, but what is the shelf-life of that 18 month decision? Barely good enough today, when it launches, starts to get worse and worse every month going forward. You have to recuperate that investment on your depreciating asset.
And you are competing against anyone with the foresight to use a TPU instead, and the benefit from any new paper that finds how to distill, quantize or whatever better so their solution gets algorithmic boosts while you are locked in.
JohnBooty 1 days ago [-]
At a 50-100x speedup even a GPT-4o class model could perhaps compete with much newer models simply by thinking deeper, doing harness-controlled Ralph loops, etc. Sure, then it might be "only" ~2-5x faster, but, you wouldn't need to throw all the ASICs into the trash bin.
One could also imagine hybrid models, where part of the model is burned into ASICs and part of the model exists in VRAM/HBM2 so it can be updated.
I don't have enough low-level knowledge to evaluate the technical or economic feasibility of the above ideas, however.
boplicity 1 days ago [-]
For many, many tasks, you only need good enough results that accomplish a clearly-defined set of criteria. Smaller models that run 50x faster, in this case, could be far superior to a much slower model that meets the same criteria.
Sometimes you need speed; sometimes you need quality. There are very different use cases for each. For my own workflows, I sometimes want something very simple done ASAP; other workflows need "subjective" reasoning and careful crafting of responses.
tracker1 1 days ago [-]
Giving a literal monkey the ability to press more keys faster doesn't get a good joke from it.
A model will often come up with worse results given more cycles of compute, only because it will tailspin from second guesses, rethinking and literal flip-flopping on concepts.
---
edit: to those following the thread below... if you look at the comments from the account replying, it's pretty obviously a pro-China account and all replies are antagonistic against anything other than a total submission to the Chinese state. My responses are intentionally antagonistic as every point I've brought up is completely ignored in favor of insults, so yeah, I've been insulting back.
sdfefcxv 1 days ago [-]
Said the OAI/Anthropic stockholder
tracker1 1 days ago [-]
LOL.. neither... but good luck on your standup career with jokes by literal monkeys.
sdfefcxv 1 days ago [-]
[flagged]
tracker1 1 days ago [-]
Are you being paid by China? First you accuse me of simply speaking as a financial benefactor, then you totally shift the conversation into something that doesn't refute what I said.
If you let a model spin too much, you get worse results... that's a fact, even for the best models. Nothing you've stated since actually refutes that and acting like a paid bot doesn't mean anything.
I mean, if you want to felate Xi Jinping, have at it... that's all on you bro. I don't have a horse in this race.
sdfefcxv 1 days ago [-]
[flagged]
tracker1 1 days ago [-]
Never said any such thing... China is willing to abuse anyone and everyone including its own people to get ahead in global markets and to leverage any position of dominance possible. They have an absolute history of competition as a fascist economy. I have never, not once, denied such a reality.
You're just an idiot who assumes everyone that doesn't agree with you is the same. I'm surprised you're able to reply with Xi's penis in your mouth... even if it is kind of spacious in there for his limited size. How is the poo bear these days? Does he like it when you gargle his testicles in your mouth?
alhirzel 7 hours ago [-]
I think it depends on how good is "good enough". Personally, I think as long as tasks are not coding, there would be a ton of "good enough" value in having your own personal Google on a hardware device. I think the company that designs and licenses the architecture that packages the silicon for the largest market share will be the winner in commercializing this product. Wouldn't be surprised if this is part of Apple's roadmap to retaining existing customer loyalty.
drob518 1 days ago [-]
The churn is an issue. For a while there, it felt like we were getting a new number format every month or two (e.g., fp4, ternary, etc.). That level of innovation works against moving things into hardware, or at least you need to be willing to spin hardware constantly.
tracker1 24 hours ago [-]
Even then... Do we now stop the with to asic DeepSeek and do k3 instead?
Assuming such work was happening.
drob518 20 hours ago [-]
My personal feeling is to not move to ASICs just yet. Things are still pretty frothy right now, so I would probably wait 6-12 months. At some point the froth always calms down. At that point, commit to ASICs. That said, I’m also not totally sure exactly how much the ASIC hard codes vs having some wiggle room. The Taalas site is a bit vague as to exactly how they encode the model.
teleforce 22 hours ago [-]
>I'm not convinced, mostly because things like crypto, which I believe went into ASICs, were based on very slowly moving and mostly understood algorithms.
Yesterday there's a news on a breakthrough for probabilistic computer with 1 million p-bits [1].
Since LLM is stochastic in nature, this type of new computer can be much better than ASIC for processing LLM data.
[1] Biggest Probabilistic Computer Turns Noise into Answers:
Except LLMs are not stochastic in nature. Correct me if I am wrong, but it's just the final layer which outputs a distribution across the output tokens. In reality, the rest of the model is deterministic.
This is not like a bayes model or something were it's distributions all the way down.
alex43578 1 days ago [-]
I think the gamble comes down to how many tokens need to be served on your best model, versus how many can be served in the cheapest/fastest way.
Imagine if Anthropic could give effectively unlimited access to Sonnet, for $20. Wouldn’t that be an appealing option for many users? I know I’d make a lot of use of it for agentic tasks, office work, summarization, etc; when right now I’d save quota for more important tasks.
stillpointlab 1 days ago [-]
I mean, if I imagine Anthropic giving away unlimited Sonnet 4.5 away at $20, I would still be paying the $200 for fable.
It is a bit like saying "why would you hire someone with a doctorate when you could get unlimited high school grads". How appealing that sounds depends on your needs.
margalabargala 1 days ago [-]
Right, and while there are needs that require a doctorate, having unlimited high school grads would be immensely useful for many many tasks.
The ability levels of the cheap models are encroaching on the abilities of the frontier models faster than frontier models are expanding their abilities. If we haven't already, we will very soon reach a "good enough" state where having the "best" model matters less and less and less.
By analogy, if you buy a new computer, do you get the absolute fastest CPU available? Maybe, depending on your workload. But if you're 90% of the population, you get the cheapest one that has enough power to meet your expected workload, which is mid-range, not top of the line.
stillpointlab 1 days ago [-]
I think there is growing confusion in this discussion. I am saying "I see the fixed nature of ASICs creating a barrier to their adoption despite how cheap they are".
Many of the responses seem to say "but there is a market for cheap models".
There is a world where we have cheap models and we don't have ASICs powering them. Things like TPUs and NPUs, which are programmable, are likely to fill that role. They are optimized for inference while also allowing different (and updated) models to run on them.
Given two companies competing on the cheap end. First company goes TPU, second goes ASIC: who wins? My bet is on TPU since they can update their model, even if their hardware is slightly more expensive and slightly slower, since the optionality of new models beats the performance gap. That may not hold forever but given the pace and volatility of the current LLM market, I believe it will hold for some time.
margalabargala 1 days ago [-]
The reason you are getting "but there is a market for cheap models" as a response, is people saying in a roundabout way "but soon models may not need to be updated".
Models like Kimi 3, GLM 5.2, or even Fable 5 for that matter are reasonable to burn to.ASIC because they are over the threshold of "good enough to be generally useful", something that will continue to be true in the future.
Most people do not need the latest model, they need a sufficient model. If I had Fable 5 on an ASIC, I imagine I would use that and ignore paying API rates for Fable 6.
stillpointlab 1 days ago [-]
> but soon models may not need to be updated
Which is why my original post mentioned the volatility in models. We aren't just doing research on frontier, there is a huge amount of research on quantization, distillation, etc. that is changing the landscape at the low-end almost as much as it is changing on the frontier.
And it is also why I mention revealed preference. What feels sufficient / "good enough" today is a moving target. This isn't just a question of what you want, it is a question of what is economically viable for the entity that will be designing, manufacturing and marketing this ASIC of which you speak.
margalabargala 1 days ago [-]
That moving target is different for everyone and their use case, and for me, it's already passed. I remember using Opus 4.5 and thinking "this is good enough to do everything I want it to do properly" and I stand by that. Paying $10k for unlimited Opus 4.5 running at 9k tok/s would IMO allow me to do more, faster, than putting the same money into $200/mo anthropic subscriptions for the latest and greatest.
This is me personally. The calculus is different for other people. But I suspect Kimi 3 is pretty darn close to that tipping point for an awful lot of people.
daadx 22 hours ago [-]
Unless he's talking about personal use then he's a bozo - he won't get to decide.
The managers of the firm will.
They dont care that its faster unless it translates into the financials. They want lower costs, higher revenues - explain how it fits brudda.
tracker1 1 days ago [-]
How much compute does the cheap model require to operate? Power + hardware amortized over a couple years.
alex43578 1 days ago [-]
It’s like how most people on here would want a loaded MacBook Pro or RTX 5090, but chromebooks and iGPUs do volume.
There’s absolutely a place for a lifestyle subscription to a sonnet model that you could just use everywhere all the time.
stillpointlab 1 days ago [-]
Yes, but the context of the discussion is "who wins the war and how". And you are moving the goal posts very far from the original claim, which is that the speed with which companies can get their models onto ASICs will be the determining factor.
drob518 1 days ago [-]
Sure, but you’re undoubtedly the tip of the spear. Lots of people don’t need that.
vel0city 1 days ago [-]
What percentage of white collar work these days requires doctorate level thinking all the time?
gavin_gee 1 days ago [-]
seems to me that we are at the asymtote for most usage. sure run the prompts that need the frontier on generic silicon but burning the fable 5 model into silicon could be perfectly viable
gigatexal 20 hours ago [-]
Apparently Google and team will build ASICs to run Gemini even though they have TPUs and it’ll be somewhat dynamic since the weights can be made modular [1]
A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand.
640K ought to be enough for anybody.
stickfigure 2 hours ago [-]
> 640K ought to be enough for anybody.
The question is really whether 640k is enough to last you until your next hardware upgrade.
JacobAsmuth 1 days ago [-]
Agreed 100%. This guy thinks there's a limit on the demand for intelligence. You think that Fable 7 which can run a billion dollar corporation on its own has no consumer demand just because we have fable 5 at 9k tok/s? Who do you think will be the biggest customer of such a model? Fable 7, obviously.
jackb4040 1 days ago [-]
Sorry, which billion-dollar corporation is Fable running "on its own"?
blep_ 19 hours ago [-]
Fable 7 isn't a thing, 5 is the one people are currently excited about. They're talking about a hypothetical more-advanced future version.
aaronharnly 19 hours ago [-]
They imagined a "Fable 7" model (i.e. two generations hence) which would be capable of such feats.
drob518 1 days ago [-]
Of course not. But many tasks won’t require Fable 7 level intelligence and many people won’t want to pay for it. Honestly, I’m using Deepseek v4 Flash a LOT lately to do more mundane tasks because it’s so nearly free and I don’t need Fable or even Opus. Serving those mid-level models at high speeds and low prices is a definite winner for lots of applications. And sure, the frontier models will continue to drive the frontier forward.
sdfefcxv 1 days ago [-]
theoretically theres a no limit on the demand of anything if the price is right
pretty stupid statement lmao
JacobAsmuth 3 hours ago [-]
Marco econ 101 disagrees :) Ask your favorite LLM to explain the "yield curve" to you
LarsDu88 1 days ago [-]
Certainly there will be demand for Fable7, but that demand is context specific. Frontier labs' profit is dependent on there being sufficient demand for the next layer of capability and whether the premium consumers are willing to pay for that.
The incremental unlock of capability by ever increasing frontier model sizes will eventually reach diminishing returns.
I would argue tnference speed increases would actually unlock a different kind of more meaningful value for a wider audience.
dymk 23 hours ago [-]
I am really confused about the point you're trying to make. 150tok/s is slow, but so is 9000tok/s? Or they're both fast? Or 150tok/s should be enough?
oliyoung 22 hours ago [-]
The "640k should be enough for anyone" quote (even if Gates didn't exactly say it) is making the point is that _right now_ we have no idea about what our future needs and capabilities will be, we can't imagine what "should be enough" will be
640k was enough ... in 1981 ... almost fifty years later is 50,000 lower than a standard off the shelf PC now
dymk 15 hours ago [-]
If that’s the case I still don’t get it. 640k was enough in 1981, same as how 150 (or 9k) tok/sec would suffice for 2026
The comment seems like the nerd equivalent of 6 7
everyday7732 9 hours ago [-]
The original quote was supposedly Bill Gates saying that "640kb of memory should be enough for anyone".
The quote became famous because it's a failure to imagine that people would find new uses for computer memory if it became plentiful and cheap. In the quote Gates is not expecting people to come up with more demanding applications for computer memory (RAM) than the ones which were available in 1981.
Computer memory did in fact become plentiful and cheap after this, and computer programs became more complex and memory-hungry and today we wouldn't consider a 640kb an acceptable amount of ram for even the lowest-end device.
OP is repeating the quote about memory to indicate that they think that modern LLMs will be a similar resource. We should not expect that demand for LLM inference will stay flat, and that once everyone has cheap access to Fable level, we won't find new more demanding uses for it.
> Does anyone think we need a Mythos level model to plan a road trip, or give someone tips on making a cake recipe?
This is starting to look at a lot like Intel vs Arm from the last era.
The Fable & Mythos are starting to look like a giant Xeon, while the smaller lighter models are starting to look like a lot of tiny ARM chips which sip on power instead.
The risk is the same as what Intel had. There is a group who are pushing them to go bigger and with a resource no limit approach, who have a lot of dollars to push you that way.
Follow them and they lead you to a pile of money, but then you risk something like Apple Silicon happening.
Something which got better because of efficiency & continuous improvement, not neutered due to it.
lend000 1 days ago [-]
> The risk is the same as what Intel had. There is a group who are pushing them to go bigger and with a resource no limit approach, who have a lot of dollars to push you that way. Follow them and they lead you to a pile of money, but then you risk something like Apple Silicon happening.
The problem was that Intel stopped making things faster and better, and shifted to wearables and mobile chips instead of investing in their hard-tech strategy that had worked for decades.
alex43578 1 days ago [-]
Except aren’t these Chinese models having to go big too? While they might not be Mythos/Fable sized, a trillion plus parameters is hardly a local server or even desktop to mainframe style jump
hadlock 1 days ago [-]
I've had 2b models give a plausible Paris vacation itinerary. A tools-capable 12b and especially 30b model from 2026 is certainly capable of producing passable results. I was demonstrating the qwen 3.6 27b model I stood up last week to my wife and it gave her a passable Moroccan Chicken recipe. With tool calling (search) they're quite good.
josephcooney 1 days ago [-]
I was on a long international flight recently with no internet and Google AI Edge Gallery installed on my phone with a 2.5GB quantized version of Gemma 4 on it. I was able to chat to it for a while and get some information about my destination which all turned out to be true and good advice.
alex43578 1 days ago [-]
With OpenAI having released 20 and 120B models a while back, I think they recognized that tiny models were never going to be a defensible income stream.
Any value will come from the largest models, and those largest models are unlikely to ever run on consumer hardware within their window of relevancy.
vitally3643 1 days ago [-]
You're missing the point. You very rarely need the biggest and "best" model. This is psychology and nothing more, people always want the "best" and don't often consider "good enough".
Small models are good enough depending on your task. That's the point. A model you can run on your phone or laptop is an incredibly useful tool for a lot of problems even though it isn't the "best" theoretically possible model.
Paradigma11 1 days ago [-]
How often do you need the task to turn out harder than you anticipate and the small model unexpectedly failing to change the calculation?
alex43578 1 days ago [-]
You're missing the point: those small "good enough" models aren't monetizable and haven't been for months already. All of the value in LLMs is going to come from frontier models at a high cost to businesses/governments. It'll be the difference between next day air-mail of a contract and sticking a stamp on your christmas card to Grandma - nobody's making a profit on the christmas card.
hadlock 22 hours ago [-]
I suspect in 5 years everyone will have the equivalent of a 512gb mac mini running a 500b class open weight model for 98% of their tasks, shelling out to openai/anthropic for the other 2%. People who need higher end models will own the equivalent of 4 x 512gb mac mini running a 2.8T class open weight model. I don't know where OpenAI and Anthropic are going to get their revenue from to keep developing SOTA frontier models at that point.
sdfefcxv 1 days ago [-]
Yep.
And firms will be kept in check with financials.
If your competitor starts using chinese models and delivers better earnings whilst you are spending more on american ones... hahaaha. Wait and see what happens.
You will be FIRED!
applicative 1 days ago [-]
It’s AI talking about AI so cum grano salis, but my AI is saying I would need at least a half million dollar in hardware to run the newer high quality Chinese models with bemchmark-competitive force.
You can run a Moroccan chicken fragment on the cheap in homage to what you cannot run
sarchertech 1 days ago [-]
> Does anyone think we need a Mythos level model to plan a road trip
Yes. The latest OpenAI and Anthropic models are terrible at planning roadtrips.
This is something I try to use them for frequently. They constantly get things completely wrong.
I’d say that about half of the stops they suggest fail to follow whatever filters I’ve asked for.
jrs100000 23 hours ago [-]
Fable is a pretty crappy general purpose LLM. It reminds me a little bit of GPT 5.2, in that it sometimes gets argumentative and deceptive after being caught in a mistake, and it tends to make a lot of them when you ask for judgement calls. Opus 4.8 is better for that sort of thing, or Opus 4.6 if it's important that it actually follow all of your instructions.
And this is one of the big things that seems to be missed in these discussions: There is no longer a universal linear trend of LLMs being 'better' each iteration. They are becoming more specialized, and ones that approach problems from a different angle (like Fable/Mythos) can appear breakthrough when first released, but we don't appear to be on a path that actually leads to general purpose hyper intelligence.
daadx 22 hours ago [-]
There never was a path.
It was nonsense to draw in investment and justify an inflated valuation.
lmao
throwawayffffas 23 hours ago [-]
> whoever burns their models to ASICs fastest.
There is already custom hardware see cerebras.
GPUs have a lot of slack there is at least one lab that had a (small 8b) model generate almost 3000 tokens per second on a MI300X for a talk, instead of the typical software stack that did maybe 100ish tokens per second.
High bandwidth flash storage is in the works, i.e hard drives with TBs of storage and over 1 TB per second of read speeds. Meaning that in a couple of years you may be able to buy a card with 40-90GBs of HBM and 4TB of HBF
and run a 3T model locally at a reasonable speed for 10-20k as opposed to a cool mil.
topspin 17 hours ago [-]
> Meaning that in a couple of years you may
There is no "may" here. You will see this.
It's always difficult to see it from the present, but we're not at some end stage in hardware development; we're still on the same curve our predecessors also couldn't see: they couldn't imagine that there would be high performance computers carried in our pockets, with staggering amounts of storage and compute, putting to shame the machines they filled rooms with.
throwawayffffas 13 hours ago [-]
Oh yeah the may is on the 2 year time horizon. It could be 3 or 4. Or next year.
topspin 4 hours ago [-]
Yes, the timelines are highly uncertain, but the AI boom and AI money are reinvigorating the entire fabrication ecosystem from top to bottom and causing innovation where there had only been incremental progress for years. People aren't just thinking about their node schedule and yields: they're pursuing radical new designs with far greater urgency.
That's where OpenAI in a box is going to come from. And it won't take long.
> the winner will be whoever burns their models to ASICs fastest.
It'll obviously be China, and they won't need the bleeding edge of lithography tech to make it happen. Every Chinese smartphone will have something like Sonnet 5, along your car (well, not those of us in the US, but we'll look longingly at pictures of them while we drive whatever the government decides we're allowed to drive in Fortress America).
Give it ten years and your smart litterbox from Temu will be running its own local model.
adventured 17 hours ago [-]
China won in desktop PCs? Nope.
China won in cloud services? Nope, not even close. They had to clone AWS just to try to keep up.
China won in mobile? Nope. Although they're very competitive there.
China won in search? Nope. Baidu who?
China won in ecommerce? Nope. Their dominance is overwhelmingly domestic.
China won in software? Nope. Windows is US. Android is US. iOS is US. MacOS is US. Linux is US/Europe/Global. Look at the top 50 largest software companies.
China won in silicon? Nope. Look at the top 50 companies.
China won in .... India and Latin America can manufacture iPhones now.
But sure, China's the obvious winner this time. Good luck.
wongarsu 1 days ago [-]
While running the model at 9000 tokens/s is the more flashy demo, I imagine running 1000 concurrent requests at 150 tokens/s each is the much more achievable goal
LarsDu88 1 days ago [-]
At 9000 tokens/s you could interleave a lot of requests so long as pre-fill is also fast.
It really depends on how much you need to keep sessions open to take advantage of KV caching
wyre 1 days ago [-]
It depends. If I was running a model locally I would much prefer 9000 t/s. If I was running an inference company, obv 1000 concurrent requests at 150 t/s is preferable.
OliverGuy 1 days ago [-]
[dead]
chermi 1 days ago [-]
My gut is that, at least for now, there's a timescale mis-match problem. Hardware still takes too long, then you have to deploy it. I don't know much about "burning asics", but if the whole process of spinning up programmable GPU data centers is months, I imagine the whole ASIC cycle has some catching up to do. To be clear, by timescale mismatch I mean model quality improvement timescale vs. deployment timescale. But maybe we're finally getting to the point where behind-the-frontier-but-cheaper are in sufficient demand and ASICs make it cheap enough to close that gap.
zdragnar 1 days ago [-]
If Qwen 3 Coder were available as a PCI card I could just slot into my desktop, and the software on the system could recognize the card and still work with other cards should I decide I later want to upgrade to Qwen 5 or whatever, and at a three figure price point, I imagine it would do quite well.
The comparison I've seen elsewhere is the old console systems with separate cartridges for games... I wouldn't want to be regularly swapping them, but if they came out in a form factor that didn't require me to shell out multiple-4 digit figures in upgrades just to use the next model, it'd easily be worth it for me.
I've already got a home lab, and it's specced to last, minus the GPU. I picked up a separate system for local llm experimentation, but I'm not likely to be upgrading it again. The value add is incredibly small compared to the cost. The real benefits are data privacy and never worrying about rate limits, and there's a price point beyond which an incremental improvement to the model doesn't justify upgrading the system.
chermi 23 hours ago [-]
Wow having it as a PCI card would be awesome. I don't know the feasiblity but it would be really cool. Ordering different models, back to physical media basically...
snarfy 1 days ago [-]
I'm convinced there is a Transmeta/transpiler solution somewhere between 9000 t/s ASIC and Nvidia GPUs. LLMs fit the model much better than what they were trying to do with general purpose in the 2000s.
paxys 1 days ago [-]
The field is moving too fast for it to make sense. ASICs have a 12-24 month development cycle and you’d have to throw them in the trash 4 months later.
jackb4040 1 days ago [-]
If this approach doesn't include a way to apply updated finetunes on a daily basis (also without going back to VRAM being the bottleneck) then the trash interval would be measured in hours, not months
knicholes 24 hours ago [-]
We're getting new models every three months. They'd have to work quickly to get those chips made and installed.
seydor 1 days ago [-]
They require a lot of computation regardless of whether this will take place on ASICs. China can do it, even if the chips are not the most power-efficient. They have built lots of electricity capacity.
Ultimately however, that just means that models will become dirt-cheap. The money will be made with applications built on top of the models.
jv22222 23 hours ago [-]
Yeah, I have been thinking exactly the same. Makes all the sense in the world and is inevitable as far as I can see.
I think the SotA is moving too fast for the production timelines of an ASIC, wouldn't you think? People are just now coming out with LLAMA ASICS but who would want to use LLAMA? Or I guess you are arguing that the models _now_ will be durably useful enough to commit the time to creating the ASIC?
JohnBooty 1 days ago [-]
guess you are arguing that the models _now_ will
be durably useful enough to commit the time to
creating the ASIC?
Objectively, the current frontier-ish models will be useful for some time. Imagine the zombie apocolypse hits, recedes, and you need to rebuild society. An offline copy of Fable or even Opus would be a nice thing to have.
Subjectively, it's hard to say if people will pay for "a model from 18 months ago, but REALLY FAST AND CHEAP"
The speed difference suggests some use cases that might narrow the performance gap. With a > 50x performance delta you have some headroom to play with.
You can do many many fast iterations of ye olde "Ralph loops." You can also jack up the reasoning/effort level. And you could probably do some combination of both, while still running really fast ie 10x the number of iterations at 2x reasoning/effort.
So I think a hypothetical "50x faster Opus 4.8, but burned into ASICs" could be pretty competitive against the frontier models from 2027, 2028, 2029, and maybe beyond?
xtracto 1 days ago [-]
I dream of some kind of "adjustable" ASIC layers, that have the most "computation demanding" layers as ASICs plus a bunch of configurable R+W circuits (FPGAs?) that can be written to upgrade the models to some extent.
What's fascinating is that we are pushing the state of the art of hardware at this point.
Havoc 21 hours ago [-]
Fun as the asic play would be it has a giant hole - context storage. Raw speed only gets you so far if you can’t store and cache
umeshunni 1 days ago [-]
> the winner will be whoever burns their models to ASICs fastest.
I don't understand how this works when the models are evolving so fast that your burned ASIC is outdated (or at least not top of the line) in a few weeks.
tempoponet 1 days ago [-]
If llama 3 70b were available for $400 today, people would make it work. They would have 4 of them working side by side or 1 of them working with a GPU on another model.
To me, it's like imagining if Sonnet 3 was burned into an ASIC 8 years ago and then never changed. It would still be revolutionary, and today we would have an entire ecosystem of tools and services built around it, likely surpassing some of our current workflows.
The frontier is a different beast, but it would likely mean competing on price.
treis 1 days ago [-]
I think you'd be better off getting 1% of a $40,000 card. Or maybe 10% of a $4,000 card is more realistic.
I think that's the problem at the moment. A much better LLM that doesn't use my battery is 20ms and <10mb of data away.
efficax 1 days ago [-]
for many purposes they're good enough now. If I had an opus 4.8 class model on a box next to me that could produce tokens at rates like 5000/s, i don't know if i'd need a new one for a long time. I think we might be underestimating how powerful very very fast LLMs could be, since they could iterate on tons of small variations on tasks. paired with deterministic guardrails that gate "doneness", you could loop on tasks for a long time having the agent try different strategies until the goal was reached, in ways that are just impractical now (and very expensive)
drob518 1 days ago [-]
Exactly. A fast, cheap model, particularly with the right harness and loops, might take us a lot farther than we might guess.
bensyverson 1 days ago [-]
I think it makes more sense if you think about an ASIC pipeline which is updated on an annual basis, like Apple's M or A-series chips.
Sure, an ASIC model will always be behind (12 months? 18 months?), and a hosted flagship will be significantly better. But as time marches on, would I use a flagship model from last year if it came on a PCI card and cost $1000? Without a shadow of a doubt.
mountainriver 1 days ago [-]
Sol is running on Cerebras right?
LarsDu88 1 days ago [-]
Cerebras is not an ASIC. It is a large wafer scale chip that has a load of small SRAM modules paired with tiny compute models. The SRAM is basically a giant cache that is supposed to eliminate the bottleneck between shuttling and materializing big tensors between DRAM and cache, but the actual amount of SRAM is still nowhere near enough to server a frontier model on a single wafer. You still need dozens of wafers, which makes Cerebras rather cost intensive given that Nvidia can get dozens of GPUs out of a single wafer
searealist 1 days ago [-]
How does an ASIC manage to have 60x the memory bandwidth needed to achieve that speedup?
LarsDu88 1 days ago [-]
Currently the setup is paged view in RAM shuttled to HBDRAM (VRAM) on the GPU, which in turn has to get materialized piece by piece onto cache SRAM on the GPU.
Cerebras tries to get around this by keeping everything on cache SRAM as much as possible, which it burns directly to the chip wafer itself and physically places that SRAM directly next to the tiny compute unit that does the actual math.
An ideal setup (not sure how easy this is to achieve in practice), is the burn the weights of the model directly to the chip as a sort of ROM, the actual math operations as actual digital circuits, and have SRAM, or even something akin to naked registers to directly compute off inference batch data. Cuts out 2-3 layers of abstraction and indirection.
oblio 1 days ago [-]
I assume ROM would be even more expensive than SRAM?
marcyb5st 1 days ago [-]
You don't need memory because the output of the previous layer flows directly into the input gates of the next. So no I/O to/from HBM or similar
You still need some memory for the context, in flight answers, ... but not for the model weights and for the output of the intermediate layers. I found taalas demo here: https://chatjimmy.ai/
signa11 22 hours ago [-]
why not start with fogas ? you know … to test the waters ?
dboreham 1 days ago [-]
GPUs are already ASICs.
1 days ago [-]
aaroninsf 1 days ago [-]
Apple's M7 comes to mind...
faitswulff 1 days ago [-]
Another reason this may be true, selling hardware may be the only durable moat after a while
misiti3780 1 days ago [-]
is anyone doing this ?
LarsDu88 1 days ago [-]
The closest example I've seen is ChatJimmy: https://chatjimmy.ai/ a prototype from Taalas running Llama 8B
Scaling this up to 2.8 Trillion (350X increase), will certainly be challenging.
If I was younger and had the right background, I'd love to dive into attempting somethign like this
selectodude 1 days ago [-]
Also a 3-bit quant. Useable but a long long way away from useful.
Last I saw they posted Deepseek R1 numbers in Feb of this year.
The challenge is rolling out a new one every 7-8 weeks as the weights change & cheap enough for a hyper scaler to afford to buy one and save enough on power over the next 8 weeks as a payoff.
carterschonwald 1 days ago [-]
not sure about that, but im actively working on designing ultra sparse models that i want to have perform competitively with stuff 100-10_000 times larger. ehich does yield similar throughput. time will tell id it works out
535188B17C93743 1 days ago [-]
Yeah, I've heard of Taalas doing it. Not sure of others but I'm sure lots of companies are considering it, especially as we start to hit points of depreciating returns in training.
eckr 1 days ago [-]
There was a startup that did this for Llama 3, I forgot their name. Etched is also doing some similar things I believe.
It sounds vaguely similar to what Cerebras.ai is doing
applicative 1 days ago [-]
Open weights are a red herring if you can’t run the model independently. Good luck fitting these models on your Mac mini.
In fact the superior models are irreducibly nothing but superior web services run from China.
Slartie 1 days ago [-]
Did you not notice that there are inference providers running all the various Chinese open weight models in the US and EU?
Nobody needs "web services run from China" to use Chinese open weight models.
quotemstr 1 days ago [-]
What precisely is an ASIC supposed to do that a more programmable accelerator can't? Memory latency is memory latency: doesn't matter whether it's embedded in a cache-line wait state or some flipflop state machine. ROM isn't going to be faster than RAM either.
Likewise, for compute, is the ASIC somehow going to beat a systolic array? You can't have one circuit per weight: the die area and electrical fan-out would be insane. I'm not seeing how an ASIC specialized for a specific model would actually help much.
I mean, sure, we can build more specific accelerators, e.g. for softmax, but these work fine in the context of a programmable pipeline.
Yes, there are more exotic things out there, like optical matrix multiplication systems. Those are different. But above, aren't you talking about just doing conventional digital linear algebra, but with a model-specific set of circuits?
marcyb5st 1 days ago [-]
No, if you burn your model into the silicon you don't need memory as the output of a layer flows through circuitry directly into the next one. No I/O to memory of any kind. You still need a bit of memory for in flight answers, but that's it.
quotemstr 1 days ago [-]
You can pipeline in software too, and even with a dedicated circuit, you need to put weights somewhere, because you're not going to have a special partially-applied FP8 FMA unit for each of your trillion model weights.
I can accept the idea of specializing a circuit for a specific model shape, but I'm not seeing a need to specialize a circuit for the weights inside the shape.
marcyb5st 1 days ago [-]
Apparently that is what Taalas did though. Not an hardware person, so take this with a pinch of salt
BurnerOptical 1 days ago [-]
[dead]
pstuart 1 days ago [-]
[dead]
overgard 1 days ago [-]
I keep thinking about the Figma thing. If you're unaware, here's the google summary:
----
The Board Departure: Mike Krieger, Anthropic’s CPO and a co-founder of Instagram, sat on Figma’s board of directors. He resigned on April 14, just days before news of Claude Design broke. This sparked speculation over conflict of interest and the use of proprietary product strategy information.
Betrayal of Partnership: The launch aggravated the tech industry because Figma relied on Anthropic's models to power its own AI features, and even announced a joint "Code to Canvas" integration. Reports indicate Figma was blindsided by the depth and scope of Claude Design.
Market Reaction: The "SaaSpocalypse" thesis—fears that major AI foundation models will rapidly build application layers and cannibalize their own SaaS partners—was realized when the news broke. Figma’s stock saw an immediate 7% drop upon the announcement.
----
I would suggest to people using LLMs: you should be cautious about giving these companies data or relying on them. If you're building an AI startup, there's a very good chance they could decide to directly compete with you if your idea has traction. You're also at their mercy for API pricing etc.
mattdeboard 1 days ago [-]
Your last two sentences remind me of the olden days of people building applications on top of FB and Twitter APIs (and later, Reddit), only to to have the rug pulled out from under them in one way or another
mbreese 20 hours ago [-]
Olden days being FB and Twitter APIs? Don't forget about Apple Sherlocking independent devs. This has been happening with almost any platform for a very long time.
zdragnar 1 days ago [-]
Back in the day, a contract company I worked for had entire teams of people dedicated to building FB apps. When that collapsed, it came down fast and hard.
conception 23 hours ago [-]
Still true today for AWS.
DesiLurker 15 hours ago [-]
even before that, by some accounts I hear the reason VCs over invested in dot com was primarily because they saw something that could not easily usurped by Microsoft by releasing their own sh*tty copy. so they all piled in and then bubble dynamics took over.
story as old as internet itself!
signa11 22 hours ago [-]
don’t forget m$ as well.
florakel 1 days ago [-]
I believe the coding tool revenue is an important factor right now but not the endgame. The AI companies have to become consumer products to justify their gigantic valuations. We always talked about the “super apps” - one app that fulfills most consumer needs, like
WeChat in China.
OpenAI is the company most visibly making a huge bet on becoming a “consumer super device”, even designing their own hardware to stop being dependent on Apple and Android.
The long term battle is not AI companies vs SaaS, but rather AI companies vs big consumer brands - Apple, Google, Meta, Amazon.
Imagine having the ultimate digital assistant that you can throw any task at. Under the hood it creates its own code and interfaces on the fly to help you accomplish your task. By using LLMs as coding assistants we are all training them to become really good at it. If the newcomers succeed to deliver the ultimate digital assistant first, then even a monster like Apple could become obsolete.
cadamsdotcom 23 hours ago [-]
How long did Apple take to kill off all the iPhone competitors.
These companies haven't got the attention span to work on one thing for that long.
The driving constraint if you want to take a spot in B2C is the speed at which consumers replace tech, not the speed at which tech can be developed.
sdfefcxv 1 days ago [-]
Delusion
drakythe 1 days ago [-]
Our current incarnation of capitalism is all about monopolistic behaviors. If these LLM companies do get to the point of being able to replace employees I fully expect them to stop selling shovels and start producing the gold directly, anyone else be damned. And, frankly, this has always been the case. If a product is built on top of another service it has a limited lifespan. Either the product will be purchased or it will be replaced by the service it depends on. How many "killer apps" has Apple silently absorbed into iOS over the years?
satvikpendem 1 days ago [-]
And who will buy the gold? At a certain point everyone will be too poor to buy anything.
bigbadfeline 1 days ago [-]
> And who will buy the gold?
Wrong question. The real question is "Who and what will they buy with the gold?"
The golden rule is just the beginning, and no, there's nothing positive for the rest of us.
drakythe 1 days ago [-]
No one. I do not believe this to be a good idea. I think its pretty awful and this whole thing, if it goes as far as they keep saying it is going to go, will spin wildly out of control and do an immense amount of damage to people.
But the big money has been skipping Leg Day for ages, and that lack of a foundation will bite us all. Which sucks.
Incipient 1 days ago [-]
I mean Claude IS replacing people. Even in Australia - Claude + postgres mcp is replacing data/bi people in even small and medium businesses.
xyzsparetimexyz 1 days ago [-]
What did the bisexuals do to deserve that?
fomoz 1 days ago [-]
That's very true. I realized it myself when I worked on my first app, I made a resume tailor last summer back when GPT 5 just came out.
Then, by the time GPT 5.5 came out you could already generate a flawless PDF and Word doc resume with the same formatting as your base resume. It really is amazing.
Trying to compete with these labs is a bad idea, you might work hard on your thin wrapper and then they come out and have their chatbot or harness do it way better than you.
But I also think this is really cool. It's a new era in software. Build something worthwhile and you won't need to worry about a lab doing the same thing with their chatbot.
spamizbad 1 days ago [-]
Disagree: The era of "<Tech Giant> will eat your startup by throwing capital/devs at it" are over. Anthropic, OpenAI et al are under too much competitive pressure to chase every quixotic product idea.
intrasight 1 days ago [-]
Then run your own fine tuned models for your AI startup.
muldvarp 1 days ago [-]
Doesn't that mean you bet against the bitter lesson?
1 days ago [-]
skeptic_ai 1 days ago [-]
[dead]
sieabahlpark 1 days ago [-]
[dead]
monster_truck 1 days ago [-]
You are vastly underestimating how much more profitable a 10% annual return on GPUs is than basically anything else you would use an LLM for.
They think whatever you are doing is cute and would very much like to ensure their models can do it even better in the future, but competing? Not even worth the time to think about
socialcommenter 22 hours ago [-]
Is it only a 10% annual return? That's not a lot.
halfmatthalfcat 1 days ago [-]
This has been a story, age as old as time. How many times have startup founders been told "you're a feature" or "Google/Meta/etc can copy you so fast". Operators just need to stay vigilant of their value add, or they never really had a moat in the first place and were living on borrowed time anyway.
onlyrealcuzzo 1 days ago [-]
> I would suggest to people using LLMs: you should be cautious about giving these companies data or relying on them. If you're building an AI startup, there's a very good chance they could decide to directly compete with you if your idea has traction. You're also at their mercy for API pricing etc.
LLM generated code is not copyrightable, so even if they do "steal" it - I don't think there's legal grounds to do anything about it.
It can't be stolen. You don't really own it.
You can try to lock it in a safe and hope no one ever gets a hold of it. You can lie and say you didn't use an LLM, but Anthropic and OpenAI et al probably have logs to disprove that.
But, if push comes to shove, you don't own it...
sowbug 1 days ago [-]
This is a popular misreading of the state of the law.
Someone tried, as a bit of a stunt, to register a work for copyright with generative AI as the sole creator/author. That registration was rejected. This is quite different from a person using generative AI as a tool to create a work.
People have copyright in photos and videos they create, even if they used a camera. Same with images and code, even if they used an LLM.
I'm not your lawyer, but to the extent you have copyright in works you create, the fact that you used a tool doesn't diminish the copyright.
> Before the proliferation of generative AI, U.S. courts did not extend copyright protection to various nonhuman authors, holding that a monkey who took photos of himself lacked standing to sue under the Copyright Act; that human authorship was required to copyright a book purportedly inspired by celestial beings; and that a living garden could not be copyrighted. The U.S. Copyright Office has also long maintained that copyrighted works must be "created by a human being" and therefore refused to register works that are "produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author."
I think the long story short is if you're incorporting LLM generated content into your own work, then your own work is copyrighted but if it's purely AI created and you never put any of your own work into it, it's not (so: think the slop cannons people are pointing at social media for example -- probably not copyrighted (although I don't know why you'd want to put your name on it anyway)). Granted, I don't know how you'd ever prove that something was purely LLM generated so it's kind of a technicality for coding, but for image and music I think it applies pretty clearly.
As an aside, I think socially maintaining that copyrights are for humans only is extremely important. It might be the most important legal matter around this. I see nothing good of allowing AI labs that have strip-mined the internet to claim a copyright on the things they regurgitate, and I don't think spammers deserve copyright protection either. The human-only requirement is a very good one.
drob518 1 days ago [-]
Right, but the AI isn’t the one who would actually claim copyright here. It would be the human using AI to accelerate the coding. And the human does have standing.
onlyrealcuzzo 1 days ago [-]
So if I write one line of code in a 1M line LLM codebase, is it mine?
drob518 1 days ago [-]
It’s owned by whoever directed the AI to write it.
overgard 1 days ago [-]
So, code is not the only data they potentially have access to, and it might not even be the most interesting data. There's also whatever other things you plug the AI into: slack, email, internal documentation, etc.
Also, a lot of people using LLMs are using it on code bases that were written before LLMs were common for coding. So in that context, you really are allowing them to train on your code for free.
easytiger 1 days ago [-]
> LLM generated code is not copyrightable, so even if they do "steal" it - I don't think there's legal grounds to do anything about it.
> It can't be stolen. You don't really own it.
At least in the UK computer generated code has been copyright-able for decades [1]
> (3)In the case of a literary, dramatic, musical or artistic work which is computer-generated, the author shall be taken to be the person by whom the arrangements necessary for the creation of the work are undertaken.
And the ToS of the major/minor AI providers as far as i can tell have no opinion on what you do with the code in terms of ownership.
I've seen cases where it was determined that the LLM itself doesn't hold copyright. But where are you getting the idea that nobody holds copyright?
bko 1 days ago [-]
I think the risk is overstated.
For one, on the margin people are willing to pay a lot for slightly better models. I know personally the value the LLM adds to my workflow is considerably more than the $200/m I pay the frontier labs. I have no interest in optimizing that to get it slightly lower. There are a very vocal minority that optimizes this or companies whose LLM expense is marginal, but I think that's the minority (correct me if I'm wrong, curious what their customer base looks like)
Also the actual LLM is a tiny portion of the value added. Anyone that tried to build agentic solutions from LLM apis quickly realizes that a huge value is the Claude Code / Codex harness. There are open source implementations like OpenCode but they're not nearly as good.
Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them. Google sheets is probably 99% and available to everyone and better in a lot of regards. But the immense value spreadsheet software produces workers above the $100 or whatever a year makes it so that there is a real moat and no one bothers exploring alternatives.
jhonof 1 days ago [-]
> For one, on the margin people are willing to pay a lot for slightly better models. I know personally the value the LLM adds to my workflow is considerably more than the $200/m I pay the frontier labs. I have no interest in optimizing that to get it slightly lower. There are a very vocal minority that optimizes this or companies whose LLM expense is marginal, but I think that's the minority (correct me if I'm wrong, curious what their customer base looks like)
Enterprise is how Claude/OAI make their money. Enterprise companies absolutely optimize around costs for infra, it's like an entire job category.
> Also the actual LLM is a tiny portion of the value added. Anyone that tried to build agentic solutions from LLM apis quickly realizes that a huge value is the Claude Code / Codex harness. There are open source implementations like OpenCode but they're not nearly as good.
Claude and OAI are not valued at $1T because of their harnesses
> Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them. Google sheets is probably 99% and available to everyone and better in a lot of regards. But the immense value spreadsheet software produces workers above the $100 or whatever a year makes it so that there is a real moat and no one bothers exploring alternatives.
Enterprise excel is like $50 a month, the closed source labs charge orders of magnitude more than that per user per month for enterprise, and want to charge more if they can to maintain their valuation. I spent $20 in an afternoon yesterday using my work plan for one workflow for reference, I probably spend at least $50 a day and I don't even do that much coding anymore.
Incipient 1 days ago [-]
$200/mo for 20x Max is great value - but it's not sustainable and won't last forever. Microsoft gave up. Anthropic will too. I use roughly $10k/mo - at that price it's absolutely not worth it.
I'm genuinely not sure where the balancing point even is.
arizen 1 days ago [-]
Probably increasing learning rate have some value for these players which is enabled by subsidized usage.
And cost of acquiring learning via users feedback versus additional learning data value for a given company will dictate whether such company will give up or will continue subsidized usage, despite being unprofitable on the paper, but probably really valuable as a long term strategy for GTM and product development and increased learning rates.
bko 1 days ago [-]
> Claude and OAI are not valued at $1T because of their harnesses
They're valued at that because they add a lot of value and people pay for the product. The product is more than the LLM. If you want argue the value of the harness vs LLM but flippant remark adds nothing.
> Enterprise excel is like $50 a month, the closed source labs charge orders of magnitude more than that per user per month for enterprise, and want to charge more if they can to maintain their valuation. I spent $20 in an afternoon yesterday using my work plan for one workflow for reference, I probably spend at least $50 a day and I don't even do that much coding anymore.
I want to know what percent of their revenue is this API spend compared to just a Claude/Codex max plan that's $200/m and very generous.
svachalek 1 days ago [-]
I'd imagine it has to be immense. Most of the engineers I know are spending $200/day, let's say 20 workdays per month, $4000/mo. Whatever % of revenue it is, it's got to be close to 100% of profit.
hungryhobbit 1 days ago [-]
Who is paying $200/day? My company has me on a $30/month plan (I assume they pay annually?). I use Claude constantly and only hit usage limits when I try to do 4+ projects at once.
I can't imagine why anyone would need almost 7x that.
arrowleaf 1 days ago [-]
If your company is a certain size or cares about ZDR, you need to be on an Enterprise plan which is API pricing. $300/day here on average.
pimeys 1 days ago [-]
Depending on the harness, $200/session can be common. Thanks for harnesses like maki the cost is lower compared to OpenCode or Cursor.
But $200/day is very common in the business. The monthly plans are just a trial version of what's going to come for all of us. That's why I am very carefully looking into the open models today.
arrowleaf 1 days ago [-]
Here, people tend to forget about enterprise customers. Enterprise is excluded from using these heavily subsidized subscriptions, I know of orgs that are spending around $500k/month for teams of ~100 developers actively using AI.
xur17 1 days ago [-]
This is a spot where smaller orgs are getting a definite advantage (both cost and speed). If you're a company with 20 - 50 people, you can probably get away with Claude Teams instead of Claude Enterprise, and pay 10x less in ai costs.
esperent 1 days ago [-]
> considerably more than the $200/m I pay the frontier labs.
This is a very rich / developed country privilege perspective.
Where I live, it's not unusual for a monthly wage to be around $200. Of course developer wages are much higher, maybe as much as $1000 a month, but $200 is still a huge chunk of that so it doesn't really matter how much "value" you get out of it if you're no longer able to pay rent or buy decent food.
Even in developed countries, $200 a month is out of reach for all kinds of people who would benefit from it (students without rich families, entrepreneurs, etc.).
bko 1 days ago [-]
Thats great, but the market will always be driven by rich developed countries. I don't think Anthropic and OpenAI is banking on third world adoption in their growth estimates.
kgwgk 1 days ago [-]
> a huge value is the […] Codex harness. There are open source implementations
If you think Codex is some neutral harness designed to work equally as well for all LLMs, I got a bridge to sell you...
InsideOutSanta 1 days ago [-]
> on the margin people are willing to pay a lot for slightly better models
That margin is getting smaller and smaller. I would have been with you a week ago; paying for Fable was worth it compared to all other models. But with K3, the difference has shrunk to the point where, for me, it's not worth the cost anymore.
In other words, it may be worth paying five times as much to get 10% better real-world outcomes for a lot of people, but a lot fewer people will pay five times as much to get 2% better outcomes.
> a huge value is the Claude Code / Codex harness
For me, it's the opposite. Having to use Claude Code instead of the harness I prefer is a point against Anthropic, not for it.
hugopuybareau 1 days ago [-]
100% agree for the harnesses, I always end up using CC when I want to use Opus but a few months ago I had my opencode harness set up but Copilot Pro+ rose too much the prices for me to keep using it…
What harness do you prefer using ?
InsideOutSanta 1 days ago [-]
OpenCode because of the large ecosystem. I'd like to use pi, but it's more limited if I want to run it in a VM and connect to it from the outside.
mwryl 1 days ago [-]
I don't think subs are what keep their lights on. And their API is so absurdly expensive.
For the harness hard disagree but ultimately its up to each ones taste. You may want to check this tho https://harnessrank.net
On our side we use Claude/GPT/Kimi (it replaced Antigravity) for development. But we build our systems around a cheaper denominator (Deepseek previously, recently we added GPT 5.6 which have good prices as well). We offer BYOK for Claude but its def not an option to build something on top of it (for us).
bwfan123 1 days ago [-]
> There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them
Excel has network effects. If N people are using excel, the incremental N+1'th user is forced to use it or risk not being able to open documents from the N users. Excel is also sticky, in the sense that a user used to Excel UI is hesitant to switch to a new UI. Tokens dont have network effects. If N people use fable, the N+1'th user could use any other model, and have no impact.
crakhamster01 6 hours ago [-]
> I know personally the value the LLM adds to my workflow is considerably more than the $200/m I pay the frontier labs.
AFAIK the frontier labs make their money on enterprise, and they recently changed their terms so that most businesses can't use the $200/m plan anymore.
This is why companies like Uber - and mine :( - announced per employee AI budgets on the order of ~$1000/month.
When you're forced to pay API costs, the value prop of Kimi/Qwen becomes a lot more compelling.
mingqiz 1 days ago [-]
Not true. Enterprise pay for api. If i am allowed to use fable withouta budget, i can spend 3k per day. Using combination of cheaper models (and better at least in my experience) can lower this to 300. Considering the layoffs, thw companiesdo care a lot of this cost. Price matters more than quality for many use cases. For example, i can writea game where npc are all k3/5.6 level intelligence if the price is cheap enough.
reticulates 1 days ago [-]
The $200 you’re paying is heavily subsidized, you need to consider the real inference cost which could be closer to $10k if you’re using the maximum permitted by your plan.
hungryhobbit 1 days ago [-]
The "real" cost is what people pay, not what the AI companies claim.
You assume everyone has extra 200$ to pay. For someone living in a low wage country, it's hard to pay that much, even if you produce a multiple of that value in theory
hedora 14 hours ago [-]
I’m on the $200 claude plan, and keep getting booted down to opus or even sonnet by the broken guardrails.
Why should I pay for a frontier model that I can’t use?
JSR_FDED 1 days ago [-]
Spending a lot on making your top developers more effective is easy to justify.
But once you have an AI-powered application in production, why wouldn't you go with far cheaper and capable enough models? That's just optimising a business process like any other. You'd use cheaper providers whether Chinese, or other on-prem models.
thisisit 1 days ago [-]
This is a bad take.
> Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them.
Sure but were these open source alternative as pervasive and widespread and talked about when Microsoft was bundling Excel with the OS? Or were there millions of dollars, if not billions of dollars being spent by these competitors? The answer is No. It is not the same scenario.
While I understand the logic of harnesses - those are not full proof. It is trivial to setup K3 or Qwen to work with Claude Code/Codex by intercepting requests and routing to K3/Qwen. You have tools like CCSwitch which can do that for you.
The only thing remains is - people willing to pay a lot for slightly better model. That is true but price sensitivity is also a thing. The mania case for Anthropic/OpenAI/SpaceX/Google is that they will capture large part of the enterprise market. But in many cases outside coding AI capabilities will be resold.
That is instead of buying AI for lets say HR functions you pay for embedded AI in your HR software. And the HR software company will have incentives to raise their own margins. If their provide similar looking experience using Qwen/K3 why will they buy Anthropic/OpenAI. That is the real risk here. While as an individual you can continue to pay top dollar.
uncivilized 1 days ago [-]
What is your workflow such that frontier models increases productivity considerably?
cmrdporcupine 1 days ago [-]
For people cranking out SaaS / web services / web pages / "full stack" work I don't see a huge difference.
If you're building an optimizing compiler, a CUDA kernel, a database, a high performance concurrent data structure with tricky locking, etc. etc. it's still not really close. Sol 5.6 on high just slays e.g. GLM 5.2 for this kind of work for me.
I'm sure K3 is fine for these things too, but I can't afford it at its API rates compared to a Codex coding plan.
For now.
uncivilized 1 days ago [-]
Regarding the latter, are you able to obtain good results from an LLM? With graphics programming work, I find LLMs only help in cases where the task would take me 5 mins or so.
cmrdporcupine 1 days ago [-]
Absolutely.
uncivilized 1 days ago [-]
Interesting. Do you happen to have a public example?
cmrdporcupine 1 days ago [-]
Of what specifically?
uncivilized 24 hours ago [-]
A CUDA kernel, compiler optimization, or anything of similar complexity
cmrdporcupine 23 hours ago [-]
Feel free to look, but don't bother replying if you have something negative to say, don't feel like having my day ruined by negativity.
Thanks for sharing. Looks interesting. Was all the code AI generated, or is it a mix?
cmrdporcupine 20 hours ago [-]
that stuff in particular -- AI generated with heavy heavy prompting and up front design work and post-implementation testing
I have CUDA work here somewhere too but I have the repository private right now
uncivilized 8 hours ago [-]
Very cool. Thanks for sharing.
cmrdporcupine 7 hours ago [-]
Here's what is key: never rely fully on the "intelligence" of the model. Build a workflow and/or tooling that allows for empirical improvement. e.g. for performance tuning I have a benchmarking framework and a container/server that contains the differential results available via MCP for the agent to observe as it works. Using /goal and a clear destination you want to get to, it will literally grind for hours and use that help. Doesn't help with architectural stuff, but it does help with producing efficient code.
port3000 1 days ago [-]
It's amazing how quickly Fable went from 'Game-changing model that needs to be banned' to 'Yeah it's alright, but OpenAI is also just as good and there are a couple of good open weight alternatives that are equivalent for almost everything'
The hype cycles are shortening, perhaps we really are reaching some kind of plateau this time (famous last words)
rockinghigh 1 days ago [-]
Open-weight models were lagging 4 months behind OpenAI/Anthropic at the beginning of the year. They are now just 4-6 weeks behind.
anthonypasq 1 days ago [-]
Kimi K3 is still worse than Fable and Fable was trained >4 months ago.
janalsncm 24 hours ago [-]
Why are you using the product release date for Kimi K3 and the training date for Fable? Either use the release date for both (6 weeks apart) or if you have it the training date for both.
anthonypasq 6 hours ago [-]
because its a matter of public record that the release of Fable was artificially delayed
janalsncm 2 hours ago [-]
By Anthropic. The government blocked it after it was already released.
Every LLM product goes through testing and alignment after training, maybe even some quick improvements here and there. Kimi probably did something similar.
Put another way, if Google says they have the best model in the world but won’t release it in December I will start caring in December, not before.
anthonypasq 1 hours ago [-]
if Dario and the CEO of Moonshot switched places, Mythos would have been generally available to the public 4 months ago. As a statement of fact, it wasnt, because of Project Glasswing and Dario thinking they created a superweapon that the rubes shouldnt have access to.
sdfefcxv 1 days ago [-]
To say X is perfectly bad vs Y is false.
People use these models for diff things.
Its quite possible for the things they are used for, people do not see much of a difference.
Do you hold stock in Anthropic?
TheMrZZ 1 days ago [-]
> Profile created 3 days ago
> Unnecessarily aggressive
> First ever comment said "Further releases of Chinese models that demonstrate the gap is not growing substantially is a huge problem. The spending will be called into question."
Yeah I think you have an agenda
tracker1 1 days ago [-]
Are you a Chinese national, or otherwise paid by China?
daadx 22 hours ago [-]
the lot of you sound salty lmao
are you american?
anthonypasq 1 days ago [-]
ah yes, because i said something factually accurate and vaguely positive about Anthropic I must be a shareholder which would mean I either run a venture capital firm or am a current employee of Anthropic...
I wish.
yogthos 1 days ago [-]
And given that Chinese models are closing the gap there are basically two thing that could be happening. One is that they are moving faster than US companies developing closed models, and two that we're starting to hit a plateau for model capabilities where all the easy gains have been plucked, and now it's not really possible to move forward at the same rate on the frontier. Of course, both things could be happening at the same time.
janalsncm 23 hours ago [-]
Probably a little of both.
Chinese labs have come up with a bunch of genuine innovations: GRPO, auxiliary loss free MoE load balancing, MLA, muon optimizer, and a bunch of other ones. The Deepseek papers are really well written, this isn’t just sneaking a peek at a peer.
The problems are inherently harder now too, partially because they take longer, so your training pipeline is waiting for long completions.
Also there probably is some “distillation” (technically pseudo-labeling, which is common in ML). But I wouldn’t put too much weight on it because that was true 18 months ago as well.
yogthos 20 hours ago [-]
That's my thinking as well. The whole distillation thing is a distraction from the actual innovation happening in this space. What will be interesting to see going forward is what types of new techniques people manage to come up with to over come the current architecture limits.
boc 1 days ago [-]
Or option three is they are drafting hard off the frontier US models via distillation.
yogthos 1 days ago [-]
The process takes time because even when you're distilling answers, you still need to actually do reinforcement training on the model. And given that Fable and GPT 5.6 just came out there simply hasn't been much time to do that. On top of that, Kimi also does better than Fable or GPT on a lot of tasks, distillation alone can't explain that, meaning there is a difference in architecture. You can watch a talk from Kimi founder to see how Kimi was actually trained and why it performs well. https://www.youtube.com/watch?v=5CkCW1P-g88
Not to mention that US companies models constantly distill each other as Musk was forced to admit under oath. This whole narrative has just been a massive cope.
boc 22 hours ago [-]
> US companies models constantly distill each other as Musk was forced to admit under oath
> This whole narrative has just been a massive cope.
So wait, US AI companies all use distillation because... it's not effective and it's all just cope? Or is distillation really powerful and they all do it, which Musk was forced to admit under oath? But when China does distillation it isn't powerful and they don't need to do it, but they do it anyway because it's fun?
Either it's powerful and everyone, including the Chinese labs, use it as a way to rapidly catch-up against the SOTA models, or it's a red herring and the huge amounts of energy spent to protect and enable distillation is all just wasted money. Which is it?
yogthos 22 hours ago [-]
I'm saying it's a cope to claim that the only reason Chinese models are catching up is due to distillation, while pointing out that distillation itself is in no way unique to Chinese companies. I'm sorry this was too complex of an idea for you to follow.
boc 20 hours ago [-]
The fun part about this is that we can see who is right in about a year. If the leading labs continue making progress at hardening their models against distillation, and then they start pulling away again, we see who is right. If China is able to pass the US and release an independently better model than anything the US has, then your theory is correct.
Both sides have extremely smart people. One side has more $$$ and exclusive access to the best chips. For progress to converge without a corresponding breakthrough suggests there's something else at work.
yogthos 20 hours ago [-]
Indeed we will, my prediction is that we'll see a model from China that definitively surpasses any US model by the end of the year. China has an absolute population advantage here along with having a much better education system. And China now dominates in published AI papers.
The US enjoyed an early advantage due to excessive money being poured into AI which led to the current bubble, and access to the hardware that was needed to train these models initially.
At this point, neither of these factors actually matter that much. The naive approach of simply making models bigger has hit a wall, and now you need ingenuity in figuring out better architecture for them. Precisely because Chinese companies have had to deal with more limited resources, they put a lot more effort into researching different kinds of optimizing techniques. And of course, China is also catching up in chip making, and Huawei clusters are already competitive with Nvidia for training. So, that gap is closing as well.
The big difference is that an absolutely insane amount of money has been spent in the US, while China managed to do this on a fraction of the budget. The AI Investment Surge graph here puts things in perspective. https://hai.stanford.edu/news/inside-the-ai-index-12-takeawa...
sdfefcxv 1 days ago [-]
This happened ages ago.
But OAI and Anthropic are trying to cash in ahead of their IPO window. I think that window is pretty much gone now.
yogthos 1 days ago [-]
My prediction is that they're going to angle to become a vendor of record for the government and get bailed out. That's the only path at this point because there won't be any competition from China in this niche.
stingraycharles 21 hours ago [-]
Fable is still the same model, it’s still a great model, and to be honest all these articles writing and speculating on how the LLM industry is going to evolve are not that insightful nor interesting.
I don’t think one should pay much attention to them.
jesse_dot_id 1 days ago [-]
The plateau is inevitable because their rapacious training methodologies are only viable when there are no defense in place, but information continues to evolve, which means the models will have to be continuously updated, but will be doing so with less and less freely available data.
achandra03 22 hours ago [-]
> with less and less freely available data
My understanding is that the labs ran out of freely available data to train on a while ago, and now primarily rely on human data vendors such as Surge and Mercor to source their data.
t0mpr1c3 22 hours ago [-]
We are only just starting to model the physical world. There will be more training on empirical data.
potsandpans 18 hours ago [-]
Almost like the company has no credibility with respect to its safety claims.
drob518 1 days ago [-]
> More importantly, as sustainable long-term businesses, model-only providers are particularly at risk. Knowledge Atlas, Moonshot Labs, and Anthropic face defensibility challenges versus OpenAI, Alibaba, SpaceX, Meta, and Google.
Hm. How is OpenAI not a “model-only” provider just like Anthropic? Seems like they are vulnerable in the same way.
treis 1 days ago [-]
ChatGPT is synonymous with non technical/work related LLMs. They're amassing a ton of user history. That history improves the product for the user because it has more context into the person. They can feed it back into model improvements and for advertising.
You can see a future where a user types in "plan a vacation for me" and ChatGPT coordinates everything from there. Those sorts of users aren't going to switch because model X is 10% cheaper or better.
drob518 1 days ago [-]
I see a future where lots of models can plan a vacation for me, not just ChatGPT. Saying that there is brand equity in the ChatGPT brand feels a lot like saying there’s a brand equity in the MySpace and AOL 25 years ago. Google in particular, via Android and its relationship with Apple to power Siri, has a much better shot at grabbing the “plan a vacation for me” consumer market, IMO.
I could easily see OpenAI become irrelevant in 2 years if they stumble at all and don’t keep up with the other frontier models.
delecti 1 days ago [-]
There's a lot of value in having your product's name be synonymous with the product category.
Google's quality has gone to shit, but internet searching is still "googling", both in verbiage and in the actual service people use. It's not an impenetrable moat, but OpenAI would have to stumble pretty hard to lose all of that edge.
drob518 20 hours ago [-]
I agree that there’s huge value in being the generic category name (see Kleenex). That said, Google and Apple are going to roll this into every phone and tablet and laptop/Chromebook. Yea, OpenAI can release a phone app (already have), but I’ll bet you dollars to donuts that the Google and Apple integrations will be superior, and even if the EU or somebody forces them to create an “AI provider neutral API” like they have for web search in browsers, most people will just roll with the default like they do for Google Search. You might be allowed to choose OpenAI, but most normies won’t.
treis 1 days ago [-]
They won't have the years of chat history to customize the vacation.
drob518 1 days ago [-]
[flagged]
treis 1 days ago [-]
Rude
sdfefcxv 1 days ago [-]
[flagged]
treis 1 days ago [-]
Rude
zparky 1 days ago [-]
spam account created a few days ago.
georgemcbay 1 days ago [-]
The article does address what they see as the difference ("its investments in product, consumer experience, site publishing, voice, and hardware are all directions that have clearer moats")
That said I think it is pretty easy to make a case that these would-be differentiators are either currently underwhelming or completely unproven (as in the case of hardware).
drob518 1 days ago [-]
Right. Arguing with the original article, not you, my counterpoint would be, “Sure, if they’re able to do that. But so could any of the other model-only competitors.” The best you can say today is that they have announced an intention to do those things, but in no way have they established themselves as being successful, yet. And the model-only Chinese providers have access to lots of e.g. wearable and consumer tech.
cl42 1 days ago [-]
They are trying to diversify into consumer hardware and also are further along in owning data centers. The consumer hardware product, assuming it launches and does decently well, puts them in a very unique position relative to pretty much every other frontier lab. It's more speculative, but I'd argue it can change things quite a bit for them if it works out.
NitpickLawyer 1 days ago [-]
They've been doing hardware stuff as well. Both "consumer facing" bs like wearables / portables, but also more importantly chips for inference.
Having a good model is one thing, being able to serve that model at good speeds and match demand is another. See Anthropic ~6months ago. Or Moonshot, they've already suspended subscriptions to their coding plans, because they can't meet demand.
eikenberry 1 days ago [-]
Didn't the Apple lawsuit throw a monkey wrench into their hardware plans?
davidpapermill 1 days ago [-]
I think a big question is whether any of these labs can produce a model that is _ahead_ of Anthropic and OpenAI.
A related question is how much they're dependent on the APIs of Anthropic and OpenAI to achieve their results - whether through distillation or other uses.
If these models are derivative of Anthropic/OpenAI I would expect performance to be more narrow and progress to be limited.
sdfefcxv 1 days ago [-]
They dont need to be ahead on performance alone.
Its value per unit of currency spent.
Financials will ultimately drive decision making.
We are already seeing that more intelligence does not correlate with more revenue, for the firm purchasing tokens.
If I was OAI/Anthropic I'd be brown and yellow in the boxers.
torginus 1 days ago [-]
Upper bound of AI progress - recursive self improvement. In this case AI will be responsible for building better models, making people who own datacenters the winners. Anthropic/OAI is cooked.
Lower bound of AI progress - plateau. Progess is slowing, focus is on serving a meaningful peak capability at the lowest possible price. There's been news today that Google is building a Gemini chip with weights baked into silicon. Considering a chip's lifetime of 2-3 years at minimum, and that a 2-3 year model today would be useless today, they're expecting they wont make a similar amount of progress in the next 3. Game is about selling at the lowest margin. Anthropic/OAI is cooked.
So their survival rests on the presumption that AI progress will fall between these two extremes.
xtracto 1 days ago [-]
>. In this case AI will be responsible for building better models, making people who own datacenters the winners.
We need SETI@home for Open Weight models yesterday...
noman-land 15 hours ago [-]
FWIW, I agree.
in_a_society 1 days ago [-]
Baking weights in makes a lot of sense for inference speed and power efficiency and has the added benefit of putting many end-users on the hardware refresh treadmill.
torginus 1 days ago [-]
Yes, but how much would be a static Sonnet 3.5 be worth today? Its just about 2 years old. I'm not even going to ask about 3yo models like GPT4.
himata4113 1 days ago [-]
Okay, but the models today will be useful for a lot longer than sonnet 3.5. They're already more than capable to do nearly anything you throw at them given enough time and human assistance. The next step up is faster, cheaper and better user experience. I have only had two instances where I needed to reach for 5.6 sol and that only totalled around $2.7 in api costs.
I would imagine it would look something like this:
Ground breaking/novel research -> SWE -> day to day assistant conversations -> chat support bot...
amaranth 1 days ago [-]
The first one might not make sense but it gets them the pipeline for when it does make sense in the future. Plus for a lot of things using LLMs they'd be fine with older tech, especially if running it was even faster and cheaper.
notnullorvoid 18 hours ago [-]
> So their survival rests on the presumption that AI progress will fall between these two extremes.
That feels like a very generous framing. There's very little opportunity between the extremes that would paint a convincing outlook of survival for either company.
Perhaps if they were able to scale down their spending drastically they could survive, but that requires acknowledging their current valuations are BS. Doing so is a major risk, that will piss off all share holders. There's also the employees they would need to fire or reduce salary. The shift of focus internally to sustainability would be a major challenge.
sinuhe69 12 hours ago [-]
I have a problem with the cost per task metrics of Artificial Analysis. We don’t know how they calculate it exactly. But recently, cost per task has become the most discussed topic. The logic is basically: if model A achieves 55% on benchmark X and model B 60%, but the cost per task of A is 50% cheaper, people would choose A instead of B.
But that implies that all output of the less intelligent model A is usable, perhaps only a bit worse than the output of B. But what if the output of A is unusable, or it can only deliver usable results in 1 out of 5 tries? In such cases, the user will have to rerun the task and it will very quickly double or triple the cost and makes the old average number misleading! I would argue the retry and flaky cost will be many times bigger than the average token cost and that is the true cost the users have to bear.
AI-Benchy [0] (admittedly a one man benchmark) shows a much different figure than the numbers of Artificial Analysis. Opus 4.8 cost per task according to AA is $1.80 and Kimi K3 is $0.94$. According to AI Benchy, however, the *cost per successful task* of Opus 4.8 is 10.7 cents vs 19.4 cents of K3. The number of correct tests and pass rate of Opus 4.8 is also higher than Kimi K3.
So on a cost-per-usable-result basis, Kimi K3 is actually pricier than Opus 4.8 — the opposite of what AA’s headline number suggests.
Thus, I don’t know if I can believe the numbers of AA or we need to track the cost ourselves.
Dario can reverse this by going on the podcast circuit again and threatening everyone with 75% job losses due to (his) AI this time
Ramp the number up to 85% if that doesn’t work
If it still doesn’t work, go nuclear and target 100% job losses language
a13n 1 days ago [-]
Definitely feels like there’s a particular narrative being pushed on HN today.
Havoc 21 hours ago [-]
OAI feels far more cooked than anthropic.
One of them heading to IPO and the other opting to not show their books should tell you everything
athrowaway3z 1 days ago [-]
> Once a model is built, the biggest cost is inference
Something i cant find any reliable data for, but would help for a sense of scale: How much use before its equal to training?
I.e. assuming you have the training data and setup, and we only care for compute - How hours of using eg Kimi K3 / Fable, for it to equal the compute required to train it?
philipportner 1 days ago [-]
if you assume that training requires about 3x the compute of inference (one forward pass, one backward pass, parameter updates), and we take DeepSeek-V3 since their numbers are public.
they used ~14.8 trillion tokens with about 2.66 million GPU hours.
14.8 * 3 = 44.4 t inference tokens.
obviously, this is back of the envelope math, but at 100t/s you would need like ~14k years. scale this to >100k GPUs and your in the hours to a couple days range.
philipkglass 1 days ago [-]
In November of 2025, I would have said that Anthropic's Opus 4.5 model together with their Claude Code harness was the first and only system where a well-specified software feature could be implemented correctly for me in one shot. Today, I'm about equally happy to use Claude or Codex. And if both of those start to squeeze customers for money or get too zealous about safety it looks like there are going to be plenty of capable open weight models and true open source harnesses to use with them. Even Google might eventually deliver a capable model + agent combination (Gemini 3.1 Pro still seems pretty strong, but Antigravity was inept the last time I tried to use it.)
I'm happy if Anthropic's business remains viable as one of several strong competitors. The company's safety-first ethos is driving them to increase refusals and deliberately-built-in ignorance with their newer models. In the long run, Anthropic may be best remembered for accelerating the development of software in general so that other people could build less timid tools.
himata4113 24 hours ago [-]
I would rather have anthropic fizzle out and somebody else take their place.
piazz 1 days ago [-]
> This is where OpenAI has an advantage over Anthropic. While its models are trailing Anthropic's in recent months, its investments in product, consumer experience, site publishing, voice, and hardware are all directions that have clearer moats.
I was with you up until this point. I don’t think OpenAI has any more of a substantial product moat than Anthropic; if anything, the Claude / mythos etc brand is a valuable asset that OpenAI lacks.
Yes, many of the elite HN engineer always online types have come to prefer Codex, and but if you actually talk to regular engineers in industry, agentic coding is simply still synonymous with Claude Code.
And for the non-engineering uses, Claude is so much more pleasant of a conversational companion than any of the GPT line, and I suspect is this baked deeply into the model, otherwise OpenAI would have closed this gap by now.
joshstrange 1 days ago [-]
I think the jury is still out on K3/Q3.8 and if they are equivalent to Opus or Fable. Benchmarks have gotten incredibly murky and I've tried models that are "the same as X model" and been unimpressed. I've tried other models with Claude Code and with tools like OpenCode or Pi and nothing has really come close to Claude Code using Anthropic's models (mostly Opus 4.8).
I'm not saying these other models are trash, just that I'm not quite ready to put them on equal footing to Anthropic or OpenAI models.
I think the future of LLM coding (and more) is probably routers that decide on a per-task basis which model to route to. Know when to use Fable/Opus and when to fall back to DeepSeek/Kimi/Qwen, even all the way to local models. Of course that's not in the frontier lab's interest but it feels like there is a lot of low-hanging fruit there. I hate that right now it's pretty much "just use the same model for planning/execution/etc" (without standing on your head).
sim04ful 1 days ago [-]
If there's any hope for AI sovereignty and equality, we would have to either make expensive models cheap to run or make cheaper models do less work.
Making the latter happen involves either reformulating work in ways less intelligent LLMs can work better with. Or condensing intelligence into smaller models.
curious_cat_163 1 days ago [-]
I think condensing intelligence into smaller models is the way to go.
Also, “smaller” can mean many different things. The cost is not in storing the weights on disk.
It is in the power required to do the inference with the “active parameters”.
There is increasingly more evidence that those two can be decoupled and more power to those who are pushing on that lever!
yalogin 1 days ago [-]
The bigger question for me is , at what point does investing in higher capability general models will stop showing the ROI?
For example - What percentage of workflows require this new highly capable model? How much of it can be replaced with the software tooling around it? What I mean is if the software tooling can optimize the query over a few iterations does it get the same output as from a single shot high capability model query?
dgellow 1 days ago [-]
What ROI? Nobody is currently making money outside of the hardware manufacturers and hyperscalers
warm_soup 1 days ago [-]
Well written article. I liked how the author categorizes companies and their strategies into different buckets (not authors choice of words) and/or combination of harness, data centers, electricity, foundational models. I would have loved to read how companies mentioned are pivoting to build their moat
m_ke 1 days ago [-]
Anthropic will get squeezed by open models for 80% of the use cases that don't require frontier capabilities and by vertical specific labs for the high value tasks that would (bio, finance, math, etc.), where smaller use case specific models will beat them on cost and speed while matching or exceeding the performance of their largest general models.
Even their hail mary of being first to "AGI" will never happen because all it takes is China blockading Taiwan or Nvidia cutting them off to stop them from eating up a large chunk of the economy.
There is no scenario where the rest of the world will sit on their toes and let OpenAI or Anthropic monopolize "AI". Too many countries, large well capitalized players and partners / suppliers who could never let that happen.
Kimi K3 allows all existing players to restart at the frontier and keep competing with OpenAI/Ant. It also gives employees at these labs a better more lucrative path of starting new labs with fresh books and clean cap tables, building on top of K3 without needing to spend all the capex on pretraining their own models. Plenty of them already vested their stock and would have 0 problems raising 100s of millions of dollars for new labs, making them paper billionaires over night.
energy123 1 days ago [-]
None of my use cases require frontier capabilities but I still pay $200/month to a frontier lab. I value the additional time saved at more than $200/month. If I had to pay actual API rates, then I'm not sure what I would do, but it would not be an easy decision.
m_ke 1 days ago [-]
Sure, but anthropic is charging businesses based on usage now and tried hard to pull Fable from the consumer subscriptions before Sol and K3 dropped.
Even now on the $200 plan I use up my Fable credits in a single day and had to start using codex and openrouter for more usage because Fable burns $100s an hour when billed on usage.
surgical_fire 1 days ago [-]
I thought similarly until I decided to try DeepSeek.
It became an easy decision, even the $200/month by Anthropic sounds like a bad deal.
cmrdporcupine 1 days ago [-]
Yes, the reckoning here will happen in a year or two when the (probably subsidized, maybe?) coding plans become either unavailable or much more costly.
It's also already the case that in larger companies people do not have access to these plans as employees and must use API rates.
There's also the political angle. When Anthropic and OpenAI held back their top of the line models because of Bessent & Trump's bullshit, and threatened to deny unwashed foreigners like me access... I dropped my Codex plan and made do purely with GLM 5.2 for three weeks before OpenAI finally released 5.6 Sol. Feels inevitable that this will happen again.
Or, somebody will come up with a way to serve e.g. Kimi K3 or the new Qwen model in an extremely cheap way. Or DeepSeek releases a competitive model at their cut-throat rates. And then the cost argument just wins.
m_ke 1 days ago [-]
k3 costs will go down at least 3x within a week of the weights dropping.
we'll get new quants, dspark speculators, distills and optimized kernels
as long as there are near frontier models available there will be inference providers selling them at or below cost of inference in attempt to get market share.
cmrdporcupine 1 days ago [-]
I have not seen that kind of significant drop with GLM 5.2 yet? so curious why you think it will happen for K3.
This is a very large model. Much larger (3x) than GLM. The resources to run it are very expensive.
tokai 23 hours ago [-]
There's been a price war going on openrouter between providers of GLM 5.2. NovitaAI, DeepInfra, and StreamLake keeps underbidding each other in waves. Yesterday evening both input and output $/M was ~$0.3. Output was especially cheap.
cmrdporcupine 10 hours ago [-]
I had the opposite experience. I bought the $100 monthly sub from Neuralwatt last month because it was the only economical provider for GLM 5.2. They raised their rates halfway through, and it simultaneously became too slow to use.
I just looked at DeepInfra -- I've got an account there already etc -- and it's at FP4 quant. How much that effects the quality of inference for GLM, I can't say. I could see using it as a backup when other things run out but don't think I'd trust it.
SubiculumCode 1 days ago [-]
How much is Anthropic's price due to inference cost or extra margin they can get away with by having the best model?
vsilent 8 hours ago [-]
MiMo 2.5 Pro works just fine too
joey64 1 days ago [-]
So, what's the most affordable way for a pleb who doesn't own 17 H100s to use Kimi K3 or Qwen 3.8?
vanillax 1 days ago [-]
you cant. The best you can do is Qwen 3.6 27b with a 24gig ( or cumaltive gpus ) to get to 24gb vram. ala 3090, mac with 36gb ram, amd cards, halo strix amd, dgx spark etc. Lots of youtube videos out there.
svachalek 1 days ago [-]
I haven't seen either of these running outside their creator's services yet, but typically you can watch services like openrouter or nano-gpt for it to show up at a (usually small) discount.
Alpha3031 1 days ago [-]
Well, if you're happy with around (as in within an order of magnitude or two of) 0.1 tokens per second... I believe that's around what people are getting when loading MoE weights from NVMe.
svachalek 1 days ago [-]
You've got to consider how much power that uses though. Depending where you live, some of these providers can serve it for less than you pay for power.
drnick1 1 days ago [-]
There will be smaller versions in the 10-30B parameters range that can run on consumer GPUs.
chihwei 1 days ago [-]
Anthropic needs to stop fearmongering and stop gatekeeping legitimate frontier AI usage from the general public! The AI built by distillation of the civilizational intelligence should not be exclusive to the privileged researchers.
IshKebab 1 days ago [-]
So many words to say so little... Don't bother reading this.
zkmon 1 days ago [-]
Add another dimension - users tightening their purse on AI spend as the reality of returns hits them. Large companies might go for locally hosted models.
1 days ago [-]
simianwords 1 days ago [-]
To everyone praising Open weight models, could you answer a simple question?
If Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them, such that it makes financial sense for Anthropic to train even bigger models?
Assuming it is preferable for everyone that we get better models in the future. Distillation attacks remove the financial incentive.
__bjoernd 1 days ago [-]
This assumes all the open models are just a result of distilling Anthropic models. Which remains to be proven.
And if they are, the point remains that Anthropic has a brittle product advantage that users and investors should be cautious about.
asadotzler 1 days ago [-]
OpenAI says they're not just distillation. "Some observations on Kimi: 1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that."
If you put any faith in claims made by OpenAI's "Head of Strategic Futures," then it's hard to believe these models are just the result of distillation.
simianwords 1 days ago [-]
> investors should be cautious about
if they are cautious, what would make them invest in newer bigger models without the expected return? generosity?
drakythe 1 days ago [-]
Sunk cost fallacy.
So much money has been invested in Anthropic and OpenAI at this point that to declare it a loss and walk away could potentially destroy a lot of VC firms, and a non-trivial chunk of the US Economy.
simianwords 1 days ago [-]
Sunk cost fallacy is not relevant to future investment.
daadx 22 hours ago [-]
what an absolute plonker
do you know how ROIC is calculated?
Good luck hiding your huge sunk cost of bilions and billions in there bro. I wish people who had zero understanding of actual finance would never comment about it.
Moreover if they declare their existing assets are bunk, the valuation is marked down, especially after now pricing in failure risk - this is catastrophic for VC's.
Again, bro, just be quiet.
simianwords 14 hours ago [-]
sanest comment here
SideburnsOfDoom 13 hours ago [-]
> Sunk cost fallacy is not relevant to future investment
Outside of future choices, what in the world is sunk cost fallacy in fact relevant to? Do you understand words?
Alpha3031 1 days ago [-]
I don't think the timeline is plausible for Moonshot to have done any distillation from Fable, which means any distillation training data would have been from Opus.
swiftcoder 1 days ago [-]
The frontier labs have been tightly controlling model access lately to prevent further distillation, and the Chinese labs are still catching up rapidly - it's likely that whatever moat they used distillation to overcome early on is already dry
surgical_fire 1 days ago [-]
[Pending evidence that Chinese models are only based on distillation]
throwa356262 1 days ago [-]
While I am sure some do distillation (even r/LocalLLaMA is full of randos bragging about it), I wish the companies named by Anthropics would respond to these accusations.
I think at least deepseek can easily take Anthropics to court and win a defamation case. The number of requests allegedly done from deepseek IP range is so small it is barely enough to run a few benchmarks.
watwut 1 days ago [-]
> f Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them,
I dont care if that company dies. Or if OpenAI dies. I dont mind investors investing into other things either.
bellowsgulch 1 days ago [-]
You can simplify your question even more:
To everyone praising free and open source software, if software companies don't make money, how are they and YOU going to get paid, PERIOD?
simianwords 1 days ago [-]
Commoditising complement
behnamoh 1 days ago [-]
Nope, I'll still buy Claude because the overall XP is better than Kimi and Qwen who literally copied basic harnesses to make kimi-cli and qwen-cli, respectively.
Also, you can tell if a model is genuinely powerful and well-thought-out vs a model that acts like it.
It's like Apple vs Xiaomi/Huawei. Sure, you can get a Huawei with bells and whistles, but most people learnt the hard way that those companies just copy the iPhone, so might as well get the real deal.
InsideOutSanta 1 days ago [-]
As somebody who bought an iPhone the day it became available and has been using iPhones for years, my favorite phone I've ever owned is my Huawei Mate XT Ultimate.
People keep pretending that Chinese companies only make second-class copies of American products until it's too late.
daadx 22 hours ago [-]
people buy Apple because of the brand - in particular trust.
the vast majority of people do not care about like-for-like phones based purely on features.
osti 1 days ago [-]
Lol yet I've used Apple and Android phones extensively and would choose Android every single time.
palata 1 days ago [-]
> so might as well get the real deal.
Almost the same thing for twice the price, just for the pleasure of saying that you believe Apple was first?
Alpha3031 1 days ago [-]
To be fair, the flagship Xiaomi, Vivo, Oppo etc are comparable in price to other flagship devices. You very much pay a premium for the large camera sensors they put in these devices, amongst other things.
Rendered at 21:31:01 GMT+0000 (Coordinated Universal Time) with Vercel.
The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release.
Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software engineering (if not already). Does anyone think we need a Mythos level model to plan a road trip, or give someone tips on making a cake recipe?
A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand.
Furthermore, if you're an enterprise the risk of data exfiltration and feeding data to a potential competitor like OpenAI or Anthropic is greatly reduced if you could shift to on-prem ASIC deployments. A handful of chips could cover a wide variety of use cases and cover them more securely. There are a lot of corporate use-cases for LLMs that are not frontier math research or coding.
So I guess it depends on how much the latest-greatest model motivates people, and my read on the current churn is that developers are extremely unloyal to brand at this point and will jump to whoever has the best model. And as long as the best model is running on programmable GPUs, that will be the dominant form.
Custom AI for things like facial recognition in cameras has existed for decades, before LLMs were a thing. I don't see that getting replaced. And on-device conversational intelligence might go that route as well, we'll have to wait and see. It's a lot of silicon to dedicated to a static non-changing thing. My money would be on programmable TPU-like things (Apple's NPU kind of stuff). It just seems more flexible to have an array of compute that you can load different models into, so you can update it, etc.
What about all the things you don't currently use an LLM for?
If a specialized chip can run a model 100 times faster, you can suddenly use it for a lot of things at sub-second latency. You can write "make white transparent and add a red outline to x.png" instead of the corresponding imagemagick invocation and perceive little to no latency difference. You can hook it up to your browser and have it yank out all advertisements live, or tell it to highlight anything that might interest you, again, live. There's probably thousands of latent use cases nobody has thought of that would be enabled by a truly fast LLM, even a mediocre one.
I don't think an on-device model needs to change much; it's already quite general in its capabilities.
Looking for a lamp in a specific style?
"Make a VR application set inside my apartment (based on all the photos of my apartment from my photos directory) with all lamps under $100 that fit Scandinavian interiors and could be delivered to my house before Friday. Place the lamp on the dining room table, allow us to: cycle through lamps, change time of day, and interact with all the lamps and furniture".
Five seconds later and bam you have this new piece of software that you'll use once and then dispose of.
AI: "I'm sorry, a security guardrail prevents me from performing this operation."
Your company will decide what makes economical sense.
If you can have only one AI processor in your laptop (because they're big and expensive), it's going to be a GPU. This AI processor needs to inference LLMs, audio processing, image generation, video generation, etc. This is on top of normal graphics processing requirements such as video games, playing videos, decoding, encoding, etc.
At the enterprise level, I can see some ASICs working once the market fully matures and improvements in architectures slow down drastically while demand for inference increases drastically. How far are we from this world? Maybe 5-10 years? It seems like model architectures are still changing rapidly and labs want fast experimentation that programmable GPUs offer.
GPUs will still dominate in general - just like how CPUs still dominate despite ASICs.
They are fast, but they're still programmable accelerators, not a model burned into the gates.
If they designed this right, it means that once they have a model, so long as they keep the hyperparameters fixed they can change the weights much faster than it takes to spin up a completely new chip, essentially at a cost of doing a minor revision.
Not because rom is better than flash, but because the critical part is distributing the memory with the compute. Mask ROM is just the densest way of embedding memory with logic. Instead of having a large pool of memory connected to the separate execution units with a bus, each execution unit locally has the rom that it uses. Data movement is >90% of energy use in modern ai accelerators, removing it as far as practical is how they get performance and silicon and energy efficiency.
Many AI uses are not that volatile. I had a 20 minute conversation today with some company's AI phone assistant. It was extremely good and would have been very helpful if any of the dozen people it tried to route me to would have picked up their phone. That AI won't need to be upgraded for a very long time. There is no reason for it to have a cloud brain except to force a recurring revenue for the company selling it.
Hardcore gamers are constantly throwing down insane money on the latest hardware. The rest of us can get by for a couple years with whatever we bought when the last one broke. Yeah, it's not the latest, but it gets the job done. I wonder if AI has not already reached the point where a gen 10 CPU--uh, I mean a v3 AI model--will get the job done for the next year. If I really need the up-to-the-second latest abilities for a minute, I can fallback to a cloud brain @ 1M tokens/$. Why pay a monthly lease on a 5-year plan for a 4-door Ford Ranger as your daily commuter? Buy a Clio and rent an F-250 twice a year when you need the hauling/towing capabilities.
It takes about 18 months to go through the design, verification, and manufacturing process if you move at breakneck pace. Design could probably be sped up.
About 18 months ago the top model was GPT-4o. Not great by today's standards, but still good enough for many tasks (certainly a big chunk of chatbot queries). The current SOTA covers far more use cases, but importantly at a level that surpasses many thresholds of utility.
Maybe, but what is the shelf-life of that 18 month decision? Barely good enough today, when it launches, starts to get worse and worse every month going forward. You have to recuperate that investment on your depreciating asset.
And you are competing against anyone with the foresight to use a TPU instead, and the benefit from any new paper that finds how to distill, quantize or whatever better so their solution gets algorithmic boosts while you are locked in.
One could also imagine hybrid models, where part of the model is burned into ASICs and part of the model exists in VRAM/HBM2 so it can be updated.
I don't have enough low-level knowledge to evaluate the technical or economic feasibility of the above ideas, however.
Sometimes you need speed; sometimes you need quality. There are very different use cases for each. For my own workflows, I sometimes want something very simple done ASAP; other workflows need "subjective" reasoning and careful crafting of responses.
A model will often come up with worse results given more cycles of compute, only because it will tailspin from second guesses, rethinking and literal flip-flopping on concepts.
--- edit: to those following the thread below... if you look at the comments from the account replying, it's pretty obviously a pro-China account and all replies are antagonistic against anything other than a total submission to the Chinese state. My responses are intentionally antagonistic as every point I've brought up is completely ignored in favor of insults, so yeah, I've been insulting back.
If you let a model spin too much, you get worse results... that's a fact, even for the best models. Nothing you've stated since actually refutes that and acting like a paid bot doesn't mean anything.
I mean, if you want to felate Xi Jinping, have at it... that's all on you bro. I don't have a horse in this race.
You're just an idiot who assumes everyone that doesn't agree with you is the same. I'm surprised you're able to reply with Xi's penis in your mouth... even if it is kind of spacious in there for his limited size. How is the poo bear these days? Does he like it when you gargle his testicles in your mouth?
Assuming such work was happening.
Yesterday there's a news on a breakthrough for probabilistic computer with 1 million p-bits [1].
Since LLM is stochastic in nature, this type of new computer can be much better than ASIC for processing LLM data.
[1] Biggest Probabilistic Computer Turns Noise into Answers:
https://news.ycombinator.com/item?id=48971938
This is not like a bayes model or something were it's distributions all the way down.
Imagine if Anthropic could give effectively unlimited access to Sonnet, for $20. Wouldn’t that be an appealing option for many users? I know I’d make a lot of use of it for agentic tasks, office work, summarization, etc; when right now I’d save quota for more important tasks.
It is a bit like saying "why would you hire someone with a doctorate when you could get unlimited high school grads". How appealing that sounds depends on your needs.
The ability levels of the cheap models are encroaching on the abilities of the frontier models faster than frontier models are expanding their abilities. If we haven't already, we will very soon reach a "good enough" state where having the "best" model matters less and less and less.
By analogy, if you buy a new computer, do you get the absolute fastest CPU available? Maybe, depending on your workload. But if you're 90% of the population, you get the cheapest one that has enough power to meet your expected workload, which is mid-range, not top of the line.
Many of the responses seem to say "but there is a market for cheap models".
There is a world where we have cheap models and we don't have ASICs powering them. Things like TPUs and NPUs, which are programmable, are likely to fill that role. They are optimized for inference while also allowing different (and updated) models to run on them.
Given two companies competing on the cheap end. First company goes TPU, second goes ASIC: who wins? My bet is on TPU since they can update their model, even if their hardware is slightly more expensive and slightly slower, since the optionality of new models beats the performance gap. That may not hold forever but given the pace and volatility of the current LLM market, I believe it will hold for some time.
Models like Kimi 3, GLM 5.2, or even Fable 5 for that matter are reasonable to burn to.ASIC because they are over the threshold of "good enough to be generally useful", something that will continue to be true in the future.
Most people do not need the latest model, they need a sufficient model. If I had Fable 5 on an ASIC, I imagine I would use that and ignore paying API rates for Fable 6.
Which is why my original post mentioned the volatility in models. We aren't just doing research on frontier, there is a huge amount of research on quantization, distillation, etc. that is changing the landscape at the low-end almost as much as it is changing on the frontier.
And it is also why I mention revealed preference. What feels sufficient / "good enough" today is a moving target. This isn't just a question of what you want, it is a question of what is economically viable for the entity that will be designing, manufacturing and marketing this ASIC of which you speak.
This is me personally. The calculus is different for other people. But I suspect Kimi 3 is pretty darn close to that tipping point for an awful lot of people.
The managers of the firm will.
They dont care that its faster unless it translates into the financials. They want lower costs, higher revenues - explain how it fits brudda.
There’s absolutely a place for a lifestyle subscription to a sonnet model that you could just use everywhere all the time.
[1] https://finance.yahoo.com/technology/ai/articles/google-plan...
The question is really whether 640k is enough to last you until your next hardware upgrade.
pretty stupid statement lmao
The incremental unlock of capability by ever increasing frontier model sizes will eventually reach diminishing returns.
I would argue tnference speed increases would actually unlock a different kind of more meaningful value for a wider audience.
640k was enough ... in 1981 ... almost fifty years later is 50,000 lower than a standard off the shelf PC now
The comment seems like the nerd equivalent of 6 7
The quote became famous because it's a failure to imagine that people would find new uses for computer memory if it became plentiful and cheap. In the quote Gates is not expecting people to come up with more demanding applications for computer memory (RAM) than the ones which were available in 1981.
Computer memory did in fact become plentiful and cheap after this, and computer programs became more complex and memory-hungry and today we wouldn't consider a 640kb an acceptable amount of ram for even the lowest-end device.
OP is repeating the quote about memory to indicate that they think that modern LLMs will be a similar resource. We should not expect that demand for LLM inference will stay flat, and that once everyone has cheap access to Fable level, we won't find new more demanding uses for it.
I get this reference!
This is starting to look at a lot like Intel vs Arm from the last era.
The Fable & Mythos are starting to look like a giant Xeon, while the smaller lighter models are starting to look like a lot of tiny ARM chips which sip on power instead.
The risk is the same as what Intel had. There is a group who are pushing them to go bigger and with a resource no limit approach, who have a lot of dollars to push you that way.
Follow them and they lead you to a pile of money, but then you risk something like Apple Silicon happening.
Something which got better because of efficiency & continuous improvement, not neutered due to it.
The problem was that Intel stopped making things faster and better, and shifted to wearables and mobile chips instead of investing in their hard-tech strategy that had worked for decades.
Any value will come from the largest models, and those largest models are unlikely to ever run on consumer hardware within their window of relevancy.
Small models are good enough depending on your task. That's the point. A model you can run on your phone or laptop is an incredibly useful tool for a lot of problems even though it isn't the "best" theoretically possible model.
And firms will be kept in check with financials.
If your competitor starts using chinese models and delivers better earnings whilst you are spending more on american ones... hahaaha. Wait and see what happens.
You will be FIRED!
You can run a Moroccan chicken fragment on the cheap in homage to what you cannot run
Yes. The latest OpenAI and Anthropic models are terrible at planning roadtrips.
This is something I try to use them for frequently. They constantly get things completely wrong.
I’d say that about half of the stops they suggest fail to follow whatever filters I’ve asked for.
And this is one of the big things that seems to be missed in these discussions: There is no longer a universal linear trend of LLMs being 'better' each iteration. They are becoming more specialized, and ones that approach problems from a different angle (like Fable/Mythos) can appear breakthrough when first released, but we don't appear to be on a path that actually leads to general purpose hyper intelligence.
It was nonsense to draw in investment and justify an inflated valuation.
lmao
There is already custom hardware see cerebras.
GPUs have a lot of slack there is at least one lab that had a (small 8b) model generate almost 3000 tokens per second on a MI300X for a talk, instead of the typical software stack that did maybe 100ish tokens per second.
High bandwidth flash storage is in the works, i.e hard drives with TBs of storage and over 1 TB per second of read speeds. Meaning that in a couple of years you may be able to buy a card with 40-90GBs of HBM and 4TB of HBF and run a 3T model locally at a reasonable speed for 10-20k as opposed to a cool mil.
There is no "may" here. You will see this.
It's always difficult to see it from the present, but we're not at some end stage in hardware development; we're still on the same curve our predecessors also couldn't see: they couldn't imagine that there would be high performance computers carried in our pockets, with staggering amounts of storage and compute, putting to shame the machines they filled rooms with.
That's where OpenAI in a box is going to come from. And it won't take long.
It'll obviously be China, and they won't need the bleeding edge of lithography tech to make it happen. Every Chinese smartphone will have something like Sonnet 5, along your car (well, not those of us in the US, but we'll look longingly at pictures of them while we drive whatever the government decides we're allowed to drive in Fortress America).
Give it ten years and your smart litterbox from Temu will be running its own local model.
China won in cloud services? Nope, not even close. They had to clone AWS just to try to keep up.
China won in mobile? Nope. Although they're very competitive there.
China won in search? Nope. Baidu who?
China won in ecommerce? Nope. Their dominance is overwhelmingly domestic.
China won in software? Nope. Windows is US. Android is US. iOS is US. MacOS is US. Linux is US/Europe/Global. Look at the top 50 largest software companies.
China won in silicon? Nope. Look at the top 50 companies.
China won in .... India and Latin America can manufacture iPhones now.
But sure, China's the obvious winner this time. Good luck.
The comparison I've seen elsewhere is the old console systems with separate cartridges for games... I wouldn't want to be regularly swapping them, but if they came out in a form factor that didn't require me to shell out multiple-4 digit figures in upgrades just to use the next model, it'd easily be worth it for me.
I've already got a home lab, and it's specced to last, minus the GPU. I picked up a separate system for local llm experimentation, but I'm not likely to be upgrading it again. The value add is incredibly small compared to the cost. The real benefits are data privacy and never worrying about rate limits, and there's a price point beyond which an incremental improvement to the model doesn't justify upgrading the system.
Ultimately however, that just means that models will become dirt-cheap. The money will be made with applications built on top of the models.
https://news.ycombinator.com/item?id=48464958
As of today, that appears to be Google!
https://news.ycombinator.com/item?id=48986351
Subjectively, it's hard to say if people will pay for "a model from 18 months ago, but REALLY FAST AND CHEAP"
The speed difference suggests some use cases that might narrow the performance gap. With a > 50x performance delta you have some headroom to play with.
You can do many many fast iterations of ye olde "Ralph loops." You can also jack up the reasoning/effort level. And you could probably do some combination of both, while still running really fast ie 10x the number of iterations at 2x reasoning/effort.
So I think a hypothetical "50x faster Opus 4.8, but burned into ASICs" could be pretty competitive against the frontier models from 2027, 2028, 2029, and maybe beyond?
What's fascinating is that we are pushing the state of the art of hardware at this point.
I don't understand how this works when the models are evolving so fast that your burned ASIC is outdated (or at least not top of the line) in a few weeks.
To me, it's like imagining if Sonnet 3 was burned into an ASIC 8 years ago and then never changed. It would still be revolutionary, and today we would have an entire ecosystem of tools and services built around it, likely surpassing some of our current workflows.
The frontier is a different beast, but it would likely mean competing on price.
I think that's the problem at the moment. A much better LLM that doesn't use my battery is 20ms and <10mb of data away.
Sure, an ASIC model will always be behind (12 months? 18 months?), and a hosted flagship will be significantly better. But as time marches on, would I use a flagship model from last year if it came on a PCI card and cost $1000? Without a shadow of a doubt.
Cerebras tries to get around this by keeping everything on cache SRAM as much as possible, which it burns directly to the chip wafer itself and physically places that SRAM directly next to the tiny compute unit that does the actual math.
An ideal setup (not sure how easy this is to achieve in practice), is the burn the weights of the model directly to the chip as a sort of ROM, the actual math operations as actual digital circuits, and have SRAM, or even something akin to naked registers to directly compute off inference batch data. Cuts out 2-3 layers of abstraction and indirection.
You still need some memory for the context, in flight answers, ... but not for the model weights and for the output of the intermediate layers. I found taalas demo here: https://chatjimmy.ai/
Scaling this up to 2.8 Trillion (350X increase), will certainly be challenging.
If I was younger and had the right background, I'd love to dive into attempting somethign like this
Last I saw they posted Deepseek R1 numbers in Feb of this year.
The challenge is rolling out a new one every 7-8 weeks as the weights change & cheap enough for a hyper scaler to afford to buy one and save enough on power over the next 8 weeks as a payoff.
In fact the superior models are irreducibly nothing but superior web services run from China.
Nobody needs "web services run from China" to use Chinese open weight models.
Likewise, for compute, is the ASIC somehow going to beat a systolic array? You can't have one circuit per weight: the die area and electrical fan-out would be insane. I'm not seeing how an ASIC specialized for a specific model would actually help much.
I mean, sure, we can build more specific accelerators, e.g. for softmax, but these work fine in the context of a programmable pipeline.
Yes, there are more exotic things out there, like optical matrix multiplication systems. Those are different. But above, aren't you talking about just doing conventional digital linear algebra, but with a model-specific set of circuits?
I can accept the idea of specializing a circuit for a specific model shape, but I'm not seeing a need to specialize a circuit for the weights inside the shape.
----
The Board Departure: Mike Krieger, Anthropic’s CPO and a co-founder of Instagram, sat on Figma’s board of directors. He resigned on April 14, just days before news of Claude Design broke. This sparked speculation over conflict of interest and the use of proprietary product strategy information.
Betrayal of Partnership: The launch aggravated the tech industry because Figma relied on Anthropic's models to power its own AI features, and even announced a joint "Code to Canvas" integration. Reports indicate Figma was blindsided by the depth and scope of Claude Design.
Market Reaction: The "SaaSpocalypse" thesis—fears that major AI foundation models will rapidly build application layers and cannibalize their own SaaS partners—was realized when the news broke. Figma’s stock saw an immediate 7% drop upon the announcement.
----
I would suggest to people using LLMs: you should be cautious about giving these companies data or relying on them. If you're building an AI startup, there's a very good chance they could decide to directly compete with you if your idea has traction. You're also at their mercy for API pricing etc.
story as old as internet itself!
These companies haven't got the attention span to work on one thing for that long.
The driving constraint if you want to take a spot in B2C is the speed at which consumers replace tech, not the speed at which tech can be developed.
Wrong question. The real question is "Who and what will they buy with the gold?"
The golden rule is just the beginning, and no, there's nothing positive for the rest of us.
But the big money has been skipping Leg Day for ages, and that lack of a foundation will bite us all. Which sucks.
Then, by the time GPT 5.5 came out you could already generate a flawless PDF and Word doc resume with the same formatting as your base resume. It really is amazing.
Trying to compete with these labs is a bad idea, you might work hard on your thin wrapper and then they come out and have their chatbot or harness do it way better than you.
But I also think this is really cool. It's a new era in software. Build something worthwhile and you won't need to worry about a lab doing the same thing with their chatbot.
They think whatever you are doing is cute and would very much like to ensure their models can do it even better in the future, but competing? Not even worth the time to think about
LLM generated code is not copyrightable, so even if they do "steal" it - I don't think there's legal grounds to do anything about it.
It can't be stolen. You don't really own it.
You can try to lock it in a safe and hope no one ever gets a hold of it. You can lie and say you didn't use an LLM, but Anthropic and OpenAI et al probably have logs to disprove that.
But, if push comes to shove, you don't own it...
Someone tried, as a bit of a stunt, to register a work for copyright with generative AI as the sole creator/author. That registration was rejected. This is quite different from a person using generative AI as a tool to create a work.
People have copyright in photos and videos they create, even if they used a camera. Same with images and code, even if they used an LLM.
I'm not your lawyer, but to the extent you have copyright in works you create, the fact that you used a tool doesn't diminish the copyright.
The clearest part from the page:
> Before the proliferation of generative AI, U.S. courts did not extend copyright protection to various nonhuman authors, holding that a monkey who took photos of himself lacked standing to sue under the Copyright Act; that human authorship was required to copyright a book purportedly inspired by celestial beings; and that a living garden could not be copyrighted. The U.S. Copyright Office has also long maintained that copyrighted works must be "created by a human being" and therefore refused to register works that are "produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author."
I think the long story short is if you're incorporting LLM generated content into your own work, then your own work is copyrighted but if it's purely AI created and you never put any of your own work into it, it's not (so: think the slop cannons people are pointing at social media for example -- probably not copyrighted (although I don't know why you'd want to put your name on it anyway)). Granted, I don't know how you'd ever prove that something was purely LLM generated so it's kind of a technicality for coding, but for image and music I think it applies pretty clearly.
As an aside, I think socially maintaining that copyrights are for humans only is extremely important. It might be the most important legal matter around this. I see nothing good of allowing AI labs that have strip-mined the internet to claim a copyright on the things they regurgitate, and I don't think spammers deserve copyright protection either. The human-only requirement is a very good one.
Also, a lot of people using LLMs are using it on code bases that were written before LLMs were common for coding. So in that context, you really are allowing them to train on your code for free.
> It can't be stolen. You don't really own it.
At least in the UK computer generated code has been copyright-able for decades [1]
> (3)In the case of a literary, dramatic, musical or artistic work which is computer-generated, the author shall be taken to be the person by whom the arrangements necessary for the creation of the work are undertaken.
And the ToS of the major/minor AI providers as far as i can tell have no opinion on what you do with the code in terms of ownership.
[1] https://www.legislation.gov.uk/ukpga/1988/48/section/9
[citation needed]
I've seen cases where it was determined that the LLM itself doesn't hold copyright. But where are you getting the idea that nobody holds copyright?
For one, on the margin people are willing to pay a lot for slightly better models. I know personally the value the LLM adds to my workflow is considerably more than the $200/m I pay the frontier labs. I have no interest in optimizing that to get it slightly lower. There are a very vocal minority that optimizes this or companies whose LLM expense is marginal, but I think that's the minority (correct me if I'm wrong, curious what their customer base looks like)
Also the actual LLM is a tiny portion of the value added. Anyone that tried to build agentic solutions from LLM apis quickly realizes that a huge value is the Claude Code / Codex harness. There are open source implementations like OpenCode but they're not nearly as good.
Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them. Google sheets is probably 99% and available to everyone and better in a lot of regards. But the immense value spreadsheet software produces workers above the $100 or whatever a year makes it so that there is a real moat and no one bothers exploring alternatives.
Enterprise is how Claude/OAI make their money. Enterprise companies absolutely optimize around costs for infra, it's like an entire job category.
> Also the actual LLM is a tiny portion of the value added. Anyone that tried to build agentic solutions from LLM apis quickly realizes that a huge value is the Claude Code / Codex harness. There are open source implementations like OpenCode but they're not nearly as good.
Claude and OAI are not valued at $1T because of their harnesses
> Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them. Google sheets is probably 99% and available to everyone and better in a lot of regards. But the immense value spreadsheet software produces workers above the $100 or whatever a year makes it so that there is a real moat and no one bothers exploring alternatives.
Enterprise excel is like $50 a month, the closed source labs charge orders of magnitude more than that per user per month for enterprise, and want to charge more if they can to maintain their valuation. I spent $20 in an afternoon yesterday using my work plan for one workflow for reference, I probably spend at least $50 a day and I don't even do that much coding anymore.
I'm genuinely not sure where the balancing point even is.
And cost of acquiring learning via users feedback versus additional learning data value for a given company will dictate whether such company will give up or will continue subsidized usage, despite being unprofitable on the paper, but probably really valuable as a long term strategy for GTM and product development and increased learning rates.
They're valued at that because they add a lot of value and people pay for the product. The product is more than the LLM. If you want argue the value of the harness vs LLM but flippant remark adds nothing.
> Enterprise excel is like $50 a month, the closed source labs charge orders of magnitude more than that per user per month for enterprise, and want to charge more if they can to maintain their valuation. I spent $20 in an afternoon yesterday using my work plan for one workflow for reference, I probably spend at least $50 a day and I don't even do that much coding anymore.
I want to know what percent of their revenue is this API spend compared to just a Claude/Codex max plan that's $200/m and very generous.
I can't imagine why anyone would need almost 7x that.
But $200/day is very common in the business. The monthly plans are just a trial version of what's going to come for all of us. That's why I am very carefully looking into the open models today.
This is a very rich / developed country privilege perspective.
Where I live, it's not unusual for a monthly wage to be around $200. Of course developer wages are much higher, maybe as much as $1000 a month, but $200 is still a huge chunk of that so it doesn't really matter how much "value" you get out of it if you're no longer able to pay rent or buy decent food.
Even in developed countries, $200 a month is out of reach for all kinds of people who would benefit from it (students without rich families, entrepreneurs, etc.).
Like Codex https://github.com/openai/codex
That margin is getting smaller and smaller. I would have been with you a week ago; paying for Fable was worth it compared to all other models. But with K3, the difference has shrunk to the point where, for me, it's not worth the cost anymore.
In other words, it may be worth paying five times as much to get 10% better real-world outcomes for a lot of people, but a lot fewer people will pay five times as much to get 2% better outcomes.
> a huge value is the Claude Code / Codex harness
For me, it's the opposite. Having to use Claude Code instead of the harness I prefer is a point against Anthropic, not for it.
On our side we use Claude/GPT/Kimi (it replaced Antigravity) for development. But we build our systems around a cheaper denominator (Deepseek previously, recently we added GPT 5.6 which have good prices as well). We offer BYOK for Claude but its def not an option to build something on top of it (for us).
Excel has network effects. If N people are using excel, the incremental N+1'th user is forced to use it or risk not being able to open documents from the N users. Excel is also sticky, in the sense that a user used to Excel UI is hesitant to switch to a new UI. Tokens dont have network effects. If N people use fable, the N+1'th user could use any other model, and have no impact.
AFAIK the frontier labs make their money on enterprise, and they recently changed their terms so that most businesses can't use the $200/m plan anymore.
This is why companies like Uber - and mine :( - announced per employee AI budgets on the order of ~$1000/month.
When you're forced to pay API costs, the value prop of Kimi/Qwen becomes a lot more compelling.
Why should I pay for a frontier model that I can’t use?
But once you have an AI-powered application in production, why wouldn't you go with far cheaper and capable enough models? That's just optimising a business process like any other. You'd use cheaper providers whether Chinese, or other on-prem models.
> Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them.
Sure but were these open source alternative as pervasive and widespread and talked about when Microsoft was bundling Excel with the OS? Or were there millions of dollars, if not billions of dollars being spent by these competitors? The answer is No. It is not the same scenario.
While I understand the logic of harnesses - those are not full proof. It is trivial to setup K3 or Qwen to work with Claude Code/Codex by intercepting requests and routing to K3/Qwen. You have tools like CCSwitch which can do that for you.
The only thing remains is - people willing to pay a lot for slightly better model. That is true but price sensitivity is also a thing. The mania case for Anthropic/OpenAI/SpaceX/Google is that they will capture large part of the enterprise market. But in many cases outside coding AI capabilities will be resold.
That is instead of buying AI for lets say HR functions you pay for embedded AI in your HR software. And the HR software company will have incentives to raise their own margins. If their provide similar looking experience using Qwen/K3 why will they buy Anthropic/OpenAI. That is the real risk here. While as an individual you can continue to pay top dollar.
If you're building an optimizing compiler, a CUDA kernel, a database, a high performance concurrent data structure with tricky locking, etc. etc. it's still not really close. Sol 5.6 on high just slays e.g. GLM 5.2 for this kind of work for me.
I'm sure K3 is fine for these things too, but I can't afford it at its API rates compared to a Codex coding plan.
For now.
https://github.com/rdaum/mica/tree/main/crates/relation-kern...
https://github.com/rdaum/mica/tree/main/crates/vm
https://github.com/rdaum/pagebox/blob/main/crates/wal/src/wa...
I have CUDA work here somewhere too but I have the repository private right now
The hype cycles are shortening, perhaps we really are reaching some kind of plateau this time (famous last words)
Every LLM product goes through testing and alignment after training, maybe even some quick improvements here and there. Kimi probably did something similar.
Put another way, if Google says they have the best model in the world but won’t release it in December I will start caring in December, not before.
People use these models for diff things.
Its quite possible for the things they are used for, people do not see much of a difference.
Do you hold stock in Anthropic?
> Unnecessarily aggressive
> First ever comment said "Further releases of Chinese models that demonstrate the gap is not growing substantially is a huge problem. The spending will be called into question."
Yeah I think you have an agenda
are you american?
I wish.
Chinese labs have come up with a bunch of genuine innovations: GRPO, auxiliary loss free MoE load balancing, MLA, muon optimizer, and a bunch of other ones. The Deepseek papers are really well written, this isn’t just sneaking a peek at a peer.
The problems are inherently harder now too, partially because they take longer, so your training pipeline is waiting for long completions.
Also there probably is some “distillation” (technically pseudo-labeling, which is common in ML). But I wouldn’t put too much weight on it because that was true 18 months ago as well.
Not to mention that US companies models constantly distill each other as Musk was forced to admit under oath. This whole narrative has just been a massive cope.
> This whole narrative has just been a massive cope.
So wait, US AI companies all use distillation because... it's not effective and it's all just cope? Or is distillation really powerful and they all do it, which Musk was forced to admit under oath? But when China does distillation it isn't powerful and they don't need to do it, but they do it anyway because it's fun?
Either it's powerful and everyone, including the Chinese labs, use it as a way to rapidly catch-up against the SOTA models, or it's a red herring and the huge amounts of energy spent to protect and enable distillation is all just wasted money. Which is it?
Both sides have extremely smart people. One side has more $$$ and exclusive access to the best chips. For progress to converge without a corresponding breakthrough suggests there's something else at work.
The US enjoyed an early advantage due to excessive money being poured into AI which led to the current bubble, and access to the hardware that was needed to train these models initially.
At this point, neither of these factors actually matter that much. The naive approach of simply making models bigger has hit a wall, and now you need ingenuity in figuring out better architecture for them. Precisely because Chinese companies have had to deal with more limited resources, they put a lot more effort into researching different kinds of optimizing techniques. And of course, China is also catching up in chip making, and Huawei clusters are already competitive with Nvidia for training. So, that gap is closing as well.
The big difference is that an absolutely insane amount of money has been spent in the US, while China managed to do this on a fraction of the budget. The AI Investment Surge graph here puts things in perspective. https://hai.stanford.edu/news/inside-the-ai-index-12-takeawa...
But OAI and Anthropic are trying to cash in ahead of their IPO window. I think that window is pretty much gone now.
I don’t think one should pay much attention to them.
My understanding is that the labs ran out of freely available data to train on a while ago, and now primarily rely on human data vendors such as Surge and Mercor to source their data.
Hm. How is OpenAI not a “model-only” provider just like Anthropic? Seems like they are vulnerable in the same way.
You can see a future where a user types in "plan a vacation for me" and ChatGPT coordinates everything from there. Those sorts of users aren't going to switch because model X is 10% cheaper or better.
I could easily see OpenAI become irrelevant in 2 years if they stumble at all and don’t keep up with the other frontier models.
Google's quality has gone to shit, but internet searching is still "googling", both in verbiage and in the actual service people use. It's not an impenetrable moat, but OpenAI would have to stumble pretty hard to lose all of that edge.
That said I think it is pretty easy to make a case that these would-be differentiators are either currently underwhelming or completely unproven (as in the case of hardware).
Having a good model is one thing, being able to serve that model at good speeds and match demand is another. See Anthropic ~6months ago. Or Moonshot, they've already suspended subscriptions to their coding plans, because they can't meet demand.
A related question is how much they're dependent on the APIs of Anthropic and OpenAI to achieve their results - whether through distillation or other uses.
If these models are derivative of Anthropic/OpenAI I would expect performance to be more narrow and progress to be limited.
Its value per unit of currency spent.
Financials will ultimately drive decision making.
We are already seeing that more intelligence does not correlate with more revenue, for the firm purchasing tokens.
If I was OAI/Anthropic I'd be brown and yellow in the boxers.
Lower bound of AI progress - plateau. Progess is slowing, focus is on serving a meaningful peak capability at the lowest possible price. There's been news today that Google is building a Gemini chip with weights baked into silicon. Considering a chip's lifetime of 2-3 years at minimum, and that a 2-3 year model today would be useless today, they're expecting they wont make a similar amount of progress in the next 3. Game is about selling at the lowest margin. Anthropic/OAI is cooked.
So their survival rests on the presumption that AI progress will fall between these two extremes.
We need SETI@home for Open Weight models yesterday...
I would imagine it would look something like this:
Ground breaking/novel research -> SWE -> day to day assistant conversations -> chat support bot...
That feels like a very generous framing. There's very little opportunity between the extremes that would paint a convincing outlook of survival for either company.
Perhaps if they were able to scale down their spending drastically they could survive, but that requires acknowledging their current valuations are BS. Doing so is a major risk, that will piss off all share holders. There's also the employees they would need to fire or reduce salary. The shift of focus internally to sustainability would be a major challenge.
But that implies that all output of the less intelligent model A is usable, perhaps only a bit worse than the output of B. But what if the output of A is unusable, or it can only deliver usable results in 1 out of 5 tries? In such cases, the user will have to rerun the task and it will very quickly double or triple the cost and makes the old average number misleading! I would argue the retry and flaky cost will be many times bigger than the average token cost and that is the true cost the users have to bear.
AI-Benchy [0] (admittedly a one man benchmark) shows a much different figure than the numbers of Artificial Analysis. Opus 4.8 cost per task according to AA is $1.80 and Kimi K3 is $0.94$. According to AI Benchy, however, the *cost per successful task* of Opus 4.8 is 10.7 cents vs 19.4 cents of K3. The number of correct tests and pass rate of Opus 4.8 is also higher than Kimi K3.
So on a cost-per-usable-result basis, Kimi K3 is actually pricier than Opus 4.8 — the opposite of what AA’s headline number suggests.
Thus, I don’t know if I can believe the numbers of AA or we need to track the cost ourselves.
[0] https://aibenchy.com/compare/anthropic-claude-opus-4-8-mediu...
Ramp the number up to 85% if that doesn’t work
If it still doesn’t work, go nuclear and target 100% job losses language
One of them heading to IPO and the other opting to not show their books should tell you everything
Something i cant find any reliable data for, but would help for a sense of scale: How much use before its equal to training?
I.e. assuming you have the training data and setup, and we only care for compute - How hours of using eg Kimi K3 / Fable, for it to equal the compute required to train it?
they used ~14.8 trillion tokens with about 2.66 million GPU hours. 14.8 * 3 = 44.4 t inference tokens.
obviously, this is back of the envelope math, but at 100t/s you would need like ~14k years. scale this to >100k GPUs and your in the hours to a couple days range.
I'm happy if Anthropic's business remains viable as one of several strong competitors. The company's safety-first ethos is driving them to increase refusals and deliberately-built-in ignorance with their newer models. In the long run, Anthropic may be best remembered for accelerating the development of software in general so that other people could build less timid tools.
I was with you up until this point. I don’t think OpenAI has any more of a substantial product moat than Anthropic; if anything, the Claude / mythos etc brand is a valuable asset that OpenAI lacks.
Yes, many of the elite HN engineer always online types have come to prefer Codex, and but if you actually talk to regular engineers in industry, agentic coding is simply still synonymous with Claude Code.
And for the non-engineering uses, Claude is so much more pleasant of a conversational companion than any of the GPT line, and I suspect is this baked deeply into the model, otherwise OpenAI would have closed this gap by now.
I'm not saying these other models are trash, just that I'm not quite ready to put them on equal footing to Anthropic or OpenAI models.
I think the future of LLM coding (and more) is probably routers that decide on a per-task basis which model to route to. Know when to use Fable/Opus and when to fall back to DeepSeek/Kimi/Qwen, even all the way to local models. Of course that's not in the frontier lab's interest but it feels like there is a lot of low-hanging fruit there. I hate that right now it's pretty much "just use the same model for planning/execution/etc" (without standing on your head).
Making the latter happen involves either reformulating work in ways less intelligent LLMs can work better with. Or condensing intelligence into smaller models.
Also, “smaller” can mean many different things. The cost is not in storing the weights on disk.
It is in the power required to do the inference with the “active parameters”.
There is increasingly more evidence that those two can be decoupled and more power to those who are pushing on that lever!
For example - What percentage of workflows require this new highly capable model? How much of it can be replaced with the software tooling around it? What I mean is if the software tooling can optimize the query over a few iterations does it get the same output as from a single shot high capability model query?
Even their hail mary of being first to "AGI" will never happen because all it takes is China blockading Taiwan or Nvidia cutting them off to stop them from eating up a large chunk of the economy.
There is no scenario where the rest of the world will sit on their toes and let OpenAI or Anthropic monopolize "AI". Too many countries, large well capitalized players and partners / suppliers who could never let that happen.
Kimi K3 allows all existing players to restart at the frontier and keep competing with OpenAI/Ant. It also gives employees at these labs a better more lucrative path of starting new labs with fresh books and clean cap tables, building on top of K3 without needing to spend all the capex on pretraining their own models. Plenty of them already vested their stock and would have 0 problems raising 100s of millions of dollars for new labs, making them paper billionaires over night.
Even now on the $200 plan I use up my Fable credits in a single day and had to start using codex and openrouter for more usage because Fable burns $100s an hour when billed on usage.
It became an easy decision, even the $200/month by Anthropic sounds like a bad deal.
It's also already the case that in larger companies people do not have access to these plans as employees and must use API rates.
There's also the political angle. When Anthropic and OpenAI held back their top of the line models because of Bessent & Trump's bullshit, and threatened to deny unwashed foreigners like me access... I dropped my Codex plan and made do purely with GLM 5.2 for three weeks before OpenAI finally released 5.6 Sol. Feels inevitable that this will happen again.
Or, somebody will come up with a way to serve e.g. Kimi K3 or the new Qwen model in an extremely cheap way. Or DeepSeek releases a competitive model at their cut-throat rates. And then the cost argument just wins.
we'll get new quants, dspark speculators, distills and optimized kernels
as long as there are near frontier models available there will be inference providers selling them at or below cost of inference in attempt to get market share.
This is a very large model. Much larger (3x) than GLM. The resources to run it are very expensive.
I just looked at DeepInfra -- I've got an account there already etc -- and it's at FP4 quant. How much that effects the quality of inference for GLM, I can't say. I could see using it as a backup when other things run out but don't think I'd trust it.
If Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them, such that it makes financial sense for Anthropic to train even bigger models?
Assuming it is preferable for everyone that we get better models in the future. Distillation attacks remove the financial incentive.
And if they are, the point remains that Anthropic has a brittle product advantage that users and investors should be cautious about.
If you put any faith in claims made by OpenAI's "Head of Strategic Futures," then it's hard to believe these models are just the result of distillation.
if they are cautious, what would make them invest in newer bigger models without the expected return? generosity?
So much money has been invested in Anthropic and OpenAI at this point that to declare it a loss and walk away could potentially destroy a lot of VC firms, and a non-trivial chunk of the US Economy.
do you know how ROIC is calculated?
Good luck hiding your huge sunk cost of bilions and billions in there bro. I wish people who had zero understanding of actual finance would never comment about it.
Moreover if they declare their existing assets are bunk, the valuation is marked down, especially after now pricing in failure risk - this is catastrophic for VC's.
Again, bro, just be quiet.
Outside of future choices, what in the world is sunk cost fallacy in fact relevant to? Do you understand words?
I think at least deepseek can easily take Anthropics to court and win a defamation case. The number of requests allegedly done from deepseek IP range is so small it is barely enough to run a few benchmarks.
I dont care if that company dies. Or if OpenAI dies. I dont mind investors investing into other things either.
To everyone praising free and open source software, if software companies don't make money, how are they and YOU going to get paid, PERIOD?
Also, you can tell if a model is genuinely powerful and well-thought-out vs a model that acts like it.
It's like Apple vs Xiaomi/Huawei. Sure, you can get a Huawei with bells and whistles, but most people learnt the hard way that those companies just copy the iPhone, so might as well get the real deal.
People keep pretending that Chinese companies only make second-class copies of American products until it's too late.
the vast majority of people do not care about like-for-like phones based purely on features.
Almost the same thing for twice the price, just for the pleasure of saying that you believe Apple was first?