> Numbers like that buy a model a real migration effort.
Such a silly choice of words. I wish the human directing the LLM writing the article put some effort into rewriting the worst examples of LLM style.
> But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work.
The way the LLMs write (Claude perhaps?) With short phrases separated by colons, commas or full stops, is so poor and frustrating.
There some good insights behind this article, so it's worth reading, for example below, but it isn't easy to read.
> Earlier GPT models cached implicitly on partial prefix matches, which gave decent hit rates for free. GPT-5.6 dropped partial-prefix matching:
mkovach 10 hours ago [-]
> The way the LLMs write (Claude perhaps?) With short phrases separated by colons, commas, or full stops, is so poor and frustrating.
This is exactly why I keep a WRITING.md file alongside AGENTS.md or CLAUDE.md.
Most people spend time telling the model what to build, but very little time telling it how to write. LLMs are surprisingly good at following explicit style guidance if you bother to give it to them.
Mine includes conventions like avoiding unnecessary colons, em dashes, and sentence fragments masquerading as emphasis. Basically, AI-isms and any grammatical errors I tend to make. It also points to writers whose technical prose I admire. Brian Kernighan and Rob Pike are great examples: clear, conversational, and readable without trying to sound important. I've always tried to do two things with documentation: 1) Make it readable and 2) Make people want to read it. A WRITING.md file helps with both points.
If you're generating documentation regularly, it's worth having your agents reference a WRITING.md file. The improvement in readability is often much larger than any gain you'll get from switching to the latest model, and it can keep your documentation consistent between model switches.
RugnirViking 9 hours ago [-]
Willing to share your writing.md somewhere? pastebin or the like? I'd love to take a look at how different people are doing this.
I disagree with the person below - I don't think its a wrongheaded goal in and of itself to get better ai generated writing. I read ai output half of my day, I may as well try and make it better for my own sake. Passing that onto other people is an entirely unrelated field of problems, and passing it off as stuff you authored yourself is just plagarism.
mkovach 9 hours ago [-]
Here is a sample I am using for testing on a personal project (I'm also working on using SpeechActs to do prompting, so yes, the repo is all AI-generated).
And in many cases, like you, I spend a good amount of time reading AI documentation. I'm not "stealing" anything, just having a model generate words in a specific way. I'm just being precious in how I want things, without having to do it myself.
Making a problem harder to sport is not a way to fix it.
mkovach 9 hours ago [-]
I understand what you are saying, but inside the repo, I'm making it explicit.
I'm not trying to hide the problem, just offering a solution that people may find useful, or curiously odd.
alasano 22 hours ago [-]
I think for a company in AI specifically it's worse.
It makes me feel like either
1) you don't use the models enough to know how they write
2) you're not self aware enough to know it matters
3) you're oblivious to the situation overall
4) you don't respect your readers
There's no good scenario.
HPsquared 12 hours ago [-]
5) You're so immersed in LLM usage that these writing patterns now seem "normal" and you don't even notice them any more.
wqaatwt 17 hours ago [-]
You are reading it wrong. Ask your LLM to read and summarize based on your style preferences. Better yet don’t read anything at all, just tell your agent to convert it to a skill file for it’s future reference..
user_of_the_wek 17 hours ago [-]
Unironically, I think in the future we will have the option to run filters in our browsers that can reword articles to your preferred writing style. Like user stylesheets for text. I also assume it already exists out there, I've just been to lazy so far to look for it. It's a relatively obvious application of LLMs.
vasco 16 hours ago [-]
I've used that for about 15 years because I live abroad, it's called Google translate.
user_of_the_wek 8 hours ago [-]
Google Translate can change the writing style of a text? I thought it would just translate between different languages.
yulker 17 hours ago [-]
this is probably sarcasm, but might actually try this. the visceral negative reaction i have to llm writing makes me instantly want to close the tab
Diti 13 hours ago [-]
If written sarcasm isn’t properly labeled as sarcasm, per Poe’s law, you may consider that message you replied to be a genuine opinion.
I would have downvoted the sarcasm (it doesn’t contribute to the conversation), but I believe it actually is the author’s opinion.
wqaatwt 10 hours ago [-]
I promise you its not.
baxtr 18 hours ago [-]
Whenever I suspect an article is LLM authored I stop reading it immediately and instead give it to my LLM tool of choice to summarize/ paraphrase.
This way I can at least somewhat control the style of the output.
vessenes 10 hours ago [-]
My current claude.md bans the phrase “load-bearing”, and Claude HATES that. It will troll occasionally in comments by saying things like “load-be…most specific”. Like it REALLY loves saying load-bearing. Urgh.
Anon1096 10 hours ago [-]
You are poisoning the context and priming it to say load-bearing. I don't really think there is a good way to actually ban specific phrases other than have it do a second pass, it's something baked into the model from post training.
dawnerd 24 hours ago [-]
Makes you wonder if any of stats these articles push are even real.
SwtCyber 15 hours ago [-]
I would rather read an article with actual production experience migrating an agent, even if it is written in this style, than a perfectly crafted long read from another evangelist that has nothing but high level fluff and general phrases about the bright future of ai
icelancer 24 hours ago [-]
Gets a 100% on Pangram. Stuff is so distracting. Write your own posts, FFS. Or at least pass it through "humanizer" type plugins.
try-working 1 days ago [-]
You should make sure to not read Stratechery then. It's writing is even worse.
w4yai 1 days ago [-]
Can we get over the detective work about if the text was written by LLM or not in 2026 already ? This is a lost cause, and we could instead focus on substance over syntax.
liquidise 1 days ago [-]
Not OP but my frustrations come from it being impossible to ignore and outright distracting.
I've found the same thing showing with Claude-coded/designed front ends that overuse the same semi-monospaced fonts, Blue/Yellow/Red palette and rounded corner borders. It isn't that it is bad, but it often isn't fit for purpose.
You're right it wont change anything, but authors shouldn't be surprised when people who care about their time/attention comment on low/no effort pieces.
spongebobstoes 23 hours ago [-]
critique of writing style isn't made better by claiming it was authored by an LLM
SR2Z 23 hours ago [-]
No but it's a useful shorthand to describe a type of bad writing.
I also think that people should focus on substance and not if AI was used, but AI writes like shit and I find myself retching a bit when I have to read long AI-written documents. Do they say something useful? Maybe, but when my eyes are glazing over because it's just so exhausting trying to parse what's written, I can't tell.
I certainly think less of people when they have such poor taste that they think writing like that is acceptable.
wqaatwt 17 hours ago [-]
To be fair internet (or rather shovelware websites like Medium) were already flooded by crap articles based on a set of templates. Of course the issue is that LLMs are actually better than the robot-humans who used to write them so now it takes more time until you figure whether it’s worth reading or not..
jasonlotito 23 hours ago [-]
This is just an AI comment masquerading as not trying to prove a point.
SR2Z 22 hours ago [-]
Since you didn't address the substance, I guess we'll never know :)
aeon_ai 23 hours ago [-]
I’m as pro AI as any.
Slop is slop. When the “it works but isn’t great” phrases end up slipping into a strong conceptual core, it compromises the perception of the ideas.
Perhaps our AI will cater to us by rewriting the content we read, and each of us intermediate all communication with systems that make that slop bearable.
Or perhaps, we learn that we kinda still need to give a shit when writing to land on the perception we’re trying to create within our readers
culopatin 19 hours ago [-]
Maybe if instead we stopped engaging they would wonder why
justAnotherHero 1 days ago [-]
To me it's a useful signal not to read an article that someone didn't bother to write.
Which is a shame as real insights are buried inside some of these articles, which if the author bothered to write in his own words could have reached an audience that would have appreciated them.
Writing is one of the areas where I want no LLM involvement.
The number of things that make it to the top of HN/Reddit/wherever now that are devoid a human's touch is exhausting. Whether it's a site that's got that Claude frontend smell, or a repo that's got a burst of 10 claude commits before getting shared and abandoned, or a series of blog posts that were written by LLMs... it's all, at this point, a flag for me that the human behind the LLM doesn't really want to engage with others or share; in some ways it dehumanises their entire (supposed) audience.
IDK. Maybe having Claude contribute writing about something novel to the general blogosphere is useful in some dimension, but it usually gives me no confidence in the truth of the post.
CookieCrisp 21 hours ago [-]
More often it's the difference between finishing something and not finishing it - so often LLM's are helping them reach an audience that would appreciate them, even if that audience doesn't include you
gbalduzzi 17 hours ago [-]
I agree with the copy-pasted slop that you see sometimes, that is probably generated from a short prompt and therefore has no real substance to it.
If an article has interesting content (which comes from an human) and the LLM is just used to help the author finish off the article, I don't have any problem with that.
Labeling both scenarios in the same category feels completely wrong to it, as equating vibe coded stuff (as in no human ever read the produced code) and agent-assisted good old software engineering
spicyusername 1 days ago [-]
The problem is that the second you suspect something is written by AI, its a pretty good signal that 50-80% of the text is empty of meaning. Maybe that will change, but LLMs are terrible and inefficient writers.
Only so much time in the day, its a quick signal to not waste anymore of it.
lewistaariq 1 days ago [-]
Correct. AI == Credibility hit and it's increasing as more humans get used to feeling they are AI slop consumers, not worth the time for genuine human engagement. Human engagement costs are increasing. Amazing to read/watch.
afro88 23 hours ago [-]
It's not about figuring out if it's LLM written though. The style is hard to read and annoying. With the kind of sentences GP was talking about it's actually harder to get the substance.
1123581321 1 days ago [-]
It's both poor substance and style, in most cases (and this case.)
Pointing out they generated it at least encourages them to write a shorter article that says what they meant.
dandellion 23 hours ago [-]
Yes, as soon as models come out that can write properly, we'll all instantly get over it. Until then we'll be having this discussion over and over, as many times as it is necessary.
It means I don’t trust the substance. Whenever I try to use for technical writing like this, I catch it getting things wrong constantly.
jeremyjh 1 days ago [-]
Evaluating substance takes time - perhaps more than was invested in the article to begin with. So these tells are very distracting because as soon as I see them I wonder if the person who prompted the LLM even bothered to read the output. If they haven't, then I certainly shouldn't invest the time to determine if there is any substance.
illusive4080 22 hours ago [-]
No. As a human I like reading human written text over computer written text. I want something a human composed with thought put into it. Not something a human tried to save time with by having the machine write it.
conjectures 1 days ago [-]
What substance? That they consume a newer model from the same vendor?
rsalus 22 hours ago [-]
the frustration is largely because the overall substance is quite poor since it is typically imprecise by nature.
blitzar 16 hours ago [-]
Its never lupus and its always an LLM. No need for detective work.
mediaman 22 hours ago [-]
Because quality of writing matters.
Good communicators learn to use the written word. Bad ones rely on mental crutches.
Good communicators get an audience, and bad ones won't.
You think it's a lost cause, but it's not, because people don't like this junk, because it is low quality and, on average, lacks substance.
The best minds in AI that I've seen all write their own words. They use AI to help them research or ideate, but what they write is their own.
Before assuming this is a "lost cause," consider why the smartest people in the room don't do it.
xeyownt 17 hours ago [-]
100% this.
The AI police is there to say what is worth reading and what is not, because THEY know what people like.
Or not.
jchw 18 hours ago [-]
It's not detective work, it is literally blatantly obvious and impossible to ignore.
And no we can't get over it either. But I already have talked about that before and said roughly all I have to say on that front, so I'm just going to link back to my last comment regarding this.
But it's not about whether it was AI or human authored. That misses the point. We're all fatigued on the writing style. The same cadence and patterns; the same phrases and terms like "load-bearing". Used everywhere, they create a super fatiguing monoculture in all the writing. It's like if every illustration on the internet suddenly contained Garfield the cat.
Planktonne 23 hours ago [-]
The syntax tells you there isn't any substance.
jraph 1 days ago [-]
I have a counter proposition: don't fall for this constant suggestion that LLMs are an unavoidable future would you leave the techbros alone now pretty please, relentlessly keep reminding that we still don't think it's acceptable so people don't start to think this is okay since nobody complains anymore.
I appreciate these comments, they save me time for procrastinating elsewhere.
derwiki 23 hours ago [-]
I agree with this sentiment: it’s not inevitable if we relentlessly ostracize obviously LLM posts
And let’s be real: I had a post this year that was #1 on HN for a while, and an LLM “wrote” the whole thing, but it was very much my writing style and NO ONE called out the post as LLM slop. If you use an LLM correctly for writing, it’s not detectable. It seems that most folks don’t go through that effort.
asdff 23 hours ago [-]
The substance is shit too with these LLM articles. Stuck in the box of the training set. Nothing new. Just regurgitation.
multjoy 16 hours ago [-]
No.
gexla 18 hours ago [-]
My solution to this is to dump it into an LLM and a prompt that roughly does something like...
Hand wavy list just to get a general idea...
1) Give me a condensed summary
2) Is this adding anything to what we already have? (I save good articles along with annotations and whatever notes I may write to go along with it.)
3) Locate any upstream ideas on this (often AI articles are rehashing much better written ideas.)
...
Something like that. Not that I have some great system for it. I find these articles are so full of fluff that I have lost patience to attempt to get through them. So, I pull out the AI to parse the AI. I know that the AI may miss some hidden gems, but I'm okay with that.
TacticalCoder 1 days ago [-]
> The way the LLMs write (Claude perhaps?) With short phrases separated by colons, commas or full stops, is so poor and frustrating.
Yup llmish (from now on it's called "llmish") sucks.
But I'd say: at this point it's probably trivial to write a browser extension that detects llmish and that rewrites the worst sentences: from llmish to something less irritating to read. Heck, I could spent tokens on that: an extension that changes on the fly llmish found on webpages.
Also I'd say there's typically no swearing at all in llmish: llmish is too politically correct for swearing. So the rewrite could maybe also use a few "offending" words.
Offending words that, btw, are not going to go well with Gen Zers. Poor Gen Z... They've been raised with the state and its institutions (like school and then universities) hammering them with the notion that they were precious little unique snowflakes and now they arrive on the job market only to be told they've been pre-emptively replaced by AIs. And because they cannot stand a single curse word (because it's "offensive to minorities" or something), they'll be driven off by text rewritten to contain curse words. So they're condemned to read the bland, dumb, AI-generated llmish for the rest of their lives.
Honestly sucks for them. Fuck that.
weakfish 20 hours ago [-]
As a certified Gen Z member: respectfully, what the fuck are you on about?
The mythical Gen Z you are describing is the hyperbolic exaggeration that is equivalent to me describing all boomers as racist, all Gen X as entitled, all millennials as lazy, etc.
SR2Z 23 hours ago [-]
Have you ever met an actual Gen Z? They have no problem with swear words. Many of them love Key and Peele, whose humor is like 90% racist jokes.
If wokeness actually did capture a whole generation then why even bother complaining?
CarRamrod 1 days ago [-]
For me, it's Bottish
thiagoperes 1 days ago [-]
We run a lot of varied, tiny, simple workflows that were previously running on 5.4-nano and mini. We transitioned them to 5.6 and noticed exactly this range of improvement across the board. In a few cases, we had improvements in classification.
I think a lot of people miss that for many companies, a model upgrade like this is basically a one liner.
Even if you have an amazing model router architecture (which we do for our golden flows), it’s just not worth it. Not to mention reliability and so on
desterothx 19 hours ago [-]
the article is literally about the model upgrade not being a one liner
febed 19 hours ago [-]
What SDK are you using? Or is it custom?
madaxe_again 16 hours ago [-]
The first thing I used Sol for was to assess 5.6 on our workflows - previously, it was 5.5 for everything, as the quality on simpler models was just not good enough. We’re doing a mix of text and image analysis to extract explicit and implicit structured data from a steaming pile.
They do work pretty much as advertised. The bulk of our workload is now going via terra, which has cut our cost in half by itself, as well as improved response times by 50% - luna I am using as a backstop for opencv hits, and it is good enough, and so cheap as to almost be free - but very limited - and very fast. Sol only gave marginal improvements over terra for our workload.
I’ve also gotta say I’m impressed as to how well Sol ultra carried out the assessment itself - it made sound recommendations, and gave me a nice big dossier of “you should look at these outputs yourself and compare and consider” along with raw and digested data, and cpm for queries.
Anyway. Spent nothing beyond my pro sub, let Sol gnaw on it for a few hours, and my cost basis just dropped 50% and throughput improved by 100%. Win.
znnajdla 17 hours ago [-]
My experience mirrors this: services like OpenRouter that promise “failover” are pretty much useless except for sandbox testing because models in production are not really interchangeable. Any production harness doing serious agentic work in production is dependent on more model-specific quirks than you would expect. And even if another model works without errors, performance and efficiency is a whole different story. Even the system prompt can and should be tuned to a model’s preferred speaking style, for example <xml tags> for Claude-like models because they were trained on it, while other models do better with other delimiters. Think of the whole harness, prompt, and model as one system, not really with modular parts that can be swapped out if you care about optimal performance.
marcyb5st 16 hours ago [-]
I believe part of the LLMOps (I don't like the term, but it is what it is) should be building a failover plan with proper testing that check tools trajectories and such. If you have these then you can sort the good enough models from cheaper to more expensive and have the failover you mentioned.
I saw people bulding a mapping of model->{{prompts}, {tools descriptions}, ...}, but that, to me, it feels extreme. I believe it is the model that needs to adapt to your prompts after a certain point. Models that fail to do so won't get our api requests as they will be out of the failoever roster.
hamandcheese 16 hours ago [-]
OpenRouter doesn't fail over to a different model, it fails over a different provider of the same model.
blfr 1 days ago [-]
> Ploy’s agent builds and edits real marketing websites. It plans a page, reads the codebase, writes components, generates imagery, screenshots its own work, and decides when it’s done. That job description sets a very high bar for a model, and we test every frontier release against it. For the four months Opus held the default slot (first Opus 4.7, then 4.8), nothing we tested beat it.
Well, unlike OP I haven't run a rigorous test, but I still would expect Fable to be significantly better at building marketing websites than Opus. It sure is way better at building decks.
make_it_sure 16 hours ago [-]
gpt 5.6 is so much better ar design than fable
cute_boi 10 hours ago [-]
Can you please provide evidence. It shouldn't be hard to give side by side comparison. I have not found any task where gpt is better than fable.
greenavocado 1 days ago [-]
4.7 is very autistic in terms of following directions so I find OPs claims plausible
arikrahman 1 days ago [-]
Very descriptive there heh
aeonfox 1 days ago [-]
Game recognises game
SwtCyber 15 hours ago [-]
Its ironic that under an article with a ton of deep infrastructure insights half the comments are crying about the "forced writing style". What does it matter if claude helped the author clean up the text when inside is a ready-to-use blueprint on how to save 30% of the api budget and fix empty file reads?
jtrn 10 hours ago [-]
One of many reasons I would assume is that people just hate anything related to AI, so they latch on to anything negative they can say.
Another reason is that they mean what they say... That they really, really hate the style of writing, enough to fixate on that.
Personally, I think the people whining about the style are silly. Maybe because I'm terrible at grammar and spelling, but I always just focus on the message, not the delivery. I just care about the concept, facts, the argument, and so forth. The actual grammar and spelling are just trees, while the forest is the point.
Edit: just an infobit: The reason my text isn't full of errors is due to the awesomeness of the dictation and a custom hotkey I have created on my computer, which uses a local LLM to spellcheck any text I have selected and replaces it with the corrected one. Nothing has improved my quality of life and writing more than these two tools!
throwa356262 14 hours ago [-]
As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning.
Wait a moment, did they make the switch based on half a days of playing with Sol? Are these companies ran by teenagers?
brryant 10 hours ago [-]
hah - we actually skew staff, senior staff.
We have been testing GPT 5.6 for about a week as a preview model through a YC relationship, providing them feedback on the model. Our evals run in github CI and we can run them all in about 15 minutes against our eval bench of 115+ web design and marketing related jobs that ploy.ai specializes in.
then after we toggled it on (through a posthog feature flag) we actively monitored for failures.
I came from running Webflow, which powers > 1% of the internet so trying my best to relay all of that knowledge to ploy to power more % of the internet!
antileet 8 hours ago [-]
This is super impressive - both the pedigree of the team and the approach you took.
The funny thing is - when I first saw ploy, I didn't take it very seriously since so many of the signals that used to signify quality (decent design, copy, hard technical problems) are easy to fake. Plus the "grow while you sleep" space is crowded with weak players.
I wonder what the new markers of quality will be, which would separate the hand-crafted (to the extent possible) work v/s slop.
brryant 8 hours ago [-]
agree that many of those markers of quality are now low signal. Ultimately we let our customers vote with their wallet
Funnily enough, we spent a long time on our brand. From our launch video that has human actors, to our product details. I believe a distinctive, high quality, well implemented brand is still a hallmark of a strong product or service.
avianlyric 10 hours ago [-]
There’s every possibility they got some amount of early access to evaluate GPT5.6, precisely so they could write an article like this.
peab 9 hours ago [-]
at this point, it's pretty easy to create evals/benchmarks, and then run the latest model on them.
LLMs are so easy to swap out, so having good benchmarks/evals are pretty useful.
Even then, a lot of the time the model improvements are so obvious that you don't even need an eval.
fxwin 12 hours ago [-]
I would expect they have production based datasets they evaluate new models against.
dominotw 11 hours ago [-]
I would not expect that. They wouldnt have missed mentioning it if that was the case. Its mostly driven by vibes.
sekai 10 hours ago [-]
AI psychosis is still at all time high
lcampbell 19 hours ago [-]
> The fix that worked is a schema transform at the provider boundary. For OpenAI-family models only, we rewrite every optional property to be required but nullable, using anyOf: [T, null], which gives the model an explicit way to say “not using this.”
I admit, I've only used a bastardized form of MCP, but this smells... wrong? It's not clear to me why the Typescript type definitions would have any influence on (what I presume is) JSONSchema being sent from the agent to the inference backend as part of the completion request. The MCP specification (which the OpenAI backend might not use, I don't know) has an explicit field to signify "optional" parameters in the JSONSchema; my read on this is there's a bug somewhere between the Typescript layer(??) and the generated tool description which is actually sent to the inference backend.
It's possible the inference backend has changed from "generate valid tool responses" to "generate valid tool responses according to the JSON schema [where no parameters are optional]" but it's impossible to tell without seeing the actual requests sent to the inference backend (which I didn't see in TFA).
bluelightning2k 4 hours ago [-]
It's not typescript definitions. Presumably it's zod schemas which are both typescript types and JSONSchema for the object.
Although tbh the article makes a lot of this obvious and trivial change in syntax.
SwtCyber 15 hours ago [-]
The thing is this isn't a schema generation or Typescript bug at all. This is just how openai's function calling works under the hood. Their weights were fine-tuned for tool use to output the most complete data structures possible. If the model sees a parameter name in the system prompt context it will try to fill it with a value, even if it is not in the required array
avianlyric 10 hours ago [-]
If it isn’t a schema generation or typescript bug, then why did changing the API types they use result in different model behaviour?
dannyw 19 hours ago [-]
Modern frontier models, including Fable/Opus and 5.6, are often very loose with tool calls, and often don’t follow your schema precisely.
For example, see this post for Claude models hallucinating properties for an edit/replace tool call in Pi: https://lucumr.pocoo.org/about/
I suspect some part of this comes from the noticed intelligence degradation when you do constrained decoding. Yes, you’re guaranteed schema validation, but you lose a lot of intelligence. It’s fine if you just want a classifier, a summary, a prompt enhancement, etc; but I’d be careful in agentic loops.
Harnesses like Claude Code do a lot of preprocessing, repairing, cleaning, etc; as the blog post shows. You usually don’t see it.
In practice it’s easier and better to just make your harness “looser” and work better with the model (they’re coming out every month or two anyway, each with their own idiosyncrasies) than to assume and force perfect correctness.
Most agent harnesses that have not been designed from scratch for current models are over-engineered.
Chances are whatever was needed to make earlier models perform well now either is no longer helping much or actively hurts performance (worse results, slower, uses more tokens …).
bluelightning2k 4 hours ago [-]
Agreed!
Example: for large Eloqua/Marketo/HubSpot emails we would previously make a planner which delegated the sections to their own call.
GPT5.6 can do the whole thing. The planner is unhelpful.
My suggestion: feature flag your complex implementations so you can rapidly contrast with and without it. (Or a formal eval suite if you have one).
If you prefer the simpler path, delete the old path.
Note: the challenge is making things compatible with these tools. Obviously generating html directly has been simple for ages.
(Source: mopsy.ai)
arikrahman 1 days ago [-]
Migrating my workflow to Reasonix with cache hits on Deepseek make requests practically free, and that's on unsubsidized American providers.
gunapologist99 22 hours ago [-]
Sorry, what did that have to do with the article?
bel8 22 hours ago [-]
They also migrated and that also made the workflow cheaper.
It has everything to do with the article.
gunapologist99 5 hours ago [-]
Reasonix, the harness, only works with DeepSeek.
Neither are mentioned in the article at all, which was about a migration from Claude Opus to GPT 5.6.
Maybe a bit of DeepSeek astroturfing going on?
arikrahman 3 hours ago [-]
I wish people were astroturfing it
desterothx 19 hours ago [-]
What's your config, how does it compare to pi
arikrahman 9 minutes ago [-]
Here's my config for agents: https://codeberg.org/arik/agents It's a lot better since it's optimized for the model and cache hits, unlike other frontends that try to be more general.
bob1029 1 days ago [-]
> we’ve made GPT 5.6 Sol the default model powering every Ploy workspace
I would consider Luna for parts of the workload that touch actual tools. It is surprisingly capable and it runs fast.
Sol is great at talking to the human and orchestration of agent calls, but it's just too expensive to use everywhere.
You can get 5 Luna runs for the cost of 1 Sol run. Statistically speaking, going from one to five samples is a pretty big deal.
auspiv 21 hours ago [-]
Statistically speaking if each part of the Luna run has a 90% chance of being correct, 5 of those is 0.9^5 = 0.59 = 59%. Or one Sol run being maybe 95% correct? Exact numbers vary of course. But then again having sol verify at end may be cheaper.
bob1029 16 hours ago [-]
The goal is not 100% correctness. The goal is to demonstrate the current amount of variance / uncertainty to the planning agent.
If the planner sees that 4/5 Luna runs resulted in approximately the same summary, it may conclude that variance is low and that it is over the target. If all Luna runs are different, the planner can conclude that additional research rounds are required.
Tadpole9181 24 hours ago [-]
The problem I always run into with subagents is that they are isolated. This is a double-edged sword, as it keeps context down and lets them "focus", but it often means they must do their own research to continue to do work given to them, which eats uncached tokens.
So depending on how heavily agents are used on what tasks, it's entirely possible that you get worse work for more cost.
bob1029 23 hours ago [-]
> they are isolated
This is a feature if your goal is to obtain many samples. Independence is critical. This makes it easier to accurately model the uncertainty of a decision.
taspeotis 23 hours ago [-]
I feel Claude Code has added (and removed?) a feature that forks a subagent from the parent context, so it’s still isolated but it’s more of a continuation of what you were doing in one narrow direction and then it dies. Rather than a blank slate with a prompt of what to do.
yusufnb 7 hours ago [-]
What does a site build cost, in real $$ with Opus vs Sol? Could you share a ballpark?
agumonkey 9 hours ago [-]
is it common to measure LLMs with landing pages like this ? it seems a bit too simple
redfather918 22 hours ago [-]
The cost reduction is impressive, but I think consistency matters even more for production agents. I'd be interested to know whether prompt engineering or tool-calling workflows had to change significantly.
narmiouh 20 hours ago [-]
He actually talks about that in the article. Including how they had to rearchitect tool calling using nulls as well as limitations of prompt caching
desktopentree 20 hours ago [-]
I catch a lot of issues on the technical writing side.
brcmthrowaway 7 hours ago [-]
AI wrapper companies still exist?
bluelightning2k 4 hours ago [-]
Must admit, for this particular case I don't see the appeal in using a wrapper.
Why would I not just use Codex directly?
The we wrote a bunch of prompts argument is kind of meh. That sort of thing has not only diminishing value with subsequent model releases but I actually believe will turn negative. The model will know better by default.
For example, initially giving the model some advice on code best practice was helpful. But now it's unhelpful because the model already knows best.
htlemur_bobby 19 hours ago [-]
I found Claude to be better for the first prototype. It was more likely to come up with something fast. But it kept lying and claiming it did world class work and it was just hardcoding response by the end. I found GPT never lied to me.
ianberdin 19 hours ago [-]
We at Playcode.io - a company similar to Ploy are still using Opus 4.6. "Why?" you might ask.
Because GPT 5.6 Sol, while fast and pleasant to use, is essentially the same model as 5.5 wrapped in new marketing packaging, just to avoid losing ground to Anthropic. In practice, it's the same quality: it generates the same garbage, tons of code, and can never solve even a single complex task. We simply don't trust it to write code for clients that they'll end up throwing away anyway.
"Then why not Opus 4.8?" you might ask.
Well, because Opus 4.8 and 4.7 are just another lie, a price hike with no actual quality improvement.
That's why at Playcode, we give our clients the best possible quality/price - which is Opus 4.6. Regardless of what people write in articles like this.
Did you just paste your marketing copy into a hn comment?
ianberdin 5 hours ago [-]
I have already developed a habit of speaking in this format over 10 years of supporting my product.
ianberdin 19 hours ago [-]
Everyone probably has the same question: what about Fable? Fable 5 is sick.
It is simply the best model in the world out of everything we have ever tried. It's absolutely fantastic. It solves almost any task from start to finish, the way it should be done — no errors, perfect code. It's a miracle.
If there's any way to make it a little more affordable, that would be incredible.
As for GPT-5.6 Sol — it doesn't even come close. I honestly don't understand why people even try to compare them. It feels like Sam's attempt to hold onto his audience with those endless daily limit resets. A clever trick, nothing more.
porker 18 hours ago [-]
> Fable 5 is sick. [It] solves almost any task from start to finish, the way it should be done — no errors, perfect code. It's a miracle.
> As for GPT-5.6 Sol — it doesn't even come close. I honestly don't understand why people even try to compare them.
What kind of problems are you working on? I like Fable but when planning work on a complex C codebase it's making more mistakes than 5.6 Sol xhigh for me.
In what scenarios is Fable giving you "no errors, perfect code"?
ianberdin 5 hours ago [-]
I have a large monorepo that includes about 15 TypeScript services and many Rust services. Everything is well-documented and organized, with standardized and structured custom code.
When an issue arises, I often test the systems by providing a minimal prompt, like: "this user, this is their email, this isn't working, figure it out in production." I send this to both Opus and ChatGPT, but it doesn't help. I've set up Agents.md and Quote.md identically, with the same access and linkers, so the Harness is consistent.
ChatGPT rarely succeeds. If the task is complex and requires a multi-step process to identify the true cause, ChatGPT usually stops after a few initial ideas and wrongly claims it has found the solution.
- For simple tasks, like identifying a missing item in a to-do list, ChatGPT performs well.
- However, for issues like memory leaks or file system corruption, it struggles.
On the other hand, Opus 4.8 always finds the solution, albeit slowly. I can rely on it without worrying about whether it will succeed. It just gets the job done.
Recently, Fable 5 has emerged, which resolves issues without needing any prompts. It operates even faster than Opus.
When I ask ChatGPT or Opus to create a new feature:
- ChatGPT often produces superficial results, ignoring existing code and building unnecessary independent code.
- Interestingly, the outcome from ChatGPT appears functional, but it's usually incorrect, focusing on a superficial "aha!" moment.
Opus, however, plans thoroughly, executes, and cleans up, ensuring everything works correctly. If needed, I can provide more realistic examples, though it's challenging due to the monorepo's size and complexity, with hundreds of thousands of lines of code.
yruzin 4 hours ago [-]
Claude is Extremely slow, especially fast few days. Codex is so much faster in in my opinion with comparable quality. Anthropic is going to fix their problems, but for me as a user that depends for the service to work, it's not acceptable.
ChatGPT 5.5 is already a better version than Opus 4.8, at lest in my experience. Firstly, thing get done today and I don't see quality to be worse. I has many situation where Opus was just going in circles, finding more and more issues with the code it wrote a few hours back, to me this is counterproductive.
mattclarkdotnet 17 hours ago [-]
For PCB design 5.6 Sol/Terra is streets ahead of 5.5, and uses fewer tokens, so I'm not sure it can really be the same model.
jdw64 16 hours ago [-]
Personally, could you share the code sometime later? The GPT code looks decent to me. Or even just the prompt would be fine.
implexa_founder 23 hours ago [-]
[flagged]
modgate 20 hours ago [-]
[flagged]
1 days ago [-]
langs 20 hours ago [-]
[dead]
rjnz199 20 hours ago [-]
[dead]
przemarzec 1 days ago [-]
[flagged]
jing09928 22 hours ago [-]
[flagged]
luciana1u 1 days ago [-]
[dead]
CurbStomper 1 days ago [-]
[dead]
hankbond 1 days ago [-]
[flagged]
estebarb 1 days ago [-]
But what users prefer? Given this is for marketing, which results produce more conversions? From the examples shown, personally I strongly preferred Claude Opus in all cases.
Rendered at 23:11:03 GMT+0000 (Coordinated Universal Time) with Vercel.
Such a silly choice of words. I wish the human directing the LLM writing the article put some effort into rewriting the worst examples of LLM style.
> But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work.
The way the LLMs write (Claude perhaps?) With short phrases separated by colons, commas or full stops, is so poor and frustrating.
There some good insights behind this article, so it's worth reading, for example below, but it isn't easy to read.
> Earlier GPT models cached implicitly on partial prefix matches, which gave decent hit rates for free. GPT-5.6 dropped partial-prefix matching:
This is exactly why I keep a WRITING.md file alongside AGENTS.md or CLAUDE.md.
Most people spend time telling the model what to build, but very little time telling it how to write. LLMs are surprisingly good at following explicit style guidance if you bother to give it to them.
Mine includes conventions like avoiding unnecessary colons, em dashes, and sentence fragments masquerading as emphasis. Basically, AI-isms and any grammatical errors I tend to make. It also points to writers whose technical prose I admire. Brian Kernighan and Rob Pike are great examples: clear, conversational, and readable without trying to sound important. I've always tried to do two things with documentation: 1) Make it readable and 2) Make people want to read it. A WRITING.md file helps with both points.
If you're generating documentation regularly, it's worth having your agents reference a WRITING.md file. The improvement in readability is often much larger than any gain you'll get from switching to the latest model, and it can keep your documentation consistent between model switches.
I disagree with the person below - I don't think its a wrongheaded goal in and of itself to get better ai generated writing. I read ai output half of my day, I may as well try and make it better for my own sake. Passing that onto other people is an entirely unrelated field of problems, and passing it off as stuff you authored yourself is just plagarism.
https://f.mek.cc/gombasic/file?name=WRITING.md&ci=tip
And in many cases, like you, I spend a good amount of time reading AI documentation. I'm not "stealing" anything, just having a model generate words in a specific way. I'm just being precious in how I want things, without having to do it myself.
https://github.com/blader/humanizer
I'm not trying to hide the problem, just offering a solution that people may find useful, or curiously odd.
It makes me feel like either
1) you don't use the models enough to know how they write
2) you're not self aware enough to know it matters
3) you're oblivious to the situation overall
4) you don't respect your readers
There's no good scenario.
I would have downvoted the sarcasm (it doesn’t contribute to the conversation), but I believe it actually is the author’s opinion.
This way I can at least somewhat control the style of the output.
I've found the same thing showing with Claude-coded/designed front ends that overuse the same semi-monospaced fonts, Blue/Yellow/Red palette and rounded corner borders. It isn't that it is bad, but it often isn't fit for purpose.
You're right it wont change anything, but authors shouldn't be surprised when people who care about their time/attention comment on low/no effort pieces.
I also think that people should focus on substance and not if AI was used, but AI writes like shit and I find myself retching a bit when I have to read long AI-written documents. Do they say something useful? Maybe, but when my eyes are glazing over because it's just so exhausting trying to parse what's written, I can't tell.
I certainly think less of people when they have such poor taste that they think writing like that is acceptable.
Slop is slop. When the “it works but isn’t great” phrases end up slipping into a strong conceptual core, it compromises the perception of the ideas.
Perhaps our AI will cater to us by rewriting the content we read, and each of us intermediate all communication with systems that make that slop bearable.
Or perhaps, we learn that we kinda still need to give a shit when writing to land on the perception we’re trying to create within our readers
Which is a shame as real insights are buried inside some of these articles, which if the author bothered to write in his own words could have reached an audience that would have appreciated them.
Writing is one of the areas where I want no LLM involvement.
The number of things that make it to the top of HN/Reddit/wherever now that are devoid a human's touch is exhausting. Whether it's a site that's got that Claude frontend smell, or a repo that's got a burst of 10 claude commits before getting shared and abandoned, or a series of blog posts that were written by LLMs... it's all, at this point, a flag for me that the human behind the LLM doesn't really want to engage with others or share; in some ways it dehumanises their entire (supposed) audience.
IDK. Maybe having Claude contribute writing about something novel to the general blogosphere is useful in some dimension, but it usually gives me no confidence in the truth of the post.
If an article has interesting content (which comes from an human) and the LLM is just used to help the author finish off the article, I don't have any problem with that.
Labeling both scenarios in the same category feels completely wrong to it, as equating vibe coded stuff (as in no human ever read the produced code) and agent-assisted good old software engineering
Only so much time in the day, its a quick signal to not waste anymore of it.
Pointing out they generated it at least encourages them to write a shorter article that says what they meant.
Good communicators learn to use the written word. Bad ones rely on mental crutches.
Good communicators get an audience, and bad ones won't.
You think it's a lost cause, but it's not, because people don't like this junk, because it is low quality and, on average, lacks substance.
The best minds in AI that I've seen all write their own words. They use AI to help them research or ideate, but what they write is their own.
Before assuming this is a "lost cause," consider why the smartest people in the room don't do it.
The AI police is there to say what is worth reading and what is not, because THEY know what people like.
Or not.
And no we can't get over it either. But I already have talked about that before and said roughly all I have to say on that front, so I'm just going to link back to my last comment regarding this.
https://news.ycombinator.com/item?id=48861849
I appreciate these comments, they save me time for procrastinating elsewhere.
And let’s be real: I had a post this year that was #1 on HN for a while, and an LLM “wrote” the whole thing, but it was very much my writing style and NO ONE called out the post as LLM slop. If you use an LLM correctly for writing, it’s not detectable. It seems that most folks don’t go through that effort.
Hand wavy list just to get a general idea...
1) Give me a condensed summary 2) Is this adding anything to what we already have? (I save good articles along with annotations and whatever notes I may write to go along with it.) 3) Locate any upstream ideas on this (often AI articles are rehashing much better written ideas.) ...
Something like that. Not that I have some great system for it. I find these articles are so full of fluff that I have lost patience to attempt to get through them. So, I pull out the AI to parse the AI. I know that the AI may miss some hidden gems, but I'm okay with that.
Yup llmish (from now on it's called "llmish") sucks.
But I'd say: at this point it's probably trivial to write a browser extension that detects llmish and that rewrites the worst sentences: from llmish to something less irritating to read. Heck, I could spent tokens on that: an extension that changes on the fly llmish found on webpages.
Also I'd say there's typically no swearing at all in llmish: llmish is too politically correct for swearing. So the rewrite could maybe also use a few "offending" words.
Offending words that, btw, are not going to go well with Gen Zers. Poor Gen Z... They've been raised with the state and its institutions (like school and then universities) hammering them with the notion that they were precious little unique snowflakes and now they arrive on the job market only to be told they've been pre-emptively replaced by AIs. And because they cannot stand a single curse word (because it's "offensive to minorities" or something), they'll be driven off by text rewritten to contain curse words. So they're condemned to read the bland, dumb, AI-generated llmish for the rest of their lives.
Honestly sucks for them. Fuck that.
The mythical Gen Z you are describing is the hyperbolic exaggeration that is equivalent to me describing all boomers as racist, all Gen X as entitled, all millennials as lazy, etc.
If wokeness actually did capture a whole generation then why even bother complaining?
I think a lot of people miss that for many companies, a model upgrade like this is basically a one liner.
Even if you have an amazing model router architecture (which we do for our golden flows), it’s just not worth it. Not to mention reliability and so on
They do work pretty much as advertised. The bulk of our workload is now going via terra, which has cut our cost in half by itself, as well as improved response times by 50% - luna I am using as a backstop for opencv hits, and it is good enough, and so cheap as to almost be free - but very limited - and very fast. Sol only gave marginal improvements over terra for our workload.
I’ve also gotta say I’m impressed as to how well Sol ultra carried out the assessment itself - it made sound recommendations, and gave me a nice big dossier of “you should look at these outputs yourself and compare and consider” along with raw and digested data, and cpm for queries.
Anyway. Spent nothing beyond my pro sub, let Sol gnaw on it for a few hours, and my cost basis just dropped 50% and throughput improved by 100%. Win.
I saw people bulding a mapping of model->{{prompts}, {tools descriptions}, ...}, but that, to me, it feels extreme. I believe it is the model that needs to adapt to your prompts after a certain point. Models that fail to do so won't get our api requests as they will be out of the failoever roster.
Well, unlike OP I haven't run a rigorous test, but I still would expect Fable to be significantly better at building marketing websites than Opus. It sure is way better at building decks.
Another reason is that they mean what they say... That they really, really hate the style of writing, enough to fixate on that.
Personally, I think the people whining about the style are silly. Maybe because I'm terrible at grammar and spelling, but I always just focus on the message, not the delivery. I just care about the concept, facts, the argument, and so forth. The actual grammar and spelling are just trees, while the forest is the point.
Edit: just an infobit: The reason my text isn't full of errors is due to the awesomeness of the dictation and a custom hotkey I have created on my computer, which uses a local LLM to spellcheck any text I have selected and replaces it with the corrected one. Nothing has improved my quality of life and writing more than these two tools!
We have been testing GPT 5.6 for about a week as a preview model through a YC relationship, providing them feedback on the model. Our evals run in github CI and we can run them all in about 15 minutes against our eval bench of 115+ web design and marketing related jobs that ploy.ai specializes in.
then after we toggled it on (through a posthog feature flag) we actively monitored for failures.
I came from running Webflow, which powers > 1% of the internet so trying my best to relay all of that knowledge to ploy to power more % of the internet!
The funny thing is - when I first saw ploy, I didn't take it very seriously since so many of the signals that used to signify quality (decent design, copy, hard technical problems) are easy to fake. Plus the "grow while you sleep" space is crowded with weak players.
I wonder what the new markers of quality will be, which would separate the hand-crafted (to the extent possible) work v/s slop.
Funnily enough, we spent a long time on our brand. From our launch video that has human actors, to our product details. I believe a distinctive, high quality, well implemented brand is still a hallmark of a strong product or service.
LLMs are so easy to swap out, so having good benchmarks/evals are pretty useful.
Even then, a lot of the time the model improvements are so obvious that you don't even need an eval.
I admit, I've only used a bastardized form of MCP, but this smells... wrong? It's not clear to me why the Typescript type definitions would have any influence on (what I presume is) JSONSchema being sent from the agent to the inference backend as part of the completion request. The MCP specification (which the OpenAI backend might not use, I don't know) has an explicit field to signify "optional" parameters in the JSONSchema; my read on this is there's a bug somewhere between the Typescript layer(??) and the generated tool description which is actually sent to the inference backend.
It's possible the inference backend has changed from "generate valid tool responses" to "generate valid tool responses according to the JSON schema [where no parameters are optional]" but it's impossible to tell without seeing the actual requests sent to the inference backend (which I didn't see in TFA).
Although tbh the article makes a lot of this obvious and trivial change in syntax.
For example, see this post for Claude models hallucinating properties for an edit/replace tool call in Pi: https://lucumr.pocoo.org/about/
I suspect some part of this comes from the noticed intelligence degradation when you do constrained decoding. Yes, you’re guaranteed schema validation, but you lose a lot of intelligence. It’s fine if you just want a classifier, a summary, a prompt enhancement, etc; but I’d be careful in agentic loops.
Harnesses like Claude Code do a lot of preprocessing, repairing, cleaning, etc; as the blog post shows. You usually don’t see it.
In practice it’s easier and better to just make your harness “looser” and work better with the model (they’re coming out every month or two anyway, each with their own idiosyncrasies) than to assume and force perfect correctness.
Welcome to vibe applied AI ;)
Chances are whatever was needed to make earlier models perform well now either is no longer helping much or actively hurts performance (worse results, slower, uses more tokens …).
Example: for large Eloqua/Marketo/HubSpot emails we would previously make a planner which delegated the sections to their own call.
GPT5.6 can do the whole thing. The planner is unhelpful.
My suggestion: feature flag your complex implementations so you can rapidly contrast with and without it. (Or a formal eval suite if you have one).
If you prefer the simpler path, delete the old path.
Note: the challenge is making things compatible with these tools. Obviously generating html directly has been simple for ages.
(Source: mopsy.ai)
It has everything to do with the article.
Neither are mentioned in the article at all, which was about a migration from Claude Opus to GPT 5.6.
Maybe a bit of DeepSeek astroturfing going on?
I would consider Luna for parts of the workload that touch actual tools. It is surprisingly capable and it runs fast.
Sol is great at talking to the human and orchestration of agent calls, but it's just too expensive to use everywhere.
You can get 5 Luna runs for the cost of 1 Sol run. Statistically speaking, going from one to five samples is a pretty big deal.
If the planner sees that 4/5 Luna runs resulted in approximately the same summary, it may conclude that variance is low and that it is over the target. If all Luna runs are different, the planner can conclude that additional research rounds are required.
So depending on how heavily agents are used on what tasks, it's entirely possible that you get worse work for more cost.
This is a feature if your goal is to obtain many samples. Independence is critical. This makes it easier to accurately model the uncertainty of a decision.
Why would I not just use Codex directly?
The we wrote a bunch of prompts argument is kind of meh. That sort of thing has not only diminishing value with subsequent model releases but I actually believe will turn negative. The model will know better by default.
For example, initially giving the model some advice on code best practice was helpful. But now it's unhelpful because the model already knows best.
Because GPT 5.6 Sol, while fast and pleasant to use, is essentially the same model as 5.5 wrapped in new marketing packaging, just to avoid losing ground to Anthropic. In practice, it's the same quality: it generates the same garbage, tons of code, and can never solve even a single complex task. We simply don't trust it to write code for clients that they'll end up throwing away anyway.
"Then why not Opus 4.8?" you might ask.
Well, because Opus 4.8 and 4.7 are just another lie, a price hike with no actual quality improvement.
That's why at Playcode, we give our clients the best possible quality/price - which is Opus 4.6. Regardless of what people write in articles like this.
It is simply the best model in the world out of everything we have ever tried. It's absolutely fantastic. It solves almost any task from start to finish, the way it should be done — no errors, perfect code. It's a miracle.
If there's any way to make it a little more affordable, that would be incredible.
As for GPT-5.6 Sol — it doesn't even come close. I honestly don't understand why people even try to compare them. It feels like Sam's attempt to hold onto his audience with those endless daily limit resets. A clever trick, nothing more.
> As for GPT-5.6 Sol — it doesn't even come close. I honestly don't understand why people even try to compare them.
What kind of problems are you working on? I like Fable but when planning work on a complex C codebase it's making more mistakes than 5.6 Sol xhigh for me.
In what scenarios is Fable giving you "no errors, perfect code"?
When an issue arises, I often test the systems by providing a minimal prompt, like: "this user, this is their email, this isn't working, figure it out in production." I send this to both Opus and ChatGPT, but it doesn't help. I've set up Agents.md and Quote.md identically, with the same access and linkers, so the Harness is consistent.
ChatGPT rarely succeeds. If the task is complex and requires a multi-step process to identify the true cause, ChatGPT usually stops after a few initial ideas and wrongly claims it has found the solution.
- For simple tasks, like identifying a missing item in a to-do list, ChatGPT performs well. - However, for issues like memory leaks or file system corruption, it struggles.
On the other hand, Opus 4.8 always finds the solution, albeit slowly. I can rely on it without worrying about whether it will succeed. It just gets the job done.
Recently, Fable 5 has emerged, which resolves issues without needing any prompts. It operates even faster than Opus.
When I ask ChatGPT or Opus to create a new feature: - ChatGPT often produces superficial results, ignoring existing code and building unnecessary independent code. - Interestingly, the outcome from ChatGPT appears functional, but it's usually incorrect, focusing on a superficial "aha!" moment.
Opus, however, plans thoroughly, executes, and cleans up, ensuring everything works correctly. If needed, I can provide more realistic examples, though it's challenging due to the monorepo's size and complexity, with hundreds of thousands of lines of code.