I liked the commentary and the metaphor with the Nyquist theorem. Mathematically speaking it's invalid but the problem being discussed is real - when the velocity of code generation (accelerated by LLMs) outpaces human cognitive capacity to review, test, and mentally model the system, structural coherence collapses.
And we're already there, it's beyond our cognitive capacity to fully understand the (AI) produced volume of work for given unit of time.
In order to really mitigate the human bottleneck, we will either have to reach the AGI, which might not come in the next 10 years or so, or change the form of how we're doing work, review included.
wrs 4 days ago [-]
Yes, the analogy is an interesting and useful framing of the problem. I mean, it should be on the analogy shelf next to the blind people observing the elephant, etc.
But also yes, the whole time I read the article, I was really hoping the 2X part wouldn’t be presented literally, because that makes no sense.
sneezychl 22 hours ago [-]
If anything I think software will become more stable, secure, and efficient as it's re-written by AI models.
I passed the torch over to Opus 5.5 once I realized it makes less mistakes than me and that 99% of problems were my own. I no longer recognize the codebase I spent years developing and maintaining, and yet the statistics show lower crash rates and fewer unhandled exceptions.
underlipton 22 hours ago [-]
I think it goes beyond development issues. Think of how quickly websites change their contents and even layouts. The internet - cyberspace - was initially conceived of by many as a separate "place", and even in its abstract nature, I think most people still do. So, think of how disoriented people are when they visit their hometown for the first time in a few years, after a lot of construction has taken place. Or when they go to a grocery store and products have been moved around. A micro version of this happens basically every time you open Youtube or even HN. I sometimes wonder if platform owners actually count on it; such a user has to be more suggestible, since a portion of their cognition has to be shunted over to just understanding where they "are."
TimTheTinker 4 days ago [-]
It's an interesting metaphor, but the math is not directly applicable to software engineering, because the shape of the problem (not to mention the terms) doesn't translate.
The main problem is that shannon-nyquist is concerned with discrete representation of a continuous/analog signal. But TA is applying it backwards: software is discrete and human understanding is analog. So human "sampling" of software is not something an A->D theory can speak to.
wjnc 10 hours ago [-]
I read this mostly as a metaphor pointing at social systems and dynamics. You cannot have /informed/ decision making below a certain level of polling. What do humans do when /informed/ decision making is off the table? They posture, predict, brag and brawl. I’ll go sociological and historical for a second: a thing like ordered religion is an institutional solution for stability when the (scientific) polling rate is too high.
That’s what we see in corporations. The YOLO one-shot bros have taken over Wall Street and all firms are expected to follow in the talk of the end of humans in corporate processes. OP gives a nice hint about why we are back to pre-scientific times in corporations.
throwawayqqq11 4 hours ago [-]
AI, us humans or any "thinking" subject is drumroll subjective. Unless you are an all-knowing god, there is no objectivity. This means that AI/humans have to be heuristical in their approach to tackle complex reality and you can see traces of this everywhere, be it software or eg. messy tax law or regulations.
IMO, "scientific times" are a noble ideal as a guideline to circle back and question previous assumptions (falsification) but we have never really been in such a time. But you are right, the finance/corporate bros are further away from it then ever.
I dont know if AI can help us rectifying past mistakes or whether we humans reject deep systemic criticism from it and call it woke AI.
AnimalMuppet 22 hours ago [-]
I'm not sure that's right. The "changes in tech" can be regarded as almost continuous (in the same way as a gas is made up of discrete particles, but can still be treated as a continuous medium because there are so many particles).
And human understanding of tech is discrete - nobody understands "software", say. They understand a number of individual features of individual languages, and some of the ways those features work together.
That doesn't necessarily mean that it's a good analogy. But I can kind of see it in, say, the new versions of C++. Many people can't or don't learn all the implications of all the changes before the next version comes out.
mikrl 22 hours ago [-]
> The "changes in tech" can be regarded as almost continuous
Wouldn’t this require a scaling constant analogous to the Boltzmann constant or ideal gas constant to be true?
I doubt there are 10^23 lines of asm in all the binaries ever compiled, let alone larger “tech changes”
yfw 4 days ago [-]
Seems like a forced analogy. Is the signal a periodic and regular frequency?
If its a straight line you need 2 samples.
If its not you probably need more.
Id argue codebase understanding is neither of these simplifications
esafak 4 days ago [-]
And the sampling theorem only applies when you only know that the signal is band-limited. If you know more than that, you can go below the Nyquist rate.
waffletower 4 days ago [-]
I love aliasing in digital synths -- embrace the foldover.
somethingsome 4 days ago [-]
Funnily enough, there are techniques that allows to go under the Nyquist limit, eg compressive sensing.
I don't like the analogy, it supposed that AI generated code will always be badly organized underneath if it is automatic, however, as time goes, I find the code better and better..
4star3star 4 days ago [-]
Agreed that it is an analogy, not necessarily a law of nature. However, my takeaway had nothing to do with the quality or organization of the generated code. The "signal" fidelity, so to speak, is the human understanding of the system, which I presume we still value since we would like human evaluation and decision making to drive the direction of change. If we're not making decisions from a foundation of understanding, we may as well include some dice rolls and ouija board advice.
ironmanszombie 4 days ago [-]
Thank you for the article.
However, I wholeheartedly disagree. AI available to the general public may produce both code and fixes to code as an endless accretion of bugs but that's not necessarily how it will always be. Human analytics with regard to any code within a system is essentially a result of accumulated (analytics) knowledge toolified as checks and constraints. What stops, say a military application of AI from having those analytics present during code generation?
It would be really interesting to see an AI create a new kernel for Linux that every piece of it's code was written and verified-secure by having it analyzed and exploited simultaneously.
We, humans, don't need to be as fast as our tools.
On a personal note, I can't wait to see new silicone and operating systems made specifically for AI. That will be cool!
frangonf 4 days ago [-]
I personally found hard to see this analogy at first, because the assumption of human understanding as fs seems hard to make, as I don't see human understanding as linear and measurable.
Also if fs mapped well, we could just increase fs to be able to reconstruct the signal better by interleaving more brains at it and we know of manmonthian consequences, and this kinda what the main point of the post is, that human comprehension don't scale. In fact, to me, some sort of digital inteligence seems more fs-shaped than a human one.
But overall cool read to make you think, where you can see a dev brain as an ADC, design paradigms or microservices as band pass filters, apis as low pass, frameworks as downsampling, legacy code as DC signal that you don't touch, etc.
FLeXMurphy 4 days ago [-]
You change what the signal is. Simple as.
vr46 4 days ago [-]
The amount of app updates - I don't have auto update turned on - is now completely ridiculous.
LLMs plus continuous delivery - what a gruesome twosome.
augment_me 23 hours ago [-]
Fine comparison but you assume that the ends are understanding and comprehension.
In business and in tech, the ends are often monetary. It doesn't matter if you release bugs into prod as long as the majority of your paying users do not complain.
At my corp, bugs in prod outside of shadow deployments are up 40% year-to-year since 2023, yet users and ad revenue is growing like never before also, so it doesn't matter if we ship garbage code.
Scramblejams 4 days ago [-]
It's like you've ended up inside your own OODA loop.
smitty1e 4 days ago [-]
DOODOO
science4sail 4 days ago [-]
The author assumes that human understanding and modeling is essential. However, this no longer appears to be the expressed preference of the industry.
AI advocates nowadays are calling for abolishing human code review. So code moves from read-often-write-rarely to read-once-write-once to read-never-write-often. The assumption is that we can rely on the agents to decipher the code if needed.
My feeling is that software "engineering" in the future will be basically a cargo cult or the Adeptus Mechanicus from Warhammer 40k: all detailed technical knowledge comes from the gods, humans only manage the rituals around technology.
nemo44x 4 days ago [-]
He talks about the problem with this empirically.
I have to say I mainly agree in a way I wouldn’t have 1 month ago. We are now starting to see bugs creep above our baseline. This is only what we are detecting. At least in certain critical areas we are not removing strict human review.
One theory is our codebase (which is large and mature) was very coherent. As we moved to more autonomy and needed generated code in the base has grown and is less mature now and likely not as coherent. The LLM has to cope with this.
amelius 22 hours ago [-]
Meh, with compression you can often reconstruct the signal with far fewer samples.
Let's not use analogies from the distant past, they will only hold us back.
cozzyd 22 hours ago [-]
AI bros work in the second or third Nyquist zone.
kould2333 3 days ago [-]
[dead]
Rendered at 16:47:01 GMT+0000 (Coordinated Universal Time) with Vercel.
And we're already there, it's beyond our cognitive capacity to fully understand the (AI) produced volume of work for given unit of time.
In order to really mitigate the human bottleneck, we will either have to reach the AGI, which might not come in the next 10 years or so, or change the form of how we're doing work, review included.
But also yes, the whole time I read the article, I was really hoping the 2X part wouldn’t be presented literally, because that makes no sense.
I passed the torch over to Opus 5.5 once I realized it makes less mistakes than me and that 99% of problems were my own. I no longer recognize the codebase I spent years developing and maintaining, and yet the statistics show lower crash rates and fewer unhandled exceptions.
The main problem is that shannon-nyquist is concerned with discrete representation of a continuous/analog signal. But TA is applying it backwards: software is discrete and human understanding is analog. So human "sampling" of software is not something an A->D theory can speak to.
That’s what we see in corporations. The YOLO one-shot bros have taken over Wall Street and all firms are expected to follow in the talk of the end of humans in corporate processes. OP gives a nice hint about why we are back to pre-scientific times in corporations.
IMO, "scientific times" are a noble ideal as a guideline to circle back and question previous assumptions (falsification) but we have never really been in such a time. But you are right, the finance/corporate bros are further away from it then ever.
I dont know if AI can help us rectifying past mistakes or whether we humans reject deep systemic criticism from it and call it woke AI.
And human understanding of tech is discrete - nobody understands "software", say. They understand a number of individual features of individual languages, and some of the ways those features work together.
That doesn't necessarily mean that it's a good analogy. But I can kind of see it in, say, the new versions of C++. Many people can't or don't learn all the implications of all the changes before the next version comes out.
Wouldn’t this require a scaling constant analogous to the Boltzmann constant or ideal gas constant to be true?
I doubt there are 10^23 lines of asm in all the binaries ever compiled, let alone larger “tech changes”
If its a straight line you need 2 samples.
If its not you probably need more.
Id argue codebase understanding is neither of these simplifications
I don't like the analogy, it supposed that AI generated code will always be badly organized underneath if it is automatic, however, as time goes, I find the code better and better..
It would be really interesting to see an AI create a new kernel for Linux that every piece of it's code was written and verified-secure by having it analyzed and exploited simultaneously.
We, humans, don't need to be as fast as our tools.
On a personal note, I can't wait to see new silicone and operating systems made specifically for AI. That will be cool!
But overall cool read to make you think, where you can see a dev brain as an ADC, design paradigms or microservices as band pass filters, apis as low pass, frameworks as downsampling, legacy code as DC signal that you don't touch, etc.
LLMs plus continuous delivery - what a gruesome twosome.
In business and in tech, the ends are often monetary. It doesn't matter if you release bugs into prod as long as the majority of your paying users do not complain.
At my corp, bugs in prod outside of shadow deployments are up 40% year-to-year since 2023, yet users and ad revenue is growing like never before also, so it doesn't matter if we ship garbage code.
AI advocates nowadays are calling for abolishing human code review. So code moves from read-often-write-rarely to read-once-write-once to read-never-write-often. The assumption is that we can rely on the agents to decipher the code if needed.
My feeling is that software "engineering" in the future will be basically a cargo cult or the Adeptus Mechanicus from Warhammer 40k: all detailed technical knowledge comes from the gods, humans only manage the rituals around technology.
I have to say I mainly agree in a way I wouldn’t have 1 month ago. We are now starting to see bugs creep above our baseline. This is only what we are detecting. At least in certain critical areas we are not removing strict human review.
One theory is our codebase (which is large and mature) was very coherent. As we moved to more autonomy and needed generated code in the base has grown and is less mature now and likely not as coherent. The LLM has to cope with this.
Let's not use analogies from the distant past, they will only hold us back.