I worry about the "brain atrophy" part, as I've felt this too. And not just atrophy, but even moreso I think it's evolving into "complacency".
Like there have been multiple times now where I wanted the code to look a certain way, but it kept pulling back to the way it wanted to do things. Like if I had stated certain design goals recently it would adhere to them, but after a few iterations it would forget again and go back to its original approach, or mix the two, or whatever. Eventually it was easier just to quit fighting it and let it do things the way it wanted.
What I've seen is that after the initial dopamine rush of being able to do things that would have taken much longer manually, a few iterations of this kind of interaction has slowly led to a disillusionment of the whole project, as AI keeps pushing it in a direction I didn't want.
I think this is especially true if you're trying to experiment with new approaches to things. LLMs are, by definition, biased by what was in their training data. You can shock them out of it momentarily, whish is awesome for a few rounds, but over time the gravitational pull of what's already in their latent space becomes inescapable. (I picture it as working like a giant Sierpinski triangle).
I want to say the end result is very akin to doom scrolling. Doom tabbing? It's like, yeah I could be more creative with just a tad more effort, but the AI is already running and the bar to seeing what the AI will do next is so low, so....
It's not just brain atrophy, I think. I think part of it is that we're actively making a tradeoff to focus on learning how to use the model rather than learning how to use our own brains and work with each other.
This would be fine if not for one thing: the meta-skill of learning to use the LLM depreciates too. Today's LLM is gonna go away someday, the way you have to use it will change. You will be on a forever treadmill, always learning the vagaries of using the new shiny model (and paying for the privilege!)
I'm not going to make myself dependent, let myself atrophy, run on a treadmill forever, for something I happen to rent and can't keep. If I wanted a cheap high that I didn't mind being dependent on, there's more fun ones out there.
> let myself atrophy, run on a treadmill forever, for something
You're lucky to afford the luxury not to atrophy.
It's been almost 4 years since my last software job interview and I know the drills about preparing for one.
Long before LLMs my skills naturally atrophy in my day job.
I remember the good old days of J2ME of writing everything from scratch. Or writing some graph editor for universiry, or some speculative, huffman coding algorithm.
That kept me sharp.
But today I feel like I'm living in that netflix series about people being in Hell and the Devil tricking them they're in Heaven and tormenting them: how on planet Earth do I keep sharp with java, streams, virtual threads, rxjava, tuning the jvm, react, kafka, kafka streams, aws, k8s, helm, jenkins pipelines, CI-CD, ECR, istio issues, in-house service discovery, hierarchical multi-regions, metrics and monitoring, autoscaling, spot instances and multi-arch images, multi-az, reliable and scalable yet as cheap as possible, yet as cloud native as possible, hazelcast and distributed systems, low level postgresql performance tuning, apache iceberg, trino, various in-house frameworks and idioms over all of this?
Oh, and let's not forget the business domain, coding standards, code reviews, mentorships and organazing technical events.
Also, it's 2026 so nobody hires QA or scrum masters anymore so take on those hats as well.
This is a very good point. Years ago working in a LAMP stack, the term LAMP could fully describe your software engineering, database setup and infrastructure. I shudder to think of the acronyms for today's tech stacks.
And yet many the same people who lament the tooling bloat of today will, in a heartbeat, make lame jokes about PHP. Most of them aren't even old enough to have ever done anything serious with it, or seen it in action beyond Wordpress or some spaghetti-code one-pager they had to refactor at their first job. Then they show up on HN with a vibe-coded side project or blog post about how they achieved a 15x performance boost by inventing server-side rendering.
Ya I agree it's totally crazy.... but, do most app deployments need even half that stuff? I feel like most apps at most companies can just build an app and deploy it using some modern paas-like thing.
> I feel like most apps at most companies can just build an app and deploy it using some modern paas-like thing.
Most companies (in the global, not SV sense) would be well served by an app that runs in a Docker container in a VPS somewhere and has PostgreSQL and maybe Garage, RabbitMQ and Redis if you wanna get fancy, behind Apache2/Nginx/Caddy.
But obviously that’s not Serious Business™ and won’t give you zero downtime and high availability.
Though tbh most mid-size companies would also be okay with Docker Swarm or Nomad and the same software clustered and running behind HAProxy.
> Most companies (in the global, not SV sense) would be well served by an app that runs in a Docker container in a VPS somewhere and has PostgreSQL and maybe Garage, RabbitMQ and Redis if you wanna get fancy, behind Apache2/Nginx/Caddy.
That’s still too much complication. Most companies would be well served by a native .EXE file they could just run on their PC. How did we get to the point where applications by default came with all of this shit?
When I was in primary school, the librarian used a computer this way, and it worked fine. However, she had to back it up daily or weekly onto a stack of floppy disks, and if she wanted to serve the students from the other computer on the other side of the room, she had to restore the backup on there, and remember which computer had the latest data, and only use that one. When doing a stock–take (scanning every book on the shelves to identify lost books), she had to bring that specific computer around the room in a cart. Such inconveniences are not insurmountable, but they're nice to get rid of. You don't need to back up a cloud service and it's available everywhere, even on smaller devices like your phone.
There's an intermediate level of convenience. The school did have an IT staff (of one person) and a server and a network. It would be possible to run the library database locally in the school but remotely from the library terminals. It would then require the knowledge of the IT person to administer, but for the librarian it would be just as convenient as a cloud solution.
I think the 'more than one user' alternative to a 'single EXE on a single computer' isn't the multilayered pie of things that KronisLV mentioned, but a PHP script[0] on an apache server[0] you access via a web browser. You don't even need a dedicated DB server as SQLite will do perfectly fine.
> but a PHP script[0] on an apache server[0] you access via a web browser
I've seen plenty of those as well - nobody knows exactly how things are setup, sometimes dependencies are quite outdated and people are afraid to touch the cPanel config (or however it's setup). Not that you can't do good engineering with enough discipline, it's just that Docker (or most methods of containerization) limits the blast range when things inevitably go wrong and at least try to give you some reproducibility.
At the same time, I think that PHP can be delightfully simple and I do use Apache2 myself (mod_php was actually okay, but PHP-FPM also isn't insanely hard to setup), it's just that most of my software lives in little Docker containers with a common base and a set of common tools, so they're decoupled from the updates and config of the underlying OS. I've moved the containers (well data+images) across servers with no issues when needed and also resintalled OSes and spun everything right back up.
> That’s still too much complication. Most companies would be well served by a native .EXE file they could just run on their PC
I doubt that.
As software has grown to solving simple personal computing problems (write a document, create a spreadsheet) to solving organizational problems (sharing and communication within and without the organization), it has necessarily spread beyond the .exe file and local storage.
That doesn't give a pass to overly complex applications doing a simple thing - that's a real issue - but to think most modern company problems could be solved with just a local executable program seems off.
It can be like that, but then IT and users complain about having to update this .exe on each computer when you add new functionality or fix some errors. When you solve all major pain points with a simple app, "updating the app" becomes top pain point, almost by definition.
> How did we get to the point where applications by default came with all of this shit?
Because when you give your clients instructions on how to setup the environment, they will ignore some of them and then they install OracleJDK while you have tested everything under OpenJDK and you have no idea why the application is performing so much worse in their environment: https://blog.kronis.dev/blog/oracle-jdk-and-openjdk-compatib...
It's not always trivial to package your entire runtime environment unless you wanna push VM images (which is in many ways worse than Docker), so Docker is like the sweet spot for the real world that we live in - a bit more foolproof, the configuration can be ONE docker-compose.yml file, it lets you manage resource limits without having to think about cgroups, as well as storage and exposed ports, custom hosts records and all the other stuff the human factor in the process inevitably fucks up.
And in my experience, shipping a self-contained image that someone can just run with docker compose up is infinitely easier than trying to get a bunch of Ansible playbooks in place.
If your app can be packaged as an AppImage or Flatpak, or even a fully self contained .deb then great... unless someone also wants to run it on Windows or vice versa or any other environment that you didn't anticipate, or it has more dependencies than would be "normal" to include in a single bundle, in which case Docker still works at least somewhat.
Software packaging and dependency management sucks, unless we all want to move over to statically compiled executables (which I'm all for). Desktop GUI software is another can of worms entirely, too.
When I come into a new project and I find all this... "stuff" in use, often what I later find is actually happening with a lot of it is:
- nobody remembers why they're using it
- a lot of it is pinned to old versions or the original configuration because the overhead of maintaining so much tooling is too much for the team and not worth the risk of breaking something
- new team members have a hard time getting the "complete picture" of how the software is built and how it deploys and where to look if something goes wrong.
Businesses too. For two years it's been "throw everything into AI." But now that shit is getting real, are they really feeling so coy about letting AI run ahead of their engineering team's ability to manage it? How long will it be until we start seeing outages that just don't get resolved because the engineers have lost the plot?
From what I am seeing, no one is feeling coy simply because of the cost savings that management is able to show the higher-ups and shareholders. At that level, there's very little understanding of anything technical and outages or bugs will simply get a "we've asked our technical resources to work on it". But every one understands that spending $50 when you were spending $100 is a great achievement. That's if you stop and not think about any downsides. Said management will then take the bonuses and disappear before the explosions start with their resume glowing about all the cost savings and team leadership achievements. I've experienced this first hand very recently.
Of all the looming tipping points whereby humans could destroy the fabric of their existence, this one has to be the stupidest. And therefore the most likely.
There really ought to be a class of professionals like forensic accountants who can show up in a corrupted organization and do a post mortem on their management of technical debt
Yeah but that's very different from an AWS outage. Everyone's website being down for a day every year or 2 is something that it's very hard to take advantage of as a competitor. That's not true for software that is just terrible all the time.
because it doesn't have any skin in the game and can't be punished, and can't be rewarded for succeeding. Its reputation, career, and dignity are nonexistent.
On the contrary - the LLM has had it's own version of "skin in the game" through the whole of it's training. Reinforcement learning is nothing but that. Why is that less real than putting a person in prison. Is it because of the LLM itself, or because you don't trust the people selling it to you?
Stopped anyone from doing what? Assigning responsibility to someone with nothing to lose, no dignity or pride, and immune from financial or social injury?
> It's not just brain atrophy, I think. I think part of it is that we're actively making a tradeoff to focus on learning how to use the model rather than learning how to use our own brains and work with each other.
I agree with the sentiment but I would have framed it differently. The LLM is a tool, just like code completion or a code generator. Right now we focus mainly on how to use a tool, the coding agent, to achieve a goal. This takes place at a strategic level. Prior to the inception of LLMs, we focused mainly on how to write code to achieve a goal. This took place at a tactical level, and required making decisions and paying attention to a multitude of details. With LLMs our focus shifts to a higher-level abstraction. Also, operational concerns change. When writing and maintaining code yourself, you focus on architectures that help you simplify some classes of changes. When using LLMs, your focus shifts to building context and aiding the model effectively implement their changes. The two goals seem related, but are radically different.
I think a fairer description is that with LLMs we stop exercising some skills that are only required or relevant if you are writing your code yourself. It's like driving with an automatic transmission vs manual transmission.
Previous tools have been deterministic and understandable. I write code with emacs and can at any point look at the source and tell you why it did what it did. But I could produce the same program with vi or vscode or whatever, at the cost of some frustration. But they all ultimately transform keystrokes to a text file in largely the same way, and the compiler I'm targeting changes that to asm and thence to binary in a predictable and visible way.
An LLM is always going to be a black box that is neither predictable nor visible (the unpredictability is necessary for how the tool functions; the invisibility is not but seems too late to fix now). So teams start cargo culting ways to deal with specific LLMs' idiosyncrasies and your domain knowledge becomes about a specific product that someone else has control over. It's like learning a specific office suite or whatever.
> An LLM is always going to be a black box that is neither predictable nor visible (the unpredictability is necessary for how the tool functions; the invisibility is not but seems too late to fix now)
So basically, like a co-worker.
That's why I keep insisting that anthropomorphising LLMs is to be embraced, not avoided, because it gives much better high-level, first-order intuition as to where they belong in a larger computing system, and where they shouldn't be put.
Sort of except it seems the more the co-worker does the job it atrophies my ability to understand.. So soon we'll all be that annoyingly ignorant manager saying, "I don't know, I want the button to be bigger". Yay?
Only if we're lucky and the LLMs cease being replaced with improved models.
Claude has already shown us people who openly say "I don't code and yet I managed this"; right now the command line UI will scare off a lot of people, and people using the LLMs still benefit from technical knowledge and product design skills, if the tools don't improve we keep that advantage…
…but how long will it be before the annoyingly ignorant customer skips the expensive annoyingly ignorant manager along with all us expensive developers, and has one of the models write them bespoke solution for less than the cost of off-the-shelf shrink-wrapped DVDs from a discount store?
> using the LLMs still benefit from technical knowledge and product design skills, if the tools don't improve we keep that advantage…
I don't think we will, because many of us are already asking LLMs for help/advice on these, so we're already close to the point where LLMs will be able to use these capabilities directly, instead of just for helping us drive the process.
Indeed, but the output of LLMs today for these kinds of task are akin to a junior product designer, a junior project manager, a junior software architect etc.
For those of us who are merely amateur at any given task, LLMs raising us to "junior" is absolutely an improvement. But just as it's possible to be a better coder than an LLM, if you're a good PM or QA or UI/UX designer, you're not obsolete yet.
> and can at any point look at the source and tell you why it did what it did
Even years later? Most people can’t unless there’s good comments and design. Which AI can replicate, so if we need to do that anyway, how is AI specially worse than a human looking back at code written poorly years ago?
I mean, Emacs's oldest source files are like 40 years old at this point, and yes they are in fact legible? I'm not sure what you're asking -- you absolutely can (and if you use it long enough, will) read the source code of your text editor.
The little experience I have with LLM confidently shows that LLMs are much better at navigating and modifying a well structured code base. And they struggle, sometimes to a point where they can't progress at all, if tasked to work on bad code. I mean, the kind of bad you always get after multiple rounds of unsupervised vibe coding.
That’s not a good time, I love these things. I’ve been able to indulge myself so much. Possibly good for job security but would suck in every other way.
This is why local models are so important. Even if the non-local ones shut down, and even if you can't run local ones on your own hardware, there will still be inference providers willing to serve your requests.
Recently I was thinking about how some (expensive) customer electronics like the Mac Studio can run pretty powerful open source models with a pretty efficient power consumption, that could pretty easily run on private renewable energy, and that are on most (all?) fronts much more powerful than the original ChatGPT especially if connected to a good knowledge base. Meaning that aside from very extreme scenarios I think it is safe to say that there will always be a way not to go back to how we used to code, as long as we can offer the correct hardware and energy.
Of course personally I think we will never need to go to such extreme ends... despite knowing of people who seem to seriously think developed countries heavily run out of electricity one day, which, while I reckon there might be tensions, seems like a laughable idea IMHO.
> the meta-skill of learning to use the LLM depreciates too. Today's LLM is gonna go away someday, the way you have to use it will change. You will be on a forever treadmill, always learning the vagaries of using the new shiny model (and paying for the privilege!)
I haven’t found this to be true at all, at least so far.
As models improve I find that I can start dropping old tricks and techniques that were necessary to keep old models in line. Prompts get shorter with each new model improvement.
It’s not really a cycle where you’re re-learning all the time or the information becomes outdated. The same prompt structure techniques are usually portable across LLMs.
Interesting, I’ve experienced the opposite in certain contexts. CC is so hastily shipped that new versions often imbalance existing workflows. E.g. people were raving about the new user prompt tools that CC used to get more context but they messed my simple git slash commands
I think you have to be aware of how you use any tool but I don’t think this is a forever treadmill. It’s pretty clear to me since early on that the goal is for you the user to not have to craft the perfect prompt. At least for my workflow it’s pretty darn close to that for me.
If it ever gets there, then anyone can use it and there's no "skill" to be learned at all.
Either it will continue to be this very flawed non-deterministic tool that requires a lot of effort to get useful code out of it, or it will be so good it'll just work.
That's why I'm not gonna heavily invest my time into it.
Good for you. Others like myself find the tools incredibly useful. I am able to knock out code at a higher cadence and it’s meeting a standard of quality our team finds acceptable.
I have deliberately moderated my use of AI in large part for this reason. For a solid two years now I've been constantly seeing claims of "this model/IDE/Agent/approach/etc is the future of writing code! It makes me 50x more productive, and will do the same for you!" And inevitabely those have all fallen by the wayside and been replaced by some new shiny thing. As someone who doesn't get intrinsic joy out of chasing the latest tech fad I usually move along and wait to see if whatever is being hyped really starts to take over the world.
This isn't to say LLMs won't change software development forever, I think they will. But I doubt anyone has any idea what kind of tools and approaches everyone will be using 5 or 10 years from now, except that I really doubt it will be whatever is being hyped up at this exact moment.
HN is where I keep hearing the “50× more productive” claims the most.
I’ve been reading 2024 annual reports and 2025 quarterlies to see whether any of this shows up on the other side of the hype.
So far, the only company making loud, concrete claims backed by audited financials is Klarna and once you dig in, their improved profitability lines up far more cleanly with layoffs, hiring freezes, business simplification, and a cyclical rebound than with Gen-AI magically multiplying output. AI helped support a smaller org that eliminated more complicated financial products that have edge cases, but it didn’t create a step-change in productivity.
If Gen-AI were making tech workers even 10× more productive at scale, you’d expect to see it reflected in revenue per employee, margins, or operating leverage across the sector.
I have friends who make such 50x productivity claims. They are correct if we define productivity as creating untested apps and games and their features that will never ship --- or be purchased, even if they were to ship. Thus, “productivity” has become just another point of contention.
100% agree. There are far more half-baked, incomplete "products" and projects out there now that it is easier to generate code. Generously, that doesn't necessarily equate to productivity.
I've agree with the fact that the last 10% of a project is the hardest part, and that's the part that Gen-AI sucks at (hell, maybe the 30%).
> If Gen-AI were making tech workers even 10× more productive at scale, you’d expect to see it reflected in revenue per employee, margins, or operating leverage across the sector.
If we’re even just talking a 2x multiplier, it should show up in some externally verifiable numbers.
I agree, and we might be seeing this but there is so much noise, so many other factors, and we're in the midst of capital re-asserting control after a temporary loss of leverage which might also be part of a productivity boost (people are scared so they are working harder).
The issue is that I'm not a professional financial analyst and I can't spend all day on comps so I can't tell through the noise yet if we're seeing even 2x related to AI.
But, if we're seeing 10x, I'd be finding it in the financials. Hell, a blind squirrel would, and it's simply not there.
Yes, I think there many issues in a big company that could hide a 2x productivity increase for a little while. But I'd expect it to be very visible in small companies and projects. Looking at things like number of games released on steam, new products launched on new product sites, or issues fixed on popular open source repos, you'd expect a 2x bump to be visible.
In my experience all technology has been like this though. We are on the treadmill of learning the new thing with our without LLMs. That's what makes tech work so fun and rewarding (for me anyway).
I assume you're living in a city. You're already renting out a lot of things to others (security, electricity, water, food, shelter, transportation), what is different with white collar work?
My apartment has been here for years and will be here for many more. I don't love paying rent on it but it certainly does get maintained without my having to do anything. And the rest of the infrastructure of my life is similarly banal. I ride Muni, eat food from Trader Joe's, and so on. These things are not going away and they don't require me to rewire my brain constantly in order to make use of them. The city infrastructure isn't stealing my ability to do my work, it just fills in some gaps that genuinely cannot be filled when working alone and I can trust it to keep doing that basically forever.
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise."
- Socrates on Writing and Reading, Phaedrus 370 BC
If one reads the dialogue, Socrates is not the one "saying" this, but he is telling a story of what King Thamus said to the Egyptian god Theuth, who is the inventor of writing. He is asking the king to give out the writing, but the king is unsure about it.
Its what is known as one of the Socratic "myths," and really just contributes to a web of concepts that leads the dialogue to its ultimate terminus of aporia (being a relatively early Plato dialogue). Socrates, characteristically, doesn't really give his take on writing. In the text, he is just trying to help his friend write a horny love letter/speech!
I can't bring it up right now, but the end of the dialogue has a rather beautiful characterization of writing in the positive, saying that perhaps logos can grow out of writing, like a garden.
I think if pressed Socrates/Plato would say that LLM's are merely doxa machines, incapable of logos. But I am just spitballing.
The public domain translations are pretty old either way. John Cooper's big book is probably still the best but im out of the game these days.
AI guys would probably love this if any of them still have the patience to read/comprehend something very challenging. Probably one of the more famous essays on the Phaedrus dialogue. Its the first long essay of this book:
Roughly: Plato's subordination of writing in this text is symptomatic of a broader kind of `logocentrism` throughout all of western canonical philosophy. Derrida argues the idea of the "externality" of writing compared to speech/logos is not justified by anything, and in fact everything (language, thought) is more like a kind "writing."
Presenting this quote without additional commentary is an interesting Rorschach test.
Thankfully more and more people are seriously considering the effects of technology on true wisdom and getting of the "all technological progress clearly is great, look at all these silly unenlightened naysayers from the past" train.
Socrates was right about the effects. Writing did indeed cause us to loose the talent of memorizing. Where he was wrong though (or rather where this quote without context is wrong) is that it turned out that memorizing was by the most part not the important skill to have.
When Socrates uses the same warnings about LLMs he may however be correct both on the effect and the importance of the skill being lost. If we loose the ability to think and solve various problems, we may indeed be loosing a very important skill of our humanity.
You're misinterpreting the quote. Socrates is saying that being able to find a written quotation will replace fully understanding a concept. It's the difference between being able to quote the pythagorean theorem and understanding it well enough to prove it. That's why Socrates says that those who rely on reading will be "hard to get along with" - they will be pedantic without being able to discuss concepts freely.
While there are dangers to LLMs -science fiction has been talking about this issue for decades (see below) and I think its overblown and the point of the Socrates quote is valid.
(future scientists discover how to add using paper and pencil instead of computer)
I mean Big Paradigm shifts are like death, we can't really predict how humanity will evolve if we really get AGI -but these LLMs as they work today are tools and humans are experts at finding out how to use tools efficiently to counter the trade offs.
Does it really matter today that most programmers don't know how to code in assembly for example?
I’m not making a Malthusian doomsday prediction, and neither was Socrates for that matter. Jobs need to be done, and there will always be somebody willing and able to acquire the relevant skills, and do the job. And in the worst case scenario, society will change it self before it is allowed to fail.
Unlike Malthus, for whom it was easier to imagine the end of the world then the end of Mercantilism, I can easily imagine a world which simply replaces capitalism as its institutions start producing existential threats for humanity.
However, I don‘t think LLMs are even that, for me they are an annoyance which I personally want gone, but next to climate change and the stagnation of population growth, they wont make a dent in upending capitalism, despite how much they suck.
But just because they are not an existential threat, that doesn’t make them harmless. Plenty of people will be harmed by this technology. Like Socrates predicted people will lose skills, this includes skilled programmers, and where previously we were getting some quality software, instead we will get less of it, replaced with a bunch of AI slop. That is my prediction at least.
That is interesting because your mental abilities seem to be correlated with orchestrating a bunch of abstractions you have previously mastered. Are these tools making us stupid because we no longer need to master any of these things? Or are they making us smarter because the abstraction is just trusting AI to handle it for us?
We can also invert that by asking: does a student become smarter by writing their essay on their own?
I would argue that the answer to questions is no. It depends on how you define “smarter”, though. You would likely gain knowledge writing the essay yourself, but is gaining knowledge equivalent to getting smarter?
If so, you could also just read the essay afterwards and gain the same knowledge. Is _that_ smarter? You’ve now reached the same benefit for much less work.
I think fundamentally I at least partially agree with your stance. That we should think carefully before taking a seemingly easier path. Weighing what we gain and lose. Sometimes the juice is, in fact, the squeeze. But it’s far from cut and dry.
It's unclear if you've presented this quote in order to support or criticize the idea that new technologies make us dumber. (Perhaps that's intentional; if so, bravo).
To me, this feels like support. I was never an adult who could not read or write, so I can't check my experience against Socrates' specific concern. But speaking to the idea of memory, I now "outsource" a lot of my memory to my smartphone.
In the past, I would just remember my shopping list, and go to the grocery store and get what I needed. Sure, sometimes I'd forget a thing or two, but it was almost always something unimportant, and rarely was a problem. Now I have my list on my phone, but on many occasions where I don't make a shopping list on my phone, when I get to the grocery store I have a lot of trouble remembering what to get, and sometimes finish shopping, check out, and leave the store, only to suddenly remember something important, and have to go back in.
I don't remember phone numbers anymore. In college (~2000) I had the campus numbers (we didn't have cell phones yet) of at least two dozen friends memorized. Today I know my phone number, my wife's, and my sister's, and that's it. (But I still remember the phone number for the first house I lived in, and we moved out of that house when I was five years old. Interestingly, I don't remember the area code, but I suppose that makes sense, as area codes weren't required for local dialing in the US back in the 80s.)
Now, some of this I will probably ascribe to age: I expect our memory gets more fallible as we get older (I'm in my mid 40s). I used to have all my credit/debit card numbers, and their expiration dates and security codes, memorized (five or six of them), but nowadays I can only manage to remember two of them. (And I usually forget or mix up the expiration dates; fortunately many payment forms don't seem to check, or are lax about it.) But maybe that is due to new technology to some extent: most/all sites where I spend money frequently remember my card for me (and at most only require me to enter the security code). And many also take Paypal or Google Pay, which saves me from having to recall the numbers.
So I think new technology making us "dumber" is a very real thing. I'm not sure if it's a good thing or a bad thing. You could say that, in all of my examples, technology serving the place of memory has freed up mental cycles to remember more important things, so it's a net positive. But I'm not so sure.
I don‘t think human memory works like that, at least not in theory. Storage is not the limiting factor of human memory, but rather retention. It takes time and effort to retain new information. In the past you spent some time and effort to memorize the shopping list and the phone number. Mulling it over in your mind (or out loud), repeated recalls, exposure, even mnemonic tricks like rhymes, alliterations, connecting with pictures, stories, etc. if what you had to remember was something more complicated/extensive/important. And retention is not forever, unless you repeat it, you will loose it. And you only have so much time for repetition and recall, so inevitably, there will be memories which won‘t be repeated, and can’t be recalled.
So when you started using technology to offload your memory, what you gained was the time and effort you previously spent encoding these things into your memory.
I think there is a fundamental difference though between phone book apps and LLMs. Loosing the ability to remember a phone number is not as severe as loosing the ability to form a coherent argument, or to look through sources, or for a programmer to work through logic, to abstract complex logic into simpler chunks. If a scholar looses the skill to look through sources, and a programmer looses the ability to abstract complex logic, they are loosing a fundamental part of their needed to do their jobs. This is like if a stage actor looses the ability to memorize the script, instead relying on a tape-recorder when they are on stage.
Now if a stage actor losses the ability to memorize the script, they will soon be out of a job, but I fear in the software industry (and academia) we are not so lucky. I suspect we will see a lot of people actually taking that tape recorder on stage, and continue to do their work as if nothing is more normal. And the drop in quality will predictably follow.
My personal counterpoint is Norman's thesis in Things That Make Us Smart.
I've long tried, and mostly failed, to consider the tradeoffs, to be ever mindful that technologies are never neutral (winners & losers), per Postman's Technopoly.
Writing/reading and AI are so categorically different that the only way you could compare them is if you fundamentally misunderstand how both of them work.
And "other people in the past predicted doom about something like this and it didn't happen" is a fallacious non-argument even when the things are comparable.
The argument Socrates is making is specifically that writing isn't a substitute for thinking, but it will be used as such. People will read things "without instruction" and claim to understand those things, even if they do not. This is a trade-off of writing. And the same thing is happening with LLMs in a widespread manner throughout society: people are having ChatGPT generate essays, exams, legal briefs and filings, analyses, etc., and submitting them as their own work. And many of these people don't understand what they have generated.
Writing's invention is presented as an "elixir of memory", but it doesn't transfer memory and understanding directly - the reader must still think to understand and internalize information. Socrates renames it an "elixir of reminding", that writing only tells readers what other people have thought or said. It can facilitate understanding, but it can also enable people to take shortcuts around thinking.
I feel that this is an apt comparison, for example, for someone who has only ever vibe-coded to an experienced software engineer. The skill of reading (in Socrates's argument) is not equivalent to the skill of understanding what is read. Which is why, I presume, the GP posted it in response to a comment regarding fear of skill atrophy - they are practicing code generation but are spending less time thinking about what all of the produced code is doing.
It takes about 30 seconds of thinking and/or searching the Internet to realize that people also predict doom when it actually happens - e.g. with people correctly predicting that TikTok will shorten people's attention spans.
It's then quite obvious that the fact that someone, somewhere, predicts a bad thing happening has ~zero bearing on whether it actually happens, and so the claim that "someone predicted doom in the past and it didn't happen then so someone predicting doom now is also wrong" is absurd. Calling that idea "intellectually lazy" is an insult to smart-but-lazy people. This is more like intellectually incapable.
The fact that people will unironically say such a thing in the face of not only widespread personal anecdotes from well-respected figures, but scientific evidence, is depressing. Maybe people who say these things are heavy LLM users?
There is always some set of people predicting all sorts of dooms though. The saying about the broken clock comes to mind.
With the right cherry picking, it can always be said that [some set of] the doomsayers were right, or that they were wrong.
As you say, someone predicting doom has no bearing on whether it happens, so why engage in it? It's just spreading FUD and dwelling on doom. There's no expected value to the individual or to others.
Personally, I don't think "TikTok will shorten people's attention spans" qualifies as doom in and of itself.
> And "other people in the past predicted doom about something like this and it didn't happen" is a fallacious non-argument even when the things are comparable.
> the claim that "someone predicted doom in the past and it didn't happen then so someone predicting doom now is also wrong" is absurd
It's pretty clear that I'm not defending engaging in baseless negative speculation, but refuting the dismissal of negative speculation based purely on the trope that "people have always predicted it".
Someone who read what they were responding to would rather easily have seen that.
> As you say, someone predicting doom has no bearing on whether it happens
That is not what I said. I'm pretty sure now that you did not read my comment before responding. That's bad.
This is what I said:
> It's then quite obvious that the fact that someone, somewhere, predicts a bad thing happening has ~zero bearing on whether it actually happens, and so the claim that "someone predicted doom in the past and it didn't happen then so someone predicting doom now is also wrong" is absurd.
I'm very clearly pointing out (with "someone, somewhere") that a random person predicting a bad thing has almost no ("~zero") impact on the future. Obviously, if someone who has the ability to affect the future (e.g. a big company executive, or a state leader (past or present)) makes a prediction, they have much more power to actually affect the future.
> so why engage in it? It's just spreading FUD and dwelling on doom.
Because (rational) discussion now has the capacity to drive change.
> There's no expected value to the individual or to others.
Trivially false - else most social movements would be utterly irrelevant, because they work through the same mechanism - talking about things that should be changed as a way of driving that change.
It's also pretty obvious that there's a huge difference between "predicting doom with nothing behind it" and "describing actual bad things that are happening that have a lot of evidence behind them" - which is what is actually happening here, so all of your arguments about the former point would be irrelevant (if they were valid, which they aren't) because that's not even the topic of discussion.
I suggest reading what you're responding to before responding.
> Personally, I don't think "TikTok will shorten people's attention spans" qualifies as doom in and of itself.
You're bringing up "doom" as a way to pedantically quarrel about word definitions. It's trivial to see that that's completely irrelevant to my argument - and worth noting that you're then conceding the point about people correctly predicting that TikTok will shorten people's attention spans, hence validating the need to have discussions about it.
Creation of the internet, social media, everyone on the planet getting a pocket sized supercomputer, beginning of the AI boom, Trump/beginning of the end of the US, are all reasons people will study this period of time.
This is really interesting because I wholeheartedly believe the original sentiment that everyone thinks their generation is special, and that "now this time they've really screwed it all up" is quite myopic -- and that human nature and the human experience are relatively constant throughout history while the world changes around us.
But, it is really hard to escape the feeling that digital technology and AI are a huge inflection point. In some ways this couple generations might be the singularity. Trump and contemporary geopolitics in general is a footnote, a silly blip that will pale in comparison over time.
I know managers who can read code just fine, they're just not able/willing to code it. Tho the ai helps with that too. I've had a few managers dabble back into coding esp scripts and whatnot where I want them to be pulling unique data and doing one off investigations.
I read grandparent comment as saying people have been claiming that the sky is falling forever… AI will be both good for learning and development and bad. It’s always up to the individual if it benefits them or atrophies their minds.
I'm not a big fan of LLMs, but while using it for day to day tasks, I get the same feeling I had when I first started the internet (I was lucky to start with broadband internet).
That feeling was one of empowerment: I was able to satisfy my curiosity about a lot of topics.
LLMs can do the same thing and save me a lot of time. It's basically a super charged Google. For programming it's a super charged auto complete coupled with a junior researcher.
My main concern is independence. LLMs in the hands of just a bunch of unchecked corporations are extremely dangerous. I kind of trusted Google, and even that trust is eroding, and LLMs can be extremely personal. The lack of trust ranges from risk of selling data and general data leaks, to intrusive and worse, hidden ads, etc.
When I first started using the internet, I was able to instant text message (IRC) random strangers, using a fake name, and lie about my age. My teacher had us send an email to our ex-classmate who had move to Australia, and she replied the next day, I was able to download the song I just heard on the radio and play it as many times as I wanted on my winamp.
These capabilities simply didn’t exist before the Internet. Apart for the email to Australia (which was possible with a fax machine; but much more expensive), LLMs don‘t give you any new capabilities. It just provides a way for you to do what you already can (and should) do with your brain, without using your brain. It is more like using replacing your social interaction with facebook, then it is to experience an instant message group chat for the first time.
Before LLMs it was incredibly tedious or expensive or both to get legal guidance for stuff like taxes, where I live. Now I can orient myself much better before I ask an actual tax expert pointed questions, saving a lot of time and money.
The list of things they can provide is endless.
They're not a creator, they're an accelerator.
And time matters. My interests are myriad but my capacity to pass the entry bar manually is low because I can only invest so much time.
If this resembles the feeling you had when you first used the internet, it is drastically different from when I used the internet.
When I first used the internet, it was not about doing things faster, it was about doing things which were previously simply unavailable to me. A 12 year old me was never gonna fax my previous classmate who moved to Australia, but I certainly emailed her.
We are not talking about a creator nor an accelerator, we are talking about an avenue (or a road if you will). When I use the internet, I am the creator, and the internet is the road that gets me there.
When I use an LLM it is doing something I can already do, but now I can do it without using my brain. So the feeling is much closer to doomscrolling on social media where previously I could just read a book or meet my pals at the pub. Doomscrolling facebook is certainly faster then reading a book, or socializing at the pub. But it is a poor replacement for either.
Prior to the internet I used my school library for that (or when I was very young, books at my grandparent’s house). So for me personally that wasn’t a new capability. It wasn’t until I started using Wikipedia around 2004 (when I was 17 years old) that the internet replaced (or rather complemented) libraries for that function.
But I can definitely see how for many people with less access to libraries (or worse quality libraries then what I had access to) the internet provided a new avenue for gaining knowledge which wasn’t available before.
To understand the impact on computer programming per se, I find it useful to imagine that the first computer programs I had encountered were, somehow, expressed in a rudimentary natural language. That (somewhat) divorces the consideration of AI from its specific impact on programming. Surely it would have pulled me in certain directions. Surely I would have had less direct exposure to the mechanics of things. But, it seems to me that’s a distinction of degree, not of kind.
I think I should write more about but I have been feeling very similar. I've been recently exploring using claude code/codex recently as the "default", so I've decided to implement a side project.
My gripe with AI tools in the past is that the kind of work I do is large and complex and with previous models it just wasn't efficient to either provide enough context or deal with context rot when working on a large application - especially when that application doesn't have a million examples online.
I've been trying to implement a multiplayer game with server authoritative networking in Rust with Bevy. I specifically chose Bevy as the latest version was after Claude's cut off, it had a number of breaking changes, and there aren't a lot of deep examples online.
Overall it's going well, but one downside is that I don't really understand the code "in my bones". If you told me tomorrow that I had optimize latency or if there was a 1 in 100 edge case, not only would I not know where to look, I don't think I could tell you how the game engine works.
In the past, I could not have ever gotten this far without really understanding my tools. Today, I have a semi functional game and, truth be told, I don't even know what an ECS is and what advantages it provides. I really consider this a huge problem: if I had to maintain this in production, if there was a SEV0 bug, am I confident enough I could fix it? Or am I confident the model could figure it out? Or is the model good enough that it could scan the entire code base and intuit a solution? One of these three questions have to be answered or else brain atrophy is a real risk.
I'm worried about that too. If the error is reproducible, the model can eventually figure it out from experience. But a ghost bug that I can't pattern? The model ends up in a "you're absolutely right" loop as it incorrectly guesses different solutions.
They're real at scale. Plenty of bugs don't suface until you're running under heavy load on distributed infrastructure. Often the culprit is low in the stack. Asking the reporter the right questions may not help in this case. You have full traces, but can't reproduce in a test environment.
When the cause is difficult to source or fix, it's sometimes easier to address the effect by coding around the problem, which is why mature code tends to have some unintuitive warts to handle edge cases.
> Unless your code is multi-threaded, to which I say, good luck!
What isn't multi-threaded these days? Kinda hard to serve HTTP without concurrency, and practically every new business needs to be on the web (or to serve multiple mobile clients; same deal).
All you need is a database and web form submission and now you have a full distributed system in your hands.
You mean in a single-threaded context like Javascript? (Or with Python GIL giving the impression of the same.) That removes some memory corruption races, but leaves all the logical problems in place. The biggest change is that you only have fixed points where interleaving can happen, limiting the possibilities -- but in either scenario, the number of possible paths is so big it's typically not human-accessible.
Webdevs not aware of race conditions -> complex page fails to load. They're lucky in how the domain sandboxes their bugs into affecting just that one page.
nginx is also from the era when fast static file serving was still a huge challenge, and "enough to run a business" for many purposes -- most software written has more mutable state, and much more potential for edge cases.
Historically I would have agreed with you. But since the rise of LLM-assisted coding, I've encountered an increasing number of things I'd call clear "ghost bugs" in single threaded code. I found a fun one today where invoking a process four times with a very specific access pattern would cause a key result of the second invocation to be overwritten. (It is not a coincidence, I don't think, that these are exactly the kind of bugs a genAI-as-a-service provider might never notice in production.)
> I've been trying to implement a multiplayer game with server authoritative networking in Rust with Bevy. I specifically chose Bevy as the latest version was after Claude's cut off, it had a number of breaking changes, and there aren't a lot of deep examples online.
I am interested in doing something similar (Bevy. not multiplayer).
I had the thought that you ought be able to provide a cargo doc or rust-analyzer equivalent over MCP? This... must exist?
I'm also curious how you test if the game is, um... fun? Maybe it doesn't apply so much for a multiplayer game, I'm thinking of stuff like the enemy patterns and timings in a soulslike, Zelda, etc.
I did use ChatGPT to get some rendering code for a retro RCT/SimCity-style terrain mesh in Bevy and it basically worked, though several times I had to tell it "yeah uh nothing shows up", at which point is said "of course! the problem is..." and then I learned about mesh winding, fine, okay... felt like I was in over my head and decided to go to a 2D game instead so didn't pursue that further.
>I had the thought that you ought be able to provide a cargo doc or rust-analyzer equivalent over MCP? This... must exist?
I've found that there are two issues that arise that I'm not sure how to solve. You can give it docs and point to it and it can generally figure out syntax, but the next issue I see is that without examples, it kind of just brute forces problems like a 14 year old.
For example, the input system originally just let you move left and right, and it popped it into an observer function. As I added more and more controls, it began to litter with more and more code, until it was ~600 line function responsible for a large chunk of game logic.
While trying to parse it I then had it refactor the code - but I don't know if the current code is idiomatic. What would be the cargo doc or rust-analyzer equivalent for good architecture?
Im running into this same problem when trying to claude code for internal projects. Some parts of the codebase just have really intuitive internal frameworks and claude code can rip through them and provide great idiomatic code. Others are bogged down by years of tech debt and performance hacks and claude code can't be trusted with anything other than multi-paragraph prompts.
>I'm also curious how you test if the game is, um... fun?
Lucky enough for me this is a learning exercise, so I'm not optimizing for fun. I guess you could ask claude code to inject more fun.
> What would be the cargo doc or rust-analyzer equivalent for good architecture?
Well, this is where you still need to know your tools. You should understand what ECS is and why it is used in games, so that you can push the LLM to use it in the right places. You should understand idiomatic patterns in the languages the LLM is using. Understand YAGNI, SOLID, DDD, etc etc.
Those are where the LLMs fall down, so that's where you come in. The individual lines of code after being told what architecture to use and what is idiomatic is where the LLM shines.
What you describe is how I use LLM tools today, but the reason I am approaching my project in this way is because I feel I need to brace myself for a future where developers are expected to "know your tools"
When I look around today - its clear more and more people are diving in head first into fully agentic workflows and I simply don't believe they can churn out 10k+ lines of code today and be intimately familiar with the code base. Therefore you are left with two futures:
* Agentic-heavy SWEs will eventually blow up under the weight of all their tech debt
* Coding models are going to continue to get better where tech debt wont matter.
If the answer if (1), then I do not need to change anything today. If the answer is (2), then you need to prepare for a world where almost all code is written by an agent, but almost all responsibility is shouldered by you.
In kind of an ignorant way, I'm actually avoiding trying to properly learn what an ECS is and how the engine is structured, as sort of a handicap. If in the future I'm managing a team of engineers (however that looks) who are building a metaphorical tower of babel, I'd like to develop to heuristic in navigating that mountain.
I ran into similar issues with context rot on a larger backend project recently. I ended up writing a tool that parses the AST to strip out function bodies and only feeds the relevant signatures and type definitions into the prompt.
It cuts down the input tokens significantly which is nice for the monthly bill, but I found the main benefit is that it actually stops the model from getting distracted by existing implementation details. It feels a bit like overengineering but it makes reasoning about the system architecture much more reliable when you don't have to dump the whole codebase into the context window.
I've been thinking along these lines. LLMs seem to have arrived right when we were all getting addicted to reels/tic tocks/whatever. For some reason we love to swipe, swipe, swipe, until we get something funny/interesting/shocking, that gives us a short-lasting dopamine hit (or whatever chemicals it is) that feels good for about 1 second, and we want MORE, so we keep swiping.
Using an LLM is almost exactly the same. You get the occasional, "wow! I've never seen it do that before!" moments (whether that thing it just did was even useful or not), get a short hit of feel goods, and then we keep using it trying to get another hit. It keeps providing them at just the right intervals for people to keep them going just like they do with tick tock
I ran into a new problem today: "reading atrophy".
As in if the LLM doesn't know about it, some devs are basically giving up and not even going to RTFM. I literally had to explain to someone today how something works by...reading through the docs and linking them the docs with screenshots and highlighted paragraphs of text.
Still got push back along the lines of "not sure if this will work". It's. Literally. In. The. Docs.
That's not really a new thing now, it just shows differently.
15 years ago I was working in an environment where they had lots of Indians as cheap labour - and the same thing will show up in any environment where you go for hiring a mass of cheap people while looking more at the cost than at qualifications: You pretty much need to trick them into reading stuff that are relevant.
I remember one case where one had a problem they couldn't solve, and couldn't give me enough info to help remotely. In the end I was sitting next to them, and made them read anything showing up on the screen out loud. Took a few tries where they were just closing dialog boxes without reading it, but eventually we had that under control enough that they were able to read the error messages to me, and then went "Oh, so _that's_ the problem?!"
Overall interacting with a LLM feels a lot like interacting with one of them back then, even down to the same excuses ("I didn't break anything in that commit, that test case was never passing") - and my expectation for what I can get out of it is pretty much the same as back then, and approach to interacting with it is pretty similar. It's pretty much an even cheaper unskilled developer, you just need to treat it as such. And you don't pair it up with other unskilled developers.
The mere existence of the phrase "RTFM" shows that this phenomenon was already a thing. LLMs are the worst thing to happen to people who couldn't read before. When HR type people ask what my "superpower" is I'm so tempted to say "I can read", because I honestly feel like it's the only difference between me and people who suck at working independently.
As someone working in technical support for a long time, this has always been the case.
You can have as many extremely detailed and easy to parse gudies, references, etc. there will always be a portion of customers who refuse to read them.
Never could figure out why because they aren't stupid or anything.
> Eventually it was easier just to quit fighting it and let it do things the way it wanted.
I wouldn't have believed it a few tears ago if you told me the industry would one day, in lockstep, decide that shipping more tech-debt is awesome. If the unstated bet doesn't pay off, that is, AI development will outpace the rate it generates cruft, then there will be hell to pay.
Don't worry. This will create the demand for even more powerful models that are able to untangle the mess created by previous models.
Once we realize the kind of mess _those_ models created, well, we'll need even more capable models.
It's a variation on the theme of Kernighan insight about the more "clever" you are while coding the harder it will be to debug.
EDIT: Simplicity is a way out but it's hard under normal circumstances, now with this kind of pressure to ship fast because the colleague with the AI chimp can outperform you, aiming at simplicity will require some widespread understanding
As someone who's been commissioned many times before to work on or salvage "rescue projects" with huge amounts of tech debt, I welcome that day. Still not there yet though I am starting to feel the vibes shifting.
This isn't anything new of course. Previously it was with projects built by looking for the cheapest bidder and letting them loose on an ill-defined problem. And you can just imagine what kind of code that produced. Except the scale is much larger.
My favorite example of this was a project that simply stopped working due to the amount of bugs generated from layers upon layers of bad code that was never addressed. That took around 2 years of work to undo. Roughly 6 months to un-break all the functionality and 6 more months to clean up the core and then start building on top.
Are you not worried that the sibling comment is right and the solution to this will be "more AI" in the future? So instead of hiring a team of human experts to cleanup, management might just dump more money into some specialized AI refactoring platform or hire a single AI coordinator... Or maybe they skip to rebuild using AI faster, because AI is good at greenfield. Then they only need a specialized migration AI to automate the regular switchovers.
I used to be unconcerned, but I admit to be a little frightened of the future now.
Well, in general worrying about the future is not useful. Regardless of what you think, it is always uncertain. I specifically stay away from taking part in such speculative threads here on HN.
What's interesting to me though is that very similar promises were being made about AI in the 80s. Then came the "AI Winter" after the hype cycle and promises got very far from reality. Generative AI is the current cycle and who knows, maybe it can fulfill all the promises and hype. Or maybe not.
There's a lot of irrationality currently and until that settles down, it is difficult to see what is real and useful and what is smoke and mirrors.
I'm aware of that particular chapter of history, my master's thesis was on conversational interfaces. I don't think the potential of the algorithms (and hardware) back then was in any way comparable to what's currently going on. There is definitely a hype cycle going on right now, but I'm nearly convinced it will actually leave many things changed even after it plays out.
Funny thing is that meanwhile (today) I've actually been on an emergency consulting project where a PO/PM kind of guy vibecoded some app that made it into production. The thing works, but a cursory audit laid open the expected flaws (like logic duplication, dead code, missing branches). So that's another point for our profession still being required in the near future.
The industry decided that decades ago. We may like to talk about quality and forethought, but when you actually go to work, you quickly discover it doesn't matter. Small companies tell you "we gotta go fast", large companies demand clear OKRs and focusing on actually delivering impact - either way, no one cares about tech debt, because they see it as unavoidable fact of life. Even more so now, as ZIRP went away and no one can afford to pay devs to polish the turd ad infinitum. The mantra is, ship it and do the next thing, clean up the old thing if it ever becomes a problem.
And guess what, I'm finally convinced they're right.
Consider: it's been that way for decades. We may tell ourselves good developers write quality code given the chance, but the truth is, the median programmer is a junior with <5 years of experience, and they cannot write quality code to save their life. That's purely the consequence of rapid growth of software industry itself. ~all production code in the past few decades was written by juniors, it continues to be so today; those who advance to senior level end up mostly tutoring new juniors instead of coding.
Or, all that put another way: tech debt is not wrong. It's a tool, a trade-off. It's perfectly fine to be loaded with it, if taking it lets you move forward and earn enough to afford paying installments when they're due. Like with housing: you're better off buying it with lump payment, or off savings in treasury bonds, but few have that money on hand and life is finite, so people just get a mortgage and move on.
--
Edited to add: There's a silver lining, though. LLMs make tech debt legible and quantifiable.
LLMs are affected by tech debt even more than human devs are, because (currently) they're dumber, they have less cognitive capability around abstractions and generalizations[0]. They make up for it by working much faster - which is a curse in terms of amplifying tech debt, but also a blessing, because you can literally see them slowing down.
Developer productivity is hard to measure in large part because the process is invisible (happens in people's heads and notes), and cause-and-effect chains play out over weeks or months. LLM agents compress that to hours to days, and the process itself is laid bare in the chat transcript, easy to inspect and analyze.
The way I see it, LLMs will finally allow us to turn software development at tactical level from art into an engineering process. Though it might be too late for it to be of any use to human devs.
--
[0] - At least the out-of-distribution ones - quirks unique to particular codebase and people behind it.
Sure, but we have high growth on top of that - meaning all those "perpetual intermediaries" are always the minority and gravitate upwards in the org chain, while ~all the coding work is done by people who just started working in the field, and didn't even learn enough yet to become mediocre.
> I wouldn't have believed it a few tears ago if you told me the industry would one day, in lockstep, decide that shipping more tech-debt is awesome.
It's not debt if you never have to pay it back. If a model can regenerate a whole relibale codebase in minutes from a spec, then your assessment of "tech debt" in that output becomes meaningless.
I'm yet to encounter an AI-bull who admits the LLM tendency towards creating tech debt- outside of footnotes stating it can be fixed by better prompting (with no examples), or solved by whatever tool they are selling
My disillusionment comes from the feeling I am just cosplaying my job. There is nothing to distinguish one cosplayer from another. I am just doordashing software, at this point, and I'm not in control.
I don’t get this at all. I’m using LLM’s all day and I’m constantly having to make smart architectural choices that other less experienced devs won’t be making. Are you just prompting and going with whatever the initial output is, letting the LLM make decisions? Every moderately sized task should start with a plan, I can spend hours planning, going off and thinking, coming back to the plan and adding/changing things, etc. Sometimes it will be days before I tell the LLM to “go”. I’m also constantly optimising the context available to the LLM, and making more specific skills to improve results. It’s very clear to me that knowledge and effort is still crucial to good long term output… Not everyone will get the same results, in fact everyone is NOT getting the same results, you can see this by reading the wildly different feedback on HN. To some LLM’s are a force multiplier while others claim they can’t get a single piece of decent output…
I think the way you’re using these tools that makes you feel this way is a choice. You’re choosing to not be in control and do as little as possible.
Once you start using it intelligently, the results can be really satisfying and helpful.
People complaining about 1000 lines of codes being generated? Ask it to generate functions one at a time and make small implementations.
People complaining about having to run a linter? Ask it to automatically run it after each code execution.
People complaining about losing track? Have it log every modifications in a file.
I think you get my point. You need to treat it as a super powerful tool that can do so many things that you have to guide it if you want to have a result that conforms to what you have in mind.
It's not that challenging, the answer is, it depends.
It's like a junior dev writing features for a product everyday vs a principle engineer. The junior might be adding a feature with O(n^2) performance while principle has seen this before and writes it O(log n).
If the feature never reaches significance, the "better" solution doesn't matter, but it might!
The principle may write once and it is solid and never touched, but the junior might be good enough to never need coming back to, same with a llm and the right operator.
There's that, but I actually think LLMs are becoming very good at not making the bad simple choice.
What they're worse at is the bits I can't easily see.
An example is that I recently was working on a project building a library with Claude. The code in pieces all looked excellent.
When I wrote some code making use of it several similar functions which were conceptually similar had signatures that were subtly mismatched.
Different programmers might have picked each patterns. And probably consistently made similar rules for the various projects they worked on.
To an LLM they are just happenstances and feel no friction.
A real project with real humans writing the code would notice the mismatch. Even if they aren't working on those parts at the same time just from working on it across say a weekend.
But how many more decisions do we make convenient only for us meat bags that a LLM doesn't notice?
Yes, but now you know about that classification of problem. So you learned something! As an Engineer have a choice now on what to do with that classification or problem.
Better yet, go up one level and and think about how to avoid the other classifications of problems you don't know about, how can the LLM catch these before it writes the code... etc.
What? Of course it makes a difference when I direct it away from a bad solution towards a good solution. I know as soon as I review the output and it has done what I asked, or it hasn't and I make a correction. Why would I need to wait 5 years? That makes no sense, I can see the output.
If you're using LLM's and you don't know what good/bad output looks like then of course you're going to have problems, but such a person would have the same problems without the LLM...
The problem is the LLMs are exceptionally good at producing output that appears good.
That's what it's ultimately been tuned to do.
The way I see this play out is output that satisfied me but that I would not produce myself.
Over a large project that adds up and typically is glaringly obvious to everyone but the person who was using the LLM.
My only guess as to why that is, is because most of what we do and why we do it we're not conscious of. The threshold we'd intervene at is higher than the original effort it takes to do the right thing.
If these things don't apply to you. Then I think you're coming up to a golden era.
100% there....it's getting to a point where a project manager reports a bug AND also pastes a response from Claude (he ran Claude against our codebase) on how to fix the bug..Like I'm just copying what Claude said and making sure the thing compiles (.NET). What makes me sleep at night...for now is the fact that Claude isn't supporting 9pm deployments and AWS Infra support ...it's already writing code but not supporting it yet...
What kind of software are you writing? Are you just a "code monkey" implementing perfectly described Jira tickets (no offense meant)? I cannot imagine feeling this way with what I'm working on, writing code is just a small part of it, most of the time is spent trying to figure out how to integrate the various (undocumented and actively evolving) external services involved together in a coherent, maintainable and resilient way. LLMs absolutely cannot figure this out themselves, I have to figure it out myself and then write it all in its context, and even then it mostly comes up with sub-par, unmaintainable solutions if I wasn't being precise engouh.
They are amazing for side projects but not for serious code with real world impact where most of the context is in multiple people's head.
No, I am not a code monkey. I have an odd role working directly for an exec in a highly regulated industry, managing their tech pursuits/projects. The work can range from exciting to boring depending on the business cycle. Currently it is quite boring, so I've leaned into using AI a bit more just to see how I like it. I don't think that I do.
I find the atrophy and zoning out or context switching problematic, because it takes a few seconds/ minutes in "thinking" and then BAM! I have 500 lines of all sorts of buggy and problematic code to review and get a sycophantic, not-enough-mature entity to correct.
At some point, I find myself needing to disconnect out of overwhelm and frustration. Faster responses isn't necessarily better. I want more observability in the development process so that I can be a party to it. I really have felt that I need to orchestrate multiple agents working in tandem, playing sort of a bad-cop, good-cop and a maybe a third trying to moderate that discussion and get a fourth to effectively incorporate a human in the mix. But that's too much to integrate in my day job.
I’ve actually found the tool that inspires the most worry about brain atrophy to be Copilot. Vscode is full of flashing suggestions all over. A couple days ago, I wanted to write a very quick program, and it was basically impossible to write any of it without Copilot suggesting a whole series of ways to do what it thought I was doing. And it seems that MS wants this: the obvious control to turn it off is actually just “snooze.”
I found the setting and turned it off for real. Good riddance. I’ll use the hotkey on occasion.
> Like there have been multiple times now where I wanted the code to look a certain way, but it kept pulling back to the way it wanted to do things. Like if I had stated certain design goals recently it would adhere to them, but after a few iterations it would forget again and go back to its original approach, or mix the two, or whatever. Eventually it was easier just to quit fighting it and let it do things the way it wanted.
Absolutely. At a certain level of usage, you just have to let it do it's thing.
People are going to take issue with that. You absolutely don't have to let it do its thing. In that case you have to be way more in the loop. Which isn't necessarily a bad thing.
But assuming you want it to basically do everything while you direct it, it becomes pointless to manage certain details. One thing in my experience is that Claude always wants to use ReactRouter. My personal preference is TanStack router, so I asked it to use it initially. That never really created any problems but after like the 3rd time of realizing I forget to specify it, I also realized that it's totally pointless. ReactRouter works fine and Claude uses it fine - its pointless to specify otherwise.
My experience is the opposite - I haven't used my brain more in a while.. Typing characters was never what developers were valued for anyway. The joy of building is back too.
Another thing I’ve experienced is scope creep into the average. Both Claude and ChatGPT keep making recommendations and suggestions that turns the original request into something that resembles other typical features. Sometimes that’s a good thing, because it means I’ve missed something. A lot of times, especially when I’m just riffing on ideas, it turns into something mundane and ordinary and I’ll have lost my earlier train of thought.
A quick example is trying to build a simple expenses app with it. I just want to store a list of transactions with it. I’ve already written the types and data model and just need the AI to give me the plumping. And it will always end up inserting recommendations about double entry bookkeeping.
yeah but that's like recommending a webserver for your Internet facing website. If you want to give an example of scope creep, you need a better example than double entry book keeping for an accounting app.
You’ve just illustrated exactly the problem. You assumed I was building an accounting app. I’ve experienced the same issue with building features for calculating the brightness of a room, or 3D visualisations of brightness patterns, managing inventory and cataloguing lighting fixtures and so on.
It’s great for churning out stuff that already exists, but that also means it’ll massage your idea into one of them.
> I want to say it's very akin to doom scrolling. Doom tabbing? It's like, yeah I could be more creative with just a tad more effort, but the AI is already running and the bar to seeing what the AI will do next is so low, so....
Yea exactly, Like we are just waiting so that it gets completed and after it gets completed then what? We ask it to do new things again.
Just as how if we are doom scrolling, we watch something for a minute then scroll down and watch something new again.
The whole notion of progress feels completely fake with this. Somehow I guess I was in a bubble of time where I had always end up using AI in web browsers (just as when chatgpt 3 came) and my workflow didn't change because it was free but recently changed it when some new free services dropped.
"Doom-tabbing" or complete out of the loop AI agentic programming just feels really weird to me sucking the joy & I wouldn't even consider myself a guy particular interested in writing code as I had been using AI to write code for a long time.
I think the problem for me was that I always considered myself a computer tinker before coder. So when AI came for coding, my tinkering skills were given a boost (I could make projects of curiosity I couldn't earlier) but now with AI agents in this autonomous esque way, it has come for my tinkering & I do feel replaced or just feel like my ability of tinkering and my interests and my knowledge and my experience is just not taken up into account if AI agent will write the whole code in multi file structure, run commands and then deploy it straight to a website.
I mean my point is tinkering was an active hobby, now its becoming a passive hobby, doom-tinkering? I feel like I have caught up on the feeling a bit earlier with just vibe from my heart but is it just me who feels this or?
> I worry about the "brain atrophy" part, as I've felt this too. And not just atrophy, but even moreso I think it's evolving into "complacency".
Not trusting the ML's output is step one here, that keeps you intellectually involved - but it's still a far cry from solving the majority of problems yourself (instead you only solve problems ML did a poor job at).
Step two: I delineate interesting and uninteresting work, and Claude becomes a pair programmer without keyboard access for the latter - I bounce ideas off of it etc. making it an intelligent rubber duck. [Edit to clarify, a caveat is that] I do not bore myself with trivialities such as retrieving a customer from the DB in a REST call (but again, I do verify the output).
> I do not bore myself with trivialities such as retrieving a customer from the DB in a REST call
Genuine question, why isn't your ORM doing that? I see a lot of use cases for LLMs that seem to be more expensive ways to do snippets and frameworks...
I've gone years without coding and when I come back to it, it's like riding a bike! In each iteration of my coding career, I have become a better developer, even after a large gap. Now I can "code" during my gap. Were I ever to hand-code again, I'm sure my skills would be there. They don't atrophy, like your ability to ride a bike doesn't atrophy. Yes you may need to warm back up, but all the connections in your brain are still there.
Have you ever learnt a foreign language (say Mongolian, or Danish) and then never spoken it, nor even read anything in it for over 10 years? It is not like riding a bike, it doesn’t just come back like that. You have to actually relearn the language, practice it, and you will suck at it for months. Comprehension comes first (within weeks) but you will be speaking with grammatical errors, mispronunciations, etc. for much longer. You won‘t have to learn the language from scratch, second time around is much easier, but you will have to put in the effort. And if you use google translate instead of your brain, you won‘t relearn the language at all. You will simply forget it.
Anecdotally, i burned out pretty hard and basically didn't open a text editor for half a year (unemployed too). Eventually i got an itch to write code again and it didn't really feel like I was really worse. Maybe it wasn't long enough atrophy but code doesn't seem to quite work like language though ime.
Six months is definitely not long enough of a break for skills to degrade. But it's not just skills, as I wrote in another comment, the biggest thing is knowledge of new tools, new versions of language and its features.
I'd say there's at most around 2 years of knowledge runtime (maybe with all this AI stuff this is even shorter). After that period if you don't keep your knowledge up to date it fairly quickly becomes obsolete.
I would imagine there is probably some reverse S-curve of skill loss going on. The first year you may retain like 90% (and the 10% are obscure words, rare grammar structures, expressions, etc.), then in the next 2 years you loose more and more every year, and by the 3rd year you’ve lost like 50% of the language, including some common words, useful grammar structures, but retain common greetings, basic structures, etc. and then after like year 5 the regression starts to slow down and by year 10 you may still know 20%, but it is the most basic stuff, and you won‘t be able to use the language in any meaningful way.
I have not and I'm actually really bad at learning human languages, but know a dozen programming languages. You would think they would be similar, but for some reason it's really easy for me to program in any language and really hard for me to pick up a human language.
I provided it as a counter example to the learning how to bike myth.
Learning how to bike requires only a handful of skills, most of them are located in the motor control centers in your brain (mostly in the Cerebellum), which is known to retain skills much better then any other parts of your brain. Your programing skills are comprised of thousands of separate skills which are mostly located in your frontal-cortex (mostly in your frontal and temporal lobes), and learning a foreign language is basically that but more (like 10x more).
So while a foreign language is not the perfect analogy (nothing is), I think it is a reasonable analogy as a counter example to the bicycle myth.
Maybe something that keeps programming skills fresh is that after you learn to think like a programmer, you do that with problems away from the keyboard. Decomposition, logic... in the years I wasn't programming, I was still solving problems like a programmer. Getting back behind the keyboard just engaged the thought processes I was already keeping warm with practice.
I studied Spanish for years in school, then never really used it. Ten years later, I started studying Japanese. Whenever I got stuck, Spanish would come out. Spanish that I didn't even consciously remember. AFAIK, foreign languages are all stored in the same part of the brain, and once you warm up those neurons, they all get activated.
Not that it's in any way relevant to programming. I will say that after dropping programming for years, I can still explain a lot of specifics, and when I dive back in, it all floods right back. Personally, I'm convinced that any competent, experienced programmer could take a multi-year break, then come back and be right up to speed with the latest software stack in only slightly longer than the stack transition would have taken without a break.
You might still have the skillset to write code, but depending on length of the break your knowledge of tools, frameworks, patterns would be fairly outdated.
I used to know a person like that - high in the company structure who would claim he was a great engineer, but all the actual engineers would make jokes about him and his ancient skills during private conversations.
I’d push back on this framing a bit. There's a subtle ageism baked into the assumption that someone who stepped away from day-to-day coding has "ancient skills" worth mocking.
Yes, specific frameworks and tooling knowledge atrophy without use, and that’s true for anyone at any career stage. A developer who spent three years exclusively in React would be rusty on backend patterns too. But you’re conflating current tool familiarity with engineering ability, and those are different things.
The fundamentals: system design, debugging methodology, reading and reasoning about unfamiliar code, understanding tradeoffs ... those transfer. Someone with deep experience often ramps up on new stacks faster than you’d expect, precisely because they’ve seen the same patterns repackaged multiple times.
If the person you’re describing was genuinely overconfident about skills they hadn’t maintained, that’s a fair critique. But "the actual engineers making jokes about his ancient skills" sounds less like a measured assessment and more like the kind of dismissiveness that writes off experienced people before seeing what they can actually do.
Worth asking: were people laughing because he was genuinely incompetent, or because he didn’t know the hot framework of the moment? Because those are very different things.
This has nothing to do about ageism. This applies to any person of any age who has ego big enough to think that their knowledge of industry is relevant after they take prolonged break and be socially inept enough to brag about how they are still "in".
I don't disagree with your point about fundamentals, but in an industry where there seems to be new JS framework any time somebody sneezes - latest tools are very much relevant too. And of course the big thing is language changes. The events I'm describing happened in the late 00s-early 10s. When language updates picked up steam: Python, JS, PHP, C++. Somebody who used C++ 98 can't claim to have up to date knowledge in C++ in 2015.
So to answer your question - people were laughing at his ego, not the fact that he didn't know some hot new framework.
I beg to differ. I started with C in the 90s, then C# in '05, then PHP in '12, then Go in '21. The things I learned in C still apply to Go, C#, and PHP. And I even started contributing to open source C projects in '24 ... all my skills and knowledge were still relevant. This sounds exactly like ageism to me, but I clearly have a different perspective than you.
Yes, we clearly have different perspectives. I observed an arrogant person who despite their multi-year break from engineering of any kind strongly believed that they still were as capable as engineers who remained in the field during that time.
I code in Vim, use Linux... all of those tools are pretty constant. New frameworks are easy to pick up. I've been able to become productive with very little downtime after multi-year breaks several times.
I feel like I'm still a couple steps behind in skill level as my lead and is trying to gain more experience I do wonder if I am shooting myself in the foot if I rely too much on AI at this stage. The senior engineer I'm trying to learn from can very effectively use ai because he has very good judgement of code quality, I feel like if I use AI too much I might lose out on chance to improve my judgement. It's a hard dilemma.
You could probably combat this somewhat with a skill that references to examples of the code you don't want and the code you do. And then each time you tell it to correct the code you ask it to put that example into the references.
You then tell your agent to always run that skill prior to moving on. If the examples are pattern matchable you can even have the agent write custom lints if your linter supports extension or even write a poor man’s linter using ast-grep.
I usually have a second session running that is mainly there to audit the code and help me add and adjust skills while I keep the main session on the task of working on the feature. I've found this far easier to stay engaged than context switching between unrelated tasks.
Honestly, this seems very much like the jump from being an individual contributor to being an engineering manager.
The time it happened for me was rather abrupt, with no training in between, and the feeling was eerily similar.
You know _exactly_ why the best solution is, you talk to your reports, but they have minds of their own, as well as egos, and they do things … their own way.
At some point I stopped obsessing with details and was just giving guidance and direction only in the cases where it really mattered, or when asked, but let people make their own mistakes.
Now LLMs don’t really learn on their own or anything, but the feeling of “letting go of small trivial things” is sorta similar. You concentrate on the bigger picture, and if it chose to do an iterative for loop instead of using a functional approach the way you like it … well the tests still pass, don’t they.
The only issue is that as an engineering manager you reasonably expect that the team learns new things, improve their skills, in general grow as engineers. With AI and its context handling you're working with a team where each member has severe brain damage that affects their ability to form long term memories.
You can rewire their brain to a degree teaching them new "skills" or giving them new tools, but they still don't actually learn from their mistakes or their experiences.
As a manager I would encourage them to use the LLM tools. I would also encourage unit tests, e2e testing, testing coverages, CI pipelines automating the testing, automatic pr reviewing etc...
It's also peeking at the big/impactful changes and ignoring the small ones.
Your job isn't to make sure they don't have "brain damage" its to keep them productive and not shipping mistakes.
Being optimistic (or pessimistic heh), if things keep the trend then the models will evolve as well and will probably be quite better in one year than they are now.
The solution for brain atrophy I personally arrived is to use coding agents at work, where, let’s be honest, velocity is a top priority and code purity doesn’t matter that much. Since we use stack I super familial with, I can quite fast verify produced code and tweak it if needed.
However, for hobby projects where I purposely use tech I’m not very familiar with, I force myself not to use LLMs at all - even as a chat. Thus, operating The old way - writing code manually, reading documentation, etc brings me a joy of learning back and, hopefully, establishes new neurone connections.
> Like if I had stated certain design goals recently it would adhere to them, but after a few iterations it would forget again and go back to its original approach, or mix the two, or whatever.
Context management, proper prompting and clear instructions, proper documentation are still relevant.
I think this is where tools like OpenSpec [1] may help. The deterioration in quality is because the context is degrading, often due to incomplete or amibiguous requests from the coder. With a more disciplined way of creating and persisting locally the specs for the work, especially if the agent got involved in creating that too, you'll have a much better chance of keeping the agent focussed and aligned.
> AI keeps pushing it in a direction I didn't want
The AI definitely has preferences and attention issues, but there are ways to overcome them.
Defining code styles in a design doc, and setting up initial examples in key files goes a long way. Claude seems pretty happy to follow existing patterns under these conditions unless context is strained.
I have pretty good results using a structured workflow that runs a core loop of steps on each change, with a hook that injects instructions to keep attention focused.
In the happy case where I have a good idea of the changes necessary, I will ask it to do small things, step by step, and examine what it does and commit.
In the unhappy case where one is faced with a massive codebase and no idea where to start, I find asking it to just “do the thing” generates slop, but enough for me to use as inspiration for the above.
yeah, because the thing is: at the end of the day: laying things out the way LLMs can understand is becoming more important than doing them the “right” way— a more insidious form of the same complacency. and one in which i am absolutely complicit.
> LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.
I’ve always said I’m a builder even though I’ve also enjoyed programming (but for an outcome, never for the sake of the code)
This perfectly sums up what I’ve been observing between people like me (builders) who are ecstatic about this new world and programmers who talk about the craft of programming, sometimes butting heads.
One viewpoint isn’t necessarily more valid, just a difference of wiring.
I noticed the same thing, but wasn't able to put it into words before reading that. Been experimenting with LLM-based coding just so I can understand it and talk intelligently about it (instead of just being that grouchy curmudgeon), and the thought in the back of my mind while using Claude Code is always:
"I got into programming because I like programming, not whatever this is..."
Yes, I'm building stupid things faster, but I didn't get into programming because I wanted to build tons of things. I got into it for the thrill of defining a problem in terms of data structures and instructions a computer could understand, entering those instructions into the computer, and then watching victoriously while those instructions were executed.
If I was intellectually excited about telling something to do this for me, I'd have gotten into management.
Same. This kind of coding feels like it got rid of the building aspect of programming that always felt nice, and it replaced it entirely with business logic concerns, product requirements, code reviews, etc. All the stuff I can generally take or leave. It's like I'm always in a meeting.
>If I was intellectually excited about telling something to do this for me, I'd have gotten into management.
Exactly this. This is the simplest and tersest way of explaining it yet.
I'd go one step higher, we're not builders, we're problem solvers.
Sometimes the problem needs building, sometimes not.
I'm an Engineer, I see a problem and want to solve it. I don't care if I have to write code, have a llm build something new, or maybe even destroy something. I want to solve the problem for the business and move to the next one, most of the time it is having a llm write code though.
That's what I'm doing on my codebases, while I still can. I only use Claude if I need to work on a different team's code that uses it heavily. Nothing quite gets a groan from me like opening up a repo and seeing CLAUDE.md
Same same. Writing the actual code is always a huge motivator behind my side projects. Yes, producing the outcome is important, but the journey taken to get there is a lot of fun for me.
I used Claude Code to implement a OpenAI 4o-vision powered receipt scanning feature in an expense tracking tool I wrote by hand four years ago. It did it in two or three shots while taking my codebase into account.
It was very neat, and it works great [^0], but I can't latch onto the idea of writing code this way. Powering through bugs while implementing a new library or learning how to optimize my test suite in a new language is thrilling.
Unfortunately (for me), it's not hard at all to see how the "builders" that see code as a means to an end would LOVE this, and businesses want builders, not crafters.
In effect, knowing the fundamentals is getting devalued at a rate I've never seen before.
[^0] Before I used Claude to implement this feature, my workflow for processing receipts looked like this: Tap iOS Shortcut, enter the amount, snap a pic of the receipt, type up the merchant, amount and description for the expense, then have the shortcut POST that to my expenses tracking toolkit which, then, POSTs that into a Google Sheet. This feature amounted the need for me to enter the merchant and amount. Unfortunately, it often took more time to confirm that the merchant, amount and date details OpenAI provided were correct (and correct it when details were wrong, which was most of the the time) than it did to type out those details manually, so I just went back to my manual workflow. However, the temptation to just glance at the details and tap "This looks correct" was extremely high, even if the info it generated was completely wrong! It's the perfect analogue to what I've been witnessing throughout the rise of the LLMs.
What I have enjoyed about programming is being able to get the computer to do exactly what I want. The possibilities are bounded by only what I can conceive in my mind. I feel like with AI that can happen faster.
The examples that you and others provide are always fundamentally uninteresting to me. Many, if not most, are some variant of a CRUD application. I have yet seen a single ai generated thing that I personally wanted to use and/or spend time with. I also can't help but wonder what we might have accomplished if we devoted the same amount of resources to developing better tools, languages and frameworks to developers instead of automating the generation of boiler plate and selling developer's own skills back to them. Imagine if open source maintainers instead had been flooded with billions of dollars in capital. What might be possible?
And also, the capacities of llms are almost besides the point. I don't use llms but I have no doubt that for any arbitrary problem that can be expressed textually and is computable in finite time, in the limit as time goes to infinity, an llm will be able to solve it. The more important and interesting questions are what _should_ we build with llms and what should we _not_ build with them. These arguments about capacity are distracting from these more important questions.
Considering how much time developers spend building uninteresting CRUD applications I would argue that if all LLMs can do is speed that process up they're already worth their weight in bytes.
The impression I get from this comment is that no example would convince you that LLMs are worthwhile.
The problem with replying to the proof-demanders is that they'll always pick it apart and find some reason it doesn't fit their definition. You must be familiar with that at this point.
I looked closely enough to confirm there were no architectural mistakes or nasty gotchas. It's code I would have been happy to write myself, only here I got it written on my phone while riding the BART.
See this is a perfect example of OPs statement! I don't care about the lines, I care about the output! It was never about the lines of code.
Your comment makes it very clear there are different viewpoints here. We care about problem->solution. You care about the actual code more than the solution.
>not include leaked secrets, malware installation, stealth cryptomining etc.
Not sure what your point is exactly, but those things don't bother me because I have no control over what happens on others computers. Maybe you insinuate that LLMs will create this, If so, I think you misunderstand the tooling. Or mistake the tooling with the operator.
This gets at the heart of the quality of results issues a lot of people are talking about elsewhere here. Right now, if you treat them as a system where you can tell it what you want and it will do it for you, you're building a sandcastle. Instead of that, also describe the correct data structures and appropriate algorithms to use against them, as well as the particulars of how you want the problem solved, it's a different situation altogether. Like most systems, the quality of output is in some way determined by the quality of input.
There is a strange insistence on not helping the LLM arrive at the best outcome in the subtext to this question a lot of times. I feel like we are living through the John Henry legend in real time
> I got into it for the thrill of defining a problem in terms of data structures and instructions a computer could understand, entering those instructions into the computer, and then watching victoriously while those instructions were executed.
You can still do that with Claude Code. In fact, Claude Code works best the more granular your instructions get.
For some reason this makes me think of a jigsaw puzzle. People usually complete these puzzles because they enjoy the process where on the end you get a picture that you can frame if you want to. Some people seem to want to get the resulting picture. No interest in process at all.
I guess that's the same people who went to all those coding camps during their hay day because they heard about software engineering salaries. They just want the money.
When I last bought a Lego Technic set because I wanted to play with making mechanisms with gears and stuff, I assembled it according to the instructions, which was fun, and then the final result was also cool and I couldn't bear to dismantle it.
IMO, this isn't entirely a "new world" either, it is just a new domain where the conversation amplifies the opinions even more (weird how that is happening in a lot of places)
What I mean by that: you had compiled vs interpreted languages, you had types vs untyped, testing strategies, all that, at least in some part, was a conversation about the tradeoffs between moving fast/shipping and maintainability.
But it isn't just tech, it is also in methodologies and the words use, from "build fast and break things" and "yagni" to "design patterns" and "abstractions"
As you say, it is a different viewpoint... but my biggest concern with where are as industry is that these are not just "equally valid" viewpoints of how to build software... it is quite literally different stages of software, that, AFAICT, pretty much all successful software has to go through.
Much of my career has been spent in teams at companies with products that are undergoing the transition from "hip app built by scrappy team" to "profitable, reliable software" and it is painful. Going from something where you have 5 people who know all the ins and outs and can fix serious bugs or ship features in a few days to something that has easy clean boundaries to scale to 100 engineers of a wide range of familiarities with the tech, the problem domain, skill levels, and opinions is just really hard. I am not convinced yet that AI will solve the problem, and I am also unsure it doesn't risk making it worse (at least in the short term)
Much of my career has been spent in teams at companies with products that are undergoing the transition from "hip app built by scrappy team" to "profitable, reliable software" and it is painful. Going from something where you have 5 people who know all the ins and outs and can fix serious bugs or ship features in a few days to something that has easy clean boundaries to scale to 100 engineers of a wide range of familiarities with the tech, the problem domain, skill levels, and opinions is just really hard. I am not convinced yet that AI will solve the problem, and I am also unsure it doesn't risk making it worse (at least in the short term)
“””
This perspective is crucial. Scale is the great equalizer / demoralizer, scale of the org and scale of the systems. Systems become complex quickly, and verifiability of correctness and function becomes harder. Companies that built from day with AI and have AI influencing them as they scale, where does complexity begin to run up against the limitations of AI and cause regression? Or if all goes well, amplification?
But how can you be a responsible builder if you don't have trust in the LLMs doing the "right thing"? Suppose you're the head of a software team where you've picked up the best candidates for a given project, in that scenario I can see how one is able to trust the team members to orchestrate the implementation of your ideas and intentions, with you not being intimately familiar with the details.
Can we place the same trust in LLM agents? I'm not sure. Even if one could somehow prove that LLM are very reliable, the fact an AI agents aren't accountable beings renders the whole situation vastly different than the human equivalent.
I test all of the code I produce via LLMs, usually doing fairly tight cycles. I also review the unit test coverage manually, so that I have a decent sense that it really is testing things - the goal is less perfect unit tests and more just quickly catching regressions. If I have a lot of complex workflows that need testing, I'll have it write unit tests and spell out the specific edge cases I'm worried about, or setup cheat codes I can invoke to test those workflows out in the UI/CLI.
Trust comes from using them often - you get a feeling for what a model is good and bad at, and what LLMs in general are good and bad at. Most of them are a bit of a mess when it comes to UI design, for instance, but they can throw together a perfectly serviceable "About This" HTML page. Any long-form text they write (such as that About page) is probably trash, but that's super-easy to edit manually. You can often just edit down what they write: they're actually decent writers, just very verbose and unfocused.
I find it similar to management: you have to learn how each employee works. Unless you're in the Top 1%, you can't rely on every employee giving 110% and always producing perfect PRs. Bugs happen, and even NASA-strictness doesn't bring that down to zero.
And just like management, some models are going to be the wrong employee for you because they think your style guide is stupid and keep writing code how they think it should be written.
You don't simply put a body in a seat and get software. There are entire systems enabling this trust: college, resume, samples, referral, interviews, tests and CI, monitoring, mentoring, and performance feedback.
And accountability can still exist? Is the engineer that created or reviewed a Pull Request using Claude Code less accountable then one that used PICO?
> And accountability can still exist? Is the engineer that created or reviewed a Pull Request using Claude Code less accountable then one that used PICO?
The point is that in the human scenario, you can hold the human agents accountable. You cannot do that with AI. Of course, you as the orchestrator of agents will be accountable to someone, but you won't have the benefit of holding your "subordinates" accountable, which is what you do in a human team. IMO, this renders the whole situation vastly different (whether good or bad I'm not sure).
I remember leaving university going into my first engineering job, thinking "Where is all the engineering? All the problem solving and building complex system? All the math and science? Have I been demoted to a lowly programmer?"
Took me a few years to realize that this wasn't a universal feeling, and that many others found the programming tasks more fulfilling than any challenging engineering. I suppose this is merely another manifestation of the same phenomena.
I think he's really getting at something there. I've been thinking about this a lot (in the context of trying to understand the persistent-on-HN skepticism about LLMs), and the framing I came up with[1] is top-down vs. bottom-up dev styles, aka architecting code and then filling in implementations, vs. writing code and having architecture evolve.
Maybe there's an intermediate category: people who like designing software? I personally find system design more engaging than coding (even though I enjoy coding as well). That's different from just producing an opaque artifact that seems to solve my problem.
> > LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.
> I’ve always said I’m a builder even though I’ve also enjoyed programming (but for an outcome, never for the sake of the code)
> This perfectly sums up what I’ve been observing between people like me (builders) who are ecstatic about this new world and programmers who talk about the craft of programming, sometimes butting heads.
That's one take, sure, but it's a specially crafted one to make you feel good about your position in this argument.
The counter-argument is that LLM coding splits up engineers based on those who primarily like engineering and those who like managing.
You're obviously one of the latter. I, OTOH, prefer engineering.
I prefer engineering too, I tried management and I hated it.
It's just the level of engineering we're split on. I like the type of engineering where I figure out the flow of data, maybe the data structures and how they move through the system.
Writing the code to do that is the most boring part of my job. The LLM does it now. I _know_ how to do it, I just don't want to.
It all boils down to communication in a way. Can you communicate what you want in a way others (in this case a language model) understands? And the parts you can't communicate in a human language, can you use tools to define those (linters, formatters, editorconfig)?
I've done all that with actual humans for ... a decade? So applying the exact same thing to a machine is weirdly more efficient, it doesn't complain about the way I like to have my curly braces - it just copies the defined style. With humans I've found out that using impersonal tooling to inspect code style and flaws has a lot less friction than complaining about it in PR reviews. If the CI computer says no, people don't complain, they fix it.
I feel like this is the core issue that will actually stall LLM coding tools short of actually replacing coding work at large.
'Coders' make 'builders' keep the source code good enough so that 'builders' can continue building without breaking what they built.
If 'builders' become x10 productive and 'coders' become unable to keep up with unsurmountable pile of unmaintainable mess that 'builders' proudly churn out, 'bullders' will start to run into impossibility to build further without starting over and over again hoping that agents will be able to get it right this time.
So far I haven't seen it actually be effective at "building" in a work context with any complexity, and this despite some on our team desperately trying to make that the case.
I have! You have to be realistic about the projects. The more irreducible local context it needs, the less useful it will be. Great for greenfield code, oneshots, write once read once run for months.
Agreed. I don’t care for engineering or coding, and would gladly give it up the moment I can. I’m also running a one man business where every hour counts (and where I’m responsible for maintaining every feature).
The fact of the matter is LLMs produce lower quality at higher volumes in more time than it would take to write it myself, and I’m a very mediocre engineer.
I find this seperation of “coding” vs “building” so offensive. It’s basically just saying some people are only concerned with “inputs”, while others with “outputs”. This kind of rhetoric is so toxic.
It’s like saying LLM art is separating people into people who like to scribble, and people who like to make art.
I mean it’s closer, but I don’t think it’s right to equate commissioning an artist with paying a multi-billion dollar corporation to steal from artists.
These tools are just lazy shortcuts. And that’s fine, there’s no problem with taking the lazy way. I’m never going to put in the time to learn to draw, so it’s cool there’s an option for me.
I just take ire with pretending it’s something grand and refined, or spitting in the face of the ones who are willing to put in the work
I can't argue that. The scale was already imbalanced as well, and vibe coding has lowered the bar even more, so the gap will continue to grow for now.
I'm just saying that LLMs aren't causing the divide. Accelerating yes, but I think simply equating AI usage to poor quality is wrong. Craftsmen now have a powerful tool as well, to analyze, nitpick, and refactor in ways that were previously difficult to justify.
It also seems premature for so many devs to jump to hardline "AI bad" stances. So far the tech is improving quite well. We may not be able to 1-shot much of quality yet, but it remains to be seen if that will hold.
Personally, I have hopes that AI will eventually push code quality much higher than it's ever been. I might be totally wrong of course, but to me it feels logical that computers would be very good at writing computer programs once the foundation is built.
I think the division is more likely tied to writing. You have to fundamentally change how you do your job, from one of writing a formal language for a compiler to one of writing natural language for a junior-goldfish-memory-allstar-developer, closer to management then to contributor.
This distinction to me separates the two primary camps
Yeah, I think this is a bit of insight I had not realized / been able to word correctly yet. There's developers who can let Claud go at it, and be fearless about it like me (though I mostly do it for side projects, but WOW) and then there's developers who will use it like a hammer or axe to help cut down or mold whatever is in their path.
I think both approaches are okay, the biggest thing for me is the former needs to test way more, and review the code more, as developers we don't read code enough, with the "prompt and forget" approach we have a lot of free time we could spend reading the code, asking the model to refactor and refine the code. I am shocked when I hear about hundreds of thousands of lines in some projects. I've rebuilt Beads from the ground up and I'm under 10 lines of code.
So we're going to have various level of AI Code Builders if you will: Junior, Mid, Senior, Architect. I don't know if models will ever pick up the slack for Juniors any time soon. We would need massive context windows for models, and who will pay for that? We need a major AI breakthrough to where the cost goes down drastically before that becomes profitable.
We have services deployed globally serving millions of customers where rigor is really important.
And we have internal users who're building browser extensions with AI that provide valuable information about the interface they're looking at including links to the internal record management, and key metadata that's affecting content placement.
These tools could be handed out on Zip drives in the street and it would just show our users some of the metadata already being served up to them, but it's amazing to strip out 75% of the process of certain things and just have our user (in this case though, it's one user who is driving all of this, so it does take some technical inclination) build out these tools that save our editors so much time when doing this before would have been months and months and months of discovery and coordination and designs that probably wouldn't actually be as useful in the end after the wants of the user are diluted through 18 layers of process.
I like building, but I don't fool myself into thinking it can be done by taking shortcuts. You could build something that looks like a house for half the cost but it won't be structurally sound. That's why I care about the details. Someone has to.
There's more to it than just coding Vs building though.
For a long time in my career now I've been in a situation where I'd be able to build more if I was willing to abstract myself and become a slide-merchant/coalition-builder. I don't want to do this though.
Yet, I'm still quite an enthusiastic vibe-coder.
I think it's less about coding Vs building and more about tolerance for abstraction and politics. And I don't think there are that many people who are so intolerant of abstraction that they won't let agents write a bunch of code for them.
I’ve heard something similar: “there are people who enjoy the process, and people who enjoy the outcome”. I think this saying comes moreso from artistic circles.
I’ve always considered myself a “process” person, I would even get hung-up on certain projects because I enjoyed crafting them so much.
LLM’s have taken a bit of that “process” enjoyment from me, but I think have also forced some more “outcome” thinking into my head, which I’m taking as a positive.
The new LLM centered workflow is really just a management job now.
Managers and project managers are valuable roles and have important skill sets. But there's really very little connection with the role of software development that used to exist.
It's a bit odd to me to include both of these roles under a single label of "builders", as they have so little in common.
I enjoy both and have ended up using AI a lot differently than vibe coders. I rarely use it for generating implementations, but I use it extensively for helping me understand docs/apis and more importantly, for debugging. AI saves me so much time trying to figure out why things aren’t working and in code review.
I deliberately avoid full vibe coding since I think doing so will rust my skills as a programmer. It also really doesn’t save much time in my experience. Once I have a design in mind, implementation is not the hard part.
To me this is similar to car enthusiasms. Some people absolutely love to build their project car, it's a major part of the hobby for them. Others just love the experience of driving, so they buy ready cars or just pay someone to work on the car.
agree completely. I used to be (and still would love to be) a process person, enjoying hand writing bulletproof artisanal code. Switching to startups many years ago gave me a whole new perspective, and its been interesting the struggle between writing code and shipping. Especially when you dont know how long the code you are writing will actually live. LLMs are fantastic in that space.
> I enjoy both and have ended up using AI a lot differently than vibe coders. I rarely use it for generating implementations, but I use it extensively for helping me understand docs/apis and more importantly, for debugging. AI saves me so much time trying to figure out why things aren’t working and in code review.
I had felt like this and still do but man, at some point, I feel like the management churn feels real & I just feel suffering from a new problem.
Suppose, I actually end up having services literally deployed from a single prompt nothing else. Earlier I used to have AI write code but I was interested in the deployment and everything around it, now there are services which do that really neatly for you (I also really didn't give into the agent hype and mostly used browsers LLM)
Like on one hand you feel more free to build projects but the whole joy of project completely got reduced.
I mean, I guess I am one of the junior dev's so to me AI writing code on topics I didn't know/prototyping felt awesome.
I mean I was still involved in say copy pasting or looking at the code it generates. Seeing the errors and sometimes trying things out myself. If AI is doing all that too, idk
For some reason, recently I have been disinterested in AI. I have used it quite a lot for prototyping but I feel like this complete out of the loop programming just very off to me with recent services.
I also feel like there is this sense of if I buy for some AI thing, to maximally extract "value" out of it.
I guess the issue could be that I can have vague terms or have a very small text file as input (like just do X alternative in Y lang) and I am now unable to understand the architectural decisions and the overwhelmed-ness out of it.
Probably gonna take either spec-driven development where I clearly define the architecture or development where I saw something primagen do recently which is that the AI will only manipulate code of that particular function, (I am imagining it for a file as well) and somehow I feel like its something that I could enjoy more because right now it feels like I don't know what I have built at times.
When I prototype with single file projects using say browser for funsies/any idea. I get some idea of what the code kind of uses with its dependencies and functions names from start/end even if I didn't look at the middle
A bit of ramble I guess but the thing which kind of is making me feel this is that I was talking to somebody and shwocasing them some service where AI + server is there and they asked for something in a prompt and I wrote it. Then I let it do its job but I was also thinking how I would architect it (it was some detect food and then find BMR, and I was thinking first to use any api but then I thought that meh it might be hard, why not use AI vision models, okay what's the best, gemini seems good/cheap)
and I went to the coding thing to see what it did and it actually went even beyond by using the free tier of gemini (which I guess didn't end up working could be some rate limit of my own key but honestly it would've been the thing I would've tried too)
So like, I used to pride myself on the architectural decisions I make even if AI could write code faster but now that is taken away as well.
I really don't want to read AI code so much so honestly at this point, I might as well write code myself and learn hands on but I have a problem with build fast in public like attitude that I have & just not finding it fun.
I feel like I should do a more active job in my projects & I am really just figuring out what's the perfect way to use AI in such contexts & when to use how much.
I retired from paid sw dev work in 2020 when COVID arrived.
I’ve worked on my small projects since with all development by hand. I’d followed the rise of AI, but not used it.
Late last year I started a project that included reverse engineering some firmware that runs on an Intel 8096 based embedded processor. I’d never worked on that processor before. There are tools available, but they cost many $. So, I started to think about a simple disassembler.
2 weeks ago we decided to try Claude to see what it could do. We now have a disassembler, assembler and a partially working emulator. No doubt there are bugs and missing features and the code is a bit messy, but boy has it sped up the work.
One thing did occur to me. Vendors of small utilities could be in trouble. For example I needed to cut out some pages from a pdf. I could have found a tool online(I’m sure there are several), write one myself. However, Claude quickly performed the task.
This is a mix of the “in the future, everyone will have a 3D printer at home and just 3D print random parts they need” and “anyone can trivially build Dropbox with rsync themselves” arguments.
Tech savvy users who know how to use LLMs aren’t how vendors of small utilities stay in business.
They stay in business because they sell things to users who are truly clueless with tech (99% of the population, which can’t even figure out the settings app on their phone), and solid distribution/marketing is how you reach those users and can’t really be trivially hacked because everyone is trying to hack it.
Or they stay in business because they offer some sort of guarantee (whether legal, technical, or other) that the users don’t want to burden themselves with because they have other, more important stuff to worry about.
I don't know. It's one thing to tell Joe or Jane User to "Get an FTP account, mount it locally with curlftpfs, and then use SVN or CVS on the mounted filesystem." But if Joe or Jane can just cut-and-paste that advice into a prompt and get their own personal Dropbox...
Except when that new Dropbox fails Joe or Jane on that Saturday evening, their only recourse is to ask the AI for help, and the AI starts spinning “oh yeah, mmm, I think I found where the problem is. Cut and paste these debugging lines in that function and let me know what the output is…”
Meanwhile, this year, that happens less often than it did last year... and it actually isn't how AI-assisted development works at all. Agentic models do the cutting-and-pasting by themselves, evaluate the results by themselves, and almost always succeed at fixing the problem by themselves.
> Vendors of small utilities could be in trouble. For example I needed to cut out some pages from a pdf. I could have found a tool online(I’m sure there are several), write one myself. However, Claude quickly performed the task.
Definitely. Making small, single-purpose utilities with LLMs is almost as easy these days as googling for them on-line - much easier, in fact, if you account for time spent filtering out all the malware, adware, "to finish the process, register an account" and plain broken "tools" that dominate SERP.
Case in point, last time my wife needed to generate a few QR codes for some printouts for an NGO event, I just had LLM make one as a static, single-page client-side tool and hosted it myself -- because that was the fastest way to guarantee it's fast, reliable, free of surveillance economy bullshit, and doesn't employ URL shorteners (surprisingly common pattern that sometimes becomes a nasty problem down the line; see e.g. a high-profile case of some QR codes on food products leading to porn sites after shortlink got recycled).
Having an LLM spit out a few hundred lines of HTML and JavaScript is not a colossal waste of resources, it's equivalent to running a microwave for a couple of seconds.
Not to mention, my little tool is using much less electricity running than just about anything else I could easily find on-line, simply by the virtue of being minimal, and completely free of superfluous visual bullshit, upsells, tracking, telemetry, and other such secondary aspects of anything people publish and advertise for others to use.
Don't get the anti-AI propaganda get to you too much. Inference is cheap on the margin.
Consider: there are models capable (if barely) of doing this job, that you can run locally, on a upper-mid-range PC with high-end consumer GPU. Take that as a baseline, assume it takes a day instead of an hour because of inference speed, tally up total electricity cost. It's not much. Won't boil oceans any more than people playing AAA video games all day will.
Sure, the big LLMs from SOTA vendors use more GPUs/TPUs for inference, but this means they finish much faster. Plus, commercial vendors have lots of optimizations (batch processing, large caches, etc.), and data centers are much more power-efficient than your local machine, so "how much it'd cost me in power bill if I did it locally" is a good starting estimate.
It's not even "users", just the user. Nice thing about LLMs is that it's cheap to develop small tools tailor-made for audience of few, or in this case, just one.
1) This was for my wife. She is not proficient in Linux or CLI in general, and (like ~all white collar workers these days) works almost exclusively in browser tools (exception being pre-O365 versions of Word and Excel we keep running on her laptop because she prefers them).
2) I never heard of `qrencode` CLI tool until today. For some reason I didn't even consider it might exist (maybe because last time I checked, which was many years ago, there was none).
4) Even if I knew about it, I'd still have to build a web frontend for it, and I'd need a proper server for it, which I'd then have to maintain properly, and secure it against the `qrencode` call becoming an attack vector.
So frankly, for my specific problem, my solution is strictly better.
> You realize that stamina is a core bottleneck to work
There has been a lot of research that shows that grit is far more correlated to success than intelligence. This is an interesting way to show something similar.
AIs have endless grit (or at least as endless as your budget). They may outperform us simply because they don't ever get tired and give up.
Full quote for context:
Tenacity. It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day. It's a "feel the AGI" moment to watch it struggle with something for a long time just to come out victorious 30 minutes later. You realize that stamina is a core bottleneck to work and that with LLMs in hand it has been dramatically increased.
>They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day.
"Listen, and understand! That Terminator is out there! It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop... ever, until you are dead!"
If I tell it to implement something it will sometimes declare their work done before it's done. But if I give Claude Code a verifiable goal like making the unit tests pass it will work tirelessly until that goal is achieved. I don't always like the solution, but the tenacity everyone is talking about is there
> but the tenacity everyone is talking about is there
I always double-check if it doesn't simply exclude the failing test.
The last time I had this, I discovered it later in the process. When I pointed this out to the LLM, it responded, that it acknowledged thefact of ignoring the test in CLAUDE.md, and this is justified because [...]. In other words, "known issue, fuck off"
> If you ever work with LLMs you know that they quite frequently give up.
If you try to single shot something perhaps. But with multiple shots, or an agent swarm where one agent tells another to try again, it'll keep going until it has a working solution.
Yeah exactly this is a scope problem, actual input/output size is always limited> I am 100% sure CC etc are using multiple LLM calls for each response, even though from the response streaming it looks like just one.
Context matters, for an LLM just like a person. When I wrote code I'd add TODOs because we cannot context switch to another problem we see every time.
But you can keep the agent fixated on the task AND have it create these TODOs, but ultimately it is your responsibility to find them and fix them (with another agent).
Using LLMs to clean those up is part of the workflow that you're responsible for (... for now). If you're hoping to get ideal results in a single inference, forget it.
I realized a long time ago that I’m better at computer stuff not because I’m smarter but because I will sit there all day and night to figure something out while others will give up. I always thought that was my superpower in the job industry but now I’m not so sure if it will transfer to getting AI to do what I need done…
Same, I barely made it through Engineering school, but would stay up all night figuring out everything a computer could do (before the internet).
I did it because I enjoyed it, and still do. I just do it with LLMs now. There is more to figure out than ever before and things get created faster than I have time to understand them.
LLM should be enabling this, not making it more depressing.
Me three. I was not as smart as many of my peers in uni but I freakin LOVE the subject matter and I also love studying and feeling that progress of learning, which led me to put in the huge number of hours necessary to be successful and have a positive attitude the whole time.
But even tenacity is not enough. You also need an internal timer. "Wait a minute, this is taking too long, it shouldn't be this hard. Is my overall approach completely wrong?"
I'm not sure AIs have that. Humans do, or at least the good ones do. They don't quit on the problem, but they know when it's time to consider quitting on the approach.
> AIs have endless grit (or at least as endless as your budget).
That is the only thing he doesn't address: the money it costs to run the AI. If you let the agents loose, they easily burn north of 100M tokens per hour. Now at $25/1M tokens that gets quickly expensive. At some point, when we are all drug^W AI dependent, the VCs will start to cash in on their investments.
LLMs do not have grit or tenacity. Tenacity doesn't desribe a machine that doesn't need sleep or experience tiredness, or stress. Grit doesn't describe a chatbot that will tirelessly spew out answers and code because it has no stake or interest in the result, never perceives that it doesn't know something, and never reflects on its shortcomings.
> What happens to the "10X engineer" - the ratio of productivity between the mean and the max engineer? It's quite possible that this grows a lot.
I was thinking about this the other day as relates to the DevOps movement.
The DevOps movement started as a way to accelerate and improve the results of dev<->ops team dynamics. By changing practices and methods, you get acceleration and improvement. That creates "high-performing teams", which is the team form of a 10x engineer. Whether or not you believe in '10x engineers', a high-performing team is real. You really can make your team deploy faster, with fewer bugs. You have to change how you all work to accomplish it, though.
To get good at using AI for coding, you have to do the same thing: continuous improvement, changing workflows, different designs, development of trust through automation and validation. Just like DevOps, this requires learning brand new concepts, and changing how a whole team works. This didn't get adopted widely with DevOps because nobody wanted to learn new things or change how they work. So it's possible people won't adapt to the "better" way of using AI for coding, even if it would produce a 10x result.
If we want this new way of working to stick, it's going to require education, and a change of engineering culture.
This is an interesting thing that I'm contemplating. I also do believe that (perhaps with very few exceptions) there are no "10x engineers" by themselves, but engineers that thrive 10x more in a context or another (like, I'm sure Jeff Dean is an absolutely awesome engineer - but if you took him out of Google and plugged him into IBM - would he have had the same impact?)
With that in mind - I think one very unexplored area is "how to make the mixed AI-human teams successful". Like, I'm fairly convinced AI changes things, but to get to the industrialization of our craft (which is what management seems to want - and, TBH, something that makes sense from an economic pov), I feel that some big changes need to happen, and nobody is talking about that too much. What are the changes that need to happen? How do we change things, if we are to attempt such industrialization?
I'm pretty happy with Copilot in VS Code. Type what change I want Claude to make in the Copilot panel, and then use the VS Code in context diffs to accept or reject the proposed changes. While being able to make other small changes on my own.
So I think this tracks with Karpathy's defense of IDEs still being necessary ?
Has anyone found it practical to forgo IDEs almost entirely?
I've found copilot chat is able to do everything I need. I tried the Claude plugin for vscode and it was a noticeably worse experience for me.
Mind you copilot has only supported agent mode relatively recently.
I really like the way copilot does changes in such a way you can accept or reject and even revert to point in time in the chat history without using git. Something about this just fits right with how my brain works.
Using Claude plugin just felt like I had one hand tied behind my back.
I find Claude Code in VS Code is sometimes horribly inefficient. I tell it to replace some print-statements with proper logging in the one file I have open and it first starts burning tokens to understand the codebase for the 13th time today, despite not needing to and having it laid out in the CLAUDE.md already.
I have been assigning issues to copilot in Github. It will then create a pull request and work on and report back on the issue in the PR. I will pull the code and make small changes locally using VSCode when needed.
But what I like about this setup is that I have almost all the context I need to review the work in a single PR. And I can go back and revisit the PR if I ever run into issues down the line. Plus you can run sessions in parallel if needed, although I don't do that too much.
Are you letting it run your tests and run little snippets of code to try them out (like "python -c 'import module; print(module.something())'") or are you just using it to propose diffs for you to accept or reject?
This stuff gets a whole lot more interesting when you let it start making changes and testing them by itself.
This stuff is a little messy and opaque, but the performance of the same model in different harnesses depends a lot on how context is managed. The last time I tried Copilot, it performed markedly worse for similar tasks compared to Claude Code. I suspect that Copilot was being very aggressive in compressing context to save on token cost, but I'm not 100% certain about this.
Also note that with Claude models, Copilot might allocate a different number of thinking tokens compared to Claude Code.
Things may have changed now compared to when I tried it out, these tools are in constant flux. In general I've found that harnesses created by the model providers (OpenAI/Codex CLI, Anthropic/Claude Code, Google/Gemini CLI) tend to be better than generalist harnesses (cheaper too, since you're not paying a middleman).
Different harnesses and agentic environments produce different results from the same model. Claude Code and Cursor are the best IME and Copilot is by far the worst.
Claude Code is a CLI tool which means it can do complete projects in a single command. Also has fantastic tools for scaffolding and harnessing the code. You can define everything from your coding style to specific instructions for designing frontpages, integrating payments, etc.
Implementation differences do matter. I haven't found Copilot to have as many issues as people say it does, but they are there. Their Gemini implementation is unusable, for example, and it's not because of the underlying models. They work fine in other harnesses.
> Tenacity. It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day.
This is true to an extent for sure and they will go much longer than most engineers without getting "tired", but I've def seen both sonnet and opus give up multiple times. They've updated code to skip tests they couldn't get to pass, given up on bugs they couldn't track down, etc. I literally had it ask "could we work on something else and come back to this"
The glorified autocomplete. Why would the LLM "work on something else then get back on this", is it's subconscious going to solve the problem during that time?
But because people say it, it says it too. Making sense is optional.
Ive found that clearing the context and getting back to it later actually DOES work. When you restart, your personal context is cleared and you might be better at describing the problem you are solving in a more informationally dense way.
LLM coding splits up engineers based on those who primarily like building and those who primarily like code reviews and quality assessment. I definitely don’t love the latter (especially when reviewing decisions not made by a human with whom I can build long-term personal rapport).
After certain experience threshold of making things from scratch, “coding” (never particularly liked that term) has always been 99% building, or architecture, and I struggle to see how often a well-architected solution today, with modern high-level abstractions, requires so much code that you’d save significant time and effort by not having to just type, possibly with basic deterministic autocomplete, exactly what you mean (especially considering you would have to also spend time and effort reviewing whatever was typed for you if you used a non-deterministic autocomplete).
"those who primarily like code reviews and quality assessment" -- I don't love those. In fact I find it tedious and love it when I can work on my own without them.
Except after 25 years of working I know how imperative they are, how easily a project can disintegrate into confused silos, and am frustrated as heck with these tools being pushed without attention to this problem.
See, I don't take it that extreme: LLMs make fantastic, never-before seen quality autocompletes. I hacked together a Neovim plugin that prompts an LLM to "finish this function" on command, and it's a big time save for the menial plumbing type operations. Think things like "this api I use expects JSON that encodes some subset of SQL, I want all the dogs with Ls in their name that were born on a Tuesday". Given an example of such API (or if the documentation ended up in its training), LLMs will consistently one-shot stuff like that.
Asking it to do entire projects? Dumb. You end up with spaghetti, unless you hand-hold it to a point that you might as well be using my autocomplete method.
HN should ban any discussion on “things I learned playing with AI” that don’t include direct artifacts of the thing built.
We’re about a year deep into “AI is changing everything” and I don’t see 10x software quality or output.
Now don’t get me wrong I’m a big fan of AI tooling and think it does meaningfully increase value. But I’m damn tired of all the talk with literally nothing to show for it or back it up.
> It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day. It's a "feel the AGI" moment to watch it struggle with something for a long time just to come out victorious 30 minutes later.
This is true... Equally I've seen it dive into a rabbit hole, make some changes that probably aren't the right direction... and then keep digging.
This is way more likely with Sonnet, Opus seems to be better at avoiding it. Sonnet would happily modify every file in the codebase trying to get a type error to go away. If I prompt "wait, are you off track?" it can usually course correct. Again, Opus seems way better at that part too.
Admittedly this has improved a lot lately overall.
I don't understand why anyone finds it interesting that a machine, or chatbot, never tires or gets demoralized. You have to anthromorphize the LLM before you can even think of those possibilities. A tractor never tires or gets demoralized either, because it can't. Chatbots don't "dive into a rabbit hole ... and then keep digging" because they have superhuman tenacity, they do it because that's what software does. If I ask my laptop to compute the millionth Fibonacci number it doesn't sigh and complain, and I don't think it shows any special qualities unless I compare it to a person given the same job.
You're a machine. You're literally a wet, analog device converting some forms of energy into other forms just like any other machine as you work, rest, type out HN comments, etc. There is nothing special about the carbon atoms in your body -- there's no metadata attached to them marking them out as belonging to a Living Person. Other living-person-machines treat "you" differently than other clusters of atoms only because evolution has taught us that doing so is a mutually beneficial social convention.
So, since you're just a machine, any text you generate should be uninteresting to me -- correct?
Alternatively, could it be that a sufficiently complex and intricate machine can be interesting to observe in its own right?
If humans are machines, they are still a subset of machines and they (among other animals) are the only ones who can be demotivated and so it is still a mistake to assume an entirely different kind of machine would have those properties.
>Other living-person-machines treat "you" differently than other clusters of atoms only because evolution has taught us that doing so is a mutually beneficial social convention
Evolution doesn't "teach" anything. It's just an emergent property of the fact that life reproduces (and sometimes doesn't). If you're going to have this radically reductionist view of humanity, you can't also treat evolution as having any kind of agency.
"If humans are machines, they are still a subset of machines and they (among other animals) are the only ones who can be demotivated and so it is still a mistake to assume an entirely different kind of machine would have those properties."
Wrong level of abstraction. And not the definition of machine.
I might feel awe or amazement at what human-made machines can do -- the reason I got into programming. But I don't attribute human qualities to computers or software, a category error. No computer ever looked at me as interesting or tenacious.
I would agree that OAIs GPT-5 family of models is a phase change over GPT-4.
In the ChatGPT product this is not immediately obvious and many people would strongly argue their preference for 4. However, once you introduce several complex tools and make tool calling mandatory, the difference becomes stark.
I've got an agent loop that will fail nearly every time on GPT-4. It works sometimes, but definitely not enough to go to production. GPT-5 with reasoning set to minimal works 100% of the time. $200 worth of tokens and it still hasn't failed to select the proper sequence of tools. It sometimes gets the arguments to the tools incorrect, but it's always holding the right ones now.
I was very skeptical based upon prior experience but flipping between the models makes it clear there has been recent stepwise progress.
I'll probably be $500 deep in tokens before the end of the month. I could barely go $20 before I called bullshit on this stuff last time.
Pretty sure there wasn't extensive training on tooling beforehand. I mean, god, during GPT-3 even getting a reliable json output was a battle and there were dedicated packages for json inference.
Agree with Karpathy's take. Finally a down to Earth analysis from a respected source in the AI space. I guess I'll be using slopocalypse a lot more now :)
> I am bracing for 2026 as the year of the slopacolypse across all of github, substack, arxiv, X/instagram, and generally all digital media
It has arrived. Github will be most affected thanks to git-terrorists at Apna College refusing to take down that stupid tutorial. IYKYK.
He ran Teslas ML division, but still doesnt know what a simple kalman filter is (in the sense where he claimed that lidar would be hard to integrate with cameras).
The Kalman filter examples I've seen always involve estimating a very simple quantity, like the location of a single 3D point, from noisy sensors. It's clear how multiple estimates can be combined into a new estimate.
I'd guess that cameras on a self-driving car are trying to estimate something much more complex, something like 3D surfaces labeled with categories ("person", "traffic light", etc.). It's not obvious to me how estimates of such things from multiple sensors and predictions can be sensibly and efficiently combined to produce a better estimate. For example, what if there is a near red object in front of a distant red background, so that the camera estimates just a single object, but the lidar sees two?
1. make prediction on the next state change of some measurable n dimentional quantity, and estimate the covariance matrix across those n dimentions, which describe essentially a probability that the i-th dimention is going to increase (or decrease) with j-th dimention, where i and j are between 0 and n (indices of the vector)
2. Gather sensor data (that can be noisy), and reconcile the predicted measurement with the measured to get the best guess. The covariance matrix acts as a kind of weight for each of the elements
3. Update the covariance matrix based on the measurements in previous step.
You can do this for any vector of numbers. For example, instead of tracking individual objects, you can have a grid where each element represents a physical object that the car should not drive into, with a value representing certainty of that object being there. Then when you combine sensor reading, you still can use your vision model but that model would be enhanced by what lidar detects, both in terms of seeing things that camera doesn't pick up and rejecting things that aren't there.
And the concept is generic enough to where you can set up a system to be able to plug in any additional sensor with its own noise, and it all works out in the end. This is used all the You can even extend the concept past Gaussian noise and linearity, there are a number of other filters that deal with that, broadly under the umbrella of sensor fusion.
The problem is that Karpathy is more of a computer scientist, so he is on his Code 2.0 train of having ML models do everything. I dunno if he is like that himself or Musks "im smarter than everyone else that came before me" rubbed off.
And of course when you think like that, its going to be difficult to integrate lidar into the model. But the problem with that thinking is that forward inference LLM is not AI, and it will never ever be able to drive a car well compared to a true "reasoning" AI with feedback loops.
> It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day. It's a "feel the AGI" moment to watch it struggle with something for a long time just to come out victorious 30 minutes later.
Somewhere, there are GPUs/NPUs running hot. You send all the necessary data, including information that you would never otherwise share. And you most likely do not pay the actual costs. It might become cheaper or it might not, because reasoning is a sticking plaster on the accuracy problem. You and your business become dependent on this major gatekeeper. It may seem like a good trade-off today. However, the personal, professional, political and societal issues will become increasingly difficult to overlook.
This quote stuck out to me as well, for a slightly different reason.
The “tenacity” referenced here has been, in my opinion, the key ingredient in the secret sauce of a successful career in tech, at least in these past 20 years. Every industry job has its intricacies, but for every engineer who earned their pay with novel work on a new protocol, framework, or paradigm, there were 10 or more providing value by putting the myriad pieces together, muddling through the ever-waxing complexity, and crucially never saying die.
We all saw others weeded out along the way for lacking the tenacity. Think the boot camp dropouts or undergrads who changed majors when first grappling with recursion (or emacs). The sole trait of stubbornness to “keep going” outweighs analytical ability, leetcode prowess, soft skills like corporate political tact, and everything else.
I can’t tell what this means for the job market. Tenacity may not be enough on its own. But it’s the most valuable quality in an employee in my mind, and Claude has it.
There is an old saying back home: an idiot never tires, only sweats.
Claude isn't tenacious. It is an idiot that never stops digging because it lacks the meta cognition to ask 'hey, is there a better way to do this?'. Chain of thought's whole raison d'etre was so the model could get out of the local minima it pushed itself in. The issue is that after a year it still falls into slightly deeper local minima.
This is fine when a human is in the loop. It isn't what you want when you have a thousand idiots each doing a depth first search on what the limit of your credit card is.
> it lacks the meta cognition to ask 'hey, is there a better way to do this?'.
Recently had an AI tell me this code (that it wrote) is a mess and suggested wiping it and starting from scratch with a more structure plan. That seems to hint at some meta cognition outlines
Haha, it has the human developer traits of thinking all old code is garbage, failing to identify oneself as the dummy who wrote this particular code, and wanting to start from scratch.
Perhaps. I've had LLMs tell me some code is deeply flawed garbage that should be rewritten about code that exact same LLM wrote minutes before. It could be a sign of deep meta cognition, or it might be due to some cognitive gaps where it has no idea why it did something a minute ago and suddenly has a different idea.
This is not a fair criticism. There is _nobody_ there, so you can't be saying 'code the exact same LLM wrote minutes before'. There is no 'exact same LLM' and no ideas for it to have, you're trying to make sense of sparkles off the surface of a pond. There's no 'it' to have an idea and then a different idea, much less deep meta cognition.
I'm not sure we disagree. I was pushing back against the idea that suggesting a rewrite of some code implies meta cognition abilities on the part of the LLM. That seems like weak evidence to me.
I asked Claude to analyze something and report back. It thought for a while said “Wow this analysis is great!” and then went back to thinking before delivering the report. They’re auto-sycophantic now!
I mean, not always. I've seen Claude step back and reconsider things after hitting a dead end, and go down a different path. There are also workflows, loops that can increase the likelihood of this occurring.
This is a major concern for junior programmers. For many senior ones, after 20 (or even 10) years of tenacious work, they realize that such work will always be there, and they long ago stopped growing on that front (i.e. they had already peaked). For those folks, LLMs are a life saver.
At a company I worked for, lots of senior engineers become managers because they no longer want to obsess over whether their algorithm has an off by one error. I think fewer will go the management route.
(There was always the senior tech lead path, but there are far more roles for management than tech lead).
I feel like if you're really spending a ton of time on off by one errors after twenty years in the field you haven't actually grown much and have probably just spent a ton of time in a single space.
Otherwise you'd be senior staff to principle range and doing architecture, mentorship, coordinating cross team work, interviewing, evaluating technical decisions, etc.
I got to code this week a bit and it's been a tremendous joy! I see many peers at similar and lower levels (and higher) who have more years and less technical experience and still write lots of code and I suspect that is more what you're talking about. In that case, it's not so much that you've peaked, it's that there's not much to learn and you're doing a bunch of the same shit over and over and that's of course tiring.
I think it also means that everything you interact with outside your space does feel much harder because of the infrequency with which you have interacted with it.
If you've spent your whole career working the whole stack from interfaces to infrastructure then there's really not going to be much that hits you as unfamiliar after a point. Most frameworks recycle the same concepts and abstractions, same thing with programming languages, algorithms, data management etc.
But if you've spent most of your career in one space cranking tickets, those unknown corners are going to be as numerous as the day you started and be much more taxing.
Aren't you still better off than the rest of us who found what they love + invested decades in it before it lost its value. Isn't it better to lose your love when you still have time to find a new one?
Depends on if their new love provides as much money as their old one, which is probably not likely. I'd rather have had those decades to stash and invest.
A lot of pre-faang engineers dont have the stash you're thinking about. What you meant was "right when I found a lucrative job that I love". What was going on in tech these last 15 years, unfortunately, probably was once in a lifetime.
It's crazy to think back in the 80's programmers had "mild" salaries despite programming back then being worlds more punishing. No libraries, no stack exchange, no forums, no endless memory and infinite compute. If you had a challenging bug you better also be proficient in reading schematics and probing circuits.
It has not lost its value yet, but the future will shift that value. All of the past experience you have is an asset for you to move with that shift. The problem will not be you losing value, it will be you not following where the value goes.
It might be a bit more difficult to love where the shift goes, but that is no different than loving being a artist which often shares a bed with loving being poor. What will make you happier?
Especially on the topic of value! We are all intuitively aware that value is highly contextual, but get in a knot trying to rationalize value long past genuine engagement!
Imagine a senior dev who just approves PRs, approves production releases, and prioritizes bug reports and feature requests. LLM watches for errors ceaslessly, reports an issue. Senior dev reviews the issue and assigns a severity to it. Another LLM has a backlog of features and errors to go solve, it makes a fix and submits a PR after running tests and verifying things work on its end.
Why are we pretending like the need for tenacity will go away? Certain problems are easier now. We can tackle larger problems now that also require tenacity.
Even right at this very moment where we have a high-tenacity AI, I'd argue that working with the AI -- that is to say, doing AI coding itself and dealing with the novel challenges that brings requires a lot of stubborn persistence.
Fittingly, George Hinton toiled away for years in relative obscurity before finally being recognized for his work. I was always quite impressed by his "tenacity".
So although I don't think he should have won the Nobel Prize because not really physics, I felt his perseverance and hard work should merit something.
I still find in these instances there's at least a 50% chance it has taken a shortcut somewhere: created a new, bigger bug in something that just happened not to have a unit test covering it, or broke an "implicit" requirement that was so obvious to any reasonable human that nobody thought to document it. These can be subtle because you're not looking for them, because no human would ever think to do such a thing.
Then even if you do catch it, AI: "ah, now I see exactly the problem. just insert a few more coins and I'll fix it for real this time, I promise!"
The value extortion plan writes itself. How long before someone pitches the idea that the models explicitly almost keep solving your problem to get you to keep spending? Would you even know?
That’s far-fetched. It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise. High value to user + lower compute costs = better pricing power and better margins overall.
What are the details of this? I'm not playing dumb, and of course I've noticed the decline, but I thought it was a combination of losing the battle with SEO shite and leaning further and further into a 'give the user what you think they want, rather than what they actually asked for' philosophy.
As recently as 15 years ago, Google _explicitly_ stated in their employee handbook that they would NOT, as a matter of principle, include ads in the search results. (Source: worked there at that time.)
Now, they do their best to deprioritize and hide non-ad results...
Only if you are paying per token on the API. If you are paying a fixed monthly fee then they lose money when you need to burn more tokens and they lose customers when you can’t solve your problems within that month and max out your session limits and end up with idle time which you use to check if the other providers have caught up or surpassed your current favourite.
> It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise. High value to user + lower compute costs = better pricing power and better margins overall.
It's only in the interests of the model builders to do that IFF the user can actually tell that the model is giving them the best value for a single dollar.
The free market proposition is that competition (especially with Chinese labs and grok) means that Anthropic is welcome to do that. They're even welcome to illegally collude with OpenAi such that ChatGPT is similarly gimped. But switching costs are pretty low. If it turns out I can one shot an issue with Qwen or Deepseek or Kimi thinking, Anthropic loses not just my monthly subscription, but everyone else's I show that too. So no, I think that's some grade A conspiracy theory nonsense you've got there.
It’s not that crazy. It could even happen by accident in pursuit of another unrelated goal. And if it did, a decent chunk of the tech industry would call it “revealed preference” because usage went up.
LLMs became sycophantic and effusive because those responses were rated higher during RLHF, until it became newsworthy how obviously eager-to-please they got, so yes, being highly factually correct and "intelligent" was already not the only priority.
This is a good point. For example if you have access to a bunch of slot machines, one of them is guaranteed to hit the jackpot. Since switching from one slot machine to another is easy, it is trivial to go from machine to machine until you hit the big bucks. That is why casinos have such large selections of them (for our benefit).
"for our benefit" lol! This is the best description of how we are all interacting with LLMs now. It's not working? Fire up more "agents" ala gas town or whatever
To be clear I don't think that's what they're doing intentionally. Especially on a subscription basis, they'd rather me maximize my value per token, or just not use them. Lulling users into using tokens unproductively is the worst possible option.
The way agents work right now though just sometimes feels that way; they don't have a good way of saying "You're probably going to have to figure this one out yourself".
As a rational consumer, how would you distinguish between some intentional "keep pulling the slot machine" failure rate and the intrinsic failure rate?
I feel like saying "the market will fix the incentives" handwaves away the lack of information on internals. After all, look at the market response to Google making their search less reliable - sure, an invested nerd might try Kagi, but Google's still the market leader by a long shot.
I was thinking more of deliberate backdoor in code. RCE is an obvious example, but another one could be bias. "I'm sorry ma'am, computer says you are ineligable for a bank account." These ideas aren't new. They were there in 90s already when we still thought about privacy and accountability regarding technology, and dystopian novels already described them long, long ago.
> These can be subtle because you're not looking for them
After any agent run, I'm always looking the git comparison between the new version and the previous one. This helps catch things that you might otherwise not notice.
You are using it wrong, or are using a weak model if your failure rate is over 50%. My experience is nothing like this. It very consistently works for me. Maybe there is a <5% chance it takes the wrong approach, but you can quickly steer it in the right direction.
A lot of people are getting good results using it on hard things. Obviously not perfect, but > 50% success.
That said, more and more people seem to be arriving at the conclusion that if you want a fairly large-sized, complex task in a large existing codebase done right, you'll have better odds with Codex GPT-5.2-Codex-XHigh than with Claude Code Opus 4.5. It's far slower than Opus 4.5 but more likely to get things correct, and complete, in its first turn.
I think a lot of it comes down to how well the user understands the problem, because that determines the quality of instructions and feedback given to the LLM.
For instance, I know some people have had success with getting claude to do game development. I have never bothered to learn much of anything about game development, but have been trying to get claude to do the work for me. Unsuccessful. It works for people who understand the problem domain, but not for those who don't. That's my theory.
It works for hard problems when the person already solves it and just needs the grunt work done
It also works for problems that have been solved a thousand times before, which impresses people and makes them think it is actually solving those problems
Which matches what they are. They're first and foremost pattern recognition engines extraordinaire. If they can identify some pattern that's out of whack in your code compared to something in the training data, or a bug that is similar to others that have been fixed in their training set, they can usually thwack those patterns over to your latent space and clean up the residuals. If comparing pattern matching alone, they are superhuman, significantly.
"Reasoning", however, is a feature that has been bolted on with a hacksaw and duct tape. Their ability to pattern match makes reasoning seem more powerful than it actually is. If your bug is within some reasonable distance of a pattern it has seen in training, reasoning can get it over the final hump. But if your problem is too far removed from what it has seen in its latent space, it's not likely to figure it out by reasoning alone.
Exactly. I go back to a recent ancestor of LLMs, seq2seq. Its purpose was to translate things. Thats all. That needed representation learning and an attention mechanism, and it lead to some really freaky emergent capabilities, but its trained to trainslate language.
And thats exactly what its good for. It works great if you already solve a tough problem and provide it the solution in natural language, because the program is already there, it just needs to translate it to python.
Anything more than that that might emerge from this is going to be unreliable sleight of next-token-prediction at best.
We need a new architectural leap to have these things reason, maybe something that involves reinforcement learning at the token represention level, idk. But scaling the context window and training data arent going to cut it
it's meant in the literal sense but with metaphorical hacksaws and duct tape.
Early on, some advanced LLM users noticed they could get better results by forcing insertion of a word like "Wait," or "Hang on," or "Actually," and then running the model for a few more paragraphs. This would increase the chance of a model noticing a mistake it made.
Not the core foundation model. The foundation model still only predicts the next token in a static way. The reasoning is tacked onto the instructGPT style finetuning step and its done through prompt engineering. Which is the shittiest way a model like this could have been done, and it shows
I mean problems not worth solving, because theyve already been solved. If you need to just do the grunt work of retrieving the solution to a trite and worn out problem from the models training data, then they work great
But if you want to do interesting things, like all the shills keep trying to claim they do. Then this wont do it for you. You have to do it for it
> But if you want to do interesting things, like all the shills keep trying to claim they do
I don't know where this is coming from. I've seen some over-enthusiastic hype for sure, but most of the day-to-day conversations I see aren't people saying they're curing cancer with Claude, they're people saying they're automating their bread and butter tasks with great success.
> But anyway, it already costs half compared to last year
You could not have bought Claude Opus 4.5 at any price one year ago I'm quite certain. The things that were available cost half of what they did then, and there are new things available. These are both true.
I'm agreeing with you, to be clear.
There are two pieces I expect to continue: inference for existing models will continue to get cheaper. Models will continue to get better.
Three things, actually.
The "hitting a wall" / "plateau" people will continue to be loud and wrong. Just as they have been since 2018[0].
As a user of LLMs since GPT-3 there was noticeable stagnation in LLM utility after the release of GPT-4. But it seems the RLHF, tool calling, and UI have all come together in the last 12 months. I used to wonder what fools could be finding them so useful to claim a 10x multiplier - even as a user myself. These days I’m feeling more and more efficiency gains with Claude Code.
That's the thing people are missing, the models plateaued a while ago, still making minor gains to this day, but not huge ones. The difference is now we've had time to figure out the tooling. I think there's still a ton of ground to cover there and maybe the models will improve given that the extra time, but I think it's foolish to consider people who predicted that completely wrong. There are also a lot of mathematical concerns that will cause problems in the near and distant future. Infinite progress is far from a given, we're already way behind where all the boosters thought we'd be my now.
I believe Sam Altman, perhaps the greatest grifter in today’s Silicon Valley, claimed that software engineering would be obsolete by the end of last year.
> The "hitting a wall" / "plateau" people will continue to be loud and wrong. Just as they have been since 2018[0].
Everybody who bet against Moore's Law was wrong ... until they weren't.
And AI is the reaction to Moore's Law having broken. Nobody gave one iota of damn about trying to make programming easier until the chips couldn't double in speed anymore.
This is exactly backwards: Dennard scaling stopped. Moore’s Law has continued and it’s what made training and running inference on these models practical at interactive timescales.
You are technically correct. The best kind of correct.
However, most people don't know the difference between the proper Moore's Law scaling (the cost of a transistor halves every 2 years) which is still continuing (sort of) and the colloquial version (the speed of a transistor doubles every 2 years) which got broken when Dennard scaling ran out. To them, Moore's Law just broke.
Nevertheless, you are reinforcing my point. Nobody gave a damn about improving the "programming" side of things until the hardware side stopped speeding up.
And rather than try to apply some human brainpower to fix the "programming" side, they threw a hideous number of those free (except for the electricity--but we don't mention that--LOL) transistors at the wall to create a broken, buggy, unpredictable machine simulacrum of a "programmer".
(Side note: And to be fair, it looks like even the strong form of Moore's Law is finally slowing down, too)
If you can turn a few dollars of electricity per hour into a junior-level programmer who never gets bored, tired, or needs breaks, that fundamentally changes the economics of information technology.
And in fact, the agentic looped LLMs are executing much better than that today. They could stop advancing right now and still be revolutionary.
i don't think it is harmless or we are incentivising people to just say whatever they want without any care for truth. people's reputations should be attached to their predictions.
Some people definitely do but how do they go and address it? A fresh example in that it addresses pure misinformation. I just screwed up and told some neighbors garbage collection was delayed for a day because of almost 2ft of snow. Turns out it was just food waste and I was distracted checking the app and read the notification poorly.
I went back to tell them (do not know them at all just everyone is chattier digging out of a storm) and they were not there. Feel terrible and no real viable remedy. Hope they check themselves and realize I am an idiot. Even harder on the internet.
That's not true. Many technologies get more expensive over time, as labor gets more expensive or as certain skills fall by the wayside, not everything is mass market. Have you tried getting a grandfather clock repaired lately?
Repairing grandfather clocks isn't more expensive now because it's gotten any harder; it's because the popularity of grandfather clocks is basically nonexistent compared to anything else to tell time.
of course it's silly to talk about manufacturing methods and yield and cost efficiency without having an economy to embed all of this into, but ... technology got cheaper means that we have practical knowledge of how to make cheap clocks (given certain supply chains, given certain volume, and so and so)
we can make very cheap very accurate clocks that can be embedded into whatever devices, but it requires the availability of fabs capable of doing MEMS components, supply materials, etc.
you can look at a basket of goods that doesn't have your specific product and compare directly
but inflation is the general price level increase, this can be used as a deflator to get the price of whatever product in past/future money amount to see how the price of the product changed in "real" terms (ie. relative to the general price level change)
Instead of advancing tenuous examples you could suggest a realistic mechanism by which costs could rise, such as a Chinese advance on Taiwan, effecting TSMC, etc.
You will get a different bridge. With very different technology. Same as "I can't repair my grandfather clock cheaply".
In general, there are several things that are true for bridges that aren't true for most technology:
* Technology has massively improved, but most people are not realizing that. (E.g. the Bay Bridge cost significantly more than the previous version, but that's because we'd like to not fall down again in the next earthquake)
* We still have little idea how to reason about the cost of bridges in general. (Seriously. It's an active research topic)
* It's a tiny market, with the major vendors forming an oligopoly
* It's infrastructure, not a standard good
* The buy side is almost exclusively governments.
All of these mean expensive goods that are completely non-repeatable. You can't build the same bridge again. And on top of that, in a distorted market.
But sure, the cost of "one bridge, please" has gone up over time.
This seems largely the same as any other technology. The prices of new technologies go down initially as we scale up and optimize it's production, but as soon as demand fades, due to newer technology or whatever, the cost of that technology goes up again.
I don't think the question is answerable in a meaningful way. Bridges are one-off projects with long life spans, comparing cost over time requires a lot of squinting just so.
Time-keeping is vastly cheaper. People don't want grandfather clocks. They want to tell time. And they can, more accurately, more easily, and much cheaper than their ancestors.
The chart shows that they’re right though. Newer models cost more than older models.
Sure they’re better but that’s moot if older models are not available or can’t solve the problem they’re tasked with.
On the link you shared, 4o vs 3.5 turbo price per 1m tokens.
There’s no such thing as ”same task by old model”, you might get comparable results or you might not (and this is why the comparison fail, it’s not a comparison), the reason you pick the newer models is to increase chances of getting a good result.
> The dataset for this insight combines data on large language model (LLM) API prices and benchmark scores from Artificial Analysis and Epoch AI. We used this dataset to identify the lowest-priced LLMs that match or exceed a given score on a benchmark. We then fit a log-linear regression model to the prices of these LLMs over time, to measure the rate of decrease in price. We applied the same method to several benchmarks (e.g. MMLU, HumanEval) and performance thresholds (e.g. GPT-3.5 level, GPT-4o level) to determine the variation across performance metrics
This should answer. In your case, GPT-3.5 definitely is cheaper per token than 4o but much much less capable. So they used a model that is cheaper than GPT-3.5 that achieved better performance for the analysis.
Not according to their pricing table. Then again I’m not sure what OpenAI model versions even mean anymore, but I would assume 5.2 is in the same family as 5 and 5.2-pro as 5-pro
Not true. Bitcoin has continued to rise in cost since its introduction (as in the aggregate cost incurred to run the network).
LLMs will face their own challenges with respect to reducing costs, since self-attention grows quadratically. These are still early days, so there remains a lot of low hanging fruit in terms of optimizations, but all of that becomes negligible in the face of quadratic attention.
I don't think computation is going to become more expensive, but there are techs that have become so: Nuclear power plants. Mobile phones. Oil extraction.
(Oil rampdown is a survival imperative due to the climate catastrophe so there it's a very positive thing of course, though not sufficient...)
There are plenty of technologies that have not become cheaper, or at least not cheap enough, to go big and change the world. You probably haven't heard of them because obviously they didn't succeed.
Supersonic jet engines, rockets to the moon, nuclear power plants, etc. etc. all have become more expensive. Superconductors were discovered in 1911, and we have been making them for as long as we have been making transistors in the 1950s, yet superconductors show no sign of becoming cheaper any time soon.
There have been plenty of technologies in history which do not in fact become cheaper. LLMs are very likely to become such, as I suspect their usefulness will be superseded by cheaper (much cheaper in fact) specialized models.
With optimizations and new hardware, power is almost a negligible cost. You can get 5.5M tokens/s/MW[1] for kimi k2(=20M/KWH=181M tokens/$) which is 400x cheaper than current pricing. It's just Nvidia/TSMC/other manufacturers eating up the profit now because they can. My bet is that China will match current Nvidia within 5 years.
Electricity is negligible but the dominant cost is the hardware depreciation itself. Also inference is typically memory bandwidth bound so you are limited by how fast you can move weights rather than raw compute efficiency.
Yes, because the margin is like 80% for Nvidia, and 80% again for the manufacturers like Samsung and TSMC. Once the fixed cost like R and D is amortized the same node technology and hardware capacity could be just few single digit percent of current.
> And you most likely do not pay the actual costs.
This is one of the weakest anti AI postures. "It's a bubble and when free VC money stops you'll be left with nothing". Like it's some kind of mystery how expensive these models are to run.
You have open weight models right now like Kimi K2.5 and GLM 4.7. These are very strong models, only months behind the top labs. And they are not very expensive to run at scale. You can do the math. In fact there are third parties serving these models for profit.
The money pit is training these models (and not that much if you are efficient like chinese models). Once they are trained, they are served with large profit margins compared to the inference cost.
OpenAI and Anthropic are without a doubt selling their API for a lot more than the cost of running the model.
> You send all the necessary data, including information that you would never otherwise share.
I've never sent the type of data that isn't already either stored by GitHub or a cloud provider, so no difference there.
> And you most likely do not pay the actual costs.
So? Even if costs double once investor subsidies stop, that doesn't change much of anything. And the entire history of computing is that things tend to get cheaper.
> You and your business become dependent on this major gatekeeper.
Not really. Switching between Claude and Gemini or whatever new competition shows up is pretty easy. I'm no more dependent on it than I am on any of another hundred business services or providers that similarly mostly also have competitors.
To me this tenacity is often like watching someone trying to get a screw into board using a hammer.
There’s often a better faster way to do it, and while it might get to the short term goal eventually, it’s often created some long term problems along the way.
I don’t understand this pov. Unfortunately, id pay 10k mo for my cc sub. I wish I could invest in anthropic, they’re going to be the most profitable company on earth
My agent struggled for 45 minutes because it tried to do `go run` on a _test.go file, which the compiler repeatedly exited after posting an error message that files named like this cannot be executed using the run command.
So yeah, that wasted a lot of GPU cycles for a very unimpressive result, but with a renewed superficial feeling of competence
We can observe how much generic inference providers like deepinfra or together-ai charge for large SOTA models. Since they are not subsidized and they don’t charge 7x of OpenAI, that means OAI also doesn’t have outrageously high per-token costs.
AI genius discover brute forcing... what a time to be alive. /s
Like... bro that's THE foundation of CS. That's the principle of The bomb in Turing's time. One can still marvel at it but it's been with us since the beginning.
> Atrophy. I've already noticed that I am slowly starting to atrophy my ability to write code manually...
> Largely due to all the little mostly syntactic details involved in programming, you can review code just fine even if you struggle to write it.
Until you struggle to review it as well. Simple exercise to prove it - ask LLM to write a function in familiar programming language, but in the area you didn't invest learning and coding yourself. Try reviewing some code involving embedding/SIMD/FPGA without learning it first.
I think understanding stellar processes and then using that understanding to theorize about other observations is a skill. My point was that observing can be a fantastic way to build a skill.. not all skills, but certainly some skills. Learning itself is as much an observation as a practice.
> It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day. It's a "feel the AGI" moment to watch it struggle with something for a long time just to come out victorious 30 minutes later.
The bits left unsaid:
1. Burning tokens, which we charge you for
2. My CPU does this when I tell it to do bogosort on a million 32-bit integers, it doesn't mean it's a good thing
> There seems to be zero output from they guy for the past 2 years (except tweets)
Well, he made Nanochat public recently and has been improving it regularly [1].
This doesn't preclude that he might be working on other projects that aren't public yet (as part of his work at Eureka Labs).
He's building Eureka Labs[1], an AI-first education company (can't wait to use it). He's both a strong researcher[2] and an unusually gifted technical communicator. His recent videos[3] are excellent educational material.
More broadly though: someone with his track record sharing firsthand observations about agentic coding shouldn't need to justify it by listing current projects. The observations either hold up or they don't.
Helper scripts for APIs for applications and tools I know well. LLMs have made my work bearable. Many software providers expose great apis, but expert use cases require data output/input that relies on 50-500 line scripts. Thanks to the models post gpt4.5 most requirements are solvable in 15 minutes when they could have taken multiple workdays to write and check by hand. The only major gap is safe ad-hoc environments to run these in. I provide these helper functions for clients that would love to keep the runtime in the same data environment as the tool, but not all popular software support FaaS style environments that provide something like a simple python env.
I don’t know, but it’s interesting that he and many others come up with this “we should act like LLMs are junior devs”. There is a reason why most junior devs work on fairly separate parts of products, most of the time parts which can be removed or replaced easily, and not an integral part of products: because their code is usually quite bad. Like every few lines contains issues, suboptimal solutions, and full with architectural problems. You basically never trust junior devs with core product features. Yet, we should pretend that an “LLM junior dev” is somehow different. These just signal to me that these people don’t work on serious code.
This is the first question I ask, and every time I get the answer of some monolith that supposedly solves something. Imo, this is completely fine for any personal thing, I am happy when someone says they made an API to compare weekly shopping prices from the stores around them, or some recipe, this makes sense.
However more often than not, someone is just building a monolithic construction that will never be looked at again. For example, someone found that HuggingFace dataloader was slow for some type of file size in combination with some disk.
What does this warrant? A 300000+ line non-reviewed repo to fix this issue. Not a 200-line PR to HuggingFace, no you need to generate 20% of the existing repo and then slap your thing on there.
For me this is puzzling, because what is this for? Who is this for? Usually people built these things for practice, but now its generated, so its not for practice because you made very little effort on it. The only thing I can see that its some type of competence signaling, but here again, if the engineer/manager looking knows that this is generated, it does not have the type of value that would come with such signaling. Either I am naive and people still look at these repos and go "whoa this is amazing", or it's some kind of induced egotrip/delusion where the LLM has convinced you that you are the best builder.
I wish the people who wrote this let us know what king of codebases they are working on. They seem mostly useless in a sufficiently large codebase especially when they are messy and interactions aren't always obvious. I don't know how much better Claude is than ChatGPT, but I can't get ChatGPT to do much useful with an existing large codebase.
I've been working in the mobile space since 2009, though primarily as a designer and then product manager. I work in kinda a hybrid engineering/PM job now, and have never been a particularly strong programmer. I definitely wouldn't have thought I could make something with that polish, let alone in 3 months.
Not sure if it's an American pronunciation thing, but I had to stare at that long and hard to see the problem and even after seeing it couldn't think of how you could possibly spell the correct word otherwise.
The pronunciation of the first r with a y sound has always been one of two possible standards, in fact "February" is a re-Latinizing spelling but English doesn’t like the br-r sound so it naturally dissimilates to by-r.
Claude and Codex are CLI tools you use to give the LLM context about the project on your local machine or dev environment. The fact that you're using the name "ChatGPT" instead of Codex leads me to believe you're talking about using the web-based ChatGPT interface to work on a large codebase, which is completely beside the point of the entire discussion. That's not the tool anyone is talking about here.
It's important to understand that he's talking about a specific set of models that were release around november/december, and that we've hit a kind of inflection point in model capabilities. Specifically Anthropic's Opus 4.5 model.
I never paid any attention to different models, because they all felt roughly equal to me. But Opus 4.5 is really and truly different. It's not a qualitative difference, it's more like it just finally hit that quantitative edge that allows me to lean much more heavily on it for routine work.
I highly suggest trying it out, alongside a well-built coding agent like the one offered by Claude Code, Cursor, or OpenCode. I'm using it on a fairly complex monorepo and my impressions are much the same as Karpathy's.
I had the same reaction. So when people were talking about this model back in December, I brushed it off. It wasn't until a couple weeks ago that I decided to try it out, and I immediately saw the difference.
My opinion isn't based on what other people are saying, it's my own experience as a fairly AI-skeptical person. Again, I highly suggest you give it an honest try and decide for yourself.
I'm not sure how big your repos are but I've been effective working with repos that have thousands of files and tens of thousands of lines of code.
If you're just prototyping it will hit wall when things get unwieldy but that's normally a sign that you need to refactor a bit.
Super strict compiler settings, static analysis, comprehensive tests, and documentation help a lot. As does basic technical design. After a big feature is shipped I do a refactor cycle with the LLM where we do a comprehensive code review and patch things up. This does require human oversight because the LLMs are still lacking judgement on what makes for good code design.
The places where I've seen them be useless is working across repositories or interfacing with things like infrastructure.
It's also very model-dependent. Opus is a good daily driver but Codex is much better are writing tests for some reason. I'll often also switch to it for hard problems that Claude can't solve. Gemini is nice for 'I need a prototype in the next 10 minutes', especially for making quick and dirty bespoke front-ends where you don't care about the design just the functionality.
Almost always, notes like these are going to be about greenfield projects.
Trying to incorporate it in existing codebases (esp when the end user is a support interaction or more away) is still folly, except for closely reviewed and/or non-business-logic modifications.
That said, it is quite impressive to set up a simple architecture, or just list the filenames, and tell some agents to go crazy to implement what you want the application to do. But once it crosses a certain complexity, I find you need to prompt closer and closer to the weeds to see real results. I imagine a non-technical prompter cannot proceed past a certain prototype fidelity threshold, let alone make meaningful contributions to a mature codebase via LLM without a human engineer to guide and review.
I'm using it on a large set of existing codebases full of extremely ugly legacy code, weird build systems, tons of business logic and shipping directly to prod at neckbreaking growth over the last two years, and it's delivering the same type of value that Karpathy writes about.
It's been especially helpful in explaining and understanding arcane bits of legacy code behavior my users ask about. I trigger Claude to examine the code and figure out how the feature works, then tell it to update the documentation accordingly.
I read through it, scanning sections that seem uncontroversial and reading more closely sections that talk about things I'm less sure about. The output cites key lines of code, which are faster to track down and look at than trying to remember where in a large codebase to look.
Inconsistencies also pop up in backtesting, for example if there's a point that the llm answers different ways in multiple iterations, that's a good candidate to improve docs on.
Similar to a coworker's work, there's a certain amount of trust in the competency involved.
Contract? These docs are information answering user queries. So if you use a chatbot to generate them, I'd like to be reasonably sure they aren't laden with the fabricated misinformation for which these chatbots are famous.
It's a very reasonable concern. My solution is to have the bot classify what the message is talking about as a first pass, and have a relatively strict filtering about what it responds to.
For example, I have it ignore messages about code freezes, because that's a policy question that probably changes over time, and I have it ignore urgent oncall messages, because the asker there probably wants a quick response from a human.
But there's a lot of questions in the vein of "How do I write a query for {results my service emits}", how does this feature work, where automation can handle a lot (and provide more complete answers than a human can off the top of their head)
These models do well changing brownfield applications that have tests because the constraints on a successful implementation are tight. Their solutions can be automatically augmented by research and documentation.
I don't exactly disagree with this but I have seen models simply deleting the tests, or updating the tests to pass and declaring the failures were "unrelated to my changes", so it helpfully fixed them
I’ve had to deal with this a handful of times. You just have to make it restore the test, or keep trying to pass a suite of explicit red-green method tests it wrote earlier.
For me, in just the golang server instance and the core functional package, `cloc` reports over 40k lines of code, not counting other supporting packages. I spent the last week having Claude rip out the external auth system and replace it with a home-grown one (and having GPT-codex review its changes). If anything, Claude makes it easier on me as a solo founder with a large codebase. Rather than having to re-familiarize myself with code I wrote a year ago, I describe it at a high level, point Claude to a couple of key files, and then tell it to figure out what it needs to do. It can use grep, language server, and other tools to poke around and see what's going on. I then have it write an "epic" in markdown containing all the key files, so that future sessions already know the key files to read.
I really enjoyed the process. As TFA says, you have to keep a close eye on it. But the whole process was a lot less effort, and I ended up doing mor than I would otherwise have done.
I don't know how big sufficiently large codebase is, but we have a 1mil loc Java application, that is ~10years old, and runs POS systems, and Claude Code has no issues with it. We have done full analyses with output details each module, and also used it to pinpoint specific issues when described. Vibe coding is not used here, just analysis.
At my dayjob my team uses it on our main dashboard, which is a pretty large CRUD application. The frontend (Vue) is a horrible mess, as it was originally built by people who know just enough to be dangerous. Over time people have introduced new standards without cleaning up the old code - for example, we have three or four different state management techologies.
For this the LLM struggles a bit, but so does a human. The main issues are it messes up some state that it didnt realise was used elsewhere, and out test coverage is not great. We've seen humans make exactly the same kind of mistakes. We use MCP for Figma so most of the time it can get a UI 95% done, just a few tweaks needed by the operator.
On the backend (Typescript + Node, good test coverage) it can pretty much one-shot - from a plan - whatever feature you give it.
We use opus-4.5 mostly, and sometimes gpt-5.2-codex, through Cursor. You aren't going to get ChatGPT (the web interface) to do anything useful, switch to Cursor, Codex or Claude Code. And right now it is worth paying for the subscription, you don't get the same quality from cheaper or free models (although they are starting to catch up, I've had promising results from GLM-4.7).
I had never used Swift before that and was able to use AI to whip up a fairly full-featured and complex application with a decent amount of code. I had to make some cross-cutting changes along the way as well that impacted quite a few files and things mostly worked fine with me guiding the AI. Mind you this was a year ago so I can only imagine how much better I would fare now with even better AI models. That whole month was spent not only on coding but on learning Swift enough to fix problems when AI started running into circles and then learning about Xcode profiler to optimize the application for speed and improving perf.
> They seem mostly useless in a sufficiently large codebase especially when they are messy and interactions aren't always obvious.
What type of documents do you have explaining the codebase and its messy interactions, and have you provided that to the LLM?
Also, have you tried giving someone brand new to the team the exact same task and information you gave to the LLM, and how effective were they compared to the LLM?
> I don't know how much better Claude is than ChatGPT, but I can't get ChatGPT to do much useful with an existing large codebase.
As others have pointed out, from your comment, it doesn't sound like you've used a tool dedicated for AI coding.
(But even if you had, it would still fail if you expect LLMs to do stuff without sufficient context).
The code base I work on at $dayjob$ is legacy, has few files with 20k lines each and a few more with around 10k lines each. It's hard to find things and connect dots in the code base. Dont think LLMs able to navigate and understand code bases of that size yet. But have seen lots of seemingly large projects shown here lately that involve thousands of files and millions of lines of code.
I’ve found that LLMs seem to work better on LLM-generated codebases.
Commercial codebases, especially private internal ones, are often messy. It seems this is mostly due to the iterative nature of development in response to customer demands.
As a product gets larger, and addresses a wider audience, there’s an ever increasing chance of divergence from the initial assumptions and the new requirements.
We call this tech debt.
Combine this with a revolving door of developers, and you start to see Conway’s law in action, where the system resembles the organization of the developers rather than the “pure” product spec.
With this in mind, I’ve found success in using LLMs to refactor existing codebases to better match the current requirements (i.e. splitting out helpers, modularizing, renaming, etc.).
Once the legacy codebase is “LLMified”, the coding agents seem to perform more predictably.
YMMV here, as it’s hard to do large refactors without tests for correctness.
(Note: I’ve dabbled with a test first refactor approach, but haven’t gone to the lengths to suggest it works, but I believe it could)
Claude by default, unless I tell it not to, will write stuff like:
// we need something to be true
somethingPasses = something()
if (!somethingPasses) {
return false
}
// we need somethingElse to be true
somethingElsePasses = somethingElse()
if (!somethingElsePasses) {
return false
}
return true
instead of the very simple boolean logic that could express this in one line, with the "this code does what it obviously does" comments added all over the place.
generally unless you tell it not to, it does things in very verbose ways that most humans would never do, and since there's an infinite number of ways that it can invent absurd verbosity, it is hard to preemptively prompt against all of them.
to be clear, I am getting a huge amount of value out of it for executing a bunch of large refactors and "modernization" of a (really) big legacy codebase at scale and in parallel. but it's not outputting the sort of code that I see when someone prompts it "build a new feature ...", and a big part of my prompts is screaming at it not to do certain things or to refuse the task if it at any point becomes unsure.
Yeah to be clear it will have the same issues as a flyby contributor if prompted to.
Meaning if you ask it “handle this new condition” it will happily throw in a hacky conditional and get the job done.
I’ve found the most success in having it reason about the current architecture (explicitly), and then to propose a set of changes to accomplish the task (2-5 ways), review, and then implement the changes that best suit the scope of the larger system.
The failure mode is missing constraints, not “coding skill”. Treat the model as a generator that must operate inside an explicit workflow: define the invariant boundaries, require a plan/diff before edits, run tests and static checks, and stop when uncertainty appears. That turns “hacky conditional” behaviour into controlled change.
Right. Each context window is a partial view, so it cannot “know the codebase” unless you supply stable artefacts. Treat project state as inputs: invariants, interfaces, constraints, and a small set of must-keep facts. Then force changes through a plan and a diff, and gate with tests and checks. That turns context limits into a controlled boundary instead of a surprise.
If you have a ChatGPT account, there's nothing stopping you from installing codex cli and using your chatgpt account with it. I haven't coded with ChatGPT for weeks. Maybe a month ago I got utility out of coding with codex and then having ChatGPT look at my open IDE page to give comments, but since 5.2 came out, it's been 100% codex.
I think its not Claude code per se itself but rather the (Opus 4.5 model?) or something in an agentic workflow.
I tried a website which offered the Opus model in their agentic workflow & I felt something different too I guess.
Currently trying out Kimi code (using their recent kimi 2.5) for the first time buying any AI product because got it for like 1.49$ per month. It does feel a bit less powerful than claude code but I feel like monetarily its worth it.
Y'know you have to like bargain with an AI model to reduce its pricing which I just felt really curious about. The psychology behind it feels fascinating because I think even as a frugal person, I already felt invested enough in the model and that became my sunk cost fallacy
Shame for me personally because they use it as a hook to get people using their tool and then charge next month 19$ (I mean really Cheaper than claude code for the most part but still comparative to 1.49$)
Also I never see anyone talking about code reviews, which is one of the primary ways that software engineering departments manage liability. We fired someone recently because they couldn’t explain any of the slop they were trying to get merged. Why tf would I accept the liability of managing code that someone else can’t even explain?
I guess this is fine when you don’t have customers or stakeholders that give a shit lol.
1. Write good documentation, architecture, how things work, code styling, etc.
2. Put your important dependencies source code in the same directory. E.g. put a `_vendor` directory in the project, in it put the codebase at the same tag you're using or whatever: postgres, redis, vue, whatever.
3. Write good plans and requirements. Acceptance criteria, context, user stories, etc. Save them in markdown files. Review those multiple times with LLMs trying to find weaknesses. Then move to implementation files: make it write a detailed plan of what it's gonna change and why, and what it will produce.
4. Write very good prompts. LLMs follow instructions well if they are clear "you should proactively do X", is a weak instruction if you mean "you must do X".
5. LLMs are far from perfect, and full of limits. Karpathy sums their cons very well in his long list. If you don't know their limits you'll mismanage the expectations and not use them when they are a huge boost and waste time on things they don't cope well with. On top of that: all LLMs are different in their "personality", how they adhere to instruction, how creative they are, etc.
I successfully use Claude Code in a large complex codebase. It's Clojure, perhaps that helps (Clojure is very concise, expressive and hence token-dense).
Perhaps it's harder to "do Closure wrong" than it is to do JavaScript or Python or whatever other extremely flexible multi-paradigm high-level language
Having spent 3 years of my career working with Clojure, I think it actually gives you even more rope to shoot yourself with than Python/JS.
E.g. macros exist in Clojure but not Python/JS, and I've definitely been plenty stumped by seeing them in the codebase. They tend to be used in very "clever" patterns.
On the other hand, I'm a bit surprised Claude can tackle a complex Clojure codebase. It's been a while since I attempted using an LLM for Clojure, but at the time it failed completely (I think because there is relatively little training data compared to other mainstream languages). I'll have to check that out myself
I've been trying Claude on my large code base today. When I give it the requirements I'd give an engineer and so "do it" it just writes garbage that doesn't make sense and doesn't seem to even meet the requirements (if it does I can't follow how - though I'll admit to giving up before I understood what it did, and I didn't try it on a real system). When I forced it to step back and do tiny steps - in TDD write one test of the full feature - it did much better - but then I spent the next 5 hours adjusting the code it wrote to meet our coding standards. At least I understand the code, but I'm not sure it is any faster (but it is a lot easier to see things wrong than come up with green field code).
Which is to say you have to learn to use the tools. I've only just started, and cannot claim to be an expert. I'll keep using them - in part because everyone is demanding I do - but to use them you clearly need to know how to do it yourself.
I've been playing around with the "Superpowers" [0] plugin in Claude Code on a new small project and really like it. Simple enough to understand quickly by reading the GitHub repo and seems to improve the output quality of my projects.
There's basically a "brainstorm" /slash command that you go back and forth with, and it places what you came up with in docs/plans/YYYY-MM-DD-<topic>-design.md.
Then you can run a "write-plan" /slash command on the docs/plans/YYYY-MM-DD-<topic>-design.md file, and it'll give you a docs/plans/YYYY-MM-DD-<topic>-implementation.md file that you can then feed to the "execute-plan" /slash command, where it breaks everything down into batches, tasks, etc, and actually implements everything (so three /slash commands total.)
There's also "GET SHIT DONE" (GSD) [1] that I want to look at, but at first glance it seems to be a bit more involved than Superpowers with more commands. Maybe it'd be better for larger projects.
Which is equal parts praise and damnation. Claude Code does do a lot of nice things that people just kind of don't bother for time cost / reward when writing TUIs that they've probably only done because they're using AI heavily, but equally it has a lot of underbaked edges (like accidentally shadowing the user's shell configuration when it tries to install terminal bindings for shift-enter even though the terminal it's configuring already sends a distinct shift-enter result), and bugs (have you ever noticed it just stop, unfinished?).
What do you even mean by "ChatGPT"? Copy pasting code into chatgpt.com?
AI assisted coding has never been like that, which would be atrocious. The typical workflow was using Cursor with some model of your choice (almost always an Anthropic model like sonnet before opus 4.5 released). Nowadays (in addition to IDEs) it's often a CLI tool like Claude Code with Opus or Codex CLI with GPT Codex 5.2 high/xhigh.
Why is plain Claude code outdated? I thought that’s what most people are using right now that are AI forward. Is it Ralph loops now that’s the new thing?
Plain Claude Code doesn’t have enough scaffolding to handle large projects
At a base level, people are “upgrading” their Claude Code with custom skills and subagents - all text files saved in .claude/agents|skills.
You can also use their new tasks primitive to basically run a Ralph-like loop
But at the edges, people are using multiple instances, each handling different aspects in parallel - stuff like Gas Town
Tbf you can still get a lot of mileage out of vanilla Claude Code. But I’ve found that even adding a simple frontend design skill improves the output substantially
Well, it may consume the AI environment. Maybe even the internet. It's not going to consume a PC with g++, though (at least if the PC doesn't update g++ any more once g++ starts accepting AI contributions).
There may come a point where having a "survivor machine" with auto-update turned off may be a really good idea.
I already do this, in the form of survivor machines made to do initial coding on a retro platform so the result will translate across all possible platforms. Got to, as I'm an Apple coder primarily, so if I want to target older machines I can only do it through a survivor machine: support is always pruned out of Xcode and it would be insane to try and patch it to keep everything in scope.
> - What does LLM coding feel like in the future? Is it like playing StarCraft? Playing Factorio? Playing music?
Starcraft and Factorio are exactly what it is not. Starcraft has a loooot of micro involved at any level beyond mid level play, despite all the "pro macros and beats gold league with mass queens" meme videos. I guess it could be like Factorio if you're playing it by plugging together blueprint books from other people but I don't think that's how most people play.
At that level of abstraction, it's more like grand strategy if you're to compare it to any video game? You're controlling high level pushes and then the units "do stuff" and then you react to the results.
I think the StarCraft analogy is fine, you have to compare it not to macro and micro RTS play, but to INDIVIDUAL UNITS. For your whole career until now, you have been a single Zergling or Probe. Now you are the Commander.
Except that pro starcraft player still micro-manage every single Zergling or probe when necessary, while vibe coders just right click on the ennemy base and hope it'll go well
It's like the Victoria 3 combat system. You just send an army and a general to a given front and let them get to work with no micro. Easy! But of course some percentage of the time they do something crazy like deciding to redeploy from your existential Franco-Prussian war front to a minor colonial uprising...
I sometimes wonder about the similarities between this paradigm switch (coding -> vibe coding) and when the industry switched from writing assembler to using high-level languages. I both cases we switched from having to specify every posibble implementation detail to focusing more on higher level concepts and letting the machine work out the rest. Maybe in the future instead of sharing source code, we will share prompts that we used to create a program. Similarly how different compilers produce different assembly now, "compiling" prompts with different agent/model would give different results. Maybe in the future an analog for "optimizing compiler" would emerge for agents, which would turn the (working) slop into something more clean.
> the ratio of productivity between the mean and the max engineer? It's quite possible that this grows *a lot*
I have a professor who has researched auto generated code for decades and about six months ago he told me he didn't think AI would make humans obsolete but that it was like other incremental tools over the years and it would just make good coders even better than other coders. He also said it would probably come with its share of disappointments and never be fully autonomous. Some of what he said was a critique of AI and some of it was just pointing out that it's very difficult to have perfect code/specs.
I think in less than a year writing code manually will be akin to doing arithmetic problems by hand. Sure you can still code manually, but it's going to be a lot faster to use an LLM (calculator).
People keep using these analogies but I think these are fundamentally different things.
1. hand arithmetic -> using a calculator
2. assembly -> using a high level language
3. writing code -> making an LLM write code
Number 3 does not belong. Number 3 is a fundamentally different leap because it's not based on deterministic logic. You can't depend on an LLM like you can depend on a calculator or a compiler. LLMs are totally different.
There are definitely parallels though. eg you could swap out your compiler for a different one that produces slightly different assembly. Similarly a LLM may implement things differently…but if it works do we care? Probably no more than when you buy software you don’t care precisely what compiler optimisation were used. The precise deterministicness isn’t a key feature
With the llm, it might work or it might not. If it doesn't work, then you have to keep iterating and hand holding it to make it work. Sometimes that process is less optimal than writing the code manually. With a calculator, you can be sure that the first attempt will work. An idiot with a calculator can still produce correct results. An idiot with an llm often cannot outside trivial solutions.
It often doesn't work. That's the point. A calculator works 100% of the time. A LLM might work 95% of the time, or 80%, or 40%, or 99% depending on what you're doing. This is difference and a key feature.
I see. I’d call that fragility/reliability rather than deterministic but semantics I suppose.
To me that isn’t a show stopper. Much of the real world works like that. We put very unreliable humans behind the wheel of 2 ton cars. So in a way this is perhaps just programmers aligning with the messy real world?
Perhaps a bit like architects can only model things so far eventually you need to build the thing and deal with the surprises and imperfection of dirt
I agree, but writing code is so different to calculations that long-term benefits are less clear.
It doesn't matter how good you are at calculations the answer to 2 + 2 is always 4. There are no methods of solving 2 + 2 which could result in you accidentally giving everyone who reads the result of your calculation write access to your entire DB. But there are different ways to code a system even if the UI is the same, and some of these may neglect to consider permissions.
I think a good parallel here would be to imagine that tomorrow we had access to humanoid robots who could do construction work. Would we want them to just go build skyscrapers and bridges and view all construction businesses which didn't embrace the humanoid robots as akin to doing arithmetic by hand?
You could of course argue that there's no problem here so long as trained construction workers are supervising the robots to make sure they're getting tolerances right and doing good welds, but then what happens 10 years down the road when humans haven't built a building in years? If people are not writing code any more then how can people be expected to review AI generated code?
I think the optimistic picture here is that humans just won't be needed in the future. In theory when models are good enough we should be able to trust the AI systems more than humans. But the less optimistic side of me questions a future in which humans no longer do, or even know how to do such fundamental things.
People who just let the agent code for them, how big of a codebase are you working on? How complex (i.e. is it a codebase that junior programmers could write and maintain)?
I've been an EM for the last 10 of my 25 year Software Engineering career. Coding is, frankly, boring to me anymore, even though I enjoyed doing it most of my career. I had this project I wanted to exist in world but couldn't be bothered to get started.
Decided to figure out what this "vibe coding" nonsense is, and now there's a certain level of joy to all of this again. Being able to clearly define everything using markdown contexts before any code is even written has been a great way to brain dump those 25 years of experience and actually watch something sane get produced.
I then realized I could feed it everything it ever needed to know. Just create a docs/* folder and tell it to read that every session.
Through discovery I learned about CLAUDE.md, and adding skills.
Now I have an /analyst, /engineer, and /devops that I talk to all day with their own logic and limitations, as well as the more general project CLAUDE.md, and dozens of docs/* files we collaborate on.
I'm at the point I'm running happy.engineering on my phone and don't even need to sit in front of the computer anymore.
I actually disagree with Andrej here re: "Generation (writing code) and discrimination (reading code) are different capabilities in the brain." and I would argue that the only reason he can read code fluently, find issues, etc. is because he has spent year in a non-AI assisted world writing code. As time goes on, he will become substantially worse.
This also bodes incredibly poorly for the next generation, who will mostly in their formative years now avoid writing code and thus fail to even develop a idea of what good code is, how it works/why it works, why you make certain decisions, and not others, etc. and ultimately you will see them become utterly dependent on AI, unable to make progress without it.
IMO outsourcing thinking is going to have incredibly negative consequences for the world at large.
Is coding like piloting, where pilots need a certain number of hours of "flight time" to gain skills, and then a certain number of additional hours each year to maintain their skills? Do developers need to schedule in a certain number of "manually written lines of code" every year?
Read your blog post and agree with some of it. Largely I agree with the premise that the 2nd and 3rd order effects of this technology will be more impactful than the 1st order “I was able to code this app I wouldn’t have otherwise even attempted to”. But they are so hard to predict!
Thanks, this rings true to me. The struggle is an investment, and it pays off in good judgement and taste. The same goes for individual codebases too. When I see some weird bug and can immediately guess what’s going wrong and why, that’s my time spent in that codebase paying off. I guess LLM-ing a feature is the inverse, incurring some kind of cognitive debt.
I coded up a crossword puzzle game using agentic dev this weekend. Claude and Codex/GPT. Had to seriously babysit and rewrite much of it, though, sure, I found it “cool” what it could do.
Writing code in many cases is faster to me than writing English (that is how PLs are designed, btw!) LLM/agentic is very “neat” but still a toy to the professional, I would say. I doubt reports like this one. For those of us building real world products with shelf-lives (Is Andrej representative of this archetype?), I just don’t see the value-add touted out there. I’d love to be proven wrong. But writing code (in code, not English), to me and many others, is still faster than reading/proving it.
I think there’s a combination of fetishizing and Stockholm syndroming going on in these enthusiastic self-reports. PMW.
> I am bracing for 2026 as the year of the slopacolypse across all of github, substack, arxiv, X/instagram, and generally all digital media.
2026 is just when it picks up - it'll get exponentially worse.
I think 2026 is the year of Business Analysts who were unable to code. Now CC et all are good enough that they can realize the vision as long as one knows exactly the requirements (software design not that important). Programmers who didn't know business could get by so far. Not anymore, because with these tools, the guy who knows business can now code fairly well.
"I think 2026 is the year of Business Analysts who were unable to code." This is interesting - I have seen far more BAs losing jobs as a result of the 'work' they did being replaced by tools (both AI and AI-generated). I logically see the connection from AI tools giving BAs far more direct ability to produce something, but I don't see it actually happening. It is possible it is too early in the AI curve for the quality of a BA built product to be sufficient. CC and Opus45 are relatively new.
It could also be BAs being lazy and not jumping ahead of the train that is coming towards them. It feels like in this race the engineer who is willing to learn business will still have an advantage over the business person who learns tech. At least for a little while.
> if you have any code you actually care about I would watch them like a hawk, in a nice large IDE on the side.
This is about where I'm at. I love pure claude code for code I don't care about, but for anything I'm working on with other people I need to audit the results - which I much prefer to do in an IDE.
> The most common category is that the models make wrong assumptions on your behalf and just run along with them without checking. They also don't manage their confusion, they don't seek clarifications, they don't surface inconsistencies, they don't present tradeoffs, they don't push back when they should, and they are still a little too sycophantic.
Does this not undercut everything going on here. Like, what?
It's predictable so you run defense around it with prompting, validation and model tuning. It generates volumes of working code in seconds from natural language prompts so it's extremely business efficient. We're talking about tools that generate correct code to 95% of a solution, the follow up human and automated test review, and second coding pass to fix the 5% are a non issue.
The way I have managed junior engineers is 90% via PR and testing, 10% via reading code in an editor or IDE.
It’s hard to let go of being the keyboard jockey, but in so many cases it is better to describe plans and acceptance criteria and just review the diffs.
Now that it's real, is there a minimum bar of non-AI-generated code that should be required in any production product? Like if 100% of the code is AI generated (or even doom-tabbed) and something goes wrong in prod, (crash, record corruption, data leak, whatever) then what? 99%? 50%? What's the bar where the risk starts outweighing the reward? When do we look around and say "maybe we should start slowing down before we do something that destroys our company"?
Granted it's not a one-size-fits-all problem, but I'm curious if any teams have started setting up additional concrete safeguards or processes to mitigate that specific threat. It feels like a ticking time bomb.
It almost begs the question, what even is the reward? A degradation of your engineering team's engineering fundamentals, in return for...are we actually shipping faster?
obviously you're not a devops eng, I think you're wildly under-estimating how much of business critical code pre-ai is completely orphaned anyway.
the people who wrote it were contractors long gone, or employees that have moved companies/departments/roles, or of projects that were long since wrapped up, or of people who got laid off, or the people who wrote it simply barely understood it in the first place and certainly don't remember what they were thinking back then now.
basically "what moron wrote this insane mess... oh me" is the default state of production code anyway. there's really no quality bar already.
I am a devops engineer and understand your point. But there's a huge difference: legacy code doesn't change. Yeah occasionally something weird will happen and you've got to dig into it, but it's pretty rare, and usually something like an expired certificate, not a logic bug.
What we're entering, if this comes to fruition, is a whole new era where massive amounts of code changes that engineers are vaguely familiar with are going to be deployed at a much faster pace than anything we've ever seen before. That's a whole different ballgame than the management of a few legacy services.
after a decade of follow-the-sun deployments by php contractors from vietnam to costa rica where our only qa was keeping an eye on the 500s graph, ai can't scare me.
That's actually a good comparison. Though even then, I imagine you at least have the ability to get on the phone and ask what they just did. Whereas LLM would just be like, "IDK, that was my twin brother. I'd ask him directly, but unfortunately he has been garbage collected. It was very sad. Would you like a cookie?"
I wonder if there's any value in some system that preserves the chat context of a coding agent and tags the commits with a reference to it, until the feature has been sufficiently battle tested. That way you can bring them back from the dead and interrogate them for insight if something goes wrong. Probably no more useful than just having a fresh agent look at the diff in most cases, but I can certainly imagine scenarios where it's like "Oh, duh, I meant to do X but looks like I accidentally did Y instead! Here's a fix." way faster than figuring it out from scratch. Especially if that whole process can be automated and fast, worst case you just waste a few tokens.
I'm genuinely curious though if there's anything you learned from those experiences that could be applied to agent driven dev processes too.
it was basically a mindless loop, very prime for being agent driven:
- observe error rate uptick
- maybe dig in with apm tooling
- read actual error messages
- compare what apm and logs said to last commit/deploy
- if they look even tangentially related deploy the previous commit (aka revert)
- if its still not fixed do a "debug push", basically stuff a bunch of print statements (or you can do better) around the problem to get more info
I won't say that solves every case but definitely 90% of them.
I think your point about preserving some amount of intent/context is good, but also like what are most of us doing with agents if not "loop on error message until it goes away".
I don't see the AI capacity jump in the recent months at all. For me it's more the opposite, CC works worse than a few months ago. Keeps forgetting the rules from CLAUDE.md, hallucinates function calls, generates tons of over-verbose plans, generates overengineered code. Where I find it a clear net-positive is pure frontend code (HTML + Tailwind), it's spaghetti but since it's just visualization, it's OK.
Hmm, your comment gave me the idea that maybe we should invent "What You Describe Is What You Get|. To replace HTML+Tailwind spaghetti with prompts generating it.
Sad to hear this attitude towards front-end code. Front-ends are so often already miswritten and full of accessibility pitfalls and I feel like LLMs are gonna dramatically magnify this problem :(
If you have a ChatGPT subscription, try Codex with GPT-5.2-High or 5.2-codex High? In my experience, while being much slower, it produces far better results than Opus and seems even more aggressively subsidized (more generous rate limits).
I've been doing vibe code interviews for nearly a year now. Most people are surprisingly bad with AI tools. We specifically ask them to bring their preferred tool, yet 20–30% still just copy-paste code from ChatGPT.
fun stats: corelation is real, people who were good at vibe code, also had offer(s) with other companies that didn't run vibe code interviews.
Interesting you say that, feels like when people were too stupid to google things and "googling something" was a skill that some had and others didn't.
From what I've heard, what few interviews there are for software engineers these days, they do have you use models and see how quickly you can build things.
The interviews I’ve given have asked about how control for AI slop without hurting your colleagues feelings. Anyone can prompt and build, the harder part, as usual for business, is knowing how and when to say, ‘no.’
The best thing I ever told Claude to do was "Swear profusely when discussing code and code changes". Probably says more about me than Claude, but it makes me snicker.
> LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.
Who doesn't like building? Building without any thought is literally a toy, like Lego or paint by numbers. That's the entire reason those things are popular. But a game is not a job. Sometimes I feel like half the people in this career are children. Never had any real responsibility. "Oh, everyone writes bugs, who tf cares". "Move fast, break stuff" was literally and unironically the tag line for a company that should have been taking far more responsibility.
This trend isn't limited to programmers either. Wherever I look I see people not taking responsibility. Lots of children in adult bodies. I do hope there are some adults who are really pulling the strings somewhere...
> What happens to the "10X engineer" - the ratio of productivity between the mean and the max engineer? It's quite possible that this grows a lot.
No doubt that good engineers will know when and how to leverage the tool, both for coding and improving processes (design-to-code, requirement collection, task tracking, basic code reviewal, etc) improving their own productivity and of those around them.
Motivated individuals will also leverage these tools to learn more and faster.
And yes, of course it's not the only tool one should use, of course there's still value in talking with proper human experts to learn from, etc, but 90% of the time you're looking for info the LLM will dig it from you reading at the source code of e.g. Postgres and its test rather than asking on chats/stack overflow.
This is a trasformative technology that will make great engineers even stronger, but it will weed out those who were merely valued for their very basic capability of churning something but never cared neither about engineering nor coding, which is 90% of our industry.
Most of this countries challenges are strictly political. The pittance
of work software can contribute is most likely negligible or destructive (e.g. software buttons in cars or palantir). In other words were picked all the low hanging fruit and all that left is to hang ourselves.
Great point about expansion vs speedup. I now have time to build custom tools, implement more features, try out different API designs, get 100% test coverage.. I can deliver more quickly, but can also deliver more overall.
I've seen the exact opposite with Claude. It literally ditched my request mid-analysis when doing a root cause analysis. It decided I was tired of the service failing and then gave me some restart commands to 'just get it working'
>Generation (writing code) and discrimination (reading code) are different capabilities in the brain. Largely due to all the little mostly syntactic details involved in programming, you can review code just fine even if you struggle to write it.
If this is how all juniors are learning nowadays, seniors are going shot up in value in the next decade.
I do feel a big mood shift after late November. I switched to using Cursor and Gemini primarily and it was big change in my ability to get my ideas into code effectively. The Cursor interface for one got to a place that I really like and enjoy using, but its probably more that the results from the agents themselves are less frustrating. I can deal with the output more now.
I'm still a little iffy on the agent swarm idea. I think I will need to see it in action in an interface that works for me. To me it feels like we are anthropomorphizing agents too much, and that results in this idea that we can put agents into roles and them combine them into useful teams. I can't help seeing all agents as the same automatons and I have trouble understanding why giving an agent with different guideliens to follow, and then having them follow along another agent would give me better results than just fixing the context in the first place. Either that or just working more on the code pipeline to spot issues early on - all the stuff we already test for.
The whole thing is about getting rid of experts and let the entry level idiots do all the work. The coders become expendable. And people do not see the chasm staring back at them :D. LLMs in their current form redistributes "intelligence" and expertise to the average joes for mere pennies. It should be much much more expensive, or it will disrupt the whole ecosystem. If it becomes even more intelligent it must be bludgeoned to death a.k.a. regulated like hell, otherwise the ensuing disruption will kill the job market and in the long term human values.
As an added plus: those, who already have wealth will benefit the most, instead of the masses. Since the distribution and dissemination of new projects is at the same level as before, meaning you would need a lot of money. So no matter how clever you are with an llm, if you don't have the means to distribute it you will be left in the dirt.
I am developing a web application for a dictionary that translates words from the national language into the local dialect.
Vibe coding and other tools, such as Google Vision, helped me download images published online, compile a PDF, perform OCR (Tesseract and Google Vision), and save everything in text format.
The OCR process was satisfactory for a first draft, but the text file has a lot of errors, as you'd expect when the dictionary has about 30,000 entries: Diacritical marks disappear, along with typographical marks and dashes, lines are moved up and down, and parts of speech (POS) are written in so many different ways due to errors that it is necessary to identify the wrong POS's one by one.
If the reasoning abilities of LLM-derived coding agents were as advanced as some claim, it would be possible for the LLM to derive the rules that must be applied to the entire dictionary from a sufficiently large set of “gold standard” examples.
If only that were the case. Every general rule applied creates other errors that propagate throughout the text, so that for every problem partially solved, two more emerge. What is evident to me is not clear to the LLM, in the sense that it is simple for me, albeit long and tedious, to do the editing work manually.
To give an example, if trans.v. (for example) indicates a transitive verb, it is clear to me that .trans.v. is a typographical error. I can tell the coding tool (I used Gemini, Claude, and Codex, with Codex being the best) that, given a standard POS, if there is a “.” before it, it must be deleted because it is a typo.
The generalization that comes easily to me but not to the coding agent is that if not one but two periods precede the POS, it means there are two typos, not to delete just one of the two dots.
This means that almost all rules have to be specified, whereas I expected the coding agent to generalize from the gigantic corpus on which it was trained (it should “understand” what the POS are, typical typos, the language in which the dictionary is written, etc.).
The transition from text to json to webapp is almost miraculous, but what is still missing from the mix is human-level reasoning and common sense (in part, I still believe that coding agents are fantastic, to be clear).
Honestly, how long do you guys think we have left as SWEs with high pay? Like the SWE job will still exist, but with a much lower technical barrier of entry, it strikes me that the pay is going to decrease a lot. Obviously BigCo codebases are extremely complex, more than Claude Code can handle right now, but I'd say there's definitely a timer running here. The big question for my life personally is whether I can reach certain financial milestones before my earnings potential permanently decreases.
It's counterintuitive but something becoming easier doesn't necessarily mean it becomes cheap. Programming has arguably been the easiest engineering discipline to break into by sheer force of will for the past 20+ years, and the pay scales you see are adapted to that reality already.
Empowering people to do 10 times as much as they could before means they hit 100 times the roadblocks. Again, in a lot of ways we've already lived in that reality for the past many years. On a task-by-task basis programming today is already a lot easier than it was 20 years ago, and we just grew our desires and the amount of controls and process we apply. Problems arise faster than solutions. Growing our velocity means we're going to hit a lot more problems.
I'm not saying you're wrong, so much as saying, it's not the whole story and the only possibility. A lot of people today are kept out of programming just because they don't want to do that much on a computer all day, for instance. That isn't going to change. There's still going to be skills involved in being better than other people at getting the computers to do what you want.
Also on a long term basis we may find that while we can produce entry-level coders that are basically just proxies to the AI by the bucketful that it may become very difficult to advance in skills beyond that, and those who are already over the hurdle of having been forced to learn the hard way may end up with a very difficult to overcome moat around their skills, especially if the AIs plateau for any period of time. I am concerned that we are pulling up the ladder in a way the ladder has never been pulled up before.
Supply and demand. There will continue to be a need for engineers to manage these systems and get them to do the thing you actually want, to understand implications of design tradeoffs and help stakeholders weigh the pros and cons. Some people will be better at it than others. Companies will continue to pay high premiums for such people if their business depends on quality software.
I think to give yourself more context you should ask about the patterns that led to SWEs having such high pay in the last 10-15 years and why it is you expected it to stay that way.
I personally think the barrier is going to get higher, not lower. And we will be back expected to do more.
I think the pay is going to skyrocket for senior devs within a few years, as training juniors that can graduate past pure LLM usage becomes more and more difficult.
Day after day the global quality of software and learning resources will degrade as LLM grey goo consumes every single nook and cranny of the Internet. We will soon see the first signs of pure cargo cult design patterns, conventions and schemes that LLMs made up and then regurgitated. Only people who learned before LLMs became popular will know that they are not to be followed.
People who aren't learning to program without LLMs today are getting left behind.
Yeah, all of this. Plus companies have avoided hiring and training juniors for 3 or 4 years now (which is more related to interest rates than AI). Plus existing seniors who deskill themselves by outsourcing their brain to AI. Seniors who know actually what they're doing are going to be in greater demand.
That is assuming that LLMs plateau in capability, if they haven't already, which I think is highly likely.
> IDEs/agent swarms/fallability. Both the "no need for IDE anymore" hype and the "agent swarm" hype is imo too much for right now.
I'm honestly considering throwing away my JetBrains subscription and this is year 9 or 10 of me having one. I only open Zed and start yappin' at Claude Code. My employer doesn't even want me using ReSharper because some contractor ruined it for everyone else by auto running all code suggestions and checking them in blindly, making for really obnoxious code diffs and probably introducing countless bugs and issues.
Meanwhile tasks that I know would take any developers months, I can hand-craft with Claude in a few hours, with the same level of detail, but no endless weeks of working on things that'll be done SoonTM.
The tenacity part is definitely true. I told it to keep trying when it kept getting stuck trying to spin up an Amazon Fargate service. I could feel its pain, and wanted to help, but I wanted to see whether the LLM could free itself from the thorny and treacherous AWS documentation forest. After a few dozen attempts and probably 50 KWh of energy it finally got it working, I was impressed. I could have done it faster myself, but the tradeoff would have been much higher blood pressure. Instead I relaxed and watched youtube while the LLM did its work.
Are there good guides about how to write Agents or good repos with examples? Also, are there big differences between how you would write one in Codex cli vs Claude code? Can there be run on it interchangeably?
I keep thinking about the TechnoCore from Dan Simmons' Hyperion, where the AIs were serving humans but secretly that was a parasitic relation, where they've been secretly using human brains as distributed processing nodes, essentially harvesting humanity's neural activity for their own computational needs without anyone's knowledge.
I know this is SF, but to me working with those LLMs feels more and more like that, and the atrophy part is real. Not that the model is literally using our brains as compute, but the relationship can become lopsided.
Slopacolypse. I am bracing for 2026 as the year of the slopacolypse across all of github, substack, arxiv, X/instagram, and generally all digital media.
Did he coin the term "slopacolypse"? It's a useful one.
Not sure how he is measuring, I'm still closer to about a 60% success rate. It's more like 20% is an acceptable one-shot, this goes to 60% acceptable with some iteration, but 40% either needs manual intervention to succeed or such significant iteration that manual is likely faster.
I can supervise maybe three agents in parallel before a task requiring significant hand-holding means I'm likely blocking an agent.
And the time an agent is 'restlessly working' on something in usually inversely correlated with the likelihood to succeed. Usually if it's going down a rabbit hole, the correct thing to do is to intervene and reorient it.
Right on especially on two things -- 1) the tools doing a disservice by not interviewing and seeking input and 2) The 2026 "Slopocalypse"
I'm hopeful that 2026 will be the year that the biggest adopters are forced to deal with the mass of product they've created that they don't fully understand, and a push for better tooling is the result.
Today's agentic tools are crude from a UX POV from where I am hoping they will end up.
I'm curious to see what effect this change has on leadership. For the last two years it's been "put everything you can into AI coding, or else!" with quotas and firings and whatever else. Now that AI is at the stage where it can actually output whole features with minimal handholding, is there going to be a Frankenstein moment where leadership realizes they now have a product whose codebase is running away from their engineering team's ability to support it? Does it change the calculus of what it means to be underinvested vs overinvested in AI, and what are the implications?
Am working on an iPhone app and impressed with how well Claude is able to generate decent/working code with prompts in plain English. I don’t have previous experience in building apps or swift but have a C++ background. Working in smaller chunks and incrementally adding features rather than a large prompt for the whole app seems more practical, is easier to review and build confidence.
Adding/prompting features one by one, reviewing code and then testing the resulting binary feels like the new programming workflow
Claude is good at writing code, not so good at reasoning, and I would never trust or deploy to production something solely written by Claude.
GPT-5.2 is not as good for coding, but much better at thinking and finding bugs, inconsistencies and edge cases.
The only decent way I found to use AI agents is by doing multiple steps between Claude and GPT, asking GPT to review every step of every plan and every single code change from Claude, and manually reviewing and tweaking questions and responses both way, until all the parties, including myself, agree. I also sometimes introduce other models like Qwen and K2 in the mix, for a different perspective.
And gosh, by doing so you immediately realize how dumb, unreliable and dangerous code generated by Claude alone is.
It's a slow and expensive process and at the end of the day, it doesn't save me time at all. But, perhaps counterintuitively, it gives me more confidence in the end result. The code is guaranteed to have tons of tests and assurance for edge cases that I may not have thought about.
Why am I not surprised that a blog was written about LLM coding going from 20% to 80% useful, yet all of the HN comments are still nit picking about some negative details rather than building positive ideas toward some progress...
Is the programmer ego really this fragile? At least luddites had an ideological reasoning, whereas here we just seem to have emotional reflexes.
It's because we see a bunch of people completely ignoring the missing 20% and flooding the world with complete slop. The push back is required to keep us sane, we need people reminding others that it's not at 100% yet even if it sometimes feels like it.
It’s usually people doing side projects or non-programmers who can’t tell the code is slop. None of these vibe coding evangelists ever shares the code they’re so amazed by, even though by their own logic anyone should be able to generate the same code with AI.
This kind of thought policing is getting to be exhausting. Perhaps we need a different kind of push back.
Do you know what my use case is? Do you know what kind of success rate I would actually achieve right now? Please show me where my missing 20% resides.
What particular setups are getting folks these sorts of results? If there’s a way I could avoid all the babysitting I have to do with AI tools that would be welcome
Is it muscle atrophy if you were a weakling since birth? Is it retina degeneration if you were born blind? No, because atrophy is a loss of a prior strength, and not an ever–existing weakness, but it's just as bad.
Like, do these guys actually dog food real user experience, or are they all admins with the fast lane to the real model while everyone outside the org has to go through the 10 layers of model sheding, caching and other means and methods of saving money.
We all know these models are expensive as fuck to run and these companies are degrading service, A+B testing, and the rest. Do they actually ponder these things directly?
Just always seems like people are on drugs when they talk about the capabilities, and like, the drugs could be pure shit (good) or ditch weed, and we call just act like the pipeline for drugs is a consistent thing but it's really not, not at this stage where they're all burning cash through infrastructure. Definitely, like drug dealers, you know they're cutting the good stuff with low cost cached gibberish.
> Definitely, like drug dealers, you know they're cutting the good stuff with low cost cached gibberish.
Can confirm. My partner's chatGPT wouldnt return anything useful for her given a specific query involving web use, while i got the desired result sitting side by side. She contacted support and they said nothing they can do about it, her account is in an A/B test group without some features removed. I imagine this saves them considerable resources despite still billing customers for them.
If you access a model through an openrouter provider it might be quantized (akin to being "cut with trash"), but when you go directly to Anthropic or OpenAI you are getting access to the same APIs as everyone else. Even top-brass folks within Microsoft use Anthropic and OpenAI proper (not worth the red-tape trouble to go directly through Azure). Also, the creator and maintainer of Claude, Boris Cherny, was a bit of an oddball but one of the comparatively nicer people at Anthropic, and he indicated he primarily uses the same Anthropic APIs as everyone else (which makes sense from a product development perspective).
The underlying models are all actually really undifferentiated under the covers except for the post-training and base prompts. If you eliminate the base prompts the models behave near identically.
A conspiracy would be a helluva lot more interesting and fun, but I've spoken to these folks firsthand and it seems they already have enough challenges keeping the beast running.
A big wow moment coming up is going to be GPT 5.* in Codex with Cerebras doing inference. The inference speed is going to be a big unlock, because many tasks are intrinsically serial.
It's going to feel literally like playing God, where you type in what you want and it happens ~instantly.
Speed is really important to me but also I would like higher weekly limits -- which means lower cost I suppose. Building out complex projects can take 6 months to a year on a Pro plan.
My trick is to attach the codebase as a txt file to 5-10 different GPT 5.2 Thinking chats, paste in the specs, and then get hard work done there, then just copy paste the final task list into codex to lower codex usage.
Thank you for the really excellent summation. I echo your thought 1 to 1. I have found it more difficult to learn new languages or coding skills, because I am no longer forced to go through the painful slow grind of learning.
A lot of these things sound cool but sometimes I'm curious what they're actually building
Like, is their bottleneck creativity now then? Are they building naything interedting or using agents to build... things that don't appeal to me, anyway?
That is motivational content, but not economics. Most startups will be noise, even more so than before. The value of being a founder ceases when everyone is a founder, when it becomes universal. You will need customers. Nobody wants to buy re-invented-the-wheel-74.0. It lacks character, it lacks soul. Without it, your product will be nothing but noise in a noisy world.
Cope. If you create something that genuinely solves a problem, people will buy no matter what.
Look entrepreneurship has never been easy. In fact it's always been one of the hardest thing ever. I'm just saying... *you don't have to do it*. Do whatever you want lol
Happy to hear what's your solution to avoid becoming totally replaceable and obsolete.
It’s a great and insightful review—not over-hyping the coding agent, and not underestimating it either. It acknowledges both its usefulness and its limitations. Embracing it and growing with it is how I see it too.
It's been a bit like the boiling frog analogy for me
I started by copy pasting more and more stuff in chatgpt. Then using more and more in-IDE prompting, then more and more agent tools (Claude etc). And suddenly I realise I barely hand code anymore
For sure there's still a place for manual coding, especially schemas/queries or other fiddly things where a tiny mistake gets amplified, but the vast majority of "basic work" is now just prompting, and honestly the code quality is _better_ that it was before, all kinds of refactors I didn't think about or couldn't be bothered with have almost automatically
I've had the opposite experience, it's been a long time listening to people going "It's really good now" before it developed to a permutation that was actually worth the time to use it.
ChatGPT 3.5/4 (2023-2024): The chat interface was verbose and clunky and it was just... wrong... like 70+% of the time. Not worth using.
CoPilot autocomplete and Gitlab Duo and Junie (late 2024-early 2025): Wayyy too aggressive at guessing exactly what I wasn't doing and hijacked my tab complete when pre-LLM type-tetris autocomplete was just more reliable.
Copilot Edit/early Cursor (early 2025): Ok, I can sort of see uses here but god is picking the right files all the time such a pain as it really means I need to have figured out what I wanted to do in such detail already that what was even the point? Also the models at that time just quickly descended into incoherency after like three prompts, if it went off track good luck ever correcting it.
Copilot Agent mode / Cursor (late 2025): Ok, great, if the scope is narrowly scoped, and I'm either going to write the tests for it or it's refactoring existing code it could do something. Like something mechanical like the library has a migration where we need to replace the use of methods A/B/C and replace them with a different combination of X/Y/Z. great, it can do that. Or like CRUD controller #341. I mean, sure, if my boss is going to pay for it, but not life changing.
Zed Agent mode / Cursor agent mode / Claude code (early 2026): Finally something where I can like describe the architecture and requirements of a feature, let it code, review that code, give it written instructions on how to clean it up / refactor / missing tests, and iterate.
But that was like 2 years of "really it's better and revolutionary now" before it actually got there. Now maybe in some languages or problem domains, it was useful for people earlier but I can understand people who don't care about "but it works now" when they're hearing it for the sixth time.
And I mean, what one hand gives the other takes away. I have a decent amount of new work dealing with MRs from my coworkers where they just grabbed the requirements from a stakeholder, shoved it into Claude or Cursor and it passed the existing tests and it's shipped without much understanding. When they wrote them themselves, they tested it more and were more prepared to support it in production...
I find myself even for small work, telling CC to fix it for me is better as it usually belongs to a thread of work, and then it understands the big picture better.
Both can be true. You're tapping into every line of code publicly available, and your day-to-day really isn't that unique. They're really good at this kind of work.
> Coding workflow. Given the latest lift in LLM coding capability, like many others I rapidly went from about 80% manual+autocomplete coding and 20% agents in November to 80% agent coding and 20% edits+touchups in December
Anyone wondering what exactly is he actually building? What? Where?
> The mistakes have changed a lot - they are not simple syntax errors anymore, they are subtle conceptual errors that a slightly sloppy, hasty junior dev might do.
I would LOVE to have jsut syntax errors produced by LLMs, "subtle conceptual errors that a slightly sloppy, hasty junior dev might do." are neither subtle nor slightly sloppy, they actually are serious and harmful, and no junior devs have no experience to fix those.
> They will implement an inefficient, bloated, brittle construction over 1000 lines of code and it's up to you to be like "umm couldn't you just do this instead?"
Why just not hand write 100 loc with the help of an LLM for tests, documentation and some autocomplete instead of making it write 1000 loc and then clean it up? Also very difficult to do, 1000 lines is a lot.
> Tenacity. It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day.
It's a computer program running in the cloud, what exactly did he expected?
> Speedups. It's not clear how to measure the "speedup" of LLM assistance.
See above
> 2) I can approach code that I couldn't work on before because of knowledge/skill issue. So certainly it's speedup, but it's possibly a lot more an expansion.
mmm not sure, if you don't have domain knowledge you could have an initial stubb at the problem, what when you need to iterate over it? You don't if you don't have domain knowledge on your own
> Fun. I didn't anticipate that with agents programming feels more fun because a lot of the fill in the blanks drudgery is removed and what remains is the creative part.
No it's not fun, eg LLMs produce uninteresting uis, mostly bloated with react/html
> Atrophy. I've already noticed that I am slowly starting to atrophy my ability to write code manually.
My bet is that sooner or later he will get back to coding by hand for periods of time to avoid that, like many others, the damage overreliance on these tools bring is serious.
> Largely due to all the little mostly syntactic details involved in programming, you can review code just fine even if you struggle to write it.
No programming it's not "syntactic details" the practice of programming it's everything but "syntactic details", one should learn how to program not the language X or Y
> What happens to the "10X engineer" - the ratio of productivity between the mean and the max engineer? It's quite possible that this grows a lot.
Yet no measurable econimic effects so far
> Armed with LLMs, do generalists increasingly outperform specialists? LLMs are a lot better at fill in the blanks (the micro) than grand strategy (the macro).
Did people with a smartphone outperformed photographers?
Not angry nor scared, I value my hard skills a lot, I'm just wondering why people believe religiously everything AI related. Maybe I'm a bit sick with the excessive hype
There's no fear (a bit of anger I must admit). I suspect nearly all of the reaction against this comes from a similar place to where mine does:
All of the real world code I have had to review created by AI is buggy slop (often with subtle, but weird bugs that don't show up for a while). But on HN I'm told "this is because your co-workers don't know how to AI right!!!!" Then when someone who supposedly must be an expert in getting things done with AI posts, it's always big claims with hand-wavy explanations/evidence.
Then the comments section is littered with no effort comments like this.
Yet oddly whenever anyone asks "show me the thing you built?" Either it looks like every other half-working vibe coded CRUD app... or it doesn't exist/can't be shown.
If you tell me you have discovered a miracle tool, just some me the results. Not taking increasingly ridiculous claims at face value is not "fear". What I don't understand is where comments like yours come from? What makes you need this to be more than it is?
It's because they don't have a substantive response to it, so they resort to ad hominems.
I've worked extensively in the AI space, and believe that it is extremely useful, but these weird claims (even from people I respect a lot) that "something big and mysterious is happening, I just can't show you yet!" set of my alarms.
When sensible questions are met with ad hominems by supporters it further sets of alarm bells.
I see way more hype that is boosted by the moderators. The scared ones are the nepo babies who founded a vaporware AI company that will be bought by daddy or friends through a VC.
They have to maintain the hype until a somewhat credible exit appears and therefore lash out with boomer memes, FOMO, and the usual insane talking points like "there are builders and coders".
You learned a new adjective? If people move beyond "nice", "mean" and "curmudgeonly" they might even read Shakespeare instead of having an LLM producing a summary.
>Anyone wondering what exactly is he actually building? What? Where?
this is trivially answerable. it seems like they did not do even the slightest bit of research before asking question after question to seem smart and detailed.
I asked many question and you focused on only one, btw yes I did my research, and I know him because I followed almost every tutorial he has on YouTube, and he never mentions clearly what weekend project worked on to make him conclude with such claims. I had a very high respect of him if not that at some point started acting like the Jesus Christ of LLMs
I don't "vibe code" but when I use an LLM with a game I usually branch out into several experiments which I don't have to commit to. Thus, it just makes that iteration process go faster.
Or slower, when the LLM doesn't understand what I want, which is a bigger issue when you spawn experiments from scratch (and have given limited context around what you are about to do).
I'm trying it out with Godot for my little side projects. It can handle writing the GUI files for nodes and settings. The workflow is asking cursor to change something, I review the code changes, then load up the game in Godot to check out the changes. Works pretty well. I'm curious if any Unity or Unreal devs are using it since I'm sure its a similar experience.
It might be biased to Reddit/Twitter users but from what I've seen game developers seem to be much more averse towards using AI (even for coding) than other fields.
Which is curious since prototyping helps a lot in gamedev.
Vibe coding in Unreal Engine is of limited use. It obviously helps with C++, but so much of your time is doing things that are not C++. It hurts a lot that UE relies heavily on blueprints, if they were code you could just vibecode a lot of that.
> Atrophy. I've already noticed that I am slowly starting to atrophy my ability to write code manually.
I've been increasingly using LLM's to code for nearly two years now - and I can definitely notice my brain atrophy. It bothers me. Actually over the last few weeks I've been looking at a major update to a product in production & considered doing the edits manually - at least typing the code from the LLM & also being much more granular with my instructions (i.e. focus on one function at a time). I feel in some ways like my brain is turning into slop & I've been coding for at least 35 years... I feel validated by Karpathy.
1. Manual coding may be less relevant (albeit ability to read code, interpret it and understand it will be more) in the future. Likely already is.
2. Any skill you don't practice becomes "weaker". Gonna give you an example. I play chess since my childhood, but sometimes I go months without playing it, even years. When I get back I start losing elo fast. If I was in the top 10% of chess.com, I drop to top 30% in the weeks after. But after few months I'm back at top 10%. Takeaway: your relative ability is more or less the same compared to other practitioners, you're simply rusty.
Thanks for your comment, it set me at ease. I know from experience that you're right on point 2. As for point one, I also tend to agree. AI is such a paradigm shift & rapid/massive change doesn't come without stress. I just need to stay cool about it all ;-)
The section on IDEs/agent swarms/fallibility resonated a lot for me; I haven't gone quite as far as Karpathy in terms of power usage of Claude Code, but some of the shifts in mistakes (and reality vs. hype) analysis he shared seems spot on in my (caveat: more limited) experience.
> "IDEs/agent swarms/fallability. Both the "no need for IDE anymore" hype and the "agent swarm" hype is imo too much for right now. The models definitely still make mistakes and if you have any code you actually care about I would watch them like a hawk, in a nice large IDE on the side. The mistakes have changed a lot - they are not simple syntax errors anymore, they are subtle conceptual errors that a slightly sloppy, hasty junior dev might do. The most common category is that the models make wrong assumptions on your behalf and just run along with them without checking. They also don't manage their confusion, they don't seek clarifications, they don't surface inconsistencies, they don't present tradeoffs, they don't push back when they should, and they are still a little too sycophantic. Things get better in plan mode, but there is some need for a lightweight inline plan mode. They also really like to overcomplicate code and APIs, they bloat abstractions, they don't clean up dead code after themselves, etc. They will implement an inefficient, bloated, brittle construction over 1000 lines of code and it's up to you to be like "umm couldn't you just do this instead?" and they will be like "of course!" and immediately cut it down to 100 lines. They still sometimes change/remove comments and code they don't like or don't sufficiently understand as side effects, even if it is orthogonal to the task at hand. All of this happens despite a few simple attempts to fix it via instructions in CLAUDE . md. Despite all these issues, it is still a net huge improvement and it's very difficult to imagine going back to manual coding. TLDR everyone has their developing flow, my current is a small few CC sessions on the left in ghostty windows/tabs and an IDE on the right for viewing the code + manual edits."
Yes, typically you take since from people who've been successful at their career. Are you suggesting we should be taking career advice from high school freshmen instead?
tl;dr - All this AI stuff is just Universal Paperclips[1]
I see a lot of comments about folks being worried about going soft, getting brain rot, or losing the fun part of coding.
As far as I'm concerned this is a bigger (albeit kinda flakey) self-driving tractor. Yeah I'd be bored if I just stuck to my one little cabbage patch I'd been tilling by hand. But my new cabbage patch is now a megafarm. Subjectively, same level of effort.
> IDEs/agent swarms/fallability. Both the "no need for IDE anymore" hype and the "agent swarm" hype is imo too much for right now. The models definitely still make mistakes and if you have any code you actually care about I would watch them like a hawk, in a nice large IDE on the side.
... here on HN [0] you get a bunch of people telling you to get with the times, grandpa.
Really makes me wonder: Who are these people and why are they doing that?
At the risk of exposing my... atypical take on contemporary occupational "morality". LLMs, and Claude Code specifically, have given me the ability to work two jobs, and retain my soft-standing as the "guy" that gets stuff done/knows my stuff/can fix anything, while still working less hours than I did when I just had a single job.
I firmly believe, at SOME point, ML is going to eat my lunch. And I'd like to be well and retired off to a countryside homestead by then. Until such a time, I am going to use and abuse this technology as much as possible to gather 4 paychecks a month, optimize my investment portfolio to scale my NW, and by any means gain financial independence before the risk of my career vaporizing materializes. Sure, I could REALLY try and be one of those engineers that pulls a $750k salary and I wouldn't need to do this; but that isn't really in the cards for me. I know where I stand and I'm simply not smart or hardworking enough to get paid that much from a single job and guarantee my financial independence in the traditional way.
To that end, these tools have been extraordinarily impactful for me in a very simple and objectively positive way. And as much as I relate to basically everything OP mentioned, at the end of the day I simply DGAF. I want to make as much money as possible before the music stops, and this is the smart way to do that right now
> Slopacolypse
Really… REALLY not looking forward to getting this word spammed at me the next 6-12 months… even less so seeing the actual manifestation.
> TLDR
This should be at the start?
I actually have been thinking of trying out ClaudeCode/OpenCode over this past week… can anyone provide experience, tips, tricks, ref docs?
My normal workflow is using Free-tier ChatGPT to help me interrogate or plan my solution/ approach or to understand some docs/syntax/best practice of which I’m not familiar. then doing the implementation myself.
fwiw, the same is true for humans. Which is why there's a whole lot of process and red tape around that button. We know how to manage risk. We can choose to do that for LLM usage, too.
If instead we believe in fantasies of a single all-knowing machine god that is 100% correct at all times, then... we really just have ourselves to blame. Might as well just have spammed that button by hand.
I don't know about you guys but most of the time it's spitting nonsense models in sqlalchemy and I have to constantly correct it to the point where I am back at writing the code myself. The bugs are just astonishing and I lose control of the codebase after some time to the point where reviewing the whole thing just takes a lot of time.
On the contrary if it was for a job in a public sector I would just let the LLM spit out some output and play stupid, since salary is very low.
Depends what we mean by specialist. If it frontend vs backend then maybe. If it general dev vs some specialist scientific programmer or other field where a generalist won’t have a clue then this seems like a recipe for disaster (literal disasters included).
I don't know if it's fair to call him an ai addict or deduce that his ego is bruised. But I do wonder whether karpathy's agentic llm experiences are based on actual production code or pet projects. Based on a few videos I have seen of his, I am guessing it's the latter. Also, he is a research scientist (probably a great one), not a software developer. I agree with the op that karpathy should not be given much attention in this topic i.e llms for software development.
I don't agree with the parent commenters characterization of Karpathy, but these projects are just simple toy projects. They're educational material, not production level software.
I'm almost a boomer and I agree. THis dichotomy is weird. I am retired EE and I love the ability to just have AI do whatever I want for me. I have it manage a 10 node proxmox cluster in my basement via ansible and terraform. I can finally do stuff I always wanted but had no time. I got sick of editing my kids sports videos for highlights in Davinci Resolve so just asked claude to write a simple app for me and then use all my random video cards in my boxes to render clips in parallel and so on. Tech is finally fun again when I do not have to dedicate days to understand some new framework. It does feel a little like late 1990's computing when everyone was making geocities webpages but those days were more fun. Now with local llms getting strong as well and speaking to my PC instead of typing it feels like SciFi, so yeah, I do not get this hacker news hand wringing about code craft.
Well, it's not really a full-blown app yet. Claude wrote a plugin for MPV. So now when I watch video I just push a button to mark in and out of highlights similar to how it works in DaVinci Resolve. Then I have a command line tool that takes those timestamps in a video file and cuts it up into individual clips and then re-renders those clips and creates a highlight reel. Another command line tool takes three or four large MP4 files that the camera generates and downloads them and combines them in the actual game video on my desktop and also uploads it to my archive and transcodes into a bunch of different formats and uploads to YouTube. And for transcoding, again, it divvies it out to the video cards, which works pretty well. I think I have five or six encoders available so it chunks it up and then reassembles. All in all, it's nothing fancy, but it reduced quite a bit the friction of coming home after games and getting a video up on YouTube for grandparents.
Same demographic, same experience. AI has been incredibly liberating for me. I get all sorts of things done now that before were previosly impossible for all practical purposes. Among other things, it cuts through the noise of all the layers of detail, and allows me to focus on ideas, design, and just getting stuff built asap.
I also don't get all the hand-wringing. AI is an amazing tool. Use it and be happy.
Even less do I get all the cope about it not being effective, or even useless at some level. When I read posts such as that, it feels like a different planet. Just not my experience at all.
Instead of a 17 paragraph twitter post with a baffling TLDR at the end why not just record your screen and _demonstrate_ all of what you're describing?
Otherwise, I think you're incidentally right, your "ego" /is/ bruised, and you're looking for a way out by trying to prognosticate on the future of the technology. You're failing in two different ways.
I worry about the "brain atrophy" part, as I've felt this too. And not just atrophy, but even moreso I think it's evolving into "complacency".
Like there have been multiple times now where I wanted the code to look a certain way, but it kept pulling back to the way it wanted to do things. Like if I had stated certain design goals recently it would adhere to them, but after a few iterations it would forget again and go back to its original approach, or mix the two, or whatever. Eventually it was easier just to quit fighting it and let it do things the way it wanted.
What I've seen is that after the initial dopamine rush of being able to do things that would have taken much longer manually, a few iterations of this kind of interaction has slowly led to a disillusionment of the whole project, as AI keeps pushing it in a direction I didn't want.
I think this is especially true if you're trying to experiment with new approaches to things. LLMs are, by definition, biased by what was in their training data. You can shock them out of it momentarily, whish is awesome for a few rounds, but over time the gravitational pull of what's already in their latent space becomes inescapable. (I picture it as working like a giant Sierpinski triangle).
I want to say the end result is very akin to doom scrolling. Doom tabbing? It's like, yeah I could be more creative with just a tad more effort, but the AI is already running and the bar to seeing what the AI will do next is so low, so....
It's not just brain atrophy, I think. I think part of it is that we're actively making a tradeoff to focus on learning how to use the model rather than learning how to use our own brains and work with each other.
This would be fine if not for one thing: the meta-skill of learning to use the LLM depreciates too. Today's LLM is gonna go away someday, the way you have to use it will change. You will be on a forever treadmill, always learning the vagaries of using the new shiny model (and paying for the privilege!)
I'm not going to make myself dependent, let myself atrophy, run on a treadmill forever, for something I happen to rent and can't keep. If I wanted a cheap high that I didn't mind being dependent on, there's more fun ones out there.
> let myself atrophy, run on a treadmill forever, for something
You're lucky to afford the luxury not to atrophy.
It's been almost 4 years since my last software job interview and I know the drills about preparing for one.
Long before LLMs my skills naturally atrophy in my day job.
I remember the good old days of J2ME of writing everything from scratch. Or writing some graph editor for universiry, or some speculative, huffman coding algorithm.
That kept me sharp.
But today I feel like I'm living in that netflix series about people being in Hell and the Devil tricking them they're in Heaven and tormenting them: how on planet Earth do I keep sharp with java, streams, virtual threads, rxjava, tuning the jvm, react, kafka, kafka streams, aws, k8s, helm, jenkins pipelines, CI-CD, ECR, istio issues, in-house service discovery, hierarchical multi-regions, metrics and monitoring, autoscaling, spot instances and multi-arch images, multi-az, reliable and scalable yet as cheap as possible, yet as cloud native as possible, hazelcast and distributed systems, low level postgresql performance tuning, apache iceberg, trino, various in-house frameworks and idioms over all of this? Oh, and let's not forget the business domain, coding standards, code reviews, mentorships and organazing technical events. Also, it's 2026 so nobody hires QA or scrum masters anymore so take on those hats as well.
So LLMs it is, the new reality.
This is a very good point. Years ago working in a LAMP stack, the term LAMP could fully describe your software engineering, database setup and infrastructure. I shudder to think of the acronyms for today's tech stacks.
And yet many the same people who lament the tooling bloat of today will, in a heartbeat, make lame jokes about PHP. Most of them aren't even old enough to have ever done anything serious with it, or seen it in action beyond Wordpress or some spaghetti-code one-pager they had to refactor at their first job. Then they show up on HN with a vibe-coded side project or blog post about how they achieved a 15x performance boost by inventing server-side rendering.
Highly relevant username!
Ya I agree it's totally crazy.... but, do most app deployments need even half that stuff? I feel like most apps at most companies can just build an app and deploy it using some modern paas-like thing.
> I feel like most apps at most companies can just build an app and deploy it using some modern paas-like thing.
Most companies (in the global, not SV sense) would be well served by an app that runs in a Docker container in a VPS somewhere and has PostgreSQL and maybe Garage, RabbitMQ and Redis if you wanna get fancy, behind Apache2/Nginx/Caddy.
But obviously that’s not Serious Business™ and won’t give you zero downtime and high availability.
Though tbh most mid-size companies would also be okay with Docker Swarm or Nomad and the same software clustered and running behind HAProxy.
But that wouldn’t pad your CV so yeah.
> Most companies (in the global, not SV sense) would be well served by an app that runs in a Docker container in a VPS somewhere and has PostgreSQL and maybe Garage, RabbitMQ and Redis if you wanna get fancy, behind Apache2/Nginx/Caddy.
That’s still too much complication. Most companies would be well served by a native .EXE file they could just run on their PC. How did we get to the point where applications by default came with all of this shit?
When I was in primary school, the librarian used a computer this way, and it worked fine. However, she had to back it up daily or weekly onto a stack of floppy disks, and if she wanted to serve the students from the other computer on the other side of the room, she had to restore the backup on there, and remember which computer had the latest data, and only use that one. When doing a stock–take (scanning every book on the shelves to identify lost books), she had to bring that specific computer around the room in a cart. Such inconveniences are not insurmountable, but they're nice to get rid of. You don't need to back up a cloud service and it's available everywhere, even on smaller devices like your phone.
There's an intermediate level of convenience. The school did have an IT staff (of one person) and a server and a network. It would be possible to run the library database locally in the school but remotely from the library terminals. It would then require the knowledge of the IT person to administer, but for the librarian it would be just as convenient as a cloud solution.
I think the 'more than one user' alternative to a 'single EXE on a single computer' isn't the multilayered pie of things that KronisLV mentioned, but a PHP script[0] on an apache server[0] you access via a web browser. You don't even need a dedicated DB server as SQLite will do perfectly fine.
[0] or similarly easy to get running equivalent
> but a PHP script[0] on an apache server[0] you access via a web browser
I've seen plenty of those as well - nobody knows exactly how things are setup, sometimes dependencies are quite outdated and people are afraid to touch the cPanel config (or however it's setup). Not that you can't do good engineering with enough discipline, it's just that Docker (or most methods of containerization) limits the blast range when things inevitably go wrong and at least try to give you some reproducibility.
At the same time, I think that PHP can be delightfully simple and I do use Apache2 myself (mod_php was actually okay, but PHP-FPM also isn't insanely hard to setup), it's just that most of my software lives in little Docker containers with a common base and a set of common tools, so they're decoupled from the updates and config of the underlying OS. I've moved the containers (well data+images) across servers with no issues when needed and also resintalled OSes and spun everything right back up.
Kubernetes is where dragons be, though.
> That’s still too much complication. Most companies would be well served by a native .EXE file they could just run on their PC
I doubt that.
As software has grown to solving simple personal computing problems (write a document, create a spreadsheet) to solving organizational problems (sharing and communication within and without the organization), it has necessarily spread beyond the .exe file and local storage.
That doesn't give a pass to overly complex applications doing a simple thing - that's a real issue - but to think most modern company problems could be solved with just a local executable program seems off.
It can be like that, but then IT and users complain about having to update this .exe on each computer when you add new functionality or fix some errors. When you solve all major pain points with a simple app, "updating the app" becomes top pain point, almost by definition.
> How did we get to the point where applications by default came with all of this shit?
Because when you give your clients instructions on how to setup the environment, they will ignore some of them and then they install OracleJDK while you have tested everything under OpenJDK and you have no idea why the application is performing so much worse in their environment: https://blog.kronis.dev/blog/oracle-jdk-and-openjdk-compatib...
It's not always trivial to package your entire runtime environment unless you wanna push VM images (which is in many ways worse than Docker), so Docker is like the sweet spot for the real world that we live in - a bit more foolproof, the configuration can be ONE docker-compose.yml file, it lets you manage resource limits without having to think about cgroups, as well as storage and exposed ports, custom hosts records and all the other stuff the human factor in the process inevitably fucks up.
And in my experience, shipping a self-contained image that someone can just run with docker compose up is infinitely easier than trying to get a bunch of Ansible playbooks in place.
If your app can be packaged as an AppImage or Flatpak, or even a fully self contained .deb then great... unless someone also wants to run it on Windows or vice versa or any other environment that you didn't anticipate, or it has more dependencies than would be "normal" to include in a single bundle, in which case Docker still works at least somewhat.
Software packaging and dependency management sucks, unless we all want to move over to statically compiled executables (which I'm all for). Desktop GUI software is another can of worms entirely, too.
When I come into a new project and I find all this... "stuff" in use, often what I later find is actually happening with a lot of it is:
- nobody remembers why they're using it
- a lot of it is pinned to old versions or the original configuration because the overhead of maintaining so much tooling is too much for the team and not worth the risk of breaking something
- new team members have a hard time getting the "complete picture" of how the software is built and how it deploys and where to look if something goes wrong.
That was on NBC.
Businesses too. For two years it's been "throw everything into AI." But now that shit is getting real, are they really feeling so coy about letting AI run ahead of their engineering team's ability to manage it? How long will it be until we start seeing outages that just don't get resolved because the engineers have lost the plot?
From what I am seeing, no one is feeling coy simply because of the cost savings that management is able to show the higher-ups and shareholders. At that level, there's very little understanding of anything technical and outages or bugs will simply get a "we've asked our technical resources to work on it". But every one understands that spending $50 when you were spending $100 is a great achievement. That's if you stop and not think about any downsides. Said management will then take the bonuses and disappear before the explosions start with their resume glowing about all the cost savings and team leadership achievements. I've experienced this first hand very recently.
Of all the looming tipping points whereby humans could destroy the fabric of their existence, this one has to be the stupidest. And therefore the most likely.
There really ought to be a class of professionals like forensic accountants who can show up in a corrupted organization and do a post mortem on their management of technical debt
How long until “the LLM did it it” is just as effective as “AWS is down, not my fault”?
Never because the only reason that works with Amazon is that everyone is down at the exact same time.
Everyone will suffer from slop code at the same time.
Yeah but that's very different from an AWS outage. Everyone's website being down for a day every year or 2 is something that it's very hard to take advantage of as a competitor. That's not true for software that is just terrible all the time.
This to me is the point.. LLMs can't be responsible for things. It sits with a human.
Why can LLMs not be responsible for things? (genuine question - I'm not certain myself).
because it doesn't have any skin in the game and can't be punished, and can't be rewarded for succeeding. Its reputation, career, and dignity are nonexistent.
On the contrary - the LLM has had it's own version of "skin in the game" through the whole of it's training. Reinforcement learning is nothing but that. Why is that less real than putting a person in prison. Is it because of the LLM itself, or because you don't trust the people selling it to you?
Are you claiming that LLMs are... sentient? Bold claim, Taylor.
This doesn't seem to have stopped anyone before.
Stopped anyone from doing what? Assigning responsibility to someone with nothing to lose, no dignity or pride, and immune from financial or social injury?
If you’re just a gladhander for an algorithm, what are you really needed for?
> It's not just brain atrophy, I think. I think part of it is that we're actively making a tradeoff to focus on learning how to use the model rather than learning how to use our own brains and work with each other.
I agree with the sentiment but I would have framed it differently. The LLM is a tool, just like code completion or a code generator. Right now we focus mainly on how to use a tool, the coding agent, to achieve a goal. This takes place at a strategic level. Prior to the inception of LLMs, we focused mainly on how to write code to achieve a goal. This took place at a tactical level, and required making decisions and paying attention to a multitude of details. With LLMs our focus shifts to a higher-level abstraction. Also, operational concerns change. When writing and maintaining code yourself, you focus on architectures that help you simplify some classes of changes. When using LLMs, your focus shifts to building context and aiding the model effectively implement their changes. The two goals seem related, but are radically different.
I think a fairer description is that with LLMs we stop exercising some skills that are only required or relevant if you are writing your code yourself. It's like driving with an automatic transmission vs manual transmission.
Previous tools have been deterministic and understandable. I write code with emacs and can at any point look at the source and tell you why it did what it did. But I could produce the same program with vi or vscode or whatever, at the cost of some frustration. But they all ultimately transform keystrokes to a text file in largely the same way, and the compiler I'm targeting changes that to asm and thence to binary in a predictable and visible way.
An LLM is always going to be a black box that is neither predictable nor visible (the unpredictability is necessary for how the tool functions; the invisibility is not but seems too late to fix now). So teams start cargo culting ways to deal with specific LLMs' idiosyncrasies and your domain knowledge becomes about a specific product that someone else has control over. It's like learning a specific office suite or whatever.
> An LLM is always going to be a black box that is neither predictable nor visible (the unpredictability is necessary for how the tool functions; the invisibility is not but seems too late to fix now)
So basically, like a co-worker.
That's why I keep insisting that anthropomorphising LLMs is to be embraced, not avoided, because it gives much better high-level, first-order intuition as to where they belong in a larger computing system, and where they shouldn't be put.
> So basically, like a co-worker.
Arguably, though I don't particularly need another co-worker. Also co-workers are not tools (except sometimes in the derogatory sense).
Sort of except it seems the more the co-worker does the job it atrophies my ability to understand.. So soon we'll all be that annoyingly ignorant manager saying, "I don't know, I want the button to be bigger". Yay?
Only if we're lucky and the LLMs cease being replaced with improved models.
Claude has already shown us people who openly say "I don't code and yet I managed this"; right now the command line UI will scare off a lot of people, and people using the LLMs still benefit from technical knowledge and product design skills, if the tools don't improve we keep that advantage…
…but how long will it be before the annoyingly ignorant customer skips the expensive annoyingly ignorant manager along with all us expensive developers, and has one of the models write them bespoke solution for less than the cost of off-the-shelf shrink-wrapped DVDs from a discount store?
Hopefully that extra stuff is further away than it seems, hopefully in a decade there will be an LLM version of this list: https://en.wikipedia.org/wiki/List_of_predictions_for_autono...
But I don't trust to hope. It has forsaken these lands.
> using the LLMs still benefit from technical knowledge and product design skills, if the tools don't improve we keep that advantage…
I don't think we will, because many of us are already asking LLMs for help/advice on these, so we're already close to the point where LLMs will be able to use these capabilities directly, instead of just for helping us drive the process.
Indeed, but the output of LLMs today for these kinds of task are akin to a junior product designer, a junior project manager, a junior software architect etc.
For those of us who are merely amateur at any given task, LLMs raising us to "junior" is absolutely an improvement. But just as it's possible to be a better coder than an LLM, if you're a good PM or QA or UI/UX designer, you're not obsolete yet.
> and can at any point look at the source and tell you why it did what it did
Even years later? Most people can’t unless there’s good comments and design. Which AI can replicate, so if we need to do that anyway, how is AI specially worse than a human looking back at code written poorly years ago?
I mean, Emacs's oldest source files are like 40 years old at this point, and yes they are in fact legible? I'm not sure what you're asking -- you absolutely can (and if you use it long enough, will) read the source code of your text editor.
Well especially the lisp parts!
The little experience I have with LLM confidently shows that LLMs are much better at navigating and modifying a well structured code base. And they struggle, sometimes to a point where they can't progress at all, if tasked to work on bad code. I mean, the kind of bad you always get after multiple rounds of unsupervised vibe coding.
> I happen to rent and can't keep
This is my fear - what happens if the AI companies can't find a path to profitability and shut down?
Don't threaten us with a good time.
That’s not a good time, I love these things. I’ve been able to indulge myself so much. Possibly good for job security but would suck in every other way.
This is why local models are so important. Even if the non-local ones shut down, and even if you can't run local ones on your own hardware, there will still be inference providers willing to serve your requests.
Recently I was thinking about how some (expensive) customer electronics like the Mac Studio can run pretty powerful open source models with a pretty efficient power consumption, that could pretty easily run on private renewable energy, and that are on most (all?) fronts much more powerful than the original ChatGPT especially if connected to a good knowledge base. Meaning that aside from very extreme scenarios I think it is safe to say that there will always be a way not to go back to how we used to code, as long as we can offer the correct hardware and energy. Of course personally I think we will never need to go to such extreme ends... despite knowing of people who seem to seriously think developed countries heavily run out of electricity one day, which, while I reckon there might be tensions, seems like a laughable idea IMHO.
> the meta-skill of learning to use the LLM depreciates too. Today's LLM is gonna go away someday, the way you have to use it will change. You will be on a forever treadmill, always learning the vagaries of using the new shiny model (and paying for the privilege!)
I haven’t found this to be true at all, at least so far.
As models improve I find that I can start dropping old tricks and techniques that were necessary to keep old models in line. Prompts get shorter with each new model improvement.
It’s not really a cycle where you’re re-learning all the time or the information becomes outdated. The same prompt structure techniques are usually portable across LLMs.
Interesting, I’ve experienced the opposite in certain contexts. CC is so hastily shipped that new versions often imbalance existing workflows. E.g. people were raving about the new user prompt tools that CC used to get more context but they messed my simple git slash commands
I think you have to be aware of how you use any tool but I don’t think this is a forever treadmill. It’s pretty clear to me since early on that the goal is for you the user to not have to craft the perfect prompt. At least for my workflow it’s pretty darn close to that for me.
If it ever gets there, then anyone can use it and there's no "skill" to be learned at all.
Either it will continue to be this very flawed non-deterministic tool that requires a lot of effort to get useful code out of it, or it will be so good it'll just work.
That's why I'm not gonna heavily invest my time into it.
Good for you. Others like myself find the tools incredibly useful. I am able to knock out code at a higher cadence and it’s meeting a standard of quality our team finds acceptable.
Looking forward for those 10x improvements to finally show up somewhere. Any day now!
Jokes aside, I never said it's not useful, but most definitely it's not even close to all this hype.
> very flawed non-deterministic tool that requires a lot of effort to get useful code out of it
We are all different but I think most of us with open minds are the flaw in your statement.
I have deliberately moderated my use of AI in large part for this reason. For a solid two years now I've been constantly seeing claims of "this model/IDE/Agent/approach/etc is the future of writing code! It makes me 50x more productive, and will do the same for you!" And inevitabely those have all fallen by the wayside and been replaced by some new shiny thing. As someone who doesn't get intrinsic joy out of chasing the latest tech fad I usually move along and wait to see if whatever is being hyped really starts to take over the world.
This isn't to say LLMs won't change software development forever, I think they will. But I doubt anyone has any idea what kind of tools and approaches everyone will be using 5 or 10 years from now, except that I really doubt it will be whatever is being hyped up at this exact moment.
HN is where I keep hearing the “50× more productive” claims the most. I’ve been reading 2024 annual reports and 2025 quarterlies to see whether any of this shows up on the other side of the hype.
So far, the only company making loud, concrete claims backed by audited financials is Klarna and once you dig in, their improved profitability lines up far more cleanly with layoffs, hiring freezes, business simplification, and a cyclical rebound than with Gen-AI magically multiplying output. AI helped support a smaller org that eliminated more complicated financial products that have edge cases, but it didn’t create a step-change in productivity.
If Gen-AI were making tech workers even 10× more productive at scale, you’d expect to see it reflected in revenue per employee, margins, or operating leverage across the sector.
We’re just not seeing that yet.
I have friends who make such 50x productivity claims. They are correct if we define productivity as creating untested apps and games and their features that will never ship --- or be purchased, even if they were to ship. Thus, “productivity” has become just another point of contention.
100% agree. There are far more half-baked, incomplete "products" and projects out there now that it is easier to generate code. Generously, that doesn't necessarily equate to productivity.
I've agree with the fact that the last 10% of a project is the hardest part, and that's the part that Gen-AI sucks at (hell, maybe the 30%).
> If Gen-AI were making tech workers even 10× more productive at scale, you’d expect to see it reflected in revenue per employee, margins, or operating leverage across the sector.
If we’re even just talking a 2x multiplier, it should show up in some externally verifiable numbers.
I agree, and we might be seeing this but there is so much noise, so many other factors, and we're in the midst of capital re-asserting control after a temporary loss of leverage which might also be part of a productivity boost (people are scared so they are working harder).
The issue is that I'm not a professional financial analyst and I can't spend all day on comps so I can't tell through the noise yet if we're seeing even 2x related to AI.
But, if we're seeing 10x, I'd be finding it in the financials. Hell, a blind squirrel would, and it's simply not there.
Yes, I think there many issues in a big company that could hide a 2x productivity increase for a little while. But I'd expect it to be very visible in small companies and projects. Looking at things like number of games released on steam, new products launched on new product sites, or issues fixed on popular open source repos, you'd expect a 2x bump to be visible.
In my experience all technology has been like this though. We are on the treadmill of learning the new thing with our without LLMs. That's what makes tech work so fun and rewarding (for me anyway).
I assume you're living in a city. You're already renting out a lot of things to others (security, electricity, water, food, shelter, transportation), what is different with white collar work?
My apartment has been here for years and will be here for many more. I don't love paying rent on it but it certainly does get maintained without my having to do anything. And the rest of the infrastructure of my life is similarly banal. I ride Muni, eat food from Trader Joe's, and so on. These things are not going away and they don't require me to rewire my brain constantly in order to make use of them. The city infrastructure isn't stealing my ability to do my work, it just fills in some gaps that genuinely cannot be filled when working alone and I can trust it to keep doing that basically forever.
>the city gets destroyed
vs.
>a company goes bankrupt or pivots
I can see a few differences.
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise." - Socrates on Writing and Reading, Phaedrus 370 BC
If one reads the dialogue, Socrates is not the one "saying" this, but he is telling a story of what King Thamus said to the Egyptian god Theuth, who is the inventor of writing. He is asking the king to give out the writing, but the king is unsure about it.
Its what is known as one of the Socratic "myths," and really just contributes to a web of concepts that leads the dialogue to its ultimate terminus of aporia (being a relatively early Plato dialogue). Socrates, characteristically, doesn't really give his take on writing. In the text, he is just trying to help his friend write a horny love letter/speech!
I can't bring it up right now, but the end of the dialogue has a rather beautiful characterization of writing in the positive, saying that perhaps logos can grow out of writing, like a garden.
I think if pressed Socrates/Plato would say that LLM's are merely doxa machines, incapable of logos. But I am just spitballing.
https://standardebooks.org/ebooks/plato/dialogues/benjamin-j...
Phaedo != Phaedrus. One is the "writing" one, the other one is, well, about Socrates' execution (also extremely good dialogue!).
The one at issue:
https://standardebooks.org/ebooks/plato/dialogues/benjamin-j...
The public domain translations are pretty old either way. John Cooper's big book is probably still the best but im out of the game these days.
AI guys would probably love this if any of them still have the patience to read/comprehend something very challenging. Probably one of the more famous essays on the Phaedrus dialogue. Its the first long essay of this book:
https://xenopraxis.net/readings/derrida_dissemination.pdf
Roughly: Plato's subordination of writing in this text is symptomatic of a broader kind of `logocentrism` throughout all of western canonical philosophy. Derrida argues the idea of the "externality" of writing compared to speech/logos is not justified by anything, and in fact everything (language, thought) is more like a kind "writing."
Presenting this quote without additional commentary is an interesting Rorschach test.
Thankfully more and more people are seriously considering the effects of technology on true wisdom and getting of the "all technological progress clearly is great, look at all these silly unenlightened naysayers from the past" train.
Socrates was right about the effects. Writing did indeed cause us to loose the talent of memorizing. Where he was wrong though (or rather where this quote without context is wrong) is that it turned out that memorizing was by the most part not the important skill to have.
When Socrates uses the same warnings about LLMs he may however be correct both on the effect and the importance of the skill being lost. If we loose the ability to think and solve various problems, we may indeed be loosing a very important skill of our humanity.
You're misinterpreting the quote. Socrates is saying that being able to find a written quotation will replace fully understanding a concept. It's the difference between being able to quote the pythagorean theorem and understanding it well enough to prove it. That's why Socrates says that those who rely on reading will be "hard to get along with" - they will be pedantic without being able to discuss concepts freely.
Huh, I think you're right. I think I failed the litmus test. Thanks for explaining!
While there are dangers to LLMs -science fiction has been talking about this issue for decades (see below) and I think its overblown and the point of the Socrates quote is valid.
e.g the Matrix Reloaded: https://youtu.be/cD4nhYR-VRA?si=bXGBI4ca-LaetLVl&t=69 Machines no one understand or can manage
Issac Asmiov's Classic - the Feeling of Power https://ia600806.us.archive.org/20/items/TheFeelingOfPower/T...
(future scientists discover how to add using paper and pencil instead of computer)
I mean Big Paradigm shifts are like death, we can't really predict how humanity will evolve if we really get AGI -but these LLMs as they work today are tools and humans are experts at finding out how to use tools efficiently to counter the trade offs.
Does it really matter today that most programmers don't know how to code in assembly for example?
I’m not making a Malthusian doomsday prediction, and neither was Socrates for that matter. Jobs need to be done, and there will always be somebody willing and able to acquire the relevant skills, and do the job. And in the worst case scenario, society will change it self before it is allowed to fail.
Unlike Malthus, for whom it was easier to imagine the end of the world then the end of Mercantilism, I can easily imagine a world which simply replaces capitalism as its institutions start producing existential threats for humanity.
However, I don‘t think LLMs are even that, for me they are an annoyance which I personally want gone, but next to climate change and the stagnation of population growth, they wont make a dent in upending capitalism, despite how much they suck.
But just because they are not an existential threat, that doesn’t make them harmless. Plenty of people will be harmed by this technology. Like Socrates predicted people will lose skills, this includes skilled programmers, and where previously we were getting some quality software, instead we will get less of it, replaced with a bunch of AI slop. That is my prediction at least.
That is interesting because your mental abilities seem to be correlated with orchestrating a bunch of abstractions you have previously mastered. Are these tools making us stupid because we no longer need to master any of these things? Or are they making us smarter because the abstraction is just trusting AI to handle it for us?
Does a student become smarter by hiring a smarter student to write his essays and take his tests for him?
We can also invert that by asking: does a student become smarter by writing their essay on their own?
I would argue that the answer to questions is no. It depends on how you define “smarter”, though. You would likely gain knowledge writing the essay yourself, but is gaining knowledge equivalent to getting smarter?
If so, you could also just read the essay afterwards and gain the same knowledge. Is _that_ smarter? You’ve now reached the same benefit for much less work.
I think fundamentally I at least partially agree with your stance. That we should think carefully before taking a seemingly easier path. Weighing what we gain and lose. Sometimes the juice is, in fact, the squeeze. But it’s far from cut and dry.
It's unclear if you've presented this quote in order to support or criticize the idea that new technologies make us dumber. (Perhaps that's intentional; if so, bravo).
To me, this feels like support. I was never an adult who could not read or write, so I can't check my experience against Socrates' specific concern. But speaking to the idea of memory, I now "outsource" a lot of my memory to my smartphone.
In the past, I would just remember my shopping list, and go to the grocery store and get what I needed. Sure, sometimes I'd forget a thing or two, but it was almost always something unimportant, and rarely was a problem. Now I have my list on my phone, but on many occasions where I don't make a shopping list on my phone, when I get to the grocery store I have a lot of trouble remembering what to get, and sometimes finish shopping, check out, and leave the store, only to suddenly remember something important, and have to go back in.
I don't remember phone numbers anymore. In college (~2000) I had the campus numbers (we didn't have cell phones yet) of at least two dozen friends memorized. Today I know my phone number, my wife's, and my sister's, and that's it. (But I still remember the phone number for the first house I lived in, and we moved out of that house when I was five years old. Interestingly, I don't remember the area code, but I suppose that makes sense, as area codes weren't required for local dialing in the US back in the 80s.)
Now, some of this I will probably ascribe to age: I expect our memory gets more fallible as we get older (I'm in my mid 40s). I used to have all my credit/debit card numbers, and their expiration dates and security codes, memorized (five or six of them), but nowadays I can only manage to remember two of them. (And I usually forget or mix up the expiration dates; fortunately many payment forms don't seem to check, or are lax about it.) But maybe that is due to new technology to some extent: most/all sites where I spend money frequently remember my card for me (and at most only require me to enter the security code). And many also take Paypal or Google Pay, which saves me from having to recall the numbers.
So I think new technology making us "dumber" is a very real thing. I'm not sure if it's a good thing or a bad thing. You could say that, in all of my examples, technology serving the place of memory has freed up mental cycles to remember more important things, so it's a net positive. But I'm not so sure.
I don‘t think human memory works like that, at least not in theory. Storage is not the limiting factor of human memory, but rather retention. It takes time and effort to retain new information. In the past you spent some time and effort to memorize the shopping list and the phone number. Mulling it over in your mind (or out loud), repeated recalls, exposure, even mnemonic tricks like rhymes, alliterations, connecting with pictures, stories, etc. if what you had to remember was something more complicated/extensive/important. And retention is not forever, unless you repeat it, you will loose it. And you only have so much time for repetition and recall, so inevitably, there will be memories which won‘t be repeated, and can’t be recalled.
So when you started using technology to offload your memory, what you gained was the time and effort you previously spent encoding these things into your memory.
I think there is a fundamental difference though between phone book apps and LLMs. Loosing the ability to remember a phone number is not as severe as loosing the ability to form a coherent argument, or to look through sources, or for a programmer to work through logic, to abstract complex logic into simpler chunks. If a scholar looses the skill to look through sources, and a programmer looses the ability to abstract complex logic, they are loosing a fundamental part of their needed to do their jobs. This is like if a stage actor looses the ability to memorize the script, instead relying on a tape-recorder when they are on stage.
Now if a stage actor losses the ability to memorize the script, they will soon be out of a job, but I fear in the software industry (and academia) we are not so lucky. I suspect we will see a lot of people actually taking that tape recorder on stage, and continue to do their work as if nothing is more normal. And the drop in quality will predictably follow.
I like this commentary on that passage : https://detective-pony.tumblr.com/post/96180330151/pages-21-...
And so we learn that over 2000 years before the microphone came to be, Socrates invented the mic drop.
In all seriousness though, it's just crazy that anybody is thinking these things at the dawn of civilization.
Yup.
My personal counterpoint is Norman's thesis in Things That Make Us Smart.
I've long tried, and mostly failed, to consider the tradeoffs, to be ever mindful that technologies are never neutral (winners & losers), per Postman's Technopoly.
Well, the wisdom part is true.
He was right. It did.
Writing/reading and AI are so categorically different that the only way you could compare them is if you fundamentally misunderstand how both of them work.
And "other people in the past predicted doom about something like this and it didn't happen" is a fallacious non-argument even when the things are comparable.
The argument Socrates is making is specifically that writing isn't a substitute for thinking, but it will be used as such. People will read things "without instruction" and claim to understand those things, even if they do not. This is a trade-off of writing. And the same thing is happening with LLMs in a widespread manner throughout society: people are having ChatGPT generate essays, exams, legal briefs and filings, analyses, etc., and submitting them as their own work. And many of these people don't understand what they have generated.
Writing's invention is presented as an "elixir of memory", but it doesn't transfer memory and understanding directly - the reader must still think to understand and internalize information. Socrates renames it an "elixir of reminding", that writing only tells readers what other people have thought or said. It can facilitate understanding, but it can also enable people to take shortcuts around thinking.
I feel that this is an apt comparison, for example, for someone who has only ever vibe-coded to an experienced software engineer. The skill of reading (in Socrates's argument) is not equivalent to the skill of understanding what is read. Which is why, I presume, the GP posted it in response to a comment regarding fear of skill atrophy - they are practicing code generation but are spending less time thinking about what all of the produced code is doing.
yes, but people just really like to predict dooms and they also like to be convinced that they live in some special era in human history
It takes about 30 seconds of thinking and/or searching the Internet to realize that people also predict doom when it actually happens - e.g. with people correctly predicting that TikTok will shorten people's attention spans.
It's then quite obvious that the fact that someone, somewhere, predicts a bad thing happening has ~zero bearing on whether it actually happens, and so the claim that "someone predicted doom in the past and it didn't happen then so someone predicting doom now is also wrong" is absurd. Calling that idea "intellectually lazy" is an insult to smart-but-lazy people. This is more like intellectually incapable.
The fact that people will unironically say such a thing in the face of not only widespread personal anecdotes from well-respected figures, but scientific evidence, is depressing. Maybe people who say these things are heavy LLM users?
There is always some set of people predicting all sorts of dooms though. The saying about the broken clock comes to mind.
With the right cherry picking, it can always be said that [some set of] the doomsayers were right, or that they were wrong.
As you say, someone predicting doom has no bearing on whether it happens, so why engage in it? It's just spreading FUD and dwelling on doom. There's no expected value to the individual or to others.
Personally, I don't think "TikTok will shorten people's attention spans" qualifies as doom in and of itself.
Did you actually read what you're responding to?
> And "other people in the past predicted doom about something like this and it didn't happen" is a fallacious non-argument even when the things are comparable.
> the claim that "someone predicted doom in the past and it didn't happen then so someone predicting doom now is also wrong" is absurd
It's pretty clear that I'm not defending engaging in baseless negative speculation, but refuting the dismissal of negative speculation based purely on the trope that "people have always predicted it".
Someone who read what they were responding to would rather easily have seen that.
> As you say, someone predicting doom has no bearing on whether it happens
That is not what I said. I'm pretty sure now that you did not read my comment before responding. That's bad.
This is what I said:
> It's then quite obvious that the fact that someone, somewhere, predicts a bad thing happening has ~zero bearing on whether it actually happens, and so the claim that "someone predicted doom in the past and it didn't happen then so someone predicting doom now is also wrong" is absurd.
I'm very clearly pointing out (with "someone, somewhere") that a random person predicting a bad thing has almost no ("~zero") impact on the future. Obviously, if someone who has the ability to affect the future (e.g. a big company executive, or a state leader (past or present)) makes a prediction, they have much more power to actually affect the future.
> so why engage in it? It's just spreading FUD and dwelling on doom.
Because (rational) discussion now has the capacity to drive change.
> There's no expected value to the individual or to others.
Trivially false - else most social movements would be utterly irrelevant, because they work through the same mechanism - talking about things that should be changed as a way of driving that change.
It's also pretty obvious that there's a huge difference between "predicting doom with nothing behind it" and "describing actual bad things that are happening that have a lot of evidence behind them" - which is what is actually happening here, so all of your arguments about the former point would be irrelevant (if they were valid, which they aren't) because that's not even the topic of discussion.
I suggest reading what you're responding to before responding.
> Personally, I don't think "TikTok will shorten people's attention spans" qualifies as doom in and of itself.
You're bringing up "doom" as a way to pedantically quarrel about word definitions. It's trivial to see that that's completely irrelevant to my argument - and worth noting that you're then conceding the point about people correctly predicting that TikTok will shorten people's attention spans, hence validating the need to have discussions about it.
We are very clearly living through a moment in history that will be studied intensely for thousands of years.
Because of the collapsing empire, mind you, not because of the LLMs.
Creation of the internet, social media, everyone on the planet getting a pocket sized supercomputer, beginning of the AI boom, Trump/beginning of the end of the US, are all reasons people will study this period of time.
This is really interesting because I wholeheartedly believe the original sentiment that everyone thinks their generation is special, and that "now this time they've really screwed it all up" is quite myopic -- and that human nature and the human experience are relatively constant throughout history while the world changes around us.
But, it is really hard to escape the feeling that digital technology and AI are a huge inflection point. In some ways this couple generations might be the singularity. Trump and contemporary geopolitics in general is a footnote, a silly blip that will pale in comparison over time.
I know managers who can read code just fine, they're just not able/willing to code it. Tho the ai helps with that too. I've had a few managers dabble back into coding esp scripts and whatnot where I want them to be pulling unique data and doing one off investigations.
I read grandparent comment as saying people have been claiming that the sky is falling forever… AI will be both good for learning and development and bad. It’s always up to the individual if it benefits them or atrophies their minds.
I'm not a big fan of LLMs, but while using it for day to day tasks, I get the same feeling I had when I first started the internet (I was lucky to start with broadband internet).
That feeling was one of empowerment: I was able to satisfy my curiosity about a lot of topics.
LLMs can do the same thing and save me a lot of time. It's basically a super charged Google. For programming it's a super charged auto complete coupled with a junior researcher.
My main concern is independence. LLMs in the hands of just a bunch of unchecked corporations are extremely dangerous. I kind of trusted Google, and even that trust is eroding, and LLMs can be extremely personal. The lack of trust ranges from risk of selling data and general data leaks, to intrusive and worse, hidden ads, etc.
When I first started using the internet, I was able to instant text message (IRC) random strangers, using a fake name, and lie about my age. My teacher had us send an email to our ex-classmate who had move to Australia, and she replied the next day, I was able to download the song I just heard on the radio and play it as many times as I wanted on my winamp.
These capabilities simply didn’t exist before the Internet. Apart for the email to Australia (which was possible with a fax machine; but much more expensive), LLMs don‘t give you any new capabilities. It just provides a way for you to do what you already can (and should) do with your brain, without using your brain. It is more like using replacing your social interaction with facebook, then it is to experience an instant message group chat for the first time.
Before LLMs it was incredibly tedious or expensive or both to get legal guidance for stuff like taxes, where I live. Now I can orient myself much better before I ask an actual tax expert pointed questions, saving a lot of time and money.
The list of things they can provide is endless.
They're not a creator, they're an accelerator.
And time matters. My interests are myriad but my capacity to pass the entry bar manually is low because I can only invest so much time.
If this resembles the feeling you had when you first used the internet, it is drastically different from when I used the internet.
When I first used the internet, it was not about doing things faster, it was about doing things which were previously simply unavailable to me. A 12 year old me was never gonna fax my previous classmate who moved to Australia, but I certainly emailed her.
We are not talking about a creator nor an accelerator, we are talking about an avenue (or a road if you will). When I use the internet, I am the creator, and the internet is the road that gets me there.
When I use an LLM it is doing something I can already do, but now I can do it without using my brain. So the feeling is much closer to doomscrolling on social media where previously I could just read a book or meet my pals at the pub. Doomscrolling facebook is certainly faster then reading a book, or socializing at the pub. But it is a poor replacement for either.
I didn't have friends in other countries.
I could however greatly enrich my general knowledge in ways I couldn't do with books I had access to.
Prior to the internet I used my school library for that (or when I was very young, books at my grandparent’s house). So for me personally that wasn’t a new capability. It wasn’t until I started using Wikipedia around 2004 (when I was 17 years old) that the internet replaced (or rather complemented) libraries for that function.
But I can definitely see how for many people with less access to libraries (or worse quality libraries then what I had access to) the internet provided a new avenue for gaining knowledge which wasn’t available before.
To understand the impact on computer programming per se, I find it useful to imagine that the first computer programs I had encountered were, somehow, expressed in a rudimentary natural language. That (somewhat) divorces the consideration of AI from its specific impact on programming. Surely it would have pulled me in certain directions. Surely I would have had less direct exposure to the mechanics of things. But, it seems to me that’s a distinction of degree, not of kind.
I think I should write more about but I have been feeling very similar. I've been recently exploring using claude code/codex recently as the "default", so I've decided to implement a side project.
My gripe with AI tools in the past is that the kind of work I do is large and complex and with previous models it just wasn't efficient to either provide enough context or deal with context rot when working on a large application - especially when that application doesn't have a million examples online.
I've been trying to implement a multiplayer game with server authoritative networking in Rust with Bevy. I specifically chose Bevy as the latest version was after Claude's cut off, it had a number of breaking changes, and there aren't a lot of deep examples online.
Overall it's going well, but one downside is that I don't really understand the code "in my bones". If you told me tomorrow that I had optimize latency or if there was a 1 in 100 edge case, not only would I not know where to look, I don't think I could tell you how the game engine works.
In the past, I could not have ever gotten this far without really understanding my tools. Today, I have a semi functional game and, truth be told, I don't even know what an ECS is and what advantages it provides. I really consider this a huge problem: if I had to maintain this in production, if there was a SEV0 bug, am I confident enough I could fix it? Or am I confident the model could figure it out? Or is the model good enough that it could scan the entire code base and intuit a solution? One of these three questions have to be answered or else brain atrophy is a real risk.
I'm worried about that too. If the error is reproducible, the model can eventually figure it out from experience. But a ghost bug that I can't pattern? The model ends up in a "you're absolutely right" loop as it incorrectly guesses different solutions.
Are ghost bugs even real?
My first job had the Devs working front-line support years ago. Due to that, I learnt an important lessons in bug fixing.
Always be able to re-create the bug first.
There are no such thing as ghost bugs, you just need to ask the reporter the right questions.
Unless your code is multi-threaded, to which I say, good luck!
They're real at scale. Plenty of bugs don't suface until you're running under heavy load on distributed infrastructure. Often the culprit is low in the stack. Asking the reporter the right questions may not help in this case. You have full traces, but can't reproduce in a test environment.
When the cause is difficult to source or fix, it's sometimes easier to address the effect by coding around the problem, which is why mature code tends to have some unintuitive warts to handle edge cases.
> Unless your code is multi-threaded, to which I say, good luck!
What isn't multi-threaded these days? Kinda hard to serve HTTP without concurrency, and practically every new business needs to be on the web (or to serve multiple mobile clients; same deal).
All you need is a database and web form submission and now you have a full distributed system in your hands.
Only superficially so, await/async isn't usually like the old spaghetti multi-threaded code people used to write.
You mean in a single-threaded context like Javascript? (Or with Python GIL giving the impression of the same.) That removes some memory corruption races, but leaves all the logical problems in place. The biggest change is that you only have fixed points where interleaving can happen, limiting the possibilities -- but in either scenario, the number of possible paths is so big it's typically not human-accessible.
Webdevs not aware of race conditions -> complex page fails to load. They're lucky in how the domain sandboxes their bugs into affecting just that one page.
nginx is single–threaded, but you're absolutely right — any concurrency leads to the same ghost bugs.
nginx is also from the era when fast static file serving was still a huge challenge, and "enough to run a business" for many purposes -- most software written has more mutable state, and much more potential for edge cases.
Historically I would have agreed with you. But since the rise of LLM-assisted coding, I've encountered an increasing number of things I'd call clear "ghost bugs" in single threaded code. I found a fun one today where invoking a process four times with a very specific access pattern would cause a key result of the second invocation to be overwritten. (It is not a coincidence, I don't think, that these are exactly the kind of bugs a genAI-as-a-service provider might never notice in production.)
> I've been trying to implement a multiplayer game with server authoritative networking in Rust with Bevy. I specifically chose Bevy as the latest version was after Claude's cut off, it had a number of breaking changes, and there aren't a lot of deep examples online.
I am interested in doing something similar (Bevy. not multiplayer).
I had the thought that you ought be able to provide a cargo doc or rust-analyzer equivalent over MCP? This... must exist?
I'm also curious how you test if the game is, um... fun? Maybe it doesn't apply so much for a multiplayer game, I'm thinking of stuff like the enemy patterns and timings in a soulslike, Zelda, etc.
I did use ChatGPT to get some rendering code for a retro RCT/SimCity-style terrain mesh in Bevy and it basically worked, though several times I had to tell it "yeah uh nothing shows up", at which point is said "of course! the problem is..." and then I learned about mesh winding, fine, okay... felt like I was in over my head and decided to go to a 2D game instead so didn't pursue that further.
>I had the thought that you ought be able to provide a cargo doc or rust-analyzer equivalent over MCP? This... must exist?
I've found that there are two issues that arise that I'm not sure how to solve. You can give it docs and point to it and it can generally figure out syntax, but the next issue I see is that without examples, it kind of just brute forces problems like a 14 year old.
For example, the input system originally just let you move left and right, and it popped it into an observer function. As I added more and more controls, it began to litter with more and more code, until it was ~600 line function responsible for a large chunk of game logic.
While trying to parse it I then had it refactor the code - but I don't know if the current code is idiomatic. What would be the cargo doc or rust-analyzer equivalent for good architecture?
Im running into this same problem when trying to claude code for internal projects. Some parts of the codebase just have really intuitive internal frameworks and claude code can rip through them and provide great idiomatic code. Others are bogged down by years of tech debt and performance hacks and claude code can't be trusted with anything other than multi-paragraph prompts.
>I'm also curious how you test if the game is, um... fun?
Lucky enough for me this is a learning exercise, so I'm not optimizing for fun. I guess you could ask claude code to inject more fun.
> What would be the cargo doc or rust-analyzer equivalent for good architecture?
Well, this is where you still need to know your tools. You should understand what ECS is and why it is used in games, so that you can push the LLM to use it in the right places. You should understand idiomatic patterns in the languages the LLM is using. Understand YAGNI, SOLID, DDD, etc etc.
Those are where the LLMs fall down, so that's where you come in. The individual lines of code after being told what architecture to use and what is idiomatic is where the LLM shines.
What you describe is how I use LLM tools today, but the reason I am approaching my project in this way is because I feel I need to brace myself for a future where developers are expected to "know your tools"
When I look around today - its clear more and more people are diving in head first into fully agentic workflows and I simply don't believe they can churn out 10k+ lines of code today and be intimately familiar with the code base. Therefore you are left with two futures:
* Agentic-heavy SWEs will eventually blow up under the weight of all their tech debt
* Coding models are going to continue to get better where tech debt wont matter.
If the answer if (1), then I do not need to change anything today. If the answer is (2), then you need to prepare for a world where almost all code is written by an agent, but almost all responsibility is shouldered by you.
In kind of an ignorant way, I'm actually avoiding trying to properly learn what an ECS is and how the engine is structured, as sort of a handicap. If in the future I'm managing a team of engineers (however that looks) who are building a metaphorical tower of babel, I'd like to develop to heuristic in navigating that mountain.
I ran into similar issues with context rot on a larger backend project recently. I ended up writing a tool that parses the AST to strip out function bodies and only feeds the relevant signatures and type definitions into the prompt.
It cuts down the input tokens significantly which is nice for the monthly bill, but I found the main benefit is that it actually stops the model from getting distracted by existing implementation details. It feels a bit like overengineering but it makes reasoning about the system architecture much more reliable when you don't have to dump the whole codebase into the context window.
> I don't really understand the code "in my bones".
Man, I absolutely hate this feeling.
I've been thinking along these lines. LLMs seem to have arrived right when we were all getting addicted to reels/tic tocks/whatever. For some reason we love to swipe, swipe, swipe, until we get something funny/interesting/shocking, that gives us a short-lasting dopamine hit (or whatever chemicals it is) that feels good for about 1 second, and we want MORE, so we keep swiping.
Using an LLM is almost exactly the same. You get the occasional, "wow! I've never seen it do that before!" moments (whether that thing it just did was even useful or not), get a short hit of feel goods, and then we keep using it trying to get another hit. It keeps providing them at just the right intervals for people to keep them going just like they do with tick tock
It's exactly the argument here:
https://www.fast.ai/posts/2026-01-28-dark-flow/
I ran into a new problem today: "reading atrophy".
As in if the LLM doesn't know about it, some devs are basically giving up and not even going to RTFM. I literally had to explain to someone today how something works by...reading through the docs and linking them the docs with screenshots and highlighted paragraphs of text.
Still got push back along the lines of "not sure if this will work". It's. Literally. In. The. Docs.
That's not really a new thing now, it just shows differently.
15 years ago I was working in an environment where they had lots of Indians as cheap labour - and the same thing will show up in any environment where you go for hiring a mass of cheap people while looking more at the cost than at qualifications: You pretty much need to trick them into reading stuff that are relevant.
I remember one case where one had a problem they couldn't solve, and couldn't give me enough info to help remotely. In the end I was sitting next to them, and made them read anything showing up on the screen out loud. Took a few tries where they were just closing dialog boxes without reading it, but eventually we had that under control enough that they were able to read the error messages to me, and then went "Oh, so _that's_ the problem?!"
Overall interacting with a LLM feels a lot like interacting with one of them back then, even down to the same excuses ("I didn't break anything in that commit, that test case was never passing") - and my expectation for what I can get out of it is pretty much the same as back then, and approach to interacting with it is pretty similar. It's pretty much an even cheaper unskilled developer, you just need to treat it as such. And you don't pair it up with other unskilled developers.
The mere existence of the phrase "RTFM" shows that this phenomenon was already a thing. LLMs are the worst thing to happen to people who couldn't read before. When HR type people ask what my "superpower" is I'm so tempted to say "I can read", because I honestly feel like it's the only difference between me and people who suck at working independently.
As someone working in technical support for a long time, this has always been the case.
You can have as many extremely detailed and easy to parse gudies, references, etc. there will always be a portion of customers who refuse to read them.
Never could figure out why because they aren't stupid or anything.
> Never could figure out why because they aren't stupid or anything.
They may be intelligent, but they don't sound wise.
> Eventually it was easier just to quit fighting it and let it do things the way it wanted.
I wouldn't have believed it a few tears ago if you told me the industry would one day, in lockstep, decide that shipping more tech-debt is awesome. If the unstated bet doesn't pay off, that is, AI development will outpace the rate it generates cruft, then there will be hell to pay.
Don't worry. This will create the demand for even more powerful models that are able to untangle the mess created by previous models.
Once we realize the kind of mess _those_ models created, well, we'll need even more capable models.
It's a variation on the theme of Kernighan insight about the more "clever" you are while coding the harder it will be to debug.
EDIT: Simplicity is a way out but it's hard under normal circumstances, now with this kind of pressure to ship fast because the colleague with the AI chimp can outperform you, aiming at simplicity will require some widespread understanding
"That's the brilliant part: when the winter comes the apes freeze to death!"
As someone who's been commissioned many times before to work on or salvage "rescue projects" with huge amounts of tech debt, I welcome that day. Still not there yet though I am starting to feel the vibes shifting.
This isn't anything new of course. Previously it was with projects built by looking for the cheapest bidder and letting them loose on an ill-defined problem. And you can just imagine what kind of code that produced. Except the scale is much larger.
My favorite example of this was a project that simply stopped working due to the amount of bugs generated from layers upon layers of bad code that was never addressed. That took around 2 years of work to undo. Roughly 6 months to un-break all the functionality and 6 more months to clean up the core and then start building on top.
Are you not worried that the sibling comment is right and the solution to this will be "more AI" in the future? So instead of hiring a team of human experts to cleanup, management might just dump more money into some specialized AI refactoring platform or hire a single AI coordinator... Or maybe they skip to rebuild using AI faster, because AI is good at greenfield. Then they only need a specialized migration AI to automate the regular switchovers.
I used to be unconcerned, but I admit to be a little frightened of the future now.
Well, in general worrying about the future is not useful. Regardless of what you think, it is always uncertain. I specifically stay away from taking part in such speculative threads here on HN.
What's interesting to me though is that very similar promises were being made about AI in the 80s. Then came the "AI Winter" after the hype cycle and promises got very far from reality. Generative AI is the current cycle and who knows, maybe it can fulfill all the promises and hype. Or maybe not.
There's a lot of irrationality currently and until that settles down, it is difficult to see what is real and useful and what is smoke and mirrors.
I'm aware of that particular chapter of history, my master's thesis was on conversational interfaces. I don't think the potential of the algorithms (and hardware) back then was in any way comparable to what's currently going on. There is definitely a hype cycle going on right now, but I'm nearly convinced it will actually leave many things changed even after it plays out.
Funny thing is that meanwhile (today) I've actually been on an emergency consulting project where a PO/PM kind of guy vibecoded some app that made it into production. The thing works, but a cursory audit laid open the expected flaws (like logic duplication, dead code, missing branches). So that's another point for our profession still being required in the near future.
> ... few tears ago
Brilliant. Even if it was a typo.
The industry decided that decades ago. We may like to talk about quality and forethought, but when you actually go to work, you quickly discover it doesn't matter. Small companies tell you "we gotta go fast", large companies demand clear OKRs and focusing on actually delivering impact - either way, no one cares about tech debt, because they see it as unavoidable fact of life. Even more so now, as ZIRP went away and no one can afford to pay devs to polish the turd ad infinitum. The mantra is, ship it and do the next thing, clean up the old thing if it ever becomes a problem.
And guess what, I'm finally convinced they're right.
Consider: it's been that way for decades. We may tell ourselves good developers write quality code given the chance, but the truth is, the median programmer is a junior with <5 years of experience, and they cannot write quality code to save their life. That's purely the consequence of rapid growth of software industry itself. ~all production code in the past few decades was written by juniors, it continues to be so today; those who advance to senior level end up mostly tutoring new juniors instead of coding.
Or, all that put another way: tech debt is not wrong. It's a tool, a trade-off. It's perfectly fine to be loaded with it, if taking it lets you move forward and earn enough to afford paying installments when they're due. Like with housing: you're better off buying it with lump payment, or off savings in treasury bonds, but few have that money on hand and life is finite, so people just get a mortgage and move on.
--
Edited to add: There's a silver lining, though. LLMs make tech debt legible and quantifiable.
LLMs are affected by tech debt even more than human devs are, because (currently) they're dumber, they have less cognitive capability around abstractions and generalizations[0]. They make up for it by working much faster - which is a curse in terms of amplifying tech debt, but also a blessing, because you can literally see them slowing down.
Developer productivity is hard to measure in large part because the process is invisible (happens in people's heads and notes), and cause-and-effect chains play out over weeks or months. LLM agents compress that to hours to days, and the process itself is laid bare in the chat transcript, easy to inspect and analyze.
The way I see it, LLMs will finally allow us to turn software development at tactical level from art into an engineering process. Though it might be too late for it to be of any use to human devs.
--
[0] - At least the out-of-distribution ones - quirks unique to particular codebase and people behind it.
> the median programmer is a junior with <5 years of experience, and they cannot write quality code to save their life
It's worse, look up perpetual intermediates. Most people in any given field aren't good at what they're doing. They're at best mediocre.
Sure, but we have high growth on top of that - meaning all those "perpetual intermediaries" are always the minority and gravitate upwards in the org chain, while ~all the coding work is done by people who just started working in the field, and didn't even learn enough yet to become mediocre.
> I wouldn't have believed it a few tears ago if you told me the industry would one day, in lockstep, decide that shipping more tech-debt is awesome.
It's not debt if you never have to pay it back. If a model can regenerate a whole relibale codebase in minutes from a spec, then your assessment of "tech debt" in that output becomes meaningless.
> unstated bet
(except where it's been stated, championed, enforced, and ultimated in no unequivocal terms by every executive in the tech industry)
I'm yet to encounter an AI-bull who admits the LLM tendency towards creating tech debt- outside of footnotes stating it can be fixed by better prompting (with no examples), or solved by whatever tool they are selling
My disillusionment comes from the feeling I am just cosplaying my job. There is nothing to distinguish one cosplayer from another. I am just doordashing software, at this point, and I'm not in control.
I don’t get this at all. I’m using LLM’s all day and I’m constantly having to make smart architectural choices that other less experienced devs won’t be making. Are you just prompting and going with whatever the initial output is, letting the LLM make decisions? Every moderately sized task should start with a plan, I can spend hours planning, going off and thinking, coming back to the plan and adding/changing things, etc. Sometimes it will be days before I tell the LLM to “go”. I’m also constantly optimising the context available to the LLM, and making more specific skills to improve results. It’s very clear to me that knowledge and effort is still crucial to good long term output… Not everyone will get the same results, in fact everyone is NOT getting the same results, you can see this by reading the wildly different feedback on HN. To some LLM’s are a force multiplier while others claim they can’t get a single piece of decent output…
I think the way you’re using these tools that makes you feel this way is a choice. You’re choosing to not be in control and do as little as possible.
Exactly.
Once you start using it intelligently, the results can be really satisfying and helpful. People complaining about 1000 lines of codes being generated? Ask it to generate functions one at a time and make small implementations. People complaining about having to run a linter? Ask it to automatically run it after each code execution. People complaining about losing track? Have it log every modifications in a file.
I think you get my point. You need to treat it as a super powerful tool that can do so many things that you have to guide it if you want to have a result that conforms to what you have in mind.
One challenge is, are those decisions making tangible differences?
We won't know until the code being produced especially greenfields hits any kind of maturity 5 years+ atleast?
It's not that challenging, the answer is, it depends.
It's like a junior dev writing features for a product everyday vs a principle engineer. The junior might be adding a feature with O(n^2) performance while principle has seen this before and writes it O(log n).
If the feature never reaches significance, the "better" solution doesn't matter, but it might!
The principle may write once and it is solid and never touched, but the junior might be good enough to never need coming back to, same with a llm and the right operator.
There's that, but I actually think LLMs are becoming very good at not making the bad simple choice.
What they're worse at is the bits I can't easily see.
An example is that I recently was working on a project building a library with Claude. The code in pieces all looked excellent.
When I wrote some code making use of it several similar functions which were conceptually similar had signatures that were subtly mismatched.
Different programmers might have picked each patterns. And probably consistently made similar rules for the various projects they worked on.
To an LLM they are just happenstances and feel no friction.
A real project with real humans writing the code would notice the mismatch. Even if they aren't working on those parts at the same time just from working on it across say a weekend.
But how many more decisions do we make convenient only for us meat bags that a LLM doesn't notice?
Yes, but now you know about that classification of problem. So you learned something! As an Engineer have a choice now on what to do with that classification or problem.
Better yet, go up one level and and think about how to avoid the other classifications of problems you don't know about, how can the LLM catch these before it writes the code... etc.
What? Of course it makes a difference when I direct it away from a bad solution towards a good solution. I know as soon as I review the output and it has done what I asked, or it hasn't and I make a correction. Why would I need to wait 5 years? That makes no sense, I can see the output.
If you're using LLM's and you don't know what good/bad output looks like then of course you're going to have problems, but such a person would have the same problems without the LLM...
The problem is the LLMs are exceptionally good at producing output that appears good.
That's what it's ultimately been tuned to do.
The way I see this play out is output that satisfied me but that I would not produce myself.
Over a large project that adds up and typically is glaringly obvious to everyone but the person who was using the LLM.
My only guess as to why that is, is because most of what we do and why we do it we're not conscious of. The threshold we'd intervene at is higher than the original effort it takes to do the right thing.
If these things don't apply to you. Then I think you're coming up to a golden era.
100% there....it's getting to a point where a project manager reports a bug AND also pastes a response from Claude (he ran Claude against our codebase) on how to fix the bug..Like I'm just copying what Claude said and making sure the thing compiles (.NET). What makes me sleep at night...for now is the fact that Claude isn't supporting 9pm deployments and AWS Infra support ...it's already writing code but not supporting it yet...
What kind of software are you writing? Are you just a "code monkey" implementing perfectly described Jira tickets (no offense meant)? I cannot imagine feeling this way with what I'm working on, writing code is just a small part of it, most of the time is spent trying to figure out how to integrate the various (undocumented and actively evolving) external services involved together in a coherent, maintainable and resilient way. LLMs absolutely cannot figure this out themselves, I have to figure it out myself and then write it all in its context, and even then it mostly comes up with sub-par, unmaintainable solutions if I wasn't being precise engouh.
They are amazing for side projects but not for serious code with real world impact where most of the context is in multiple people's head.
No, I am not a code monkey. I have an odd role working directly for an exec in a highly regulated industry, managing their tech pursuits/projects. The work can range from exciting to boring depending on the business cycle. Currently it is quite boring, so I've leaned into using AI a bit more just to see how I like it. I don't think that I do.
I find the atrophy and zoning out or context switching problematic, because it takes a few seconds/ minutes in "thinking" and then BAM! I have 500 lines of all sorts of buggy and problematic code to review and get a sycophantic, not-enough-mature entity to correct.
At some point, I find myself needing to disconnect out of overwhelm and frustration. Faster responses isn't necessarily better. I want more observability in the development process so that I can be a party to it. I really have felt that I need to orchestrate multiple agents working in tandem, playing sort of a bad-cop, good-cop and a maybe a third trying to moderate that discussion and get a fourth to effectively incorporate a human in the mix. But that's too much to integrate in my day job.
I’ve actually found the tool that inspires the most worry about brain atrophy to be Copilot. Vscode is full of flashing suggestions all over. A couple days ago, I wanted to write a very quick program, and it was basically impossible to write any of it without Copilot suggesting a whole series of ways to do what it thought I was doing. And it seems that MS wants this: the obvious control to turn it off is actually just “snooze.”
I found the setting and turned it off for real. Good riddance. I’ll use the hotkey on occasion.
Yes! I spent more time trying to figure out how to turn off that garbage copilot suggesting then I did editing this 5 year old python program.
I use claude daily, no problems with it. But vscode + copilot suggestions was garbage!
> Like there have been multiple times now where I wanted the code to look a certain way, but it kept pulling back to the way it wanted to do things. Like if I had stated certain design goals recently it would adhere to them, but after a few iterations it would forget again and go back to its original approach, or mix the two, or whatever. Eventually it was easier just to quit fighting it and let it do things the way it wanted.
Absolutely. At a certain level of usage, you just have to let it do it's thing.
People are going to take issue with that. You absolutely don't have to let it do its thing. In that case you have to be way more in the loop. Which isn't necessarily a bad thing.
But assuming you want it to basically do everything while you direct it, it becomes pointless to manage certain details. One thing in my experience is that Claude always wants to use ReactRouter. My personal preference is TanStack router, so I asked it to use it initially. That never really created any problems but after like the 3rd time of realizing I forget to specify it, I also realized that it's totally pointless. ReactRouter works fine and Claude uses it fine - its pointless to specify otherwise.
My experience is the opposite - I haven't used my brain more in a while.. Typing characters was never what developers were valued for anyway. The joy of building is back too.
100% same, I had brain fog before the llms, I got tired of reading new docs over and over again for new languages. I became a manager and lost it all.
Now back to IC with 25+ years of experience + LLM = god mode, and its fun again.
Same. I feel I need to be way more into the domain and what the user is trying to do than ever before.
Another thing I’ve experienced is scope creep into the average. Both Claude and ChatGPT keep making recommendations and suggestions that turns the original request into something that resembles other typical features. Sometimes that’s a good thing, because it means I’ve missed something. A lot of times, especially when I’m just riffing on ideas, it turns into something mundane and ordinary and I’ll have lost my earlier train of thought.
A quick example is trying to build a simple expenses app with it. I just want to store a list of transactions with it. I’ve already written the types and data model and just need the AI to give me the plumping. And it will always end up inserting recommendations about double entry bookkeeping.
yeah but that's like recommending a webserver for your Internet facing website. If you want to give an example of scope creep, you need a better example than double entry book keeping for an accounting app.
You’ve just illustrated exactly the problem. You assumed I was building an accounting app. I’ve experienced the same issue with building features for calculating the brightness of a room, or 3D visualisations of brightness patterns, managing inventory and cataloguing lighting fixtures and so on.
It’s great for churning out stuff that already exists, but that also means it’ll massage your idea into one of them.
> I want to say it's very akin to doom scrolling. Doom tabbing? It's like, yeah I could be more creative with just a tad more effort, but the AI is already running and the bar to seeing what the AI will do next is so low, so....
Yea exactly, Like we are just waiting so that it gets completed and after it gets completed then what? We ask it to do new things again.
Just as how if we are doom scrolling, we watch something for a minute then scroll down and watch something new again.
The whole notion of progress feels completely fake with this. Somehow I guess I was in a bubble of time where I had always end up using AI in web browsers (just as when chatgpt 3 came) and my workflow didn't change because it was free but recently changed it when some new free services dropped.
"Doom-tabbing" or complete out of the loop AI agentic programming just feels really weird to me sucking the joy & I wouldn't even consider myself a guy particular interested in writing code as I had been using AI to write code for a long time.
I think the problem for me was that I always considered myself a computer tinker before coder. So when AI came for coding, my tinkering skills were given a boost (I could make projects of curiosity I couldn't earlier) but now with AI agents in this autonomous esque way, it has come for my tinkering & I do feel replaced or just feel like my ability of tinkering and my interests and my knowledge and my experience is just not taken up into account if AI agent will write the whole code in multi file structure, run commands and then deploy it straight to a website.
I mean my point is tinkering was an active hobby, now its becoming a passive hobby, doom-tinkering? I feel like I have caught up on the feeling a bit earlier with just vibe from my heart but is it just me who feels this or?
What could be a name for what I feel?
> I worry about the "brain atrophy" part, as I've felt this too. And not just atrophy, but even moreso I think it's evolving into "complacency".
Not trusting the ML's output is step one here, that keeps you intellectually involved - but it's still a far cry from solving the majority of problems yourself (instead you only solve problems ML did a poor job at).
Step two: I delineate interesting and uninteresting work, and Claude becomes a pair programmer without keyboard access for the latter - I bounce ideas off of it etc. making it an intelligent rubber duck. [Edit to clarify, a caveat is that] I do not bore myself with trivialities such as retrieving a customer from the DB in a REST call (but again, I do verify the output).
> I do not bore myself with trivialities such as retrieving a customer from the DB in a REST call
Genuine question, why isn't your ORM doing that? I see a lot of use cases for LLMs that seem to be more expensive ways to do snippets and frameworks...
An ORM doesn't generate REST endpoints?
I've gone years without coding and when I come back to it, it's like riding a bike! In each iteration of my coding career, I have become a better developer, even after a large gap. Now I can "code" during my gap. Were I ever to hand-code again, I'm sure my skills would be there. They don't atrophy, like your ability to ride a bike doesn't atrophy. Yes you may need to warm back up, but all the connections in your brain are still there.
Have you ever learnt a foreign language (say Mongolian, or Danish) and then never spoken it, nor even read anything in it for over 10 years? It is not like riding a bike, it doesn’t just come back like that. You have to actually relearn the language, practice it, and you will suck at it for months. Comprehension comes first (within weeks) but you will be speaking with grammatical errors, mispronunciations, etc. for much longer. You won‘t have to learn the language from scratch, second time around is much easier, but you will have to put in the effort. And if you use google translate instead of your brain, you won‘t relearn the language at all. You will simply forget it.
Anecdotally, i burned out pretty hard and basically didn't open a text editor for half a year (unemployed too). Eventually i got an itch to write code again and it didn't really feel like I was really worse. Maybe it wasn't long enough atrophy but code doesn't seem to quite work like language though ime.
Six months is definitely not long enough of a break for skills to degrade. But it's not just skills, as I wrote in another comment, the biggest thing is knowledge of new tools, new versions of language and its features.
I'd say there's at most around 2 years of knowledge runtime (maybe with all this AI stuff this is even shorter). After that period if you don't keep your knowledge up to date it fairly quickly becomes obsolete.
I would imagine there is probably some reverse S-curve of skill loss going on. The first year you may retain like 90% (and the 10% are obscure words, rare grammar structures, expressions, etc.), then in the next 2 years you loose more and more every year, and by the 3rd year you’ve lost like 50% of the language, including some common words, useful grammar structures, but retain common greetings, basic structures, etc. and then after like year 5 the regression starts to slow down and by year 10 you may still know 20%, but it is the most basic stuff, and you won‘t be able to use the language in any meaningful way.
I have not and I'm actually really bad at learning human languages, but know a dozen programming languages. You would think they would be similar, but for some reason it's really easy for me to program in any language and really hard for me to pick up a human language.
Learning human languages is not a similar process to learning programming languages at all. I've never been sure why so many people think it is.
I provided it as a counter example to the learning how to bike myth.
Learning how to bike requires only a handful of skills, most of them are located in the motor control centers in your brain (mostly in the Cerebellum), which is known to retain skills much better then any other parts of your brain. Your programing skills are comprised of thousands of separate skills which are mostly located in your frontal-cortex (mostly in your frontal and temporal lobes), and learning a foreign language is basically that but more (like 10x more).
So while a foreign language is not the perfect analogy (nothing is), I think it is a reasonable analogy as a counter example to the bicycle myth.
Maybe something that keeps programming skills fresh is that after you learn to think like a programmer, you do that with problems away from the keyboard. Decomposition, logic... in the years I wasn't programming, I was still solving problems like a programmer. Getting back behind the keyboard just engaged the thought processes I was already keeping warm with practice.
I studied Spanish for years in school, then never really used it. Ten years later, I started studying Japanese. Whenever I got stuck, Spanish would come out. Spanish that I didn't even consciously remember. AFAIK, foreign languages are all stored in the same part of the brain, and once you warm up those neurons, they all get activated.
Not that it's in any way relevant to programming. I will say that after dropping programming for years, I can still explain a lot of specifics, and when I dive back in, it all floods right back. Personally, I'm convinced that any competent, experienced programmer could take a multi-year break, then come back and be right up to speed with the latest software stack in only slightly longer than the stack transition would have taken without a break.
You might still have the skillset to write code, but depending on length of the break your knowledge of tools, frameworks, patterns would be fairly outdated.
I used to know a person like that - high in the company structure who would claim he was a great engineer, but all the actual engineers would make jokes about him and his ancient skills during private conversations.
I’d push back on this framing a bit. There's a subtle ageism baked into the assumption that someone who stepped away from day-to-day coding has "ancient skills" worth mocking.
Yes, specific frameworks and tooling knowledge atrophy without use, and that’s true for anyone at any career stage. A developer who spent three years exclusively in React would be rusty on backend patterns too. But you’re conflating current tool familiarity with engineering ability, and those are different things.
The fundamentals: system design, debugging methodology, reading and reasoning about unfamiliar code, understanding tradeoffs ... those transfer. Someone with deep experience often ramps up on new stacks faster than you’d expect, precisely because they’ve seen the same patterns repackaged multiple times.
If the person you’re describing was genuinely overconfident about skills they hadn’t maintained, that’s a fair critique. But "the actual engineers making jokes about his ancient skills" sounds less like a measured assessment and more like the kind of dismissiveness that writes off experienced people before seeing what they can actually do.
Worth asking: were people laughing because he was genuinely incompetent, or because he didn’t know the hot framework of the moment? Because those are very different things.
This has nothing to do about ageism. This applies to any person of any age who has ego big enough to think that their knowledge of industry is relevant after they take prolonged break and be socially inept enough to brag about how they are still "in".
I don't disagree with your point about fundamentals, but in an industry where there seems to be new JS framework any time somebody sneezes - latest tools are very much relevant too. And of course the big thing is language changes. The events I'm describing happened in the late 00s-early 10s. When language updates picked up steam: Python, JS, PHP, C++. Somebody who used C++ 98 can't claim to have up to date knowledge in C++ in 2015.
So to answer your question - people were laughing at his ego, not the fact that he didn't know some hot new framework.
I beg to differ. I started with C in the 90s, then C# in '05, then PHP in '12, then Go in '21. The things I learned in C still apply to Go, C#, and PHP. And I even started contributing to open source C projects in '24 ... all my skills and knowledge were still relevant. This sounds exactly like ageism to me, but I clearly have a different perspective than you.
Yes, we clearly have different perspectives. I observed an arrogant person who despite their multi-year break from engineering of any kind strongly believed that they still were as capable as engineers who remained in the field during that time.
Maybe you had to be there.
I code in Vim, use Linux... all of those tools are pretty constant. New frameworks are easy to pick up. I've been able to become productive with very little downtime after multi-year breaks several times.
I feel like I'm still a couple steps behind in skill level as my lead and is trying to gain more experience I do wonder if I am shooting myself in the foot if I rely too much on AI at this stage. The senior engineer I'm trying to learn from can very effectively use ai because he has very good judgement of code quality, I feel like if I use AI too much I might lose out on chance to improve my judgement. It's a hard dilemma.
You could probably combat this somewhat with a skill that references to examples of the code you don't want and the code you do. And then each time you tell it to correct the code you ask it to put that example into the references.
You then tell your agent to always run that skill prior to moving on. If the examples are pattern matchable you can even have the agent write custom lints if your linter supports extension or even write a poor man’s linter using ast-grep.
I usually have a second session running that is mainly there to audit the code and help me add and adjust skills while I keep the main session on the task of working on the feature. I've found this far easier to stay engaged than context switching between unrelated tasks.
Honestly, this seems very much like the jump from being an individual contributor to being an engineering manager.
The time it happened for me was rather abrupt, with no training in between, and the feeling was eerily similar.
You know _exactly_ why the best solution is, you talk to your reports, but they have minds of their own, as well as egos, and they do things … their own way.
At some point I stopped obsessing with details and was just giving guidance and direction only in the cases where it really mattered, or when asked, but let people make their own mistakes.
Now LLMs don’t really learn on their own or anything, but the feeling of “letting go of small trivial things” is sorta similar. You concentrate on the bigger picture, and if it chose to do an iterative for loop instead of using a functional approach the way you like it … well the tests still pass, don’t they.
The only issue is that as an engineering manager you reasonably expect that the team learns new things, improve their skills, in general grow as engineers. With AI and its context handling you're working with a team where each member has severe brain damage that affects their ability to form long term memories. You can rewire their brain to a degree teaching them new "skills" or giving them new tools, but they still don't actually learn from their mistakes or their experiences.
As a manager I would encourage them to use the LLM tools. I would also encourage unit tests, e2e testing, testing coverages, CI pipelines automating the testing, automatic pr reviewing etc...
It's also peeking at the big/impactful changes and ignoring the small ones.
Your job isn't to make sure they don't have "brain damage" its to keep them productive and not shipping mistakes.
Being optimistic (or pessimistic heh), if things keep the trend then the models will evolve as well and will probably be quite better in one year than they are now.
The solution for brain atrophy I personally arrived is to use coding agents at work, where, let’s be honest, velocity is a top priority and code purity doesn’t matter that much. Since we use stack I super familial with, I can quite fast verify produced code and tweak it if needed.
However, for hobby projects where I purposely use tech I’m not very familiar with, I force myself not to use LLMs at all - even as a chat. Thus, operating The old way - writing code manually, reading documentation, etc brings me a joy of learning back and, hopefully, establishes new neurone connections.
"I wanted the code to look a certain way, but it kept pulling back to the way it wanted to do things."
I would argue this is ok for front-end. For back-end? very, very bad- if you can't get a usable output do it by hand.
"rip it out" is a phrase I've been saying more often to the robots.
> Like if I had stated certain design goals recently it would adhere to them, but after a few iterations it would forget again and go back to its original approach, or mix the two, or whatever.
Context management, proper prompting and clear instructions, proper documentation are still relevant.
I think this is where tools like OpenSpec [1] may help. The deterioration in quality is because the context is degrading, often due to incomplete or amibiguous requests from the coder. With a more disciplined way of creating and persisting locally the specs for the work, especially if the agent got involved in creating that too, you'll have a much better chance of keeping the agent focussed and aligned.
[1] - https://openspec.dev/
> AI keeps pushing it in a direction I didn't want
The AI definitely has preferences and attention issues, but there are ways to overcome them.
Defining code styles in a design doc, and setting up initial examples in key files goes a long way. Claude seems pretty happy to follow existing patterns under these conditions unless context is strained.
I have pretty good results using a structured workflow that runs a core loop of steps on each change, with a hook that injects instructions to keep attention focused.
My advice: keep it on a tight leash.
In the happy case where I have a good idea of the changes necessary, I will ask it to do small things, step by step, and examine what it does and commit.
In the unhappy case where one is faced with a massive codebase and no idea where to start, I find asking it to just “do the thing” generates slop, but enough for me to use as inspiration for the above.
He didn't say "brain atrophy", he was talking about coding abilities.
it's not about brain atrophy, it's skill atrophy
is that not the sam thing?
LLMs are yet another layer between us and the end result. I remain wary of this distance and am super grateful I learned coding the hard way.
yeah, because the thing is: at the end of the day: laying things out the way LLMs can understand is becoming more important than doing them the “right” way— a more insidious form of the same complacency. and one in which i am absolutely complicit.
[dead]
[dead]
LLMs have some terrible patterns, don't know what do ? Just chuck a class named Service in.
Have to really look out for the crap.
> LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.
I’ve always said I’m a builder even though I’ve also enjoyed programming (but for an outcome, never for the sake of the code)
This perfectly sums up what I’ve been observing between people like me (builders) who are ecstatic about this new world and programmers who talk about the craft of programming, sometimes butting heads.
One viewpoint isn’t necessarily more valid, just a difference of wiring.
I noticed the same thing, but wasn't able to put it into words before reading that. Been experimenting with LLM-based coding just so I can understand it and talk intelligently about it (instead of just being that grouchy curmudgeon), and the thought in the back of my mind while using Claude Code is always:
"I got into programming because I like programming, not whatever this is..."
Yes, I'm building stupid things faster, but I didn't get into programming because I wanted to build tons of things. I got into it for the thrill of defining a problem in terms of data structures and instructions a computer could understand, entering those instructions into the computer, and then watching victoriously while those instructions were executed.
If I was intellectually excited about telling something to do this for me, I'd have gotten into management.
Same. This kind of coding feels like it got rid of the building aspect of programming that always felt nice, and it replaced it entirely with business logic concerns, product requirements, code reviews, etc. All the stuff I can generally take or leave. It's like I'm always in a meeting.
>If I was intellectually excited about telling something to do this for me, I'd have gotten into management.
Exactly this. This is the simplest and tersest way of explaining it yet.
Because you are not coding, you are building. I've been coding since I was 7 years old, now I'm building.
I'd go one step higher, we're not builders, we're problem solvers.
Sometimes the problem needs building, sometimes not.
I'm an Engineer, I see a problem and want to solve it. I don't care if I have to write code, have a llm build something new, or maybe even destroy something. I want to solve the problem for the business and move to the next one, most of the time it is having a llm write code though.
Maybe I don't entirely get it, but what is stopping you to just continue coding?
Speaking for myself, speed. I’d be noticeably slower than my peers if I was crafting code by hand all day.
That's what I'm doing on my codebases, while I still can. I only use Claude if I need to work on a different team's code that uses it heavily. Nothing quite gets a groan from me like opening up a repo and seeing CLAUDE.md
Same same. Writing the actual code is always a huge motivator behind my side projects. Yes, producing the outcome is important, but the journey taken to get there is a lot of fun for me.
I used Claude Code to implement a OpenAI 4o-vision powered receipt scanning feature in an expense tracking tool I wrote by hand four years ago. It did it in two or three shots while taking my codebase into account.
It was very neat, and it works great [^0], but I can't latch onto the idea of writing code this way. Powering through bugs while implementing a new library or learning how to optimize my test suite in a new language is thrilling.
Unfortunately (for me), it's not hard at all to see how the "builders" that see code as a means to an end would LOVE this, and businesses want builders, not crafters.
In effect, knowing the fundamentals is getting devalued at a rate I've never seen before.
[^0] Before I used Claude to implement this feature, my workflow for processing receipts looked like this: Tap iOS Shortcut, enter the amount, snap a pic of the receipt, type up the merchant, amount and description for the expense, then have the shortcut POST that to my expenses tracking toolkit which, then, POSTs that into a Google Sheet. This feature amounted the need for me to enter the merchant and amount. Unfortunately, it often took more time to confirm that the merchant, amount and date details OpenAI provided were correct (and correct it when details were wrong, which was most of the the time) than it did to type out those details manually, so I just went back to my manual workflow. However, the temptation to just glance at the details and tap "This looks correct" was extremely high, even if the info it generated was completely wrong! It's the perfect analogue to what I've been witnessing throughout the rise of the LLMs.
What I have enjoyed about programming is being able to get the computer to do exactly what I want. The possibilities are bounded by only what I can conceive in my mind. I feel like with AI that can happen faster.
> get the computer to do exactly what I want.
> with AI that can happen faster.
well, not exactly that.
For simple things it can. But then for more complex things that's where I step it
Have you an example of getting a coding chatbot to do exactly what you want?
https://gisthost.github.io/?a41ce6304367e2ced59cd237c576b817... - which built https://github.com/datasette/datasette-transactions exactly the way I wanted it to be built
The examples that you and others provide are always fundamentally uninteresting to me. Many, if not most, are some variant of a CRUD application. I have yet seen a single ai generated thing that I personally wanted to use and/or spend time with. I also can't help but wonder what we might have accomplished if we devoted the same amount of resources to developing better tools, languages and frameworks to developers instead of automating the generation of boiler plate and selling developer's own skills back to them. Imagine if open source maintainers instead had been flooded with billions of dollars in capital. What might be possible?
And also, the capacities of llms are almost besides the point. I don't use llms but I have no doubt that for any arbitrary problem that can be expressed textually and is computable in finite time, in the limit as time goes to infinity, an llm will be able to solve it. The more important and interesting questions are what _should_ we build with llms and what should we _not_ build with them. These arguments about capacity are distracting from these more important questions.
Considering how much time developers spend building uninteresting CRUD applications I would argue that if all LLMs can do is speed that process up they're already worth their weight in bytes.
The impression I get from this comment is that no example would convince you that LLMs are worthwhile.
The problem with replying to the proof-demanders is that they'll always pick it apart and find some reason it doesn't fit their definition. You must be familiar with that at this point.
Worse, they might even attempt to verify your claims e.g. "When AI 'builds a browser,' check the repo before believing the hype" https://www.theregister.com/2026/01/26/cursor_opinion/
> exactly the way I wanted it to be built
You verified each line?
I looked closely enough to confirm there were no architectural mistakes or nasty gotchas. It's code I would have been happy to write myself, only here I got it written on my phone while riding the BART.
What? Why would you want to?
See this is a perfect example of OPs statement! I don't care about the lines, I care about the output! It was never about the lines of code.
Your comment makes it very clear there are different viewpoints here. We care about problem->solution. You care about the actual code more than the solution.
> I don't care about the lines, I care about the output! It was never about the lines of code.
> Your comment makes it very clear there are different viewpoints here.
Agreed.
I care that code output not include leaked secrets, malware installation, stealth cryptomining etc.
Some others don't.
>not include leaked secrets, malware installation, stealth cryptomining etc.
Not sure what your point is exactly, but those things don't bother me because I have no control over what happens on others computers. Maybe you insinuate that LLMs will create this, If so, I think you misunderstand the tooling. Or mistake the tooling with the operator.
Is this a joke? Are you genuinely implying that no one has ever got an LLM to write code that does exactly what they want?
No. Mashing up other peoples' code scraped from the web is not what I'd call writing code.
Can you not see how you truly, deep down, are afraid you might be wrong?
It's clouding your vision.
This gets at the heart of the quality of results issues a lot of people are talking about elsewhere here. Right now, if you treat them as a system where you can tell it what you want and it will do it for you, you're building a sandcastle. Instead of that, also describe the correct data structures and appropriate algorithms to use against them, as well as the particulars of how you want the problem solved, it's a different situation altogether. Like most systems, the quality of output is in some way determined by the quality of input.
There is a strange insistence on not helping the LLM arrive at the best outcome in the subtext to this question a lot of times. I feel like we are living through the John Henry legend in real time
> I got into it for the thrill of defining a problem in terms of data structures and instructions a computer could understand, entering those instructions into the computer, and then watching victoriously while those instructions were executed.
You can still do that with Claude Code. In fact, Claude Code works best the more granular your instructions get.
> Claude Code works best the more granular your instructions get.
So best feed it machine code?
Funny you say that. Because I have never enjoyed management as much as being hands on and directly solving problems.
So maybe our common ground is that we are direct problem solvers. :-)
For some reason this makes me think of a jigsaw puzzle. People usually complete these puzzles because they enjoy the process where on the end you get a picture that you can frame if you want to. Some people seem to want to get the resulting picture. No interest in process at all.
I guess that's the same people who went to all those coding camps during their hay day because they heard about software engineering salaries. They just want the money.
When I last bought a Lego Technic set because I wanted to play with making mechanisms with gears and stuff, I assembled it according to the instructions, which was fun, and then the final result was also cool and I couldn't bear to dismantle it.
IMO, this isn't entirely a "new world" either, it is just a new domain where the conversation amplifies the opinions even more (weird how that is happening in a lot of places)
What I mean by that: you had compiled vs interpreted languages, you had types vs untyped, testing strategies, all that, at least in some part, was a conversation about the tradeoffs between moving fast/shipping and maintainability.
But it isn't just tech, it is also in methodologies and the words use, from "build fast and break things" and "yagni" to "design patterns" and "abstractions"
As you say, it is a different viewpoint... but my biggest concern with where are as industry is that these are not just "equally valid" viewpoints of how to build software... it is quite literally different stages of software, that, AFAICT, pretty much all successful software has to go through.
Much of my career has been spent in teams at companies with products that are undergoing the transition from "hip app built by scrappy team" to "profitable, reliable software" and it is painful. Going from something where you have 5 people who know all the ins and outs and can fix serious bugs or ship features in a few days to something that has easy clean boundaries to scale to 100 engineers of a wide range of familiarities with the tech, the problem domain, skill levels, and opinions is just really hard. I am not convinced yet that AI will solve the problem, and I am also unsure it doesn't risk making it worse (at least in the short term)
“””
Much of my career has been spent in teams at companies with products that are undergoing the transition from "hip app built by scrappy team" to "profitable, reliable software" and it is painful. Going from something where you have 5 people who know all the ins and outs and can fix serious bugs or ship features in a few days to something that has easy clean boundaries to scale to 100 engineers of a wide range of familiarities with the tech, the problem domain, skill levels, and opinions is just really hard. I am not convinced yet that AI will solve the problem, and I am also unsure it doesn't risk making it worse (at least in the short term)
“””
This perspective is crucial. Scale is the great equalizer / demoralizer, scale of the org and scale of the systems. Systems become complex quickly, and verifiability of correctness and function becomes harder. Companies that built from day with AI and have AI influencing them as they scale, where does complexity begin to run up against the limitations of AI and cause regression? Or if all goes well, amplification?
But how can you be a responsible builder if you don't have trust in the LLMs doing the "right thing"? Suppose you're the head of a software team where you've picked up the best candidates for a given project, in that scenario I can see how one is able to trust the team members to orchestrate the implementation of your ideas and intentions, with you not being intimately familiar with the details. Can we place the same trust in LLM agents? I'm not sure. Even if one could somehow prove that LLM are very reliable, the fact an AI agents aren't accountable beings renders the whole situation vastly different than the human equivalent.
Trust but verify:
I test all of the code I produce via LLMs, usually doing fairly tight cycles. I also review the unit test coverage manually, so that I have a decent sense that it really is testing things - the goal is less perfect unit tests and more just quickly catching regressions. If I have a lot of complex workflows that need testing, I'll have it write unit tests and spell out the specific edge cases I'm worried about, or setup cheat codes I can invoke to test those workflows out in the UI/CLI.
Trust comes from using them often - you get a feeling for what a model is good and bad at, and what LLMs in general are good and bad at. Most of them are a bit of a mess when it comes to UI design, for instance, but they can throw together a perfectly serviceable "About This" HTML page. Any long-form text they write (such as that About page) is probably trash, but that's super-easy to edit manually. You can often just edit down what they write: they're actually decent writers, just very verbose and unfocused.
I find it similar to management: you have to learn how each employee works. Unless you're in the Top 1%, you can't rely on every employee giving 110% and always producing perfect PRs. Bugs happen, and even NASA-strictness doesn't bring that down to zero.
And just like management, some models are going to be the wrong employee for you because they think your style guide is stupid and keep writing code how they think it should be written.
You don't simply put a body in a seat and get software. There are entire systems enabling this trust: college, resume, samples, referral, interviews, tests and CI, monitoring, mentoring, and performance feedback.
And accountability can still exist? Is the engineer that created or reviewed a Pull Request using Claude Code less accountable then one that used PICO?
> And accountability can still exist? Is the engineer that created or reviewed a Pull Request using Claude Code less accountable then one that used PICO?
The point is that in the human scenario, you can hold the human agents accountable. You cannot do that with AI. Of course, you as the orchestrator of agents will be accountable to someone, but you won't have the benefit of holding your "subordinates" accountable, which is what you do in a human team. IMO, this renders the whole situation vastly different (whether good or bad I'm not sure).
You can switch to another LLM provider or stop using them altogether. It's even easier than firing a developer.
It is as easy as getting rid of Microsoft Teams at your org.
Of course he is - because he invested so much less.
I remember leaving university going into my first engineering job, thinking "Where is all the engineering? All the problem solving and building complex system? All the math and science? Have I been demoted to a lowly programmer?"
Took me a few years to realize that this wasn't a universal feeling, and that many others found the programming tasks more fulfilling than any challenging engineering. I suppose this is merely another manifestation of the same phenomena.
I think he's really getting at something there. I've been thinking about this a lot (in the context of trying to understand the persistent-on-HN skepticism about LLMs), and the framing I came up with[1] is top-down vs. bottom-up dev styles, aka architecting code and then filling in implementations, vs. writing code and having architecture evolve.
[1] https://www.klio.org/theory-of-llm-dev-skepticism/
I like this framing. Nice typography btw, a pleasure to read.
Maybe there's an intermediate category: people who like designing software? I personally find system design more engaging than coding (even though I enjoy coding as well). That's different from just producing an opaque artifact that seems to solve my problem.
> > LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.
> I’ve always said I’m a builder even though I’ve also enjoyed programming (but for an outcome, never for the sake of the code)
> This perfectly sums up what I’ve been observing between people like me (builders) who are ecstatic about this new world and programmers who talk about the craft of programming, sometimes butting heads.
That's one take, sure, but it's a specially crafted one to make you feel good about your position in this argument.
The counter-argument is that LLM coding splits up engineers based on those who primarily like engineering and those who like managing.
You're obviously one of the latter. I, OTOH, prefer engineering.
I prefer engineering too, I tried management and I hated it.
It's just the level of engineering we're split on. I like the type of engineering where I figure out the flow of data, maybe the data structures and how they move through the system.
Writing the code to do that is the most boring part of my job. The LLM does it now. I _know_ how to do it, I just don't want to.
It all boils down to communication in a way. Can you communicate what you want in a way others (in this case a language model) understands? And the parts you can't communicate in a human language, can you use tools to define those (linters, formatters, editorconfig)?
I've done all that with actual humans for ... a decade? So applying the exact same thing to a machine is weirdly more efficient, it doesn't complain about the way I like to have my curly braces - it just copies the defined style. With humans I've found out that using impersonal tooling to inspect code style and flaws has a lot less friction than complaining about it in PR reviews. If the CI computer says no, people don't complain, they fix it.
I feel like this is the core issue that will actually stall LLM coding tools short of actually replacing coding work at large.
'Coders' make 'builders' keep the source code good enough so that 'builders' can continue building without breaking what they built.
If 'builders' become x10 productive and 'coders' become unable to keep up with unsurmountable pile of unmaintainable mess that 'builders' proudly churn out, 'bullders' will start to run into impossibility to build further without starting over and over again hoping that agents will be able to get it right this time.
"Coders" can code tools that programmatically define quality. We have like 80% of those already.
Then force the builders to use those tools to constrain their output.
So far I haven't seen it actually be effective at "building" in a work context with any complexity, and this despite some on our team desperately trying to make that the case.
I have! You have to be realistic about the projects. The more irreducible local context it needs, the less useful it will be. Great for greenfield code, oneshots, write once read once run for months.
Agreed. I don’t care for engineering or coding, and would gladly give it up the moment I can. I’m also running a one man business where every hour counts (and where I’m responsible for maintaining every feature).
The fact of the matter is LLMs produce lower quality at higher volumes in more time than it would take to write it myself, and I’m a very mediocre engineer.
I find this seperation of “coding” vs “building” so offensive. It’s basically just saying some people are only concerned with “inputs”, while others with “outputs”. This kind of rhetoric is so toxic.
It’s like saying LLM art is separating people into people who like to scribble, and people who like to make art.
Would you accept 'people who like to make art, and people who like to commission somebody to make art and give them lots of notes in the process'?
I mean it’s closer, but I don’t think it’s right to equate commissioning an artist with paying a multi-billion dollar corporation to steal from artists.
These tools are just lazy shortcuts. And that’s fine, there’s no problem with taking the lazy way. I’m never going to put in the time to learn to draw, so it’s cool there’s an option for me.
I just take ire with pretending it’s something grand and refined, or spitting in the face of the ones who are willing to put in the work
> > LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.
This is much less significant than the fact LLMs split engineers on those who primarily like quality v. those who primarily like speed.
That split has always existed. LLMs can be used on either side of the divide.
We see a ton of "AI let me code a program X faster than ever before."
We see almost no "AI let me code a program X better than ever before."
See this episode of Oxide and Friends, where they discuss just that: https://oxide-and-friends.transistor.fm/episodes/engineering...
I can't argue that. The scale was already imbalanced as well, and vibe coding has lowered the bar even more, so the gap will continue to grow for now.
I'm just saying that LLMs aren't causing the divide. Accelerating yes, but I think simply equating AI usage to poor quality is wrong. Craftsmen now have a powerful tool as well, to analyze, nitpick, and refactor in ways that were previously difficult to justify.
It also seems premature for so many devs to jump to hardline "AI bad" stances. So far the tech is improving quite well. We may not be able to 1-shot much of quality yet, but it remains to be seen if that will hold.
Personally, I have hopes that AI will eventually push code quality much higher than it's ever been. I might be totally wrong of course, but to me it feels logical that computers would be very good at writing computer programs once the foundation is built.
I think the division is more likely tied to writing. You have to fundamentally change how you do your job, from one of writing a formal language for a compiler to one of writing natural language for a junior-goldfish-memory-allstar-developer, closer to management then to contributor.
This distinction to me separates the two primary camps
Yeah, I think this is a bit of insight I had not realized / been able to word correctly yet. There's developers who can let Claud go at it, and be fearless about it like me (though I mostly do it for side projects, but WOW) and then there's developers who will use it like a hammer or axe to help cut down or mold whatever is in their path.
I think both approaches are okay, the biggest thing for me is the former needs to test way more, and review the code more, as developers we don't read code enough, with the "prompt and forget" approach we have a lot of free time we could spend reading the code, asking the model to refactor and refine the code. I am shocked when I hear about hundreds of thousands of lines in some projects. I've rebuilt Beads from the ground up and I'm under 10 lines of code.
So we're going to have various level of AI Code Builders if you will: Junior, Mid, Senior, Architect. I don't know if models will ever pick up the slack for Juniors any time soon. We would need massive context windows for models, and who will pay for that? We need a major AI breakthrough to where the cost goes down drastically before that becomes profitable.
I think there's a place for both.
We have services deployed globally serving millions of customers where rigor is really important.
And we have internal users who're building browser extensions with AI that provide valuable information about the interface they're looking at including links to the internal record management, and key metadata that's affecting content placement.
These tools could be handed out on Zip drives in the street and it would just show our users some of the metadata already being served up to them, but it's amazing to strip out 75% of the process of certain things and just have our user (in this case though, it's one user who is driving all of this, so it does take some technical inclination) build out these tools that save our editors so much time when doing this before would have been months and months and months of discovery and coordination and designs that probably wouldn't actually be as useful in the end after the wants of the user are diluted through 18 layers of process.
I like building, but I don't fool myself into thinking it can be done by taking shortcuts. You could build something that looks like a house for half the cost but it won't be structurally sound. That's why I care about the details. Someone has to.
There's more to it than just coding Vs building though.
For a long time in my career now I've been in a situation where I'd be able to build more if I was willing to abstract myself and become a slide-merchant/coalition-builder. I don't want to do this though.
Yet, I'm still quite an enthusiastic vibe-coder.
I think it's less about coding Vs building and more about tolerance for abstraction and politics. And I don't think there are that many people who are so intolerant of abstraction that they won't let agents write a bunch of code for them.
I’ve heard something similar: “there are people who enjoy the process, and people who enjoy the outcome”. I think this saying comes moreso from artistic circles.
I’ve always considered myself a “process” person, I would even get hung-up on certain projects because I enjoyed crafting them so much.
LLM’s have taken a bit of that “process” enjoyment from me, but I think have also forced some more “outcome” thinking into my head, which I’m taking as a positive.
The new LLM centered workflow is really just a management job now.
Managers and project managers are valuable roles and have important skill sets. But there's really very little connection with the role of software development that used to exist.
It's a bit odd to me to include both of these roles under a single label of "builders", as they have so little in common.
EDIT: this goes into more detail about how coding (and soon other kinds of knowledge work) is just a management task now: https://www.oneusefulthing.org/p/management-as-ai-superpower...
i don't disagree. at some point LLM's might become good enough that we wouldn't need exact technical expertise.
I enjoy both and have ended up using AI a lot differently than vibe coders. I rarely use it for generating implementations, but I use it extensively for helping me understand docs/apis and more importantly, for debugging. AI saves me so much time trying to figure out why things aren’t working and in code review.
I deliberately avoid full vibe coding since I think doing so will rust my skills as a programmer. It also really doesn’t save much time in my experience. Once I have a design in mind, implementation is not the hard part.
To me this is similar to car enthusiasms. Some people absolutely love to build their project car, it's a major part of the hobby for them. Others just love the experience of driving, so they buy ready cars or just pay someone to work on the car.
Alternatively, others just want to get to their destination.
agree completely. I used to be (and still would love to be) a process person, enjoying hand writing bulletproof artisanal code. Switching to startups many years ago gave me a whole new perspective, and its been interesting the struggle between writing code and shipping. Especially when you dont know how long the code you are writing will actually live. LLMs are fantastic in that space.
makes sense if you are a data scientist where people need to be boxed into tidy little categories. but some people probably fall into both categories.
> I enjoy both and have ended up using AI a lot differently than vibe coders. I rarely use it for generating implementations, but I use it extensively for helping me understand docs/apis and more importantly, for debugging. AI saves me so much time trying to figure out why things aren’t working and in code review.
I had felt like this and still do but man, at some point, I feel like the management churn feels real & I just feel suffering from a new problem.
Suppose, I actually end up having services literally deployed from a single prompt nothing else. Earlier I used to have AI write code but I was interested in the deployment and everything around it, now there are services which do that really neatly for you (I also really didn't give into the agent hype and mostly used browsers LLM)
Like on one hand you feel more free to build projects but the whole joy of project completely got reduced.
I mean, I guess I am one of the junior dev's so to me AI writing code on topics I didn't know/prototyping felt awesome.
I mean I was still involved in say copy pasting or looking at the code it generates. Seeing the errors and sometimes trying things out myself. If AI is doing all that too, idk
For some reason, recently I have been disinterested in AI. I have used it quite a lot for prototyping but I feel like this complete out of the loop programming just very off to me with recent services.
I also feel like there is this sense of if I buy for some AI thing, to maximally extract "value" out of it.
I guess the issue could be that I can have vague terms or have a very small text file as input (like just do X alternative in Y lang) and I am now unable to understand the architectural decisions and the overwhelmed-ness out of it.
Probably gonna take either spec-driven development where I clearly define the architecture or development where I saw something primagen do recently which is that the AI will only manipulate code of that particular function, (I am imagining it for a file as well) and somehow I feel like its something that I could enjoy more because right now it feels like I don't know what I have built at times.
When I prototype with single file projects using say browser for funsies/any idea. I get some idea of what the code kind of uses with its dependencies and functions names from start/end even if I didn't look at the middle
A bit of ramble I guess but the thing which kind of is making me feel this is that I was talking to somebody and shwocasing them some service where AI + server is there and they asked for something in a prompt and I wrote it. Then I let it do its job but I was also thinking how I would architect it (it was some detect food and then find BMR, and I was thinking first to use any api but then I thought that meh it might be hard, why not use AI vision models, okay what's the best, gemini seems good/cheap)
and I went to the coding thing to see what it did and it actually went even beyond by using the free tier of gemini (which I guess didn't end up working could be some rate limit of my own key but honestly it would've been the thing I would've tried too)
So like, I used to pride myself on the architectural decisions I make even if AI could write code faster but now that is taken away as well.
I really don't want to read AI code so much so honestly at this point, I might as well write code myself and learn hands on but I have a problem with build fast in public like attitude that I have & just not finding it fun.
I feel like I should do a more active job in my projects & I am really just figuring out what's the perfect way to use AI in such contexts & when to use how much.
Thoughts?
I retired from paid sw dev work in 2020 when COVID arrived. I’ve worked on my small projects since with all development by hand. I’d followed the rise of AI, but not used it. Late last year I started a project that included reverse engineering some firmware that runs on an Intel 8096 based embedded processor. I’d never worked on that processor before. There are tools available, but they cost many $. So, I started to think about a simple disassembler. 2 weeks ago we decided to try Claude to see what it could do. We now have a disassembler, assembler and a partially working emulator. No doubt there are bugs and missing features and the code is a bit messy, but boy has it sped up the work. One thing did occur to me. Vendors of small utilities could be in trouble. For example I needed to cut out some pages from a pdf. I could have found a tool online(I’m sure there are several), write one myself. However, Claude quickly performed the task.
> Vendors of small utilities could be in trouble
This is a mix of the “in the future, everyone will have a 3D printer at home and just 3D print random parts they need” and “anyone can trivially build Dropbox with rsync themselves” arguments.
Tech savvy users who know how to use LLMs aren’t how vendors of small utilities stay in business.
They stay in business because they sell things to users who are truly clueless with tech (99% of the population, which can’t even figure out the settings app on their phone), and solid distribution/marketing is how you reach those users and can’t really be trivially hacked because everyone is trying to hack it.
Or they stay in business because they offer some sort of guarantee (whether legal, technical, or other) that the users don’t want to burden themselves with because they have other, more important stuff to worry about.
Im definitely going to build some small tools when I need them. One tool I use occasionally, but not so often I want to subscribe is Insomnia.
I don't know. It's one thing to tell Joe or Jane User to "Get an FTP account, mount it locally with curlftpfs, and then use SVN or CVS on the mounted filesystem." But if Joe or Jane can just cut-and-paste that advice into a prompt and get their own personal Dropbox...
Except when that new Dropbox fails Joe or Jane on that Saturday evening, their only recourse is to ask the AI for help, and the AI starts spinning “oh yeah, mmm, I think I found where the problem is. Cut and paste these debugging lines in that function and let me know what the output is…”
Meanwhile, this year, that happens less often than it did last year... and it actually isn't how AI-assisted development works at all. Agentic models do the cutting-and-pasting by themselves, evaluate the results by themselves, and almost always succeed at fixing the problem by themselves.
Fair
> Vendors of small utilities could be in trouble. For example I needed to cut out some pages from a pdf. I could have found a tool online(I’m sure there are several), write one myself. However, Claude quickly performed the task.
Definitely. Making small, single-purpose utilities with LLMs is almost as easy these days as googling for them on-line - much easier, in fact, if you account for time spent filtering out all the malware, adware, "to finish the process, register an account" and plain broken "tools" that dominate SERP.
Case in point, last time my wife needed to generate a few QR codes for some printouts for an NGO event, I just had LLM make one as a static, single-page client-side tool and hosted it myself -- because that was the fastest way to guarantee it's fast, reliable, free of surveillance economy bullshit, and doesn't employ URL shorteners (surprisingly common pattern that sometimes becomes a nasty problem down the line; see e.g. a high-profile case of some QR codes on food products leading to porn sites after shortlink got recycled).
Whatever happened to just typing "apt install qrencode"? It's definitely "fast, reliable, free of surveillance economy bullshit, and doesn't employ URL shorteners".
You need to know "qrencode" exists under that exact name. Claude already knows about it and how to use it.
Sure, but that's entirely different from vibe-coding a tool, which sounds like a colossal waste of resources.
Having an LLM spit out a few hundred lines of HTML and JavaScript is not a colossal waste of resources, it's equivalent to running a microwave for a couple of seconds.
Not to mention, my little tool is using much less electricity running than just about anything else I could easily find on-line, simply by the virtue of being minimal, and completely free of superfluous visual bullshit, upsells, tracking, telemetry, and other such secondary aspects of anything people publish and advertise for others to use.
as long as that wast and the associated cost is heavily subsidized as it is today, nobody will care
Don't get the anti-AI propaganda get to you too much. Inference is cheap on the margin.
Consider: there are models capable (if barely) of doing this job, that you can run locally, on a upper-mid-range PC with high-end consumer GPU. Take that as a baseline, assume it takes a day instead of an hour because of inference speed, tally up total electricity cost. It's not much. Won't boil oceans any more than people playing AAA video games all day will.
Sure, the big LLMs from SOTA vendors use more GPUs/TPUs for inference, but this means they finish much faster. Plus, commercial vendors have lots of optimizations (batch processing, large caches, etc.), and data centers are much more power-efficient than your local machine, so "how much it'd cost me in power bill if I did it locally" is a good starting estimate.
Users can't use command–line tools. They just can't. It has to be user–friendly or it doesn't exist.
It's not even "users", just the user. Nice thing about LLMs is that it's cheap to develop small tools tailor-made for audience of few, or in this case, just one.
1) This was for my wife. She is not proficient in Linux or CLI in general, and (like ~all white collar workers these days) works almost exclusively in browser tools (exception being pre-O365 versions of Word and Excel we keep running on her laptop because she prefers them).
2) I never heard of `qrencode` CLI tool until today. For some reason I didn't even consider it might exist (maybe because last time I checked, which was many years ago, there was none).
3) Notably, no one mentioned it the last time I shared this story on HN - https://news.ycombinator.com/item?id=44385049.
4) Even if I knew about it, I'd still have to build a web frontend for it, and I'd need a proper server for it, which I'd then have to maintain properly, and secure it against the `qrencode` call becoming an attack vector.
So frankly, for my specific problem, my solution is strictly better.
A "static, single-page client-side tool" is so much better than "Step 1: install Linux..."
> You realize that stamina is a core bottleneck to work
There has been a lot of research that shows that grit is far more correlated to success than intelligence. This is an interesting way to show something similar.
AIs have endless grit (or at least as endless as your budget). They may outperform us simply because they don't ever get tired and give up.
Full quote for context:
Tenacity. It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day. It's a "feel the AGI" moment to watch it struggle with something for a long time just to come out victorious 30 minutes later. You realize that stamina is a core bottleneck to work and that with LLMs in hand it has been dramatically increased.
>They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day.
"Listen, and understand! That Terminator is out there! It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop... ever, until you are dead!"
If you ever work with LLMs you know that they quite frequently give up.
Sometimes it's a
or a"this feature would require extensive logic and changes to the existing codebase".
Sometimes they just declare their work done. Ignoring failing tests and builds.
You can nudge them to keep going but I often feel like, when they behave like this, they are at their limit of what they can achieve.
If I tell it to implement something it will sometimes declare their work done before it's done. But if I give Claude Code a verifiable goal like making the unit tests pass it will work tirelessly until that goal is achieved. I don't always like the solution, but the tenacity everyone is talking about is there
> but the tenacity everyone is talking about is there
I always double-check if it doesn't simply exclude the failing test.
The last time I had this, I discovered it later in the process. When I pointed this out to the LLM, it responded, that it acknowledged thefact of ignoring the test in CLAUDE.md, and this is justified because [...]. In other words, "known issue, fuck off"
Tools in a loop people, tools in a loop.
If you don't give the agent the tools to deterministically test what it did, you're just vibe coding in its worst form.
tenacity == while loop
> If you ever work with LLMs you know that they quite frequently give up.
If you try to single shot something perhaps. But with multiple shots, or an agent swarm where one agent tells another to try again, it'll keep going until it has a working solution.
Yeah exactly this is a scope problem, actual input/output size is always limited> I am 100% sure CC etc are using multiple LLM calls for each response, even though from the response streaming it looks like just one.
Nope, not for me, unless I tell it to.
Context matters, for an LLM just like a person. When I wrote code I'd add TODOs because we cannot context switch to another problem we see every time.
But you can keep the agent fixated on the task AND have it create these TODOs, but ultimately it is your responsibility to find them and fix them (with another agent).
Using LLMs to clean those up is part of the workflow that you're responsible for (... for now). If you're hoping to get ideal results in a single inference, forget it.
I realized a long time ago that I’m better at computer stuff not because I’m smarter but because I will sit there all day and night to figure something out while others will give up. I always thought that was my superpower in the job industry but now I’m not so sure if it will transfer to getting AI to do what I need done…
Same, I barely made it through Engineering school, but would stay up all night figuring out everything a computer could do (before the internet).
I did it because I enjoyed it, and still do. I just do it with LLMs now. There is more to figure out than ever before and things get created faster than I have time to understand them.
LLM should be enabling this, not making it more depressing.
Me three. I was not as smart as many of my peers in uni but I freakin LOVE the subject matter and I also love studying and feeling that progress of learning, which led me to put in the huge number of hours necessary to be successful and have a positive attitude the whole time.
There are dozens of us!
The tenacity aspect makes me worried about the paper clip AI misalignment scenario more than before.
But even tenacity is not enough. You also need an internal timer. "Wait a minute, this is taking too long, it shouldn't be this hard. Is my overall approach completely wrong?"
I'm not sure AIs have that. Humans do, or at least the good ones do. They don't quit on the problem, but they know when it's time to consider quitting on the approach.
> AIs have endless grit (or at least as endless as your budget).
That is the only thing he doesn't address: the money it costs to run the AI. If you let the agents loose, they easily burn north of 100M tokens per hour. Now at $25/1M tokens that gets quickly expensive. At some point, when we are all drug^W AI dependent, the VCs will start to cash in on their investments.
LLMs do not have grit or tenacity. Tenacity doesn't desribe a machine that doesn't need sleep or experience tiredness, or stress. Grit doesn't describe a chatbot that will tirelessly spew out answers and code because it has no stake or interest in the result, never perceives that it doesn't know something, and never reflects on its shortcomings.
[dead]
> What happens to the "10X engineer" - the ratio of productivity between the mean and the max engineer? It's quite possible that this grows a lot.
I was thinking about this the other day as relates to the DevOps movement.
The DevOps movement started as a way to accelerate and improve the results of dev<->ops team dynamics. By changing practices and methods, you get acceleration and improvement. That creates "high-performing teams", which is the team form of a 10x engineer. Whether or not you believe in '10x engineers', a high-performing team is real. You really can make your team deploy faster, with fewer bugs. You have to change how you all work to accomplish it, though.
To get good at using AI for coding, you have to do the same thing: continuous improvement, changing workflows, different designs, development of trust through automation and validation. Just like DevOps, this requires learning brand new concepts, and changing how a whole team works. This didn't get adopted widely with DevOps because nobody wanted to learn new things or change how they work. So it's possible people won't adapt to the "better" way of using AI for coding, even if it would produce a 10x result.
If we want this new way of working to stick, it's going to require education, and a change of engineering culture.
This is an interesting thing that I'm contemplating. I also do believe that (perhaps with very few exceptions) there are no "10x engineers" by themselves, but engineers that thrive 10x more in a context or another (like, I'm sure Jeff Dean is an absolutely awesome engineer - but if you took him out of Google and plugged him into IBM - would he have had the same impact?)
With that in mind - I think one very unexplored area is "how to make the mixed AI-human teams successful". Like, I'm fairly convinced AI changes things, but to get to the industrialization of our craft (which is what management seems to want - and, TBH, something that makes sense from an economic pov), I feel that some big changes need to happen, and nobody is talking about that too much. What are the changes that need to happen? How do we change things, if we are to attempt such industrialization?
I'm pretty happy with Copilot in VS Code. Type what change I want Claude to make in the Copilot panel, and then use the VS Code in context diffs to accept or reject the proposed changes. While being able to make other small changes on my own.
So I think this tracks with Karpathy's defense of IDEs still being necessary ?
Has anyone found it practical to forgo IDEs almost entirely?
I've found copilot chat is able to do everything I need. I tried the Claude plugin for vscode and it was a noticeably worse experience for me.
Mind you copilot has only supported agent mode relatively recently.
I really like the way copilot does changes in such a way you can accept or reject and even revert to point in time in the chat history without using git. Something about this just fits right with how my brain works. Using Claude plugin just felt like I had one hand tied behind my back.
I find Claude Code in VS Code is sometimes horribly inefficient. I tell it to replace some print-statements with proper logging in the one file I have open and it first starts burning tokens to understand the codebase for the 13th time today, despite not needing to and having it laid out in the CLAUDE.md already.
I have been assigning issues to copilot in Github. It will then create a pull request and work on and report back on the issue in the PR. I will pull the code and make small changes locally using VSCode when needed.
But what I like about this setup is that I have almost all the context I need to review the work in a single PR. And I can go back and revisit the PR if I ever run into issues down the line. Plus you can run sessions in parallel if needed, although I don't do that too much.
Are you letting it run your tests and run little snippets of code to try them out (like "python -c 'import module; print(module.something())'") or are you just using it to propose diffs for you to accept or reject?
This stuff gets a whole lot more interesting when you let it start making changes and testing them by itself.
Coplilot is not on par with cc or cursor even
I use it to access Claude. So what's the difference?
This stuff is a little messy and opaque, but the performance of the same model in different harnesses depends a lot on how context is managed. The last time I tried Copilot, it performed markedly worse for similar tasks compared to Claude Code. I suspect that Copilot was being very aggressive in compressing context to save on token cost, but I'm not 100% certain about this.
Also note that with Claude models, Copilot might allocate a different number of thinking tokens compared to Claude Code.
Things may have changed now compared to when I tried it out, these tools are in constant flux. In general I've found that harnesses created by the model providers (OpenAI/Codex CLI, Anthropic/Claude Code, Google/Gemini CLI) tend to be better than generalist harnesses (cheaper too, since you're not paying a middleman).
Different harnesses and agentic environments produce different results from the same model. Claude Code and Cursor are the best IME and Copilot is by far the worst.
Why not? You can select Opus 4.5, Gemini 3 Pro, and others.
Claude Code is a CLI tool which means it can do complete projects in a single command. Also has fantastic tools for scaffolding and harnessing the code. You can define everything from your coding style to specific instructions for designing frontpages, integrating payments, etc.
It's not about the model. It's about the harness
Claude Code is a CLI tool which means it can do complete projects in a single command
https://github.com/features/copilot/cli/
This would make some sense if VS Code didn't have a terminal built into it. The LLMs have the same bash capabilities in either form.
Huh? There is nothing stopping copilot from doing an entire project in one go.
Ive done it 10s of times.
"Copilot has done 10 tool calls, do you want to continue" or whatever was the bane of my existence before our company approved Claude for use.
Like I asked you to do this task, then you spent time looking around and now want me to pat you on the back so you can continue?
The model is the engine, the framework is the rest of the car.
With Copilot Microsoft has basically put the meanest leanest triple-turbo'd V8 engine in a rickety 80's soviet car.
You can kinda drive it fast in a straight line if you're careful, but you can also crash and burn really hard.
it's not a model limit anymore, it's tools , skills, background agents, etc. It's an entire agentic environment.
Github copilot has support for this stuff as well. Agent skills, background/subagents, etc.
Implementation differences do matter. I haven't found Copilot to have as many issues as people say it does, but they are there. Their Gemini implementation is unusable, for example, and it's not because of the underlying models. They work fine in other harnesses.
> Tenacity. It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day.
This is true to an extent for sure and they will go much longer than most engineers without getting "tired", but I've def seen both sonnet and opus give up multiple times. They've updated code to skip tests they couldn't get to pass, given up on bugs they couldn't track down, etc. I literally had it ask "could we work on something else and come back to this"
The glorified autocomplete. Why would the LLM "work on something else then get back on this", is it's subconscious going to solve the problem during that time?
But because people say it, it says it too. Making sense is optional.
Ive found that clearing the context and getting back to it later actually DOES work. When you restart, your personal context is cleared and you might be better at describing the problem you are solving in a more informationally dense way.
not impossible right? the new context can provide some needed hints, etc...
Reminiscent of a time just a year or two ago where the LLMs would get downright frustrated and sassy
Oh, definitely. Also, they end up getting stuck in a loop, adding and removing the same code endlessly.
And then someone comes and "improves" their agent with additional "do not repeat yourself" prompts scattered all over the place, to no avail.
"Asinine" describes my experience perfectly.
LLM coding splits up engineers based on those who primarily like building and those who primarily like code reviews and quality assessment. I definitely don’t love the latter (especially when reviewing decisions not made by a human with whom I can build long-term personal rapport).
After certain experience threshold of making things from scratch, “coding” (never particularly liked that term) has always been 99% building, or architecture, and I struggle to see how often a well-architected solution today, with modern high-level abstractions, requires so much code that you’d save significant time and effort by not having to just type, possibly with basic deterministic autocomplete, exactly what you mean (especially considering you would have to also spend time and effort reviewing whatever was typed for you if you used a non-deterministic autocomplete).
"those who primarily like code reviews and quality assessment" -- I don't love those. In fact I find it tedious and love it when I can work on my own without them.
Except after 25 years of working I know how imperative they are, how easily a project can disintegrate into confused silos, and am frustrated as heck with these tools being pushed without attention to this problem.
See, I don't take it that extreme: LLMs make fantastic, never-before seen quality autocompletes. I hacked together a Neovim plugin that prompts an LLM to "finish this function" on command, and it's a big time save for the menial plumbing type operations. Think things like "this api I use expects JSON that encodes some subset of SQL, I want all the dogs with Ls in their name that were born on a Tuesday". Given an example of such API (or if the documentation ended up in its training), LLMs will consistently one-shot stuff like that.
Asking it to do entire projects? Dumb. You end up with spaghetti, unless you hand-hold it to a point that you might as well be using my autocomplete method.
Depends on the scope of the project. If it's small, and you direct it correctly, it can one-shot yes. Or 2-3-shot.
HN should ban any discussion on “things I learned playing with AI” that don’t include direct artifacts of the thing built.
We’re about a year deep into “AI is changing everything” and I don’t see 10x software quality or output.
Now don’t get me wrong I’m a big fan of AI tooling and think it does meaningfully increase value. But I’m damn tired of all the talk with literally nothing to show for it or back it up.
[dead]
> It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day. It's a "feel the AGI" moment to watch it struggle with something for a long time just to come out victorious 30 minutes later.
This is true... Equally I've seen it dive into a rabbit hole, make some changes that probably aren't the right direction... and then keep digging.
This is way more likely with Sonnet, Opus seems to be better at avoiding it. Sonnet would happily modify every file in the codebase trying to get a type error to go away. If I prompt "wait, are you off track?" it can usually course correct. Again, Opus seems way better at that part too.
Admittedly this has improved a lot lately overall.
I don't understand why anyone finds it interesting that a machine, or chatbot, never tires or gets demoralized. You have to anthromorphize the LLM before you can even think of those possibilities. A tractor never tires or gets demoralized either, because it can't. Chatbots don't "dive into a rabbit hole ... and then keep digging" because they have superhuman tenacity, they do it because that's what software does. If I ask my laptop to compute the millionth Fibonacci number it doesn't sigh and complain, and I don't think it shows any special qualities unless I compare it to a person given the same job.
You're a machine. You're literally a wet, analog device converting some forms of energy into other forms just like any other machine as you work, rest, type out HN comments, etc. There is nothing special about the carbon atoms in your body -- there's no metadata attached to them marking them out as belonging to a Living Person. Other living-person-machines treat "you" differently than other clusters of atoms only because evolution has taught us that doing so is a mutually beneficial social convention.
So, since you're just a machine, any text you generate should be uninteresting to me -- correct?
Alternatively, could it be that a sufficiently complex and intricate machine can be interesting to observe in its own right?
If humans are machines, they are still a subset of machines and they (among other animals) are the only ones who can be demotivated and so it is still a mistake to assume an entirely different kind of machine would have those properties.
>Other living-person-machines treat "you" differently than other clusters of atoms only because evolution has taught us that doing so is a mutually beneficial social convention
Evolution doesn't "teach" anything. It's just an emergent property of the fact that life reproduces (and sometimes doesn't). If you're going to have this radically reductionist view of humanity, you can't also treat evolution as having any kind of agency.
"If humans are machines, they are still a subset of machines and they (among other animals) are the only ones who can be demotivated and so it is still a mistake to assume an entirely different kind of machine would have those properties."
Yet.
Sure but the entire context of the discussion is surprisial that they don't.
Agreed - There is no guarantee of what will happen in the future. I'm not for or against the outcome, but certainly curious to see what it is.
Humans and all other organisms are "literally" not machines or devices by the simple fact that those terms refer to works made for a purpose.
Even as an analogy "wet machine" fails again and again to adequately describe anything interesting or useful in life sciences.
Wrong level of abstraction. And not the definition of machine.
I might feel awe or amazement at what human-made machines can do -- the reason I got into programming. But I don't attribute human qualities to computers or software, a category error. No computer ever looked at me as interesting or tenacious.
I would agree that OAIs GPT-5 family of models is a phase change over GPT-4.
In the ChatGPT product this is not immediately obvious and many people would strongly argue their preference for 4. However, once you introduce several complex tools and make tool calling mandatory, the difference becomes stark.
I've got an agent loop that will fail nearly every time on GPT-4. It works sometimes, but definitely not enough to go to production. GPT-5 with reasoning set to minimal works 100% of the time. $200 worth of tokens and it still hasn't failed to select the proper sequence of tools. It sometimes gets the arguments to the tools incorrect, but it's always holding the right ones now.
I was very skeptical based upon prior experience but flipping between the models makes it clear there has been recent stepwise progress.
I'll probably be $500 deep in tokens before the end of the month. I could barely go $20 before I called bullshit on this stuff last time.
Pretty sure there wasn't extensive training on tooling beforehand. I mean, god, during GPT-3 even getting a reliable json output was a battle and there were dedicated packages for json inference.
Now imagine local models with 95%+ reliable tool calling, you can do insane things when that's the reality.
Agree with Karpathy's take. Finally a down to Earth analysis from a respected source in the AI space. I guess I'll be using slopocalypse a lot more now :)
> I am bracing for 2026 as the year of the slopacolypse across all of github, substack, arxiv, X/instagram, and generally all digital media
It has arrived. Github will be most affected thanks to git-terrorists at Apna College refusing to take down that stupid tutorial. IYKYK.
The respect is unwarranted.
He ran Teslas ML division, but still doesnt know what a simple kalman filter is (in the sense where he claimed that lidar would be hard to integrate with cameras).
The Kalman filter examples I've seen always involve estimating a very simple quantity, like the location of a single 3D point, from noisy sensors. It's clear how multiple estimates can be combined into a new estimate.
I'd guess that cameras on a self-driving car are trying to estimate something much more complex, something like 3D surfaces labeled with categories ("person", "traffic light", etc.). It's not obvious to me how estimates of such things from multiple sensors and predictions can be sensibly and efficiently combined to produce a better estimate. For example, what if there is a near red object in front of a distant red background, so that the camera estimates just a single object, but the lidar sees two?
https://www.bzarg.com/p/how-a-kalman-filter-works-in-picture...
Kalman filters basic concept is essentially this.
1. make prediction on the next state change of some measurable n dimentional quantity, and estimate the covariance matrix across those n dimentions, which describe essentially a probability that the i-th dimention is going to increase (or decrease) with j-th dimention, where i and j are between 0 and n (indices of the vector)
2. Gather sensor data (that can be noisy), and reconcile the predicted measurement with the measured to get the best guess. The covariance matrix acts as a kind of weight for each of the elements
3. Update the covariance matrix based on the measurements in previous step.
You can do this for any vector of numbers. For example, instead of tracking individual objects, you can have a grid where each element represents a physical object that the car should not drive into, with a value representing certainty of that object being there. Then when you combine sensor reading, you still can use your vision model but that model would be enhanced by what lidar detects, both in terms of seeing things that camera doesn't pick up and rejecting things that aren't there.
And the concept is generic enough to where you can set up a system to be able to plug in any additional sensor with its own noise, and it all works out in the end. This is used all the You can even extend the concept past Gaussian noise and linearity, there are a number of other filters that deal with that, broadly under the umbrella of sensor fusion.
The problem is that Karpathy is more of a computer scientist, so he is on his Code 2.0 train of having ML models do everything. I dunno if he is like that himself or Musks "im smarter than everyone else that came before me" rubbed off.
And of course when you think like that, its going to be difficult to integrate lidar into the model. But the problem with that thinking is that forward inference LLM is not AI, and it will never ever be able to drive a car well compared to a true "reasoning" AI with feedback loops.
> It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day. It's a "feel the AGI" moment to watch it struggle with something for a long time just to come out victorious 30 minutes later.
Somewhere, there are GPUs/NPUs running hot. You send all the necessary data, including information that you would never otherwise share. And you most likely do not pay the actual costs. It might become cheaper or it might not, because reasoning is a sticking plaster on the accuracy problem. You and your business become dependent on this major gatekeeper. It may seem like a good trade-off today. However, the personal, professional, political and societal issues will become increasingly difficult to overlook.
This quote stuck out to me as well, for a slightly different reason.
The “tenacity” referenced here has been, in my opinion, the key ingredient in the secret sauce of a successful career in tech, at least in these past 20 years. Every industry job has its intricacies, but for every engineer who earned their pay with novel work on a new protocol, framework, or paradigm, there were 10 or more providing value by putting the myriad pieces together, muddling through the ever-waxing complexity, and crucially never saying die.
We all saw others weeded out along the way for lacking the tenacity. Think the boot camp dropouts or undergrads who changed majors when first grappling with recursion (or emacs). The sole trait of stubbornness to “keep going” outweighs analytical ability, leetcode prowess, soft skills like corporate political tact, and everything else.
I can’t tell what this means for the job market. Tenacity may not be enough on its own. But it’s the most valuable quality in an employee in my mind, and Claude has it.
There is an old saying back home: an idiot never tires, only sweats.
Claude isn't tenacious. It is an idiot that never stops digging because it lacks the meta cognition to ask 'hey, is there a better way to do this?'. Chain of thought's whole raison d'etre was so the model could get out of the local minima it pushed itself in. The issue is that after a year it still falls into slightly deeper local minima.
This is fine when a human is in the loop. It isn't what you want when you have a thousand idiots each doing a depth first search on what the limit of your credit card is.
> it lacks the meta cognition to ask 'hey, is there a better way to do this?'.
Recently had an AI tell me this code (that it wrote) is a mess and suggested wiping it and starting from scratch with a more structure plan. That seems to hint at some meta cognition outlines
Haha, it has the human developer traits of thinking all old code is garbage, failing to identify oneself as the dummy who wrote this particular code, and wanting to start from scratch.
It's like NIH syndrome but instead "not invented here today". Also a very human thing.
More like NIITS: Not Invented in this Session.
Perhaps. I've had LLMs tell me some code is deeply flawed garbage that should be rewritten about code that exact same LLM wrote minutes before. It could be a sign of deep meta cognition, or it might be due to some cognitive gaps where it has no idea why it did something a minute ago and suddenly has a different idea.
This is not a fair criticism. There is _nobody_ there, so you can't be saying 'code the exact same LLM wrote minutes before'. There is no 'exact same LLM' and no ideas for it to have, you're trying to make sense of sparkles off the surface of a pond. There's no 'it' to have an idea and then a different idea, much less deep meta cognition.
I'm not sure we disagree. I was pushing back against the idea that suggesting a rewrite of some code implies meta cognition abilities on the part of the LLM. That seems like weak evidence to me.
They should’ve named him tom instead of Claude in homage to Ten second Tom from fifty first dates
I asked Claude to analyze something and report back. It thought for a while said “Wow this analysis is great!” and then went back to thinking before delivering the report. They’re auto-sycophantic now!
Someone will say "you just need to instruct Claude.md to be more meta and do a wiggum loop on it"
Metacognition As A Service, you say?
Running on the Meta Cognition Protocol server near you.
You’ll get sued by Meta for this!
I think that’s called “consulting”.
lol no it doesn’t. It hints at convincing language models
I mean, not always. I've seen Claude step back and reconsider things after hitting a dead end, and go down a different path. There are also workflows, loops that can increase the likelihood of this occurring.
[dead]
This is a major concern for junior programmers. For many senior ones, after 20 (or even 10) years of tenacious work, they realize that such work will always be there, and they long ago stopped growing on that front (i.e. they had already peaked). For those folks, LLMs are a life saver.
At a company I worked for, lots of senior engineers become managers because they no longer want to obsess over whether their algorithm has an off by one error. I think fewer will go the management route.
(There was always the senior tech lead path, but there are far more roles for management than tech lead).
I feel like if you're really spending a ton of time on off by one errors after twenty years in the field you haven't actually grown much and have probably just spent a ton of time in a single space.
Otherwise you'd be senior staff to principle range and doing architecture, mentorship, coordinating cross team work, interviewing, evaluating technical decisions, etc.
I got to code this week a bit and it's been a tremendous joy! I see many peers at similar and lower levels (and higher) who have more years and less technical experience and still write lots of code and I suspect that is more what you're talking about. In that case, it's not so much that you've peaked, it's that there's not much to learn and you're doing a bunch of the same shit over and over and that's of course tiring.
I think it also means that everything you interact with outside your space does feel much harder because of the infrequency with which you have interacted with it.
If you've spent your whole career working the whole stack from interfaces to infrastructure then there's really not going to be much that hits you as unfamiliar after a point. Most frameworks recycle the same concepts and abstractions, same thing with programming languages, algorithms, data management etc.
But if you've spent most of your career in one space cranking tickets, those unknown corners are going to be as numerous as the day you started and be much more taxing.
That's just sad. Right when I found love in what I do, my work has no value anymore.
Aren't you still better off than the rest of us who found what they love + invested decades in it before it lost its value. Isn't it better to lose your love when you still have time to find a new one?
I don't think so. Those of us who found what we love and invested decades into it got to spend decades getting paid well to do what we love.
Depends on if their new love provides as much money as their old one, which is probably not likely. I'd rather have had those decades to stash and invest.
A lot of pre-faang engineers dont have the stash you're thinking about. What you meant was "right when I found a lucrative job that I love". What was going on in tech these last 15 years, unfortunately, probably was once in a lifetime.
It's crazy to think back in the 80's programmers had "mild" salaries despite programming back then being worlds more punishing. No libraries, no stack exchange, no forums, no endless memory and infinite compute. If you had a challenging bug you better also be proficient in reading schematics and probing circuits.
on the bright side software evolved much more slowly in the 80s. You could go very far by being an expert in 1 thing.
People had real offices with actual quiet focus time.
User expectations were also much lower.
pros and cons i guess?
"it lost its value".
It has not lost its value yet, but the future will shift that value. All of the past experience you have is an asset for you to move with that shift. The problem will not be you losing value, it will be you not following where the value goes.
It might be a bit more difficult to love where the shift goes, but that is no different than loving being a artist which often shares a bed with loving being poor. What will make you happier?
This is genuinely such a good take
Especially on the topic of value! We are all intuitively aware that value is highly contextual, but get in a knot trying to rationalize value long past genuine engagement!
Imagine a senior dev who just approves PRs, approves production releases, and prioritizes bug reports and feature requests. LLM watches for errors ceaslessly, reports an issue. Senior dev reviews the issue and assigns a severity to it. Another LLM has a backlog of features and errors to go solve, it makes a fix and submits a PR after running tests and verifying things work on its end.
Why are we pretending like the need for tenacity will go away? Certain problems are easier now. We can tackle larger problems now that also require tenacity.
Even right at this very moment where we have a high-tenacity AI, I'd argue that working with the AI -- that is to say, doing AI coding itself and dealing with the novel challenges that brings requires a lot of stubborn persistence.
Fittingly, George Hinton toiled away for years in relative obscurity before finally being recognized for his work. I was always quite impressed by his "tenacity".
So although I don't think he should have won the Nobel Prize because not really physics, I felt his perseverance and hard work should merit something.
... The person who embezzled from the SDC in 2018? https://eu.jsonline.com/story/news/investigations/2024/04/19...
Haha, my bad. Yes, that "George" Hinton!
I still find in these instances there's at least a 50% chance it has taken a shortcut somewhere: created a new, bigger bug in something that just happened not to have a unit test covering it, or broke an "implicit" requirement that was so obvious to any reasonable human that nobody thought to document it. These can be subtle because you're not looking for them, because no human would ever think to do such a thing.
Then even if you do catch it, AI: "ah, now I see exactly the problem. just insert a few more coins and I'll fix it for real this time, I promise!"
The value extortion plan writes itself. How long before someone pitches the idea that the models explicitly almost keep solving your problem to get you to keep spending? Would you even know?
First time I've seen this idea, I have a tingling feeling it might become reality sooner rather than later.
That’s far-fetched. It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise. High value to user + lower compute costs = better pricing power and better margins overall.
> far-fetched
Remember Google?
Once it was far-fetched that they would make the search worse just to show you more ads. Now, it is a reality.
With tokens, it is even more direct. The more tokens users spend, the more money for providers.
> Now, it is a reality.
What are the details of this? I'm not playing dumb, and of course I've noticed the decline, but I thought it was a combination of losing the battle with SEO shite and leaning further and further into a 'give the user what you think they want, rather than what they actually asked for' philosophy.
https://www.wheresyoured.at/the-men-who-killed-google/
As recently as 15 years ago, Google _explicitly_ stated in their employee handbook that they would NOT, as a matter of principle, include ads in the search results. (Source: worked there at that time.)
Now, they do their best to deprioritize and hide non-ad results...
Only if you are paying per token on the API. If you are paying a fixed monthly fee then they lose money when you need to burn more tokens and they lose customers when you can’t solve your problems within that month and max out your session limits and end up with idle time which you use to check if the other providers have caught up or surpassed your current favourite.
Indeed, unlimited plan seems like the only way that makes sense to not have it be guaranteed to be abused by the provider
> It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise.
Unless you’re paying by the token.
> It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise. High value to user + lower compute costs = better pricing power and better margins overall.
It's only in the interests of the model builders to do that IFF the user can actually tell that the model is giving them the best value for a single dollar.
Right now you can't tell.
Why not? Seems like you'd just build the same app on each of the models you want to test and judge how they did.
> Why not? Seems like you'd just build the same app on each of the models you want to test and judge how they did.
I tried that on a few problems; even on the same model the results have too much variation.
When comparing different models, repeating the experiment gives you different results.
The free market proposition is that competition (especially with Chinese labs and grok) means that Anthropic is welcome to do that. They're even welcome to illegally collude with OpenAi such that ChatGPT is similarly gimped. But switching costs are pretty low. If it turns out I can one shot an issue with Qwen or Deepseek or Kimi thinking, Anthropic loses not just my monthly subscription, but everyone else's I show that too. So no, I think that's some grade A conspiracy theory nonsense you've got there.
It’s not that crazy. It could even happen by accident in pursuit of another unrelated goal. And if it did, a decent chunk of the tech industry would call it “revealed preference” because usage went up.
LLMs became sycophantic and effusive because those responses were rated higher during RLHF, until it became newsworthy how obviously eager-to-please they got, so yes, being highly factually correct and "intelligent" was already not the only priority.
This is a good point. For example if you have access to a bunch of slot machines, one of them is guaranteed to hit the jackpot. Since switching from one slot machine to another is easy, it is trivial to go from machine to machine until you hit the big bucks. That is why casinos have such large selections of them (for our benefit).
"for our benefit" lol! This is the best description of how we are all interacting with LLMs now. It's not working? Fire up more "agents" ala gas town or whatever
gas is the transaction fees in Ethereum. It's a fitting name.
last time I was at a casino I checked to see what company built the machines, imagine my surprise that it was (by my observation) a single vendor.
> But switching costs are pretty low
Switching costs are currently low. Once you're committed to the workflow the providers will switch to prepaying for a year's worth of tokens.
To be clear I don't think that's what they're doing intentionally. Especially on a subscription basis, they'd rather me maximize my value per token, or just not use them. Lulling users into using tokens unproductively is the worst possible option.
The way agents work right now though just sometimes feels that way; they don't have a good way of saying "You're probably going to have to figure this one out yourself".
As a rational consumer, how would you distinguish between some intentional "keep pulling the slot machine" failure rate and the intrinsic failure rate?
I feel like saying "the market will fix the incentives" handwaves away the lack of information on internals. After all, look at the market response to Google making their search less reliable - sure, an invested nerd might try Kagi, but Google's still the market leader by a long shot.
In a market for lemons, good luck finding a lime.
FWIW, kagi is better than Google
yes, that was their point. Everyone uses Google anyway.
And we all know the market always gives us the best quality product ...
I was thinking more of deliberate backdoor in code. RCE is an obvious example, but another one could be bias. "I'm sorry ma'am, computer says you are ineligable for a bank account." These ideas aren't new. They were there in 90s already when we still thought about privacy and accountability regarding technology, and dystopian novels already described them long, long ago.
Is this from a page of dating apps playbook?
yes
> These can be subtle because you're not looking for them
After any agent run, I'm always looking the git comparison between the new version and the previous one. This helps catch things that you might otherwise not notice.
And after manually coding I often have an LLM review the diff. 90% of the problems it finds can be discounted, but it’s still a net positive.
And there is this paradox where it becomes harder to detect the problems as the models 'improve'.
You are using it wrong, or are using a weak model if your failure rate is over 50%. My experience is nothing like this. It very consistently works for me. Maybe there is a <5% chance it takes the wrong approach, but you can quickly steer it in the right direction.
you are using it on easy questions. some of us are not.
A lot of people are getting good results using it on hard things. Obviously not perfect, but > 50% success.
That said, more and more people seem to be arriving at the conclusion that if you want a fairly large-sized, complex task in a large existing codebase done right, you'll have better odds with Codex GPT-5.2-Codex-XHigh than with Claude Code Opus 4.5. It's far slower than Opus 4.5 but more likely to get things correct, and complete, in its first turn.
yes, i also get good results. that's why i use it on the hard things.
I think a lot of it comes down to how well the user understands the problem, because that determines the quality of instructions and feedback given to the LLM.
For instance, I know some people have had success with getting claude to do game development. I have never bothered to learn much of anything about game development, but have been trying to get claude to do the work for me. Unsuccessful. It works for people who understand the problem domain, but not for those who don't. That's my theory.
It works for hard problems when the person already solves it and just needs the grunt work done
It also works for problems that have been solved a thousand times before, which impresses people and makes them think it is actually solving those problems
Which matches what they are. They're first and foremost pattern recognition engines extraordinaire. If they can identify some pattern that's out of whack in your code compared to something in the training data, or a bug that is similar to others that have been fixed in their training set, they can usually thwack those patterns over to your latent space and clean up the residuals. If comparing pattern matching alone, they are superhuman, significantly.
"Reasoning", however, is a feature that has been bolted on with a hacksaw and duct tape. Their ability to pattern match makes reasoning seem more powerful than it actually is. If your bug is within some reasonable distance of a pattern it has seen in training, reasoning can get it over the final hump. But if your problem is too far removed from what it has seen in its latent space, it's not likely to figure it out by reasoning alone.
Exactly. I go back to a recent ancestor of LLMs, seq2seq. Its purpose was to translate things. Thats all. That needed representation learning and an attention mechanism, and it lead to some really freaky emergent capabilities, but its trained to trainslate language.
And thats exactly what its good for. It works great if you already solve a tough problem and provide it the solution in natural language, because the program is already there, it just needs to translate it to python.
Anything more than that that might emerge from this is going to be unreliable sleight of next-token-prediction at best.
We need a new architectural leap to have these things reason, maybe something that involves reinforcement learning at the token represention level, idk. But scaling the context window and training data arent going to cut it
>"Reasoning", however, is a feature that has been bolted on with a hacksaw and duct tape.
What do you mean by this? Especially for tasks like coding where there is a deterministic correct or incorrect signal it should be possible to train.
it's meant in the literal sense but with metaphorical hacksaws and duct tape.
Early on, some advanced LLM users noticed they could get better results by forcing insertion of a word like "Wait," or "Hang on," or "Actually," and then running the model for a few more paragraphs. This would increase the chance of a model noticing a mistake it made.
Reasoning is basically this.
It's not just force inserting a word. Reasoning is integrated into the training process of the model.
Not the core foundation model. The foundation model still only predicts the next token in a static way. The reasoning is tacked onto the instructGPT style finetuning step and its done through prompt engineering. Which is the shittiest way a model like this could have been done, and it shows
> It also works for problems that have been solved a thousand times before
So you mean it works on almost all problems?
I mean problems not worth solving, because theyve already been solved. If you need to just do the grunt work of retrieving the solution to a trite and worn out problem from the models training data, then they work great
But if you want to do interesting things, like all the shills keep trying to claim they do. Then this wont do it for you. You have to do it for it
> But if you want to do interesting things, like all the shills keep trying to claim they do
I don't know where this is coming from. I've seen some over-enthusiastic hype for sure, but most of the day-to-day conversations I see aren't people saying they're curing cancer with Claude, they're people saying they're automating their bread and butter tasks with great success.
Don’t use it for hard questions like this then; you wouldn’t use a hammer to cut a plank, you’d try to make a saw instead
> It might become cheaper or it might not
If it does not, this is going to be first technology in the history of mankind that has not become cheaper.
(But anyway, it already costs half compared to last year)
> But anyway, it already costs half compared to last year
You could not have bought Claude Opus 4.5 at any price one year ago I'm quite certain. The things that were available cost half of what they did then, and there are new things available. These are both true.
I'm agreeing with you, to be clear.
There are two pieces I expect to continue: inference for existing models will continue to get cheaper. Models will continue to get better.
Three things, actually.
The "hitting a wall" / "plateau" people will continue to be loud and wrong. Just as they have been since 2018[0].
[0]: https://blog.irvingwb.com/blog/2018/09/a-critical-appraisal-...
As a user of LLMs since GPT-3 there was noticeable stagnation in LLM utility after the release of GPT-4. But it seems the RLHF, tool calling, and UI have all come together in the last 12 months. I used to wonder what fools could be finding them so useful to claim a 10x multiplier - even as a user myself. These days I’m feeling more and more efficiency gains with Claude Code.
That's the thing people are missing, the models plateaued a while ago, still making minor gains to this day, but not huge ones. The difference is now we've had time to figure out the tooling. I think there's still a ton of ground to cover there and maybe the models will improve given that the extra time, but I think it's foolish to consider people who predicted that completely wrong. There are also a lot of mathematical concerns that will cause problems in the near and distant future. Infinite progress is far from a given, we're already way behind where all the boosters thought we'd be my now.
I believe Sam Altman, perhaps the greatest grifter in today’s Silicon Valley, claimed that software engineering would be obsolete by the end of last year.
> The "hitting a wall" / "plateau" people will continue to be loud and wrong. Just as they have been since 2018[0].
Everybody who bet against Moore's Law was wrong ... until they weren't.
And AI is the reaction to Moore's Law having broken. Nobody gave one iota of damn about trying to make programming easier until the chips couldn't double in speed anymore.
This is exactly backwards: Dennard scaling stopped. Moore’s Law has continued and it’s what made training and running inference on these models practical at interactive timescales.
You are technically correct. The best kind of correct.
However, most people don't know the difference between the proper Moore's Law scaling (the cost of a transistor halves every 2 years) which is still continuing (sort of) and the colloquial version (the speed of a transistor doubles every 2 years) which got broken when Dennard scaling ran out. To them, Moore's Law just broke.
Nevertheless, you are reinforcing my point. Nobody gave a damn about improving the "programming" side of things until the hardware side stopped speeding up.
And rather than try to apply some human brainpower to fix the "programming" side, they threw a hideous number of those free (except for the electricity--but we don't mention that--LOL) transistors at the wall to create a broken, buggy, unpredictable machine simulacrum of a "programmer".
(Side note: And to be fair, it looks like even the strong form of Moore's Law is finally slowing down, too)
If you can turn a few dollars of electricity per hour into a junior-level programmer who never gets bored, tired, or needs breaks, that fundamentally changes the economics of information technology.
And in fact, the agentic looped LLMs are executing much better than that today. They could stop advancing right now and still be revolutionary.
interesting post. i wonder if these people go back and introspect on how incorrect they have been? do they feel the need to address it?
No, people do not do that.
This is harmless when it comes to tech opinions but causes real damage in politics and activism.
People get really attached to ideals and ideas, and keep sticking to those after they fail to work again and again.
i don't think it is harmless or we are incentivising people to just say whatever they want without any care for truth. people's reputations should be attached to their predictions.
Some people definitely do but how do they go and address it? A fresh example in that it addresses pure misinformation. I just screwed up and told some neighbors garbage collection was delayed for a day because of almost 2ft of snow. Turns out it was just food waste and I was distracted checking the app and read the notification poorly.
I went back to tell them (do not know them at all just everyone is chattier digging out of a storm) and they were not there. Feel terrible and no real viable remedy. Hope they check themselves and realize I am an idiot. Even harder on the internet.
Do _you_ do that?
i try to yes
That's not true. Many technologies get more expensive over time, as labor gets more expensive or as certain skills fall by the wayside, not everything is mass market. Have you tried getting a grandfather clock repaired lately?
Repairing grandfather clocks isn't more expensive now because it's gotten any harder; it's because the popularity of grandfather clocks is basically nonexistent compared to anything else to tell time.
Doesn't need to be any particular reason to disprove the notion that technology only gets cheaper.
"repairing a unique clock" getting costlier doesn't mean technology hasn't gotten cheaper.
check out whether clocks have gotten cheaper in general. the answer is that it has.
there is no economy of scale here in repairing a single clock. its not relevant to bring it up here.
Clocks prices have gone up since 2020. Unless a cheaper better way to make clocks has emerged inflation causes prices to grow.
Luxury watches have gone up, 'clocks' as a technology is cheaper than ever.
You can buy one for 90 cents on temu.
The landing cost for that 90 cent watch has gone way up. Shipping and to some degree taxes has pushed the price higher.
that's not the technology
of course it's silly to talk about manufacturing methods and yield and cost efficiency without having an economy to embed all of this into, but ... technology got cheaper means that we have practical knowledge of how to make cheap clocks (given certain supply chains, given certain volume, and so and so)
we can make very cheap very accurate clocks that can be embedded into whatever devices, but it requires the availability of fabs capable of doing MEMS components, supply materials, etc.
not true, clocks have gone down after accounting for inflation. verified using ChatGPT.
You cannot verify anything using a tool that cannot validate truth.
You can't account for inflation because the price increase is inflation.
you can look at a basket of goods that doesn't have your specific product and compare directly
but inflation is the general price level increase, this can be used as a deflator to get the price of whatever product in past/future money amount to see how the price of the product changed in "real" terms (ie. relative to the general price level change)
this is not true
Instead of advancing tenuous examples you could suggest a realistic mechanism by which costs could rise, such as a Chinese advance on Taiwan, effecting TSMC, etc.
No. You don't get to make "technology gets more expensive over time" statements for deprecated technologies.
Getting a bespoke flintstone axe is also pretty expensive, and has also absolutely no relevance to modern life.
These discussions must, if they are to be useful, center in a population experience, not in unique personal moments.
I purchased a 5T drive in 2019 and the price is higher now despite newer better drives going on the market since.
Not much has down in price over the last few years.
Price volatility exists.
Meanwhile the overall price of storage has been going down consistently: https://ourworldindata.org/grapher/historical-cost-of-comput...
okay how about the Francis Scott Key Bridge?
https://marylandmatters.org/2025/11/17/key-bridge-replacemen...
You will get a different bridge. With very different technology. Same as "I can't repair my grandfather clock cheaply".
In general, there are several things that are true for bridges that aren't true for most technology:
* Technology has massively improved, but most people are not realizing that. (E.g. the Bay Bridge cost significantly more than the previous version, but that's because we'd like to not fall down again in the next earthquake) * We still have little idea how to reason about the cost of bridges in general. (Seriously. It's an active research topic) * It's a tiny market, with the major vendors forming an oligopoly * It's infrastructure, not a standard good * The buy side is almost exclusively governments.
All of these mean expensive goods that are completely non-repeatable. You can't build the same bridge again. And on top of that, in a distorted market.
But sure, the cost of "one bridge, please" has gone up over time.
This seems largely the same as any other technology. The prices of new technologies go down initially as we scale up and optimize it's production, but as soon as demand fades, due to newer technology or whatever, the cost of that technology goes up again.
> But sure, the cost of "one bridge, please" has gone up over time.
Even if you adjust for inflation?
Depends, do we care about TCO? Also, can I pick the set of bridges I compare?
OK, kidding aside: If you deeply care, you can probably mine the Federal Highway Administration's bridge construction database: https://fhwaapps.fhwa.dot.gov/upacsp/tm?transName=MenuSystem...
I don't think the question is answerable in a meaningful way. Bridges are one-off projects with long life spans, comparing cost over time requires a lot of squinting just so.
Bought any RAM lately? Phone? GPU in the last decade?
The latest iphone has gone down in price? It's double. I guess the marketing is working.
"Pens are not cheaper, look at this Montblanc" is not a good faith response.
'84 Motorola DynaTAC - ~$12k AfI (adjusted for inflation)
'89 MicroTAC ~$8k AfI
'96 StarTAC ~$2k AfI
`07 iPhone ~$673 AfI
The current average smartphone sells for around $280. Phones are getting cheaper.
Phones, or smartphones?
Time-keeping is vastly cheaper. People don't want grandfather clocks. They want to tell time. And they can, more accurately, more easily, and much cheaper than their ancestors.
Or riding in an uber?
Sure, running an LLM is cheaper, but the way we use LLMs now requires way more tokens than last year.
10x more tokens today cost less than than half of X tokens from ~mid 2024.
ok but the capabilities are also rising. what point are you trying to make?
That it's not getting cheaper?
But it is, capability adjusted, which is the only way it makes sense. You can definitely produce last years capability at a huge discount.
you are wrong. https://epoch.ai/data-insights/llm-inference-price-trends
this is accounting for the fact that more tokens are used.
The chart shows that they’re right though. Newer models cost more than older models. Sure they’re better but that’s moot if older models are not available or can’t solve the problem they’re tasked with.
this is incorrect. the cost to achieve the same task by old models is way higher than by new models.
> Newer models cost more than older models
where did you see this?
On the link you shared, 4o vs 3.5 turbo price per 1m tokens.
There’s no such thing as ”same task by old model”, you might get comparable results or you might not (and this is why the comparison fail, it’s not a comparison), the reason you pick the newer models is to increase chances of getting a good result.
> The dataset for this insight combines data on large language model (LLM) API prices and benchmark scores from Artificial Analysis and Epoch AI. We used this dataset to identify the lowest-priced LLMs that match or exceed a given score on a benchmark. We then fit a log-linear regression model to the prices of these LLMs over time, to measure the rate of decrease in price. We applied the same method to several benchmarks (e.g. MMLU, HumanEval) and performance thresholds (e.g. GPT-3.5 level, GPT-4o level) to determine the variation across performance metrics
This should answer. In your case, GPT-3.5 definitely is cheaper per token than 4o but much much less capable. So they used a model that is cheaper than GPT-3.5 that achieved better performance for the analysis.
OpenAI has always priced newer models lower than older ones.
not true! 4o was costlier than 3.5 turbo
https://platform.openai.com/docs/pricing
Not according to their pricing table. Then again I’m not sure what OpenAI model versions even mean anymore, but I would assume 5.2 is in the same family as 5 and 5.2-pro as 5-pro
Check GPT 5.2 vs it's predecessor the 'o' series of reasoning models.
Not true. Bitcoin has continued to rise in cost since its introduction (as in the aggregate cost incurred to run the network).
LLMs will face their own challenges with respect to reducing costs, since self-attention grows quadratically. These are still early days, so there remains a lot of low hanging fruit in terms of optimizations, but all of that becomes negligible in the face of quadratic attention.
For Bitcoin that’s by design!
> bitcoin
so close! that is a commodity
I don't think computation is going to become more expensive, but there are techs that have become so: Nuclear power plants. Mobile phones. Oil extraction.
(Oil rampdown is a survival imperative due to the climate catastrophe so there it's a very positive thing of course, though not sufficient...)
There are plenty of technologies that have not become cheaper, or at least not cheap enough, to go big and change the world. You probably haven't heard of them because obviously they didn't succeed.
cheaper doesnt mean cheap enough to be viable after the bills come due
Supersonic jet engines, rockets to the moon, nuclear power plants, etc. etc. all have become more expensive. Superconductors were discovered in 1911, and we have been making them for as long as we have been making transistors in the 1950s, yet superconductors show no sign of becoming cheaper any time soon.
There have been plenty of technologies in history which do not in fact become cheaper. LLMs are very likely to become such, as I suspect their usefulness will be superseded by cheaper (much cheaper in fact) specialized models.
Concorde?
With optimizations and new hardware, power is almost a negligible cost. You can get 5.5M tokens/s/MW[1] for kimi k2(=20M/KWH=181M tokens/$) which is 400x cheaper than current pricing. It's just Nvidia/TSMC/other manufacturers eating up the profit now because they can. My bet is that China will match current Nvidia within 5 years.
[1]: https://developer-blogs.nvidia.com/wp-content/uploads/2026/0...
Electricity is negligible but the dominant cost is the hardware depreciation itself. Also inference is typically memory bandwidth bound so you are limited by how fast you can move weights rather than raw compute efficiency.
Yes, because the margin is like 80% for Nvidia, and 80% again for the manufacturers like Samsung and TSMC. Once the fixed cost like R and D is amortized the same node technology and hardware capacity could be just few single digit percent of current.
> And you most likely do not pay the actual costs.
This is one of the weakest anti AI postures. "It's a bubble and when free VC money stops you'll be left with nothing". Like it's some kind of mystery how expensive these models are to run.
You have open weight models right now like Kimi K2.5 and GLM 4.7. These are very strong models, only months behind the top labs. And they are not very expensive to run at scale. You can do the math. In fact there are third parties serving these models for profit.
The money pit is training these models (and not that much if you are efficient like chinese models). Once they are trained, they are served with large profit margins compared to the inference cost.
OpenAI and Anthropic are without a doubt selling their API for a lot more than the cost of running the model.
Humans run hot too. Once you factor in the supply chain that keeps us alive, things become surprisingly equivalent.
Eating burgers and driving cars around costs a lot more than whatever # of watts the human brain consumes.
I mean, “equivalent” is an understatement! There’s a reason Claude Code costs less than hiring a full time software engineer…
(it's VC money burn)
> Somewhere, there are GPUs/NPUs running hot.
Running at their designed temperature.
> You send all the necessary data, including information that you would never otherwise share.
I've never sent the type of data that isn't already either stored by GitHub or a cloud provider, so no difference there.
> And you most likely do not pay the actual costs.
So? Even if costs double once investor subsidies stop, that doesn't change much of anything. And the entire history of computing is that things tend to get cheaper.
> You and your business become dependent on this major gatekeeper.
Not really. Switching between Claude and Gemini or whatever new competition shows up is pretty easy. I'm no more dependent on it than I am on any of another hundred business services or providers that similarly mostly also have competitors.
It is also amazing seeing Linux kernel work, scheduling threads, proving interrupts and API calls all without breaking a sweat or injuring its ACL.
To me this tenacity is often like watching someone trying to get a screw into board using a hammer.
There’s often a better faster way to do it, and while it might get to the short term goal eventually, it’s often created some long term problems along the way.
I don’t understand this pov. Unfortunately, id pay 10k mo for my cc sub. I wish I could invest in anthropic, they’re going to be the most profitable company on earth
My agent struggled for 45 minutes because it tried to do `go run` on a _test.go file, which the compiler repeatedly exited after posting an error message that files named like this cannot be executed using the run command.
So yeah, that wasted a lot of GPU cycles for a very unimpressive result, but with a renewed superficial feeling of competence
> you most likely do not pay the actual costs. It might become cheaper or it might not
Why would this be the first technology that doesn't become cheaper at scale over time?
> And you most likely do not pay the actual costs
Oh my lord you absolutely do not. The costs to oai per token inference ALONE are at least 7x. AT LEAST and from what I’ve heard, much higher.
We can observe how much generic inference providers like deepinfra or together-ai charge for large SOTA models. Since they are not subsidized and they don’t charge 7x of OpenAI, that means OAI also doesn’t have outrageously high per-token costs.
AI genius discover brute forcing... what a time to be alive. /s
Like... bro that's THE foundation of CS. That's the principle of The bomb in Turing's time. One can still marvel at it but it's been with us since the beginning.
> Atrophy. I've already noticed that I am slowly starting to atrophy my ability to write code manually... > Largely due to all the little mostly syntactic details involved in programming, you can review code just fine even if you struggle to write it.
Until you struggle to review it as well. Simple exercise to prove it - ask LLM to write a function in familiar programming language, but in the area you didn't invest learning and coding yourself. Try reviewing some code involving embedding/SIMD/FPGA without learning it first.
People would struggle to review code in a completely unfamiliar domain or part of the stack even before LLMs.
That’s why you need to write code to learn it.
No-one has ever learned skill just by reading/observing
"No-one has ever learned skill just by reading/observing" - Except of course all of those people in Cosmology who, you know, observe.
what skill do they have? making stars? no they are skilled at observing, which is what they do.
I think understanding stellar processes and then using that understanding to theorize about other observations is a skill. My point was that observing can be a fantastic way to build a skill.. not all skills, but certainly some skills. Learning itself is as much an observation as a practice.
How would you find yourself in that situation before AI?
No, because they wouldn't be so foolish as to try it.
Boris Cherny (Claude Code creator) replies to Andrej Karpathy
https://xcancel.com/bcherny/status/2015979257038831967
> It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day. It's a "feel the AGI" moment to watch it struggle with something for a long time just to come out victorious 30 minutes later.
The bits left unsaid:
1. Burning tokens, which we charge you for
2. My CPU does this when I tell it to do bogosort on a million 32-bit integers, it doesn't mean it's a good thing
So what is he even coding there all the time?
Does anybody have any info on what he is actually working on besides all the vibe-coding tweets?
There seems to be zero output from they guy for the past 2 years (except tweets)
> There seems to be zero output from they guy for the past 2 years (except tweets)
Well, he made Nanochat public recently and has been improving it regularly [1]. This doesn't preclude that he might be working on other projects that aren't public yet (as part of his work at Eureka Labs).
1: https://github.com/karpathy/nanochat
So, it's generative pre-trained transformers again?
He's building Eureka Labs[1], an AI-first education company (can't wait to use it). He's both a strong researcher[2] and an unusually gifted technical communicator. His recent videos[3] are excellent educational material.
More broadly though: someone with his track record sharing firsthand observations about agentic coding shouldn't need to justify it by listing current projects. The observations either hold up or they don't.
[1] https://x.com/EurekaLabsAI
[2] PhD in DL, early OpenAI, founding head of AI at Tesla
[3] https://www.youtube.com/@AndrejKarpathy/videos
If LLM coding is a 10x productivity enhancer, why aren't we seeing 10x more software of the same quality level, or 100x as much shitty software?
Helper scripts for APIs for applications and tools I know well. LLMs have made my work bearable. Many software providers expose great apis, but expert use cases require data output/input that relies on 50-500 line scripts. Thanks to the models post gpt4.5 most requirements are solvable in 15 minutes when they could have taken multiple workdays to write and check by hand. The only major gap is safe ad-hoc environments to run these in. I provide these helper functions for clients that would love to keep the runtime in the same data environment as the tool, but not all popular software support FaaS style environments that provide something like a simple python env.
I don’t know, but it’s interesting that he and many others come up with this “we should act like LLMs are junior devs”. There is a reason why most junior devs work on fairly separate parts of products, most of the time parts which can be removed or replaced easily, and not an integral part of products: because their code is usually quite bad. Like every few lines contains issues, suboptimal solutions, and full with architectural problems. You basically never trust junior devs with core product features. Yet, we should pretend that an “LLM junior dev” is somehow different. These just signal to me that these people don’t work on serious code.
This is the first question I ask, and every time I get the answer of some monolith that supposedly solves something. Imo, this is completely fine for any personal thing, I am happy when someone says they made an API to compare weekly shopping prices from the stores around them, or some recipe, this makes sense.
However more often than not, someone is just building a monolithic construction that will never be looked at again. For example, someone found that HuggingFace dataloader was slow for some type of file size in combination with some disk. What does this warrant? A 300000+ line non-reviewed repo to fix this issue. Not a 200-line PR to HuggingFace, no you need to generate 20% of the existing repo and then slap your thing on there.
For me this is puzzling, because what is this for? Who is this for? Usually people built these things for practice, but now its generated, so its not for practice because you made very little effort on it. The only thing I can see that its some type of competence signaling, but here again, if the engineer/manager looking knows that this is generated, it does not have the type of value that would come with such signaling. Either I am naive and people still look at these repos and go "whoa this is amazing", or it's some kind of induced egotrip/delusion where the LLM has convinced you that you are the best builder.
I wish the people who wrote this let us know what king of codebases they are working on. They seem mostly useless in a sufficiently large codebase especially when they are messy and interactions aren't always obvious. I don't know how much better Claude is than ChatGPT, but I can't get ChatGPT to do much useful with an existing large codebase.
This is an antidotal example, but I released this last week after 3 months of work on it as a "nights and weekdends" project: https://apps.apple.com/us/app/skyscraper-for-bluesky/id67541...
I've been working in the mobile space since 2009, though primarily as a designer and then product manager. I work in kinda a hybrid engineering/PM job now, and have never been a particularly strong programmer. I definitely wouldn't have thought I could make something with that polish, let alone in 3 months.
That code base is ~98% Claude code.
I don’t know if “antidotal example” is a pun or a typo but I quite like it.
Lol typing on my phone during lunch and meant anecdotal. But let's leave it anyways. :)
That is fun.
Not sure if it's an American pronunciation thing, but I had to stare at that long and hard to see the problem and even after seeing it couldn't think of how you could possibly spell the correct word otherwise.
> Not sure if it's an American pronunciation thing
It's a bad American pronunciation thing like "Febuwary" and "nuculer".
If you pronounce the syllables correctly, "an-ec-dote", "Feb-ru-ar-y", "nu-cle-ar" the spellings follow.
English has it's fair share of spelling stupidities, but if people don't even pronounce the words correctly there is no hope.
https://en.wiktionary.org/wiki/February
The pronunciation of the first r with a y sound has always been one of two possible standards, in fact "February" is a re-Latinizing spelling but English doesn’t like the br-r sound so it naturally dissimilates to by-r.
Claude and Codex are CLI tools you use to give the LLM context about the project on your local machine or dev environment. The fact that you're using the name "ChatGPT" instead of Codex leads me to believe you're talking about using the web-based ChatGPT interface to work on a large codebase, which is completely beside the point of the entire discussion. That's not the tool anyone is talking about here.
It's important to understand that he's talking about a specific set of models that were release around november/december, and that we've hit a kind of inflection point in model capabilities. Specifically Anthropic's Opus 4.5 model.
I never paid any attention to different models, because they all felt roughly equal to me. But Opus 4.5 is really and truly different. It's not a qualitative difference, it's more like it just finally hit that quantitative edge that allows me to lean much more heavily on it for routine work.
I highly suggest trying it out, alongside a well-built coding agent like the one offered by Claude Code, Cursor, or OpenCode. I'm using it on a fairly complex monorepo and my impressions are much the same as Karpathy's.
People have said this about every single model release.
I had the same reaction. So when people were talking about this model back in December, I brushed it off. It wasn't until a couple weeks ago that I decided to try it out, and I immediately saw the difference.
My opinion isn't based on what other people are saying, it's my own experience as a fairly AI-skeptical person. Again, I highly suggest you give it an honest try and decide for yourself.
Are you using Codex?
I'm not sure how big your repos are but I've been effective working with repos that have thousands of files and tens of thousands of lines of code.
If you're just prototyping it will hit wall when things get unwieldy but that's normally a sign that you need to refactor a bit.
Super strict compiler settings, static analysis, comprehensive tests, and documentation help a lot. As does basic technical design. After a big feature is shipped I do a refactor cycle with the LLM where we do a comprehensive code review and patch things up. This does require human oversight because the LLMs are still lacking judgement on what makes for good code design.
The places where I've seen them be useless is working across repositories or interfacing with things like infrastructure.
It's also very model-dependent. Opus is a good daily driver but Codex is much better are writing tests for some reason. I'll often also switch to it for hard problems that Claude can't solve. Gemini is nice for 'I need a prototype in the next 10 minutes', especially for making quick and dirty bespoke front-ends where you don't care about the design just the functionality.
> tens of thousands of lines of code
Perhaps this is part of it? Tens of thousands of lines of code seems like a very small repo to me.
Almost always, notes like these are going to be about greenfield projects.
Trying to incorporate it in existing codebases (esp when the end user is a support interaction or more away) is still folly, except for closely reviewed and/or non-business-logic modifications.
That said, it is quite impressive to set up a simple architecture, or just list the filenames, and tell some agents to go crazy to implement what you want the application to do. But once it crosses a certain complexity, I find you need to prompt closer and closer to the weeds to see real results. I imagine a non-technical prompter cannot proceed past a certain prototype fidelity threshold, let alone make meaningful contributions to a mature codebase via LLM without a human engineer to guide and review.
I'm using it on a large set of existing codebases full of extremely ugly legacy code, weird build systems, tons of business logic and shipping directly to prod at neckbreaking growth over the last two years, and it's delivering the same type of value that Karpathy writes about.
That was true for me, but is no longer.
It's been especially helpful in explaining and understanding arcane bits of legacy code behavior my users ask about. I trigger Claude to examine the code and figure out how the feature works, then tell it to update the documentation accordingly.
> I trigger Claude to examine the code and figure out how the feature works, then tell it to update the documentation accordingly.
And how do you verify its output isn't total fabrication?
I read through it, scanning sections that seem uncontroversial and reading more closely sections that talk about things I'm less sure about. The output cites key lines of code, which are faster to track down and look at than trying to remember where in a large codebase to look.
Inconsistencies also pop up in backtesting, for example if there's a point that the llm answers different ways in multiple iterations, that's a good candidate to improve docs on.
Similar to a coworker's work, there's a certain amount of trust in the competency involved.
Your docs are a contact. You can verify that contract using integration tests
Contract? These docs are information answering user queries. So if you use a chatbot to generate them, I'd like to be reasonably sure they aren't laden with the fabricated misinformation for which these chatbots are famous.
It's a very reasonable concern. My solution is to have the bot classify what the message is talking about as a first pass, and have a relatively strict filtering about what it responds to.
For example, I have it ignore messages about code freezes, because that's a policy question that probably changes over time, and I have it ignore urgent oncall messages, because the asker there probably wants a quick response from a human.
But there's a lot of questions in the vein of "How do I write a query for {results my service emits}", how does this feature work, where automation can handle a lot (and provide more complete answers than a human can off the top of their head)
OK, but little of that applies to this use case, to "then tell it to update the documentation accordingly."
These models do well changing brownfield applications that have tests because the constraints on a successful implementation are tight. Their solutions can be automatically augmented by research and documentation.
I don't exactly disagree with this but I have seen models simply deleting the tests, or updating the tests to pass and declaring the failures were "unrelated to my changes", so it helpfully fixed them
I’ve had to deal with this a handful of times. You just have to make it restore the test, or keep trying to pass a suite of explicit red-green method tests it wrote earlier.
Yes. You have to treat the model like an eager yet incompetent worker, i.e. don't go full yolo mode and review everything they do.
For me, in just the golang server instance and the core functional package, `cloc` reports over 40k lines of code, not counting other supporting packages. I spent the last week having Claude rip out the external auth system and replace it with a home-grown one (and having GPT-codex review its changes). If anything, Claude makes it easier on me as a solo founder with a large codebase. Rather than having to re-familiarize myself with code I wrote a year ago, I describe it at a high level, point Claude to a couple of key files, and then tell it to figure out what it needs to do. It can use grep, language server, and other tools to poke around and see what's going on. I then have it write an "epic" in markdown containing all the key files, so that future sessions already know the key files to read.
I really enjoyed the process. As TFA says, you have to keep a close eye on it. But the whole process was a lot less effort, and I ended up doing mor than I would otherwise have done.
I don't know how big sufficiently large codebase is, but we have a 1mil loc Java application, that is ~10years old, and runs POS systems, and Claude Code has no issues with it. We have done full analyses with output details each module, and also used it to pinpoint specific issues when described. Vibe coding is not used here, just analysis.
At my dayjob my team uses it on our main dashboard, which is a pretty large CRUD application. The frontend (Vue) is a horrible mess, as it was originally built by people who know just enough to be dangerous. Over time people have introduced new standards without cleaning up the old code - for example, we have three or four different state management techologies.
For this the LLM struggles a bit, but so does a human. The main issues are it messes up some state that it didnt realise was used elsewhere, and out test coverage is not great. We've seen humans make exactly the same kind of mistakes. We use MCP for Figma so most of the time it can get a UI 95% done, just a few tweaks needed by the operator.
On the backend (Typescript + Node, good test coverage) it can pretty much one-shot - from a plan - whatever feature you give it.
We use opus-4.5 mostly, and sometimes gpt-5.2-codex, through Cursor. You aren't going to get ChatGPT (the web interface) to do anything useful, switch to Cursor, Codex or Claude Code. And right now it is worth paying for the subscription, you don't get the same quality from cheaper or free models (although they are starting to catch up, I've had promising results from GLM-4.7).
Another personal example. I spent around a month last year in January on this application: https://apps.apple.com/us/app/salam-prayer-qibla-quran/id674...
I had never used Swift before that and was able to use AI to whip up a fairly full-featured and complex application with a decent amount of code. I had to make some cross-cutting changes along the way as well that impacted quite a few files and things mostly worked fine with me guiding the AI. Mind you this was a year ago so I can only imagine how much better I would fare now with even better AI models. That whole month was spent not only on coding but on learning Swift enough to fix problems when AI started running into circles and then learning about Xcode profiler to optimize the application for speed and improving perf.
> They seem mostly useless in a sufficiently large codebase especially when they are messy and interactions aren't always obvious.
What type of documents do you have explaining the codebase and its messy interactions, and have you provided that to the LLM?
Also, have you tried giving someone brand new to the team the exact same task and information you gave to the LLM, and how effective were they compared to the LLM?
> I don't know how much better Claude is than ChatGPT, but I can't get ChatGPT to do much useful with an existing large codebase.
As others have pointed out, from your comment, it doesn't sound like you've used a tool dedicated for AI coding.
(But even if you had, it would still fail if you expect LLMs to do stuff without sufficient context).
The code base I work on at $dayjob$ is legacy, has few files with 20k lines each and a few more with around 10k lines each. It's hard to find things and connect dots in the code base. Dont think LLMs able to navigate and understand code bases of that size yet. But have seen lots of seemingly large projects shown here lately that involve thousands of files and millions of lines of code.
I’ve found that LLMs seem to work better on LLM-generated codebases.
Commercial codebases, especially private internal ones, are often messy. It seems this is mostly due to the iterative nature of development in response to customer demands.
As a product gets larger, and addresses a wider audience, there’s an ever increasing chance of divergence from the initial assumptions and the new requirements.
We call this tech debt.
Combine this with a revolving door of developers, and you start to see Conway’s law in action, where the system resembles the organization of the developers rather than the “pure” product spec.
With this in mind, I’ve found success in using LLMs to refactor existing codebases to better match the current requirements (i.e. splitting out helpers, modularizing, renaming, etc.).
Once the legacy codebase is “LLMified”, the coding agents seem to perform more predictably.
YMMV here, as it’s hard to do large refactors without tests for correctness.
(Note: I’ve dabbled with a test first refactor approach, but haven’t gone to the lengths to suggest it works, but I believe it could)
are LLM codebases not messy?
Claude by default, unless I tell it not to, will write stuff like:
instead of the very simple boolean logic that could express this in one line, with the "this code does what it obviously does" comments added all over the place.generally unless you tell it not to, it does things in very verbose ways that most humans would never do, and since there's an infinite number of ways that it can invent absurd verbosity, it is hard to preemptively prompt against all of them.
to be clear, I am getting a huge amount of value out of it for executing a bunch of large refactors and "modernization" of a (really) big legacy codebase at scale and in parallel. but it's not outputting the sort of code that I see when someone prompts it "build a new feature ...", and a big part of my prompts is screaming at it not to do certain things or to refuse the task if it at any point becomes unsure.
Yeah to be clear it will have the same issues as a flyby contributor if prompted to.
Meaning if you ask it “handle this new condition” it will happily throw in a hacky conditional and get the job done.
I’ve found the most success in having it reason about the current architecture (explicitly), and then to propose a set of changes to accomplish the task (2-5 ways), review, and then implement the changes that best suit the scope of the larger system.
The failure mode is missing constraints, not “coding skill”. Treat the model as a generator that must operate inside an explicit workflow: define the invariant boundaries, require a plan/diff before edits, run tests and static checks, and stop when uncertainty appears. That turns “hacky conditional” behaviour into controlled change.
Yes, exactly.
The LLM is onboarding to your codebase with each context window, all it knows is what it’s seen already.
Right. Each context window is a partial view, so it cannot “know the codebase” unless you supply stable artefacts. Treat project state as inputs: invariants, interfaces, constraints, and a small set of must-keep facts. Then force changes through a plan and a diff, and gate with tests and checks. That turns context limits into a controlled boundary instead of a surprise.
Surely because LLM generated code is part of the training data for the model, so code/patterns it can work with is closer to its training data.
If you have a ChatGPT account, there's nothing stopping you from installing codex cli and using your chatgpt account with it. I haven't coded with ChatGPT for weeks. Maybe a month ago I got utility out of coding with codex and then having ChatGPT look at my open IDE page to give comments, but since 5.2 came out, it's been 100% codex.
Try Claude code. It’s different.
After you tried it, come back.
I think its not Claude code per se itself but rather the (Opus 4.5 model?) or something in an agentic workflow.
I tried a website which offered the Opus model in their agentic workflow & I felt something different too I guess.
Currently trying out Kimi code (using their recent kimi 2.5) for the first time buying any AI product because got it for like 1.49$ per month. It does feel a bit less powerful than claude code but I feel like monetarily its worth it.
Y'know you have to like bargain with an AI model to reduce its pricing which I just felt really curious about. The psychology behind it feels fascinating because I think even as a frugal person, I already felt invested enough in the model and that became my sunk cost fallacy
Shame for me personally because they use it as a hook to get people using their tool and then charge next month 19$ (I mean really Cheaper than claude code for the most part but still comparative to 1.49$)
Also I never see anyone talking about code reviews, which is one of the primary ways that software engineering departments manage liability. We fired someone recently because they couldn’t explain any of the slop they were trying to get merged. Why tf would I accept the liability of managing code that someone else can’t even explain?
I guess this is fine when you don’t have customers or stakeholders that give a shit lol.
1. Write good documentation, architecture, how things work, code styling, etc.
2. Put your important dependencies source code in the same directory. E.g. put a `_vendor` directory in the project, in it put the codebase at the same tag you're using or whatever: postgres, redis, vue, whatever.
3. Write good plans and requirements. Acceptance criteria, context, user stories, etc. Save them in markdown files. Review those multiple times with LLMs trying to find weaknesses. Then move to implementation files: make it write a detailed plan of what it's gonna change and why, and what it will produce.
4. Write very good prompts. LLMs follow instructions well if they are clear "you should proactively do X", is a weak instruction if you mean "you must do X".
5. LLMs are far from perfect, and full of limits. Karpathy sums their cons very well in his long list. If you don't know their limits you'll mismanage the expectations and not use them when they are a huge boost and waste time on things they don't cope well with. On top of that: all LLMs are different in their "personality", how they adhere to instruction, how creative they are, etc.
I successfully use Claude Code in a large complex codebase. It's Clojure, perhaps that helps (Clojure is very concise, expressive and hence token-dense).
Perhaps it's harder to "do Closure wrong" than it is to do JavaScript or Python or whatever other extremely flexible multi-paradigm high-level language
Having spent 3 years of my career working with Clojure, I think it actually gives you even more rope to shoot yourself with than Python/JS.
E.g. macros exist in Clojure but not Python/JS, and I've definitely been plenty stumped by seeing them in the codebase. They tend to be used in very "clever" patterns.
On the other hand, I'm a bit surprised Claude can tackle a complex Clojure codebase. It's been a while since I attempted using an LLM for Clojure, but at the time it failed completely (I think because there is relatively little training data compared to other mainstream languages). I'll have to check that out myself
I've been trying Claude on my large code base today. When I give it the requirements I'd give an engineer and so "do it" it just writes garbage that doesn't make sense and doesn't seem to even meet the requirements (if it does I can't follow how - though I'll admit to giving up before I understood what it did, and I didn't try it on a real system). When I forced it to step back and do tiny steps - in TDD write one test of the full feature - it did much better - but then I spent the next 5 hours adjusting the code it wrote to meet our coding standards. At least I understand the code, but I'm not sure it is any faster (but it is a lot easier to see things wrong than come up with green field code).
Which is to say you have to learn to use the tools. I've only just started, and cannot claim to be an expert. I'll keep using them - in part because everyone is demanding I do - but to use them you clearly need to know how to do it yourself.
Have you tried showing it a copy of your coding standards?
I also find pointing it to an existing folder full of code that conforms to certain standards can work really well.
Yeah let's share all your IP for the vague promise that it will somehow work ;)
You just gave me a revelation as to why some people report being unable to get decent results out of coding agents!
(Shrug) If you're not willing to make that tradeoff, you'll be outcompeted by people who are. Your call.
At least some of them that it violated it has seen.
I've been playing around with the "Superpowers" [0] plugin in Claude Code on a new small project and really like it. Simple enough to understand quickly by reading the GitHub repo and seems to improve the output quality of my projects.
There's basically a "brainstorm" /slash command that you go back and forth with, and it places what you came up with in docs/plans/YYYY-MM-DD-<topic>-design.md.
Then you can run a "write-plan" /slash command on the docs/plans/YYYY-MM-DD-<topic>-design.md file, and it'll give you a docs/plans/YYYY-MM-DD-<topic>-implementation.md file that you can then feed to the "execute-plan" /slash command, where it breaks everything down into batches, tasks, etc, and actually implements everything (so three /slash commands total.)
There's also "GET SHIT DONE" (GSD) [1] that I want to look at, but at first glance it seems to be a bit more involved than Superpowers with more commands. Maybe it'd be better for larger projects.
[0] https://github.com/obra/superpowers
[1] https://github.com/glittercowboy/get-shit-done
it's all about the context. observe what files it opened, etc. good luck
They build Claude Code fully with Claude Code.
Which is equal parts praise and damnation. Claude Code does do a lot of nice things that people just kind of don't bother for time cost / reward when writing TUIs that they've probably only done because they're using AI heavily, but equally it has a lot of underbaked edges (like accidentally shadowing the user's shell configuration when it tries to install terminal bindings for shift-enter even though the terminal it's configuring already sends a distinct shift-enter result), and bugs (have you ever noticed it just stop, unfinished?).
i haven't used Claude Code but come on.. it is a production level quality application used seriously by millions.
If you haven't used it, how can you judge its quality level?
Look up the flickering issue. The program was created by dunces.
Ah, now I understand why @autocomplete suddenly got broken between versions and still not fixed )
What do you even mean by "ChatGPT"? Copy pasting code into chatgpt.com?
AI assisted coding has never been like that, which would be atrocious. The typical workflow was using Cursor with some model of your choice (almost always an Anthropic model like sonnet before opus 4.5 released). Nowadays (in addition to IDEs) it's often a CLI tool like Claude Code with Opus or Codex CLI with GPT Codex 5.2 high/xhigh.
chatGPT is not made to write code. Get out of stone age :)
I'm afraid that we're entering a time when the performance difference between the really cutting edge and even the three-month-old tools is vast
If you're using plain vanilla chatgpt, you're woefully, woefully out of touch. Heck, even plain claude code is now outdated
Why is plain Claude code outdated? I thought that’s what most people are using right now that are AI forward. Is it Ralph loops now that’s the new thing?
Plain Claude Code doesn’t have enough scaffolding to handle large projects
At a base level, people are “upgrading” their Claude Code with custom skills and subagents - all text files saved in .claude/agents|skills.
You can also use their new tasks primitive to basically run a Ralph-like loop
But at the edges, people are using multiple instances, each handling different aspects in parallel - stuff like Gas Town
Tbf you can still get a lot of mileage out of vanilla Claude Code. But I’ve found that even adding a simple frontend design skill improves the output substantially
Is there anywhere where we can learn more about creating your own agents/skills? Maybe some decent public repos that you could recommend.
You can just ask Claude to create them. They’re just markdown files
Anthropic’s own repo is as good place as any
https://github.com/anthropics/skills
The Slopocalypse - an unexpected variant of Gray Goo:
https://en.wikipedia.org/wiki/Gray_goo
Well, it may consume the AI environment. Maybe even the internet. It's not going to consume a PC with g++, though (at least if the PC doesn't update g++ any more once g++ starts accepting AI contributions).
There may come a point where having a "survivor machine" with auto-update turned off may be a really good idea.
I already do this, in the form of survivor machines made to do initial coding on a retro platform so the result will translate across all possible platforms. Got to, as I'm an Apple coder primarily, so if I want to target older machines I can only do it through a survivor machine: support is always pruned out of Xcode and it would be insane to try and patch it to keep everything in scope.
Aslopalypse, a slop ellipse.
> - What does LLM coding feel like in the future? Is it like playing StarCraft? Playing Factorio? Playing music?
Starcraft and Factorio are exactly what it is not. Starcraft has a loooot of micro involved at any level beyond mid level play, despite all the "pro macros and beats gold league with mass queens" meme videos. I guess it could be like Factorio if you're playing it by plugging together blueprint books from other people but I don't think that's how most people play.
At that level of abstraction, it's more like grand strategy if you're to compare it to any video game? You're controlling high level pushes and then the units "do stuff" and then you react to the results.
I think the StarCraft analogy is fine, you have to compare it not to macro and micro RTS play, but to INDIVIDUAL UNITS. For your whole career until now, you have been a single Zergling or Probe. Now you are the Commander.
Except that pro starcraft player still micro-manage every single Zergling or probe when necessary, while vibe coders just right click on the ennemy base and hope it'll go well
It's like the Victoria 3 combat system. You just send an army and a general to a given front and let them get to work with no micro. Easy! But of course some percentage of the time they do something crazy like deciding to redeploy from your existential Franco-Prussian war front to a minor colonial uprising...
I sometimes wonder about the similarities between this paradigm switch (coding -> vibe coding) and when the industry switched from writing assembler to using high-level languages. I both cases we switched from having to specify every posibble implementation detail to focusing more on higher level concepts and letting the machine work out the rest. Maybe in the future instead of sharing source code, we will share prompts that we used to create a program. Similarly how different compilers produce different assembly now, "compiling" prompts with different agent/model would give different results. Maybe in the future an analog for "optimizing compiler" would emerge for agents, which would turn the (working) slop into something more clean.
> the ratio of productivity between the mean and the max engineer? It's quite possible that this grows *a lot*
I have a professor who has researched auto generated code for decades and about six months ago he told me he didn't think AI would make humans obsolete but that it was like other incremental tools over the years and it would just make good coders even better than other coders. He also said it would probably come with its share of disappointments and never be fully autonomous. Some of what he said was a critique of AI and some of it was just pointing out that it's very difficult to have perfect code/specs.
I can sense two classes of coders emerging.
Billionaire coder: a person who has "written" billion lines.
Ordinary coders : people with only couple of thousands to their git blame.
I think in less than a year writing code manually will be akin to doing arithmetic problems by hand. Sure you can still code manually, but it's going to be a lot faster to use an LLM (calculator).
People keep using these analogies but I think these are fundamentally different things.
1. hand arithmetic -> using a calculator
2. assembly -> using a high level language
3. writing code -> making an LLM write code
Number 3 does not belong. Number 3 is a fundamentally different leap because it's not based on deterministic logic. You can't depend on an LLM like you can depend on a calculator or a compiler. LLMs are totally different.
There are definitely parallels though. eg you could swap out your compiler for a different one that produces slightly different assembly. Similarly a LLM may implement things differently…but if it works do we care? Probably no more than when you buy software you don’t care precisely what compiler optimisation were used. The precise deterministicness isn’t a key feature
With the llm, it might work or it might not. If it doesn't work, then you have to keep iterating and hand holding it to make it work. Sometimes that process is less optimal than writing the code manually. With a calculator, you can be sure that the first attempt will work. An idiot with a calculator can still produce correct results. An idiot with an llm often cannot outside trivial solutions.
> but if it works so we care?
It often doesn't work. That's the point. A calculator works 100% of the time. A LLM might work 95% of the time, or 80%, or 40%, or 99% depending on what you're doing. This is difference and a key feature.
I see. I’d call that fragility/reliability rather than deterministic but semantics I suppose.
To me that isn’t a show stopper. Much of the real world works like that. We put very unreliable humans behind the wheel of 2 ton cars. So in a way this is perhaps just programmers aligning with the messy real world?
Perhaps a bit like architects can only model things so far eventually you need to build the thing and deal with the surprises and imperfection of dirt
I agree, but writing code is so different to calculations that long-term benefits are less clear.
It doesn't matter how good you are at calculations the answer to 2 + 2 is always 4. There are no methods of solving 2 + 2 which could result in you accidentally giving everyone who reads the result of your calculation write access to your entire DB. But there are different ways to code a system even if the UI is the same, and some of these may neglect to consider permissions.
I think a good parallel here would be to imagine that tomorrow we had access to humanoid robots who could do construction work. Would we want them to just go build skyscrapers and bridges and view all construction businesses which didn't embrace the humanoid robots as akin to doing arithmetic by hand?
You could of course argue that there's no problem here so long as trained construction workers are supervising the robots to make sure they're getting tolerances right and doing good welds, but then what happens 10 years down the road when humans haven't built a building in years? If people are not writing code any more then how can people be expected to review AI generated code?
I think the optimistic picture here is that humans just won't be needed in the future. In theory when models are good enough we should be able to trust the AI systems more than humans. But the less optimistic side of me questions a future in which humans no longer do, or even know how to do such fundamental things.
This is true if your calculator sometimes gave the wrong answer and you had to check each time
People who just let the agent code for them, how big of a codebase are you working on? How complex (i.e. is it a codebase that junior programmers could write and maintain)?
rust compiler and redox operating system with modified Qemu for Mac Vulcan metal pipeline ... probably not junior stuff
you might think I'm kidding but Search redox on github, you will find that project and the anonymous contributions
I am curious. What do you want us to see in that github repo?
I've been an EM for the last 10 of my 25 year Software Engineering career. Coding is, frankly, boring to me anymore, even though I enjoyed doing it most of my career. I had this project I wanted to exist in world but couldn't be bothered to get started.
Decided to figure out what this "vibe coding" nonsense is, and now there's a certain level of joy to all of this again. Being able to clearly define everything using markdown contexts before any code is even written has been a great way to brain dump those 25 years of experience and actually watch something sane get produced.
Here are the stats Claude Code gave me:
In case anyone is curious, here was my epiphany project from 2 weeks ago: https://github.com/boj/the-project
I then realized I could feed it everything it ever needed to know. Just create a docs/* folder and tell it to read that every session.
Through discovery I learned about CLAUDE.md, and adding skills.
Now I have an /analyst, /engineer, and /devops that I talk to all day with their own logic and limitations, as well as the more general project CLAUDE.md, and dozens of docs/* files we collaborate on.
I'm at the point I'm running happy.engineering on my phone and don't even need to sit in front of the computer anymore.
Interesting!
I wonder if this line
> It will configure an auth_backend.rs and wire up a basic user
over a big enough number of projects will lead to at least 2-3 different user names.
How much did this type of project cost you to make?
What kind of costs are you thinking?
money
Touching on the atrophy point, I actually wrote a few thoughts about this yesterday: https://www.neilwithdata.com/outsourced-thinking
I actually disagree with Andrej here re: "Generation (writing code) and discrimination (reading code) are different capabilities in the brain." and I would argue that the only reason he can read code fluently, find issues, etc. is because he has spent year in a non-AI assisted world writing code. As time goes on, he will become substantially worse.
This also bodes incredibly poorly for the next generation, who will mostly in their formative years now avoid writing code and thus fail to even develop a idea of what good code is, how it works/why it works, why you make certain decisions, and not others, etc. and ultimately you will see them become utterly dependent on AI, unable to make progress without it.
IMO outsourcing thinking is going to have incredibly negative consequences for the world at large.
Is coding like piloting, where pilots need a certain number of hours of "flight time" to gain skills, and then a certain number of additional hours each year to maintain their skills? Do developers need to schedule in a certain number of "manually written lines of code" every year?
Read your blog post and agree with some of it. Largely I agree with the premise that the 2nd and 3rd order effects of this technology will be more impactful than the 1st order “I was able to code this app I wouldn’t have otherwise even attempted to”. But they are so hard to predict!
Thanks, this rings true to me. The struggle is an investment, and it pays off in good judgement and taste. The same goes for individual codebases too. When I see some weird bug and can immediately guess what’s going wrong and why, that’s my time spent in that codebase paying off. I guess LLM-ing a feature is the inverse, incurring some kind of cognitive debt.
great article and many great points here
Feels like a combination of writing very detailed task descriptions and reviewing junior devs. It's horrible. I very much hope this won't be my job.
You let AI write your task descriptions. Try plan mode on CC.
Probably won't be if you don't get good at it.
I coded up a crossword puzzle game using agentic dev this weekend. Claude and Codex/GPT. Had to seriously babysit and rewrite much of it, though, sure, I found it “cool” what it could do.
Writing code in many cases is faster to me than writing English (that is how PLs are designed, btw!) LLM/agentic is very “neat” but still a toy to the professional, I would say. I doubt reports like this one. For those of us building real world products with shelf-lives (Is Andrej representative of this archetype?), I just don’t see the value-add touted out there. I’d love to be proven wrong. But writing code (in code, not English), to me and many others, is still faster than reading/proving it.
I think there’s a combination of fetishizing and Stockholm syndroming going on in these enthusiastic self-reports. PMW.
>Writing code in many cases is faster to me than writing English
True, I feel as though i'd have to become Stienbeck to get it to do what i "really" wanted, with all the true nuance.
> I am bracing for 2026 as the year of the slopacolypse across all of github, substack, arxiv, X/instagram, and generally all digital media.
2026 is just when it picks up - it'll get exponentially worse.
I think 2026 is the year of Business Analysts who were unable to code. Now CC et all are good enough that they can realize the vision as long as one knows exactly the requirements (software design not that important). Programmers who didn't know business could get by so far. Not anymore, because with these tools, the guy who knows business can now code fairly well.
"I think 2026 is the year of Business Analysts who were unable to code." This is interesting - I have seen far more BAs losing jobs as a result of the 'work' they did being replaced by tools (both AI and AI-generated). I logically see the connection from AI tools giving BAs far more direct ability to produce something, but I don't see it actually happening. It is possible it is too early in the AI curve for the quality of a BA built product to be sufficient. CC and Opus45 are relatively new.
It could also be BAs being lazy and not jumping ahead of the train that is coming towards them. It feels like in this race the engineer who is willing to learn business will still have an advantage over the business person who learns tech. At least for a little while.
Agree here, the code barrier (creating software) was hiding the real mountain: creating software business. The two are very different beasts.
with these tools, the guy who knows business can now code fairly well.
... until CC doesn't get it quite right and the guy who knows business doesn't know code.
The future of the programmer profession: This AI-generated mess of a codebase does 80% of what I want. Now fix the last 20%, should be easy, right?
Apart from the "AI-generated mess" part, that's too often been the past of the programmer profession, too.
> if you have any code you actually care about I would watch them like a hawk, in a nice large IDE on the side.
This is about where I'm at. I love pure claude code for code I don't care about, but for anything I'm working on with other people I need to audit the results - which I much prefer to do in an IDE.
> The most common category is that the models make wrong assumptions on your behalf and just run along with them without checking. They also don't manage their confusion, they don't seek clarifications, they don't surface inconsistencies, they don't present tradeoffs, they don't push back when they should, and they are still a little too sycophantic.
Does this not undercut everything going on here. Like, what?
It's predictable so you run defense around it with prompting, validation and model tuning. It generates volumes of working code in seconds from natural language prompts so it's extremely business efficient. We're talking about tools that generate correct code to 95% of a solution, the follow up human and automated test review, and second coding pass to fix the 5% are a non issue.
Oh wow! Guy who's current project depends on AI being good is talking about AI being good.
Interesting.
The way I have managed junior engineers is 90% via PR and testing, 10% via reading code in an editor or IDE.
It’s hard to let go of being the keyboard jockey, but in so many cases it is better to describe plans and acceptance criteria and just review the diffs.
Now that it's real, is there a minimum bar of non-AI-generated code that should be required in any production product? Like if 100% of the code is AI generated (or even doom-tabbed) and something goes wrong in prod, (crash, record corruption, data leak, whatever) then what? 99%? 50%? What's the bar where the risk starts outweighing the reward? When do we look around and say "maybe we should start slowing down before we do something that destroys our company"?
Granted it's not a one-size-fits-all problem, but I'm curious if any teams have started setting up additional concrete safeguards or processes to mitigate that specific threat. It feels like a ticking time bomb.
It almost begs the question, what even is the reward? A degradation of your engineering team's engineering fundamentals, in return for...are we actually shipping faster?
obviously you're not a devops eng, I think you're wildly under-estimating how much of business critical code pre-ai is completely orphaned anyway.
the people who wrote it were contractors long gone, or employees that have moved companies/departments/roles, or of projects that were long since wrapped up, or of people who got laid off, or the people who wrote it simply barely understood it in the first place and certainly don't remember what they were thinking back then now.
basically "what moron wrote this insane mess... oh me" is the default state of production code anyway. there's really no quality bar already.
I am a devops engineer and understand your point. But there's a huge difference: legacy code doesn't change. Yeah occasionally something weird will happen and you've got to dig into it, but it's pretty rare, and usually something like an expired certificate, not a logic bug.
What we're entering, if this comes to fruition, is a whole new era where massive amounts of code changes that engineers are vaguely familiar with are going to be deployed at a much faster pace than anything we've ever seen before. That's a whole different ballgame than the management of a few legacy services.
after a decade of follow-the-sun deployments by php contractors from vietnam to costa rica where our only qa was keeping an eye on the 500s graph, ai can't scare me.
That's actually a good comparison. Though even then, I imagine you at least have the ability to get on the phone and ask what they just did. Whereas LLM would just be like, "IDK, that was my twin brother. I'd ask him directly, but unfortunately he has been garbage collected. It was very sad. Would you like a cookie?"
I wonder if there's any value in some system that preserves the chat context of a coding agent and tags the commits with a reference to it, until the feature has been sufficiently battle tested. That way you can bring them back from the dead and interrogate them for insight if something goes wrong. Probably no more useful than just having a fresh agent look at the diff in most cases, but I can certainly imagine scenarios where it's like "Oh, duh, I meant to do X but looks like I accidentally did Y instead! Here's a fix." way faster than figuring it out from scratch. Especially if that whole process can be automated and fast, worst case you just waste a few tokens.
I'm genuinely curious though if there's anything you learned from those experiences that could be applied to agent driven dev processes too.
it was basically a mindless loop, very prime for being agent driven:
I won't say that solves every case but definitely 90% of them.I think your point about preserving some amount of intent/context is good, but also like what are most of us doing with agents if not "loop on error message until it goes away".
I don't see the AI capacity jump in the recent months at all. For me it's more the opposite, CC works worse than a few months ago. Keeps forgetting the rules from CLAUDE.md, hallucinates function calls, generates tons of over-verbose plans, generates overengineered code. Where I find it a clear net-positive is pure frontend code (HTML + Tailwind), it's spaghetti but since it's just visualization, it's OK.
> Where I find it a clear net-positive is pure frontend code (HTML + Tailwind), it's spaghetti but since it's just visualization, it's OK.
This makes it sound like we're back in the days of FrontPage/Dreamweaver WYSIWYG. Goodness.
Hmm, your comment gave me the idea that maybe we should invent "What You Describe Is What You Get|. To replace HTML+Tailwind spaghetti with prompts generating it.
Sad to hear this attitude towards front-end code. Front-ends are so often already miswritten and full of accessibility pitfalls and I feel like LLMs are gonna dramatically magnify this problem :(
Are you using Opus 4.5? Sounds more like Sonnet.
Yes I'm using Sonnet 4.5. Thanks for the tip, will try Opus 4.5, although costs might become an issue.
> although costs might become an issue.
If you have a ChatGPT subscription, try Codex with GPT-5.2-High or 5.2-codex High? In my experience, while being much slower, it produces far better results than Opus and seems even more aggressively subsidized (more generous rate limits).
The AGI vibes with Claude Code are real, but the micromanagement tax is heavy. I spend most of my time babysitting agents.
I expect interviews will evolve into "build project X with an LLM while we watch" and audit of agent specs
I've been doing vibe code interviews for nearly a year now. Most people are surprisingly bad with AI tools. We specifically ask them to bring their preferred tool, yet 20–30% still just copy-paste code from ChatGPT.
fun stats: corelation is real, people who were good at vibe code, also had offer(s) with other companies that didn't run vibe code interviews.
Interesting you say that, feels like when people were too stupid to google things and "googling something" was a skill that some had and others didn't.
Copy pasting from chatgpt is the most secure option.
Not going from home is the most secure way of going out.
It doesn’t work you can’t be productive without agent capable of doing queries to db etc
>Not going from home is the most secure way of going out.
What? I can't parse this sentence. Maybe get an ai to rewrite it?
Also the method that will result in the higher quality codebase.
Sounds great to me. Leetcode is outdated and heavily abused by people who share the questions ahead of time in various forums and chats.
From what I've heard, what few interviews there are for software engineers these days, they do have you use models and see how quickly you can build things.
The interviews I’ve given have asked about how control for AI slop without hurting your colleagues feelings. Anyone can prompt and build, the harder part, as usual for business, is knowing how and when to say, ‘no.’
The best thing I ever told Claude to do was "Swear profusely when discussing code and code changes". Probably says more about me than Claude, but it makes me snicker.
> LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.
Who doesn't like building? Building without any thought is literally a toy, like Lego or paint by numbers. That's the entire reason those things are popular. But a game is not a job. Sometimes I feel like half the people in this career are children. Never had any real responsibility. "Oh, everyone writes bugs, who tf cares". "Move fast, break stuff" was literally and unironically the tag line for a company that should have been taking far more responsibility.
This trend isn't limited to programmers either. Wherever I look I see people not taking responsibility. Lots of children in adult bodies. I do hope there are some adults who are really pulling the strings somewhere...
> What happens to the "10X engineer" - the ratio of productivity between the mean and the max engineer? It's quite possible that this grows a lot.
No doubt that good engineers will know when and how to leverage the tool, both for coding and improving processes (design-to-code, requirement collection, task tracking, basic code reviewal, etc) improving their own productivity and of those around them.
Motivated individuals will also leverage these tools to learn more and faster.
And yes, of course it's not the only tool one should use, of course there's still value in talking with proper human experts to learn from, etc, but 90% of the time you're looking for info the LLM will dig it from you reading at the source code of e.g. Postgres and its test rather than asking on chats/stack overflow.
This is a trasformative technology that will make great engineers even stronger, but it will weed out those who were merely valued for their very basic capability of churning something but never cared neither about engineering nor coding, which is 90% of our industry.
> - How much of society is bottlenecked by digital knowledge work?
Any qualified guesses?
I'm not convinced more traders on wall street will allocate capital more effectively leading to economic growth.
Will more programmers grow the economy? Or should we get real jobs ;)
Most of this countries challenges are strictly political. The pittance of work software can contribute is most likely negligible or destructive (e.g. software buttons in cars or palantir). In other words were picked all the low hanging fruit and all that left is to hang ourselves.
I actually disagree. Having software (AI) that can cut through the technological stuff faster will make people more aware of political problems.
edit: country's* all that is left*
Great point about expansion vs speedup. I now have time to build custom tools, implement more features, try out different API designs, get 100% test coverage.. I can deliver more quickly, but can also deliver more overall.
>Tenacity
I've seen the exact opposite with Claude. It literally ditched my request mid-analysis when doing a root cause analysis. It decided I was tired of the service failing and then gave me some restart commands to 'just get it working'
>Generation (writing code) and discrimination (reading code) are different capabilities in the brain. Largely due to all the little mostly syntactic details involved in programming, you can review code just fine even if you struggle to write it.
If this is how all juniors are learning nowadays, seniors are going shot up in value in the next decade.
I do feel a big mood shift after late November. I switched to using Cursor and Gemini primarily and it was big change in my ability to get my ideas into code effectively. The Cursor interface for one got to a place that I really like and enjoy using, but its probably more that the results from the agents themselves are less frustrating. I can deal with the output more now.
I'm still a little iffy on the agent swarm idea. I think I will need to see it in action in an interface that works for me. To me it feels like we are anthropomorphizing agents too much, and that results in this idea that we can put agents into roles and them combine them into useful teams. I can't help seeing all agents as the same automatons and I have trouble understanding why giving an agent with different guideliens to follow, and then having them follow along another agent would give me better results than just fixing the context in the first place. Either that or just working more on the code pipeline to spot issues early on - all the stuff we already test for.
The whole thing is about getting rid of experts and let the entry level idiots do all the work. The coders become expendable. And people do not see the chasm staring back at them :D. LLMs in their current form redistributes "intelligence" and expertise to the average joes for mere pennies. It should be much much more expensive, or it will disrupt the whole ecosystem. If it becomes even more intelligent it must be bludgeoned to death a.k.a. regulated like hell, otherwise the ensuing disruption will kill the job market and in the long term human values.
As an added plus: those, who already have wealth will benefit the most, instead of the masses. Since the distribution and dissemination of new projects is at the same level as before, meaning you would need a lot of money. So no matter how clever you are with an llm, if you don't have the means to distribute it you will be left in the dirt.
I am developing a web application for a dictionary that translates words from the national language into the local dialect.
Vibe coding and other tools, such as Google Vision, helped me download images published online, compile a PDF, perform OCR (Tesseract and Google Vision), and save everything in text format.
The OCR process was satisfactory for a first draft, but the text file has a lot of errors, as you'd expect when the dictionary has about 30,000 entries: Diacritical marks disappear, along with typographical marks and dashes, lines are moved up and down, and parts of speech (POS) are written in so many different ways due to errors that it is necessary to identify the wrong POS's one by one.
If the reasoning abilities of LLM-derived coding agents were as advanced as some claim, it would be possible for the LLM to derive the rules that must be applied to the entire dictionary from a sufficiently large set of “gold standard” examples.
If only that were the case. Every general rule applied creates other errors that propagate throughout the text, so that for every problem partially solved, two more emerge. What is evident to me is not clear to the LLM, in the sense that it is simple for me, albeit long and tedious, to do the editing work manually.
To give an example, if trans.v. (for example) indicates a transitive verb, it is clear to me that .trans.v. is a typographical error. I can tell the coding tool (I used Gemini, Claude, and Codex, with Codex being the best) that, given a standard POS, if there is a “.” before it, it must be deleted because it is a typo. The generalization that comes easily to me but not to the coding agent is that if not one but two periods precede the POS, it means there are two typos, not to delete just one of the two dots.
This means that almost all rules have to be specified, whereas I expected the coding agent to generalize from the gigantic corpus on which it was trained (it should “understand” what the POS are, typical typos, the language in which the dictionary is written, etc.).
The transition from text to json to webapp is almost miraculous, but what is still missing from the mix is human-level reasoning and common sense (in part, I still believe that coding agents are fantastic, to be clear).
Honestly, how long do you guys think we have left as SWEs with high pay? Like the SWE job will still exist, but with a much lower technical barrier of entry, it strikes me that the pay is going to decrease a lot. Obviously BigCo codebases are extremely complex, more than Claude Code can handle right now, but I'd say there's definitely a timer running here. The big question for my life personally is whether I can reach certain financial milestones before my earnings potential permanently decreases.
It's counterintuitive but something becoming easier doesn't necessarily mean it becomes cheap. Programming has arguably been the easiest engineering discipline to break into by sheer force of will for the past 20+ years, and the pay scales you see are adapted to that reality already.
Empowering people to do 10 times as much as they could before means they hit 100 times the roadblocks. Again, in a lot of ways we've already lived in that reality for the past many years. On a task-by-task basis programming today is already a lot easier than it was 20 years ago, and we just grew our desires and the amount of controls and process we apply. Problems arise faster than solutions. Growing our velocity means we're going to hit a lot more problems.
I'm not saying you're wrong, so much as saying, it's not the whole story and the only possibility. A lot of people today are kept out of programming just because they don't want to do that much on a computer all day, for instance. That isn't going to change. There's still going to be skills involved in being better than other people at getting the computers to do what you want.
Also on a long term basis we may find that while we can produce entry-level coders that are basically just proxies to the AI by the bucketful that it may become very difficult to advance in skills beyond that, and those who are already over the hurdle of having been forced to learn the hard way may end up with a very difficult to overcome moat around their skills, especially if the AIs plateau for any period of time. I am concerned that we are pulling up the ladder in a way the ladder has never been pulled up before.
I think the senior devs will be fine. They're like lawyers at this point - everyone is too scared they'll screw up and will keep them around
The juniors though will radically have to upskill. The standard junior dev portfolio can be replicated by claude code in like three prompts
The game has changed and I don't think all the players are ready to handle it
> like the SWE job will still exist, but with a much lower technical barrier of entry
its opposite, now in addition to all other skills, you need skill how to handle giant codebases of viobe-coded mess using AI.
Supply and demand. There will continue to be a need for engineers to manage these systems and get them to do the thing you actually want, to understand implications of design tradeoffs and help stakeholders weigh the pros and cons. Some people will be better at it than others. Companies will continue to pay high premiums for such people if their business depends on quality software.
I think to give yourself more context you should ask about the patterns that led to SWEs having such high pay in the last 10-15 years and why it is you expected it to stay that way.
I personally think the barrier is going to get higher, not lower. And we will be back expected to do more.
I think the pay is going to skyrocket for senior devs within a few years, as training juniors that can graduate past pure LLM usage becomes more and more difficult.
Day after day the global quality of software and learning resources will degrade as LLM grey goo consumes every single nook and cranny of the Internet. We will soon see the first signs of pure cargo cult design patterns, conventions and schemes that LLMs made up and then regurgitated. Only people who learned before LLMs became popular will know that they are not to be followed.
People who aren't learning to program without LLMs today are getting left behind.
Yeah, all of this. Plus companies have avoided hiring and training juniors for 3 or 4 years now (which is more related to interest rates than AI). Plus existing seniors who deskill themselves by outsourcing their brain to AI. Seniors who know actually what they're doing are going to be in greater demand.
That is assuming that LLMs plateau in capability, if they haven't already, which I think is highly likely.
> IDEs/agent swarms/fallability. Both the "no need for IDE anymore" hype and the "agent swarm" hype is imo too much for right now.
I'm honestly considering throwing away my JetBrains subscription and this is year 9 or 10 of me having one. I only open Zed and start yappin' at Claude Code. My employer doesn't even want me using ReSharper because some contractor ruined it for everyone else by auto running all code suggestions and checking them in blindly, making for really obnoxious code diffs and probably introducing countless bugs and issues.
Meanwhile tasks that I know would take any developers months, I can hand-craft with Claude in a few hours, with the same level of detail, but no endless weeks of working on things that'll be done SoonTM.
The tenacity part is definitely true. I told it to keep trying when it kept getting stuck trying to spin up an Amazon Fargate service. I could feel its pain, and wanted to help, but I wanted to see whether the LLM could free itself from the thorny and treacherous AWS documentation forest. After a few dozen attempts and probably 50 KWh of energy it finally got it working, I was impressed. I could have done it faster myself, but the tradeoff would have been much higher blood pressure. Instead I relaxed and watched youtube while the LLM did its work.
Are there good guides about how to write Agents or good repos with examples? Also, are there big differences between how you would write one in Codex cli vs Claude code? Can there be run on it interchangeably?
> How much of society is bottlenecked by digital knowledge work?
I think not much. The real society bottleneck is that a growing number of peeps try to convince each other that life and society are a zero sum game.
They are so much more if we don't do that.
I keep thinking about the TechnoCore from Dan Simmons' Hyperion, where the AIs were serving humans but secretly that was a parasitic relation, where they've been secretly using human brains as distributed processing nodes, essentially harvesting humanity's neural activity for their own computational needs without anyone's knowledge.
I know this is SF, but to me working with those LLMs feels more and more like that, and the atrophy part is real. Not that the model is literally using our brains as compute, but the relationship can become lopsided.
I prefer slopocalypse.
That works better.
“slopacolypse” does not make any sense both in writing and pronunciation.
not aslopalypse?
or even aislopalypse?
It's refreshing to see one of the top minds in AI converge on the same set of thoughts and frustrations as me.
For as fast as this is all moving, it's good to remember that most of us are actually a lot closer to the tip of the spear than we think.
Not sure how he is measuring, I'm still closer to about a 60% success rate. It's more like 20% is an acceptable one-shot, this goes to 60% acceptable with some iteration, but 40% either needs manual intervention to succeed or such significant iteration that manual is likely faster.
I can supervise maybe three agents in parallel before a task requiring significant hand-holding means I'm likely blocking an agent.
And the time an agent is 'restlessly working' on something in usually inversely correlated with the likelihood to succeed. Usually if it's going down a rabbit hole, the correct thing to do is to intervene and reorient it.
Right on especially on two things -- 1) the tools doing a disservice by not interviewing and seeking input and 2) The 2026 "Slopocalypse"
I'm hopeful that 2026 will be the year that the biggest adopters are forced to deal with the mass of product they've created that they don't fully understand, and a push for better tooling is the result.
Today's agentic tools are crude from a UX POV from where I am hoping they will end up.
I'm curious to see what effect this change has on leadership. For the last two years it's been "put everything you can into AI coding, or else!" with quotas and firings and whatever else. Now that AI is at the stage where it can actually output whole features with minimal handholding, is there going to be a Frankenstein moment where leadership realizes they now have a product whose codebase is running away from their engineering team's ability to support it? Does it change the calculus of what it means to be underinvested vs overinvested in AI, and what are the implications?
Am working on an iPhone app and impressed with how well Claude is able to generate decent/working code with prompts in plain English. I don’t have previous experience in building apps or swift but have a C++ background. Working in smaller chunks and incrementally adding features rather than a large prompt for the whole app seems more practical, is easier to review and build confidence.
Adding/prompting features one by one, reviewing code and then testing the resulting binary feels like the new programming workflow
Prompt/REview/Test - PRET.
Claude is good at writing code, not so good at reasoning, and I would never trust or deploy to production something solely written by Claude.
GPT-5.2 is not as good for coding, but much better at thinking and finding bugs, inconsistencies and edge cases.
The only decent way I found to use AI agents is by doing multiple steps between Claude and GPT, asking GPT to review every step of every plan and every single code change from Claude, and manually reviewing and tweaking questions and responses both way, until all the parties, including myself, agree. I also sometimes introduce other models like Qwen and K2 in the mix, for a different perspective.
And gosh, by doing so you immediately realize how dumb, unreliable and dangerous code generated by Claude alone is.
It's a slow and expensive process and at the end of the day, it doesn't save me time at all. But, perhaps counterintuitively, it gives me more confidence in the end result. The code is guaranteed to have tons of tests and assurance for edge cases that I may not have thought about.
Finally, literate programming!
https://en.wikipedia.org/wiki/Literate_programming
Why am I not surprised that a blog was written about LLM coding going from 20% to 80% useful, yet all of the HN comments are still nit picking about some negative details rather than building positive ideas toward some progress...
Is the programmer ego really this fragile? At least luddites had an ideological reasoning, whereas here we just seem to have emotional reflexes.
It's because we see a bunch of people completely ignoring the missing 20% and flooding the world with complete slop. The push back is required to keep us sane, we need people reminding others that it's not at 100% yet even if it sometimes feels like it.
Then you have Anthropic that states on his own blog that engineers fully delegate to claude code only from 0 to 20% https://www.anthropic.com/research/how-ai-is-transforming-wo...
The fact that people keep pushing figures like 80% is total bs to me
It’s usually people doing side projects or non-programmers who can’t tell the code is slop. None of these vibe coding evangelists ever shares the code they’re so amazed by, even though by their own logic anyone should be able to generate the same code with AI.
This kind of thought policing is getting to be exhausting. Perhaps we need a different kind of push back.
Do you know what my use case is? Do you know what kind of success rate I would actually achieve right now? Please show me where my missing 20% resides.
Thought policing, lol. People are just sharing their perspectives, no need to take it personally. Glad it's working well for you.
I used CC in year age and it was not good. But one month ago I paid for max and started to rebuild my company web shop using it.
It is like plowing land with hand one year age and now is like I'm in brend new John Deere. It's amazing.
Of course its not perfect but if you understand code and problem it needs to solve then it works really good.
What particular setups are getting folks these sorts of results? If there’s a way I could avoid all the babysitting I have to do with AI tools that would be welcome
> If there’s a way I could avoid all the babysitting I have to do with AI tools that would be welcome
OP mentions that they are actually doing the “babysitting”
i use codex cli. work on giving it useful skills. work on the other instruction files. take Karpathy tips around testing and declarativeness
use many simultaneously, and bounce between them to unblock them as needed
build good tools and tests. you will soon learn all the things you did manually -- script them all
Is it really brain atrophy if I never learned to code in ASM in my entire career as compiler has been doing that for me?
A part of me really want to say yes and wear it as a badge to have been coding before LLMs were a thing, but at the same time, it's not unprecedented.
Is it muscle atrophy if you were a weakling since birth? Is it retina degeneration if you were born blind? No, because atrophy is a loss of a prior strength, and not an ever–existing weakness, but it's just as bad.
The thing is the compiler does exactly what you want it to 99.999…% of the time so you never have to drop down into ASM
That’s not really true in this case
I think a person with zero coding knowledge would have a lot tougher time using these tools successfully
So I'm curious, whats the actual quality control.
Like, do these guys actually dog food real user experience, or are they all admins with the fast lane to the real model while everyone outside the org has to go through the 10 layers of model sheding, caching and other means and methods of saving money.
We all know these models are expensive as fuck to run and these companies are degrading service, A+B testing, and the rest. Do they actually ponder these things directly?
Just always seems like people are on drugs when they talk about the capabilities, and like, the drugs could be pure shit (good) or ditch weed, and we call just act like the pipeline for drugs is a consistent thing but it's really not, not at this stage where they're all burning cash through infrastructure. Definitely, like drug dealers, you know they're cutting the good stuff with low cost cached gibberish.
> Definitely, like drug dealers, you know they're cutting the good stuff with low cost cached gibberish.
Can confirm. My partner's chatGPT wouldnt return anything useful for her given a specific query involving web use, while i got the desired result sitting side by side. She contacted support and they said nothing they can do about it, her account is in an A/B test group without some features removed. I imagine this saves them considerable resources despite still billing customers for them.
how much this is occurring is anyones guess
If you access a model through an openrouter provider it might be quantized (akin to being "cut with trash"), but when you go directly to Anthropic or OpenAI you are getting access to the same APIs as everyone else. Even top-brass folks within Microsoft use Anthropic and OpenAI proper (not worth the red-tape trouble to go directly through Azure). Also, the creator and maintainer of Claude, Boris Cherny, was a bit of an oddball but one of the comparatively nicer people at Anthropic, and he indicated he primarily uses the same Anthropic APIs as everyone else (which makes sense from a product development perspective).
The underlying models are all actually really undifferentiated under the covers except for the post-training and base prompts. If you eliminate the base prompts the models behave near identically.
A conspiracy would be a helluva lot more interesting and fun, but I've spoken to these folks firsthand and it seems they already have enough challenges keeping the beast running.
A big wow moment coming up is going to be GPT 5.* in Codex with Cerebras doing inference. The inference speed is going to be a big unlock, because many tasks are intrinsically serial.
It's going to feel literally like playing God, where you type in what you want and it happens ~instantly.
When?
I don't know when but I'm going off:
- "OpenAI is partnering with Cerebras to add 750MW of ultra low-latency AI compute"
- Sam Altman saying that users want faster inference more than lower cost in his interview.
- My understanding that many tasks are serial in nature.
Speed is really important to me but also I would like higher weekly limits -- which means lower cost I suppose. Building out complex projects can take 6 months to a year on a Pro plan.
Same experience with Pro.
My trick is to attach the codebase as a txt file to 5-10 different GPT 5.2 Thinking chats, paste in the specs, and then get hard work done there, then just copy paste the final task list into codex to lower codex usage.
Thank you for the really excellent summation. I echo your thought 1 to 1. I have found it more difficult to learn new languages or coding skills, because I am no longer forced to go through the painful slow grind of learning.
Painful slow grind? I have always found the learning part what I enjoy most about programming. I don't intend to outsource that a chatbot.
Does one ever still need to learn new languages or coding skills if an AI will be able to do it?
This question makes me unbelievably sad. Why should anyone learn anything?
I'm not disagreeing.
Probably not. But as someone who has learned a few languages, having to outsource a conversation to a machine will never not feel incredibly lame.
I doubt most people feel the same, though.
> 80% agent coding
A lot of these things sound cool but sometimes I'm curious what they're actually building
Like, is their bottleneck creativity now then? Are they building naything interedting or using agents to build... things that don't appeal to me, anyway?
I guess it depends what appeal to you.
As an example finding myself in a similar 80% situation, over the last few months I built
- a personal website with my projects and poems
- an app to rework recipes in a format I like from any source (text, video,...)
- a 3d visual version of a project my nephew did for work
- a gym class finder in my area with filters the websites don't provide
- a football data game
- working on a saas for work so typical saas stuff
I was never that productive on personal projects, so this is great for me.
Also the coding part of these projects was not very appealing to me, only the output, so it fits well with AI using.
In the meanwhile I did Advent of Code as usual for the fun of code. Different objectives.
"you can review code just fine even if you struggle to write it."
Well, merely approving code takes no skill at all.
Seriously, that’s a completely nonsense line.
> It hurts the ego a bit but the power to operate over software in large "code actions" is just too net useful
It does hurt, that's why all programmers now need an entrepreneurial mindset... you become if you use your skills + new AI power to build a business.
That is motivational content, but not economics. Most startups will be noise, even more so than before. The value of being a founder ceases when everyone is a founder, when it becomes universal. You will need customers. Nobody wants to buy re-invented-the-wheel-74.0. It lacks character, it lacks soul. Without it, your product will be nothing but noise in a noisy world.
Cope. If you create something that genuinely solves a problem, people will buy no matter what.
Look entrepreneurship has never been easy. In fact it's always been one of the hardest thing ever. I'm just saying... *you don't have to do it*. Do whatever you want lol
Happy to hear what's your solution to avoid becoming totally replaceable and obsolete.
What about the people who dont want to be entrepreneurs?
They have to pivot to something else
Or stay ahead of the curve as long as possible, e.g. work on the loop/ralphing
permanent underclass...
It’s a great and insightful review—not over-hyping the coding agent, and not underestimating it either. It acknowledges both its usefulness and its limitations. Embracing it and growing with it is how I see it too.
It's been a bit like the boiling frog analogy for me
I started by copy pasting more and more stuff in chatgpt. Then using more and more in-IDE prompting, then more and more agent tools (Claude etc). And suddenly I realise I barely hand code anymore
For sure there's still a place for manual coding, especially schemas/queries or other fiddly things where a tiny mistake gets amplified, but the vast majority of "basic work" is now just prompting, and honestly the code quality is _better_ that it was before, all kinds of refactors I didn't think about or couldn't be bothered with have almost automatically
And people still call them stochastic parrots
I've had the opposite experience, it's been a long time listening to people going "It's really good now" before it developed to a permutation that was actually worth the time to use it.
ChatGPT 3.5/4 (2023-2024): The chat interface was verbose and clunky and it was just... wrong... like 70+% of the time. Not worth using.
CoPilot autocomplete and Gitlab Duo and Junie (late 2024-early 2025): Wayyy too aggressive at guessing exactly what I wasn't doing and hijacked my tab complete when pre-LLM type-tetris autocomplete was just more reliable.
Copilot Edit/early Cursor (early 2025): Ok, I can sort of see uses here but god is picking the right files all the time such a pain as it really means I need to have figured out what I wanted to do in such detail already that what was even the point? Also the models at that time just quickly descended into incoherency after like three prompts, if it went off track good luck ever correcting it.
Copilot Agent mode / Cursor (late 2025): Ok, great, if the scope is narrowly scoped, and I'm either going to write the tests for it or it's refactoring existing code it could do something. Like something mechanical like the library has a migration where we need to replace the use of methods A/B/C and replace them with a different combination of X/Y/Z. great, it can do that. Or like CRUD controller #341. I mean, sure, if my boss is going to pay for it, but not life changing.
Zed Agent mode / Cursor agent mode / Claude code (early 2026): Finally something where I can like describe the architecture and requirements of a feature, let it code, review that code, give it written instructions on how to clean it up / refactor / missing tests, and iterate.
But that was like 2 years of "really it's better and revolutionary now" before it actually got there. Now maybe in some languages or problem domains, it was useful for people earlier but I can understand people who don't care about "but it works now" when they're hearing it for the sixth time.
And I mean, what one hand gives the other takes away. I have a decent amount of new work dealing with MRs from my coworkers where they just grabbed the requirements from a stakeholder, shoved it into Claude or Cursor and it passed the existing tests and it's shipped without much understanding. When they wrote them themselves, they tested it more and were more prepared to support it in production...
I find myself even for small work, telling CC to fix it for me is better as it usually belongs to a thread of work, and then it understands the big picture better.
> And people still call them stochastic parrots
Both can be true. You're tapping into every line of code publicly available, and your day-to-day really isn't that unique. They're really good at this kind of work.
Minor nitpick: The original measure of a 10x programmer was not the productivity multiplier max/mean, but rather max/min.
maybe its just me doing stuff that's out the usual loop
even dealing with api's that have MCP servers the so called agents make a mess of everything.
my stuff is just regular data stuff - ingest data from x - transform it | make it real time - then pipe it to y
No idea why the poster wants to deprive this author of engagement on his post, but here's the original link: https://x.com/karpathy/status/2015883857489522876
> Coding workflow. Given the latest lift in LLM coding capability, like many others I rapidly went from about 80% manual+autocomplete coding and 20% agents in November to 80% agent coding and 20% edits+touchups in December
Anyone wondering what exactly is he actually building? What? Where?
> The mistakes have changed a lot - they are not simple syntax errors anymore, they are subtle conceptual errors that a slightly sloppy, hasty junior dev might do.
I would LOVE to have jsut syntax errors produced by LLMs, "subtle conceptual errors that a slightly sloppy, hasty junior dev might do." are neither subtle nor slightly sloppy, they actually are serious and harmful, and no junior devs have no experience to fix those.
> They will implement an inefficient, bloated, brittle construction over 1000 lines of code and it's up to you to be like "umm couldn't you just do this instead?"
Why just not hand write 100 loc with the help of an LLM for tests, documentation and some autocomplete instead of making it write 1000 loc and then clean it up? Also very difficult to do, 1000 lines is a lot.
> Tenacity. It's so interesting to watch an agent relentlessly work at something. They never get tired, they never get demoralized, they just keep going and trying things where a person would have given up long ago to fight another day.
It's a computer program running in the cloud, what exactly did he expected?
> Speedups. It's not clear how to measure the "speedup" of LLM assistance.
See above
> 2) I can approach code that I couldn't work on before because of knowledge/skill issue. So certainly it's speedup, but it's possibly a lot more an expansion.
mmm not sure, if you don't have domain knowledge you could have an initial stubb at the problem, what when you need to iterate over it? You don't if you don't have domain knowledge on your own
> Fun. I didn't anticipate that with agents programming feels more fun because a lot of the fill in the blanks drudgery is removed and what remains is the creative part.
No it's not fun, eg LLMs produce uninteresting uis, mostly bloated with react/html
> Atrophy. I've already noticed that I am slowly starting to atrophy my ability to write code manually.
My bet is that sooner or later he will get back to coding by hand for periods of time to avoid that, like many others, the damage overreliance on these tools bring is serious.
> Largely due to all the little mostly syntactic details involved in programming, you can review code just fine even if you struggle to write it.
No programming it's not "syntactic details" the practice of programming it's everything but "syntactic details", one should learn how to program not the language X or Y
> What happens to the "10X engineer" - the ratio of productivity between the mean and the max engineer? It's quite possible that this grows a lot.
Yet no measurable econimic effects so far
> Armed with LLMs, do generalists increasingly outperform specialists? LLMs are a lot better at fill in the blanks (the micro) than grand strategy (the macro).
Did people with a smartphone outperformed photographers?
Lots of very scared, angry developers in these comment sections recently...
Not angry nor scared, I value my hard skills a lot, I'm just wondering why people believe religiously everything AI related. Maybe I'm a bit sick with the excessive hype
FOMO really
There's no fear (a bit of anger I must admit). I suspect nearly all of the reaction against this comes from a similar place to where mine does:
All of the real world code I have had to review created by AI is buggy slop (often with subtle, but weird bugs that don't show up for a while). But on HN I'm told "this is because your co-workers don't know how to AI right!!!!" Then when someone who supposedly must be an expert in getting things done with AI posts, it's always big claims with hand-wavy explanations/evidence.
Then the comments section is littered with no effort comments like this.
Yet oddly whenever anyone asks "show me the thing you built?" Either it looks like every other half-working vibe coded CRUD app... or it doesn't exist/can't be shown.
If you tell me you have discovered a miracle tool, just some me the results. Not taking increasingly ridiculous claims at face value is not "fear". What I don't understand is where comments like yours come from? What makes you need this to be more than it is?
Also note that I'm a heavy LLM user, not anti ai for sure
This is extremely reductive and incredibly dismissive of everything they wrote above.
It's because they don't have a substantive response to it, so they resort to ad hominems.
I've worked extensively in the AI space, and believe that it is extremely useful, but these weird claims (even from people I respect a lot) that "something big and mysterious is happening, I just can't show you yet!" set of my alarms.
When sensible questions are met with ad hominems by supporters it further sets of alarm bells.
I see way more hype that is boosted by the moderators. The scared ones are the nepo babies who founded a vaporware AI company that will be bought by daddy or friends through a VC.
They have to maintain the hype until a somewhat credible exit appears and therefore lash out with boomer memes, FOMO, and the usual insane talking points like "there are builders and coders".
society doesn't take kindly to the hyper-aware. tone it down.
i'm not sure what kind of conspiracy you are hallucinating. do you think people have to "maintain the hype"? it is doing quite well organically.
So well that they're losing billions and OpenAI may go bankrupt this year
what if it doesn't?
better for them! the heck i care about it
This is a low quality curmudgeonly comment
Now that you contributed zero net to the discussion and learned a new word you can go out and play with toys! Good job
You learned a new adjective? If people move beyond "nice", "mean" and "curmudgeonly" they might even read Shakespeare instead of having an LLM producing a summary.
cool.
>Anyone wondering what exactly is he actually building? What? Where?
this is trivially answerable. it seems like they did not do even the slightest bit of research before asking question after question to seem smart and detailed.
I asked many question and you focused on only one, btw yes I did my research, and I know him because I followed almost every tutorial he has on YouTube, and he never mentions clearly what weekend project worked on to make him conclude with such claims. I had a very high respect of him if not that at some point started acting like the Jesus Christ of LLMs
its not clear why you asked that question if you knew the answer to it?
the xcancel link is amusing.
9/10 of the most important social media users use X, like or loath it
Stop whining
Are game developers vibe coding with agents?
It's such a visual and experiential thing that writing true success criteria it can iterate on seems like borderline impossible ahead of time.
I don't "vibe code" but when I use an LLM with a game I usually branch out into several experiments which I don't have to commit to. Thus, it just makes that iteration process go faster.
Or slower, when the LLM doesn't understand what I want, which is a bigger issue when you spawn experiments from scratch (and have given limited context around what you are about to do).
I'm trying it out with Godot for my little side projects. It can handle writing the GUI files for nodes and settings. The workflow is asking cursor to change something, I review the code changes, then load up the game in Godot to check out the changes. Works pretty well. I'm curious if any Unity or Unreal devs are using it since I'm sure its a similar experience.
It might be biased to Reddit/Twitter users but from what I've seen game developers seem to be much more averse towards using AI (even for coding) than other fields.
Which is curious since prototyping helps a lot in gamedev.
Vibe coding in Unreal Engine is of limited use. It obviously helps with C++, but so much of your time is doing things that are not C++. It hurts a lot that UE relies heavily on blueprints, if they were code you could just vibecode a lot of that.
A big problem is that a lot of game logic is done in visual scripting (e.g unreal blueprints) which AI tools have no idea about
> Atrophy. I've already noticed that I am slowly starting to atrophy my ability to write code manually.
I've been increasingly using LLM's to code for nearly two years now - and I can definitely notice my brain atrophy. It bothers me. Actually over the last few weeks I've been looking at a major update to a product in production & considered doing the edits manually - at least typing the code from the LLM & also being much more granular with my instructions (i.e. focus on one function at a time). I feel in some ways like my brain is turning into slop & I've been coding for at least 35 years... I feel validated by Karpathy.
Don't be too worried about it.
1. Manual coding may be less relevant (albeit ability to read code, interpret it and understand it will be more) in the future. Likely already is.
2. Any skill you don't practice becomes "weaker". Gonna give you an example. I play chess since my childhood, but sometimes I go months without playing it, even years. When I get back I start losing elo fast. If I was in the top 10% of chess.com, I drop to top 30% in the weeks after. But after few months I'm back at top 10%. Takeaway: your relative ability is more or less the same compared to other practitioners, you're simply rusty.
Thanks for your comment, it set me at ease. I know from experience that you're right on point 2. As for point one, I also tend to agree. AI is such a paradigm shift & rapid/massive change doesn't come without stress. I just need to stay cool about it all ;-)
>LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.
Quite insightful.
The section on IDEs/agent swarms/fallibility resonated a lot for me; I haven't gone quite as far as Karpathy in terms of power usage of Claude Code, but some of the shifts in mistakes (and reality vs. hype) analysis he shared seems spot on in my (caveat: more limited) experience.
> "IDEs/agent swarms/fallability. Both the "no need for IDE anymore" hype and the "agent swarm" hype is imo too much for right now. The models definitely still make mistakes and if you have any code you actually care about I would watch them like a hawk, in a nice large IDE on the side. The mistakes have changed a lot - they are not simple syntax errors anymore, they are subtle conceptual errors that a slightly sloppy, hasty junior dev might do. The most common category is that the models make wrong assumptions on your behalf and just run along with them without checking. They also don't manage their confusion, they don't seek clarifications, they don't surface inconsistencies, they don't present tradeoffs, they don't push back when they should, and they are still a little too sycophantic. Things get better in plan mode, but there is some need for a lightweight inline plan mode. They also really like to overcomplicate code and APIs, they bloat abstractions, they don't clean up dead code after themselves, etc. They will implement an inefficient, bloated, brittle construction over 1000 lines of code and it's up to you to be like "umm couldn't you just do this instead?" and they will be like "of course!" and immediately cut it down to 100 lines. They still sometimes change/remove comments and code they don't like or don't sufficiently understand as side effects, even if it is orthogonal to the task at hand. All of this happens despite a few simple attempts to fix it via instructions in CLAUDE . md. Despite all these issues, it is still a net huge improvement and it's very difficult to imagine going back to manual coding. TLDR everyone has their developing flow, my current is a small few CC sessions on the left in ghostty windows/tabs and an IDE on the right for viewing the code + manual edits."
Imagine taking career advice from people who will never need to be employed again in order to survive.
Yes, typically you take since from people who've been successful at their career. Are you suggesting we should be taking career advice from high school freshmen instead?
I'm nitpicking on the atrophy bit. He can afford to have his skills or his brain atrophied. His followers though?
Nevermind the fact he became successful _because_ of his skills and his brain.
tl;dr - All this AI stuff is just Universal Paperclips[1]
I see a lot of comments about folks being worried about going soft, getting brain rot, or losing the fun part of coding.
As far as I'm concerned this is a bigger (albeit kinda flakey) self-driving tractor. Yeah I'd be bored if I just stuck to my one little cabbage patch I'd been tilling by hand. But my new cabbage patch is now a megafarm. Subjectively, same level of effort.
[1]: https://en.wikipedia.org/wiki/Universal_Paperclips
xcancel? What is the purpose or benefit of providing a free mirror to x? Doesn't it end up sparing the x servers and causing their costs to decrease?
I prefer xcancel in part because Twitter doesn't let you view replies etc when not logged in.
Guessing x loses ad revenue when traffic goes to xcancel.
my screen is 60 percent banners about cookies and account creation when I use x
Basically mirrors my experience.
Interestingly, when you point out this ...
> IDEs/agent swarms/fallability. Both the "no need for IDE anymore" hype and the "agent swarm" hype is imo too much for right now. The models definitely still make mistakes and if you have any code you actually care about I would watch them like a hawk, in a nice large IDE on the side.
... here on HN [0] you get a bunch of people telling you to get with the times, grandpa.
Really makes me wonder: Who are these people and why are they doing that?
[0] https://news.ycombinator.com/item?id=46745039
At the risk of exposing my... atypical take on contemporary occupational "morality". LLMs, and Claude Code specifically, have given me the ability to work two jobs, and retain my soft-standing as the "guy" that gets stuff done/knows my stuff/can fix anything, while still working less hours than I did when I just had a single job.
I firmly believe, at SOME point, ML is going to eat my lunch. And I'd like to be well and retired off to a countryside homestead by then. Until such a time, I am going to use and abuse this technology as much as possible to gather 4 paychecks a month, optimize my investment portfolio to scale my NW, and by any means gain financial independence before the risk of my career vaporizing materializes. Sure, I could REALLY try and be one of those engineers that pulls a $750k salary and I wouldn't need to do this; but that isn't really in the cards for me. I know where I stand and I'm simply not smart or hardworking enough to get paid that much from a single job and guarantee my financial independence in the traditional way.
To that end, these tools have been extraordinarily impactful for me in a very simple and objectively positive way. And as much as I relate to basically everything OP mentioned, at the end of the day I simply DGAF. I want to make as much money as possible before the music stops, and this is the smart way to do that right now
> LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building
as the former, i've never felt _more ahead_ than now due to all of the latter succumbing to the llm hype
> Slopacolypse Really… REALLY not looking forward to getting this word spammed at me the next 6-12 months… even less so seeing the actual manifestation.
> TLDR This should be at the start?
I actually have been thinking of trying out ClaudeCode/OpenCode over this past week… can anyone provide experience, tips, tricks, ref docs?
My normal workflow is using Free-tier ChatGPT to help me interrogate or plan my solution/ approach or to understand some docs/syntax/best practice of which I’m not familiar. then doing the implementation myself.
Claude code official docs are quite nice - that's where I started.
Next milestone: solving authoritarian LLM dependencies. We can’t always get trapped in local minima. Or is that actually okay?
> The most common category is that the models make wrong assumptions on your behalf and just run along with them without checking.
If current LLMs are ever deployed in systems harboring the big red button, they WILL most definitely somehow press that button.
US MIC are already planning on integrating fucking Grok into military systems. No comment.
Including classified systems. What could possibly go wrong
the US is going to stop the chinese by mass production of illegal pornography?
fwiw, the same is true for humans. Which is why there's a whole lot of process and red tape around that button. We know how to manage risk. We can choose to do that for LLM usage, too.
If instead we believe in fantasies of a single all-knowing machine god that is 100% correct at all times, then... we really just have ourselves to blame. Might as well just have spammed that button by hand.
I don't know about you guys but most of the time it's spitting nonsense models in sqlalchemy and I have to constantly correct it to the point where I am back at writing the code myself. The bugs are just astonishing and I lose control of the codebase after some time to the point where reviewing the whole thing just takes a lot of time.
On the contrary if it was for a job in a public sector I would just let the LLM spit out some output and play stupid, since salary is very low.
> do generalists outperform specialists?
Depends what we mean by specialist. If it frontend vs backend then maybe. If it general dev vs some specialist scientific programmer or other field where a generalist won’t have a clue then this seems like a recipe for disaster (literal disasters included).
Senpai has taken the words out of my mouth and put them on the page.
[dead]
[dead]
[dead]
[flagged]
I don't know if it's fair to call him an ai addict or deduce that his ego is bruised. But I do wonder whether karpathy's agentic llm experiences are based on actual production code or pet projects. Based on a few videos I have seen of his, I am guessing it's the latter. Also, he is a research scientist (probably a great one), not a software developer. I agree with the op that karpathy should not be given much attention in this topic i.e llms for software development.
"addict"
Great idea! Le's pathalogize another thing! I love quickly othering whole concepts and putting them in my brain's "bad" box so I can feel superior.
https://github.com/karpathy/nanochat
https://github.com/karpathy/llm.c
The proof is in the pudding. Let's see your code
I don't agree with the parent commenters characterization of Karpathy, but these projects are just simple toy projects. They're educational material, not production level software.
You just proved the parent’s point.
He said “…who has never written any production software…” yet you show toy projects instead.
Well done.
[dead]
Once again, 80% of the comments here are from boomers.
HN used to be a proper place for people actually curious about technology
I'm almost a boomer and I agree. THis dichotomy is weird. I am retired EE and I love the ability to just have AI do whatever I want for me. I have it manage a 10 node proxmox cluster in my basement via ansible and terraform. I can finally do stuff I always wanted but had no time. I got sick of editing my kids sports videos for highlights in Davinci Resolve so just asked claude to write a simple app for me and then use all my random video cards in my boxes to render clips in parallel and so on. Tech is finally fun again when I do not have to dedicate days to understand some new framework. It does feel a little like late 1990's computing when everyone was making geocities webpages but those days were more fun. Now with local llms getting strong as well and speaking to my PC instead of typing it feels like SciFi, so yeah, I do not get this hacker news hand wringing about code craft.
So what is your workflow now with this app for kids sports highlights?
Well, it's not really a full-blown app yet. Claude wrote a plugin for MPV. So now when I watch video I just push a button to mark in and out of highlights similar to how it works in DaVinci Resolve. Then I have a command line tool that takes those timestamps in a video file and cuts it up into individual clips and then re-renders those clips and creates a highlight reel. Another command line tool takes three or four large MP4 files that the camera generates and downloads them and combines them in the actual game video on my desktop and also uploads it to my archive and transcodes into a bunch of different formats and uploads to YouTube. And for transcoding, again, it divvies it out to the video cards, which works pretty well. I think I have five or six encoders available so it chunks it up and then reassembles. All in all, it's nothing fancy, but it reduced quite a bit the friction of coming home after games and getting a video up on YouTube for grandparents.
Also interested
Same demographic, same experience. AI has been incredibly liberating for me. I get all sorts of things done now that before were previosly impossible for all practical purposes. Among other things, it cuts through the noise of all the layers of detail, and allows me to focus on ideas, design, and just getting stuff built asap.
I also don't get all the hand-wringing. AI is an amazing tool. Use it and be happy.
Even less do I get all the cope about it not being effective, or even useless at some level. When I read posts such as that, it feels like a different planet. Just not my experience at all.
Ya it's so weird lol
Instead of a 17 paragraph twitter post with a baffling TLDR at the end why not just record your screen and _demonstrate_ all of what you're describing?
Otherwise, I think you're incidentally right, your "ego" /is/ bruised, and you're looking for a way out by trying to prognosticate on the future of the technology. You're failing in two different ways.