We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong."
Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
This is all complicated by the fact that we're also losing our grasp on reality from the other direction because we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
Nor even €50k/year.
Two things are true:
1) The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to *write*.
2) My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
That last one, "given the time constraints", is a place where my experience is still useful. The AI is as lazy (or as optimised for fast wins) as the humans whose examples it was trained upon; but an LLM costs so little that the answer is always "do it right" rather than "do it fast". A lot of people don't know what "do it right" even looks like, having only ever known e.g. websites that take 5 seconds to load because of all the adverts and analytics, and never the world where machines with 1% of the CPU and no GPU at all could fit a fully playable first person shooter in the same memory footprint as that page.
At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Engineers have done amazing work of abstracting away the hard algorithms and data structures. In the meantime, there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms. In contrast, getting LLM to work is a new field, so it requires tons of new implementations: KV caches, speculative decoding, all kinds of variants of attention like FlashAttention, all kinds of parallel processing techniques, RL pipelines, post-training pipelines, and etc. It's just that the field is so concentrated that only luck few get to work on them.
So, maybe it's not LLM per se that removes the need of writing code. It is the maturity of the software engineering that has done so. It's just that LLM fills the last gap: making knowledge transfer so much faster and cheaper - if all that's left for most of us is slicing and dicing of what has been already been implemented, then LLM can reliably take over.
Before, if I would start working / thinking on a problem and discover it was harder than anticipated, that often was a signal that the feature may not be worth it. Because implementing it doesn't fit the current model of how things work or similar. Then it was back to the drawing table and find a better way of solving the underlying issue.
But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.
So exactly true! With ideas people unbridled, the painting-yourself-in-a-corner tendency that was always there has gone exponential, add to it the not-completely-aligned incentives, and the oh-so-clever impressive writer of long prose LLMs just end up churning. The real value add may end up being the lesson in honesty and humility to us all...
I first brought this up late last year – that there used to exist a kind of selection pressure against both developers and against features which would prevent certain functionality from ever being shipped, specifically:
- Functionality which exceed the technical ability or knowledge of the developer which built it; and
- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).
Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.
Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.
I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.
Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.
Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.
At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.
If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.
Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.
In my personal project I've found having a hard lines of code limit has been a good limit on complexity. Makes me really ask if I need things.
One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple
>My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
I feel like there is also the "Nerd Factor" to consider which is this unrelenting passion to type away at a screen all day, whether that is yesterday's code or tomorrow's prompt. Few non-technical people have the attention span to suffer such monotony. (Even most devs don't have that to varying extents)
I'm trying to write a novel, have been for a while now; one thing I hear from professional writers is that one should only do this if you are prepared to re-read whole thing six times before you even send it to the editor, and when you do send it to the editor, you and they will likely spend several months repeatedly re-reading the whole thing as you respond to a long list of changes they give you.
It's only monotone from someone looking, not from someone doing. If you look at digital painting tutorial, you'll see a lot of timelapse, because it's a very slow process where you spend the majority of time correcting stuff. But if you're the one doing it, each individual brush stroke is the result of a conscious decision.
Working on a passion project or working through the hairy details of a complex algorithm or whatever can be fun. But that's not really what the job is like most of the time.
All the boring parts I've encountered have soon been automated out of my workflow. There's the waiting part of some process, but that's why I have HN for.
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
That is certainly not true. LLMs can't write code worth a damn still, and you have to babysit them to make sure they aren't doing stupid stuff. A human is still by far the best choice for doing programming, and when trend-chasing companies realize the damage they've done to their businesses they will stop pushing LLMs so hard.
I think it depends on the - ahem - context. In my experience they usually can, sometimes they can't.
But generally their sense of software architecture remains abysmal, so even if their writing is ok, you need to have architectural authority the very least to guide them.
The fact that they sometimes can't means you can't have someone who is not a software engineer. Otherwise there is no way to verify whether the LLM wrote correct code or not.
I guess that somewhat describes me - and guess what, that also transfers to "should we even throw AI at that specific problem, or are there other solutions?". Early curiousity in figuring out the limits of LLMs now is turning into revenue by customers with problems that didn't exist a few years ago at all - which currently is quite an exciting field to work in as a lot of things just don't have pre-established solutions yet.
It's a similar level of excitement as when we started doing devops stuff before devops even was a thing (which has been one of our main things for a very long time now).
I have also tried developing games with them. I find them very weak at these things.
Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.
They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.
My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.
Works for a demo, but quickly stagnates. Future models may of course push the limits further, but right now it quickly goes sideways without someone knowledgeable at the helm.
You don't design things with never ending details though. If your developing a feature there's a limited scope to it. If your designing the direction of a large project you're working with higher level concepts.
The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.
> The point is that AI is aware of constraints and can manage a round it.
Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.
I have found it intriguingly difficult in my own tests to make LLMs deliberately fuck up grammar in any way similar to how humans do. I suppose this is actually an architectural limitation.
write a comment about how to use ChatGPT to write a game like you're a barely literate reddit user, short answer, maximum incoherency
and the result was a pretty good simulation. I tried less insulting prompts first, this did not produce a good simulation.
Not perfect, gptzero.me still knew it was AI generated (tayo42 is human by the same measure), but if for some reason someone was using an LLM prompted with that pattern, I suspect it would fool me in a Turing test unless I found the keyword to force the agent to change the role it was playing.
I'll take your word for it, I think I'm in zero groups on Facebook, I only even look at the feed when I see a notification in Messenger; and I barely use Messenger, too.
We have a product guy on the team who was in a deeply not technical role before AI who is trying to do the “hey Claude, read this Jira ticket, implement” thing.
It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
The spacer between the keyboard and the chair still matters in my experience.
Not everyone has this, but I always felt that a significant percentage of the value I bring is in immediately recognizing what you shouldn’t do.
I have a hard enough time explaining why “yet another bespoke application on some unmanaged azure resources” is a bad idea when they have more reasonable alternatives at hand.
Now these goofs can (very nearly) press a button and do it anyway, with no comprehension of the consequences. It’s high fives and pats on the back, until I’m cleaning up the mess.
This lands. Llms are bad precisely at following what not to do. They work best off of positive constraints.
I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps. I call it a bias field, pushes them toward hopefully the happy and harmonious (with the rest of the system) paths. Obviously this is only partial and imperfect enforcement, but if it's applied to everything consistently it naturally encodes some self-consistency and harmony.
> I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps.
I’ve done the same but it’s a moving target as models advance and I find half of my points are ignored until I’m prompting “No wtf why are you still trying to symlink the global Python executable just use the virtual environment that’s already activated”.
Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time. There’s no way my hobby fiddling will outpace whatever gets released in the next couple months.
In the meantime, real linting does work pretty well, if you can write a detector for whatever antipattern you find LLMs fall into (like multiline comments).
Oh I'm certain I'm not doing it optimally or close to best practices. It's just an example I had in my head related to my comment.
Although, given opus 4.8->5.0 experience within claude code, I don't know if I'd agree they necessarily improve things with time. But that's just my experience. I should probably read their release notes, maybe just a tweak to my claude.md would help
> Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time.
I get patches into the biggest AI projects all the time to make inference faster on my 3090 and all you have to do is read the contribution docs and open a small PR.
It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI.
You can trivially outpace what big companies are doing it’s insane, why do you think so many innovations are coming out of scrappy Chinese labs? They are willing to engage instead of being defeatist about it.
> It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI. You can trivially outpace what big companies are doing it’s insane
Not my core competence and I’m not interested in making it a core competence while the field is still rapidly changing. I find harness engineering interesting, but not lucrative enough to push one of the many very important other things on my plate to the backburner.
I don’t think Python language developers are super geniuses either, and I’ve never submitted code for consideration.
I've had some success with: here is this android bug ticket in a product that I don't know much about, fix it! And it works sometimes. I'm still technical, so it isn't the same, but it was crazy that AI could just solve some problems automagically without me know much about the system being modified.
> It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
Remove the dev from the equation. Suppose you need to produce circumstances for the slop machine to succeed on its own (imagine a loop).
What is missing? A good enough description of the current behavior, desired changes and how it should work? Understanding of the business domain or the system in question, its architectural choices? Enough guardrails and linters and tests to ensure that the solution will indeed be testable and work end to end? Cause a lot of the things that will help human devs will also help LLMs make their output more workable - for example ADRs in the repo and custom in-house tools to ensure they follow architectural conventions and practices (or just something like ArchUnit). Realistically, how close to a finished implementation must you get and how far backwards from that can you go before its output degrades too much?
I tried figuring that out myself on a personal project where I mostly want what I want in GitHub issues and then feed them to Kepler, which spawns OpenCode in worktrees and also gives me a UI to follow the execution. Most models, even SOTA, occasionally implement things wrong, but it's suprising how far you can take things when you've spent a dozen hours planning out what you want the end result to be vs not to be (talking about the goals for an entire system, maybe an hour of planning for a more bounded type of task).
In general, when I’ve worked on projects where I’ve spent that much time planning, having a magic AI box that wrote perfect code wouldn’t have helped that much. When you spec every everything out like that, writing the code is the most trivial part of this whole process.
For the completion to work, the source text needs to be ‘good’. That’s a basic kernel of how the thing works. Even with a perfect oracle autocomplete if the source text is ‘bullshit’, the output is too.
Or, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
fucking is it? AI labs certainly want you to think so but they're running out of time and money. they've been telling me that the advancements of the last three months have made this a reality for the last five years. it's starting to feel a lot like Elon promising fully automated taxi services by next year... 11 years ago
So far, the LLMs I've used need problems to be fairly specifically scoped, or they don't produce the correct solution. Scoping problems correctly is a different skill-set than implementation, but it's still a technical skill that is expected from mid-level and higher engineers.
Yep and I just ran a simple enterprisey "ambiguity" bench on the big three (US) model providers: same ambiguous initial-prompt with same clarifications and pushback prompt sequence afterwards.
The edge of correct/better when facing ambiguity is very fuzzy, all models from the past 6 month or so have similar random ways of spinning between too-literal avenues and oddly misplaced misled fixations. Taking the right initiatives in face of uncertainty is definitely AGI, and its not there, and perceptrons + attention layers just ain't got what it takes no matter how hard you push.
Probably worth not getting too comfortable. It's only a matter of time before a new generation of product people can do this. Also, I have said this before, instead of 1 product person and 10 engineers on a team, you may have 2-3 product+eng people and 3 engineers, etc.
Boris: "I don't prompt Claude anymore. I have loops prompting Claude and figuring what to do".
Boris: "I haven’t written a line of code by hand in, I think, eight months now… Claude Code, 100% written by Claude Code".
Boris: "There’s no manually written code anywhere at the company… All of the SQL is written by models. Everything is just built by the models... Claude instances communicate with each other (e.g., over Slack) in autonomous loops"
This does not sound like they review the code either. So, either the frontier labs like Anthropic have figured out something that very few companies could replicate, or they are being incredibly deceptive. I don't know which is true.
They work on research problems you can define a clear solution criteria for.
The kinds of business software I work on don't have those characteristics. If I needed something like a utils library, I think I could easily have Claude write the whole thing and not read the code.
Mythical man month should still apply for AI coding. If you look at the code you're shipping, and you can't quickly edit it by hand, or fix a bug with it, you will be in trouble if the models are down, and your website is down at the same unfortunate time.
1. I think LLMs will end up pretty dramatically shrinking MTTR, a lot of that will be tooling to proactively resolve problems as soon as they start, but a lot of it is that agents are very good at finding and fixing problems by correlating data, something humans can do but I think we're going to be slower at
2. "The models are down" doesn't seem like a very realistic problem unless you have a single provider/set of endpoints. I do not recommend this setup but I guess if you have a single point of failure that is a risk.
People are fundamentally short-term thinkers. Just imagine spraying neurotoxins on your lawn. You're "saved" from the "terror" of noticing more than one kind of grass, but of course you're incrementally increased the cancer risk for yourself, your children, and who knows who else. This is just one example; people do this constantly. Plastic fleece puts an untold amount of poison into the environment, but "I need to be warm RIGHT NOW." etc. Or lead paint. "It's only toxic when it breaks down, and that won't be for decades!"
All the issues you mention here have a death horizon: the people engaged in the behaviour will be dead when its effects are most acute. That they engage in it then isnt very mysterious.
> the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it
The word "engineering" means doing things using a repeatable process to get predictable results. If you can get predictable results (e.g. guaranteeing the absence of data integrity issues, security issues, anything that will cause downtime, etc.) without looking at the code, you're still doing the work of engineering.
Your job as an engineer is to choose high-value problems to work on, solve them in the correct way, and guarantee that the functional and non-functional requirements are met. If you no longer need to read the code to guarantee that it has the right set of ilities, then I'm not sure that's obviously a bad thing, as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
> as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
Perfect example of begging the question.
Vibe coding works great as long as vibe coding works great.
Personally, I'd love to see how you enforce security and long-term maintainability without looking at the code cuz we need to add that to our build pipelines ASAP.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
When you factor in overhead and benefits, many companies were regularly paying that much for someone (many someones) to "read this Jira ticket and implement the feature".
We are currently in the "centaur" phase where a human-AI combination produces the best output, but I think some people are betting on the fact that the AI only product will eventually outperform the centaur. And with the cost of tokens falling thanks to fierce competition from the Chinese open weight models, it's definitely possible that those that bet on AI early will reap payoffs in lower expenditures for more output.
Honestly not sure which side I land on that bet but I definitely can't rule it out.
We've gone through this with chess, and the days when a human can do anything to help AI there are long past; the human can only drag it down. It seems to me to be a form of the bitter lesson. I expect this will happen in every field where we don't add guardrails to require humans in the network. I have a hard time seeing regulation happening around this in the US but vanity may also serve, will CEOs be satisfied ruling armies of virtual assistants, or do they need big buildings full of human thralls?
This is not a silly question, because it's kind of silly:
Has an AI ever actually gone through all of the steps necessary to become a chess master? Go to competitions, raise in the ranks, take a plane to the masters, organize support and all the real life details necessary? Playing chess is just the small formal part here, and all of the obvious things humans obviously do, well...
> the days when a human can do anything to help AI there are long past; the human can only drag it down.
I disagree, currently a human guiding an AI agent is far better (more productive for less money) than an AI agent with very loose non-technical guidance and The Ralph Wiggum Loop.
I don't know how much longer humans have, but I don't think they can be cut out of the loop entirely just yet.
1. Execution of engineering task - Companies happen to focus on this mostly and this is the metric to measure easily. Using LLM tools gives a impression of improvement on this area which is what everyone is chasing
2. Growth of the Engineer - This is the one which was always a side product of company culture, individuals interest, work being done, time being spent to understand, learn from failures. A job being executed perfectly for the first time itself does not gives the opportunity for learning, no memories/experiences are built up mind of the person executing the job after a while.
The second part is the one which is under appreciated in current scheme of things since it was a by product. There is a concept of muscle memory which pretty much applies to everything.
My job improved significantly since I managed to get into their (managers) thick skulls to tell Claude to not be verbose. I'm tired of 1200-word tickets for "put the totals in the automated emails".
It's like a tool that has caused a fan out of both good and bad outcomes.
Good: you can refactor your codebase at will, throw out legacy cruft by the megabyte, improve build/CI time, and simplify ruthlessly. Not to mention kick out new features in simple and consistent ways that align with what a user actually wants.
Or you could add megabytes of vibe coded crap, solutions that add a ton of mass but don't actually solve the problem at hand (seriously!), implement abstractions that are logically inconsistent with the rest of the system, etc.
Both of these are happening right now, and I think the latter is happening at a rate far higher than the former. But our fundamental dynamics are still at play - the ball of mud is still a ball of mud, even if AI lets you make it 100x bigger. The problem just gets more entrenched.
Eventually AI will learn how to simplify code, understand coupling, etc - and hopefully it will just iron out problems as it goes. I think we're a long way away from that. But this is uncharted territory, and I don't think anyone really knows. I certainly don't hear anyone focusing on that as a target of optimization however.
My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue, and perhaps we can re-learn these lessons yet again. It kind of feels like GitHub might be the first example of this.
> My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue
If you worked in SV like I did from 2010-2020 you know that the exact opposite of that will happen. I saw hundreds of companies successfully scale out of their garbage stack such as Facebook and PHP.
The only thing that matters is the problem you’re solving and the quality of the code is irrelevant.
> A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
I had a recent experience with this. Although it was months and not weeks and there were other factors into play such as the market etc but the last company i was in had a poor engineering culture with inexperienced engineers equipped with AI output some of the lowest quality features for both customer-facing products and internal dev tools. A lot of customers churned, and im sure quality us part of the reason.
That led to a reduction in knowledge of assembly in the average programmer but the people who specialize in it haven't gotten any worse at it.
The result was a generation of programmers who make useful software while very few of them understand the machine they program. You could easily make either a positive or negative value judgement about that result.
> We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
Our jobs have never been to write code. We’ve been saying for decades that LoC isn’t a rational way to measure engineering output, and now we have our chance to structurally change that system before processes re-solidify. In fact, I think the flexibility to adopt new systems and the experience and foresight to choose a path that is better than the status quo without throwing out everything we’ve learned is going to be what sets companies apart and makes individual careers over the next few years.
> we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
There’s an important criticism here IMO - the relationship between “business” and “engineering” is changing drastically. It’s going to be a challenge to set the expectation that just because Marketing was able to vibe-code a prototype in a day, actual implementation may well take weeks or months. Engineering should be considering things like security, scalability, and systems integration that aren’t a concern for Marketing - that’s why we’re being paid!
> So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
I think it’s an extremely heterogenous landscape right now. Where I work I’m struggling mostly with organization - keeping the (literal) dozens of inbound features that come in every day straight long enough to hook up an agent harness, review, validate, and deploy. At other companies the issues seem to revolve around their Agile-based processes. Or maybe it’s non-technical vibe-coders and their expectations. Or maybe it’s executive leadership flirting with AI psychosis.
Everything is in flux. It’s stressful and exciting, and I’m thankful to be around for it, even if I am in my 40s at this point and looking at the core skills I’ve built rapidly become worth exponentially less. It’s a huge opportunity for growth.
Engineers, free from writing thousands of lines of boilerplate, now have more time to review code and be more efficient in that armed with LLMs. E.g. instead of asking a question about the code and waiting for the author to respond, I can ask an LLM and get the answer almost immediately. Or, if I see some strange code being changed, I am always curious why it was there in the first place and, instead of tracing it through git blame over formatting, renaming, and moving files around dirs I can delegate this to an LLM. If I have doubts about it working in some environment I can, again, make an LLM test it out instead of asking the author etc. etc.
And no, telling Claude to implement a Jira ticket is not worth $200K/year. Checking if it has not done something stupid and correcting it when it's trying to - is.
Alternate take: How many people have known how to program in assembly for the last 30 years or so? It's been a skill on the decline. Good compilers made that happen.
// The need for ongoing friction in long-term skill formation.
The subtitle of the story tells it all.
There are some people who seek out friction. Think about an athlete or a hardcore nerd.
The best engineers are ones who were fascinated with computers and learning as kids and persued it at every opportunity. Found their own friction in other words.
For those kinds of people, friction-seeking is the constant and what LLMs did is moved the point of where the friction occurs.
For example - the best engineers I've worked with didn't necessarily have lots of experience coding in assembly because that kind of friction was no longer necessary. But they could solve hard problems (and if a problem really required assembly they could go learn it)
What I think will be hit much harder by AI is the low tier engineer. Someone who was never truly curious and committed to it, for whom it was just a job. For example a typical offshore ticket pusher kind of person. That kind of person never went out to find friction and that's the kind of thing that's never going to fly again - if I want mediocre or average, the LLMs are sufficient
I think this expresses what I've been trying to form in my mind.
I've been trying to map the LLM advancements and the current state of software development onto prior technological improvements. History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
For the people that fall in love with a single abstraction layer or don't have an interest in learning new paradigms, when their known pattern is abstracted away, they're condemned to being left behind, either unwilling or unable to adapt.
I don't think any industry is free from this, any person in any industry/profession over a period of 20 years or more has likely had to undergo massive adjustments as technology changed their field.
We're not unique, but that doesn't stop it from feeling so jarring when it happens to us
> History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
But there's no abstraction layer that ends up getting lifted. When I use a library like SDL or a standard like POSIX, I don't tend to look at the underlying implementation. Instead I work with the high level concepts that they come up with. There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
I've learned something at every abstraction layer in computing from electronics (hardware), theory of computation (software) to high level programming languages with their paradigms. Same with several domains embodied by libraries. LLM tooling is more like shamanic ritual than engineering.
> There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
This is the same “but the ai can’t even do fingers” argument all over again, and it will age just a well.
The whole vibe coding thing only really started taking hold with the release of opus 4.5. That wasn’t even a year ago. you absolutely can be quality software without looking at the code. The only thing up for debate in my mind is how big that software can get before you need to look under the hood.
But that’s dictated by current model capabilities. And I don’t see any reason to believe that they will suddenly stop improving on this front.
Yeah, a bespoke program that does exactly what I need it to do, at a speed that I had forgotten was possible on computers, with customization that is an exact fit to me, at a cost that is smaller than a rounding error.
I get that LLMs struggle with the old paradigm of a single piece of software meant to serve every conceivable use case of every conceivable user, but I kinda hope that paradigm dies.
I think coding experts who are only experts at writing code will also be hit pretty hard. Every problem given to them is solved by writing code, and only code, their way and only their way. I think people like that are having a very hard time relinquishing control regardless of the quality and correctness of the LLM output. If it didn't come from them, or conform to their conventions and style, then it's wrong. These are the people that seem to be terrific developers but then when asked to be a team lead and bring a group of developers to their level completely fail.
Curiosity is really key, and I find that LLMs diminish curiosity in people that might otherwise have it because the allure of the answer is often a lot stronger than the allure of the friction. I guess that's just the distilling of the industry over time, but that could certainly lead to a shortage of expertise.
If that is representative of your personal experience, then I'm saddened to hear it. I believe your article is an argument from incredulity [1]. You may not personally know anyone who thrives in learning from and applying knowledge with AI. But being unaware that it is even possible is not evidence that expertise will inevitably collapse.
I mean if you think about it, that sounds like a miserable existence to begin with.
I would not rule out that some people will use this.. uh.. disruption, to start pursuing something that actually brings them joy.
Of course, bills need to be paid and it's not all that simple. But at least there might be such a silver lining.
The snake eating it's own tail for llm software development has really been met with a shoulder shrug whenever it gets brought up. At best you might have a small cohort of developers that don't cook their brains with AI and their reward for that appears to be having to review terrible AI code written by people who have cooked their brains.
As a tech educator I 100% agree. LLMs are not going to become a "new compiler" where we don't have to worry about the code any more. There's a reason we trust deterministic systems.
I've been worried about this a lot, I even created an agent skill called do-i-understand that's designed for novice devs (and experienced too, because atrophy) where the LLM asks you questions about the PR you're about to submit. I've found it helps a lot: https://github.com/AnthonyPAlicea/skills/blob/main/skills/do...
One way or another, there will be a skill reckoning.
It absolutely might make mistakes. But the skill requires all questions to be based on lines of code with specific line numbers which the human needs to review. That's intentional to serve as a safeguard against hallucinations.
There are 2 levels of determinism that matter here IMO. Level 1, given the same input you get the same output. Level 2, you know what to change on the input to get a specific change on the output.
LLMs are currently capable of level 1, but not capable of level 2. Trust follows what is actually deterministic.
they probably mean deterministic in the sense of traditional computer programs consisting of if-else decision points and ordinary cpu logic, as opposed to numerical models (which can of course be deterministic too, as you pointed out)
Then I think I'd rephrase you: There's a reason we trust trusted systems.
:)
Even their fans agree, LLMs are inherently unreliable. The fact some people trust them regardless is due to a deep flaw in human psychology that I believe has not been significantly exploited by any previous tech. There will be tears.
It believe the most likely consequence of AI is to promote generalists: people who have a domain expertise, can work cross discipline, and has enough programming knowledge to keep the LLMs on track. I don't think 'pure' software engineers will end up being as highly valued as the last decade though I also think that will be true for other disciplines as well. Just as an example, in signal processing, it's not uncommon to have a person designing the general algorithm and another person dedicated to implementing the algorithm in the embedded system. With the quality of coding agents, it's not really necessary to have both of those people anymore. A person who is moderately experienced in both can do the job now.
I strongly agree with the concept that cognitive friction is the engine of learning.
First and foremost, it's an issue of "dependency": if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself.
A historical example that brought about a similar shift is this: when the production process moved from the craftsman's mind and hands to the Fordist factory (and the assembly line), the skill of building things shifted from human craftsmanship to anonymous, structured processes.
Bit by bit, traditional artisans lost their knowledge and "know-how." Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
> cognitive friction is the engine of learning.
I agree 100% too.
This is why in order to understand new codebase or to ramp up to new projects, I use AI to generate a textbook style reading material for me along with "verify yourself" types of exercises along the way. Then I print them on a paper and read it using a pencil/pen and take notes.
Because of my math training, I am in the habit of slowing down to read textbook style texts which helps.
Exactly the type of usage I would hope to see advocated for in the future. As I wrote, when leveraged with Socratic methods and a sort of "interactive documentation", they can really boost one's understanding. Of course, this is predicated on the individual having enough information to know the right questions to ask.
I also do this. It would be better I think to generate exact references to primary and canonical sources to avoid learning LLMisms, or to feed it these and have it ensure its material is anchored to those good references.
> "...if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself."
For a bit of extra unease, consider that these people will retain their right to vote, both in making business decisions as employees, in shareholder meetings as shareholders, and in government elections as voters, despite their atrophied reasoning abilities. Who will wind up whispering what to vote for in the ears of these reverse centaurs?
Another example is autocorrect absolutely destroying the world's ability to spell even the most basic words. Ever since mobile phones came along and removed the friction of walking to the dictionary, everywhere I look people write as if they have extremely serious brain trauma - grown adults struggling to write as well as we used to as children!
> Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
Tangential, but this is one of many reasons that I (and I suspect many others here) have taken up wood working.
> Bainbridge argues that new, severe problems are caused by automating most of the work, while the human operator is responsible for tasks that can not be automated. Thus, operators will not practice skills as part of their ongoing work. Their work now also includes exhausting monitoring tasks. Thus, rather than needing less training, operators need to be trained more to be ready for the rare but crucial interventions
TBH it was already pretty bad. There is a stark difference between the best and the average in my experience. The top, say, ten percent of coders are vastly better than anyone else when it comes to anything but boilerplate glue code(which is still needed and is better done by average coders anyway).
This is speaking from my experience as a systems/c/c++ guy. If you are a js web frontend guy, python, or whatever I have no idea if this applies to you.
Writing code in programming languages is a skill/necessity created by us to instruct computers what we want them to do
Initially in the 60s, this was done by connecting circuits one way or another (think ENIAC). Then we devised "programmable" computers and devised a bunch of codes (computer code instructions) that abstracted away those cables.
The we created Programming Languages to further abstract away the hardware complexity, and to be able to write down our wishes in a way that is more transferable between us humans, but that is still computable by machines.
But with LLMs and neural networks, at some point these abstractions wont be necessary.
The computers will still be making computations, but the way we tell them what we want is going to evolve.
Another way to look at it is that the AI prompting is the new Programming Language or Compiler.
Like going from coding in Assembly to coding in Python; we are just writing the instructions in a simpler way.
And like going from Assembly to Python; the simpler way of writing the code makes us lose some of the deep control and efficiency of code execution that we would otherwise have; but, it comes at the benefit of being easier to create software and (hopefully) maintain that software.
Not to detract from your point, but there were software-programmable computers in the 50s. Check out Usagi Electric's videos about the Bendix G-15, where he loads software from punched tape.
This seems sadly very true.
Just yesterday I was implementing some slightly tricky javascript (not my main language) on hover show n neighbours to each side, and if a deficit on either side, expand to the other side. After about 20 minutes of struggling to get the offsets just right, I succumbed to just asking an agent to do it.
I'm sure I'd still be able to do it, but was saddened that I didn't get it as quickly as I think I used to be able to. Atrophy might already be in play.
I often think back to the early GPT days before agents as maybe the last time I will deeply learn a technical subject. I think that was when I actually learned the most as I had to stay completely in the loop. I had recently started a new job and had to work with k8s for the first time. Using GPT to help me implement new services and help me diagnose and fix issues with kubectl taught me so much.
Now with agents I don't have any insights into what its doing with kubectl and would have no reason to learn how it works (other than my own curiosity).
I agree that the human no longer gains expertise, but I am not sure that will matter in the long run.
The end result is a total and complete reliance on AI providers. It's not really a secret; that's why I started the article with the direct quote from Altman. They would love to see a world where nobody is able to do any type of development work without an LLM subscription enabled...even though they also admit that these models perform best when they are steered by someone who is highly capable and knowledgeable in the first place. They're powerful, but they are also VERY fallible.
Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.
While I agree Local AI would be a good goal, even the huge open weight models that can't be run locally are already thwarting the US AI lab's duopoly. Other companies can and do serve up these open weight models, and without having to amortize R&D they are able to serve the tokens at a pretty cheap rate.
In some ways, it feel like I'd be "stuck" at my 2024 capabilities if I didn't push myself constantly, and I'm really glad for every hour I spent learning/applying new stuff before that, because now it feels way harder.
Most computer science programs don't teach "coding" though. There is pretty heavy emphasis on datastructures, algorithms, and system design. I don't think AI atrophies those skills / knowledge as much as the coding language and I think in many cases actually improves them, especially if you are reading the code that is produced and at least understanding the flow of it.
I agree that AI reliance is hurting engineer's language understanding, but I don't know how much we should care.
I don't think that's quite correct; every CS program I've seen involves writing code, quite a lot of it. What they don't teach is /software engineering/: working large codebases, maintaining code, dealing with legacy code, long-term collaboration, working with product managed and designers, etc. That's stuff that you mostly end up having to pickup in the real world outside school.
I did B.S. in software engineering instead computer science in college, and there were classes that indeed taught the industry software engineering process like you mentioned.
> especially if you are reading the code that is produced and at least understanding the flow of it.
This is the key point. Should we stay vigilant and review all the code, ensuring we have a decept grasp of what it does. Or just green light everything and treat the code itself as a black box. I would hope we do the former, but many are pushing for the latter.
The issue with this in practice is that the rate at which code is produced is often substantially higher than what any human reviewer is capable of processing mentally.
Example from my workplace: pull requests that were already too long before LLMs by a few hundred or a couple of thousand lines are now several thousand lines longer and they get opened much more frequently. The primary source of these PRs is a well-respected senior team member with a ton more experience than everyone else by a measure of decades so nobody dares tell him to stop. Other team members aren’t far behind in the PR opening rate and admittedly I am one of them, but I try to mitigate the issue by doing multiple LLM reviews and anticipating when it’s ok to just approve it myself (have never had an issue in making this judgement thankfully). Part of this IMO is almost certainly because there’s now peer-pressure within the team to churn out a ton more code and that’s adding to the already clear message from leadership that we need to leverage AI to deliver things at unprecedented levels.
My point is that it’s not just that some people are pushing for greenlighting everything without due diligence, it’s also that they’re pushing others to toward that outcome with their extreme AI enthusiasm which they don’t see the ramifications of or just don’t care.
> Most computer science programs don't teach "coding" though. There is pretty heavy emphasis on datastructures, algorithms, and system design
Those skills are often taught in an idealized situation like very small projects. In the real world, there are a lot more practices and techniques to develop software in a pragmatic way. Those "engineering practices" are much more useful than the above, unless you're in some specific domains.
I fall into the category of senior engineers who benefit from LLMs for all the reasons mentioned in this post. I find it's possible to agree completely with sentiments like this and still feel as if this is all written in the sand below the high tide line, and ten years from now nobody will care about this.
Horsemanship and sailing were both specialized skills of high value to society, and now they're not. But in each case there was probably a liminal period, when being an accomplished horseman or sailor was still valuable, even as motors were taking over. Eventually that period ended, as the new generations without those skills found ways to get by with cars and motorboats.
This analogy breaks down because computers aren't going away unlike horse drawn carriages or sailing vessels. Computers aren't being replaced by something. It's that fewer people will understand how they work while the efficiency and reach of computers extends into areas of life previously unreachable due to the scale of having humans build it or the refusal of humans.
I could wax philosophical about what that leads to but enough people already have.
I'm looking forward to having the same experience as COBOL engineers did after retirement: companies backing up the brinks truck to anyone with the skills to keep the lights on.
now that's a phrase I have only heard in one other context: as an alleged quote from former Boston Celtics player Isaiah Thomas (who was really about 5'7 despite his official listed 5'9; yes, professional basketball player in NBA). This was around 2017 when he had just come off a career year and was soon due for a contract extension in the low 9 figures. He got injured and bounced around for a few years before eventually leaving the league, and never got that payday.
It’s commonly used in college sports where one school’s fan base argue for poaching a coach at another school. “Back up the Brinks truck” == throw money at him.
Yeah, every COBOL job paid pretty terrible relative to other types of work (like web dev).
Sure there might be a dozen individuals making over $300k but most COBOL devs live in the deep south making less than $100k for their entire careers. Fine salary, but that line of work is severely limited and the salaries do not reflect how valuable that work is.
There have always been opportunities for those who seek out the less traveled paths.
If all your peers are only accepting what LLMs tell them then you should do the opposite. Spend some of your time writing code by hand, especially when working at a new layer of the system or on new things you've never looked at before.
While the agents churn out more CRUD crap in the background spend your attention learning how everything works.
I do way more experiments with way more things by taking advantage of LLMS. When I have them do research I use that as a starting point for my own reading (but don't blindly trust them).
I find I am learning more faster.
Change is an opportunity. Use it to your advantage. Let other people while and moan while you get things done.
So far I think I have learned far more from LLMs than I've lost to them. I forget some syntax, definitely. But I now reach for a much wider range of tools that I have become familiar with because of LLMs.
Yeah, I have to say some of this deskilling argument sounds like Socrates complaining about the invention of writing ruining people's ability to truly own the text. I really don't mourn my lost assembly skills or my lost C skills. And I'm not missing the details of whatever web framework du jour my Claude is vibing for me on my extremely useful dataviz internal tools; and I have learned how to rein the agents in to not ruin my high-performance Go code and yet debug concurrency issues. Maybe coding qua coding is what some people find their summum bonum, but for me programming is instrumentive - to build real things in the world that do things to make something better. When I want pure virtuosity of intellectual construction, I read maths papers in arxiv.
(Not to say that the current state of the art with LLMs merits borrowing money for a multi-trillion spend on hardware that will be obsolete in 5 years, but for me that is a different issue).
The master decrying the invention that resulted in more wisdom for more people than anything prior:
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise." (quoting from https://www.historyofinformation.com/detail.php?id=3439)
I posted this elsewhere, but that story has a lot more nuance.
Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
There is an argument to be made though that writing stuff down yourself results in better retention than simply reading the same material. People still write by hand and take notes. And they still process ideas and thoughts verbally. Doing so is often more effective than going in circles with a chatbot.
Yeah. Thought in Socrates' defense I think the jury is still out on writing, in the scale of human existence it's still very new. Give it a hundred thousand years or so.
It's basically laid out in the article: the reason that you are able to gain so much from LLMs is because of your decades of experience that predated their existence. Ability begets ability, but if that ability doesn't take root, these more advanced toolsets don't yield the same outcome (and are likely to do harm).
To be clear, I feel the same way as you. I have been doing this for about twenty-five years, and I have developed a keen sense of taste and judgment to know how to properly scrutinize and utilize these systems. I definitely feel like I am able to learn things faster, but that is directly correlated to having experienced a lot of friction over the years that cultivated said taste and judgment in the first place.
I imagine it's sort of similar to someone who's just getting started in mathematics, but is introduced to WolframAlpha, versus someone with decades of experience in the field and what they learn with it.
We are spending sooo much for our intellectual time and energy on discussions about alchemy-like experiments with different models, agents, harnesses, orchestrations.
And there are huge amounts of time, energy, and writing sunk into arguing about AI or trying to convince people of the danger or problems.
Sadly all of this is taking time away that could be spent on real progress and learning about coding/FP/PL/algorithms/type theory/etc.
I don’t think there’s a big societal problem here, except that the agentic programming developers may be out of a job like the rest of us. If AI continues improving it will just keep taking over whatever skills are involved in the current AI coding meta are. If it stops improving then the meta will stabilize and after 10 years everyone will have longevity in it.
I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
Depends on what you are programming. I have been using it to build things that have never existed before, that I have wanted for over a decade. Experience gives one more imagination.
> I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
I do not think anyone could do the job I do with AI without my decades of experience, is my point. For now.
The AI programming meta that keeps constantly evolving (last month it was loops) seems like a relatively trivial skill to adopt compared to learning what it’s like to do high quality work and to participate in a team, and these kinds of things. If the AI stops improving then everyone will master the AI skill set that everyone else is using, and people will acquire maturity and experience over time, like always. This is my point.
I hope that this will expose the true value of education. It's a new scale perhaps, but not a new problem. We sit in classrooms even though books and YouTube videos make available the teachings of far more illustrious teachers than the one in front of us. We watch things written out agonizingly slowly in chalk. We learned to do algebra and calculus even though we had our TI-86s and Mathematica. We learn in Haskell and Scheme even though those are rarely used later. We write toy compilers and write essays and repeat experiments with known outcomes in labs. We do it to learn how to think, how to build. We just have to be even more deliberate about it now.
My personal antidote is to write unassisted Zig in my spare time (though C would be fine as well). While I can't back with data I certainly feel sharper.
I think we now entering this state where code is cheap but good well made programs will be rarer then ever, a lot of crap can be made and a lot of programs made without ever thinking the problem through properly will occur.
It might be like how an archeologist since the invention of plastic can date the period of the soil as post plastic, if one could cut open the software stack of the coming systems that will be built in the near future one would could data that code as post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
I am not Anti-AI but I think it's going to be interesting and I am surprised at how bad a program ends up when one tries to "vibe code", though often ends up working, though in isolation and when used with discipline (that the tools themselves psychologically make it harder to do) can produce some very good code and being able to use loops to solve difficult problems or problems that simply would have require banging ones head against the problem many times is very profitable.
Yes every time I tried to understand ffmpeg I got a little of it for a little bit and then lost it by 2 days or so. Just getting LLM's to do their thing for ffmpeg is great. the other one is socat an utility I used before but LLM's can do magic with it.
The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!
I can totally see where that could be the case. I'm sure there have been breakthroughs already, but it's not mainstream news. The mountain of AI slop is all over the mainstream though. You have to wonder why.
> This applied friction is directly what builds "developer intuition" (or "taste"). The Germans have a great word for this: Fingerspitzengefühl (fingertip feeling). It’s the muscle memory that triggers when a developer looks at something and thinks, “yeah...this is probably going to cause problems.”
No, we do not have that word for that.
Yes, we have that word. No, it does not mean that.
Any LLM proofreading could've told the author that, for that matter.
__
What Fingerspitzengefühl actually means could be described as "tact", a precise approach to something and general attention to detail that leads to success.
Or.. the lack of all of that leading to blunt failure.
__
I would of course comment with more Fingerspitzengefühl for the emotional needs of the author if the writing wasn't just a sales funnel for their courses and whatever else (+ posted by them themselves.)
This happened before with “real” engineering. Engineers a hundred years ago used to be proficient with slide rules and mental math and 3D assembly by hand. Products designed in that era often lasted a hundred years, like Singer sewing machines and Lada cars and 500 year old churches. Now things aren’t built like they used to be, and few people can rebuild civilization by hand like they did. But the world moves on fine!
It's different though because for structural engineering finite element models (set up correctly) will give you a deterministic, mathematically correct answer.
In fact FEM is arguably more deterministic than the old school pen and paper methods (yield line analysis etc.) which often rely more on correct judgement of the structural engineer doing the analysis.
I do understand the sentiment of applying "constraints" over AI usage to certain scopes of our development work.
This has an issue in practice.
As an example, Function Programming (FP) has a high constraint on programming, where some don't even have loops (gotta use recursions all the way), and almost no state mutation (Exlixir I believe have a mutable state? forgot).
FP sounds great with all those constraints. But what about the adoption?
It's hard to switch your mindset, and not as natural, thus long leraning curve, and not as wide adopted <- productivity goes down as all others need to know and use it well. That's why still imperative and OOP languages are ruling the world.
Same for these constraints. The author promoting using AIs only to subsets (no coding? wtf) is something the majority won't follow. Yes, the constraints sound great, but at what cost? By the time one learns everything, and everyone moves along with AI building stuff fast and cheap, they will be left behind.
---
I do not like the author trying to make his points authoritative by using Halo effect using quotes from the industry and could have had been taken out of context (That's where AIs mess up the most, they only get chunks of text and lost context of before and after exerpts).
Your inclusion of FP is interesting in an unintended way: LLMs actually thrive in a highly constrained system like you describe. "It's hard to switch your mindset" - isn't nearly as true for an LLM, it can usually adopt those languages quite readily.
LLMs don't particularly care how hard the type system or borrow checker is (as long as it has good error messages that can guide it to a solution), they can grind away at these things, and what you describe as a negative actually becomes a strong positive. I think we'll see strongly typed systems plus some level of formal verification become popular as an AI-preferred language.
When the burden of review is lowered because you have purely functional building blocks, with lots of formally verified chunks - people will choose that simply out of (productive, beneficial) laziness. Not to mention we're in a ram crunch, and being able to run your stuff far cheaper has an immediate financial incentive.
Substantiveness of this article notwithstanding, I do want to point out that the author also runs (or is planning on running) a programming course which to me may be a slight conflict of interest. I think the author may altogether be well-intentioned, but it IS something that I'm going to keep in the back of my mind as I consume this content.
> LLMs can detect patterns at a scale that no human ever could, but patterns only get you so far.
This is asserted without evidence, and from a scientific standpoint, unjustified.
First, "detect patterns" makes it sound like a classification task, "is this a picture of a cat". But LLMs transform the input.
For example, an LLM can translate text between two languages while properly handling the names of the people described, no matter what those names are. That shows they are representing the text in a somewhat abstract way, that they can perform operations on that representation, and also convert it to useful output.
And, what I just described is the most general form of information processing algorithm. Science is not aware of any limitations in principle on such systems.
I am not saying LLMs have no limits, but "they only recognize patterns, and that is a true limit" is not a good argument.
Writing code was never the job. It was a small part of the job that can now be delegated to AI.
Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage. You know `Engineering`. Mechanical and Electrical engineers didn't go away because of CAD.
> Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage.
Besides your first point, the rest can be delegated to AI as well. And you don't need to be a "software engineer" to spec out requirements and user needs.
Literally all those skills are the result of decades of friction, challenge, and deep domain knowledge...all of which are circumvented when LLMs are utilized early in the process.
CAD is also useless to Mechanical and Electrical engineers who didn't learn fundamental the skills that it streamlines.
Does anyone else feel like the web has become so "readable" it is illegible?
I have such a hard time quickly reading pages like TFA. Headings, bullets, line-spacing, width, link hover animations, even the font. Even reader mode defaults aren't great for me.
I think this is more “gray” than this. I feel like I can rip through ideas more quickly then ever before and as a result get a lot better at designing systems and (for my work) get a lot better at designing data/model experiments.
But I empathize with the author. I think the frustration comes from the fact that other folks in your organization use these tools to “get by” more effectively then before. In the past, a disinterested engineer might ship less under the umbrage of a difficult engineering problem. With the pressure to “ship more” now that there’s AI, the same engineer is using AI to spam PRs to prove they’re working, even the quality of work is low (because quality of thought is low).
Yes, but it's also building other skills. There's no friction to learning something new anymore, and that's really quite amazing.
E.g. I've gotten pretty good at git thanks to AI because it forced me into more complex workflows. I still can't remember the commands but I really grok the concepts and terminology on a far deeper level than before, where I made a conscious effort to stay on the happy path lest I get into some odd state I don't understand. Pretty sure those "odd states" would seem quite clear to me now.
It's surprisingly late for him to notice. But wait til you think through the consequences of the already collapsing human capacity for extended written composition, which is orders of magnitude more significant.
I've, more than once, gone long spans of time doing purely management and architecture level work, then returned to programming. A little rusty? Sure. But I never forgot how to program or lost the skills that actually matter. Seems mostly like a non-issue for people already in the field.
The real issue will be with people that never learn it in the first place.
And even that isn't so bad, learning programming isn't some insurmountable hurdle. It took me less than 6 months from no knowledge to my first job.
> It took me less than 6 months from no knowledge to my first job.
I agreed with you up until this sentence. Entry-level devs with 6 months of require an enormous amount of time and mentoring until they're able to really perform well. I think the greater concern being expressed is not will people be able to type syntactically correct code?, it's will people be able to write software well?.
In my experience human design -> agent code -> agent review -> release -> agent observer, does not (at all!) work. One absolutely has to be understand what is going on, how the system works, how the H/W works, etc, to catch agent errors.
The proposition of us losing those skills is terrifying.
Maybe AI will get better and we truly won't need a human in the loop. Then please also have all PRs signed by whatever model wrote them, and do not come to me for trouble-shooting.
But what we're gonna manage the codebase in, then? What kind of language?
Will it be a natural language, or a formal one?
I feel that "coding" cannot collapse. Coding is just translation from natural to formal language. Somebody needs to write the specs. And writing/maintaining them in natural language brings a lot of fun - shifting interpretation, inconsistency, missing specification, etc.
I don't think it's progress for the field of SW engineering. But at least more "shareholder value" will be created.
Then you make it write code in the tickets for the tricky stuff.
Have it reference relevant documentation/APIs.
Then there is no lost knowledge.
(Still, sometimes it sneaks in some "helpful" belt and suspenders, but for the most part, I know everything that is going on in the code base with this method.)
Coders should be treated as as well as as other software users. Tools like Claude code, Antigravity, and others are applying AI to solving that problem.
I know how to create a multi year P&L projection from scratch. It gives me some insight into how much capital a startup will need to get to breakeven. A good AI tool that creates that same spreadsheet in minutes should also be able to explain why different businesses have different shaped cash flow troughs. I don't think people using tools like that is bad even if they don't gain deep insight.
At some point it won’t be realistic to ask a human to solve a technical problem any more. I’m not suggesting it will happen soon but I can at least imagine that day coming. Whereas before modern “AI”, that possibility wasn’t even on my radar.
Something I noticed trying to learn rust through models, they're inherently tuned to just finish everything with even the slightest provocation so it's very hard to use them to actually learn anything, they kinda just want to do everything themselves.
This just feels like someone whining about how ORMs result in $#1tty data access code because lazy programmers don't take the time to learn how to use a database.
When that happens, (developer thinks using an ORM means they don't need to know how to use a database,) the project eventually fails. It's the same thing with AI; just like companies learned that they need to make sure developers actually know how to use a database, companies will learn that their engineers actually need to know how to program.
Speaking on a handful of these that I'm familiar enough with to opine on:
- Photography hasn't erased other visual arts. Both require a separate skillset; both have their uses and their enthusiasts. Photography benefits from having a lower skill floor, in my experience doing it, and this in turn has led to commercial photography eclipsing commercial painting (barring things like painting buildings) as a viable business model, largely.
I'm not sure if we're better or worse off as a society for having gone through that shift.
- Television is arguably not a net positive in society (I won't say net negative yet, but there are some reasonable arguments for that position). It enables a form of passive consumption that allows bad arguments to come through as persuasive through the gradual (on the scale of months and years) numbing of critical appraisal of the media being consumed.
This has more- and less-pronounced areas. In the US (which is geographically close to me and which has an outsize proportion on my own country's culture), Fox News is an excellent example of how TV can erode critical thinking.
- Smartphones arguably reproduce and amplify this problem, and tack on others: they make it increasingly easy to dissociate from daily life, and make it easier to continue consuming content from actors increasingly interested in steering conversations in one direction or another (on the most benign end, it's influencers trying to get you to buy things, which obviously has negative economic impacts on an individual scale; on the more overtly harmful end, they serve as a hardware channel for less-regulated or unregulated extreme and hateful content).
Just like with TV, smartphones have clear benefits and utility in daily life. They aren't abjectly bad inventions, but they have a serious societal toll, and it's difficult to ignore that.
- In aviation, the first generation of (relatively modern) automation did lead to a high number of accidents and loss of life. American Airlines flight 965, in 1995, is a well-studied case of an over-reliance on flight automation before it was widely understood among pilots what the limits of these early-modern incarnations of autoflight were.
In the wake of that accident, many aviation regulators instituted or began requiring that manufacturers and airlines formalize training and operational procedures that protect thoroughly against this sort of cognitive gap.
I think that last example (and many aviation accidents) are an interesting example to study in the context of software. Aviation's extremely and conspicuously responsive to accidents, often requiring procedural and hardware changes as a result of these. They can be as simple as a software patch, or as complex as the redesign of a component because a human factors analysis found that it could be confusing under stress.
In software, we like to think that we're a serious industry, sometimes. We spent much of the 2010s going in what I found to be a good direction: the development of IaC, the popularization and standardization of tools like Git, registries for vulnerabilities like the NVD, and even the development of languages like Rust which address historical technical shortfalls of high-performance, low-level languages like C; but we always fell short, industry-wide, on matching these toolchain upgrades with technically-rigorous procedures.
Ever since I was a teenager, I started to develop cyclical bouts of existential crises.
While I have been feeling better for quite some time, I feel another starting to brew. While I was never one of the fabled 10x or rockstar developers, I still derived a lot of enjoyment and and sense of purpose from my work.
With the utility of LLMs becoming more ubiquitous by the day, what purpose can I provide anymore? I struggled greatly to get where I am today, and while it might not be impressive to any of my fellow programmers, I am still proud to be a programmer.
I obtained a degree in CS because I found the entire field to be fascinating. I had great passion for learning many of the subjects. What I learned in my degree changed my life forever. I seriously went from one of the, "What's the point in learning most math if you never use it?" to being absolutely infatuated with the subject. (I'm still bad at math, but that does not detract from my pleasure).
With the utmost certainty, I did not push through all my struggles just to prompt LLMs all day. Maybe's its my insecurities manifesting, but I want people to say, "You did a great job" and not "Claude did a great job." I understand the Buddhist were likely correct. My suffering really does come from my attachment, but at the same time, I do not want to let it go.
I have no opposition to using LLMs as tools. I think a fine balance can be achieved. LLMs can even be exceptional learning tools when used appropriately.
If programming is just a means to an end to you, then you will receive no ill judgements from me. We all have different motives in life. But programming has always meant something more to me.
look to past examples. when the greeks developed writing, people complained that the youth were losing the ability to memorize much content. and that is true, but with writing they were able to do better still and that old skill -- though useful -- was lost.
Plato wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse.
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
I agree the tradeoffs are real but consider it obvious that reading and writing were in net a boon and I also expect the same for AI / programming. Of course having some understanding is still required in both cases to make good use of the tools.
This seems right to me, but I think that it's likely that we haven't fully figured out how to integrate AI into coding imo. I think there is user side thing that we haven’t fully worked out yet.
And so what? How do you think people are going to be working on software in 10 years. They're going to use coding agents and harnesses that work much better than the current ones (although I don't think we're getting anything but incremental improvement from here on out). They will run on commodity hardware. I first got paid for writing code in 1999, I've worked at publicly traded software companies and failed startups, in every language out there. And if LLMs "atrophy" the expertise I built up over those years, then so what? What is lost, exactly? I stopped programming in C++ about 10 years ago when Rust seemed to get good. I no longer remember a lot of the specialized knowledge I had in C++. My life goes on. I am now becoming an expert at building software with agents. The problem is still the same, you still have to apply engineering principles. You just don't have to write the code.
It kinda seems like wishful thinking on our part as software engineers that one day we'll be able to go "Hah, miss me now?" but that just isn't the writing that I see on the wall.
Over the past year, I've AI-generated two large pieces of software over 2000 commits without having much depth in either domain, and my ability to deliver value as a veteran software engineer has only shrunk over the months or it can be trivially extracted into a reusable markdown file like "design bar: ensure things 'by construction' where possible", a lesson I learned viscerally over decades.
Because I can still reason and dig into things and ask questions and have the domain explained to me out of curiosity, the same way I could build anything that was originally out of my depth.
But what I've noticed is that the need for any corrective power has gone or is going to nil and I'm mainly doing directional work. And even then, I can constantly have sota models "rank the top options and recommend one", and it's almost always the right way forward.
I used to read every line of every plan that LLMs generated, but over time I've realized I have less and less to correct or, more impressively, the LLM had foresights I never considered. The same thing happens with implementation.
Periodically I can spawn a bunch of agents to evaluate the system adversarially to find improvements, and the findings have been so good that it's evident that soon I can just automate that too.
In other words, the skills I need to excel here are more curiosity and patience than tech expertise -- the things that got me into software engineering in the first place since that was the only way to build things.
You got it. You may well be right, but from my standpoint it's hard to see how you'd know if you weren't. Yeah, the models fill in vagueness with detail sourced from "what's likely and helpful". (In one of my particular domains, that's "use SHAP values for everything everywhere".) Yeah, that detail's increasingly unlikely to be "wrong" in a way that you could explicitly argue. Is it what an expert would come up with? Is it the best it could be? Is there something else that we would have done had we been forced to think it through? Will it ever be _surprising_?
I know, all irrelevant little philosophical roadbumps. The artifact satisfies the buyer. I think I'm going to have to step away from this industry pretty soon, at least for a while.
As always, the innovators will thrive and become the new experts in the field. Everyone else will die out.
It's impossible to buy the FUD that expertise will somehow vanish because people use AI. People come in a variety of capabilities, challenges, and personal interests. Expertise varies from person to person. Those with personality traits that are more likely to take risks and persist against adversity will be the innovators and experts. We've seen this played out countless times.
You have to ignore everything you know intrinsically but can't articulate about innovation, technology, and evolution to drink this FUD Kool-Aid.
One interesting feature of vibe coding is that it allows you to write code in specialized areas where you have no expert knowledge. That makes the code effectively impossible to review in any useful way.
For example, I have a vibe coded Python monstrosity which I use for stitching together scans of large format negatives. The enlarging lens that I use for scanning has strong chromatic aberration in the red channel but is otherwise very sharp†. I got Codex to figure out some way of using the green and blue channels to sharpen the luminance while preserving the color information from the red channel. I sort of vaguely understand how this works, but not nearly well enough to review all the horrid numpy code that implements it.
For my use case, there is really no reason to look at the code, as I am only interested in the output. As disconcerting as it is, I feel that this is going to become more and more normalized. Looking at the code that your LLM generates will become like looking at the assembly output of your C compiler.
† This makes a lot of sense, if you think about the color of a darkroom safelight...
I used to think this was a problem. But now? I don’t think it matters.
Why? Because people are going to get so much more ambitious about the stuff they build with AI, that code expertise for the stuff they are building is already scarce anyway. People will choose difficult and less popular languages, and will replace open source packages with stuff built entirely in-house by AI.
Going forward, there is no choice except for AI to fully replace the need for code expertise. Depending on humans won’t be scalable.
Do we even need to get LLMs to create code in the end then? Can we imagine a future where the thing simply does it all, you deploy the LLM like a docker image and have it process all requests directly. Sounds far fetched today, coming to your cluster in five years.
In theory yes, but you would need LLMs that basically have frontier level capabilities but with zero latency, meaning you enter a prompt and as you type the prompt is regenerating instantly like some kind of autocomplete.
I think that would feel like crack for some people who are addicted to using or building stuff with LLMs.
Very few people know how to hand set typography anymore, but many of the conventions still exist and people don't have to understand the origin of a lot of them to use them properly.
I'm seeing it happen in real time. I've seen several folks claim they "vibe-coded" something and upon review it's some seriously low quality garbage. Unfortunately it's just "functional enough" where it's encouraging the one who talked to the robot to continue thinking they can code for some reason.
It is important to ensure that any LLM code is named and shamed.
The industry has always had niches where quality is really important, such as NASA where you install your product in a multi-billion-dollar, one-of-a-kind robot and launch it into space. However, for the most part all that matters is that it roughly does what it advertises to do and ships as quickly as possible. LLMs are ideally suited to the latter form of software and the need for humans to ever see or think about the code will drop to zero soon, if it hasn't already.
Coding expertise was on a downward trend way before AI arrived. Does nobody remember how most "engineers" were literally copying and pasting snippets from StackOverflow they didn't understand, and the snippets usually had tons of bugs? How many of you know Assembly well? How many of you can decompile a program, or write your own compiler or VM? How many of you could pass a computer science course (that you didn't take at Uni)?
We've been hearing about the demise of skills since before the invention of the loom. The only skills we've truly lost are for things nobody uses or makes anymore. What actually happens is, we develop a new tool that's better than the old tool, and we get good at using the new tool. We then have a class of people whose job it is to maintain the new tools. We don't need that many of them because the whole point is to eliminate that labor.
Your jobs as software engineers are going away, it's that simple. Your new job is to use the AI tools to make products. Somebody else's job will be to make the AI a better software engineer.
In an abstract sense, yes. Compilers let programmers work at a higher level of abstraction. LLMs bring it to an even higher level, though the abstraction is much leakier.
For the time being, a good understanding of programming and domain knowledge gets you much farther than without it. There might not be room for as many (professional) programmers as before, though.
That's because the institutional knowledge was passed along through the act of coding. Skills that I learned in Assembly and FORTRAN are applicable to even modern programming languages...but not to "natural language."
I don't agree. Certain optimization skills are learned on the job if people work in specific areas of compilers or performance.
Also the reality is that modern CPU microarchitectures are so different from what old-school assembly programmers used that their supposed optimizations are likely to be counterproductive.
I'm also concerned about people asking an opaque machine for an answer and stopping there instead of going to different sources, evaluating their credibility, exercising their own ability to construct their own worldview from disparate facts and opinions.
Flagged for the expressing the most obvious and legitimate concern about LLM-backed "research".
"I use AI to generate a summary which I print out and read slowly every time I use a new codebase" is the new propaganda line pushed by Anthropic and OpenAI. They cannot tolerate this rebuttal.
Are engineers still making 100s of thousands a year still just for vibe coding while a nurse and other skilled / highly trained medical professionals working their butt off make half of that?
Do we think there is going to be a correction for the vibe coders?
Pardon I know such question is controversial but thinking ahead.
There's something weird about nurse pay where it doesn't seem to raise even during large shortages of qualified nurses. I think it's artificially lowered by hospital admin, with temporary traveling nurses acting as "scabs" to avoid the normal wage increasing
They do, it’s called the AMA, American Medical Association. Very tight control of who can be a doctor, how many new doctors per year, etc. which ensures salaries stay high. They also fight against other roles like nurses doing too much doctor-like work, helping doctor pay while harming nursing pay.
At least where I live there’s a shortage of doctors so you can’t negotiate with them . There’s a shortage of nurses too, but if you lose 1 nurse because they’re striking it’s bad but things don’t collapse.
With 1 fewer doctor who also knows and is allowed to do things nurse don’t / can’t things get really bad. They also have access to power, are often elected, and are respected.
Nurses need to unionize I think and strike to get something .
Is it my job to make sure that nurses are paid more than I am?
I'll readily admit that they deserve it, I'm a lazy dickhead and they help people, but I think your blame is misplaced if you're insinuating that software engineers are to blame for nurses being paid less than ourselves.
While it depends on the market, job, and hours, but the bedside nurses I know make in the low $100k range. Typical shift structure at the hospital is three 12 hour shifts in a row and then four days off. Most of the time, they are union jobs, so they receive many benefits from collective bargaining such as increased holiday and overtime pay in the range of 1.5-2x normal rate. A nurse practitioner tends to make something in the 180-220k realm.
Again, to be clear, this is highly dependent on the market, job, and hours and one needs to understand the difference between a CNA, RN, NP, etc. However, it's a regulated typically union job that's very secure and in high demand. It probably pays more than what you expect.
Yes, they are still making that much. I am. I don't fully understand why yet but I think it is because of a few reasons:
1. We still need to keep the lights on. If things go wrong, which they still do, humans have to debug it.
2. Jevon's paradox: now the expectation is that we ship stuff faster and there is still a LOT of stuff to do, not less.
3. We are still in the very early days of this and it is too soon for corporations to make a dramatic call like laying off 80% of engineering. We just don't know how that would look like.
ASM to a higher level language is a jump to a clearly defined higher level language through a deterministic compile step. The jump away to LLMs is something completely different.
1. function trampolines, such as in libraries like ATL (Microsoft's COM template library);
2. hand-optimized encoders/decoders for things like ffmpeg;
3. niche applications such as Ben Eater's 6502 breadboard computer (using a modified WozMon and Microsoft BASIC);
4. reading ASM output from C/C++ code in things like the GodBolt site for performance comparisons/analysis;
5. reverse engineering old DOS games;
6. people working on things like compilers (generating optimized assembly), JIT engines, or other code generation -- though this will typically be machine code, possibly with assembly instruction comments;
and other performance critical places where assembly can speed up that part of the code.
Keep in mind also that LLMs are currenctly heavily subsidized. Once VCs money are gonna run out, you will see the real price, and your 200K per year dev is probably nothing.
I learned BASIC, Pascal and x86 Assembly language by the age of 14-15. None of which I remember (nor care to). The strict instructions (aka language) we gave the compiler was not the point. Those experiences taught me how to think logically, and over time develop a taste/structure for how software should be built, as well as foot guns to avoid. I don't know how you teach that without coding, but the "coding expertise" was always a means to an end. LLMs are just another abstraction layer above programming languages, machine language, etc.
Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:
In contrast with Pascal, I could see in my mind the PDP-11 instructions behind my C code. By the time I started running code on x86, there were debuggers that could step through source code, and I didn't really want to think like an i386.
> LLMs are just another abstraction layer above programming languages, machine language, etc.
Programming languages are (largely) deterministic. LLMs are not. The boundary between an abstract concept and something executable sits comfortably in the mind of the programmer in non-LLM contexts. This boundary shifts when using an LLM and becomes shared between the programmer and the matrix multiplication machine.
The "layer of abstraction" analogy is therefore technically true but essentially false. "Just another" papers over how seismic of a paradigm shift all of this is.
Sure, you're right - at the end of the day, LLMs are non-deterministic code generators. I certainly wouldn't trust one for a medical device manufacturer's software. For line of business apps it's totally adequate. And to be clear, I don't love it. I'm just coming to terms with the fact that these LLMs have gotten good enough, that with enough instruction and guidance, they can generate code/features at speeds we can only dream of if we were to code them by hand.
That's the whole scam. It's a ransom with more steps.
First they fuck up your codebase with so much slop that no human does (or could) understand it.
Now if you want to keep making bug fixes, you're on the hook for whatever Anthropic wants to charge you.
The solution is to make AI developers the scapegoat. Fire the people at the top of the AI leaderboards, with malice. Otherwise, lazy AI-addict developers are going to destroy what remains of your company's IP.
Actually, they have for a large majority of the population. And fun fact: that debate is still going on, and the Mathematical Association of America (MAA) enacted a rule prohibiting calculator use in AMC.
Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...
My question would be if "math" is the right tool for that. It can achieve that as a side-effect, but how we express it and talk about it is full of legacy cruft, cargo-culting and general bullshit that makes it not exactly snap into place for all kinds of brain wirings.
Math I believe is not an end in itself. Maybe with that perspective, solutions that satisfy our needs there can be found
Because the 1890s and 1930s were famous for their population wide critical thinking, of course. Good heavens, what do they teach in the schools these days.
I mean you may be trying to make a point but the counterpoint is Einstein, Bohrs, Planck, Shannon, Turing, Von Nuemann, Feynman, Bohr and most of the other big names all came out during that time.
In contrast since then we've had almost not noteable physicists or Computer Scientists that I'm aware, we've got a couple, Hawking and Knuth. But apart from that I don't know if there is a single scientist I could point out today that has revolutionized our understanding or even done any real breakthrough work in the field of Physics since Einsteins era.
We have gotten better at working as a group. The standard model is a tour de force. It was a collaborative effort over decades but is a lot more complex than anything Einstein did. To some extent we haven’t gotten new physics since then because of reality - we have built higher energy experiments but haven’t got new results.
But my reply was I guess implicitly assuming the post I was replying to was talking about the median level of critical thinking. Your reply is talking about the peaks of creative accomplishment which is not what I was addressing.
Not a good argument. Those were big names because they were (co)founders of major fields. Physics and CS aren’t like fashion where you can come out with a new color or design every six months.
You also seem to be restricting yourself to pop science knowledge of who’s made breakthroughs. That’s because our collective understanding of physics has become much more detailed, to the point where it starts to lose the ability to be meaningfully comprehensible to the average layperson.
There’ve been many major developments just in that area since the early 1970s, to through the late 90s all the way to 2019 (the island formula), with many major contributors like Bekenstein, ‘t Hooft, Susskind, Maldacena, and many less well-known names.
If you resurrected some of the scientists you mentioned, they’d have a lot of work to do to catch up, and making field-changing major breakthroughs would be just as difficult for them as for anyone else alive today.
Something similar is true in CS. There’s been plenty of progress in all fields of CS, but similar to the situation with physics, its nature is not as simple as what the founders of the field were dealing with, so there’s less awareness of it outside the field. Just to pick one small example, Conor McBride’s insight that the derivative of a regular type is the type of its one-hole context ( https://news.ycombinator.com/item?id=34358663 ) is beautiful and kind of amazing, but not a result that you’re going to hear much about outside of PhD programs and the like. CS is chock-full of such work. The entire field of programming language theory has come a long, long way since Turing and Von Neumann’s time, to pick a broader example.
A more apt analogy is the diminishing understanding of assembly language and machine instruction sets. I love programming and something got lost in both transitions, first to high level languages and now to coding agents. I am quite sure some contexts will require programming by hand for a long while as there are still people who are writing amd64/avx512/sass by hand. I hope in time we will find new ways to enjoy solving problems with this new programming modality. By now, I am pretty sure it is here to stay and evolve from its current crude form.
> the diminishing understanding of assembly language and machine instruction sets
This is true and nobody complains that GCC or LLVM is making programmers dumber. They are code-generation tools from higher-level source code. Much like LLMs are code-generation tools for higher-level language/prompting.
Man, Usenet had the best online discourse - hardly any angry personal flaming because people dared to disagree. And Twitter, wow that was just pure rationality. slashdot - practically a masters program.
Now, they all did have gems of interaction that would could find by sifting thru the dumb stuff, but I find that property of discourse to be pretty steady. I've even found some interesting comments on YouTube videos and newpaper websites.
Not really the same, since your average calculator doesn't take over executive function. The user thought and decided what needs to happen, down to one extremely well-specified task, which is so unambiguous that it can be completed by rote and without context.
This is more like paying someone else to read your math homework, generate answers, and maybe even submit them on your behalf. If that had been the (average) student experience, math skills really would have collapsed.
I think we'll find that the people doing that, like the people spending all their time and energy reading slasher pulp fiction in the 1950s, won't be doing as well as the people learning to enhance the power of their education and their expertise via LLMs and strategic alternating between GenAI and thinking about problems. And eventually, if society ever steadies up a bit, people will learn the useful ways of using technology in school and youth; and we'll be glad to be alive to see such marvels.
Assuming we can re-establish the will for democracy and taking care of the whole group.
I don't think it's quite as bad, but I'm imagining someone in the 1950s going: "People just have to use methamphetamines responsibly in their work, and then we'll see the massive benefits as widespread adoption unlocks the potential of the human mind."
In other words, a positive outcome with thing X is possible but there are also big behavioral and cybernetic pitfalls, ones which are suspiciously persistent.
I mean I use coffee quite a lot to prop up my software career. I don’t really find purity to be useful for anything except to answer questions like “should I steal this money now that I am alone” or “should I backstab this friend who pissed me off today.”
> questions like “should I steal this money now that I am alone” or “should I backstab this friend who pissed me off today.”
Ah, yes, a very BOFH approach [0] to workplace quandaries.
> I mean I use coffee quite a lot
Perhaps, but I don't think either of us would agree with the claim: "The way we integrated caffeine for productivity offers a useful and practical model for integrating meth for productivity."
There's an extremely detectable difference in the feedback loops that form between the tool and the people using it. There's something which remains unsolved ~90 years on. If we assume for the sake of argument that coffee is "good" and "productive", then that leads us to the question:
Are these LLM systems going to be more like coffee, or more like meth?
Those are some big ifs and assumptions unfortunately, and the use of AI unfortunately has an isolating effect that is only likely to move the needle in the wrong direction.
I also can't make butter by churning or soap from lye and animal fat or what ever.
However, I can use mathematica to demonstrate the criteria for various forms topological stability of solutions to many types of differential equations. I'd rather live now.
That's a very good point. People don't pull out their calculator in the grocery store to do analysis of unit costs or whatever, it's too much of a hassle for that small kind of analysis. But calculators have robbed us of the need to be good at doing that math in our heads. So the math doesn't get done.
Many grocery stores will already give you a per oz or per unit price so you can compare pricing for different brands that have different sizes. It's a very handy feature.
My most-common frustration with it is when products I want to compare don't have the same quantity-denominator. (Either a different unit, or sometimes a different dimension like weight versus volume.)
I've many times seen it wrong but more often the same item will have some tags expressed on cost per ounce, others per lb, etc. To discourage comparison. But I dont think people are paying much attention to those numbers anyway. Math has become a scary topic for them altogether.
This is exactly the right analogy. Granted LLMs are a more sophisticated calculator and engineering is still adapting to this wonky/evolving technology. But to pretend it's a coming catastrophe is to ignore every past parallel development - where this has not happened.
Math skills are at their lowest point ever in modern society, and many are still advocating for the reduction in reliance on calculators for this reason.
"Mental math skills" literally did collapse tho??? "Math skills" to write proofs (which a calculator cannot do) is a fundamentally different skill set than that "math skills" of mechanically doing arithmetic (which a calculator does do).
Seems like critical reading skills have already collapsed.
With a calculator you still need to understand the fundamentals of mathematics. You also need to understand how to operate a calculator and know its functions.
Even when removing numbers and manual calculation, being a function or formula oriented mathematician still requires you to know the theory and functionality.
With an LLM, you don't even need to know anything other than asking the robot to perform. You don't understand the output, nor can an unskilled person explain it.
This is the same kind of analogy people make when they "poo-pooh" any sort of technological advance, especially with AI.
"Oh, that old simple device didn't displace anything, so this one which is now quickly taking over every job will be just like that simple one". So I guess people today know how to sew and knit and bake bread?
"Oh, humans were always needed, so it'll be the same this time." Is that true of horses and oxen after cars and tractors were invented?
100%
We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong."
Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
This is all complicated by the fact that we're also losing our grasp on reality from the other direction because we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
Nor even €50k/year.
Two things are true:
1) The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to *write*.
2) My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
That last one, "given the time constraints", is a place where my experience is still useful. The AI is as lazy (or as optimised for fast wins) as the humans whose examples it was trained upon; but an LLM costs so little that the answer is always "do it right" rather than "do it fast". A lot of people don't know what "do it right" even looks like, having only ever known e.g. websites that take 5 seconds to load because of all the adverts and analytics, and never the world where machines with 1% of the CPU and no GPU at all could fit a fully playable first person shooter in the same memory footprint as that page.
At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Engineers have done amazing work of abstracting away the hard algorithms and data structures. In the meantime, there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms. In contrast, getting LLM to work is a new field, so it requires tons of new implementations: KV caches, speculative decoding, all kinds of variants of attention like FlashAttention, all kinds of parallel processing techniques, RL pipelines, post-training pipelines, and etc. It's just that the field is so concentrated that only luck few get to work on them.
So, maybe it's not LLM per se that removes the need of writing code. It is the maturity of the software engineering that has done so. It's just that LLM fills the last gap: making knowledge transfer so much faster and cheaper - if all that's left for most of us is slicing and dicing of what has been already been implemented, then LLM can reliably take over.
Before, if I would start working / thinking on a problem and discover it was harder than anticipated, that often was a signal that the feature may not be worth it. Because implementing it doesn't fit the current model of how things work or similar. Then it was back to the drawing table and find a better way of solving the underlying issue.
But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.
So exactly true! With ideas people unbridled, the painting-yourself-in-a-corner tendency that was always there has gone exponential, add to it the not-completely-aligned incentives, and the oh-so-clever impressive writer of long prose LLMs just end up churning. The real value add may end up being the lesson in honesty and humility to us all...
I first brought this up late last year – that there used to exist a kind of selection pressure against both developers and against features which would prevent certain functionality from ever being shipped, specifically:
- Functionality which exceed the technical ability or knowledge of the developer which built it; and
- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).
Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.
Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.
I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.
Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.
Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.
At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.
If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.
Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.
In my personal project I've found having a hard lines of code limit has been a good limit on complexity. Makes me really ask if I need things.
One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple
There is a limit, after a few 10,000's of LOC ai projects start disintegrating and becoming unmanageable.
10,000 LOC, hell my LLM can do that in a day!
Current project has over 60,000 LOC, I find using an antagonistic agent code and architecture reviews are really handy for keeping things grounded.
>My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
This is not non-coding work. This is coding.
this is software development
There's a reason it's called Software Engineering
Coding < Development < Engineering.
My claim is just that LLMs are basically fine now with the "coding" part. Not so much with development, definitely not with engineering.
I feel like there is also the "Nerd Factor" to consider which is this unrelenting passion to type away at a screen all day, whether that is yesterday's code or tomorrow's prompt. Few non-technical people have the attention span to suffer such monotony. (Even most devs don't have that to varying extents)
Aye. This is true for many professions, I think.
I'm trying to write a novel, have been for a while now; one thing I hear from professional writers is that one should only do this if you are prepared to re-read whole thing six times before you even send it to the editor, and when you do send it to the editor, you and they will likely spend several months repeatedly re-reading the whole thing as you respond to a long list of changes they give you.
It's only monotone from someone looking, not from someone doing. If you look at digital painting tutorial, you'll see a lot of timelapse, because it's a very slow process where you spend the majority of time correcting stuff. But if you're the one doing it, each individual brush stroke is the result of a conscious decision.
I'm not sure; I find the job pretty boring.
Working on a passion project or working through the hairy details of a complex algorithm or whatever can be fun. But that's not really what the job is like most of the time.
All the boring parts I've encountered have soon been automated out of my workflow. There's the waiting part of some process, but that's why I have HN for.
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
That is certainly not true. LLMs can't write code worth a damn still, and you have to babysit them to make sure they aren't doing stupid stuff. A human is still by far the best choice for doing programming, and when trend-chasing companies realize the damage they've done to their businesses they will stop pushing LLMs so hard.
"LLMs can't write code worth a damn"
I think it depends on the - ahem - context. In my experience they usually can, sometimes they can't.
But generally their sense of software architecture remains abysmal, so even if their writing is ok, you need to have architectural authority the very least to guide them.
The fact that they sometimes can't means you can't have someone who is not a software engineer. Otherwise there is no way to verify whether the LLM wrote correct code or not.
FWIW: The highest paid devs I know excel at 2, and aren't even that good at 1.
I guess that somewhat describes me - and guess what, that also transfers to "should we even throw AI at that specific problem, or are there other solutions?". Early curiousity in figuring out the limits of LLMs now is turning into revenue by customers with problems that didn't exist a few years ago at all - which currently is quite an exciting field to work in as a lot of things just don't have pre-established solutions yet.
It's a similar level of excitement as when we started doing devops stuff before devops even was a thing (which has been one of our main things for a very long time now).
Ai can work through all of those questions in number 2 and design around it.
I was just doing that yesterday designing a game demo lol
I have also tried developing games with them. I find them very weak at these things.
Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.
They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.
My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.
Works for a demo, but quickly stagnates. Future models may of course push the limits further, but right now it quickly goes sideways without someone knowledgeable at the helm.
You don't design things with never ending details though. If your developing a feature there's a limited scope to it. If your designing the direction of a large project you're working with higher level concepts.
The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.
> The point is that AI is aware of constraints and can manage a round it.
Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.
Are you an AI?
Your right to push back on that...
> Your right to push back on that...
Human test passed. AIs make giant flashy mistakes. They don't fuck up basic grammar.
I have found it intriguingly difficult in my own tests to make LLMs deliberately fuck up grammar in any way similar to how humans do. I suppose this is actually an architectural limitation.
Out of curiosity, I just tried this:
and the result was a pretty good simulation. I tried less insulting prompts first, this did not produce a good simulation.Not perfect, gptzero.me still knew it was AI generated (tayo42 is human by the same measure), but if for some reason someone was using an LLM prompted with that pattern, I suspect it would fool me in a Turing test unless I found the keyword to force the agent to change the role it was playing.
Redditor is too easy.
Conveying the errors typical in a Raise The Colours group on Facebook, that's the hard stuff.
I'll take your word for it, I think I'm in zero groups on Facebook, I only even look at the feed when I see a notification in Messenger; and I barely use Messenger, too.
We have a product guy on the team who was in a deeply not technical role before AI who is trying to do the “hey Claude, read this Jira ticket, implement” thing.
It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
The spacer between the keyboard and the chair still matters in my experience.
Not everyone has this, but I always felt that a significant percentage of the value I bring is in immediately recognizing what you shouldn’t do.
I have a hard enough time explaining why “yet another bespoke application on some unmanaged azure resources” is a bad idea when they have more reasonable alternatives at hand.
Now these goofs can (very nearly) press a button and do it anyway, with no comprehension of the consequences. It’s high fives and pats on the back, until I’m cleaning up the mess.
Yes, there is a post that made the rounds not long ago entitled Taste Is All That's Left: https://notashelf.dev/posts/taste-is-all-thats-left
Knowing what not to do has always been important, but it is perhaps more important than ever.
This lands. Llms are bad precisely at following what not to do. They work best off of positive constraints.
I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps. I call it a bias field, pushes them toward hopefully the happy and harmonious (with the rest of the system) paths. Obviously this is only partial and imperfect enforcement, but if it's applied to everything consistently it naturally encodes some self-consistency and harmony.
> I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps.
I’ve done the same but it’s a moving target as models advance and I find half of my points are ignored until I’m prompting “No wtf why are you still trying to symlink the global Python executable just use the virtual environment that’s already activated”.
Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time. There’s no way my hobby fiddling will outpace whatever gets released in the next couple months.
In the meantime, real linting does work pretty well, if you can write a detector for whatever antipattern you find LLMs fall into (like multiline comments).
Oh I'm certain I'm not doing it optimally or close to best practices. It's just an example I had in my head related to my comment.
Although, given opus 4.8->5.0 experience within claude code, I don't know if I'd agree they necessarily improve things with time. But that's just my experience. I should probably read their release notes, maybe just a tweak to my claude.md would help
> Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time.
I get patches into the biggest AI projects all the time to make inference faster on my 3090 and all you have to do is read the contribution docs and open a small PR.
It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI.
You can trivially outpace what big companies are doing it’s insane, why do you think so many innovations are coming out of scrappy Chinese labs? They are willing to engage instead of being defeatist about it.
> It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI. You can trivially outpace what big companies are doing it’s insane
Not my core competence and I’m not interested in making it a core competence while the field is still rapidly changing. I find harness engineering interesting, but not lucrative enough to push one of the many very important other things on my plate to the backburner.
I don’t think Python language developers are super geniuses either, and I’ve never submitted code for consideration.
I would be interested in see that. I have reems of rules I use with Claude.
I've had some success with: here is this android bug ticket in a product that I don't know much about, fix it! And it works sometimes. I'm still technical, so it isn't the same, but it was crazy that AI could just solve some problems automagically without me know much about the system being modified.
> It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
Remove the dev from the equation. Suppose you need to produce circumstances for the slop machine to succeed on its own (imagine a loop).
What is missing? A good enough description of the current behavior, desired changes and how it should work? Understanding of the business domain or the system in question, its architectural choices? Enough guardrails and linters and tests to ensure that the solution will indeed be testable and work end to end? Cause a lot of the things that will help human devs will also help LLMs make their output more workable - for example ADRs in the repo and custom in-house tools to ensure they follow architectural conventions and practices (or just something like ArchUnit). Realistically, how close to a finished implementation must you get and how far backwards from that can you go before its output degrades too much?
I tried figuring that out myself on a personal project where I mostly want what I want in GitHub issues and then feed them to Kepler, which spawns OpenCode in worktrees and also gives me a UI to follow the execution. Most models, even SOTA, occasionally implement things wrong, but it's suprising how far you can take things when you've spent a dozen hours planning out what you want the end result to be vs not to be (talking about the goals for an entire system, maybe an hour of planning for a more bounded type of task).
In general, when I’ve worked on projects where I’ve spent that much time planning, having a magic AI box that wrote perfect code wouldn’t have helped that much. When you spec every everything out like that, writing the code is the most trivial part of this whole process.
But isn't it only a matter of time before the AI generated autocomplete will contain the correct solution? What then?
For the completion to work, the source text needs to be ‘good’. That’s a basic kernel of how the thing works. Even with a perfect oracle autocomplete if the source text is ‘bullshit’, the output is too.
Or, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
Same with any field.
fucking is it? AI labs certainly want you to think so but they're running out of time and money. they've been telling me that the advancements of the last three months have made this a reality for the last five years. it's starting to feel a lot like Elon promising fully automated taxi services by next year... 11 years ago
Come on bro, PhD level intelligence!
next year I PROMISE!
So far, the LLMs I've used need problems to be fairly specifically scoped, or they don't produce the correct solution. Scoping problems correctly is a different skill-set than implementation, but it's still a technical skill that is expected from mid-level and higher engineers.
Yep and I just ran a simple enterprisey "ambiguity" bench on the big three (US) model providers: same ambiguous initial-prompt with same clarifications and pushback prompt sequence afterwards.
The edge of correct/better when facing ambiguity is very fuzzy, all models from the past 6 month or so have similar random ways of spinning between too-literal avenues and oddly misplaced misled fixations. Taking the right initiatives in face of uncertainty is definitely AGI, and its not there, and perceptrons + attention layers just ain't got what it takes no matter how hard you push.
The best I've come up with is to ask an agent to come up with questions to help get to a focused design. But it still needs me in the loop for that.
Probably worth not getting too comfortable. It's only a matter of time before a new generation of product people can do this. Also, I have said this before, instead of 1 product person and 10 engineers on a team, you may have 2-3 product+eng people and 3 engineers, etc.
Boris: "I don't prompt Claude anymore. I have loops prompting Claude and figuring what to do".
Boris: "I haven’t written a line of code by hand in, I think, eight months now… Claude Code, 100% written by Claude Code".
Boris: "There’s no manually written code anywhere at the company… All of the SQL is written by models. Everything is just built by the models... Claude instances communicate with each other (e.g., over Slack) in autonomous loops"
This does not sound like they review the code either. So, either the frontier labs like Anthropic have figured out something that very few companies could replicate, or they are being incredibly deceptive. I don't know which is true.
Judging by how it feels being a customer and using their products, I can confidently say:
1. Yes, we know, and can tell.
2. No, they haven't figured out anything. Just vibe coding it with their bleeding dege models.
They work on research problems you can define a clear solution criteria for.
The kinds of business software I work on don't have those characteristics. If I needed something like a utils library, I think I could easily have Claude write the whole thing and not read the code.
As always, the essential skill in engineering is not providing the right answer it's about asking the right questions.
Mythical man month should still apply for AI coding. If you look at the code you're shipping, and you can't quickly edit it by hand, or fix a bug with it, you will be in trouble if the models are down, and your website is down at the same unfortunate time.
1. I think LLMs will end up pretty dramatically shrinking MTTR, a lot of that will be tooling to proactively resolve problems as soon as they start, but a lot of it is that agents are very good at finding and fixing problems by correlating data, something humans can do but I think we're going to be slower at 2. "The models are down" doesn't seem like a very realistic problem unless you have a single provider/set of endpoints. I do not recommend this setup but I guess if you have a single point of failure that is a risk.
People are fundamentally short-term thinkers. Just imagine spraying neurotoxins on your lawn. You're "saved" from the "terror" of noticing more than one kind of grass, but of course you're incrementally increased the cancer risk for yourself, your children, and who knows who else. This is just one example; people do this constantly. Plastic fleece puts an untold amount of poison into the environment, but "I need to be warm RIGHT NOW." etc. Or lead paint. "It's only toxic when it breaks down, and that won't be for decades!"
Why would programming be any different?
All the issues you mention here have a death horizon: the people engaged in the behaviour will be dead when its effects are most acute. That they engage in it then isnt very mysterious.
That doesnt clearly apply here
The problem is further away when the next quarter, which for some companies, might as well be in the year 3000.
> the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it
The word "engineering" means doing things using a repeatable process to get predictable results. If you can get predictable results (e.g. guaranteeing the absence of data integrity issues, security issues, anything that will cause downtime, etc.) without looking at the code, you're still doing the work of engineering.
Your job as an engineer is to choose high-value problems to work on, solve them in the correct way, and guarantee that the functional and non-functional requirements are met. If you no longer need to read the code to guarantee that it has the right set of ilities, then I'm not sure that's obviously a bad thing, as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
> as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
Perfect example of begging the question.
Vibe coding works great as long as vibe coding works great.
Personally, I'd love to see how you enforce security and long-term maintainability without looking at the code cuz we need to add that to our build pipelines ASAP.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
When you factor in overhead and benefits, many companies were regularly paying that much for someone (many someones) to "read this Jira ticket and implement the feature".
We are currently in the "centaur" phase where a human-AI combination produces the best output, but I think some people are betting on the fact that the AI only product will eventually outperform the centaur. And with the cost of tokens falling thanks to fierce competition from the Chinese open weight models, it's definitely possible that those that bet on AI early will reap payoffs in lower expenditures for more output.
Honestly not sure which side I land on that bet but I definitely can't rule it out.
We've gone through this with chess, and the days when a human can do anything to help AI there are long past; the human can only drag it down. It seems to me to be a form of the bitter lesson. I expect this will happen in every field where we don't add guardrails to require humans in the network. I have a hard time seeing regulation happening around this in the US but vanity may also serve, will CEOs be satisfied ruling armies of virtual assistants, or do they need big buildings full of human thralls?
Well, maybe. But most fields of human endeavor aren't very much like a board game with clear, unchanging rules and a single, unambiguous objective.
Have we really though?
This is not a silly question, because it's kind of silly:
Has an AI ever actually gone through all of the steps necessary to become a chess master? Go to competitions, raise in the ranks, take a plane to the masters, organize support and all the real life details necessary? Playing chess is just the small formal part here, and all of the obvious things humans obviously do, well...
I don't agree.
Chess is a perfect information game with objective, easily measurable outcomes.
Show me a software business that looks like that and I'll agree with your hypothesis.
Otherwise I don't believe your metaphor proves much of anything.
> the days when a human can do anything to help AI there are long past; the human can only drag it down.
I disagree, currently a human guiding an AI agent is far better (more productive for less money) than an AI agent with very loose non-technical guidance and The Ralph Wiggum Loop.
I don't know how much longer humans have, but I don't think they can be cut out of the loop entirely just yet.
In chess? That ended in ~2017.
Sorry, I thought we were talking about Human + LLM agent for software development. I misinterpreted your initial statement.
At some point the whims of the CEOs don’t win. The shareholders demand more growth.
Demand from whom, if not the CEO?
Of course the demand will be directed at the CEO and they will have to provide results or be replaced.
The board, presumably
Yes, but aren't you just restating the premise in more words?
I think there are two parts to it
1. Execution of engineering task - Companies happen to focus on this mostly and this is the metric to measure easily. Using LLM tools gives a impression of improvement on this area which is what everyone is chasing
2. Growth of the Engineer - This is the one which was always a side product of company culture, individuals interest, work being done, time being spent to understand, learn from failures. A job being executed perfectly for the first time itself does not gives the opportunity for learning, no memories/experiences are built up mind of the person executing the job after a while.
The second part is the one which is under appreciated in current scheme of things since it was a by product. There is a concept of muscle memory which pretty much applies to everything.
My job improved significantly since I managed to get into their (managers) thick skulls to tell Claude to not be verbose. I'm tired of 1200-word tickets for "put the totals in the automated emails".
Up to now, manual coding has produced mountains of code, faster than humans can un derstand it! Already!
The nincompoops at the top think that if mountains of code can appear by snapping your fingers, they will be less of a problem.
If the mountain doesn't work, no problem; since it was so cheap to produce, just scrap it and snap you fingers again. The next one will be better.
Comparatively evaluating 2, 3, ... mountains of code is harder than trying to understand one.
Maybe some day, not in the short term. I've seen output of bad developers using Fable and after that I sleep safe.
A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
It's like a tool that has caused a fan out of both good and bad outcomes.
Good: you can refactor your codebase at will, throw out legacy cruft by the megabyte, improve build/CI time, and simplify ruthlessly. Not to mention kick out new features in simple and consistent ways that align with what a user actually wants.
Or you could add megabytes of vibe coded crap, solutions that add a ton of mass but don't actually solve the problem at hand (seriously!), implement abstractions that are logically inconsistent with the rest of the system, etc.
Both of these are happening right now, and I think the latter is happening at a rate far higher than the former. But our fundamental dynamics are still at play - the ball of mud is still a ball of mud, even if AI lets you make it 100x bigger. The problem just gets more entrenched.
Eventually AI will learn how to simplify code, understand coupling, etc - and hopefully it will just iron out problems as it goes. I think we're a long way away from that. But this is uncharted territory, and I don't think anyone really knows. I certainly don't hear anyone focusing on that as a target of optimization however.
My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue, and perhaps we can re-learn these lessons yet again. It kind of feels like GitHub might be the first example of this.
> My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue
If you worked in SV like I did from 2010-2020 you know that the exact opposite of that will happen. I saw hundreds of companies successfully scale out of their garbage stack such as Facebook and PHP.
The only thing that matters is the problem you’re solving and the quality of the code is irrelevant.
> A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
I had a recent experience with this. Although it was months and not weeks and there were other factors into play such as the market etc but the last company i was in had a poor engineering culture with inexperienced engineers equipped with AI output some of the lowest quality features for both customer-facing products and internal dev tools. A lot of customers churned, and im sure quality us part of the reason.
Decades ago, enterprises said, "If you're writing assembly you're doing it wrong".
That led to a reduction in knowledge of assembly in the average programmer but the people who specialize in it haven't gotten any worse at it.
The result was a generation of programmers who make useful software while very few of them understand the machine they program. You could easily make either a positive or negative value judgement about that result.
> We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
Our jobs have never been to write code. We’ve been saying for decades that LoC isn’t a rational way to measure engineering output, and now we have our chance to structurally change that system before processes re-solidify. In fact, I think the flexibility to adopt new systems and the experience and foresight to choose a path that is better than the status quo without throwing out everything we’ve learned is going to be what sets companies apart and makes individual careers over the next few years.
> we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
There’s an important criticism here IMO - the relationship between “business” and “engineering” is changing drastically. It’s going to be a challenge to set the expectation that just because Marketing was able to vibe-code a prototype in a day, actual implementation may well take weeks or months. Engineering should be considering things like security, scalability, and systems integration that aren’t a concern for Marketing - that’s why we’re being paid!
> So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
I think it’s an extremely heterogenous landscape right now. Where I work I’m struggling mostly with organization - keeping the (literal) dozens of inbound features that come in every day straight long enough to hook up an agent harness, review, validate, and deploy. At other companies the issues seem to revolve around their Agile-based processes. Or maybe it’s non-technical vibe-coders and their expectations. Or maybe it’s executive leadership flirting with AI psychosis.
Everything is in flux. It’s stressful and exciting, and I’m thankful to be around for it, even if I am in my 40s at this point and looking at the core skills I’ve built rapidly become worth exponentially less. It’s a huge opportunity for growth.
Engineers, free from writing thousands of lines of boilerplate, now have more time to review code and be more efficient in that armed with LLMs. E.g. instead of asking a question about the code and waiting for the author to respond, I can ask an LLM and get the answer almost immediately. Or, if I see some strange code being changed, I am always curious why it was there in the first place and, instead of tracing it through git blame over formatting, renaming, and moving files around dirs I can delegate this to an LLM. If I have doubts about it working in some environment I can, again, make an LLM test it out instead of asking the author etc. etc.
And no, telling Claude to implement a Jira ticket is not worth $200K/year. Checking if it has not done something stupid and correcting it when it's trying to - is.
Alternate take: How many people have known how to program in assembly for the last 30 years or so? It's been a skill on the decline. Good compilers made that happen.
// The need for ongoing friction in long-term skill formation.
The subtitle of the story tells it all.
There are some people who seek out friction. Think about an athlete or a hardcore nerd.
The best engineers are ones who were fascinated with computers and learning as kids and persued it at every opportunity. Found their own friction in other words.
For those kinds of people, friction-seeking is the constant and what LLMs did is moved the point of where the friction occurs.
For example - the best engineers I've worked with didn't necessarily have lots of experience coding in assembly because that kind of friction was no longer necessary. But they could solve hard problems (and if a problem really required assembly they could go learn it)
What I think will be hit much harder by AI is the low tier engineer. Someone who was never truly curious and committed to it, for whom it was just a job. For example a typical offshore ticket pusher kind of person. That kind of person never went out to find friction and that's the kind of thing that's never going to fly again - if I want mediocre or average, the LLMs are sufficient
I think this expresses what I've been trying to form in my mind.
I've been trying to map the LLM advancements and the current state of software development onto prior technological improvements. History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
For the people that fall in love with a single abstraction layer or don't have an interest in learning new paradigms, when their known pattern is abstracted away, they're condemned to being left behind, either unwilling or unable to adapt.
I don't think any industry is free from this, any person in any industry/profession over a period of 20 years or more has likely had to undergo massive adjustments as technology changed their field.
We're not unique, but that doesn't stop it from feeling so jarring when it happens to us
> History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
But there's no abstraction layer that ends up getting lifted. When I use a library like SDL or a standard like POSIX, I don't tend to look at the underlying implementation. Instead I work with the high level concepts that they come up with. There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
I've learned something at every abstraction layer in computing from electronics (hardware), theory of computation (software) to high level programming languages with their paradigms. Same with several domains embodied by libraries. LLM tooling is more like shamanic ritual than engineering.
> There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
This is the same “but the ai can’t even do fingers” argument all over again, and it will age just a well.
The whole vibe coding thing only really started taking hold with the release of opus 4.5. That wasn’t even a year ago. you absolutely can be quality software without looking at the code. The only thing up for debate in my mind is how big that software can get before you need to look under the hood.
But that’s dictated by current model capabilities. And I don’t see any reason to believe that they will suddenly stop improving on this front.
>and everyone knows the quality of the result.
Yeah, a bespoke program that does exactly what I need it to do, at a speed that I had forgotten was possible on computers, with customization that is an exact fit to me, at a cost that is smaller than a rounding error.
I get that LLMs struggle with the old paradigm of a single piece of software meant to serve every conceivable use case of every conceivable user, but I kinda hope that paradigm dies.
With AI you learn contributor, project, and product management.
I think coding experts who are only experts at writing code will also be hit pretty hard. Every problem given to them is solved by writing code, and only code, their way and only their way. I think people like that are having a very hard time relinquishing control regardless of the quality and correctness of the LLM output. If it didn't come from them, or conform to their conventions and style, then it's wrong. These are the people that seem to be terrific developers but then when asked to be a team lead and bring a group of developers to their level completely fail.
Curiosity is really key, and I find that LLMs diminish curiosity in people that might otherwise have it because the allure of the answer is often a lot stronger than the allure of the friction. I guess that's just the distilling of the industry over time, but that could certainly lead to a shortage of expertise.
If that is representative of your personal experience, then I'm saddened to hear it. I believe your article is an argument from incredulity [1]. You may not personally know anyone who thrives in learning from and applying knowledge with AI. But being unaware that it is even possible is not evidence that expertise will inevitably collapse.
[1]: https://en.wikipedia.org/wiki/Argument_from_incredulity
I mean if you think about it, that sounds like a miserable existence to begin with. I would not rule out that some people will use this.. uh.. disruption, to start pursuing something that actually brings them joy.
Of course, bills need to be paid and it's not all that simple. But at least there might be such a silver lining.
The snake eating it's own tail for llm software development has really been met with a shoulder shrug whenever it gets brought up. At best you might have a small cohort of developers that don't cook their brains with AI and their reward for that appears to be having to review terrible AI code written by people who have cooked their brains.
Completely unsustainable.
As a tech educator I 100% agree. LLMs are not going to become a "new compiler" where we don't have to worry about the code any more. There's a reason we trust deterministic systems.
I've been worried about this a lot, I even created an agent skill called do-i-understand that's designed for novice devs (and experienced too, because atrophy) where the LLM asks you questions about the PR you're about to submit. I've found it helps a lot: https://github.com/AnthonyPAlicea/skills/blob/main/skills/do...
One way or another, there will be a skill reckoning.
Why do you trust the non-deterministic LLM with your skill?
It absolutely might make mistakes. But the skill requires all questions to be based on lines of code with specific line numbers which the human needs to review. That's intentional to serve as a safeguard against hallucinations.
A deterministic LLM is possible. Would you trust it?
There are 2 levels of determinism that matter here IMO. Level 1, given the same input you get the same output. Level 2, you know what to change on the input to get a specific change on the output.
LLMs are currently capable of level 1, but not capable of level 2. Trust follows what is actually deterministic.
they probably mean deterministic in the sense of traditional computer programs consisting of if-else decision points and ordinary cpu logic, as opposed to numerical models (which can of course be deterministic too, as you pointed out)
Model inference is just that same logic but faster.
I think the point is determined. No-one can determine what the LLM might do.
I mean in the traditional software sense. We trust autopilots to fly planes. Would you get on a plane flown by an LLM?
Then I think I'd rephrase you: There's a reason we trust trusted systems.
:)
Even their fans agree, LLMs are inherently unreliable. The fact some people trust them regardless is due to a deep flaw in human psychology that I believe has not been significantly exploited by any previous tech. There will be tears.
It believe the most likely consequence of AI is to promote generalists: people who have a domain expertise, can work cross discipline, and has enough programming knowledge to keep the LLMs on track. I don't think 'pure' software engineers will end up being as highly valued as the last decade though I also think that will be true for other disciplines as well. Just as an example, in signal processing, it's not uncommon to have a person designing the general algorithm and another person dedicated to implementing the algorithm in the embedded system. With the quality of coding agents, it's not really necessary to have both of those people anymore. A person who is moderately experienced in both can do the job now.
I strongly agree with the concept that cognitive friction is the engine of learning.
First and foremost, it's an issue of "dependency": if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself.
A historical example that brought about a similar shift is this: when the production process moved from the craftsman's mind and hands to the Fordist factory (and the assembly line), the skill of building things shifted from human craftsmanship to anonymous, structured processes.
Bit by bit, traditional artisans lost their knowledge and "know-how." Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
The exact same thing is happening to software.
We are the (now "former") software craftsmen.
> cognitive friction is the engine of learning. I agree 100% too.
This is why in order to understand new codebase or to ramp up to new projects, I use AI to generate a textbook style reading material for me along with "verify yourself" types of exercises along the way. Then I print them on a paper and read it using a pencil/pen and take notes.
Because of my math training, I am in the habit of slowing down to read textbook style texts which helps.
Exactly the type of usage I would hope to see advocated for in the future. As I wrote, when leveraged with Socratic methods and a sort of "interactive documentation", they can really boost one's understanding. Of course, this is predicated on the individual having enough information to know the right questions to ask.
I also do this. It would be better I think to generate exact references to primary and canonical sources to avoid learning LLMisms, or to feed it these and have it ensure its material is anchored to those good references.
> "...if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself."
For a bit of extra unease, consider that these people will retain their right to vote, both in making business decisions as employees, in shareholder meetings as shareholders, and in government elections as voters, despite their atrophied reasoning abilities. Who will wind up whispering what to vote for in the ears of these reverse centaurs?
Another example is autocorrect absolutely destroying the world's ability to spell even the most basic words. Ever since mobile phones came along and removed the friction of walking to the dictionary, everywhere I look people write as if they have extremely serious brain trauma - grown adults struggling to write as well as we used to as children!
> Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
Tangential, but this is one of many reasons that I (and I suspect many others here) have taken up wood working.
theres even more recent things to learn from > https://en.wikipedia.org/wiki/Ironies_of_Automation
> Bainbridge argues that new, severe problems are caused by automating most of the work, while the human operator is responsible for tasks that can not be automated. Thus, operators will not practice skills as part of their ongoing work. Their work now also includes exhausting monitoring tasks. Thus, rather than needing less training, operators need to be trained more to be ready for the rare but crucial interventions
TBH it was already pretty bad. There is a stark difference between the best and the average in my experience. The top, say, ten percent of coders are vastly better than anyone else when it comes to anything but boilerplate glue code(which is still needed and is better done by average coders anyway).
This is speaking from my experience as a systems/c/c++ guy. If you are a js web frontend guy, python, or whatever I have no idea if this applies to you.
Yes, and it doesn't matter.
Writing code in programming languages is a skill/necessity created by us to instruct computers what we want them to do
Initially in the 60s, this was done by connecting circuits one way or another (think ENIAC). Then we devised "programmable" computers and devised a bunch of codes (computer code instructions) that abstracted away those cables.
The we created Programming Languages to further abstract away the hardware complexity, and to be able to write down our wishes in a way that is more transferable between us humans, but that is still computable by machines.
But with LLMs and neural networks, at some point these abstractions wont be necessary.
The computers will still be making computations, but the way we tell them what we want is going to evolve.
It's fascinating.
Another way to look at it is that the AI prompting is the new Programming Language or Compiler.
Like going from coding in Assembly to coding in Python; we are just writing the instructions in a simpler way.
And like going from Assembly to Python; the simpler way of writing the code makes us lose some of the deep control and efficiency of code execution that we would otherwise have; but, it comes at the benefit of being easier to create software and (hopefully) maintain that software.
Not to detract from your point, but there were software-programmable computers in the 50s. Check out Usagi Electric's videos about the Bendix G-15, where he loads software from punched tape.
This seems sadly very true. Just yesterday I was implementing some slightly tricky javascript (not my main language) on hover show n neighbours to each side, and if a deficit on either side, expand to the other side. After about 20 minutes of struggling to get the offsets just right, I succumbed to just asking an agent to do it.
I'm sure I'd still be able to do it, but was saddened that I didn't get it as quickly as I think I used to be able to. Atrophy might already be in play.
I often think back to the early GPT days before agents as maybe the last time I will deeply learn a technical subject. I think that was when I actually learned the most as I had to stay completely in the loop. I had recently started a new job and had to work with k8s for the first time. Using GPT to help me implement new services and help me diagnose and fix issues with kubectl taught me so much.
Now with agents I don't have any insights into what its doing with kubectl and would have no reason to learn how it works (other than my own curiosity).
I agree that the human no longer gains expertise, but I am not sure that will matter in the long run.
The end result is a total and complete reliance on AI providers. It's not really a secret; that's why I started the article with the direct quote from Altman. They would love to see a world where nobody is able to do any type of development work without an LLM subscription enabled...even though they also admit that these models perform best when they are steered by someone who is highly capable and knowledgeable in the first place. They're powerful, but they are also VERY fallible.
Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.
> Local AI would thwart this vision
While I agree Local AI would be a good goal, even the huge open weight models that can't be run locally are already thwarting the US AI lab's duopoly. Other companies can and do serve up these open weight models, and without having to amortize R&D they are able to serve the tokens at a pretty cheap rate.
yes, this seems sadly very true for me, too.
In some ways, it feel like I'd be "stuck" at my 2024 capabilities if I didn't push myself constantly, and I'm really glad for every hour I spent learning/applying new stuff before that, because now it feels way harder.
Most computer science programs don't teach "coding" though. There is pretty heavy emphasis on datastructures, algorithms, and system design. I don't think AI atrophies those skills / knowledge as much as the coding language and I think in many cases actually improves them, especially if you are reading the code that is produced and at least understanding the flow of it.
I agree that AI reliance is hurting engineer's language understanding, but I don't know how much we should care.
I don't think that's quite correct; every CS program I've seen involves writing code, quite a lot of it. What they don't teach is /software engineering/: working large codebases, maintaining code, dealing with legacy code, long-term collaboration, working with product managed and designers, etc. That's stuff that you mostly end up having to pickup in the real world outside school.
Writing code and being formally taught a coding languages syntax and toolchain are quite different.
I did B.S. in software engineering instead computer science in college, and there were classes that indeed taught the industry software engineering process like you mentioned.
> especially if you are reading the code that is produced and at least understanding the flow of it.
This is the key point. Should we stay vigilant and review all the code, ensuring we have a decept grasp of what it does. Or just green light everything and treat the code itself as a black box. I would hope we do the former, but many are pushing for the latter.
The issue with this in practice is that the rate at which code is produced is often substantially higher than what any human reviewer is capable of processing mentally.
Example from my workplace: pull requests that were already too long before LLMs by a few hundred or a couple of thousand lines are now several thousand lines longer and they get opened much more frequently. The primary source of these PRs is a well-respected senior team member with a ton more experience than everyone else by a measure of decades so nobody dares tell him to stop. Other team members aren’t far behind in the PR opening rate and admittedly I am one of them, but I try to mitigate the issue by doing multiple LLM reviews and anticipating when it’s ok to just approve it myself (have never had an issue in making this judgement thankfully). Part of this IMO is almost certainly because there’s now peer-pressure within the team to churn out a ton more code and that’s adding to the already clear message from leadership that we need to leverage AI to deliver things at unprecedented levels.
My point is that it’s not just that some people are pushing for greenlighting everything without due diligence, it’s also that they’re pushing others to toward that outcome with their extreme AI enthusiasm which they don’t see the ramifications of or just don’t care.
Opening large PRs is bad engineering practice, regardless of if an LLM or human wrote the code.
Would recommend your senior team look into stacked PRs, which was made to handle this scenario exactly.
> Most computer science programs don't teach "coding" though. There is pretty heavy emphasis on datastructures, algorithms, and system design
Those skills are often taught in an idealized situation like very small projects. In the real world, there are a lot more practices and techniques to develop software in a pragmatic way. Those "engineering practices" are much more useful than the above, unless you're in some specific domains.
I fall into the category of senior engineers who benefit from LLMs for all the reasons mentioned in this post. I find it's possible to agree completely with sentiments like this and still feel as if this is all written in the sand below the high tide line, and ten years from now nobody will care about this.
Horsemanship and sailing were both specialized skills of high value to society, and now they're not. But in each case there was probably a liminal period, when being an accomplished horseman or sailor was still valuable, even as motors were taking over. Eventually that period ended, as the new generations without those skills found ways to get by with cars and motorboats.
This analogy breaks down because computers aren't going away unlike horse drawn carriages or sailing vessels. Computers aren't being replaced by something. It's that fewer people will understand how they work while the efficiency and reach of computers extends into areas of life previously unreachable due to the scale of having humans build it or the refusal of humans.
I could wax philosophical about what that leads to but enough people already have.
If my horse is ‘a useful piece of software’, what are the cars and motorboats that AI coding is going to give me?
A faster horse.
I'm looking forward to having the same experience as COBOL engineers did after retirement: companies backing up the brinks truck to anyone with the skills to keep the lights on.
If you actually look at Cobol job listings, they pay like 50k and want a master's degree. The myth of them being paid really well is not true at all.
> backing up the brinks truck
now that's a phrase I have only heard in one other context: as an alleged quote from former Boston Celtics player Isaiah Thomas (who was really about 5'7 despite his official listed 5'9; yes, professional basketball player in NBA). This was around 2017 when he had just come off a career year and was soon due for a contract extension in the low 9 figures. He got injured and bounced around for a few years before eventually leaving the league, and never got that payday.
It’s commonly used in college sports where one school’s fan base argue for poaching a coach at another school. “Back up the Brinks truck” == throw money at him.
It's like I said in another article, I've spent my life in C# and JavaScript. I'm a low-level programmer now, did even have to learn C or assembly.
that would be nice, but unfortunately I think the supply vs demand situation is quite different this time
Yeah, every COBOL job paid pretty terrible relative to other types of work (like web dev).
Sure there might be a dozen individuals making over $300k but most COBOL devs live in the deep south making less than $100k for their entire careers. Fine salary, but that line of work is severely limited and the salaries do not reflect how valuable that work is.
There have always been opportunities for those who seek out the less traveled paths.
If all your peers are only accepting what LLMs tell them then you should do the opposite. Spend some of your time writing code by hand, especially when working at a new layer of the system or on new things you've never looked at before.
While the agents churn out more CRUD crap in the background spend your attention learning how everything works.
I do way more experiments with way more things by taking advantage of LLMS. When I have them do research I use that as a starting point for my own reading (but don't blindly trust them).
I find I am learning more faster.
Change is an opportunity. Use it to your advantage. Let other people while and moan while you get things done.
So far I think I have learned far more from LLMs than I've lost to them. I forget some syntax, definitely. But I now reach for a much wider range of tools that I have become familiar with because of LLMs.
So, partly agree, partly disagree.
Yeah, I have to say some of this deskilling argument sounds like Socrates complaining about the invention of writing ruining people's ability to truly own the text. I really don't mourn my lost assembly skills or my lost C skills. And I'm not missing the details of whatever web framework du jour my Claude is vibing for me on my extremely useful dataviz internal tools; and I have learned how to rein the agents in to not ruin my high-performance Go code and yet debug concurrency issues. Maybe coding qua coding is what some people find their summum bonum, but for me programming is instrumentive - to build real things in the world that do things to make something better. When I want pure virtuosity of intellectual construction, I read maths papers in arxiv.
(Not to say that the current state of the art with LLMs merits borrowing money for a multi-trillion spend on hardware that will be obsolete in 5 years, but for me that is a different issue).
The master decrying the invention that resulted in more wisdom for more people than anything prior:
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise." (quoting from https://www.historyofinformation.com/detail.php?id=3439)
I posted this elsewhere, but that story has a lot more nuance.
Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
There is an argument to be made though that writing stuff down yourself results in better retention than simply reading the same material. People still write by hand and take notes. And they still process ideas and thoughts verbally. Doing so is often more effective than going in circles with a chatbot.
Yeah. Thought in Socrates' defense I think the jury is still out on writing, in the scale of human existence it's still very new. Give it a hundred thousand years or so.
May I ask how many years experience you have?
20ish. What's your hypothesis?
It's basically laid out in the article: the reason that you are able to gain so much from LLMs is because of your decades of experience that predated their existence. Ability begets ability, but if that ability doesn't take root, these more advanced toolsets don't yield the same outcome (and are likely to do harm).
To be clear, I feel the same way as you. I have been doing this for about twenty-five years, and I have developed a keen sense of taste and judgment to know how to properly scrutinize and utilize these systems. I definitely feel like I am able to learn things faster, but that is directly correlated to having experienced a lot of friction over the years that cultivated said taste and judgment in the first place.
I imagine it's sort of similar to someone who's just getting started in mathematics, but is introduced to WolframAlpha, versus someone with decades of experience in the field and what they learn with it.
We are spending sooo much for our intellectual time and energy on discussions about alchemy-like experiments with different models, agents, harnesses, orchestrations.
And there are huge amounts of time, energy, and writing sunk into arguing about AI or trying to convince people of the danger or problems.
Sadly all of this is taking time away that could be spent on real progress and learning about coding/FP/PL/algorithms/type theory/etc.
I don’t think there’s a big societal problem here, except that the agentic programming developers may be out of a job like the rest of us. If AI continues improving it will just keep taking over whatever skills are involved in the current AI coding meta are. If it stops improving then the meta will stabilize and after 10 years everyone will have longevity in it.
I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
Depends on what you are programming. I have been using it to build things that have never existed before, that I have wanted for over a decade. Experience gives one more imagination.
Me too. I’m failing to connect the dots from your post to mine though.
> I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
I do not think anyone could do the job I do with AI without my decades of experience, is my point. For now.
I’m assuming you’re a software engineer.
The AI programming meta that keeps constantly evolving (last month it was loops) seems like a relatively trivial skill to adopt compared to learning what it’s like to do high quality work and to participate in a team, and these kinds of things. If the AI stops improving then everyone will master the AI skill set that everyone else is using, and people will acquire maturity and experience over time, like always. This is my point.
I hope that this will expose the true value of education. It's a new scale perhaps, but not a new problem. We sit in classrooms even though books and YouTube videos make available the teachings of far more illustrious teachers than the one in front of us. We watch things written out agonizingly slowly in chalk. We learned to do algebra and calculus even though we had our TI-86s and Mathematica. We learn in Haskell and Scheme even though those are rarely used later. We write toy compilers and write essays and repeat experiments with known outcomes in labs. We do it to learn how to think, how to build. We just have to be even more deliberate about it now.
My personal antidote is to write unassisted Zig in my spare time (though C would be fine as well). While I can't back with data I certainly feel sharper.
I think we now entering this state where code is cheap but good well made programs will be rarer then ever, a lot of crap can be made and a lot of programs made without ever thinking the problem through properly will occur.
It might be like how an archeologist since the invention of plastic can date the period of the soil as post plastic, if one could cut open the software stack of the coming systems that will be built in the near future one would could data that code as post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
I am not Anti-AI but I think it's going to be interesting and I am surprised at how bad a program ends up when one tries to "vibe code", though often ends up working, though in isolation and when used with discipline (that the tools themselves psychologically make it harder to do) can produce some very good code and being able to use loops to solve difficult problems or problems that simply would have require banging ones head against the problem many times is very profitable.
How many programs can dispense with 50% of their complexity by just offering an API for LLMs to execute?
By way of example I use Kdenlive 90% less and just use Codex to issue the ffmpeg CLI commands directly when I want to crop/format video on my PC.
All those crazy complicated CLIs are now super useful for LLMs!
Yes every time I tried to understand ffmpeg I got a little of it for a little bit and then lost it by 2 days or so. Just getting LLM's to do their thing for ffmpeg is great. the other one is socat an utility I used before but LLM's can do magic with it.
> JetBrains, a major player in developer tools, recently completed a study of junior and novice developers
The article links to this as the study: https://dl.acm.org/doi/epdf/10.1145/3632620.3671116. It doesn't appear to be related to JetBrains in any way?
I originally found the paper through this video:
https://www.youtube.com/watch?v=HTUh0OO6Kmo
The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!
Would you work for less money and use AI ? Or would you step out - professionally - of the industry and program for passion and fun ?
I think that is a question we are all mulling about these years.
Our expertise has been packaged into a 200$ product and the competition has become ourselves.
I'm still waiting to see how LLMs have made the world a better place because it honestly feels like its done the opposite.
I think a lot of good might come from medical sciences or chemistry, where LLMs will be used in coming up with new medicines, methods or synthesis.
I can totally see where that could be the case. I'm sure there have been breakthroughs already, but it's not mainstream news. The mountain of AI slop is all over the mainstream though. You have to wonder why.
They haven't. They have made the world significantly worse.
> This applied friction is directly what builds "developer intuition" (or "taste"). The Germans have a great word for this: Fingerspitzengefühl (fingertip feeling). It’s the muscle memory that triggers when a developer looks at something and thinks, “yeah...this is probably going to cause problems.”
No, we do not have that word for that.
Yes, we have that word. No, it does not mean that.
Any LLM proofreading could've told the author that, for that matter.
__
What Fingerspitzengefühl actually means could be described as "tact", a precise approach to something and general attention to detail that leads to success. Or.. the lack of all of that leading to blunt failure.
__
I would of course comment with more Fingerspitzengefühl for the emotional needs of the author if the writing wasn't just a sales funnel for their courses and whatever else (+ posted by them themselves.)
Hobbyist programming expertise unaffected
Genuine curiosity in learning how stuff works, not necessarily for financial gain, e.g., from a salary or using web to deliver ads
No real "pressure" for hobbyist to use "AI"
IMO as end user, some of the best software available was originally written by a single unpaid author, not salaried teams
Some of the worst software ever written was produced by overpaid teams
Hobbyists may use relatively old, handwritten code to learn from
That's not necessarily bad. Software quality has declined over time
It appears to be a pattern where I work "expert novice" eventually causes serious incident but is unable to root-cause on their own.
They've become a meat proxy[0] prompting LLM to root cause for them and the ability to fix is capped by LLMs ability, not the engineer's.
I've tried to convince management before this happened and yet here we are.
[0] https://gruhn.me/blog/2026-08-03/
This happened before with “real” engineering. Engineers a hundred years ago used to be proficient with slide rules and mental math and 3D assembly by hand. Products designed in that era often lasted a hundred years, like Singer sewing machines and Lada cars and 500 year old churches. Now things aren’t built like they used to be, and few people can rebuild civilization by hand like they did. But the world moves on fine!
It's different though because for structural engineering finite element models (set up correctly) will give you a deterministic, mathematically correct answer.
In fact FEM is arguably more deterministic than the old school pen and paper methods (yield line analysis etc.) which often rely more on correct judgement of the structural engineer doing the analysis.
So I don't think the analogy quite holds
I do understand the sentiment of applying "constraints" over AI usage to certain scopes of our development work.
This has an issue in practice.
As an example, Function Programming (FP) has a high constraint on programming, where some don't even have loops (gotta use recursions all the way), and almost no state mutation (Exlixir I believe have a mutable state? forgot).
FP sounds great with all those constraints. But what about the adoption?
It's hard to switch your mindset, and not as natural, thus long leraning curve, and not as wide adopted <- productivity goes down as all others need to know and use it well. That's why still imperative and OOP languages are ruling the world.
Same for these constraints. The author promoting using AIs only to subsets (no coding? wtf) is something the majority won't follow. Yes, the constraints sound great, but at what cost? By the time one learns everything, and everyone moves along with AI building stuff fast and cheap, they will be left behind.
---
I do not like the author trying to make his points authoritative by using Halo effect using quotes from the industry and could have had been taken out of context (That's where AIs mess up the most, they only get chunks of text and lost context of before and after exerpts).
Your inclusion of FP is interesting in an unintended way: LLMs actually thrive in a highly constrained system like you describe. "It's hard to switch your mindset" - isn't nearly as true for an LLM, it can usually adopt those languages quite readily.
LLMs don't particularly care how hard the type system or borrow checker is (as long as it has good error messages that can guide it to a solution), they can grind away at these things, and what you describe as a negative actually becomes a strong positive. I think we'll see strongly typed systems plus some level of formal verification become popular as an AI-preferred language.
When the burden of review is lowered because you have purely functional building blocks, with lots of formally verified chunks - people will choose that simply out of (productive, beneficial) laziness. Not to mention we're in a ram crunch, and being able to run your stuff far cheaper has an immediate financial incentive.
In my company the knowledge is starting to be devalued.
People are distrusting whole swaths of concepts because they do not understand how things actually work.
Like going away from zerotrust architecture to holding open a long polling connection.
At least I seem to remember that long polling wasn’t that great, but i start to even doubt myself.
Anyone wondering how the Roman Empire was wiped out? I guess we will find out soon
Saw a quote comparing pre-LLM code to pre-WWII steel, low-background steel, essentially.
No matter how robust my code is and however many eventualities I considered, it certainly feels like AI can find and address issues.
> it certainly feels like AI can find and address issues.
LLM chatbots are good at making you feel good. Facts don't care about your feelings.
Substantiveness of this article notwithstanding, I do want to point out that the author also runs (or is planning on running) a programming course which to me may be a slight conflict of interest. I think the author may altogether be well-intentioned, but it IS something that I'm going to keep in the back of my mind as I consume this content.
> LLMs can detect patterns at a scale that no human ever could, but patterns only get you so far.
This is asserted without evidence, and from a scientific standpoint, unjustified.
First, "detect patterns" makes it sound like a classification task, "is this a picture of a cat". But LLMs transform the input.
For example, an LLM can translate text between two languages while properly handling the names of the people described, no matter what those names are. That shows they are representing the text in a somewhat abstract way, that they can perform operations on that representation, and also convert it to useful output.
And, what I just described is the most general form of information processing algorithm. Science is not aware of any limitations in principle on such systems.
I am not saying LLMs have no limits, but "they only recognize patterns, and that is a true limit" is not a good argument.
Writing code was never the job. It was a small part of the job that can now be delegated to AI.
Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage. You know `Engineering`. Mechanical and Electrical engineers didn't go away because of CAD.
> Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage.
Besides your first point, the rest can be delegated to AI as well. And you don't need to be a "software engineer" to spec out requirements and user needs.
Literally all those skills are the result of decades of friction, challenge, and deep domain knowledge...all of which are circumvented when LLMs are utilized early in the process.
CAD is also useless to Mechanical and Electrical engineers who didn't learn fundamental the skills that it streamlines.
> Mechanical and Electrical engineers didn't go away because of CAD.
that is a very good point! I'm going to use it in the future.
Does anyone else feel like the web has become so "readable" it is illegible?
I have such a hard time quickly reading pages like TFA. Headings, bullets, line-spacing, width, link hover animations, even the font. Even reader mode defaults aren't great for me.
Compared to, for example, https://www.paulgraham.com/best.html
I think this is more “gray” than this. I feel like I can rip through ideas more quickly then ever before and as a result get a lot better at designing systems and (for my work) get a lot better at designing data/model experiments.
But I empathize with the author. I think the frustration comes from the fact that other folks in your organization use these tools to “get by” more effectively then before. In the past, a disinterested engineer might ship less under the umbrage of a difficult engineering problem. With the pressure to “ship more” now that there’s AI, the same engineer is using AI to spam PRs to prove they’re working, even the quality of work is low (because quality of thought is low).
Yes, but it's also building other skills. There's no friction to learning something new anymore, and that's really quite amazing.
E.g. I've gotten pretty good at git thanks to AI because it forced me into more complex workflows. I still can't remember the commands but I really grok the concepts and terminology on a far deeper level than before, where I made a conscious effort to stay on the happy path lest I get into some odd state I don't understand. Pretty sure those "odd states" would seem quite clear to me now.
It's surprisingly late for him to notice. But wait til you think through the consequences of the already collapsing human capacity for extended written composition, which is orders of magnitude more significant.
I've, more than once, gone long spans of time doing purely management and architecture level work, then returned to programming. A little rusty? Sure. But I never forgot how to program or lost the skills that actually matter. Seems mostly like a non-issue for people already in the field.
The real issue will be with people that never learn it in the first place.
And even that isn't so bad, learning programming isn't some insurmountable hurdle. It took me less than 6 months from no knowledge to my first job.
> It took me less than 6 months from no knowledge to my first job.
I agreed with you up until this sentence. Entry-level devs with 6 months of require an enormous amount of time and mentoring until they're able to really perform well. I think the greater concern being expressed is not will people be able to type syntactically correct code?, it's will people be able to write software well?.
Wasn't everyone making these arguments for the last 50 years with each wave of advances and convenience? "Real programmers use [current_tech - 1]."
I mean, they were right, then, too. My meaning is that expertise already collapsed several decades ago.
--
One of my favorite talks of all time:
Jonathan Blow, Preventing the Collapse of Civilization (2019)
https://www.youtube.com/watch?v=ZSRHeXYDLko
Yep.
In my experience human design -> agent code -> agent review -> release -> agent observer, does not (at all!) work. One absolutely has to be understand what is going on, how the system works, how the H/W works, etc, to catch agent errors.
The proposition of us losing those skills is terrifying.
Maybe AI will get better and we truly won't need a human in the loop. Then please also have all PRs signed by whatever model wrote them, and do not come to me for trouble-shooting.
Fair enough, let's say that happens.
But what we're gonna manage the codebase in, then? What kind of language?
Will it be a natural language, or a formal one?
I feel that "coding" cannot collapse. Coding is just translation from natural to formal language. Somebody needs to write the specs. And writing/maintaining them in natural language brings a lot of fun - shifting interpretation, inconsistency, missing specification, etc.
I don't think it's progress for the field of SW engineering. But at least more "shareholder value" will be created.
It's pretty simple.
You treat the AI like a junior.
You make it write pseudo code in the tickets.
Then you make it write code in the tickets for the tricky stuff.
Have it reference relevant documentation/APIs.
Then there is no lost knowledge.
(Still, sometimes it sneaks in some "helpful" belt and suspenders, but for the most part, I know everything that is going on in the code base with this method.)
Coders should be treated as as well as as other software users. Tools like Claude code, Antigravity, and others are applying AI to solving that problem.
I know how to create a multi year P&L projection from scratch. It gives me some insight into how much capital a startup will need to get to breakeven. A good AI tool that creates that same spreadsheet in minutes should also be able to explain why different businesses have different shaped cash flow troughs. I don't think people using tools like that is bad even if they don't gain deep insight.
Sewing expertise is going to collapse from sewing machine reliance
At some point it won’t be realistic to ask a human to solve a technical problem any more. I’m not suggesting it will happen soon but I can at least imagine that day coming. Whereas before modern “AI”, that possibility wasn’t even on my radar.
Something I noticed trying to learn rust through models, they're inherently tuned to just finish everything with even the slightest provocation so it's very hard to use them to actually learn anything, they kinda just want to do everything themselves.
just add to your prompt or to an agents.md "don't write any code, just present me the solution so i can learn from it". that should do it.
This just feels like someone whining about how ORMs result in $#1tty data access code because lazy programmers don't take the time to learn how to use a database.
When that happens, (developer thinks using an ORM means they don't need to know how to use a database,) the project eventually fails. It's the same thing with AI; just like companies learned that they need to make sure developers actually know how to use a database, companies will learn that their engineers actually need to know how to program.
A list of things, that were supposed to lead to our intellectual demise:
- Writing (starting from Platos Phaedrus)
- Printed books
- Photography
- Typewriters, keyboards
- Telephones
- Recorded music
- Television
- Wikipedia
- GPS navigation
- Smartphones
- Automation in aviation, medicine, industries
If current trends are anything to go by, who's to say that these aren't stepping stones towards it?
Dear lord
Speaking on a handful of these that I'm familiar enough with to opine on:
- Photography hasn't erased other visual arts. Both require a separate skillset; both have their uses and their enthusiasts. Photography benefits from having a lower skill floor, in my experience doing it, and this in turn has led to commercial photography eclipsing commercial painting (barring things like painting buildings) as a viable business model, largely.
- Television is arguably not a net positive in society (I won't say net negative yet, but there are some reasonable arguments for that position). It enables a form of passive consumption that allows bad arguments to come through as persuasive through the gradual (on the scale of months and years) numbing of critical appraisal of the media being consumed. - Smartphones arguably reproduce and amplify this problem, and tack on others: they make it increasingly easy to dissociate from daily life, and make it easier to continue consuming content from actors increasingly interested in steering conversations in one direction or another (on the most benign end, it's influencers trying to get you to buy things, which obviously has negative economic impacts on an individual scale; on the more overtly harmful end, they serve as a hardware channel for less-regulated or unregulated extreme and hateful content). - In aviation, the first generation of (relatively modern) automation did lead to a high number of accidents and loss of life. American Airlines flight 965, in 1995, is a well-studied case of an over-reliance on flight automation before it was widely understood among pilots what the limits of these early-modern incarnations of autoflight were. I think that last example (and many aviation accidents) are an interesting example to study in the context of software. Aviation's extremely and conspicuously responsive to accidents, often requiring procedural and hardware changes as a result of these. They can be as simple as a software patch, or as complex as the redesign of a component because a human factors analysis found that it could be confusing under stress.In software, we like to think that we're a serious industry, sometimes. We spent much of the 2010s going in what I found to be a good direction: the development of IaC, the popularization and standardization of tools like Git, registries for vulnerabilities like the NVD, and even the development of languages like Rust which address historical technical shortfalls of high-performance, low-level languages like C; but we always fell short, industry-wide, on matching these toolchain upgrades with technically-rigorous procedures.
For every borrow checker, there's vibe coding.
Hand weaving expertise has also collapsed.
Ever since I was a teenager, I started to develop cyclical bouts of existential crises.
While I have been feeling better for quite some time, I feel another starting to brew. While I was never one of the fabled 10x or rockstar developers, I still derived a lot of enjoyment and and sense of purpose from my work.
With the utility of LLMs becoming more ubiquitous by the day, what purpose can I provide anymore? I struggled greatly to get where I am today, and while it might not be impressive to any of my fellow programmers, I am still proud to be a programmer.
I obtained a degree in CS because I found the entire field to be fascinating. I had great passion for learning many of the subjects. What I learned in my degree changed my life forever. I seriously went from one of the, "What's the point in learning most math if you never use it?" to being absolutely infatuated with the subject. (I'm still bad at math, but that does not detract from my pleasure).
With the utmost certainty, I did not push through all my struggles just to prompt LLMs all day. Maybe's its my insecurities manifesting, but I want people to say, "You did a great job" and not "Claude did a great job." I understand the Buddhist were likely correct. My suffering really does come from my attachment, but at the same time, I do not want to let it go.
I have no opposition to using LLMs as tools. I think a fine balance can be achieved. LLMs can even be exceptional learning tools when used appropriately.
If programming is just a means to an end to you, then you will receive no ill judgements from me. We all have different motives in life. But programming has always meant something more to me.
look to past examples. when the greeks developed writing, people complained that the youth were losing the ability to memorize much content. and that is true, but with writing they were able to do better still and that old skill -- though useful -- was lost.
Plato wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse.
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
I agree the tradeoffs are real but consider it obvious that reading and writing were in net a boon and I also expect the same for AI / programming. Of course having some understanding is still required in both cases to make good use of the tools.
This seems right to me, but I think that it's likely that we haven't fully figured out how to integrate AI into coding imo. I think there is user side thing that we haven’t fully worked out yet.
I believe it. I haven't written code since March.
And so what? How do you think people are going to be working on software in 10 years. They're going to use coding agents and harnesses that work much better than the current ones (although I don't think we're getting anything but incremental improvement from here on out). They will run on commodity hardware. I first got paid for writing code in 1999, I've worked at publicly traded software companies and failed startups, in every language out there. And if LLMs "atrophy" the expertise I built up over those years, then so what? What is lost, exactly? I stopped programming in C++ about 10 years ago when Rust seemed to get good. I no longer remember a lot of the specialized knowledge I had in C++. My life goes on. I am now becoming an expert at building software with agents. The problem is still the same, you still have to apply engineering principles. You just don't have to write the code.
While true, I'm not sure it matters.
It kinda seems like wishful thinking on our part as software engineers that one day we'll be able to go "Hah, miss me now?" but that just isn't the writing that I see on the wall.
Over the past year, I've AI-generated two large pieces of software over 2000 commits without having much depth in either domain, and my ability to deliver value as a veteran software engineer has only shrunk over the months or it can be trivially extracted into a reusable markdown file like "design bar: ensure things 'by construction' where possible", a lesson I learned viscerally over decades.
> without having much depth in either domain
I don't mean to pick on you, but to present this in the service of this kind of fatalism... you know the inevitable follow-up question, right?
Is it: "How do you know the solution is good?"
Because I can still reason and dig into things and ask questions and have the domain explained to me out of curiosity, the same way I could build anything that was originally out of my depth.
But what I've noticed is that the need for any corrective power has gone or is going to nil and I'm mainly doing directional work. And even then, I can constantly have sota models "rank the top options and recommend one", and it's almost always the right way forward.
I used to read every line of every plan that LLMs generated, but over time I've realized I have less and less to correct or, more impressively, the LLM had foresights I never considered. The same thing happens with implementation.
Periodically I can spawn a bunch of agents to evaluate the system adversarially to find improvements, and the findings have been so good that it's evident that soon I can just automate that too.
In other words, the skills I need to excel here are more curiosity and patience than tech expertise -- the things that got me into software engineering in the first place since that was the only way to build things.
You got it. You may well be right, but from my standpoint it's hard to see how you'd know if you weren't. Yeah, the models fill in vagueness with detail sourced from "what's likely and helpful". (In one of my particular domains, that's "use SHAP values for everything everywhere".) Yeah, that detail's increasingly unlikely to be "wrong" in a way that you could explicitly argue. Is it what an expert would come up with? Is it the best it could be? Is there something else that we would have done had we been forced to think it through? Will it ever be _surprising_?
I know, all irrelevant little philosophical roadbumps. The artifact satisfies the buyer. I think I'm going to have to step away from this industry pretty soon, at least for a while.
As always, the innovators will thrive and become the new experts in the field. Everyone else will die out.
It's impossible to buy the FUD that expertise will somehow vanish because people use AI. People come in a variety of capabilities, challenges, and personal interests. Expertise varies from person to person. Those with personality traits that are more likely to take risks and persist against adversity will be the innovators and experts. We've seen this played out countless times.
You have to ignore everything you know intrinsically but can't articulate about innovation, technology, and evolution to drink this FUD Kool-Aid.
I personally write lots of code manually and use AI to review it.
Sounds like a good compromise, if not even positive from many points of view.
Of course it will. Doesn't matter though. Coding expertise is no longer valuable.
What is valuable now is productivity. Which in today's day and age is the ability to quickly and efficiently drive AI.
Hence the never-ending growth and spewing out of tooling: harnesses, TUIs, multiplexers, agent frameworks, software "factories", etc, etc.
Software engineers are chasing opportunity and expertise in the only thing that's left.
One interesting feature of vibe coding is that it allows you to write code in specialized areas where you have no expert knowledge. That makes the code effectively impossible to review in any useful way.
For example, I have a vibe coded Python monstrosity which I use for stitching together scans of large format negatives. The enlarging lens that I use for scanning has strong chromatic aberration in the red channel but is otherwise very sharp†. I got Codex to figure out some way of using the green and blue channels to sharpen the luminance while preserving the color information from the red channel. I sort of vaguely understand how this works, but not nearly well enough to review all the horrid numpy code that implements it.
For my use case, there is really no reason to look at the code, as I am only interested in the output. As disconcerting as it is, I feel that this is going to become more and more normalized. Looking at the code that your LLM generates will become like looking at the assembly output of your C compiler.
† This makes a lot of sense, if you think about the color of a darkroom safelight...
I think the hope of the business end is that coding expertise will at some point no longer be needed at all.
I used to think this was a problem. But now? I don’t think it matters.
Why? Because people are going to get so much more ambitious about the stuff they build with AI, that code expertise for the stuff they are building is already scarce anyway. People will choose difficult and less popular languages, and will replace open source packages with stuff built entirely in-house by AI.
Going forward, there is no choice except for AI to fully replace the need for code expertise. Depending on humans won’t be scalable.
Do we even need to get LLMs to create code in the end then? Can we imagine a future where the thing simply does it all, you deploy the LLM like a docker image and have it process all requests directly. Sounds far fetched today, coming to your cluster in five years.
In theory yes, but you would need LLMs that basically have frontier level capabilities but with zero latency, meaning you enter a prompt and as you type the prompt is regenerating instantly like some kind of autocomplete.
I think that would feel like crack for some people who are addicted to using or building stuff with LLMs.
Very few people know how to hand set typography anymore, but many of the conventions still exist and people don't have to understand the origin of a lot of them to use them properly.
I'm seeing it happen in real time. I've seen several folks claim they "vibe-coded" something and upon review it's some seriously low quality garbage. Unfortunately it's just "functional enough" where it's encouraging the one who talked to the robot to continue thinking they can code for some reason.
It is important to ensure that any LLM code is named and shamed.
> It is important to ensure that any LLM code is named and shamed.
You need to take it further.
We need to shame and mock the people (read: delusional imbeciles) who think this is acceptable if we want it to stop.
AI users need to fear for their careers and reputations if we want to get software development back on the right track.
The industry has always had niches where quality is really important, such as NASA where you install your product in a multi-billion-dollar, one-of-a-kind robot and launch it into space. However, for the most part all that matters is that it roughly does what it advertises to do and ships as quickly as possible. LLMs are ideally suited to the latter form of software and the need for humans to ever see or think about the code will drop to zero soon, if it hasn't already.
> AI models lack judgment
Very interesting is how divisive is this statement. In that almost every response is either hard agree or hard disagree.
Does agreement come from observer judgement, or lack thereof?
We're all posers - unless you can implement square root in machine code, you're not a Real Programmer™
Coding expertise was on a downward trend way before AI arrived. Does nobody remember how most "engineers" were literally copying and pasting snippets from StackOverflow they didn't understand, and the snippets usually had tons of bugs? How many of you know Assembly well? How many of you can decompile a program, or write your own compiler or VM? How many of you could pass a computer science course (that you didn't take at Uni)?
We've been hearing about the demise of skills since before the invention of the loom. The only skills we've truly lost are for things nobody uses or makes anymore. What actually happens is, we develop a new tool that's better than the old tool, and we get good at using the new tool. We then have a class of people whose job it is to maintain the new tools. We don't need that many of them because the whole point is to eliminate that labor.
Your jobs as software engineers are going away, it's that simple. Your new job is to use the AI tools to make products. Somebody else's job will be to make the AI a better software engineer.
Coding expertise is going to collapse from compiler reliance (1985 Turbo Pascal detractors)
Do you think LLMs and compilers have a lot of overlap?
In an abstract sense, yes. Compilers let programmers work at a higher level of abstraction. LLMs bring it to an even higher level, though the abstraction is much leakier.
For the time being, a good understanding of programming and domain knowledge gets you much farther than without it. There might not be room for as many (professional) programmers as before, though.
Almost nobody knows how to program in assembly but we're doing fine.
That's because the institutional knowledge was passed along through the act of coding. Skills that I learned in Assembly and FORTRAN are applicable to even modern programming languages...but not to "natural language."
It really wasn't. Ask the average young coder these days how many registers their CPU has, or even the difference between the stack and the heap.
I don't agree. Certain optimization skills are learned on the job if people work in specific areas of compilers or performance.
Also the reality is that modern CPU microarchitectures are so different from what old-school assembly programmers used that their supposed optimizations are likely to be counterproductive.
I'm also concerned about people asking an opaque machine for an answer and stopping there instead of going to different sources, evaluating their credibility, exercising their own ability to construct their own worldview from disparate facts and opinions.
Flagged for the expressing the most obvious and legitimate concern about LLM-backed "research".
"I use AI to generate a summary which I print out and read slowly every time I use a new codebase" is the new propaganda line pushed by Anthropic and OpenAI. They cannot tolerate this rebuttal.
Going to? It already is.
It may bifurcate the average skill from the skill of figuring out new solutions.
Yeah I'm not convinced people are just going to give up solving problems
Are engineers still making 100s of thousands a year still just for vibe coding while a nurse and other skilled / highly trained medical professionals working their butt off make half of that?
Do we think there is going to be a correction for the vibe coders?
Pardon I know such question is controversial but thinking ahead.
There's something weird about nurse pay where it doesn't seem to raise even during large shortages of qualified nurses. I think it's artificially lowered by hospital admin, with temporary traveling nurses acting as "scabs" to avoid the normal wage increasing
It is weird how it works with anyone there but the doctors... Do they have some type of cartel or mafia that nurses and other staff don't have?
They do, it’s called the AMA, American Medical Association. Very tight control of who can be a doctor, how many new doctors per year, etc. which ensures salaries stay high. They also fight against other roles like nurses doing too much doctor-like work, helping doctor pay while harming nursing pay.
At least where I live there’s a shortage of doctors so you can’t negotiate with them . There’s a shortage of nurses too, but if you lose 1 nurse because they’re striking it’s bad but things don’t collapse. With 1 fewer doctor who also knows and is allowed to do things nurse don’t / can’t things get really bad. They also have access to power, are often elected, and are respected.
Nurses need to unionize I think and strike to get something .
The Labor Theory of Value was defeated decades ago, get with the times
Unfortunately, we haven't found a better alternative.
I'm solving harder and harder problems for real users/customers than I was a year ago and our sales are increasing as a result.
Are you saving lives?
Is it my job to make sure that nurses are paid more than I am?
I'll readily admit that they deserve it, I'm a lazy dickhead and they help people, but I think your blame is misplaced if you're insinuating that software engineers are to blame for nurses being paid less than ourselves.
While it depends on the market, job, and hours, but the bedside nurses I know make in the low $100k range. Typical shift structure at the hospital is three 12 hour shifts in a row and then four days off. Most of the time, they are union jobs, so they receive many benefits from collective bargaining such as increased holiday and overtime pay in the range of 1.5-2x normal rate. A nurse practitioner tends to make something in the 180-220k realm.
Again, to be clear, this is highly dependent on the market, job, and hours and one needs to understand the difference between a CNA, RN, NP, etc. However, it's a regulated typically union job that's very secure and in high demand. It probably pays more than what you expect.
Yes, they are still making that much. I am. I don't fully understand why yet but I think it is because of a few reasons:
1. We still need to keep the lights on. If things go wrong, which they still do, humans have to debug it.
2. Jevon's paradox: now the expectation is that we ship stuff faster and there is still a LOT of stuff to do, not less.
3. We are still in the very early days of this and it is too soon for corporations to make a dramatic call like laying off 80% of engineering. We just don't know how that would look like.
I want a world where teachers on average make more than software engineers. If we figure that out, everything else will be fine in the long run.
I think nurses in the Bay Area make like 200k a year
Don't you worry about salaries of any trained medical professionals:)
The effort/pay ratio never made sense though.
"Assembly expertise is going to collapse from compiler reliance".
Did this not happen?
It did, it’s just a question of “would it have been worth preventing compiler adoption/usage to attempt to preserve assembly knowledge”
Ah yes. Selling courses is the oldest grift in town, which is what the author here is doing.
To be fair nobody codes in ASM and things are fine for decades.
ASM to a higher level language is a jump to a clearly defined higher level language through a deterministic compile step. The jump away to LLMs is something completely different.
ASM is still used in some areas, such as:
1. function trampolines, such as in libraries like ATL (Microsoft's COM template library);
2. hand-optimized encoders/decoders for things like ffmpeg;
3. niche applications such as Ben Eater's 6502 breadboard computer (using a modified WozMon and Microsoft BASIC);
4. reading ASM output from C/C++ code in things like the GodBolt site for performance comparisons/analysis;
5. reverse engineering old DOS games;
6. people working on things like compilers (generating optimized assembly), JIT engines, or other code generation -- though this will typically be machine code, possibly with assembly instruction comments;
and other performance critical places where assembly can speed up that part of the code.
Absolutely, by 1% of developers. So analogy would be that those 1% would understand the code fully in the coming years.
Except sloppy experts.
Keep in mind also that LLMs are currenctly heavily subsidized. Once VCs money are gonna run out, you will see the real price, and your 200K per year dev is probably nothing.
A $1200 GPU is all a reasonably capable engineer needs for each work stream. The cloud prices are a scam.
I learned BASIC, Pascal and x86 Assembly language by the age of 14-15. None of which I remember (nor care to). The strict instructions (aka language) we gave the compiler was not the point. Those experiences taught me how to think logically, and over time develop a taste/structure for how software should be built, as well as foot guns to avoid. I don't know how you teach that without coding, but the "coding expertise" was always a means to an end. LLMs are just another abstraction layer above programming languages, machine language, etc.
> I don't know how you teach that without coding
That's the neat part: you don't!
Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:
https://www.youtube.com/watch?v=RxxxGkFIUJ0&t=1356s
Robert Martin has caused me so much frustration over the years, but he's spot on. "Young people with power tools tend to lose fingers"...I like that!
In contrast with Pascal, I could see in my mind the PDP-11 instructions behind my C code. By the time I started running code on x86, there were debuggers that could step through source code, and I didn't really want to think like an i386.
> LLMs are just another abstraction layer above programming languages, machine language, etc.
Programming languages are (largely) deterministic. LLMs are not. The boundary between an abstract concept and something executable sits comfortably in the mind of the programmer in non-LLM contexts. This boundary shifts when using an LLM and becomes shared between the programmer and the matrix multiplication machine.
The "layer of abstraction" analogy is therefore technically true but essentially false. "Just another" papers over how seismic of a paradigm shift all of this is.
Sure, you're right - at the end of the day, LLMs are non-deterministic code generators. I certainly wouldn't trust one for a medical device manufacturer's software. For line of business apps it's totally adequate. And to be clear, I don't love it. I'm just coming to terms with the fact that these LLMs have gotten good enough, that with enough instruction and guidance, they can generate code/features at speeds we can only dream of if we were to code them by hand.
We don't teach kids math with access to calculators, and so on. Making students code by hand at first is a very moderate proposition
Yeah I was taught coding in a fake programming language so it couldn't be compiled or run. It really was "by hand" coding since we had written tests.
That's the whole scam. It's a ransom with more steps.
First they fuck up your codebase with so much slop that no human does (or could) understand it.
Now if you want to keep making bug fixes, you're on the hook for whatever Anthropic wants to charge you.
The solution is to make AI developers the scapegoat. Fire the people at the top of the AI leaderboards, with malice. Otherwise, lazy AI-addict developers are going to destroy what remains of your company's IP.
"Punching card expersite is going to collapse from interactive terminals reliance"
And math skills are going to collapse from calculator reliance.
Actually, they have for a large majority of the population. And fun fact: that debate is still going on, and the Mathematical Association of America (MAA) enacted a rule prohibiting calculator use in AMC.
https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...
Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...
> Math fosters critical thinking
I kinda agree, but I also kinda disagree.
My question would be if "math" is the right tool for that. It can achieve that as a side-effect, but how we express it and talk about it is full of legacy cruft, cargo-culting and general bullshit that makes it not exactly snap into place for all kinds of brain wirings.
Math I believe is not an end in itself. Maybe with that perspective, solutions that satisfy our needs there can be found
Because the 1890s and 1930s were famous for their population wide critical thinking, of course. Good heavens, what do they teach in the schools these days.
I mean you may be trying to make a point but the counterpoint is Einstein, Bohrs, Planck, Shannon, Turing, Von Nuemann, Feynman, Bohr and most of the other big names all came out during that time.
In contrast since then we've had almost not noteable physicists or Computer Scientists that I'm aware, we've got a couple, Hawking and Knuth. But apart from that I don't know if there is a single scientist I could point out today that has revolutionized our understanding or even done any real breakthrough work in the field of Physics since Einsteins era.
We have gotten better at working as a group. The standard model is a tour de force. It was a collaborative effort over decades but is a lot more complex than anything Einstein did. To some extent we haven’t gotten new physics since then because of reality - we have built higher energy experiments but haven’t got new results.
But my reply was I guess implicitly assuming the post I was replying to was talking about the median level of critical thinking. Your reply is talking about the peaks of creative accomplishment which is not what I was addressing.
Not a good argument. Those were big names because they were (co)founders of major fields. Physics and CS aren’t like fashion where you can come out with a new color or design every six months.
You also seem to be restricting yourself to pop science knowledge of who’s made breakthroughs. That’s because our collective understanding of physics has become much more detailed, to the point where it starts to lose the ability to be meaningfully comprehensible to the average layperson.
Consider AdS/CFT, for example: https://en.wikipedia.org/wiki/AdS/CFT_correspondence
There’ve been many major developments just in that area since the early 1970s, to through the late 90s all the way to 2019 (the island formula), with many major contributors like Bekenstein, ‘t Hooft, Susskind, Maldacena, and many less well-known names.
If you resurrected some of the scientists you mentioned, they’d have a lot of work to do to catch up, and making field-changing major breakthroughs would be just as difficult for them as for anyone else alive today.
Something similar is true in CS. There’s been plenty of progress in all fields of CS, but similar to the situation with physics, its nature is not as simple as what the founders of the field were dealing with, so there’s less awareness of it outside the field. Just to pick one small example, Conor McBride’s insight that the derivative of a regular type is the type of its one-hole context ( https://news.ycombinator.com/item?id=34358663 ) is beautiful and kind of amazing, but not a result that you’re going to hear much about outside of PhD programs and the like. CS is chock-full of such work. The entire field of programming language theory has come a long, long way since Turing and Von Neumann’s time, to pick a broader example.
> we've been in a critical thinking deficit for decades and decades.
Compared to when?
I have heard stories of cashiers being incapable of giving correct change. So yes it has happened.
A more apt analogy is the diminishing understanding of assembly language and machine instruction sets. I love programming and something got lost in both transitions, first to high level languages and now to coding agents. I am quite sure some contexts will require programming by hand for a long while as there are still people who are writing amd64/avx512/sass by hand. I hope in time we will find new ways to enjoy solving problems with this new programming modality. By now, I am pretty sure it is here to stay and evolve from its current crude form.
> the diminishing understanding of assembly language and machine instruction sets
This is true and nobody complains that GCC or LLVM is making programmers dumber. They are code-generation tools from higher-level source code. Much like LLMs are code-generation tools for higher-level language/prompting.
Math skills had never anything to do with arithmetic. Maybe Ramanujan discovered a couple of theorems by dealing with numbers themselves. That is it.
Online discourse though is collapsing under dumb AI talking points.
Man, Usenet had the best online discourse - hardly any angry personal flaming because people dared to disagree. And Twitter, wow that was just pure rationality. slashdot - practically a masters program.
Now, they all did have gems of interaction that would could find by sifting thru the dumb stuff, but I find that property of discourse to be pretty steady. I've even found some interesting comments on YouTube videos and newpaper websites.
Arithmetic skills. And they probably have?
Not really the same, since your average calculator doesn't take over executive function. The user thought and decided what needs to happen, down to one extremely well-specified task, which is so unambiguous that it can be completed by rote and without context.
This is more like paying someone else to read your math homework, generate answers, and maybe even submit them on your behalf. If that had been the (average) student experience, math skills really would have collapsed.
I think we'll find that the people doing that, like the people spending all their time and energy reading slasher pulp fiction in the 1950s, won't be doing as well as the people learning to enhance the power of their education and their expertise via LLMs and strategic alternating between GenAI and thinking about problems. And eventually, if society ever steadies up a bit, people will learn the useful ways of using technology in school and youth; and we'll be glad to be alive to see such marvels.
Assuming we can re-establish the will for democracy and taking care of the whole group.
I don't think it's quite as bad, but I'm imagining someone in the 1950s going: "People just have to use methamphetamines responsibly in their work, and then we'll see the massive benefits as widespread adoption unlocks the potential of the human mind."
In other words, a positive outcome with thing X is possible but there are also big behavioral and cybernetic pitfalls, ones which are suspiciously persistent.
I mean I use coffee quite a lot to prop up my software career. I don’t really find purity to be useful for anything except to answer questions like “should I steal this money now that I am alone” or “should I backstab this friend who pissed me off today.”
> questions like “should I steal this money now that I am alone” or “should I backstab this friend who pissed me off today.”
Ah, yes, a very BOFH approach [0] to workplace quandaries.
> I mean I use coffee quite a lot
Perhaps, but I don't think either of us would agree with the claim: "The way we integrated caffeine for productivity offers a useful and practical model for integrating meth for productivity."
There's an extremely detectable difference in the feedback loops that form between the tool and the people using it. There's something which remains unsolved ~90 years on. If we assume for the sake of argument that coffee is "good" and "productive", then that leads us to the question:
Are these LLM systems going to be more like coffee, or more like meth?
[0] https://www.theregister.com/bofh
Those are some big ifs and assumptions unfortunately, and the use of AI unfortunately has an isolating effect that is only likely to move the needle in the wrong direction.
> And math skills are going to collapse from calculator reliance.
People can no longer drive a stick-shift car, whereas it used to be that a valet would always be able to drive one to park it.
I also can't make butter by churning or soap from lye and animal fat or what ever.
However, I can use mathematica to demonstrate the criteria for various forms topological stability of solutions to many types of differential equations. I'd rather live now.
That's a very good point. People don't pull out their calculator in the grocery store to do analysis of unit costs or whatever, it's too much of a hassle for that small kind of analysis. But calculators have robbed us of the need to be good at doing that math in our heads. So the math doesn't get done.
Many grocery stores will already give you a per oz or per unit price so you can compare pricing for different brands that have different sizes. It's a very handy feature.
And it's so often wrong that you better be doing sanity checks if you use it for anything.
Is it often flat-out wrong where you live?
My most-common frustration with it is when products I want to compare don't have the same quantity-denominator. (Either a different unit, or sometimes a different dimension like weight versus volume.)
I've many times seen it wrong but more often the same item will have some tags expressed on cost per ounce, others per lb, etc. To discourage comparison. But I dont think people are paying much attention to those numbers anyway. Math has become a scary topic for them altogether.
That's mandated by law, not because the grocery story wants to.
This is exactly the right analogy. Granted LLMs are a more sophisticated calculator and engineering is still adapting to this wonky/evolving technology. But to pretend it's a coming catastrophe is to ignore every past parallel development - where this has not happened.
Math skills are at their lowest point ever in modern society, and many are still advocating for the reduction in reliance on calculators for this reason.
https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...
While on my high school math team, we weren't allowed to use calculators at school. Being able to work quickly without one was too important.
"Mental math skills" literally did collapse tho??? "Math skills" to write proofs (which a calculator cannot do) is a fundamentally different skill set than that "math skills" of mechanically doing arithmetic (which a calculator does do).
Seems like critical reading skills have already collapsed.
No.
With a calculator you still need to understand the fundamentals of mathematics. You also need to understand how to operate a calculator and know its functions.
Even when removing numbers and manual calculation, being a function or formula oriented mathematician still requires you to know the theory and functionality.
With an LLM, you don't even need to know anything other than asking the robot to perform. You don't understand the output, nor can an unskilled person explain it.
They probably have though.
Seems like we're doing fine though, yeah?
Could be a doing a whole lot better. Do you think there's an abundance of critical thinking in society? Because math skills aid in that.
I honestly can't tell what reward there is for this "critical thinking" since the entire world is bathing in "AI will replace you"
what point is there to be a human calculator if nobody rewards you for it?
and I'm not talking about wealthy Bay Area tech workers doing it so that they can write their substack blogs but rather the average person
Getting to live in a civilized society with a responsible government. Witness the counterexample.
When I look at the state of software engineering, "fine" is not the word that comes to mind https://xkcd.com/2030/.
If we're going to be the last generation that understands code at this level, we need leave it in a much better state before we die.
Not really, no. It's embarrassing how many people lack the ability to do basic arithmetic.
Do they necessarily need to? We have ubiquitous tools for this.
This is the same kind of analogy people make when they "poo-pooh" any sort of technological advance, especially with AI.
"Oh, that old simple device didn't displace anything, so this one which is now quickly taking over every job will be just like that simple one". So I guess people today know how to sew and knit and bake bread?
"Oh, humans were always needed, so it'll be the same this time." Is that true of horses and oxen after cars and tractors were invented?
Your calculator isn’t designed to keep you addicted to it and doesn’t have baked in orders like “be humorous and irreverent when appropriate.”
Also, yes I am dependent on a calculator.
1) have they not? 2) not a level comparison