Please write your blog posts yourself. I don't want to have to parse through this stupid claude output to get to your point. Literally just posting the prompt would be better than this.
The issue with this is that we don't know how it works. Generally speaking, we know how the level of abstraction that we were born with works. We might have some understanding of one or two previous levels, but that decreases the farther down you go. We might understand the next level, and some of the next after that, but eventually people will be making things that we don't have the context to understand without having to unlearn a lot of what we know now.
I'm old enough to see this process in action; I used to be young and in possession of esoteric knowledge that made me infinitely in demand and now most of the things that young people have esoteric knowledge about is things that I don't particularly care about, and I'm left with a lot of finely honed skills to solve problems that have mostly been abstracted away.
Exactly, I find this sort of nostalgizing to be unhelpful. I am not _that_ old but I did cut my teeth on C++ and learned about manual memory management, programming language design and had a few (even at the time) esoteric courses about assembly language. I am not assembly fluent and have no desire to be - but getting an understanding of how higher level code compiles into lower level assembly/machine statements is invaluable to get a sense of how performance can be manipulated and the how impactful it can be to do "silly" things like unroll loops to minimize the number of instructions needed for some higher level operation.
Issac Newton/Bernard of Chartres said we stand on the shoulders of giants and that allows us to see further afield but it also loses the detail of the ground beneath us - modern folks will no longer have the broad expertise that someone struggling through building a computer from transistors would gain through overcoming that[1], but we should learn about it academically to garner the bits of knowledge that remain salient. It is incredibly useful when experts in these highly specific fields have the communication skills and knowledge to share what is relevant to others without overwhelming them[2] with detail.
1. Real modern consumer grade computers are far too complex to understand fully without dedicating your life (or a significant portion of it) solely to that. It is more pragmatic to allow specialization and small realm expertise instead.
2. Of course, if you want to geek out and learn the minutiae that's awesome - but we can only geek so much in our limited time here.
I do think there's something to be said for the big difference in the current abstraction layer jump - the loss of determinism. It is meaningfully different.
There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism. Only the former can be statistically modeled to characterize and work around the noise with any reliability. If you measure hallucination rate for one prompt it tells you nothing about hallucination rate for another prompt.
In engineering you have tolerances to deal with non-determinism. ”Within these bounds” and ”given these assumptions then…” is the foundation of building something _on top of_ those things. LLMs are the same, it relies on heavily exact Turing machines as input but its output is entirely unstable. Even if you can get determinism it will never get anything resembling ”bounds” out of the box. That makes it a poor foundation for building on top of. Ie it’s not a screwdriver, it’s the monkey who’s holding it.
I do not understand the need to argue that monkeys are better than screwdrivers at screwing. Just let the monkey be the best version of a monkey.
I would argue that management is a better discipline to pull from for employing LLMs; It is better equipped to deal with non-determinism and going completely off the rails.
I think it's safe to say that computing should be mostly deterministic. I know compilers use heuristics that are stochastic, but come on. Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
> Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
Computing has never really been deterministic all the way down. Storage gets corrupted, RAM has soft errors, networks drop/reorder packets, schedulers race, caches go stale, distributed systems partition, query planners change plans, ...
Obviously those become tolerable because we've developed layers of contracts and understanding the bounds around them. Error correcting codes, checksums, retries, consensus, idempotency, false-positive rates, SLAs, etc.
If the abstraction is “delegate this task to a junior engineer, analyst, lawyer, designer, or support rep,” you dont expect deterministic behavior. There's review, constraints, escalation, checklists, tests, and accountability.
So now instead of improving the tools of biology so we can actually understand it deeply - we increase complexity of IT so we have to rely on muddy, side-effecty tools of biology to try to infer some of the properties of the systems we made. That's depressing.
People use the word "determinism" when they really mean something akin to "linearity", i.e. the predictability of a change in input on a change in output. Compilers for example are both deterministic and "more linear" in the sense that I can tell what the output will look like given a change in input (yes yes optimizations violate this to a small degree). LLMs can be made totally deterministic, but a seemingly insignificant change in input can create a drastic change in output, which is the characterstic we don't want.
I'm not sure that's true - you may be over generalizing
Probably not many here know the processes needed to turn sand into silicon and the expertise needed for hand grinding fancy lenses used in lithography. But we do know those things are needed the approximate philosophical need.
I think the danger comes when or if we fully automate a lot of the low level infrastructure tasks to the point where future generations do not even have the conceptual framework for how the tech they use is created
I agree, but having abstractions allow us to learn things in a system that are broadly applied.
This is out the door for AI generated systems. Unless extreme care is taken there is is no consistency in these new-found codebases. New paradigms are introduced left and right, because in the eye of the LLM, and the prompter, they worked.
It didn't matter that the same pattern was repeated 37 times in a slightly different manner.
So your knowledge now is no longer portable.
It used to be that you could look at code, and ask why? And usually (not always) the answer was something like: 'I tried x, y, and z, and those didn't work', either by past experience or current experimentation. (we could argue if the current experimentation should be documented).
But for LLMs they put in something complex for no other reason than that it does what is requested. Reading a string from a valid source byte by byte until you hit \0 is valid, and it'll work. But if you take a step back at, read the API docs of what you're consuming, and then consider: if the API says X, why am I testing whether that's correct?
> Generally speaking, we know how the level of abstraction that we were born with works
I'd say it's generally true that the majority of jobs of an era deal with a similar level of abstraction, and that's why most people stay on it. However, I frame this as being born with technical debt, and it's my obligation as an engineer to understand what the previous generations have built, and where it makes sense for me to work, directionally.
I think it's more that "Some human some time has known how it works", not that "Any given human knows how it (all) works.".
But yea this glosses over a bit trial-and-error designs and, so to speak, "genetic optimization" kinds of designs where we just try random stuff and say "Hey, this works. Not sure why, but it works.".
There was a period of a century or more where steam engine design parameters were determined by experiment and gut feel. The physics of fluid dynamics required to explain why a design is optimal just did not exist yet.
There's a lot of places in history where engineering far outpaced the science required to explain it.
I disagree. For example, the Wright brothers did air foil engineering empirically without full knowledge of the underlying fluid mechanics. Early and innovative engineering will often be less based on existing scientific or mathematical knowledge, while still using some of the methods. It's still absolutely defined as engineering.
Of course it is. The science comes from reproducing what you tried and getting the same results.
You may not have a good enough model for this observation to allow you to predict the results of other starting conditions, but it’s a model nonetheless, one that narrowly applies to the conditions you tried.
The level of abstraction when I started was that you had to solder the components (resistors, capacitors, chips - well IC holders, chips were expensive) to the PCB board - that was "building your computer", none of this nancy-boy rubbish of "plugging stuff together" with "slots that can only take one type of card".
I'm fairly certain I know how it works. Being a physicist helps with the even-lower-level-details if you want to start talking about transistor doping, or electrical circuit theory, for example
> The issue with this is that we don't know how it works. Generally speaking, we know how the level of abstraction that we were born with works.
What? Definitely not. I went to university and my first two years were subjects where I had to understand really deep levels of abstractions. I had to build logic gates, I had to work with hardware, wires, etc. I didnt see the point back then (I never used any of that professionally). The same about algorithms, databases, and a lot of things. But now I find it valuable and thankful that my professors (and whoever designed the career) considered important topics that I had to lear.
You started with logic gates. How much EE did you do, or the actual physics that makes the transistor possible? Those are the previous (deeper) levels that people had to know before they got abstracted away.
When I took software focused computer engineering around 2010, we still had courses that took us all the way down to transistors and even the physics of P and N junctions and how that applies to CMOS. (And even some basic analog electronics.)
Did I end up an expert at those layers? Of course not, but I know the basics and I know enough that if I need to I know where to start learning more. Just like I wasn't a C++ or hard realtime expert after university either, but now a decade and a half later I am pretty good at those (and a bunch of other skills that ended up relevant to my line of work).
Basically, none of the layers are "magic" to me. Even if I don't know the details of it, I know the general principle and I know I could learn more if I need it.
(I think you naturally end up an expert at the layer(s) you work in, and the knowledge tapers off as you go down (or up) the stack. For example, I know a fair bit about how the CPU works (cache coherency, pipeline stalls etc), I can passably read x86 assembly, etc. Because they affect the layer I work at (hard realtime systems C++ and now also Rust). I know far less about web dev than hardware.)
Similar experience in the 90s, but we don't really know the intricacies of doping silicone, or smelting metal to make the pins. And what about mining it?
I think the last time people knew how things were made was in preindustrial societies because they had to build everything themselves (whatever little things they had)
I was once in a museum in Bolzano, Italy. And they had the ice man mummy there (we call him Ötzi). He died like 5000years ago. And his "axe" was made of copper from some mine very far away. So even this guy probably did not know how everything he had was made.
There is a theory that he was probably very rich. So maybe less rich people were more in touch with what they owned. But still I found it fascinating that even so far back people relied on technology and materials that they didn't really know about.
I partially disagree. I know the basic high level concept of all those. Would I be able to reproduce state of the art results on my own? Obviously not.
But a core part of the engineer or scientist mindset is curiosity for the sake of curiosity. Just the fact that I don't know something is enough reason for me to poke at it or otherwise learn more. Same reason I still take apart broken electronics as an adult and try to find the fault (and sometimes even repair it).
By the way, mining silicon is particularly easy: it's basically sand. The difficult part is purifying it, especially to the levels needed for modern nm scale chips.
A more useful question than making high end silicon would be: could you with reasonable tooling reproduce basic electric components? I'm talking things like light bulbs, resistors, generators, perhaps capacitors even? Just the basics crappy versions, not modern highly optimised surface mount components. And I think the anwer is yes (for me personally) if I had access to metal wire and sheet stock and industrial revolution era tools.
I have similar experience with the poster, and the way I read it is, "from the things I build here are some examples". I did learn about advanced physics topics that enable transistors, and even did some experiments, but for fundamental stuff you "don't build stuff".
Did I do all physics or all electronic circuit design or all software stacks? Definitely not. But I spent 3 years learning (and building) about lots of stuff.
The one thing I appreciated about my Computer Engineering undergrad - and it took me a few years to fully appreciate it - is that yes, we did cover those levels.
The first two years were shared with Electrical Engineering. The second two years started to specialize towards Computer Engineering topics.
* Physics and chemistry.
* Circuits.
* Transistors.
* Logic gates.
* FPGAs.
* Assembly.
* Compilers.
* CPU and hardware design.
* Operating systems.
* Networking layers.
* Programming languages.
* Computer graphics.
Did I master all of the above - absolutely not. I loved some of them, struggled with others. Generally the cut-off for how my brain works is logic gates, I was never strong at the levels below that.
But we did cover them, and I could honestly say I had at least a rough understanding and mental map of everything that happens inside a computer from the point where it's plugged into an outlet, to the point where pixels show up on the screen.
how did you make the transistors? made your own vacuum chambers?
i had to make logic gates and so on, but i wouldnt say i really learnt it, even if back in highschool i learned all the different things a 555 timer can do
Please get used to this sort of depressive, absurd and out of touch tone from HNers, it’s literally all they do now. Don’t bother calling people here hackers anymore, they have checked out emotionally and spiritually.
What's absurd is thinking because you took a logic class and made a flip flop 30 years ago that that is the ground floor and that it means you "understand how it all works". You're not building a CPU from logic gates and you don't know how it works. If I put you or OP in a room for a year I highly doubt you could build an 8 bit Atari-like CPU from scratch. I worked with wires and logic too but I'm not arrogant enough to say that I know how it all works.
Don't you think there is a difference from "knowing how it works" and "reproducing every aspect of it at the level of the state of the art" ?
Also, your example seems flawed if you restrict to a certain product. Can I build a compiler from scratch? Yes. Can I reproduce in a year a compiler with LLVM/GCC performance level? No. Can I build a compiler from scratch in a year from a room if I need to starting mining from metals, building transistors, then building the first assembler and then implementing the compiler? You can imagine the answer.
Not really, at least not in this discussion. An assembly programmer gets logic gates, a C programmer gets assembly, etc etc. And each one can say to the one above "well you don't really get it if you can't bang out some meaningful work". Otherwise idk what we're even talking about here. Having a vague understanding of something isn't "knowing how it works".
Yeah, but if we use your criteria, people have to invent their own atoms out of individual electrons, protons, neutrons, quarks, etc before you'd admit that "hey, they did build a compiler from scratch".
I think a fair bar for "from scratch" is stopping at the point they need an expensive fab process (or how else will they reproduce the workings of semiconductors?)
The parent's argument can be boiled down to "You don't know absolutely everything therefore it's fine to know little or nothing." fallacy. It's the Chewbacca defense of AI boosterism.
> If I put you or OP in a room for a year I highly doubt you could build an 8 bit Atari-like CPU from scratch.
I believe I can, depending on what "from scratch" means. With nothing but transistors, resistors, inductors, coils, capacitors, I can probably do apretty poor general-purpose CPU. Maybe something like a stack-based ISA or, if I had more time, an accumulator-based processor like a 6502, but with an 8-bit bus.
If you're going to ask me to create transistors themselves, well I believe that needs specialised equipment.
That was one the major criticism I have of my CS degree's program. Perhaps the knowledge I wish for is not truly CS in the mathematical/theoretical perspective, but I believe it would have been absolutely valuable to learn.
Yes, thats exactly what uni is for.. To learn all those previous centuries of stuff before you can either start contributing to the corpus yourself, or go get a job (Where you will pick up more immediately useful stuff)
We really dont understand how AI is working, even the earliest "genetic algorithms" could be incomprehensible, but computer systems in general, they're not really that complicated.. its like an audio mixing desk..it looks insanely complicated until you realise it's just the same few knobs repeated many times for many channels. High level languages, compilers, assembly, machine code, nand, mosfets. A single person really can understand it all.
Exactly, I'm over 50 and I remember all the complaints about script kiddies who looked at windows bat files as opposed to all the 'real programmers' who knew C and Assembly and used VIM and linux (which is still going strong)
I expect 20 years from now there will be people reminiscing "remember claude.md? Back then we knew how it all worked. We had deep control. We counted tokens, chose the model..."
A (sad/funny) anecdote. Someone told me earlier this week "Not using a coding agent today is like using Vim for development instead of an IDE, you're just lagging behind".
As someone who's been exclusively using Vim for my development (and can definitely integrate it with AI workflows), that's just an insanely silly opinion. But I guess it shows how the next generation thinks about these tools that they've heard of but never actually bothered to learn.
Script kiddies didn't look at the .bat files. They ran them without a clue how they worked. There was some respect for the people who wrote those scripts but none for users who couldn't write them
It is no small feat to put in words that we are losing something almost as quickly as we are gaining something. The undertone, despite leaning into nostalgia boils down to losing control and this uneasiness I feel growing daily. It is already shocking to a certain degree seeing very young people not being able to use a computer in the narrow sense because all they ever learned was touch interfaces and apps. Curated content, curated interfaces - everything that resembles some kind of hardship ironed out in thousand steps of iterations to appease the market which means the lowest common denominator.
But I also see that the people who can create the absolute most and the good things and the working things and the maintainable things nowadays are the people that have gained a tool, but not lost the knowledge of the medium we are using it on because we are tied to this old world so perfectly put under the spotlight in this blog post.
The financialization of everything is what's ruining everything. In the computer and internet realm, there's warm nostalgia from hours spent tinkering, building one's own PC, reformatting C drives due to malware, searching for "snippets" to add a forum or pimp a myspace page. But inevitably the money incentives come to dominate. It's all of our doing, we all dream of a better life to put it charitably.
Now everything is a means to a commercial end. Tinkering for fun and knowledge just isn't profitable. And it matters less and less what each our stance is on money and capital if the people that optimize for money and capital gobble up all the money and capital. Of all that's going on, the wealth gap is what's most troubling to me, closely followed because it's closely related is "post truth". I think post-truth is roughly caused by the fact that people are happy to believe what they want to believe toward some commercialized and/or idealogical end. You're much more likely to hate and blame your neighbor when you look around and you're the one not doing too well.
In my mind, 'tinkering for fun and knowledge' was never meant to be profitable - if you're tinkering with the express aim of making a profit i'm not sure you're really 'tinkering' so much as you're trying to create a product/service/output for someone else to pay for?
There's absolutely nothing wrong with doing that - I'm just not sure the 'ethos' of tinkering has anything to do with trying to make money and is usually reserved for describing someone playing about with something for their own enjoyment/fun with no desire to make money.
Now, of course some people did find that their tinkerings were able to make them money, but I think at its base it's a term I'd tend to say implies doing something for fun/for themselves, rather than doing it for profit?
In my experience there's still plenty of people out there tinkering just for their own personal satisfaction, but of course there's almost certain a whole load more people out there 'tinkering' to try make a profit.
Agreed, I'd say tinkering for profit has become the defacto optimization and tinkering for its own sake is less fashionable because it commands no attention. and everyone is obsessed with the modern currency: attention.
To be clear, I'm seeing this as an observed phenomenon, not that everyone made up their mind that they hate tinkering and love money. I just think it's getting really really hard to exist in the world as a normal person when the entire human collective is getting pumped commercialized hyper-media from all angles at all times now 100X'd by genAI bots. It's really exhausting and so you either opt-in to the game, monetize your now AI side-hustle to pay the rent, or opt-out and live in the woods. and get packages delivered by Amazon. heh.
hah, that made me laugh. To this day I'm so proud to have learned how to repeatedly just wipe my PC C drive when needed then reinstall XP or whatever. I'm just a web-dev that got their start by googling "how to make a website". As I leaned in there came a turning point where I wanted badly to switch form windows, php, scripts, to Ruby on Rails. I worked up the courage - eventually - to partition my drive and install linux. After that, I got to experience a proper terminal and the rest is history.
We are on average losing control, but you individually can choose whether to lose control or not.
Programming languages, UNIX, debuggers aren’t going anywhere. There is more to computing than what your boss demands and what is hyped on tech forums.
In fact I believe the indie/handmade scene will grow substantial if not boom, even if just as a hobby for most. Showing what you have made with your blood sweat and tears will elicit more praise and delight when you could have just asked a machine to do it all along.
> It is already shocking to a certain degree seeing very young people not being able to use a vehicle in the narrow sense because all they ever learned were the mechanical controls of the so-called automobile.
I think there's a fallacy where someone points out one instance of a larger trend which will, when taken to its logical progression, lead to an undesired effect; and then someone attempts to rebut the claim by pointing out that the trend has existed before and the undesired effect hasn't happened yet, so any concern is nugatory. I'd call it the grippery slope fallacy, complement to the slippery one: we haven't fallen down the slope yet, so we can't fall down it. What if an individual instance of ignorance is acceptable because people still need to have understanding in other areas, but if all understanding everywhere is eliminated then we all suffer?
the difference is that an automatic transmission doesn't make the car work worse. The modern UX landscape would rather board up a room because the door has a sharp handle than figure out how to make the handle less sharp
ETA: Or, to put it in car terms, we were all forced to take cabs (except for the people who were interested in driving, who became cab drivers) because car crashes happen or my sand eating neighbour couldn't tell which pedal was the brakes
Absolutely - you used to have to control the richness of the fuel mixture manually. You used to have to crank it to start it, manually interact with a clutch to shift gears, etc.
I appreciate the tactile joy of interacting with simple systems like those, but most times I just want to get where I'm going. Freeing my attention from those tasks allows me to pay more attention to the (inattentive) drivers around me, and try my best to not die.
Eventually a computer will handle driving for most of us, and we can lament about all the things we've lost there too. If you zoom out, most of us don't have an in-depth understanding of how an entire city works (power, garbage, sewage, maintenance, public services, politics, etc), and couldn't coordinate the various activities to keep it running if we had to. We live in towers of abstraction.
My read is that there does seem a clear difference between simple -> advanced machines vs simple -> "smart" machines. Nearly every smart machine is bullshit enshitification-in-waiting. Rent-seeking in-waiting. Smart tvs, smart cars with touch-screens. some would argue apple products. These things proclaim advancements but what they really do is black-box and dumb everything down to the lowest common denominator, then quite literally impose control over the air, and shove ads to you.
I'm all for just getting to where I need to go by using the appropriate tool, like a reliable car. But no not if it means foregoing the liberty of other options.
This time is different though (which has also been said every single time). But I'm worried this time it's true (also said every time). Doesn't help with the unease though.
The scale of it is certainly different, if nothing else
We have never before seen every single profession disrupted to this degree, not even the introduction of the personal computer introduced such a dramatic shift
No. No economy ever had essentially every single major company spending a significant fraction of its budget on hiring auto mechanics. Which is to say, for all the changes the automobile wrought, the role of the computer in industrialized society eclipses it tenfold, a hundredfold, a thousandfold. For an individual in many modern societies, being denied access to a car is already effectively crippling, and the idea of being denied access to computation threatens to be somehow even worse.
There was a sweet spot with computer technologies for some decades where hobbyists could afford to experiment and even push the envelope in the nascent field of computing - similar to genetic radiation, many niches were formed and rapidly filled. The computing biome has evolved to the point where most entities are not operating at the low-level abstractions that were once the only means of interacting with the computing environment, instead they operate now at the highest levels of abstraction we are capable, so called "natural language".
"The difficulty was the knowledge. You came to know that machine the way you come to know anything that pushes back. The resistance was the whole medium. You only ever know the things that you can lose to."
We who grew up in this era formed a hands-on engineer's knowledge of these systems, built from experience and practice, learning these layers of abstraction as the bleeding edge developed. Many these days have entered into a world where there are easy answers abound, they just might not be right, and one has to gauge how much they care about correctness.
This is definitely true, and we definitely need something similar again. I've been using a game ("The Farmer Was Replaced") as a jumping-off point for teaching the kids Python, but the more I think about it, the more I think that they need some sort of hardware package similar to an old Apple //e that gives them just enough rope to hang themselves. It was easy, back in the day, to learn a ton (even assembler) on a system like that, and I feel like there's some value in rewinding the clock back to that point, forking the experience from there, and seeing what a new generation of kids will cook up.
Zach Barth and his former studio Zachtronics released several entertaining puzzle games based on idealized fake assembly languages. These are a lot of fun and useful for introducing registers and multiprocessing.
I keep coming back to the idea of taking that concept and just running with it, IE building up a toolchain alongside some concrete tasks to solve with it and just adding more detail until we reach a reasonably sophisticated level
I've been staring at 0x10c, carwars and battletech and there's a sense that I could build a sort of programming / engineering Zach like
I'm one of the greybeards who has the 2400 BAUD modem negotiation tone sequences emblazened in my neurons.
For a while I've been meaning to set up some Wireguard connections among some of my systems. Being as busy as I am with work and family, I've relinquished that to Tailscale for now.
Sure, I could have sat down and jumped through the hoops to get everything set up and working across my various hosts, including network routes, firewall rules, key pairs, systemd units, and so forth. But the "cheap and easy" alternative was right there and worked (except when it forces re-authentication).
With LLM agents, I was able to effortlessly analyze my existing network and produce tailored scripts to do precisely what I wanted. All I had to do was review the scripts for potential security issues and what not. Looking at the script, there are 3 or 4 specific tweaks that needed to be made to my network routing rules given my network topology. I could have read a few man pages and iterated on the script by hand to eventually get there after maybe an hour or two of futzing.
The availability and effectiveness of the agents is simply too tempting for me. I'm not sure what this means about my skillset, or if that even matters any more. I am fairly confident that, so long as my brain still works well enough, I'll always be able to RTFM and figure things like this out myself. At this rate I wonder whether my kids will have the same ability. And I also wonder how much that will matter.
Regardless, I'm still helping them figure things out the "old way" without over-reliance on LLMs. One thing I'm fairly certain about is that failure to develop problem-solving skills can only put them in a worse position in life, no matter how capable AI becomes.
The thing is - everyone complains about AI stealing our attention and understanding. But you can just as easily use an LLM as a tool to gain a deeper understanding. It's just the default path for most folks is "Hey clanker, do the thing" rather than "Hello clanker, please tell me about how that thing works".
I've done at least a little of the latter, and it's amazing how underrated it is as an educational tool - especially for the solo individual.
> When I was young I fixed my parents’ computer and now that I’m older I fix computers for my kids. Are we the only generation that knows how computers work?
No, that person is just the only one in his family who knows how computers work. There's plenty of people in generations both older and younger that know too, just apparently not his parents or his kids. Some of the most impressive assembly written today is by people younger than 20. It'll be fine.
It's broadly true though, that younger people struggle with the fundamentals of computing, even if there exists specific examples of young people accomplishing exceptional feats.
We raised a generation of people on consumption-based computing devices, which means they don't develop the skills necessary to produce things using computers. Of course, we can teach these skills, but to do that, we must first acknowledge it's a skill which needs to be taught.
I agree; the situation "young people know all about computers" was maybe a twenty-five year phenomenon; before that, there weren't enough computers for it to be generally true, and at some point you no longer needed to know all about computers to successfully use a computer for the sorts of things the general population does.
maybe another angle to this that I observed: my working class dad who had little education happily taught himself a bit of programming with computer magazines in the 80s an 90s. He actually could figure out his way around a command prompt.
I went on to study CS and teach undergrad courses and what I noticed, this started already maybe a decade ago, is that CS students who we handed bootable linux usbs couldn't figure out how to set their system up. And not just that, they just kept emailing us with statements like "it didn't work, what do I do?". It's not just lack of knowledge but complete helplessness when something doesn't work in 2 minutes. That's the biggest problem with this reliance on ChatGPT or whatever.
I think the young generation is in an even worse position. Not only do they not know how computers work, they don't even have the basic DIY problem solving attitude our parents have.
“The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads.”
I disagree. If you ask a model for a manual and it regurgitates that manual from its training data, it’s over-fitted. It will regurgitate something that looks like a training manual. Or whatever fits your query about training manuals.
You still have to push back on them sometimes when you spot an error. And you can only spot them if you already know what you’re looking for and should expect. Otherwise you have to ignore the output and just get the links which… could be outdated or made up as well. You’ll never know until you verify the results.
And this degrades with compression and time.
There’s no royal road. I agree that trying and getting frustrated and having to take the effort to understand something pays off in spades. I just think it’s still worth it and vastly under appreciated in this era of “everything fast, now.”
I came into the US tech industry in 2011 from a CS program (programming on my own longer), which was basically a pipeline into Java or web dev at that time (the classes on other things like OS, compilers, and FPGA were a thing, but seen by most as a hurdle not a promising or practical career pathways).
Most of the people I dealt with for the decade following that (i.e. pre-LLM) had little clue how any part of computers worked, and I don't intend that to be particularly pejorative, it's just that it's been somewhat easy to carve out a niche doing something to do with programming while having little understanding of the small or big picture.. all heuristics and applying working patterns.. for a few decades.
You can still understand whatever you want, today, with added tools. It's just a choice to turn your brain on or off. I think LLMs are perfectly fine as a learning tool to interrogate a subject, do comparatives, and then formalize your understanding by reading the sources. At a macro level brain rot is real, but it cuts across all generations and it started long before LLMs.
> At a macro level brain rot is real, but it cuts across all generations and it started long before LLMs.
The thing is, if there’s a way to do something easier people will generally just do that. All the nostalgia of the olden days has as a component that there were no alternatives as the example in the article highlights.
We can try to preserve the hard way to things that gets improved, but the hard way will only ever be niche if it doesn’t become extinct entirely. There is no going back.
Sure, humans are attracted to laziness by default and at its best technology must be used to lower toil and suffering (unfortunately there are unintended consequences etc for philosophical debates).
The bigger issue with brain rot is attention spans. Social media has ruined attention spans.
This might just be a personality trait of me, but I really like technology that's transparent enough that I can fix/modify/understand it. Opaque technology gives me an ick. It's not really a complexity thing -- Linux has this property of being transparent even though it's extremely complex. The way he describes old computers.. man, I loved that about them, there was nothing _secret_ going on, you weren't worried about telemetry or weird background processes. Then again, I still drive a manual transmission internal combustion car in 2026, so maybe I don't represent the world.
Then again, maybe there's light at the end of the tunnel. Even though AI boosters kind of annoy me, I do appreciate that they're tinkering and hacking and working to understand the technology on some level. I think there's always going to be some small number of people that are like this, in the same way there are still people that look at disassembly and understand CPU caches.
> You opened files like autoexec.bat and you read them. Sometimes you built a boot disk for a single game. A floppy whose entire reason to exist was to start the computer in the configuration that one program demanded.
Is the premise here correct? I'm not sure that I'm convinced that a 1990s computer user who knew how to edit autoexec.bat or insert a floppy to boot their computer "knew how it worked" in a meaningful sense.
The stack of abstractions is deeper now, and all indications suggest it's not going to stop deepening. But I think the abstractions were already quite deep by the 1990s.
(I think the classic error here is a demographic one: computer nerds always poke through abstractions, because they do it for fun. I don't think that's going to stop, anymore than web browsers stopped people from writing kernels. If anything, we write more low-level code than ever, because access to the prerequisite knowledge is less gatekept than before.)
I doubt you ever understood the solid state physics, semiconductor fabrication processes, supply chain logistics, monetary policy, shipping routes, mining engineering, etc. "Knowing how things work" is a stone-age attitude.
Obviously no one knows every detail all the way down the stack. It's not possible, but having some level of understanding of how a system operates is useful especially if you ever have to fix it yourself or explain why/how it is broken.
Modding communities are still going. Kids, afaik, are still playing around with hosting minecraft servers or whatever is en vogue/cool/meta/etc nowadays. DIY 8-bit computers are gaining popularity.
IMO the fact that something's become very mainstream doesn't necessarily mean it's been watered down for everybody. There will always be people with various levels of curiosity and enthusiasm.
I resonate with every example given - setting jumpers by hand, sound card interrupts, autoexec.bat. I'm also a happy user of LLMs and agents. This article captured for me what is lost - which, as others point out, has long since been lost, if ever had, in other fields (e.g. modern cars vs. the Model T). I wouldn't go back, but I can still have a sense of loss.
With how seriously rise of LLMs damages schools and university education, and thus next generation of comp sci graduates, at least we can hope to secure our arcane knowledge jobs for long.
Speak for yourself but I certainly don't know how it works.
Also, if you don't think the difficulty is the knowledge, I'm sorry but I must disagree. There has been more than one government or corporation or large institution that has tried to destroy history and knowledge in the past.
I think it's fair to say that knowledge is also at risk.
>The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads. They will recite, flawlessly and forever, exactly how all machines work. If this were only about competence, it would be the most secure moment in the history of computing.
Obviously this person doesn't have enough experience with the current crop of AIs. This is the exact reason why we should be concerned in the first place. Someone has to design and build the next version of everything, and AI isn't qualified to do the job, and might never be.
>What is dying is acquaintance. The plain, unglamorous intimacy of having fought a particular machine, and lost, and gone back, and finally felt the thing give.
This is true. But then again, who among us had to get up in the middle of the night, and trudge out to an outhouse? Who had to shovel a bit more coal into the furnace? Who had to pump water, and heat it on the stove? Who had to split wood, and stack it for the winter?
There are many, many things we're no longer acquainted with, and that's sad, but still ok. What's not ok is losing the competence required to maintaining the infrastructure and supply chain supporting society and civilization.
1: This is why I prefer console games. I just want to have fun without fighting with the machine.
2: There are plenty of people who appreciate old techniques and methods; and keep them alive. Think of going to a museum and seeing someone demonstrate an old craft or reenact how a craftsman did their job. For example, in my town there is an old, water-powered corn mill that still runs and sells corn meal.
> The graybeards are aging out, nobody compiles their kernel anymore, and someday something deep will break and there will be no one left who can climb down and fix it. Maybe. But I think competence is the part that’s fine.
> knew a beige computer in 1995 that wouldn’t run a game until I had rearranged its bits by hand. More dependent than ever
If you look at previous article from this author, it says how Mac is amazing and how Linux sucks. Kids like that in 1990ties would buy expensive consoles, and would not deal with hack PC's to get free games.
Many people today are still dealing with cheap shitty hardware, 7 years old Android phones and sketchy ROMs... Just because there is no other option!
>What is dying is acquaintance. The plain, unglamorous intimacy of having fought a particular machine, and lost, and gone back, and finally felt the thing give.
I think this is a universal feeling that accompanies any technological innovation. My phrasing is that new technology unbundles the thing people want from the craft that was formerly required to get it; any craft requires someone to learn and achieve through struggle.
I can't help but think this type of step function change has happened with computers before.
The change from punch cards to magnetic storage certainly made it so you didn't have to "know how it works" for every single bit.
The change from machine code to a language like Fortran brought about such an abstraction that a Fortran dev didn't "know how it works" at that same level anymore.
At this point, the layers of abstractions between using a React component and something being rendered is immense. React VDOM, the real DOM, browser render engine (which sits on abstractions like ANGLE, skia, etc), calls OS APIs, which call driver APIs, and the lowest level of anything it is still C/C++ that is compiled (abstracted) to something closer to what the hardware expects.
I won't bore you with the ChatGPT output details, but it estimated at least 35 meaningful layers of abstraction between a React component and being rendered to the screen. LLMs seem to be the latest level of abstraction to make it so we "know how it works" less than before.
It's hard to articulate why, but 35 layers of deterministic abstraction feels very different from 1 layer of nondeterministic one.
In particular, I think every developer has experienced the need to jump "down the stack" to debug or understand something (even if not all the way down). Certainly, I think any senior developer should be at least conversant in the first few levels below wherever they "live". But this seemingly ends up looking fundamentally different in the interaction mode of an LLM, because you'd just ask it to jump down the stack for you
This is also a fear I have, I am answering it with building bigger things with AI: what I loose in expertise in the details, I gain in the pleasure of architecturing bigger things.
This is sort of the story of the telephone system of the 1950s-1970s, or electricity in the early 1900s, or cars from 1950-1980s, or airplanes from 1910-1939.
I have no idea how an electrical transformer works (well, other than the bare theory I learned in physics courses), or how power gets from the power company to my house, or how the circuits in my home are setup. I plug something in, and it works, and occasionally I throw a breaker if something is malfunctioning. There's no resistance there (pun not intended), and there shouldn't be. People got killed trying to wire their own homes.
I used to read about phone phreaks from the 1970s that could do black magic to get free long-distance phone calls. When I grew up in the 80s, that was basically gone. You picked up the phone, got a dial-tone, and called. And now it's really gone, with everyone having an encrypted cell phone connection over 5G, and your IMEI and IMSI being phoned home to every tower you connect to.
It's the nature of technology and capitalism. As the technology matures, it gets hidden away to become increasingly invisible to the end user, so you just do what you want to do with it. And then the engineering resources get spent on new problems.
> And now it's really gone, with everyone having an encrypted cell phone connection over 5G, and your IMEI and IMSI being phoned home to every tower you connect to.
people will hack enterprise pbx systems. The stream just flowed somewhere else.
I'm still nostalgic for the times when I had to enter the bootloader using the keys on front panel of the PDP-11 to then loaded the OS from paper tape. (not to mention multi pass compilers using paper tape between the phases...)
> They will have a tool that does everything and asks for nothing, and they will be as easy with it as you are with the light switch you never once thought about.
Oh come on! Of course I wondered how a light switch work, and I then learnt. I remember taking apart broken electronics as a kid, and that later morphed into also trying to repair things. Including the computer (both hardware and software wise). I remember ending up reinstalling the OS so many times as a kid on the computer I had access to when I broke it in various ways past what I was able to fix.
Sure, not everyone will have that drive to understand how the world around them works under the hood. But for engineers and scientists I would expect a far higher percentage to have that sort of personality.
It doesn't matter if it isn't "pushing back", just that I don't understand it is enough to catch my interest and
a reason to go poking at the thing.
The thing is, yes it will still happen, but at substantially decreased rate. Groups of people banding together to solve a problem or figure a solution will slowly stop forming. Everyone will be out there learning just for themselves.
The collective knowledge of people will likely ebb. Things will be different and individuals will have to adapt.
There is a lot of useful information on how things work out there, if you want to learn. YouTube videos took the place of popular science magazines largely, and perhaps it will be some different medium tomorrow. But the knowledge is there, and just like in the past you have to go and look for it, just in different places.
There are also still courses you can take to learn practical skills, I for example took a short blacksmithing course a few years ago. A lot of fun and way harder than it looks (at least if you want to get good at it, my goal was just to give it a try and have fun, and 5 evenings was enough for that).
The graybeards are aging out, nobody compiles their kernel anymore, and someday something deep will break and there will be no one left who can climb down and fix it.
I dunno. I suspect it'll be slightly different in the OSS world[1]. There will be some folks who can climb down there and fix it, but since that skill is no longer valued (as in, being able to do that isn't valued by most users now, much less in some hypothetical future). And they will be expected to do that, now, without thanks. And when they say "I'm too old for this shit...you can go pound sand" because this was one unpaid insult too many, users will shrug their shoulders and load windows or buy a mac because those companies are paying & tasking people to keep things running. And by inches oss ceases to be a thing.
[1] Yes, I know there are many oss devs who get paid for their work. Many critical contributors do not.
I wouldn't calling it learning more and more like "cycling through SoundBlaster DMA and IRQ options until the sound work". Still, there was an intrinsic curiosity that isn't as prevalent.
There are more important battles for humanity to fight than someone trying to figure out how to configure Linux correctly for their hardware. I won't miss the need to waste time fighting the computer going away.
We always were the only people who ever knew how it worked. In 1990 people fellow students called me to fix their computer, they had absolutely no idea how any of this worked. No. Idea. Yes, the machine was being difficult; but their reaction wasn't to fight it, or understand it. It was to call someone to do it in their stead.
Maybe the difference is more of the professionals in the field now haven't built that same muscle, as there's a broader group of people working in tech. Whereas the folks that could fix things in the 90s mainly gravitated to computers as a profession. Just random musing though I truly don't now.
I feel that things are pretty different. Although the example of interrupts and jumper settings being common knowledge is a bit of a stretch... it's still amazing to recall that MS-DOS was regarded as an end-user / consumer OS, and that, more generally, it really was regarded as totally normal to need to invest some time in learning about the system, files, directories, typing, configuring/customizing settings and network options just to be able to do what you wanted to do.
I find the current expectations around consumer "apps" to be totally infantile in comparison, where everything is now a single-purpose "app" that does exactly one thing when you push a button, and if you want something even a tiny bit different.. you can't, and that even basic things like files and settings are no longer accessible.
Compare with what the drivers/operators of the earliest automobiles were expected to know vs. what the same category of people are expected to know today.
There's nothing new about this particular progression - we've been through it in dozens of technologies already.
I think the difference is that with those technologies, not knowing is seen as a hindrance (I imagine most people wished they knew how their engine worked, that's useful) that one can simply live with, whereas with computers nowadays I see people just not care about these skills. When I do something interesting on a computer that other people see, the response isn't "huh, that's a neat skill" it's "why?". People don't care anymore. They don't see knowledge as useful or something that is beneficial to pursue
Folks who keep mentioning that this is no different than any previous upward leap of abstraction in human history are missing a key point that the frontier labs are certainly not missing: this is the first time in computing that you are becoming completely dependent on a _subscription service_. I don't need to know how my CPU works because it continues to work once built. Once I outsource all cognition to a billable service, I am forever and continuously in thrall to someone else's revenue strategy.
But this too shall pass, it used to be you were dependent on a subscription to just have access to a computer itself (mainframe) and now we all have one in our pockets. I’m sure some day we will be able to run something as good as Claude at home.
I may not be a "typical" engineer as much anymore, in that I don't seem to be excited by knowing the intricate details of how things work as much as I enjoy knowing the capabilities of the systems that enable the projects I work on, and working at a higher level that might be more abstract and conceptual. AI enables that very well in that it enables jumping between paradigms and enabling creativity by removing a lot of the grind that was necessary in gathering technical details. I find it less appealing now to get into the weeds of how a thing works, even though at 10 I very much did. Like the author, I dug into the details of exactly which bits needed to be activated to make a particular program work. But as I aged, I was drawn to higher levels of abstraction, I think because I realized I could do more overall by piecing together bigger blocks. Yet I know colleagues who like to be involved in a very granular level of detail. I have many friends who almost revel in their knowledge that they are the ones who know precisely why something works the way it does. I think about the story of the NASA engineer on the Apollo 11 program who understood the assembly so well that he could confidently answer on the spot the moment the radar overload "1201 alarm" error flashed that the mission was still good to go. Some folks get so deep in this perspective that it almost seems like an affront to suggest they may be wrong about one of those details. And well they should, because precise people are very much needed to understand intricate systems.
But it makes me wonder how they feel with AI being an indeterminate system - one where you can never really know what the answer will be every time. Likely still correct, and getting better every day, but also possibly different every time. Added to that, it's exponential spread across the software engineering domain. For me, even as someone who likes high-level systems, I still relate to the feeling that the detailed part of me that still likes some of the internals may be less focused on in the future. Even for me that's kind of sad. But I am also curious about my colleagues who are super detail oriented - how do you feel about AI and how it changes the focus on that detail aspect?
Moral panic. We all have private tutors on every subject at our disposal now. It takes some special kind of mental gymnastics to conclude from this that no one will learn anything anymore henceforth. The article is just a long-winded way of saying "Kids today have it too easy", or, equivalently, "I'm getting old".
When I was young I was so naïve. The whole world seemed to be barreling straight towards the things I liked and was interested in. Every year there were more and more nerds, more people with computers, more people playing video games. It felt so much like I was the future, that in a couple of decades everyone would be chit-chatting about what was coming, buying and comparing devices, etc. I could not have conceived that people younger than me would have less computer knowledge than me. If at twenty you had told me that I would wake up one day and teenagers no longer knew what a file extension is I would've asked you who won World War 3.
"To play a computer game in in the 1990s, you first had to understand how the computer worked.
So you learned. You opened files like autoexec.bat and you read them."
Ehh I dunno about that. I rarely, if ever, had to mess with any of that junk after Windows 3... I also didn't have to deal with any IRQ issues. So seems like it was already mostly abstracted in the "1990s" lol
Yeah same. I remember playing a bunch of Sierra games on my first PC as a kid on either. Most I had to deal with was installing the drivers for my SoundBlaster card from the included disk. Most I dealt with was putting in the CD, double clicking the installer and entering the product key.
That said I did run into my fair share of other problems, and that early era of personal computing and my access to machines is the only reason I work in computing/tech today. If my childhood wasn't full of tinkering with these fascinating machines, and I only ever had an iPhone or iPad, I likely would have turned out much different.
Anecdotally, I think it depends on age and family more than anything. I grew up with hand-me-down computers, games because we couldn't afford anything new. Started with win98 and games from the mid 90s, but running them in 2002.
I do remember having to look lots of things up and figuring out why some things wouldnt work. Then getting into building our own computers (because it was cheaper) and figuring out how to get halflife mods working...
Windows 3.x did not cohabitate well with certain games that required nearly the entire 640k of conventional memory to run. These are mainly games from the old DOS days; games that relied on DPMI were largely exempt. But to play certain Sierra titles, for instance, you had to set up one CONFIG.SYS configuration that loaded HIMEM.SYS and EMM386.EXE for Windows and another that left them out to free up that memory for your games. In later versions of DOS you could set up a menu to select the appropriate configuration at boot.
And of course IRQ diddling was still necessary to configure sound, network, and game controller hardware throughout the DOS era of gaming, which lasted well into the Windows 9x era.
I've been thinking that there might not be new programming languages. The amount of code in the current popular ones will explode, so that's what all LLMs will be trained on.
Good luck coming up with a new language and getting enough content out there that LLMs will be fluent with it.
If true, I think that's a shame. There's plenty of innovation still to be done.
How pessimistic you are on this subject depends on how much you believe in domain transfer for these tools. No programming language sprung out of a vacuum, and it seems reasonable to think that a LLM sufficiently trained on all the C++, Haskell, and OCaml in the world could probably do a decent job at writing Rust, even if it had never seen it before, given the rust compiler as a tool and rust documentation as input
It certainly does worry me, though. As does the increasing amount of materials and manuals that seem to be written assuming the LLMs will be the (only? primary?) audience
This has always been true. I never fixed my car. I knew how it worked well enough to know, hey, that sounds like its coming from the exhaust pipe. Then I took it to the mechanic. I can do basic maintenance on my bike, but I still take it to the bike shop. I have a small collection of vintage cameras, which means tracking down the few people left who know how some particular model works, might have parts. If your Synchro-Compur shutter needs parts, forget it. For most people, most of the time, the assumption has always been that someone else knows how to do that.
> The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads. They will recite, flawlessly and forever, exactly how all machines work.
That's wrong, and that's exactly why the loss of knowledge is such a problem. LLMs do not, and cannot, actually know a single thing. They are a statistical model, not knowledge. When they give out wrong information (and they always will, by their very nature), you need someone with actual knowledge to be able to recognize the BS and correct it. But we are losing the knowledge, and unless things change we will be no better off than the people in dystopian sci-fi stories who pray to the machine god because nobody knows how it actually works.
I can already see future where there is group of people goading AI to right direction by repeatedly changing what they ask slightly. And then memorise the times when you got right incantation for seemingly right outcome. Without actually understanding or being able to reason about process in the middle. Possibly a seemingly inconsequential misspoken word or typo can lead to better or worse outcome. Or maybe just not saying "Please" will sometimes produce wrong output. and other times you must not use it...
Yes. And since LLMs can not improve knowledge - I mean, they can generate new arrangements of information, but they have no idea whether any of it real or making sense, unless humans - explicitly or through training - tell them, the more we rely on LLM knowledge the less the quality of it would be. Right now the LLMs are mainly in auxiliary role, so most of the knowledge erosion they generate is laughed at and relatively quickly corrected. But would this hold once the role of generative AIs increases? We already essentially entered the chaos period with news content - there's so much noise that it's basically impossible to know if any news message you read is true or manipulated somehow. This is going to start happening to more fundamental knowledge too, either on purpose or just by the force of the probabilistic nature of generative AI.
I was just thinking. What is the future when LLM is used for both code running itself and then also designing the hardware it runs on? Will it be sustainable? Or will there be some sort of error cascade that destroys the whole thing?
Maybe AGI is impossible with current model as it simply can not reliably improve itself... Enough errors in any part of loop will stop the progression.
This comment might have gone hard in 2023. Now it seems out of place. LLM's do hold knowledge empirically. They sometimes give out wrong information, like humans. And you need someone with better knowledge to correct it.
> But we are losing the knowledge
No we aren't and this is spreading FUD. Things have always been like this. Its called specialisation and this is how society progresses. I don't know how the supply chain worked to get the food to my table. That's why its so cheap!
There have always been layers of abstraction. I've been around for a while, and when I was a kid, the two choices I remember seeing were assembly code and simple semantic languages like BASIC.
Assembly seemed like too cryptic for me to really even follow and I never really did learn it, but at the time I remember people would say that assembly was easy and basically plain English compared to machine code.
As recently as fifteen or twenty years ago, I would occasionally check in and think of how unbelievably far away we had gotten from how the computer actually works. Like, you can just write "open window" and a window opens. Amazing.
Of course, those people writing machine code didn't need to really understand what P and N were in a transistor, let alone how an integrated circuit pulls it all together. And I'm not sure how much those guys knew about silicon dioxide.
The more complex things get and the more layers of abstraction there are, the more impossible it gets to really master things all the way down to first principles.
So what? People can carve out whatever chunk of the stack they want to really understand if they want to focus their lives on it. And for everyone else who's just trying to accomplish some other goal with computers as the tool, they will naturally use the highest level of abstraction and the simplest one for them to use, which is exactly what they should do.
This and it was demonstrated in Star Trek The Next Generation: Season 1 Episode 17 "When The Bough Breaks" [1]. They did not know how to fix their computer system or even where it resided. It did everything for them but was also killing them and making them sterile. There was nobody left to figure this out until the Enterprise crew happened along. This could happen on Earth. Who will save us?
>The difficulty was the knowledge.
Please write your blog posts yourself. I don't want to have to parse through this stupid claude output to get to your point. Literally just posting the prompt would be better than this.
A shame that Pangram flags this as AI.
https://www.pangram.com/history/c0a9cde2-7a5c-4588-83a3-0269...
Given that I’m the author, I promise you I’m a real human. It’s not AI.
Obviously there’s a rise in pure AI writing but I find these sort of services that “analyze” content more harmful than good.
“Pure AI” so you haven’t used any LLM at all, or just partially?
I trust these detectors even less than I trust LLMs in the first place.
I know you know this, but for those reading, these detectors are bullshit.
Damn, that is depressing. What a shitshow we've created.
The issue with this is that we don't know how it works. Generally speaking, we know how the level of abstraction that we were born with works. We might have some understanding of one or two previous levels, but that decreases the farther down you go. We might understand the next level, and some of the next after that, but eventually people will be making things that we don't have the context to understand without having to unlearn a lot of what we know now.
I'm old enough to see this process in action; I used to be young and in possession of esoteric knowledge that made me infinitely in demand and now most of the things that young people have esoteric knowledge about is things that I don't particularly care about, and I'm left with a lot of finely honed skills to solve problems that have mostly been abstracted away.
Exactly, I find this sort of nostalgizing to be unhelpful. I am not _that_ old but I did cut my teeth on C++ and learned about manual memory management, programming language design and had a few (even at the time) esoteric courses about assembly language. I am not assembly fluent and have no desire to be - but getting an understanding of how higher level code compiles into lower level assembly/machine statements is invaluable to get a sense of how performance can be manipulated and the how impactful it can be to do "silly" things like unroll loops to minimize the number of instructions needed for some higher level operation.
Issac Newton/Bernard of Chartres said we stand on the shoulders of giants and that allows us to see further afield but it also loses the detail of the ground beneath us - modern folks will no longer have the broad expertise that someone struggling through building a computer from transistors would gain through overcoming that[1], but we should learn about it academically to garner the bits of knowledge that remain salient. It is incredibly useful when experts in these highly specific fields have the communication skills and knowledge to share what is relevant to others without overwhelming them[2] with detail.
1. Real modern consumer grade computers are far too complex to understand fully without dedicating your life (or a significant portion of it) solely to that. It is more pragmatic to allow specialization and small realm expertise instead.
2. Of course, if you want to geek out and learn the minutiae that's awesome - but we can only geek so much in our limited time here.
I do think there's something to be said for the big difference in the current abstraction layer jump - the loss of determinism. It is meaningfully different.
Plenty of physical (and biological) processes are subjectively non-deterministic outside clean rooms. Doesn’t mean our ancestors couldn’t forge and selectively breed.
There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism. Only the former can be statistically modeled to characterize and work around the noise with any reliability. If you measure hallucination rate for one prompt it tells you nothing about hallucination rate for another prompt.
In engineering you have tolerances to deal with non-determinism. ”Within these bounds” and ”given these assumptions then…” is the foundation of building something _on top of_ those things. LLMs are the same, it relies on heavily exact Turing machines as input but its output is entirely unstable. Even if you can get determinism it will never get anything resembling ”bounds” out of the box. That makes it a poor foundation for building on top of. Ie it’s not a screwdriver, it’s the monkey who’s holding it.
I do not understand the need to argue that monkeys are better than screwdrivers at screwing. Just let the monkey be the best version of a monkey.
You’ve clearly not seen monkeys. /sarcasm
I would argue that management is a better discipline to pull from for employing LLMs; It is better equipped to deal with non-determinism and going completely off the rails.
I think it's safe to say that computing should be mostly deterministic. I know compilers use heuristics that are stochastic, but come on. Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
> Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
Computing has never really been deterministic all the way down. Storage gets corrupted, RAM has soft errors, networks drop/reorder packets, schedulers race, caches go stale, distributed systems partition, query planners change plans, ...
Obviously those become tolerable because we've developed layers of contracts and understanding the bounds around them. Error correcting codes, checksums, retries, consensus, idempotency, false-positive rates, SLAs, etc.
If the abstraction is “delegate this task to a junior engineer, analyst, lawyer, designer, or support rep,” you dont expect deterministic behavior. There's review, constraints, escalation, checklists, tests, and accountability.
There's a pretty cool article about trying to understand electronics and fix a radio the way we try to understand biology. https://mappingignorance.org/2014/07/02/biologists-cant-unde...
So now instead of improving the tools of biology so we can actually understand it deeply - we increase complexity of IT so we have to rely on muddy, side-effecty tools of biology to try to infer some of the properties of the systems we made. That's depressing.
Our ancestors also had an infant mortality ratio of 4/10.
Determinism is way undervalued.
People use the word "determinism" when they really mean something akin to "linearity", i.e. the predictability of a change in input on a change in output. Compilers for example are both deterministic and "more linear" in the sense that I can tell what the output will look like given a change in input (yes yes optimizations violate this to a small degree). LLMs can be made totally deterministic, but a seemingly insignificant change in input can create a drastic change in output, which is the characterstic we don't want.
If there is something to be said, I have to ask. What specifically is there to be said?
Non-determinism is not inherently bad. We have had useful non-deterministic tools for (in computer years) several generations.
Non-determinism combined with bugginess?
That's a terrible combination. It is impossible to gradient-descent your way into a working prompt.
I'm not sure that's true - you may be over generalizing
Probably not many here know the processes needed to turn sand into silicon and the expertise needed for hand grinding fancy lenses used in lithography. But we do know those things are needed the approximate philosophical need.
I think the danger comes when or if we fully automate a lot of the low level infrastructure tasks to the point where future generations do not even have the conceptual framework for how the tech they use is created
I agree, but having abstractions allow us to learn things in a system that are broadly applied.
This is out the door for AI generated systems. Unless extreme care is taken there is is no consistency in these new-found codebases. New paradigms are introduced left and right, because in the eye of the LLM, and the prompter, they worked.
It didn't matter that the same pattern was repeated 37 times in a slightly different manner.
So your knowledge now is no longer portable.
It used to be that you could look at code, and ask why? And usually (not always) the answer was something like: 'I tried x, y, and z, and those didn't work', either by past experience or current experimentation. (we could argue if the current experimentation should be documented).
But for LLMs they put in something complex for no other reason than that it does what is requested. Reading a string from a valid source byte by byte until you hit \0 is valid, and it'll work. But if you take a step back at, read the API docs of what you're consuming, and then consider: if the API says X, why am I testing whether that's correct?
If you wish to make an apple pie from scratch you must first invent the universe
> Generally speaking, we know how the level of abstraction that we were born with works
I'd say it's generally true that the majority of jobs of an era deal with a similar level of abstraction, and that's why most people stay on it. However, I frame this as being born with technical debt, and it's my obligation as an engineer to understand what the previous generations have built, and where it makes sense for me to work, directionally.
I think it's more that "Some human some time has known how it works", not that "Any given human knows how it (all) works.".
But yea this glosses over a bit trial-and-error designs and, so to speak, "genetic optimization" kinds of designs where we just try random stuff and say "Hey, this works. Not sure why, but it works.".
There was a period of a century or more where steam engine design parameters were determined by experiment and gut feel. The physics of fluid dynamics required to explain why a design is optimal just did not exist yet.
There's a lot of places in history where engineering far outpaced the science required to explain it.
If your design work is not based upon science and mathematics, it is not engineering.
Not saying engineering is required to build or improve a product; it is not.
"try this" is not engineering.
I disagree. For example, the Wright brothers did air foil engineering empirically without full knowledge of the underlying fluid mechanics. Early and innovative engineering will often be less based on existing scientific or mathematical knowledge, while still using some of the methods. It's still absolutely defined as engineering.
Of course it is. The science comes from reproducing what you tried and getting the same results.
You may not have a good enough model for this observation to allow you to predict the results of other starting conditions, but it’s a model nonetheless, one that narrowly applies to the conditions you tried.
The level of abstraction when I started was that you had to solder the components (resistors, capacitors, chips - well IC holders, chips were expensive) to the PCB board - that was "building your computer", none of this nancy-boy rubbish of "plugging stuff together" with "slots that can only take one type of card".
I'm fairly certain I know how it works. Being a physicist helps with the even-lower-level-details if you want to start talking about transistor doping, or electrical circuit theory, for example
> The issue with this is that we don't know how it works. Generally speaking, we know how the level of abstraction that we were born with works.
What? Definitely not. I went to university and my first two years were subjects where I had to understand really deep levels of abstractions. I had to build logic gates, I had to work with hardware, wires, etc. I didnt see the point back then (I never used any of that professionally). The same about algorithms, databases, and a lot of things. But now I find it valuable and thankful that my professors (and whoever designed the career) considered important topics that I had to lear.
You started with logic gates. How much EE did you do, or the actual physics that makes the transistor possible? Those are the previous (deeper) levels that people had to know before they got abstracted away.
When I took software focused computer engineering around 2010, we still had courses that took us all the way down to transistors and even the physics of P and N junctions and how that applies to CMOS. (And even some basic analog electronics.)
Did I end up an expert at those layers? Of course not, but I know the basics and I know enough that if I need to I know where to start learning more. Just like I wasn't a C++ or hard realtime expert after university either, but now a decade and a half later I am pretty good at those (and a bunch of other skills that ended up relevant to my line of work).
Basically, none of the layers are "magic" to me. Even if I don't know the details of it, I know the general principle and I know I could learn more if I need it.
(I think you naturally end up an expert at the layer(s) you work in, and the knowledge tapers off as you go down (or up) the stack. For example, I know a fair bit about how the CPU works (cache coherency, pipeline stalls etc), I can passably read x86 assembly, etc. Because they affect the layer I work at (hard realtime systems C++ and now also Rust). I know far less about web dev than hardware.)
Similar experience in the 90s, but we don't really know the intricacies of doping silicone, or smelting metal to make the pins. And what about mining it?
I think the last time people knew how things were made was in preindustrial societies because they had to build everything themselves (whatever little things they had)
I was once in a museum in Bolzano, Italy. And they had the ice man mummy there (we call him Ötzi). He died like 5000years ago. And his "axe" was made of copper from some mine very far away. So even this guy probably did not know how everything he had was made. There is a theory that he was probably very rich. So maybe less rich people were more in touch with what they owned. But still I found it fascinating that even so far back people relied on technology and materials that they didn't really know about.
I partially disagree. I know the basic high level concept of all those. Would I be able to reproduce state of the art results on my own? Obviously not.
But a core part of the engineer or scientist mindset is curiosity for the sake of curiosity. Just the fact that I don't know something is enough reason for me to poke at it or otherwise learn more. Same reason I still take apart broken electronics as an adult and try to find the fault (and sometimes even repair it).
By the way, mining silicon is particularly easy: it's basically sand. The difficult part is purifying it, especially to the levels needed for modern nm scale chips.
A more useful question than making high end silicon would be: could you with reasonable tooling reproduce basic electric components? I'm talking things like light bulbs, resistors, generators, perhaps capacitors even? Just the basics crappy versions, not modern highly optimised surface mount components. And I think the anwer is yes (for me personally) if I had access to metal wire and sheet stock and industrial revolution era tools.
Simply by the fact that you say computer engineering, you already went deeper than 99% of "computer people" in 2010
I have similar experience with the poster, and the way I read it is, "from the things I build here are some examples". I did learn about advanced physics topics that enable transistors, and even did some experiments, but for fundamental stuff you "don't build stuff".
Did I do all physics or all electronic circuit design or all software stacks? Definitely not. But I spent 3 years learning (and building) about lots of stuff.
The one thing I appreciated about my Computer Engineering undergrad - and it took me a few years to fully appreciate it - is that yes, we did cover those levels.
The first two years were shared with Electrical Engineering. The second two years started to specialize towards Computer Engineering topics.
* Physics and chemistry.
* Circuits.
* Transistors.
* Logic gates.
* FPGAs.
* Assembly.
* Compilers.
* CPU and hardware design.
* Operating systems.
* Networking layers.
* Programming languages.
* Computer graphics.
Did I master all of the above - absolutely not. I loved some of them, struggled with others. Generally the cut-off for how my brain works is logic gates, I was never strong at the levels below that.
But we did cover them, and I could honestly say I had at least a rough understanding and mental map of everything that happens inside a computer from the point where it's plugged into an outlet, to the point where pixels show up on the screen.
Yes, they taught that too.
how did you make the transistors? made your own vacuum chambers?
i had to make logic gates and so on, but i wouldnt say i really learnt it, even if back in highschool i learned all the different things a 555 timer can do
We had a photolithography lab when I was at school. It's not that complicated to make a few npn transistors. The difficult part is doing it well.
Exactly.
Please get used to this sort of depressive, absurd and out of touch tone from HNers, it’s literally all they do now. Don’t bother calling people here hackers anymore, they have checked out emotionally and spiritually.
What's absurd is thinking because you took a logic class and made a flip flop 30 years ago that that is the ground floor and that it means you "understand how it all works". You're not building a CPU from logic gates and you don't know how it works. If I put you or OP in a room for a year I highly doubt you could build an 8 bit Atari-like CPU from scratch. I worked with wires and logic too but I'm not arrogant enough to say that I know how it all works.
Don't you think there is a difference from "knowing how it works" and "reproducing every aspect of it at the level of the state of the art" ?
Also, your example seems flawed if you restrict to a certain product. Can I build a compiler from scratch? Yes. Can I reproduce in a year a compiler with LLVM/GCC performance level? No. Can I build a compiler from scratch in a year from a room if I need to starting mining from metals, building transistors, then building the first assembler and then implementing the compiler? You can imagine the answer.
Not really, at least not in this discussion. An assembly programmer gets logic gates, a C programmer gets assembly, etc etc. And each one can say to the one above "well you don't really get it if you can't bang out some meaningful work". Otherwise idk what we're even talking about here. Having a vague understanding of something isn't "knowing how it works".
Yeah, but if we use your criteria, people have to invent their own atoms out of individual electrons, protons, neutrons, quarks, etc before you'd admit that "hey, they did build a compiler from scratch".
I think a fair bar for "from scratch" is stopping at the point they need an expensive fab process (or how else will they reproduce the workings of semiconductors?)
Code: The Hidden Language of Computer Hardware and Software
https://www.amazon.com/dp/0137909101
Explains how to build an 8 bit calculator with wires, switches, and light bulbs and then keeps on going.
The parent's argument can be boiled down to "You don't know absolutely everything therefore it's fine to know little or nothing." fallacy. It's the Chewbacca defense of AI boosterism.
This is a very cringe take.
Me and OP aren’t super heroes, we can’t do what a team of talented individuals created even if that team existed many moons ago. That isn’t the point.
We both questioned the tone and the conclusion of the comment.
> If I put you or OP in a room for a year I highly doubt you could build an 8 bit Atari-like CPU from scratch.
I believe I can, depending on what "from scratch" means. With nothing but transistors, resistors, inductors, coils, capacitors, I can probably do apretty poor general-purpose CPU. Maybe something like a stack-based ISA or, if I had more time, an accumulator-based processor like a 6502, but with an 8-bit bus.
If you're going to ask me to create transistors themselves, well I believe that needs specialised equipment.
That was one the major criticism I have of my CS degree's program. Perhaps the knowledge I wish for is not truly CS in the mathematical/theoretical perspective, but I believe it would have been absolutely valuable to learn.
Name checks out
come on now.
so even if you're right, checking out emotionally and spiritually just means more life lived. That ain't some kind of bad thing.
life is good sometimes. hard sometimes. and it's long sometimes, so give people a break.
> checking out emotionally and spiritually just means more life lived
What exactly about checking out makes it 'more living'? I sense a false dichotomy here
easy: we all need breaks. breaks are not bad. living doesn't only mean "doing actions"
I don't think OP argued for any type of relentless work.
Yes, thats exactly what uni is for.. To learn all those previous centuries of stuff before you can either start contributing to the corpus yourself, or go get a job (Where you will pick up more immediately useful stuff)
We really dont understand how AI is working, even the earliest "genetic algorithms" could be incomprehensible, but computer systems in general, they're not really that complicated.. its like an audio mixing desk..it looks insanely complicated until you realise it's just the same few knobs repeated many times for many channels. High level languages, compilers, assembly, machine code, nand, mosfets. A single person really can understand it all.
Wait a second, its layers all the way down
Exactly, I'm over 50 and I remember all the complaints about script kiddies who looked at windows bat files as opposed to all the 'real programmers' who knew C and Assembly and used VIM and linux (which is still going strong)
(ie https://www.explainxkcd.com/wiki/index.php/378:_Real_Program...)
but now also as an AI engineer we have to learn how harnesses, sys prompts, various models, tokens LLMs etc all work so a new abstraction is born..
layers changes, nerds and ultra-specialist nerds will remain
nostaliga is always for the last layer- the one you remembered from your teens and 20s.
I expect 20 years from now there will be people reminiscing "remember claude.md? Back then we knew how it all worked. We had deep control. We counted tokens, chose the model..."
A (sad/funny) anecdote. Someone told me earlier this week "Not using a coding agent today is like using Vim for development instead of an IDE, you're just lagging behind".
As someone who's been exclusively using Vim for my development (and can definitely integrate it with AI workflows), that's just an insanely silly opinion. But I guess it shows how the next generation thinks about these tools that they've heard of but never actually bothered to learn.
Script kiddies didn't look at the .bat files. They ran them without a clue how they worked. There was some respect for the people who wrote those scripts but none for users who couldn't write them
It is no small feat to put in words that we are losing something almost as quickly as we are gaining something. The undertone, despite leaning into nostalgia boils down to losing control and this uneasiness I feel growing daily. It is already shocking to a certain degree seeing very young people not being able to use a computer in the narrow sense because all they ever learned was touch interfaces and apps. Curated content, curated interfaces - everything that resembles some kind of hardship ironed out in thousand steps of iterations to appease the market which means the lowest common denominator.
But I also see that the people who can create the absolute most and the good things and the working things and the maintainable things nowadays are the people that have gained a tool, but not lost the knowledge of the medium we are using it on because we are tied to this old world so perfectly put under the spotlight in this blog post.
The financialization of everything is what's ruining everything. In the computer and internet realm, there's warm nostalgia from hours spent tinkering, building one's own PC, reformatting C drives due to malware, searching for "snippets" to add a forum or pimp a myspace page. But inevitably the money incentives come to dominate. It's all of our doing, we all dream of a better life to put it charitably.
Now everything is a means to a commercial end. Tinkering for fun and knowledge just isn't profitable. And it matters less and less what each our stance is on money and capital if the people that optimize for money and capital gobble up all the money and capital. Of all that's going on, the wealth gap is what's most troubling to me, closely followed because it's closely related is "post truth". I think post-truth is roughly caused by the fact that people are happy to believe what they want to believe toward some commercialized and/or idealogical end. You're much more likely to hate and blame your neighbor when you look around and you're the one not doing too well.
In my mind, 'tinkering for fun and knowledge' was never meant to be profitable - if you're tinkering with the express aim of making a profit i'm not sure you're really 'tinkering' so much as you're trying to create a product/service/output for someone else to pay for?
There's absolutely nothing wrong with doing that - I'm just not sure the 'ethos' of tinkering has anything to do with trying to make money and is usually reserved for describing someone playing about with something for their own enjoyment/fun with no desire to make money.
Now, of course some people did find that their tinkerings were able to make them money, but I think at its base it's a term I'd tend to say implies doing something for fun/for themselves, rather than doing it for profit?
In my experience there's still plenty of people out there tinkering just for their own personal satisfaction, but of course there's almost certain a whole load more people out there 'tinkering' to try make a profit.
Agreed, I'd say tinkering for profit has become the defacto optimization and tinkering for its own sake is less fashionable because it commands no attention. and everyone is obsessed with the modern currency: attention.
To be clear, I'm seeing this as an observed phenomenon, not that everyone made up their mind that they hate tinkering and love money. I just think it's getting really really hard to exist in the world as a normal person when the entire human collective is getting pumped commercialized hyper-media from all angles at all times now 100X'd by genAI bots. It's really exhausting and so you either opt-in to the game, monetize your now AI side-hustle to pay the rent, or opt-out and live in the woods. and get packages delivered by Amazon. heh.
"warm nostalgia" "reformatting C drives due to malware"
Processing error.
hah, that made me laugh. To this day I'm so proud to have learned how to repeatedly just wipe my PC C drive when needed then reinstall XP or whatever. I'm just a web-dev that got their start by googling "how to make a website". As I leaned in there came a turning point where I wanted badly to switch form windows, php, scripts, to Ruby on Rails. I worked up the courage - eventually - to partition my drive and install linux. After that, I got to experience a proper terminal and the rest is history.
We are on average losing control, but you individually can choose whether to lose control or not.
Programming languages, UNIX, debuggers aren’t going anywhere. There is more to computing than what your boss demands and what is hyped on tech forums.
In fact I believe the indie/handmade scene will grow substantial if not boom, even if just as a hobby for most. Showing what you have made with your blood sweat and tears will elicit more praise and delight when you could have just asked a machine to do it all along.
> It is already shocking to a certain degree seeing very young people not being able to use a vehicle in the narrow sense because all they ever learned were the mechanical controls of the so-called automobile.
We could do this forever.
I think there's a fallacy where someone points out one instance of a larger trend which will, when taken to its logical progression, lead to an undesired effect; and then someone attempts to rebut the claim by pointing out that the trend has existed before and the undesired effect hasn't happened yet, so any concern is nugatory. I'd call it the grippery slope fallacy, complement to the slippery one: we haven't fallen down the slope yet, so we can't fall down it. What if an individual instance of ignorance is acceptable because people still need to have understanding in other areas, but if all understanding everywhere is eliminated then we all suffer?
the difference is that an automatic transmission doesn't make the car work worse. The modern UX landscape would rather board up a room because the door has a sharp handle than figure out how to make the handle less sharp
ETA: Or, to put it in car terms, we were all forced to take cabs (except for the people who were interested in driving, who became cab drivers) because car crashes happen or my sand eating neighbour couldn't tell which pedal was the brakes
It wasn't all that long ago that automatic transmissions had significantly worse reliability and fuel efficiency than manual transmissions.
Absolutely - you used to have to control the richness of the fuel mixture manually. You used to have to crank it to start it, manually interact with a clutch to shift gears, etc.
I appreciate the tactile joy of interacting with simple systems like those, but most times I just want to get where I'm going. Freeing my attention from those tasks allows me to pay more attention to the (inattentive) drivers around me, and try my best to not die.
Eventually a computer will handle driving for most of us, and we can lament about all the things we've lost there too. If you zoom out, most of us don't have an in-depth understanding of how an entire city works (power, garbage, sewage, maintenance, public services, politics, etc), and couldn't coordinate the various activities to keep it running if we had to. We live in towers of abstraction.
My read is that there does seem a clear difference between simple -> advanced machines vs simple -> "smart" machines. Nearly every smart machine is bullshit enshitification-in-waiting. Rent-seeking in-waiting. Smart tvs, smart cars with touch-screens. some would argue apple products. These things proclaim advancements but what they really do is black-box and dumb everything down to the lowest common denominator, then quite literally impose control over the air, and shove ads to you.
I'm all for just getting to where I need to go by using the appropriate tool, like a reliable car. But no not if it means foregoing the liberty of other options.
This time is different though (which has also been said every single time). But I'm worried this time it's true (also said every time). Doesn't help with the unease though.
The scale of it is certainly different, if nothing else
We have never before seen every single profession disrupted to this degree, not even the introduction of the personal computer introduced such a dramatic shift
(in recent memory). Surely the industrial revolution was a much larger disruption.
Maybe? Either way, we're still talking about one of the largest disruptions that global society has ever had
No. No economy ever had essentially every single major company spending a significant fraction of its budget on hiring auto mechanics. Which is to say, for all the changes the automobile wrought, the role of the computer in industrialized society eclipses it tenfold, a hundredfold, a thousandfold. For an individual in many modern societies, being denied access to a car is already effectively crippling, and the idea of being denied access to computation threatens to be somehow even worse.
A person dies every 26 seconds in a road traffic incident somewhere in the world. A big part of that is people using machines they do not understand.
In what sense do they not understand it? What could we teach them that they don't already know?
Almost no part of that is due to people using machines they do not understand.
There was a sweet spot with computer technologies for some decades where hobbyists could afford to experiment and even push the envelope in the nascent field of computing - similar to genetic radiation, many niches were formed and rapidly filled. The computing biome has evolved to the point where most entities are not operating at the low-level abstractions that were once the only means of interacting with the computing environment, instead they operate now at the highest levels of abstraction we are capable, so called "natural language".
"The difficulty was the knowledge. You came to know that machine the way you come to know anything that pushes back. The resistance was the whole medium. You only ever know the things that you can lose to."
We who grew up in this era formed a hands-on engineer's knowledge of these systems, built from experience and practice, learning these layers of abstraction as the bleeding edge developed. Many these days have entered into a world where there are easy answers abound, they just might not be right, and one has to gauge how much they care about correctness.
This is definitely true, and we definitely need something similar again. I've been using a game ("The Farmer Was Replaced") as a jumping-off point for teaching the kids Python, but the more I think about it, the more I think that they need some sort of hardware package similar to an old Apple //e that gives them just enough rope to hang themselves. It was easy, back in the day, to learn a ton (even assembler) on a system like that, and I feel like there's some value in rewinding the clock back to that point, forking the experience from there, and seeing what a new generation of kids will cook up.
Zach Barth and his former studio Zachtronics released several entertaining puzzle games based on idealized fake assembly languages. These are a lot of fun and useful for introducing registers and multiprocessing.
I keep coming back to the idea of taking that concept and just running with it, IE building up a toolchain alongside some concrete tasks to solve with it and just adding more detail until we reach a reasonably sophisticated level
I've been staring at 0x10c, carwars and battletech and there's a sense that I could build a sort of programming / engineering Zach like
If you want to go the old computer route, find an old vic-20. It has the best manual for any computer ever made imo
I'm one of the greybeards who has the 2400 BAUD modem negotiation tone sequences emblazened in my neurons.
For a while I've been meaning to set up some Wireguard connections among some of my systems. Being as busy as I am with work and family, I've relinquished that to Tailscale for now.
Sure, I could have sat down and jumped through the hoops to get everything set up and working across my various hosts, including network routes, firewall rules, key pairs, systemd units, and so forth. But the "cheap and easy" alternative was right there and worked (except when it forces re-authentication).
With LLM agents, I was able to effortlessly analyze my existing network and produce tailored scripts to do precisely what I wanted. All I had to do was review the scripts for potential security issues and what not. Looking at the script, there are 3 or 4 specific tweaks that needed to be made to my network routing rules given my network topology. I could have read a few man pages and iterated on the script by hand to eventually get there after maybe an hour or two of futzing.
The availability and effectiveness of the agents is simply too tempting for me. I'm not sure what this means about my skillset, or if that even matters any more. I am fairly confident that, so long as my brain still works well enough, I'll always be able to RTFM and figure things like this out myself. At this rate I wonder whether my kids will have the same ability. And I also wonder how much that will matter.
Regardless, I'm still helping them figure things out the "old way" without over-reliance on LLMs. One thing I'm fairly certain about is that failure to develop problem-solving skills can only put them in a worse position in life, no matter how capable AI becomes.
The thing is - everyone complains about AI stealing our attention and understanding. But you can just as easily use an LLM as a tool to gain a deeper understanding. It's just the default path for most folks is "Hey clanker, do the thing" rather than "Hello clanker, please tell me about how that thing works".
I've done at least a little of the latter, and it's amazing how underrated it is as an educational tool - especially for the solo individual.
And you can use a TV to read books, but nobody does.
Every technology has ways it can be used, and ways it wants to be used. This one wants to be used in a way that produces outcomes we won't like.
Nobody has time anymore to read a textbook.
@ryancbriggs - 18 Oct 2024
> When I was young I fixed my parents’ computer and now that I’m older I fix computers for my kids. Are we the only generation that knows how computers work?
https://x.com/ryancbriggs/status/1847391612428517844
https://xcancel.com/ryancbriggs/status/1847391612428517844
No, that person is just the only one in his family who knows how computers work. There's plenty of people in generations both older and younger that know too, just apparently not his parents or his kids. Some of the most impressive assembly written today is by people younger than 20. It'll be fine.
It's broadly true though, that younger people struggle with the fundamentals of computing, even if there exists specific examples of young people accomplishing exceptional feats.
We raised a generation of people on consumption-based computing devices, which means they don't develop the skills necessary to produce things using computers. Of course, we can teach these skills, but to do that, we must first acknowledge it's a skill which needs to be taught.
I agree; the situation "young people know all about computers" was maybe a twenty-five year phenomenon; before that, there weren't enough computers for it to be generally true, and at some point you no longer needed to know all about computers to successfully use a computer for the sorts of things the general population does.
Most people in my generation didn't know the fundamentals of computing either. That's why I get paid the big bucks to make computers work for them.
maybe another angle to this that I observed: my working class dad who had little education happily taught himself a bit of programming with computer magazines in the 80s an 90s. He actually could figure out his way around a command prompt.
I went on to study CS and teach undergrad courses and what I noticed, this started already maybe a decade ago, is that CS students who we handed bootable linux usbs couldn't figure out how to set their system up. And not just that, they just kept emailing us with statements like "it didn't work, what do I do?". It's not just lack of knowledge but complete helplessness when something doesn't work in 2 minutes. That's the biggest problem with this reliance on ChatGPT or whatever.
I think the young generation is in an even worse position. Not only do they not know how computers work, they don't even have the basic DIY problem solving attitude our parents have.
“The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads.”
I disagree. If you ask a model for a manual and it regurgitates that manual from its training data, it’s over-fitted. It will regurgitate something that looks like a training manual. Or whatever fits your query about training manuals.
You still have to push back on them sometimes when you spot an error. And you can only spot them if you already know what you’re looking for and should expect. Otherwise you have to ignore the output and just get the links which… could be outdated or made up as well. You’ll never know until you verify the results.
And this degrades with compression and time.
There’s no royal road. I agree that trying and getting frustrated and having to take the effort to understand something pays off in spades. I just think it’s still worth it and vastly under appreciated in this era of “everything fast, now.”
I came into the US tech industry in 2011 from a CS program (programming on my own longer), which was basically a pipeline into Java or web dev at that time (the classes on other things like OS, compilers, and FPGA were a thing, but seen by most as a hurdle not a promising or practical career pathways).
Most of the people I dealt with for the decade following that (i.e. pre-LLM) had little clue how any part of computers worked, and I don't intend that to be particularly pejorative, it's just that it's been somewhat easy to carve out a niche doing something to do with programming while having little understanding of the small or big picture.. all heuristics and applying working patterns.. for a few decades.
You can still understand whatever you want, today, with added tools. It's just a choice to turn your brain on or off. I think LLMs are perfectly fine as a learning tool to interrogate a subject, do comparatives, and then formalize your understanding by reading the sources. At a macro level brain rot is real, but it cuts across all generations and it started long before LLMs.
> At a macro level brain rot is real, but it cuts across all generations and it started long before LLMs.
The thing is, if there’s a way to do something easier people will generally just do that. All the nostalgia of the olden days has as a component that there were no alternatives as the example in the article highlights.
We can try to preserve the hard way to things that gets improved, but the hard way will only ever be niche if it doesn’t become extinct entirely. There is no going back.
Sure, humans are attracted to laziness by default and at its best technology must be used to lower toil and suffering (unfortunately there are unintended consequences etc for philosophical debates).
The bigger issue with brain rot is attention spans. Social media has ruined attention spans.
This might just be a personality trait of me, but I really like technology that's transparent enough that I can fix/modify/understand it. Opaque technology gives me an ick. It's not really a complexity thing -- Linux has this property of being transparent even though it's extremely complex. The way he describes old computers.. man, I loved that about them, there was nothing _secret_ going on, you weren't worried about telemetry or weird background processes. Then again, I still drive a manual transmission internal combustion car in 2026, so maybe I don't represent the world.
Then again, maybe there's light at the end of the tunnel. Even though AI boosters kind of annoy me, I do appreciate that they're tinkering and hacking and working to understand the technology on some level. I think there's always going to be some small number of people that are like this, in the same way there are still people that look at disassembly and understand CPU caches.
> You opened files like autoexec.bat and you read them. Sometimes you built a boot disk for a single game. A floppy whose entire reason to exist was to start the computer in the configuration that one program demanded.
Is the premise here correct? I'm not sure that I'm convinced that a 1990s computer user who knew how to edit autoexec.bat or insert a floppy to boot their computer "knew how it worked" in a meaningful sense.
The stack of abstractions is deeper now, and all indications suggest it's not going to stop deepening. But I think the abstractions were already quite deep by the 1990s.
(I think the classic error here is a demographic one: computer nerds always poke through abstractions, because they do it for fun. I don't think that's going to stop, anymore than web browsers stopped people from writing kernels. If anything, we write more low-level code than ever, because access to the prerequisite knowledge is less gatekept than before.)
I doubt you ever understood the solid state physics, semiconductor fabrication processes, supply chain logistics, monetary policy, shipping routes, mining engineering, etc. "Knowing how things work" is a stone-age attitude.
Obviously no one knows every detail all the way down the stack. It's not possible, but having some level of understanding of how a system operates is useful especially if you ever have to fix it yourself or explain why/how it is broken.
I do at least try to understand all of those things.
It's not hard to understand the physics and fabrication.
The rest of that isn't part of how a computer works.
There is plenty to learn about hardware / integrated circuits.
> "Knowing how things work" is a stone-age attitude.
Should we petition to rename this site to "Stone Age News" or something then?
Maybe I'm wrong but I always thought figuring out how things work was pretty core to the Hacker mentality.
If being a Luddite means I'm the only one who cares to understand anything, I'll accept the label gladly.
Modding communities are still going. Kids, afaik, are still playing around with hosting minecraft servers or whatever is en vogue/cool/meta/etc nowadays. DIY 8-bit computers are gaining popularity.
IMO the fact that something's become very mainstream doesn't necessarily mean it's been watered down for everybody. There will always be people with various levels of curiosity and enthusiasm.
Yeah, I don't think human curiosity can be extinguished. It can be disincentivized, yes, but not extinguished. Nerds will always be nerds.
I resonate with every example given - setting jumpers by hand, sound card interrupts, autoexec.bat. I'm also a happy user of LLMs and agents. This article captured for me what is lost - which, as others point out, has long since been lost, if ever had, in other fields (e.g. modern cars vs. the Model T). I wouldn't go back, but I can still have a sense of loss.
Beautiful writing.
With how seriously rise of LLMs damages schools and university education, and thus next generation of comp sci graduates, at least we can hope to secure our arcane knowledge jobs for long.
Speak for yourself but I certainly don't know how it works.
Also, if you don't think the difficulty is the knowledge, I'm sorry but I must disagree. There has been more than one government or corporation or large institution that has tried to destroy history and knowledge in the past.
I think it's fair to say that knowledge is also at risk.
>The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads. They will recite, flawlessly and forever, exactly how all machines work. If this were only about competence, it would be the most secure moment in the history of computing.
Obviously this person doesn't have enough experience with the current crop of AIs. This is the exact reason why we should be concerned in the first place. Someone has to design and build the next version of everything, and AI isn't qualified to do the job, and might never be.
>What is dying is acquaintance. The plain, unglamorous intimacy of having fought a particular machine, and lost, and gone back, and finally felt the thing give.
This is true. But then again, who among us had to get up in the middle of the night, and trudge out to an outhouse? Who had to shovel a bit more coal into the furnace? Who had to pump water, and heat it on the stove? Who had to split wood, and stack it for the winter?
There are many, many things we're no longer acquainted with, and that's sad, but still ok. What's not ok is losing the competence required to maintaining the infrastructure and supply chain supporting society and civilization.
Two thoughts:
1: This is why I prefer console games. I just want to have fun without fighting with the machine.
2: There are plenty of people who appreciate old techniques and methods; and keep them alive. Think of going to a museum and seeing someone demonstrate an old craft or reenact how a craftsman did their job. For example, in my town there is an old, water-powered corn mill that still runs and sells corn meal.
I wanted things to be a little easier, but not this easy
I worry less about whether people know how it all works.
I worry more about whether people care and consider it a problem when nobody knows.
> The graybeards are aging out, nobody compiles their kernel anymore, and someday something deep will break and there will be no one left who can climb down and fix it. Maybe. But I think competence is the part that’s fine.
> knew a beige computer in 1995 that wouldn’t run a game until I had rearranged its bits by hand. More dependent than ever
If you look at previous article from this author, it says how Mac is amazing and how Linux sucks. Kids like that in 1990ties would buy expensive consoles, and would not deal with hack PC's to get free games.
Many people today are still dealing with cheap shitty hardware, 7 years old Android phones and sketchy ROMs... Just because there is no other option!
https://unix.foo/posts/it-will-never-be-the-year-of-the-linu...
>What is dying is acquaintance. The plain, unglamorous intimacy of having fought a particular machine, and lost, and gone back, and finally felt the thing give.
I think this is a universal feeling that accompanies any technological innovation. My phrasing is that new technology unbundles the thing people want from the craft that was formerly required to get it; any craft requires someone to learn and achieve through struggle.
> The machine was made of edges and you cut yourself on them, and that is how you found out where they were
> ...having fought a particular machine, and lost, and gone back, and finally felt the thing give
mayhaps this is because our computing paradigms are stuck in the 70's
I can't help but think this type of step function change has happened with computers before.
The change from punch cards to magnetic storage certainly made it so you didn't have to "know how it works" for every single bit.
The change from machine code to a language like Fortran brought about such an abstraction that a Fortran dev didn't "know how it works" at that same level anymore.
At this point, the layers of abstractions between using a React component and something being rendered is immense. React VDOM, the real DOM, browser render engine (which sits on abstractions like ANGLE, skia, etc), calls OS APIs, which call driver APIs, and the lowest level of anything it is still C/C++ that is compiled (abstracted) to something closer to what the hardware expects.
I won't bore you with the ChatGPT output details, but it estimated at least 35 meaningful layers of abstraction between a React component and being rendered to the screen. LLMs seem to be the latest level of abstraction to make it so we "know how it works" less than before.
It's hard to articulate why, but 35 layers of deterministic abstraction feels very different from 1 layer of nondeterministic one.
In particular, I think every developer has experienced the need to jump "down the stack" to debug or understand something (even if not all the way down). Certainly, I think any senior developer should be at least conversant in the first few levels below wherever they "live". But this seemingly ends up looking fundamentally different in the interaction mode of an LLM, because you'd just ask it to jump down the stack for you
This is also a fear I have, I am answering it with building bigger things with AI: what I loose in expertise in the details, I gain in the pleasure of architecturing bigger things.
Why do you have to know how the whole thing works?
You can know how your entire farm’s water system works. But a city?
Somewhere in between those extremes, you just have to let go.
This is sort of the story of the telephone system of the 1950s-1970s, or electricity in the early 1900s, or cars from 1950-1980s, or airplanes from 1910-1939.
I have no idea how an electrical transformer works (well, other than the bare theory I learned in physics courses), or how power gets from the power company to my house, or how the circuits in my home are setup. I plug something in, and it works, and occasionally I throw a breaker if something is malfunctioning. There's no resistance there (pun not intended), and there shouldn't be. People got killed trying to wire their own homes.
I used to read about phone phreaks from the 1970s that could do black magic to get free long-distance phone calls. When I grew up in the 80s, that was basically gone. You picked up the phone, got a dial-tone, and called. And now it's really gone, with everyone having an encrypted cell phone connection over 5G, and your IMEI and IMSI being phoned home to every tower you connect to.
It's the nature of technology and capitalism. As the technology matures, it gets hidden away to become increasingly invisible to the end user, so you just do what you want to do with it. And then the engineering resources get spent on new problems.
> And now it's really gone, with everyone having an encrypted cell phone connection over 5G, and your IMEI and IMSI being phoned home to every tower you connect to.
people will hack enterprise pbx systems. The stream just flowed somewhere else.
I'm still nostalgic for the times when I had to enter the bootloader using the keys on front panel of the PDP-11 to then loaded the OS from paper tape. (not to mention multi pass compilers using paper tape between the phases...)
> They will have a tool that does everything and asks for nothing, and they will be as easy with it as you are with the light switch you never once thought about.
Oh come on! Of course I wondered how a light switch work, and I then learnt. I remember taking apart broken electronics as a kid, and that later morphed into also trying to repair things. Including the computer (both hardware and software wise). I remember ending up reinstalling the OS so many times as a kid on the computer I had access to when I broke it in various ways past what I was able to fix.
Sure, not everyone will have that drive to understand how the world around them works under the hood. But for engineers and scientists I would expect a far higher percentage to have that sort of personality.
It doesn't matter if it isn't "pushing back", just that I don't understand it is enough to catch my interest and a reason to go poking at the thing.
The thing is, yes it will still happen, but at substantially decreased rate. Groups of people banding together to solve a problem or figure a solution will slowly stop forming. Everyone will be out there learning just for themselves.
The collective knowledge of people will likely ebb. Things will be different and individuals will have to adapt.
There is a lot of useful information on how things work out there, if you want to learn. YouTube videos took the place of popular science magazines largely, and perhaps it will be some different medium tomorrow. But the knowledge is there, and just like in the past you have to go and look for it, just in different places.
There are also still courses you can take to learn practical skills, I for example took a short blacksmithing course a few years ago. A lot of fun and way harder than it looks (at least if you want to get good at it, my goal was just to give it a try and have fun, and 5 evenings was enough for that).
The graybeards are aging out, nobody compiles their kernel anymore, and someday something deep will break and there will be no one left who can climb down and fix it.
I dunno. I suspect it'll be slightly different in the OSS world[1]. There will be some folks who can climb down there and fix it, but since that skill is no longer valued (as in, being able to do that isn't valued by most users now, much less in some hypothetical future). And they will be expected to do that, now, without thanks. And when they say "I'm too old for this shit...you can go pound sand" because this was one unpaid insult too many, users will shrug their shoulders and load windows or buy a mac because those companies are paying & tasking people to keep things running. And by inches oss ceases to be a thing.
[1] Yes, I know there are many oss devs who get paid for their work. Many critical contributors do not.
Gee, I sure hope pump 6 never fails.
Yes, it was not easy to install and play a game on the computer in the 90's. Today, we also don't want to wait for the game to boot.
I wouldn't calling it learning more and more like "cycling through SoundBlaster DMA and IRQ options until the sound work". Still, there was an intrinsic curiosity that isn't as prevalent.
Some of the best systems programmers are under 20 years old what do you mean
There are more important battles for humanity to fight than someone trying to figure out how to configure Linux correctly for their hardware. I won't miss the need to waste time fighting the computer going away.
We always were the only people who ever knew how it worked. In 1990 people fellow students called me to fix their computer, they had absolutely no idea how any of this worked. No. Idea. Yes, the machine was being difficult; but their reaction wasn't to fight it, or understand it. It was to call someone to do it in their stead.
I'm not sure things are very different now.
Maybe the difference is more of the professionals in the field now haven't built that same muscle, as there's a broader group of people working in tech. Whereas the folks that could fix things in the 90s mainly gravitated to computers as a profession. Just random musing though I truly don't now.
I feel that things are pretty different. Although the example of interrupts and jumper settings being common knowledge is a bit of a stretch... it's still amazing to recall that MS-DOS was regarded as an end-user / consumer OS, and that, more generally, it really was regarded as totally normal to need to invest some time in learning about the system, files, directories, typing, configuring/customizing settings and network options just to be able to do what you wanted to do.
I find the current expectations around consumer "apps" to be totally infantile in comparison, where everything is now a single-purpose "app" that does exactly one thing when you push a button, and if you want something even a tiny bit different.. you can't, and that even basic things like files and settings are no longer accessible.
Compare with what the drivers/operators of the earliest automobiles were expected to know vs. what the same category of people are expected to know today.
There's nothing new about this particular progression - we've been through it in dozens of technologies already.
I think the difference is that with those technologies, not knowing is seen as a hindrance (I imagine most people wished they knew how their engine worked, that's useful) that one can simply live with, whereas with computers nowadays I see people just not care about these skills. When I do something interesting on a computer that other people see, the response isn't "huh, that's a neat skill" it's "why?". People don't care anymore. They don't see knowledge as useful or something that is beneficial to pursue
People nerd sniping the op add nothing of value to the discussion. Take a break from HN if you’re just going to shitpost.
Folks who keep mentioning that this is no different than any previous upward leap of abstraction in human history are missing a key point that the frontier labs are certainly not missing: this is the first time in computing that you are becoming completely dependent on a _subscription service_. I don't need to know how my CPU works because it continues to work once built. Once I outsource all cognition to a billable service, I am forever and continuously in thrall to someone else's revenue strategy.
But this too shall pass, it used to be you were dependent on a subscription to just have access to a computer itself (mainframe) and now we all have one in our pockets. I’m sure some day we will be able to run something as good as Claude at home.
I may not be a "typical" engineer as much anymore, in that I don't seem to be excited by knowing the intricate details of how things work as much as I enjoy knowing the capabilities of the systems that enable the projects I work on, and working at a higher level that might be more abstract and conceptual. AI enables that very well in that it enables jumping between paradigms and enabling creativity by removing a lot of the grind that was necessary in gathering technical details. I find it less appealing now to get into the weeds of how a thing works, even though at 10 I very much did. Like the author, I dug into the details of exactly which bits needed to be activated to make a particular program work. But as I aged, I was drawn to higher levels of abstraction, I think because I realized I could do more overall by piecing together bigger blocks. Yet I know colleagues who like to be involved in a very granular level of detail. I have many friends who almost revel in their knowledge that they are the ones who know precisely why something works the way it does. I think about the story of the NASA engineer on the Apollo 11 program who understood the assembly so well that he could confidently answer on the spot the moment the radar overload "1201 alarm" error flashed that the mission was still good to go. Some folks get so deep in this perspective that it almost seems like an affront to suggest they may be wrong about one of those details. And well they should, because precise people are very much needed to understand intricate systems. But it makes me wonder how they feel with AI being an indeterminate system - one where you can never really know what the answer will be every time. Likely still correct, and getting better every day, but also possibly different every time. Added to that, it's exponential spread across the software engineering domain. For me, even as someone who likes high-level systems, I still relate to the feeling that the detailed part of me that still likes some of the internals may be less focused on in the future. Even for me that's kind of sad. But I am also curious about my colleagues who are super detail oriented - how do you feel about AI and how it changes the focus on that detail aspect?
Moral panic. We all have private tutors on every subject at our disposal now. It takes some special kind of mental gymnastics to conclude from this that no one will learn anything anymore henceforth. The article is just a long-winded way of saying "Kids today have it too easy", or, equivalently, "I'm getting old".
https://en.wikipedia.org/wiki/When_the_Bough_Breaks_(Star_Tr...
Amazing how we always have a Trek episode for everything.
When I was young I was so naïve. The whole world seemed to be barreling straight towards the things I liked and was interested in. Every year there were more and more nerds, more people with computers, more people playing video games. It felt so much like I was the future, that in a couple of decades everyone would be chit-chatting about what was coming, buying and comparing devices, etc. I could not have conceived that people younger than me would have less computer knowledge than me. If at twenty you had told me that I would wake up one day and teenagers no longer knew what a file extension is I would've asked you who won World War 3.
"To play a computer game in in the 1990s, you first had to understand how the computer worked.
So you learned. You opened files like autoexec.bat and you read them."
Ehh I dunno about that. I rarely, if ever, had to mess with any of that junk after Windows 3... I also didn't have to deal with any IRQ issues. So seems like it was already mostly abstracted in the "1990s" lol
If you wanted to play a PC game in the Windows 3 era, you were playing a DOS game. Playing it inside of windows just made it even more difficult.
Yeah same. I remember playing a bunch of Sierra games on my first PC as a kid on either. Most I had to deal with was installing the drivers for my SoundBlaster card from the included disk. Most I dealt with was putting in the CD, double clicking the installer and entering the product key.
That said I did run into my fair share of other problems, and that early era of personal computing and my access to machines is the only reason I work in computing/tech today. If my childhood wasn't full of tinkering with these fascinating machines, and I only ever had an iPhone or iPad, I likely would have turned out much different.
Anecdotally, I think it depends on age and family more than anything. I grew up with hand-me-down computers, games because we couldn't afford anything new. Started with win98 and games from the mid 90s, but running them in 2002.
I do remember having to look lots of things up and figuring out why some things wouldnt work. Then getting into building our own computers (because it was cheaper) and figuring out how to get halflife mods working...
Windows 3.x did not cohabitate well with certain games that required nearly the entire 640k of conventional memory to run. These are mainly games from the old DOS days; games that relied on DPMI were largely exempt. But to play certain Sierra titles, for instance, you had to set up one CONFIG.SYS configuration that loaded HIMEM.SYS and EMM386.EXE for Windows and another that left them out to free up that memory for your games. In later versions of DOS you could set up a menu to select the appropriate configuration at boot.
And of course IRQ diddling was still necessary to configure sound, network, and game controller hardware throughout the DOS era of gaming, which lasted well into the Windows 9x era.
Just wait until they shred all the old useless books
I've been thinking that there might not be new programming languages. The amount of code in the current popular ones will explode, so that's what all LLMs will be trained on.
Good luck coming up with a new language and getting enough content out there that LLMs will be fluent with it.
If true, I think that's a shame. There's plenty of innovation still to be done.
How pessimistic you are on this subject depends on how much you believe in domain transfer for these tools. No programming language sprung out of a vacuum, and it seems reasonable to think that a LLM sufficiently trained on all the C++, Haskell, and OCaml in the world could probably do a decent job at writing Rust, even if it had never seen it before, given the rust compiler as a tool and rust documentation as input
It certainly does worry me, though. As does the increasing amount of materials and manuals that seem to be written assuming the LLMs will be the (only? primary?) audience
This has always been true. I never fixed my car. I knew how it worked well enough to know, hey, that sounds like its coming from the exhaust pipe. Then I took it to the mechanic. I can do basic maintenance on my bike, but I still take it to the bike shop. I have a small collection of vintage cameras, which means tracking down the few people left who know how some particular model works, might have parts. If your Synchro-Compur shutter needs parts, forget it. For most people, most of the time, the assumption has always been that someone else knows how to do that.
It doesn’t have to be this way, but the cost is performance (and falling behind competitors).
> The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads. They will recite, flawlessly and forever, exactly how all machines work.
That's wrong, and that's exactly why the loss of knowledge is such a problem. LLMs do not, and cannot, actually know a single thing. They are a statistical model, not knowledge. When they give out wrong information (and they always will, by their very nature), you need someone with actual knowledge to be able to recognize the BS and correct it. But we are losing the knowledge, and unless things change we will be no better off than the people in dystopian sci-fi stories who pray to the machine god because nobody knows how it actually works.
I can already see future where there is group of people goading AI to right direction by repeatedly changing what they ask slightly. And then memorise the times when you got right incantation for seemingly right outcome. Without actually understanding or being able to reason about process in the middle. Possibly a seemingly inconsequential misspoken word or typo can lead to better or worse outcome. Or maybe just not saying "Please" will sometimes produce wrong output. and other times you must not use it...
Sounds like absolutely horrifying dystopia.
Communing with the machine becomes spiritual. Computer operators will become a priesthood.
Welcome to the future. Make no mistakes.
Yes. And since LLMs can not improve knowledge - I mean, they can generate new arrangements of information, but they have no idea whether any of it real or making sense, unless humans - explicitly or through training - tell them, the more we rely on LLM knowledge the less the quality of it would be. Right now the LLMs are mainly in auxiliary role, so most of the knowledge erosion they generate is laughed at and relatively quickly corrected. But would this hold once the role of generative AIs increases? We already essentially entered the chaos period with news content - there's so much noise that it's basically impossible to know if any news message you read is true or manipulated somehow. This is going to start happening to more fundamental knowledge too, either on purpose or just by the force of the probabilistic nature of generative AI.
I was just thinking. What is the future when LLM is used for both code running itself and then also designing the hardware it runs on? Will it be sustainable? Or will there be some sort of error cascade that destroys the whole thing?
Maybe AGI is impossible with current model as it simply can not reliably improve itself... Enough errors in any part of loop will stop the progression.
I think what you describe is called the Model Collapse.
> Yes. And since LLMs can not improve knowledge
1. GPT proved Erdős Unit Distance Conjecture entirely on its own
2. GPT-5.4 Pro Solves Erdős Problem #1196 (April 2026)
In fact there's a whole benchmark that's measuring this: https://epoch.ai/frontiermath
I'm sure this comment will get buried, but I wholeheartedly agree with your take.
Imagine if we had LLMs back then to write xorg.conf for you and it hallucinated the modeline and you blew up your monitor.
This comment might have gone hard in 2023. Now it seems out of place. LLM's do hold knowledge empirically. They sometimes give out wrong information, like humans. And you need someone with better knowledge to correct it.
> But we are losing the knowledge
No we aren't and this is spreading FUD. Things have always been like this. Its called specialisation and this is how society progresses. I don't know how the supply chain worked to get the food to my table. That's why its so cheap!
Paging Vernor Vinge to the white courtesy phone.
Who cares?
There have always been layers of abstraction. I've been around for a while, and when I was a kid, the two choices I remember seeing were assembly code and simple semantic languages like BASIC.
Assembly seemed like too cryptic for me to really even follow and I never really did learn it, but at the time I remember people would say that assembly was easy and basically plain English compared to machine code.
As recently as fifteen or twenty years ago, I would occasionally check in and think of how unbelievably far away we had gotten from how the computer actually works. Like, you can just write "open window" and a window opens. Amazing.
Of course, those people writing machine code didn't need to really understand what P and N were in a transistor, let alone how an integrated circuit pulls it all together. And I'm not sure how much those guys knew about silicon dioxide.
The more complex things get and the more layers of abstraction there are, the more impossible it gets to really master things all the way down to first principles.
So what? People can carve out whatever chunk of the stack they want to really understand if they want to focus their lives on it. And for everyone else who's just trying to accomplish some other goal with computers as the tool, they will naturally use the highest level of abstraction and the simplest one for them to use, which is exactly what they should do.
https://en.wikipedia.org/wiki/Profession_(novella)
Competence Does Not Mean What You Think it Means.
Please don't fucking tell people that AI is competent. It is not and it cannot ever be because it is not alive.
AI can't be a competent person, but inanmate things can be competent. https://www.merriam-webster.com/dictionary/competence
I disagree wholeheartedly. I do not give a fraction of a flying fuck what MW thinks competence is. I'm a goddamn engineer and I know what it means
If you use your own definition for words, don't be upset when there's confusion about what they mean when you try to communicate.
This and it was demonstrated in Star Trek The Next Generation: Season 1 Episode 17 "When The Bough Breaks" [1]. They did not know how to fix their computer system or even where it resided. It did everything for them but was also killing them and making them sterile. There was nobody left to figure this out until the Enterprise crew happened along. This could happen on Earth. Who will save us?
[1] - https://www.youtube.com/watch?v=5WWyKQThSKg