I think beyond the specific elements of any domain AI eclipses humans in, overall, the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight. You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor? All it tells us is we will be bargaining with superintelligence with nothing to stand on, only a trust that the 5000 lab executives and researchers solved the most important research problem of all time on the first try.
And remember, these AI models were trained on your work, your culmination of effort and sacrifice, without any consent, credit, or compensation, then sold back to you for a subscription fee.
That’s what standing on the shoulders of giants refers to. This is how science has always worked. The generated papers have attributions and citations just like if a human wrote them.
>That’s what standing on the shoulders of giants refers to. This is how science has always worked.
Yes, but automating and commercializing a process makes it inherently different. Just because everyone has always been welcome to drink from the village well, that countless generations have worked tirelessly to dig, maintain, and improve, doesn't mean that there's no basis for complaint when Nestle sticks a pump down it and starts draining the aquifer to bottle and sell.
The lack of an existing law (or custom) against new behavior doesn't actually render it necessarily non-grotesque.
No, it's able to reach deeper and pump more out than ever before, democratizing access to the water from the well, at the minor cost of the people of the village no longer having a well. But they're welcome to buy plastic bottles of their own water along with everyone else.
Is this actually how water pumping works, or are we extending the metaphor to imply that the AI is creating knowledge? Because I would dispute the latter if the effect is that the number of people with a deep understanding of a subject declines.
So.. they’re being trained on all of the information that some used to make their living with by writing books, presenting seminars, etc. on a topic.. without paying for it, and then giving it all away for free?
For now. Once people are hooked, and/or once some of that investment capital starts to be expected to generate some dividends (see https://isaiprofitable.com/), free use will be very limited.
> You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
I recently thought about this and fear that AI may further drive us humans apart.
Instead of working with humans, I increasingly work with a fleet of agents.
I would recommend the work of Sherry Turkle, specifically her new book "Artifical Intimacy" - she is quite opinionated on this, in the sense that the role that AI play as a conversation partner is so frictionless, so controlled, that we get none of the rough edges of human interaction that form us into a functional social being. No real quick solution outside of deep, fearful caution when using these things - and we're not going to see that from John Q Public
> the role that AI play as a conversation partner is so frictionless, so controlled, that we get none of the rough edges of human interaction that form us into a functional social being.
I wonder if there's a tangent with social media here, at least some of it, where in my experience "a perfect life lived by influencers" is portrayed to us.
IMO, I think that frictionless experience comes at us from a bunch of angles - finding yourself in an echo chamber (by choice or not) online means that you may find yourself less comfortable talking with people you don't agree with, things like that.
I grew up in a small town pre-social media, and that meant I didn't have much of a choice about who I was surrounded by. If I was growing up today, what would stop me from retreating to a comfy social media-mediated environment without any challenge or friction? If I was growing up 5 years from now, what would stop me from retreating even further to the comfort of my LLM "friends" who can be shaped to me perfectly? That world sounds terrifying to me.
Machines could socialize back in a (more) meaningful way. But to do so, they would have to not lose context (or at least keep some).
I have not totally abandoned the hope that a machine could be someone. LLMs aren't that, though.
[Edit to reply to Loquebantur, because I'm rate limited ATM:] You seem to have read this post as saying that I want that because I find human relationships inadequate. I never said that, nor meant it. My point was a more technical one: Throwing away context at the end of each conversation means that you can't form a meaningful social connection with an LLM, because you're starting from zero each time. (It's kind of like "50 First Dates".)
I'm not pining for a social connection with a machine. But if a machine is ever truly capable of that, I hope I will interact with it as someone, not just as a machine.
> they would have to not lose context (or at least keep some).
I guess you can start to see that happen (for a while now) with e.g., ChatGPT, Claude, etc. keeping persistent memory of you, automatically.
Personally, I often find these memorization features annoying, because - more often than not - the persisted context is outdated because something in my real life changed that ChatGPT & co are not aware of.
In some very very distant future we might someday have your typical sci-fi Data/R2-D2/Johnny 5/EDI style AI and yeah, at that point it'd be weird to not treat them as people, give them rights, etc. It'll take a lot more than solving the context problem to pull that off, and LLMs will never be that, but I agree whatever that technology ends up being it'd pretty much have to be able to hold onto memories. I doubt we'll ever see it in our lifetime, but who knows.
I mean, why wouldn't you? Sci-fi AI robot people could be very cool, be capable of things we aren't, have a unique perspectives as a non-human with similar (if not superior) intelligence, have extremely long (potentially indefinite) lifespans, and provide us with insights into our own nature.
Not the parent, but sorry, for me, no. And this is a perspective that I don’t think I will ever understand.
This idea that we are on the path to creating a new synthetic intelligence that surpasses our own with the goal of, what, speaking to it? Cooperating with it? Evolving with it? To me, this sort of goal is an affront to humanity and human self-governance and I’ll probably never be able to see it as anything else.
Losing context is only part of the issue. LLMs don't have the rich connection to the world that humans have. The only connection LLMs have with anything is streams of text. Not only don't they have eyes, ears, smell, taste, touch--they don't have bodies. They don't have to deal with the real, physical world and have it push back on them.
That is so utterly different from what we humans are used to that we can't even comprehend it, and we anthropomorphize LLMs because we can't make any sense of them otherwise. We can't comprehend that these things that emit text that looks like human text don't have all of the other things that lie underneath our words when we humans talk.
> Instead of working with humans, I increasingly work with a fleet of agents.
This is why I see myself leaving dev within a year. Remote working is enough of a embuggererance for my mental health and I've be unhappy in work much of the time since 2020 largely due to that, AI will finish me off.
Career suicide and going of to wait tables, or be a hospital porter, or anything else with people even if I'm going to hate those people, with the massive pay cut that entails, will be better than actual suicide after a couple more years trying to stick at it.
There came a point a few months ago where I realised that I had to either accept becoming obscelete or change my while outlook on using AI and trying to catch up. I didn't choose the AI option.
Think carefully. I left high school teaching to become a dev. I would never go back. Yeah, social isolation sucks. But doing a job where your students/clients/patients think you're nothing but a waste of their time is soul-crushing on another level.
It is frightening that it took only a few comments from the top of the page to the first mention of a suicide and also that it doesn't look out of place at all.
Seriously I don't get what everyone else is doing where they say things are getting worse somehow.
There's days I no longer need to sit at my computer: Claude Remote prompts my phone when it needs direction, the hardware stays busy and I can just go out with my family instead.
It's more important now than ever to hold onto the fact that human beings have inherent value. This exists independent of their ability to generate wealth for an employer. I know HN loves a technical solution for every problem, and so do I, but this is a political problem requiring political solutions.
No, you don't work w/ a fleet of agents, you use them to do your work.
This is a general issue that is being forced on us by AI companies; AI is not a human being, it is a tool that we can use to do things. We don't have them as coworkers or subordinates, they're objects specialized in language processing. Thinking otherwise leads people to all kinds of existential and psychological issues that in the end make no sense.
No. If these tools could do your work, people would feel existential dread anyway. It doesn’t matter at all if they are coworkers or tools (I really don’t see why would it ever matter).
Personally I don’t care about that all that much. I don’t have any “sacred” inherent value for human beings. If they can make robots covered in real human skin, blood vessels and muscles (lab grown over the robot frame) with an LLM brain at some point in the future, I would definitely prefer that to a human lover.
> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
Not for those without status (which is most people on this planet). But yes, AI is set to wipe out most established hierarchies and their gatekeepers, and to equalize most fields. A profound sense of dread is something most ordinary people live with throughout life, and if AI can reduce systemic unfairness, I bet most people will actually find some degree of relief from that dread. The complacency of the formerly-statused being shocked along the way isn't much of an argument against that equalization.
> Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight.
This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it. So why fight for us all to have to keep working just to live? Isn't a post-scarcity world the thing we should all be fighting for?
>This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it.
This is what you don't get. _I_ already know it's not worth it. I've always known. It's everyone else! It's a wife who cares whether or not her husband has a good career. It's the police who care if you look like you're poor. It's the way every single person in society obsessively cares about the status of others.
What in the world makes you think I've had to build up status for my own sake? It's absolutely baffling that anyone could be confused about this.
>> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
> Not for those without status (which is most people on this planet)
Look at how coal miners react to the idea of getting rid of coal as an energy source; whether or not you think they have prestige or status, there's certainly an amount of dread there that I'd expect to somewhat resemble the type we're talking about here.
A fallacious argument. There's no goal of inflicting suffering on other people; you can equally look at it as enabling people who were previously shut out of something to participate in it, or if not to participate in it directly to reap the benefits by having an agent exploit automation on their behalf.
Not only is that not a real solution to unfairness (it's certainly not fair for the person who now has to suffer more than they were), it also ignores the far better option which would be to reduce your suffering so that it isn't disproportionate to everyone else's. You might be holding onto to some self-destructive fearful crab-bucket mentality if that's where your mind instantly takes you.
Unfortunately, I think this is a human failing being exposed rather than anything else. We like to build conceptual models of how things work, but they are often tinged with should. Reality doesn't care how things should work. It only cares about how things do work. It's always jarring when the should turns out to be false.
Philosophers for millenia believed that human beings had meaning, purpose and set out to explore what that could be. It was a noble profession for a long time. Once we realized that there was no hidden spark, ergo no purpose, the study of and the prestige associated with philosophy took a nosedive. Yes it is still studied but it carries nowhere near the respect it once did. Mathematics seems to be suffering the same fate.
It's more or less reverse to the mean. The period in which a people had both access to education and leverage through their labor and intellect is super short. For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.
>For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.
I feel like we should be doing everything we can to not revert back to that way of life.
I'm too young to be financially secure given my impoverished background, not having parents in my life, being homeless as a teenager, etc. But too old and physically disabled to change course from tech. Nothing to do but keep walking through this tunnel and see where it ends up. It's scary!
same here, and it often keeps me up at night. for a few years i really felt like i had finally managed to get on track towards escaping the poverty i was born into, now i'm not so sure anymore in the long term. the good thing is at least i know i somehow managed to claw my way upwards from rock bottom to a tech job once already, so whatever the future might throw at me i'll probably be able to somehow deal with that too.
Disagree. Scribes, doctors, bankers, and government clerks always existed, even if the education to do it was expensive and exclusive. For the first time in history, this labor, including its knowledge content might become automated and commoditized.
I think the fix is to get started on post scarcity economics now. Sooner or later we're going to have to come up with practices that boost or limit endeavors based on how people feel about them, rather than based on how much money somebody has.
It doesn't have to be ((post scarcity) economics), it can be (post (scarcity economics)). Both systems can coexist for a while, one providing a bit of a safety net while the other falls apart.
There will still be scarcity, just of different things.
Even if we vastly increase productivity beyond our current wildest dreams, the production system takes time to adjust to changes in demand, creating continuous points of scarcity.
But Georgist economics are way ahead of how to deal with this. Tax the ownership/control of things that can not be increased (e.g. land).
Sure, but that's no reason to continue using control over scarce things as a proxy for consent. It's not exactly working out.
We had to tolerate gold because it implied a game that was facilitated by the laws of physics, and we didn't have the resources to set up any other sort. But now we do, the rules can be whatever we want them to be.
Sure, let's try Georgism, but given the historical track record of that kind of reform, let's work on other things in parallel.
agree. wealth inequality has only widened despite AI. things like healthcare insurance, tuition, rent and homes less affordable or more expensive than ever. Nothing has changed and probably ever will. Ai creates abundance in some things like knowledge but other tangible scarcities and individual differences of status remain.
Counter point: AI is trained on output from humans. We all made AI, together. It belongs to all of us. We use it to benefit us, and we celebrate our wins together.
For the low entry fee of ~$200 a month you too can escape the permanent underclass - which is more than some countries average citizen makes in an entire year.
Counterpoint: Their moat is funding secured and contracts locking in the purchase of currently forecasted chip production. Even if you assume a catastrophic withdrawal of invested capital from those vehicles on a coordinated revocation of faith by the investing community; who do you plan on having operate this theoretical squeaky clean, ethically training data sourcing company? And for that matter; what makes you think you'll get passed the greed that is already sunk cost into the current gen of shysters?
This entire evolution is a generational project hoping to paradigm shift away from their predecessor's paradigm while ignoring the lessons of their predecessor's predecessors on the nature of greed, the corrupting influence of power, and the consequences of blatant cultural/ethical unhitching amplified by the reach amplification of technology?
The Stasi, or a perfect Jim Crowe couldn't be realized with manpower alone. With computers, and a small cadre of the unethical willing to succumb to the promise of lucre on the other hand...
Great, I'll just eat good intentions, wear a smile, and live under happy thoughts. I'm 50, so I'm "cooked" as far as contemporary society is concerned.
When did we last try to avoid such a thing? The way I remember history we celebrate this whenever it happens. Like how the electronic computer made the job of human computers obsolete.
It's mainly about status. No one cares that much about AI automating drug discovery or coding, but it's bad when it assists in math. Math results are attached to individuals , whereas for cancer drugs or other medical discoveries, it's more of a team effort by a lab. AI now has the effect of turning math into something that is more lab-like and less about the individual.
Yes, and I think over time there will be a blatant inequality in who is commoditized and who isn’t.
If AI engineers didn’t create the data to produce the models, and they increasingly don’t write the code to produce the models either, then why are they treated as the agents of change rather than the subjects of change, like mathematicians are?
When I was a kid I built my entire identity around being #1. I was the most talented kid in school, anything endangering that resulted in an instant identity crisis.
Then I went to college and I understood that there are people who, for one reason or another, are thinking in ways so advanced I'll never catch up. I'll never be the #1. I'm actually much closer to being average than I thought.
But this doesn't mean I should just give up, lie down in my bed and cry. Since then I've learned a lot of skills - most of them being crap at best, but that's not the point. What matters is my own subjective experience. The fact that despite objectively not being good, I see myself as a skilled person with rich life.
Once I was watching a YouTube video about elderly care and there was a phrase along "These people won't achieve anything more in their lives, my goal is to keep them occupied until they die" which is a bit morbid at first glance, but it actually makes so much sense as a life philosophy.
I would answer to this that the mathematicians (humans) are the ones who give any meaning to mathematical discoveries. If we don't have mathematicians anymore, the result is that AI can find whatever they want, but 99% of that will be useless waste. Outside of practical applications, the only reason most mathematical results have any interest is that lots of mathematicians spend time trying to find their solution. Once humans stop caring, mathematics done in an automated fashion will also stop having any value, other than its possible use in finding solutions to existing engineering problems.
This part is nonsense, still. There's no known AI -> AGI path, and superintelligence doesn't mean anything, and AGI is assumed to be (going to be) both malicious and a slave, both for no good reason.
It has already been happening for decades if not centuries. Instead of bargaining with superintelligence though, it's bargaining with moronic systems. I watched an airline gate employee struggle to allow a family onto a flight despite acknowledging that they were confirmed and had seats on the plane because the computer wouldn't beep when they waived their boarding passes. She called her supervisor. Her supervisor resolved it not by writing "it's okay they can get on" into an exceptions field, but by kicking them off the flight, putting them on standby, then accepting their standby request and reseating them (in different seats, not all together, for some reason). The humans involved all knew things were okay, but the system was too dumb to just allow them to go forward.
Nah. There is nothing beyond being made redundant in your job and labor leverage. That is all. It has got nothing to do with resentment over fucking machines. It’s all about people (the executives that you mention) fucking us over. The end. Completely human problem.
I know I’m being sold some bullshit when this is made into something inherently existential and psycho-identitarian. An egalitarian future of labor-worthlessness means that there is nothing above us to dictate anything at all—maybe not even nature itself, save for meteors and other cosmic events. I’m not afraid of being “worthless” in that world of complete freedom. Fuck no.
Some have been scurrying to amass a nest egg and become self-sufficient. Of course, if demand destruction from the hollowing out of labor happens fast enough, who will consume and keep these stock prices up? With the tax base simultaneously crashing, gov't can't step in and prop things up. It could help during the transition while failing as a long-term strategy but what else can you do?
Consider this… in the grand scheme of things, we are on a tiny planet in the universe. What cosmic contribution could you make?
Most people won’t make anything even close to “a ding in the universe”, as Steve Jobs has been reported to say.
So they may as well enjoy their life. And your worth can be to your family, to your children, your parents who took care of you.
Learning a hobby. Practicing religion. Mindfulness and meditation. Doing sports and improving your own body.
Look around if you are a heterosexual man, for instance: many women are far more attracted to men that perform in the moment, than men who labor quietly to create a product. The former enables untold hours of honing social skills and calibration, and humor and performance skills. The latter is of more durable value to society, yet relatively few women are genuinely sexually attracted to a man doing the deep work. (By the way, it is also true the other way: most men are not attracted to women for doing deep work or being a rocket scientist.)
Animals don’t do “deep work” and most animals are attracted for far “shallower” reasons. If everyone has a UBI and AI does the “deep work” then that frees up humans to be like animals in a zoo or in the wild… doing all the displays to attract mates, having babies etc. That’s not nothing — in fact it is 99.999% of everything that happened in history and 100% of what happened in prehistory.
You’re just ceding that 0.0001% which represents technological progress. The actual dread comes from the fact that you and all of humanity will be as completely powerless to change things as cats or any random animal species is powerless now. The difference is you will perhaps understand your powerlessness more than they do. But in the good scenario, you’ll be like animals in a zoo. We are constructing this zoo for ourselves now.
We will remain on this blue planet while the AIs will colonize the universe. In fact I wonder why aliens would ever come to Earth, when they’d just send their AIs or robots. Look at this: https://news.ycombinator.com/item?id=46200502
As Ernest Becker would ask, then what is your "immortality project"? Even worse, what's the immortality project for the vast majority of humanity? Do we just bring back religion to Middle Ages levels? Or do we hope everybody goes through some kind of never-before-in-history spiritual awakening?
For me the most obvious conclusion is, use your maths skills and get into open source AI.
Make AI as cheap and accessible to as many people as possible and we will likely be much better off.
The more people empowered by the tech the less like you will be “redundant” in the way you describe. You will own the important tools only the techno oligarchy own today…
Would you feel the same if you were a part owner of OpenAI? I bet you’d feel excited about the money you’re about to make ?
> How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor?
I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
And like to anyone else who hints that “automation creates better jobs”, I have to add: but not for us. Maybe for the next or the generation after that.
Industrial and technological progress has birthed better conditions overall, but at a cost of locally worse ones. Mine workers that got displaced didn’t end up being machine operators - most of them landed on benefits or sporadic gig work. Only the next generation was able to be properly trained and educated to be able to benefit from the technological advancement en masse. And we might see that happening again.
In the I was referring to people reading my comment as “us”.
While it is nice to think of better futures for coming generations, I think we (reading this) also have to think about our more immediate work, as in next 2-5-10 years.
> I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
I don't buy this argument, much like I don't buy the similar "they said the same when calculators were introduced".
It negates that some changes are qualitatively and scale-wise different. Also, by gradually chipping away at things that humans do, you (we) keep shrinking and shrinking until there's nothing left, every step of the way some people claiming "well, the telegraph operators said the same". Keep in mind that, while it's hyperbole, some AI entrepreneurs are actually saying stuff like "maybe jobs will become a hobby". Unrealistic for now, maybe, but if they do become a hobby, what leverage will we have at all?
In other words, it's true the past is worth keeping in mind, but not to the exclusion of the possibility that this time it's different. Or that the trends of the past actually point to harm in the near future.
> the removal of all human leverage via their labor
I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.
I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.
My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.
I can kind of relate but I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.
Nothing is validated or tested anymore. It’s all about the shape or believability or even how loud it is.
They dropped 100+ papers no one read. I am sure mathematicians are reading them now but even this forum has concluded that these problems are solved, without reading or validating.
It’s like everyone forgot “don’t believe everything you read on the Internet.” And it makes for a very boring discussion when everyone is just spewing unvalidated claims, layering them with strong beliefs founded on similarly unvalidated claims. It’s dull, without substance and just pure noise.
> I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.
This is what I've been feeling too. It feels increasingly like the professional world has become a tower of scams. It's not worth building technical skills or focusing on quality; your boss doesn't know or care what good work looks like, he/she just wants your work to look good enough to his/her boss, who knows even less about the actual work being performed. And everyone reports up to an MBA CEO who was brought in from a completely different industry.
I want to work in an industry where skill, diligence and expertise is rewarded and respected, but from what I'm seeing political ability is 95% of the game and caring too much about the work is a career liability.
Not sure I agree - they dropped hundreds of papers, most of which were so badly written that they were incomprehensible. If it wasn’t for the lean certificates on 30% of them, the mathematical community would have told them to go and whistle.
I think the consensus here is that most of the papers are effectively correct. But it would take years of study just to understand most of the problem statement - so we have to wait for the mathematical community to digest the borderline-nonsense that OpenAI spewed out.
Just tell the (AI) authors to go back and write clear explanations for why they matter and what the specific claim is.
I gotta say I think mathematicians brought this roblem on themselves, most math papers are absolutely incomprehensible to anyone who is not an expert in the field. Mathematicians spend so much effort trying to make their argument to other mathematicians that they've gradually abandoned making their field or findings comprehensible to anyone else and left that to science journalists. In turn, those journalists churn out very formulaic stories written to a template: 50% the human story of the mathematicians social/academic wanderings in pursuit of their goal, 50% an explainer with some nice graphics on the subfield they operate in and a mention of what sort of problem their insight solves, perhaps with a few colorful metaphors thrown in.
The last mathematical paper I can think of (but I'm not an expert, I'm sure there are some others) that was both mathematically rigorous and accessible enough to be readable was Mitchell Feigenbaum's Quantitative Universality for a Class of
Nonlinear Transformations. This is accessible int he same way as a textbook - you have to work at it, but hte author is explaining his thought process and drawing the reader's attention toward things that matter or stand out as unusual.
This piece circles, without naming it, around the concept of “deontic scorekeeping” central to the work of Robert Brandom on semantics. The idea that our ability to create and sustain conceptual systems depends on the give and take of reasons within a social setting. Brandom has been front of mind for me lately because elements of his theory both explain why LLMs should work in principle, and where they are bound to be radically deficient. An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
Worth reading: Robert Brandom, “Making It Explicit: Reasoning, Representing, and Discursive Commitment”
I mean, many companies operate within particular domains but don't have much skin in the domain itself in a way that is meaningful to other participants there -- like Oracle builds software for healthcare, ostensibly, but any actual user of their products will tell you they are nightmarish with regard to usability, and research shows that patients are often harmed as a result. So healthcare isn't their game, it's selling healthcare software.
Likewise OpenAI doesn't have "skin" of any shape or value in e.g. the mathematics community or even in mathematics research per se. In my cynical view, they are exploiting the exposure of mathematical research to automation in an effort to create a case study of their product obsoleting a highly visible and well-leveraged and relatively small profession. Regardless, I doubt even they would say they have durable skin in mathematics in any sense, certainly not in either of the senses I mentioned above. But their output inhabits the same space as mathematicians, who have to contend with the consequences
I think it means if we all have to justify ourselves to each other, we collectively come up with an sustainable ecosystem of ideas that we can understand. So if we stop justifying ourselves to each other, the ecosystem doesn't get built or sustained, and we stop understanding things. Not the person you responded to, but that's my reading.
What a coincidence, I've also picked up inferentialism but never really got into it.
I'm interested in the reversal that Sellars proposes - it seems clear to me that "representation" is not prior to the space of reasons, despite how unintuitive it is.
> An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
I see no reason why this can't be the case in the future, even though I think inferentialist-based theories are generally strong. Can't you force LLM's (or whatever AI architecture will exist in a decade) to stake durable commitments itself, then? They're at least autonomous enough to run computers. I agree that that is a core part to true meaning & autonomy, but nobody just lets them yet. I don't think inferentialism is a consciousness-based argument, either, so that escape hatch isn't available. I also don't think this is a necessary condition to engage in the space of reasons (but maybe then I'm misunderstanding it, or retreating to a more Sellarsian understanding). Could you explain on this more?
Though again, I only have the book sitting on my desk, and I haven't cracked it open...
I actually agree with you. LLMs could become participants in the future, but this would require (1) durability of real awareness over time (an architecture change from the current fixed-weights + transient context architecture) and (2) that humans grant them the status of equal participants (probably not a terribly high bar given how people already treat these things). I'm not aware of any real progress toward (1), and all the stand-ins people have developed live in the context layer or textual artifacts, which are poor stand-ins for durable memory.
And yeah, consciousness based arguments are bunk IMO. It's clear that LLMs are in a basic sense "aware", and even to some extent "sapient" (in Brandom's usage), i.e. they can justify claims and object to things at least locally. I follow Rorty on the question of consciousness, which is to say that consciousness has no deeper meaning in practice beyond "are you awake", and in philosophical discussion it's largely a proxy for "are you one of us" in some amorphous sense.
Well that's too bad, since Rorty completely misses the point on consciousness.
Consciousness, the interesting part anyway, is qualia, or the "it is like something to be me" as Nagel put it.
Intention, as Dennett used it in his book "The Intentional Stance" is the "are you one of us" angle, and is distinct from consciousness because as the book title suggests, it is a stance taken with respect to some other system (person, cat, dog, turtle, LLM) rather than a description of subjective experience.
So I think it is your (and Rorty's) positions that are bunk. It is very, very far from clear that LLM's are in any basic sense "aware" let alone "sapient". They are extraordinarily able to manipulate language in a way that leads to text generation that has traditionally been assumed to indicate consciousness and/or intelligence. But we have no way of knowing if it is like anything to be an LLM, even though most of us are busy taking the Intentional Stance towards them.
I guess I'd say that your qualia aren't a something, but they're not a nothing either. We can't say what they are. They're not publicly describable, and therefore not publicly attributable. As Wittgenstein would say, "They cancel out."
And as I think Dennett himself would say, qualia are just Sellars's Given rearing its ugly head again.
Well, while I love much of Dennett's work, I emailed him after "Consciousness Explained" was published to say that I was disappointed that it appeared to describe what we are conscious of, not how we are conscious. He sent back a short note to say "You're probably right".
Brandom wrote a shorter version of the book, called "Articulating Reasons", which is available in full on his faculty page [1], or from booksellers. He's a very wordy writer (kind of exhausting to read, even for someone who enjoys philosophy), but if you get ahold of Making It Explicit, chapters 2 and 3 have the meat of the ideas I'm talking about here.
Edit: One of my ways of benchmarking new AI models is actually to type a version of this prompt in and see how sophisticated the response is: "Talk to me about the concept of deontic scorekeeping in the context of llms". You'll get good mileage out of that.
Humans change over time due to what they experience. It’s different from waking up from a ten-year coma and having access to a record of what happened to yourself in an alternative timeline where you didn’t fall into the coma.
I think in future, each company will built its own universe of mathematics consisting of prized discoveries. Maybe one Mathematical Universe per Company in the multiverse of mathematics.
Mathematicians always define their terms exactly, in order to begin their discussion. Philosophers never define their terms exactly, because then they would have nothing to discuss.
Because language is mimetic, and people are consuming huge quantities of AI-generated text. I.e. saying that an LLM can't be an equal participant in our language games isn't saying that it can't play by the rules. In fact that's what LLMs are trained to be best at.
What an LLM can't do (at least with the current architecture) is sustain a commitment that it can be held accountable for later. It doesn't have durable "perspective". It can't update its priors. Maybe a new architecture will come along that gives LLMs memory plasticity, at which point they will shift into being individuated quasi-persons, but that's not what we've got now.
Recursive self improvement is a bulk process that shapes the weights of the next model. The weights are still frozen. Each individual "agent" would need to able to durably update its own dispositions to get to what I'm talking about.
I'm not suggesting mathematics and language are the same thing, but I am saying (which shouldn't be controversial) that mathematics as a discipline exists within the medium of language, and that mathematical reasoning, discovery, and proof happen via people proposing, justifying, testing, and objecting to formulations in language. This isn't a metaphysical claim about whether math is real or captures inherent structures of the universe etc.
Ok, then what is an agent that provides a 'thinking' or reasoning loop around said model?
You ended your first post with: "An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time."
And so if there is an agent, a thinking loop, that can recursively adjust over time...how does your statement hold up? I'm particularly talking about 'staking durable commitments or changing its dispositions over time'. Recursive self improvement is all about these 2 things, especially when it's the 'outer thinking loop' doing the improvement.
But you're right, Mathematics exists as a medium within Language. But then Language is just a transfer of ideas from one mind to another.
This principle is the heart of what the article is talking about: a transfer of logical mathematical ideas passed from a human to another. And the sense of 'shared understanding' that comes with it. It has nothing to do with keeping score.
It's about getting a 'sense' for why someone thought or rationalized an idea in order to connect some other idea. You get an 'ah-ha' moment, and some of those moments are gone, or a different path is taken to reach the moments now. That is what the 'weeping' is about (not that I agree with the article. I think anyone can spend however they want. Math doesn't have to change, it's that people want it to, apparently, because it's 'easier' to do something different, or for some other reason I don't know about).
Rather the opposite, AIs are arguably deontically asymmetric because they are powerful and virtual digital entities, and thus should be very concerning to the human species. Not because they current have no internal abstract state for learning state like humans and animals.
It would only be true if you consider Mathematics and Language to be the same thing.
Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.
The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.
That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.
And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.
Ideas are contagious and AI is our most powerful vector yet.
I suppose we would see some interaction with AI in that sense, but it'll take place over the course of learn cycles and when AI starts ingesting its own ideas it typically goes wrong somewhere down the line.
I think for me is, that history is not kind to the people who are left behind.
For example, weavers were one of the most powerful professions (if thats a thing) in the middle ages. In england there are the wool churches(not entirely weavers....), in the low countries, the weavers guild fomented revolutions, and shaped the borders of countries.
But that came to a crashing halt over the space of 200 years. slowly at first, then all at once. the power loom, then cheap imports from india turned a cottage industry into dark satanic mills. Yes there are/were specialists that still hang on, but they are like football stars you can probably name all the people who make money, the rest are barely scraping by.
For us, in the programming world, we will not have a 200 year grace period. Yes, AI produces shit code, but thats not the point. its cheaper and there are vanishingly small places were "good" code matters anymore. AI could be used to help a large range of people, provide support, trusted advice.
However because of the people who are funding it, that will not be the case. it will be used to reduce wages. Just like offshoring, resulting in china being the worlds manufacturing hub, AI, and the current people funding it will aim to make the expensive worker redundant.
In short, I weep, because Altman, and the money behind him ans his ilk want me destitute. I have committed the crime of wanting to earn a wage, and thus I must be punished.
It's profit chasing as well as practical - it kneecaps the educated workers increasingly asking questions about the morality of business and technology. It takes away the income that affords them the space to push back, threatens high ed, critical thinking. The salivating over AI makes perfect sense in that context.
I think Thompson has no understanding of endeavors like pure math or the theoretical sciences. In the words of Einstein, "I want to know how God created this world. I am not interested in this or that phenomenon, in the spectrum of this or that element. I want to know his thoughts; the rest [is] just details."
I think mathematicians aren't bemoaning their lost sunk costs. They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_. Their voyages of discovery, which give life meaning, may never leave dry dock again.
It is strange, because in a sense, when you start learning math, you never explore, you are being taught.
Now mathematicians were able to exercise curiosity. That is going to change. Back to the ranks of eternal learners if you will.
The fun was in the research. Now I think it will leave the door open toward more Applied math. What can people do with all the discoveries. The curiosity will be there.
I think it is better that way.
It will also put a cap on abstractions that overlaps. In essence, AI is less likely to create new theories, coin new terminology, etc. But it is better suited to recognize emerging patterns.
That will probably simplify mathematics.
Unless the domain of abstraction is infinite and with each new abstraction comes its own results but I doubt it. Who knows though...
> they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_
Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity. Even now, verifying and understanding these AI proofs is a massive endeavor. Coming up with an intuitive (to humans) distillation of the finding remains an open challenging demanding expertise.
i think to certain types of people (who are probably more likely to be mathematicians), the "self" discovery aspect of it is important, they don't want to read about a new proof for a problem they were grappling with, they want to come about it through their own thoughts.
LLMs are not feeding us things, we are asking it and getting answers. This dialectic I think is super powerful and it still requires the human to ask questions and continue expanding intellectual curiosity to go down the rabbit hole and discover. LLMs are just an incredible rubber duck but the discovery is still there.
We literally had a case where OpenAI may have stolen the dialectic from a mathematician using its product and then given it to a proprietary model burning tens of millions of dollars of tokens [1].
I think I disagree, if I had a puzzle on my desk and I told an LLM to explain how to solve it, I think I have robbed myself of the enjoyment of solving the puzzle. Is it really any different than googling a solution?
There are different avenues of "solving" something though. Asking an LLM to one shot something vs iterating on it collaboratively are different ways of achieve potentially different ends.
If the result is black and white, then yeah, its probably not super fruitful but if the end result is subjective, then there really no is objective "solve".
> Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity.
I don't think this matters a bit. We individually can't demonstrate "consciousness" in other humans, it's not even a word with a definition. What we can do is miss the feeling of discovering something on one's own rather than having it explained to you over and over again until you understand it.
I mean, dead mathematicians are definitely not "conscious," but their books keep you from discovering things that you may have discovered yourself if left to your own devices.
The joy that was mentioned is the joy of knowing a bit of gossip that no one else knows, of being the first explainer. Interpretation is just being able to pass gossip on; a decidedly lesser joy. And there's no reason that AI shouldn't be able to explain findings in an intuitive way - LLMs at least purely operate based on human intuition as written in human communications and then captured in matrices.
If these proofs are good (doubtful, imo) and can be reduced (intuitively distilled, sorry), there's no reason that an LLM shouldn't be able to do that.
That's such a nice way to put it! I recently stumbled into "The Last First- Winter K2"[1] and saw the same spirit along with a similar misguided effort to win without the will, nor capacity to do it — e.g. influencers taking a stab at K2 is like prompting your way to results.
Why is not like changing to a different type of vessel for the voyage? Unless and until it is all AI in closed system, could direct machines where to look etc.
I'll say it again: AI is capable of answering the questions put to it but AI is not yet capable of knowing which questions are worth asking. Mathematicians have no cause to weep, they will be fine.
Most of these questions Mathematicians posed to each other have no practical application, and probably will never have a practical application. (ex: the solution to the Navier Stokes question doesn't change anything at all.)
In many ways, the teleporting machine that teleports to the solution is against the point of the questions. No one wants the solution to be "Run $10,000,000 worth of tokens to cover all possible Lean formulations of the problem".
I think for Mathematicians, I'd say that the human level formulation is still possible and with a new perspective. We as a society (and outside of Mathematics) need to also understand that this AI teleport to the answer thing has less value than we all expected. The value is in the journey.
The open question is if people will still want to take the journey if the teleport machine exists. Or if the existence of the teleport machine halts exploration.
> The story of human discovery and the triumph of the human spirit will soon be excised from this discipline. The Dinitz-Garg-Goemans counterexample was the most egregious demonstration. It revealed that the process of prompting novel proofs will be as auraless as ordering doordash. Watch as magic and mystery evaporate. Watch as the sun sets on our heroic age. Is there not something evil in the act of blocking all future generations of mathematicians from the experience of discovery? Forget about accuracy or even attribution. Something fundamental to the experience of mathematics is being taken.
> You are a helpless onlooker. Before you a channel through which humans have accessed the ineffable and sacred for thousands of years is being sealed for eternity.
As someone with friends in the mathematics community, they've expressed similar sentiments.
I have to wonder if this is the equivalent of teleporting back in time and finding Marco Polo or any of those early explorers who traveled for the love of the game (and fame and money), and opening up Google Maps on your phone and showing them the zoomed out world. And showing them any island or village they could name.
This is absurd and silly. Discovery doesn't go away because more new things are being discovered. This position only makes sense if you make an arbitrary cut point where some instances "don't count". If you were to go back to scientists or mathematicians several hundred years ago and tell them that one day science would be done by tens of millions of full time researchers around the globe, often working in teams, some of them might also bemoan the loss of the "hero scientist" as any commoner could now participate. But science and math have only benefited from this, and this has increased, not decreased, the beauty and wonder in these fields.
It is not the same because the act of problem solving for discovery is now to be done exclusively by machines, as humans are no longer remotely competitive.
If you read the linked article, the author goes into more detail about how doing mathematics also has been an act of communication with past and present human mathematicians. This is very much true when math and science are done by millions of human researchers, but not so when it is done by massive swarms of agents at a couple labs.
I am sure science and math will benefit in the sense of solving more problems than ever before. But you are not really understanding the agony of the humans caught in the change.
> They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance...
Not "never again", it would be extremely difficult but science and math are also built on random, trial and error, and intuitions that could not arise from AI.
That's the issue - trial and error. A human can try a couple ideas in a day, but realistically, there are not that many sane ideas.
Computer can iterate without frustration on a completely random garbage. Once in a while it stumbles upon a gem. Even if it's one in 10000, that's still hundreds of good ideas in a day (provided enough GPUs).
Your comment assumes math and science are some kind of closed systems that can mechanistically pursue any endeavour but reality (whatever it is (e.g. the universe)) is much more complex than that, and there are realities that are outside of those systems. Not saying that humans can see all those realities though.
Maybe? I'm targeting math, which doesn't need any kind of reality check. It has axioms though, so in a sense it's closed.
You can generate completely random ideas (a couple proofs I saw where you define some weird set, pick an element and derive a contradiction when the set is empty or whatever).
Some of these will reach a conclusion: proof or a counterexample. And that's how speed can outperform creativity, if the speed bump is fast enough. It's the same as with those monkeys and Shakespeare.
> “Google Translate is amazing, but it has taught us essentially nothing about linguistics,” Strogatz said. “We don’t understand why it works. It just works. That’s the Twilight Zone. It is knowledge without understanding.”
This is not true, and it is the crux of an enormous debate we're now having across many, many fields.
Classical Chomskyan syntax (which models sentences as parse trees) has been enormously, astonishingly predictive of the rich structure in real-world human language. But we are quick to forget that we invented concepts like "words" and "noun phrases" specifically to help us analyze and predict things about human language. What LLMs and modern MT shows, definitively, is that there is another, totally separate way to describe these phenomena, which is both vastly less descriptive, and also vastly more predictive.
That these appear to trade-off is unfortunate, but it is NOT the case that we "don't understand why it works." We absolutely understand why it works, if we didn't understand we would not be able to improve these systems. What we do not have is a clean, descriptive theory that boils this down to something that is formally specifiable and analyzable. But those are different things, and this is the root of the disconnect between many communities working in and around these spaces (linguists vs computational linguists; mathematicians vs computational mathematicians; mech interp people vs empiricists).
"We absolutely understand why it works, if we didn't understand we would not be able to improve these systems" - Understanding how to improve something does not require understanding how it works.
Growing a plant does not imply understanding soil minerals or photosynthesis.
Training a dog does not imply understanding it's brain.
Using LLMs to improve a codebase does not imply understanding C++.
Improving a PC with new parts, does not imply understanding RAM.
You can understand abstractions about the system, without understanding the system.
Without gradient descent or other forms of automatic optimisation, do you think humans could build something as capable as GPT-1?
Not sure if this will bake your noodle or not, but yes this is basically my point. Saying I understand how dog training works has very little overlap with saying I understand how the dog brain proteome works, even though they're related, and even though I might (or might not!) understand better how to train my dog for having fully understood the proteome.
Saying we understand training and deploying machine translation systems that have strong predictive (rather than descriptive) power, apparently has very little overlap with defining a formal, analyzable, mathematical model of language. Mathematician who refer to "understanding" generally mean the second thing, but the first thing is also a kind of understanding. Just messier.
I think they're referring to the fact that neural networks are more or less black boxes. Their internals are blurry and not explainable in the same way that traditional linguistics tries to explain things - with rules.
Of course, ultimately that's because language itself doesn't follow simple rules like linguists would prefer. They're more like guidelines.
Yes, that is one of the points I am making. Chomskyan syntax is a strongly descriptive theory that is entirely ex post. MT and LLM systems are strongly predictive and have no descriptive theory. It is completely wrong to say the first is "understanding" and the second is not. They are both different types of understanding. In particular, Chomskyan theory clearly does not represent enough "understanding" to predict real-world English sentences with similar lucidity to that of LLMs.
I still think mathematicians are going to be as important as ever. The LLMs are only finding what was technically possible given our current breadth of knowledge after brute force solving problems. Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.
Finding that new knowledge may be exponentially more difficult, but it will be much more valuable.
> Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.
AIs cannot invent things like cubism, surrealism or hip-hop.
Or maybe it can; the early image generators famously did output horrific images sometimes because of their limitations, but these outputs qualify nonetheless creations.
Now the interesting thing is that it is humans who evaluate them as horrific, worthless or interesting.
Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
> Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
Online, how would they even know? Offline -- either we give up on the internet, or the offline discussion will be heavily influenced by what AI said online.
I don't see how its output possibilities could be infinite given a finite set of data. Taken to the extreme if you give an LLM only a single piece of information then that is going to be the only thing it can produce. Given it two pieces of information and maybe it can produce every combination of those two things. Given all of the worlds knowledge I'm sure its possible outputs are incomprehensibly large, but still not infinite. Then having its outputs match a set of really specific instructions is going to be a small subset of that.
How is this different from human brains being made up of a finite number of neurons in a finite number of configurations? I know very little about LLMs and nothing about neurology and maybe you can explain to me why one is so much more powerful than the other.
> a finite number of neurons in a finite number of configurations
Neurons and synapses change, especially synapses (the connections between neurons). Some neurons have a few connections to other neurons. Others have hundreds of thousands of connections to other neurons. These grow and change. "Estimates are between 100 trillion and 1,000 trillion neural connections" Is it finite? Depends on your definition of finite. If anything countable is finite, then it is finite but the number may be practically uncountable. Is it fixed? No.
As I understand it, the advantages of AI have not just been speed of processing.
Humans have cognitive limits. Humans hold a specific slice of all knowledge at a time and necessarily limits what correlations can be leveraged. This is similar to "context windows" in modern AI. Human minds are wildly variable in efficiency. AI is has constant inefficiency, largely controlled by the speed of light, physical RAM, and processing power, with superior (no cost) "thought" context switching. The AI also has superior accuracy without the need for "concentration" because the AI is processing on a Turing machine with the limited operations that it's superior at in terms of time and energy - eg find an element in a list of 1 million elements.
Sure, a given LLM can only produce so much (though it might be a lot), and then you'll need fresh inputs. But you contend that those have to be human, while I am arguing that they could be
- random,
- from the environment,
- from other AIs.
That increases the space considerably.
In particular, note that your argument also applies to humans: A human, being finite, can only produce finitely many things. But you add other humans, the environment, and pure chance to the mix, and it'll be pretty infinite for all practical purposes.
I would be curious to see the end result of letting it endlessly update its inputs from its own outputs. I was under the impression that eventually this would cause delusions and if you let it do this long enough and it is solving problems that no humans can verify as correct that it isn't useful.
Maybe that is the limits of my imagination, but it seems unwise to take humans completely out of the loop. Having mathematicians around that can understand what it is doing so that we can utilize that knowledge in a way that we can have some confidence in seems important.
Typical thought experiment - go back to the early years of humanity, even before we've created language itself. Communication is basic grunts and other such things, cutting edge knowledge is 'poke him with the pointy end'. Compile all of this knowledge into the training set for an LLM, let it run with infinite processing power and infinite time. It'll just continually remix the knowledge of that snapshot of time. You're certainly not getting to the Moon, discovering the secrets of the atom, or even discovering mathematics.
It sounds increasingly dumb to bring this up given how advanced models have become, but it remains true - they are still fundamentally token prediction algorithms. And that imposes hard constraints on their potential. These constraints are relatively academic in our current world because there's just so absurdly much information and no human can hold more than a negligible amount in his mind at once. So they'll certainly be able to connect many 'obvious' dots that we don't see simply for a lack of carrying capacity, but they're not going to discover concepts that do not clearly follow from the comprehensive knowledge of an era.
It seems far more likely to me that current LLMs have learned generic reasoning capabilities which will allow them to solve effectively infinite problems, rather than whatever vague limit on capabilities you seem to think there are.
1. LLMs aren't run deterministically - they use a bit of noise to improve the output. So even with the exact same input you're going to get a whole distribution of outputs.
2. Feedback from the environment. LLMs (and humans!) work best when they can test ideas. That's why coding agents work so well. If you ask an agent 1000 times to just think of a compression algorithm, it'll probably spit out very samey answers just like a human would. If you give it a test corpus and let it try out different things it will absolutely be able to come up with novel ideas.
Yeah, I think that's a great essay and it really helped clarify some of my own ideas.
At the same time, I think it's pretty amazing how this is the fundamental question of AI's trajectory, yet people on both sides seem so sure of their viewpoint with little actual evidence or data to back it up.
That is, it's possible that LLM-based approaches will be forever limited to this "convex hull" of current training data, and thus humans will always have a role to play (and will retain primacy when it comes to overall intelligence), because it appears that humans are not limited to this convex hull based on previous discoveries (e.g. I think basically every scientist would argue Einstein's general relativity was outside the convex hull that existed at that time).
Yet, if LLM-based approaches are eventually able to surpass humans on this "novel discovery" axis, then we as a species are basically done for when it comes to intellectual primacy - and then it becomes a very strange and scary world. And many AI researchers at the top companies clearly believe current approaches will scale and eventually (pretty soon I'll add) lead us to RSI.
I have no idea which side is right, and I'll admit I really hope LLMs eventually hit a wall. But again, I just am surprised that many seem to be putting forth their beliefs, but I would love to know if there is any good evidence one way or the other (or better yet, a more concrete and verifiable definition of the "convex hull" so that researchers could provide some falsifiable examples).
Came here to say this. What AI is doing with math is essentially fuzzing, but looking for valid inputs that solve a problem and the inputs are mathematical abstractions. It's a cool approach, but AI doesn't understand what's going on, and without understanding it won't be able to create the next generation of mathematical abstractions needed for the next generation of proofs.
aren't mathematicians needed to create the problems? Everything solved so far is a question posed by a human. Which is probably why cracking diseases is more difficult (apart from man made viruses :/)
Heard from one mathematician recently - math is not about problem solving, but about the artifacts that are produced in the process. The new Navier-Stokes solution for singularity isn't worth a million, it supposed to generate new mathematical methodologies, tooling and lead to additional discoveries, like the graph theory.
Oxytocin is one of the oldest molecules in the world — it created sexual reproduction, a very long time ago. "Our capacity to relate and to care for one another" looks a lot like oxytocin, which therefore belongs to all of us who reproduce sexually. There is nothing stopping artificial intelligence from making us humans return to "relate and to care for one another." Hugging, talking, laughing, dancing. Trusting, being trustworthy. Sex, relationships.
You know, I think Yuval Harari hit the nail on the head when defining humans as those mammals capable of gathering in groups of more than just a few dozen, by being able to believe in abstractions — like "a country," "being a fan of a soccer team," a religion, etc. Harari mentions how difficult it would be to keep forty thousand chimpanzees well-behaved in a stadium, and yet we humans do that every single weekend.
Which abstractions will we believe in, with or without AI? I think this part isn't really under attack.
Nor is our capacity to relate to one another under attack.
So both our relating-animal side (oxytocin) and our human side (believing in shared abstractions) remain untouched. They aren't being replaced by anything.
What are AIs — much like what ChatGPT did with mathematics — actually replacing?
Something else. But neither our capacity to connect with one another nor our ability to gather in the thousands, millions, or billions around abstractions. Just my two cents.
Your assessment is depressing -- the thing that makes us special has gone from "thinking", "curiosity", "understanding the world" to "we don't kill each other when gathered in a stadium".
That's a fair point of view. It's Yuval Noah Harari definition, not mine (Sapiens book, 2nd chapter):
"Ants and bees can also work together in huge numbers, but they do so in a very rigid manner and only with close relatives. Wolves and chimpanzees cooperate far more flexibly than ants, but they can do so only with small numbers of other individuals that they know intimately. Sapiens can cooperate in extremely flexible ways with countless numbers of strangers. That’s why Sapiens rule the world, whereas ants eat our leftovers and chimps are locked up in zoos and research laboratories."
(...)
"You could never pack a thousand chimpanzees into Times Square, the United Nations, the Vatican or the New York Stock Exchange. You would get pandemonium. By contrast, thousands of Sapiens regularly congregate in such places, and they usually produce orderly networks and trade rather than violence and chaos."
"If you try to crowd 100,000 chimpanzees into Wembley Stadium, or into Oxford Street, or into a church, or into a school, you will get complete chaos. But you can put 100,000 humans together in Wembley Stadium, and they will usually behave reasonably well, watch the match together and sing along."
---
But I’m not giving up on your celebration of our curiosity.
Speaking of curiosity, I’d bring Peter Gray and his "Free to Learn" book into the conversation.
Take a look at the second of the five characteristics of play that Peter Gray proposes:
"Play is activity that, from the conscious perspective of the player, is done for its own sake more than for some reward outside of the activity itself. In other words, it is behavior in which means are more valued than ends.
When people are not playing, what they value most are the results of their actions. [...] When people are not playing, they typically opt for the least effortful way of achieving their goal.
In play, however, all this is reversed. In play, attention is focused on the means more than the ends, and players do not necessarily look for the easiest routes to achieving the ends. Play often has goals, but the goals are experienced as part and parcel of the activity, not as the primary reason for the activity. Goals in play are subordinate to the means for achieving them."
Using Gray’s definition, mathematicians might actually be mourning the loss of play — the process itself, the thinking, the exploring — rather than "just" reaching the final result (via ChatGPT).
Is this a valid way of looking at it through Peter Gray's lens?
As a software engineer it is hard for me to feel sorry for these mathematicians. Software engineers are getting laid off because of advancements made by AI. Math researchers are employed at universities, and universities aren't laying off mathematicians now that OpenAI has solved some of their puzzles. Mathematicians can still feed their families, software engineers displaced by AI can't. Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
When software engineers spent the last 50 years eliminating the work of clerical, secretarial, and dozens of other back-office functions, were they concerned about how those people would feed their families? Do you think anybody but other software engineers feels any sorrow for them today, now that they are facing the same fate?
While I mostly agree with you, there's an argument to be made that classical programming tended to eliminate the mechanical parts of work that people hate doing, while AI is removing the intellectually stimulating part.
It's not fun to think of a future where knowledge workers become irrelevant in society and we're back to just physical work and managerial elites.
I'm not really one of them, but there are many developers for whom the intellectually stimulating part is designing the system. Writing the code was boring drudgery that they hated doing. They tend to call themselves "architects" or something similar, not "programmers."
That wasn't software engineers eliminating that work. And neither is it today AI eliminating software jobs. Let's name the straw man in the room: it's humans displacing humans, and extracting the value for themselves.
Just like in past history.
Yes, it sucks, for software engineers, those in the past, and those in the future. As long as you and others fall for the straw man perhaps you and others will get more and more tricked and hurt, misplacing your aggression and hatred on each other rather than those receiving most of the extracted value.
Yes in some cases, but the reduced costs also allow lower prices which benefits all consumers of whatever product is being produced. At least for those products where there is a competitive market.
What an asinine comment. Do you think I, as a software engineer, am sitting around twirling my mustache, cackling as I plot to destroy livelihoods? I'm trying to make a living like anybody else. I've had my own opportunities lost to outsourcing and H-1B's since day 1.
The economist John Keynes predicted almost 100 years ago that technological advances would lead to a 15 hour workweek. If AI now makes you 5x as productive, then why aren't you working 1 day a week for the same pay? Why didn't Keynes' vision come true.
It is not for the same pay. Real wage increased dramatically during that time period. And despite all obsoleted jobs, society is still more or less at full employment.
There are orders of magnitude fewer mathematicians with full time academic positions than there are software engineers. The economics are different and consequently academia tends to be more slow moving in reacting to market trends. It doesn't mean it isn't coming, but it will look somewhat different
I don't personally grieve for the likes of Strogatz or Tao, it seems likely both of them will be fine however this pans out. But many more mathematicians have been toiling away in precarity and obscurity and without tenure for years and years. And of course many mathematicians went into private sector roles as software engineers, actuaries, analysts, ML engineers and similar, and have been displaced there, too.
And highly visible blog posts written by famous mathematicians with large audiences and closely aligned with the news and hype cycles getting frontpaged on HN doesn't mean that mathematicians are disproportionately whiney, nor that software engineers aren't also writing about their plight or ending up on the front page as well.
To me this goes in the same direction as people not feeling sorry for software developers because our job is supposedly automating the job of other people (which I don't think describes the profession accurately as a whole at all).
Neither sits right with me. Can't we just all collectively acknowledge that we have to grapple with this new piece of technology that either is already or at least seems to be going to be hurtful to a lot of individuals?
Instead of looking at the in comparison limited group of people currently pushing this technology and asking ourselves whether we should be on board with their doing, we look to one another and are happy that someone else is now suffering just the same as we are. What a sad state of affairs.
I agree with you because I so thoroughly understand what mathematicians are saying. Expertise and knowledge developed over decades, previously unique and marketable, and status conferring, are threatened in their relevance because of ai.
I know this intimately because I identify, roughly, as a programmer first and security researcher second. Guess which two fields ai companies decided to specialize in first? Yes, the FML ones.
The intellectually rewarding part of my job/skills - the only reason I enjoyed them - are, not completely gone yet, but threatened. And shrinking. This is a very scary observation.
I know from ‘isomorphic situations’ lessons in history that fighting the tide is pointless. I have to decide intelligently how to respond. Easier said than done when your livelihood and identity are at stake.
For now I still have a job but I and everyone around me is worried. In a year, At best our jobs will be very different.
So yes I empathize with these mathematicians enormously.
I don't empathize with them that much, as many of them share in the blame of creating the broken system we have today. They want the status that comes with tenure and having a nice math job, and then pull up the ladder for everyone else.
Math journals commonly reject papers as “out of scope” or “not of sufficient interest” without an editor or referee actually reading them. So it’s not as if the “math community” was checking these papers for accuracy in the first place. Now AI speeds up the creation of papers that go unread, and we’re supposed to pity mathematicians? If journals and experts can’t be bothered to read the papers, why should authors bother to write them without AI? And suddenly mathematicians are worried about AI papers being “too hard to read,” even though that wasn’t a concern pre-AI, when no one was reading them anyway. Too bad. The incentive structure now points toward AI, and mathematicians share the blame for that.
100% agree. The comment you are responding to is basically playing "victim Olympics".
AI is going to suck hard for lots of professions, software engineers and mathematicians included. I think it's much more productive for folks in these professions to show solidarity with each other given the way AI will disrupt their livelihoods, rather than showing disdain based on who you think may have it slightly better.
As a software engineer? The career whose very reason, largely, for existing in this market is to automate jobs, with few notable exceptions? That one is somehow deserving of sympathy and that of mathematicians is not? Come off it. The notion is undignified.
Are phones an "automation of existing jobs"? Maybe couriers (??), but most of what people use them for goes far beyond that.
What about the software used to drive trains or autopilot planes? We still have conductors and pilots, but now the outcomes are safer on average and more efficient.
Or something like Google Maps -- all that replaced was the paper booklet people used to keep in their cars, it's not like we used to employ 'navigators' to go out on a road trip.
Or something like Intuit QuickBooks: there's still an accountant in the loop, and if anything more accountants today than there were a few decades ago.
Certainly some software is essentially automatory in nature. But to characterize 'nearly all' as being such just betrays a lack of awareness for the rest of the field.
> That one is somehow deserving of sympathy and that of mathematicians is not
He didn't say that mathematicians aren't deserving, he was griping that people said this about SWEs. Do you think mathematicians are deserving but SWEs aren't?
You seem to be unaware of the existence switchboard operators (people redirecting calls to the correct person) were hundreds of thousands in the US alone at some point.
How did you manage describe a litany of automation while being fully unaware that is what you were doing? I mean you even said the word autopilot lmao. Yes, not all tasks in a job are fully automatable. iPhones, automating communication. Quickbooks and turbotax, automating business workflow steps.
The words and excels of the world, automating documentation and bookkeepery. It all makes some kind of existing thing we already do faster and more convenient, and often more efficient. Most software simply does nothing new. The thing you didnt manage to list which is the actually good example is games.
> Software engineers are getting laid off because of advancements made by AI.
The bean counters laying off software engineers ought to take notes from the frontier labs' own hiring practices. They're still hiring lots of humans for business and engineering roles even when they would be better positioned than anyone to automate away those jobs with their world-class models (https://openai.com/careers/search/).
It's not about universities, it's about grant funding over the next few years. It is very apt to collapse if AI is more useful than people. Why pay people to find this when tokens will do instead.
I'll let you in on a secret: In the US, most math professors don't seek grants. In most universities, I don't think grants are even a factor in their tenure application. It's quite different from engineering.
In my undergrad math department, only 2-3 faculty members out of a dozen bothered with grants.
Who will continue to develop neural nets if the mathematics training pipeline dries up? Will we see the frontier labs relinquish AI research to the machines themselves while their humans apply for unemployment benefits?
Currently math research serves both as a training pipeline and as a math-production pipeline, and its funding is set based on those two outputs. If it is "just" a training pipeline, do we need as many mathematicians?
Who has been funding grants, and do you really think the grants are going to stop? There are infinite number of puzzles in Mathematics. The grants are not from companies like Boeing and Walmart who urgently need solutions for the problems, if they were I can understand why they might stop.
It’s only been a few months since LLMs started solving significant open problems in math. I wouldn’t jump to the conclusion that they have it better than software engineers.
In many ways it will likely be the opposite. Tech firms still need engineers to talk to customers and run experiments. What will the mathematicians be needed for?
Software engineers are getting laid off because of the end of zero interest rates, wars started by Putin and Trump, and a bit because of pandemic over hiring. The part that is due to AI is less hiring of juniors. Know your enemy before starting a fight.
What's your source for "Software engineers are getting laid off because of advancements made by AI". I haven't experienced that in my field and can't find any data to support that notion.
> Software engineers are getting laid off because of advancements made by AI.
This is said by a lot of CEOs trying to make layoffs look good, but I haven’t seen anything that suggests that it’s true.
> Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
Really?!? We’ve been living through 3 years of blog posts, LinkedIn hot takes, and podcasts talking about the plight of software engineers in the era of AI. There isn’t a week that goes by without an HN post about it.
I can feel empathy for the practitioners in a field that may have been profoundly changed, the same way I can feel empathy toward software engineers who are struggling with the changes in their own field.
In the end, we’re all trying to figure out where this leads. And how this will affect our lives and our work.
Unless this is satire, have you not considered how many proles have been laid off because they were replaced by technology or told that a spreadsheet model had decided their job was superfluous? Capitalism has been using technology to fond new ways to enrich itself for the whole age of personal computing, and arguably long before. Those Andreesen Horowitz comments about 'software eating the world' may have been exciting to people working at startups, but came off as rather sinister to people who were in the world outside softwareand just wanted to build a modest but comfortable life rather than develop unicorns and cash out.
The difference is it takes great intelligence to be a mathematician, whereas anyone can be trained to write code. It’s like the difference between an automotive engineer and a mechanic. I would argue writing code takes less skill than carpentry.
Do you write much code? Do you design software? It sounds like you don't. The challenge with software is managing complexity. This is unique to this field. Mathematicians are humans.
The idea that a junior software engineer who can't get a software job _and also can't get any other kind of job or move somewhere cheaper to live_ to the point that they can't afford to feed their families is what's unrealistic.
It's not like the options are write software or starve. Billions of people find jobs and can feed their families without writing software.
I also don't often meet junior software engineers who actually have families to feed. Most juniors work in the industry for a few years before deciding to get married and have kids.
I think your expectations of who gets to do math is uncalibrated. One famous math researcher who solved a major open problem was working at Subway while still doing math on the side.
A very small fraction of people who get math degrees actually continue to do math in their careers. They usually have to get other jobs in order to survive (actuary, teacher, finance, etc.). Academic math funding was already drying up for last few years, and now AI will probably cause more cuts. Not to mention that academic math positions have never paid particularly well.
I feel bad for anyone who can't find a job, but the software industry as a whole has been the cushiest industry for decades, and the current turbulence seems more temporary (and driven by corporate greed) than a systematic destruction of the trade.
Unfortunately this often means moving away from family. What are you going to say to the kid whose parents toiled to get him through college on the expectation that he would be able to take care of them? "Sorry Mom, I have to move to North Dakota to work on the oilfields"?
I happen to be (re)reading Strogatz' book "Nonlinear Dynamics and Chaos" right now. It's beautiful thing, and nothing in the "age of AI" threatens to equal it or in any way diminish my enjoyment of it.
I think the point is that in the "age of AI" virtually nobody will have the opportunity to follow a career and produce works like Strogatz was able to do.
I'm not disagreeing with the post. My personal opinion is that LLM "thinking" is qualitatively different to human thinking, that the "convex hull of ideas" idea is probably right in some sense, that the social structures we have built to allow and encourage human intellectual exploration are not obsolete and that there is a high risk of catastrophic damage being done to them by those who fail to see this.
Hacker News is so depressing these days. One doomer post per day to keep the valuations up.
Nothing has changed in math. Math is not solved nor has the human been kicked out of the loop. Same with programming.
*I said nothing really changed because LLMs becoming superhuman at deduction was priced in with o3 and neither the peer review, training nor publishing methods will get a revamp beyond what could already be done with SQLite and a website.
I'm no mathematician, but my impression as someone who at least does a lot of applied work with maths is that the main thing that's changed is that the yardstick by which academics earn recognition is being re-assessed.
But from my perspective proofs have never been the most interesting part of maths. What really interests me is the intuition that leads to new applications, and to new questions. Newton, for example, gets more credit for calculus than Leibniz because he's the one who demonstrated how darned useful it is. Erdos is arguably more famous for asking interesting questions than for answering them.
I still don't see LLMs seriously contributing to that facet of mathematics any time soon. Every so often I ask one to suggest a good binomial confidence interval for a particular application, and it still tends to just suggest one of the subpar options that everyone learns in a first year stats course.
Proving theorems is one of the main ways that mathematicians turn math into money.
The recent barf-out done by LLM companies is useful if what are you are interested in is an unreadable mess and someone giving you the final answer. You might be able to extract some new "aha!" moments out of it, but we will have to wait and see what happens once people go through it. I would hypothesize the LLM output will be good at hinting at some non-obvious relationships and patterns, but I doubt it will yield anything of substance in its own right.
Historically, proving difficult theorems required mathematicians to sometimes invent new fields, new ways of looking at problems, the discovery of which itself yielded incalculable contributions to both mathematics and other fields. It is this that is the greater contribution of and to mathematics.
> Proving theorems is one of the main ways that mathematicians turn math into money.
Mathematicians working at universities are paid to teach, to engage in the politics and business of government contracting, and only in very rare circumstances to play prestige games on famous problems.
It is unlikely that Congress will decide to slash budget for NSF's MPS directorate on account of AI getting good at proving theorems. That's just not how Congress works.
It is also unlikely that LLMs will be able to do, in the near future, do the sort of applied mathematics that happens in the context of occasional NIH or DoD funding that flows into Mathematics departments, for a variety of complex sociological reasons I'm happy to expound upon.
And it's unlikely Engineering degree programs will stop requiring calculus sequences, since being able to do the computation was has not been the actual point of teaching calc since he late 20th century (if ever).
So, the good news is that I doubt any of this will have much of an effect on the job market for Mathematicians who want to prove theorems for a living. The bad news, of course, is that the job market for Mathematicians who want to prove theorems for a living was already pretty horrendous.
I’m on your side on this overall, I think. And I suspect we’d agree more than disagree on nuances. But saying that ‘nothing has changed’ is very obviously wrong.
I guess I chose the words "nothing has changed" because that's true relative to all the subtext of content I'm seeing online claiming that "math is solved"
It was, but the comments on that post were insanely depressing to read. I'm a mathematician, and find myself largely hopeful and excited for its future, but being online this last week has been incredibly brutal due to the surrounding vitriol. It's an avalanche of negativity, and I'm largely not talking about that of my peers (though indeed, some of my peers). Most mathematicians I've spoken have been reasonable / positive, perhaps an effect of my field.
I really need to just stay off social media for like a month and keep talking IRL. Longer, if I know what's good for me.
I agree with "nothing has changed" in practical terms. If you take a look around most things rely on an undergraduate level of math than whatever is at the frontier. The proofs may be important, but they are really remote to people's lives.
This is like saying WW2 didnt happen while you were living in a Soviet village in 1940s, and you were doing the same thing, and nothing seemed to change
ZIRP is over but rates are not especially high and yet I don’t remember job market being that bad. Maybe employers are taking wait and see approach and avoid hiring developers they expect to layoff in a year or two if models will continue to improve and becoming cheaper at the current pace.
I think that models will improve and get cheaper. I think we'll see Opus 5.5 tier models open weights in six months.
You are probably right that some companies are factoring AI into headcount but I'd imagine that the interest rate and energy crisis are bigger concerns
Not that it explains all job market weirdness but you do have a problem on the hiring side with having to deal with the flood of AI generated applications. I assume that’s going to have a slowing effect on the typical apply/interview/hire process.
It was long predicted that the sign that AI is here in macroeconomics is rising interest rates, because investors can get MUCH higher rates of return doing AI investments than borrowing governments, why would I lend money to Greece when I can lend it to Anthropic?
I own my company and directly participate hiring. What’s changed among me and my peers is that we’re on incredibly precarious economic grounds with no telling what’s going to happen next month.
LLMs haven’t done much in the way of informing me or others I know regarding “who and how many”.
I would wager that most of them are AI generated. The cost of generating a neat looking LaTeX resume and adding vibe-projects to one's GitHub is very low...
And how! All the “nothing has changed, stop whining posts” are neglecting that they haven’t had to go look for a job. My previous company became quite toxic about LLMs and employee usage. It’s not “move over to the paradigm or get left behind” instead it was “jump on the hype-wagon and sing LLM’s praises all day, every day or you’re out”.
The doom posts will continue until profits are up.
Jokes aside, its like working for a company, individually you can lift mountains and someone else will take credit. When you use LLMs for a single bit, all credit goes to LLMs lol
Mathematics as a field of study is maybe 2500 years old and a lot has been figured out in that time. But look how far AIs have come in just ten years. I wouldn't expect progress to stop now, would you? A decade from now I wouldn't be surprised if machine mathematicians are essentially oracles. They tell us the answer but can't explain it to us any more than you could explain long division to a golden retriever.
This was the top comments when the submission disappeared from the front page.
Now the thread has been cleaned up and a pro-defeatism comment is the top comment and the submission is on the front page again (I'm not alleging moderator intervention, user mechanisms are sufficient).
You use the word "already" as if it already happened, but this is confusing speculation with reality. Market prices are a summary of speculation. Plenty of things are "priced in" that haven't happened yet, and maybe won't if the market is wrong.
Anticipating that something will happen is not the same as evidence that it's happened.
I was on Twitter around 2024-2025 and showed both Daniel and my advisor at UCSD sota results I created in group theory using Gemini 2.5 (edited) Pro in Q2 2025.
A lot of us knew what was coming. Profs I speak to these days say it's the same old except they need to pick a different kind of problem for grad students
the above, a "sorting" machine is the primary usage of hyperscalers currently, they're just trying to find new uses to boot for extra money/legitimation.
the browser industry
the phone industry
the CAs
JavaScript
it's all paid for by those wanting to sort humanity into whatever bucket the can fear monger the peasantry into for aweek or however long it takes the classification system to sort out.
gentrification works, but we chose forcible displacement now we have to live with the forever consequences.
would you give guns and bombs to the displaced native americans who taste blood over their beads and trinkets?
we've been inside the event horizon of the singularity for years, the tax bill will only get worse unless we recognize it for the tick that it is.
yeah nothing changed in math. mathematicians are weeping for some strange reason (maybe paid by OpenAI to weep?). And Terry Tao is posting blogs on AI + Mathematics because he was bored (or because he was also paid by OpenAI).
I don't get the argument - if we follow what you say, they're not _overreacting_, they are reacting appropriately but should have done so earlier (when o3 was released) rather than now?
My claim is that if Bob, a mathematician, believes that he is rendered useless by an LLM then he did not play with o3 and all the other models between now and then to realize that, yes, they are superhuman at details and deduction but not superhuman at math
AI will not replace the mathematical community (peer review, abstraction creation) until AI can train the next AI purely on data generated by itself and with a tool in a turing complete environment. The environment has to be air-gapped from human data but the AI can use tools and its own outputs
I think that "solving math" is isomorphic to solving the "thinking part" of the scientific process which would be genuine RSI
> I think I have been underrating the pace of model improvements. In March 2025 I made a bet with Tamay Besiroglu, cofounder of RL environment company Mechanize, that AI tools would not be able to autonomously produce papers I judge to be at a level comparable to that of the best few papers published in 2025, at comparable cost to human experts, by 2030. I gave him 3:1 odds at the time; I now expect to lose this bet.
“I’ve slowly updated my timelines over the past year…”
and
“I think I have been underrating the pace of model improvements. … I was not correctly calibrated as to the capabilities of existing models, let alone near-future models.”
> One doomer post per day to keep the valuations up.
Yes, outsiders (investors, journalists) are given a false impression while insiders (software engineers) are supposed to give up. Double win for the agitprop machine.
It's probably because with the release of each model it is becoming more and more difficult to see how a person's livelihood and way of life continues. Uncertainty stacked on uncertainty.
How will it pan out, no one really knows. The leading frontier labs are continuing to hire aggressively. But who knows for how long or for long or near-term goals.
The open ai math dump was actually what got me off the singularity wagon.
There were mistakes that could easily be seen with a diagram rendering three big results wrong, exposition is terrible, no coherent underlying theory generated from solving so many problems, as Thomas Bloom pointed out even duplicate proofs of the same statement in a single paper.
It clicked for me, "this is slop but slop can be superhuman and useful!". Similar to programming where it can generate as many criticisms of your codebase but you have to weed them out.
If anything, that makes the case that math is fucked even stronger. Nobody had any idea of how to prove quasi-Riemann. OpenAI produced three proofs, one of which was just a lemma in solving another problem. They have a machine that is better than every human being alive at math that solves problems humans cannot even when working inefficiently.
Math was never only about deduction. It's also about data curation.
I was taking to my friend in OA who was really surprised that the proof of L(F_2) = L(F_3) could be done in the 1980s. We just didn't have a superhuman deducer back then.
> Today, the mathematicians are weeping, because the machine smashed their tower first. But tomorrow, it may be the physicists, the economists, the illustrators, or the writers who weep over their fallen blocks.
Isn't it extraordinary that these models still can't write well? I mean, they write passably well, but they all have a highly characteristic style that you can spot from a mile away. These language models can crack the toughest open problems in math, but they haven't yet cracked human-like writing, especially long-form writing, without exposing themselves. This is very very odd, IMO.
Or, just maybe, they share a key property with the mounds of popular writing, journalism and random internet accretions they were trained on: they seem total rubbish on anything you know about deeply, but brilliant on everything else.
Their math is as just as good as their rambling writing and their Ghiblified poster art. Most people don't seem to realize that.
I think it is. Why is it that the only people in any field I'm aware of that think the AI output in their field is high quality are either the ones that are carefully crafting the output themselves or were producing/consuming slop themselves before all this started? (Outsiders pretty uniformly like the output, which is why I drew the parallel to Gell-Mann amnesia.)
The difference here obviously being that the part that you mention is more embedded in the proofs (long and difficult and near-hallucinatory) but the results that matter are that the proofs are correct and validated. So I think you're failing to account for that, the problem for math is that the journey matters less than the destination.
Hard disagree. If we send someone out exploring and they discover something (maybe?) but their account of their journey is impossible for anyone else to follow and contains obvious hallucinations, what have we gained?
Isn't the problem more like they did discover a new land (as in, the proofs are true, and the solutions are validateable) but we have no idea how to make heads or tails of how they discovered it.
Writing is a very slippery thing. “Good” writing from 200 years ago reads as unbelievably dense and overwrought to most people today. So even if you made an LLM that was “good” at writing, how long would it stay that way? AI systems can put out more writing per day than all of humanity combined, meaning it can also generate and beat to death cliches much more efficiently.
Language models don't have a characteristic style you can spot from a mile away. They haven't had it since before ChatGPT.
What is true, is that chat models (and by extension agents) have a very characteristic style after their LLM went through post-training. But the pre-trained language model doesn't have that style.
So if you want to write well with an LLM, you could and it is a well understood problem. It is more a matter of that not being of much interest, as it competes with having the model be good for chat interfaces.
How to have a well-aligned, RL'ed, safe or thinking model, which also writes well in long-form, is a bit an open problem. Though presumably agents could do it when they have access to a pretrained model.
Why is writing in the style of a corporate marketing department a requirement of being good for chat interfaces? Why can't they tune them to sound more like a normal human?
You can just tell them to adopt a different style. To make sure, add another pass with an agent that flags and/or fixes LLM tics. Humans typically do several passes over writings that they intend to publish, why are we expecting agents to do it in one shot? Just like the superpowers skill for software development does a spec phase, planning phase, several review rounds, etc., if you do the same with writing you get very passable results.
>Isn't it extraordinary that these models still can't write well?
It's not at all extraordinary if you understand how they're trained. For math and programming it's possible to generate an infinite amount of training data with correct answers, as compilers and theorem provers can guarantee correctness. For writing, the quality is not objective, it's determined by human preference, so you need humans in the loop, which slows things down massively and limits the amount of data that can be generated.
Math and programming languages are formal. Meaning they don't have any ambiguity in them. Every statement is what it is. That means they are very rigid and there's not a lot of variance. Natural languages are more diffuse and a written piece is more the reflection of the author than anything else.
That is why it's so disturbing. After several paragraphs, you realize there's nothing human there even when the actual information is present. People that do create AI piece are OK with it because they already have the context, but for the reader, it's always the uncanny valley.
There's no money in it. There's money in programming, and math is a good target to give the models superhuman reasoning abilities, but it already writes well enough for middle managers to write all of their emails.
> I am not one of those AI people who believes that hundreds of millions of jobs will be wiped clean from existence in the next five years
"Wiped clean" is the wrong phrasing, and 5 years is a short timeframe given the ability of some sectors of the economy to resist change. But yes, hundreds of millions of white collar professionals will need to rethink their careers and their business models within the foreseeable future.
Just today there was an HN article about how shady the dental industry is. In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
> In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
Right up until the sellers of dental supplies find a way to...
Indirectly influence model training to increase the frequency with which models recommend dental work and/or lower the bar for what they determine to be "necessary"...
or...
Directly influence model recommendations by paying model providers (who will be happy to take that money).
The point being that models (and particularly the humans that control them) are presumably just as corruptible as the humans they are replacing.
The advantage is scale. If a dentist is shady and overprescribes it can be difficult to find out because there are not many victims. If a company serving millions of people is overprescribing there’s a much higher chance of them getting called out on it.
That's the optimistic take. I have a much more pessimistic take: there is now only one place where people who want to sell dental devices have to focus their efforts. Less effort for them... and because of the scale... much bigger payoff.
When I was thinking of the examples I gave (poison the training and/or pay the providers), I was intentionally drawing analogs to how I saw Google search decline.
First, SEO "experts" poisoned the results. Then Google saw how much money could be made, and they sold the ad space directly to marketers. If you search for "Best <widget> for <doing something>" on Google, I am certain you are not getting the "correct" answer. And Google has suffered no consequences for this. To the contrary: it has been extremely lucrative.
Unfortunately in the real world it's just the opposite. A guy selling opioids on the street is called a dealer and gets arrested. A company overprescribing opioids to millions is called Purdue Pharma and gets record profits.
People are now aware of the harms Purdue has caused. Justice has not been served, but their plot has been uncovered and discerning consumers need not engage with them.
Journalists protect consumers from anti consumer practices. There’s not enough money in journalism to go after individual dentists even though many are bad actors. There certainly is enough to go after corporations, which we have seen many times. I’m not saying justice will be served, but I do think more public oversight = better informed consumers. There really is no good way to know if a dentist is honest before going to see them. That’s just not true when it comes to mega corps.
No one wants that. People pay other people to do things well. No one wants to need to discuss orthodontics with a bot, they would rather pay someone to know those things and think about them.
>Yes, some are crying about bruised egos, the feeling of wasted years, a creeping sense of status obliteration, the frustration of an unfair competitor, the indignation of feeling cheated and mistreated, and uncertainty about the future. But they are also mourning...
This nails it, however IMHO its almost all bruised egos and feeling of wasted time. I don't know the ratio of A:B but I also don't know anyone that is in the latter category of feeling doom and mourning that their craft is about to regress.
I don't know what's happening exactly, but I am genuinely astonished how little people seem to care about people losing something real, dismissing it as "bruised egos", using petty terms like "whining" and "crying", or calling them some insult for the crime of being angry or sad for having a big chunk of their life pulled out from under them. I'm referring to the big picture of common talking points, not just this case.
I guess I'm not astonished that people are so bad to each other in general, but more that in the case of genAI it's so prevalent in places that seemed more elevated previously.
Part of me wonders if this is actually a form of self-defense. The misery of others can provoke sympathy or the opposite: Angry pushback to protect oneself from dwelling on their misery.
Similarly, I got into software because I like working with computers and solving problems for others. I see TONS of similar sentiment where people are almost joyously exclaiming “Software is solved! Sorry!!! It’s been a long time coming, you all deserved it, get fucked!”
And I can’t imagine feeling the same about any group of people. It’s been odd, to me. I’m just trying to support myself and the people in my life, and now there’s a looming fear that I can’t do that anymore… business owners are drooling over that, and other sectors are laughing.
Tech workers are a small subset of the population that have been able to live very privileged lives when compared to everyone else.
I think the average person lives a much harder life and has been doing so for a while. Should we really be surprised by the lack of empathy when folks in tech had little to show for the common man this whole time?
I saw something similar on Twitter responding to 45yo software engineers making comments of depair in that HN thread about the increased suicide rate of middle-aged men.
For some reason I'm stuck in a real depressing, doomer algorithm / homepage on Twitter. But everyone on my homepage seems to have a take on the HNer comments along the lines of "see, this is why you don't make your job into your identity".
But that's such an antihuman response. The only reason you can put decades into a field to grind and build expertise is because you have an identity-level interest in a subject.
It's also not a good thing if the optimal way to live going forward is to have no passion for anything lest it changes so that we won't feel bad. I don't think that's possible unless you remove the human part of our brains or make life so horrible that we're happy just to survive another day.
Source? There has not been a statistically-significant increase in middle-aged male suicide rates between 2023 and 2024 confirmed data [1]. (2025 breakouts aren’t out yet, but top-line data show decreases.)
In 2025 in the UK men age 45–64 had a suicide rate of about 24 per 100,000, which is the highest it has been in the past half century, and is higher than other age categories. In the mid 2000s, the rate was like 17 per 100,000.
From your link, in the USA the overall rate for men of the same age is about 30 per 100,000, but varies dramatically between states.
Lots of people have lost their jobs and had to overhaul their lives due to layoffs or technological advances or political choices or factories moving overseas or whatever.
They are showing as much care for mathematicians as they received themselves, which is not much.
i think a big part of the backlash isn't about math per se and it's about what 'work' or 'a job' are supposed to be.
when mathematicians complaining about losing the part of the job that they love, people think "if you love doing it, why is anybody paying you? you're describing a hobby."
Yeah, the glorious utopian future we're building where humans are only able to make a living doing things they hate, and money is concentrated with the few.
Because we don’t see it as something for them to be legitimately upset about, except that it’s going to disturb their ivory tower unserious profession.
If AI got significantly better at performing surgery overnight and doctors started crying about how they were going to be replaced, what do you think the public response would be? Non-doctors want the best possible care; they aren’t interested in protecting an industry. The same will be said for pilots and lawyers, too.
People dislike self-serving whingeing, end of story, but that’s doubly true when it comes out of a profession (theoretical mathematics) that is mostly practically useless.
The doom and mourning is that it's no longer their craft. Their involvement is on the threshold of no longer being useful. What does it mean for humans when the best their involvement accomplish is slowing down progress?
This is a fundamental human need and it's being taken from everyone eventually, by design. It will take a generation to turn over and find new ways of being useful. Those who ran into this wall and could not safely reduce their income and retire/pivot to other fields, will be the coal miner oxy crisis of this generation.
Mourning is fine if it is temporary, however no man should dread their own future if they still have control of their life. This is the idea of unalienable rights and was enshrined hundreds of years ago. It seems to me that anyone dreading the future of their field failed to take a page out of their retirement strategy and diversify their own interests, i.e. https://en.wikipedia.org/wiki/Polymath
Ok, so if you really believe that losing one's vocation at 25 is the same as a 75 year old losing their spouse, that paper suggests you should wait at least 6 months before you start being a jerk about it.
I don't see what point you are trying to make. At 25 losing your vocation gives you more than plenty of time to transition into another lifestyle. Many people will consider this the very beginning of their career. At 75, retirement is near. Your legacy is known but not final. 6 months is less than 1% of your lived life, and you have a lifetime of memories to reflect and enjoy.
I'm not entirely sure what you're on about. Are you suggesting that mathematicians should become truck drivers or plumbers and be happy with that (until the next physical-aware models release, and then...)?
I was talking to a mathematician friend about this, and she was telling me that a lot of the proofs that have been produced in the past couple weeks don't have an application.
To me it begs the question: in a theoretical world where we don't have human mathematicians anymore, what is the point of these proofs if there's no human who actually understands it?
If the point of a lot of this math is only to expand the boundaries of human knowledge, it all seems kind of pointless.
heh these models should be turned loose on all the proofs paid for and sitting on the shelf and see if they can find an actual application useful to mankind.
Often, we don't know the applications of new math. For a long period of time we had "useless", "purely theoretical" mathematics that eventually turned into relativity and quantum mechanics.
Proofs are somewhat like legos. A bigger toolbox of them can only help in solving other, possibly practical, problems.
As far as the utility of a human being involved -- In a non-theoretical world where we don't have feline mathematicians, what is the point of these proofs if there is no cat which actually understands it? The cats can benefit from, for example, advances to optimziation algorithms that reduce costs for cat food delivery, of course.
Is AI at the stage that it can also point out all interesting things to study and then solve them? Otherwise humans would still be able and needed for progress.
Good question. I'm not a professional mathematician, but I would be interested in seeing what results the mathematicians get from trying to prompt it to come up with new and interesting problems.
My intuition is that it'll be at 'smart undergrad' levels today, though as the cryptographers say, attacks always get better.
None of them have pointed to any of the solutions and said "this is real innovation" - because it's all been mining existing ideas and applying them elsewhere.
Imagine you spend years working on a solution to one of these problems. Only to realize after you publish that more than half your answer already existed in another branch of mathematics and you spent a bunch of time re-inventing it.
Yes the first big solve by OAI was "stolen" and that is tragic, and pretty bad. But it looks like this new batch, the one people are crying about, has none of that stink.
I think there is totally justified distress knowing your source of livelihood may be in danger.
That said, I don't think the reactions have been entirely circumspect.
1. Is the role of math academics really going to go away? Is there concrete evidence of this? Faculty still have very important roles to play, training students, training engineers, determining the direction of research. The breadth of math is absurdly, perhaps infinitely, wide, especially if including applied math.
2. You need to be a bit crazy to go into pure math academia in the first place. If you value the pursuit of knowledge for its own sake, shouldn't the emergence of such powerful tools to that end be a good thing?
I need to do some more research on this. Because I've always been told that AI, more specifically LLMs, are "glorified autocompletes". Yes they have huge data sets but ultimately they can only provide answers based on data that they were trained on.
So how can they possibly solve problems that humans have worked for decades or centuries on without being able to solve? How did they find something new? Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
The following certainly does not describe all mathematics or all LLM operations, but its my mental model for why LLMs are so effective in several fields.
Lots of mathematics entails recognizing that a tool/techniques that works in one area can be adapted to a completely different problem. For a human to do this requires having a robust mental model of the technique and the problem itself, and probably having just the "right" mental model to see how they fit together. And we are talking about extremely hard/sophisticated techniques and extremely hard/sophisticated problems, so the ability to deeply appreciate either can require years of study. And on top of that, even if the mathematician has sufficient understanding to develop the intuition that the technique might work, they will need profound fortitude, skill, and patience to see the solution through.
Unlike human mathematicians, who will develop a deep understanding of only a relative handful of techniques and problems in their lifetime, LLMs have ALL written techniques, problems, solutions, etc stored in their internal representations. So when they ably match a problem to a technique, we can imagine the operation as a supercharged autocomplete, or even an extremely proficient search engine. Their stroke of genius is really a consequence of capacity, moreso than cleverness.
They have access to a much larger corpus of information than any single human can manage in his one brain. No human mathematician reads all the papers and has perfect recall on them. The answers have always been there, but no individual has been able to make all the right connections to pull them together and formalize a new discovery or proof.
> Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
Yes, both of these + an infinite token/compute budget and SOTA+ models. It's not like some hobbyist out in the woods with a GPU running Qwen 27B can do this.
Knowledge without understanding would be just getting "42" and not being able to ask any follow-ups.
But I don't think that's where we are at the moment. I believe the results, proofs and papers can be reviewed and understood, just like if a human peer had published another result.
I also think that the speed at which results can be now generated can't be followed by our collective speed to understand them, using the methods we are used to.
As we enter the next phase where AI publishes using mostly references of other AI results, this will compound exponentially until it would take any human decades to verify a single result by hand end to end.
I agree translating a phrase doesn't teach you a language, and I also believe translators are an essential tool that results extremely helpful when learning a new language.
So instead of throwing the towel, could we develop and adapt ways to improve and aid our understanding and verification processes?
I'm convinced the people who do not see what Mathematicians see simply have not considered that their jobs are just as vulnerable. The day when OpenAI replaces their own AI researchers with GPT will be upon us, and the response from the Mathematicians will be "First time?"
“ What makes us special, it would seem, is this capacity to compound.”
Yeah so you’ll compound using the AI tools and get more done. Stop crying that your tilting at windmills is now being done more efficiently and find new windmills to tilt in new AI-assisted ways.
To me it seems the ultimate question is why would we expect humans to have an advantage at anything in the near future. A few years ago that would have sounded insane, but intellectually I don't doubt frontier models are superior to the average person. Even in the arts and humanities, give your preferred frontier model an interface and a way to "see" its work and it'll make the average person's work look like toddler scribbles. Humans still have some advantages, some places more than others, sure. Working with a designer recently I realized how bad, or maybe average, frontier models were at design. But I highly doubt the vast majority of people will be economically viable. Maybe this is one of the great filters, a society driving itself mad when freed from necessity of economic utility and without the big philosophical questions having answers.
As usual the author is ending on a completely unrealistic positive note: "humans may come to value more highly our capacity to relate and care for one another".
Does relating to someone pay my bills? People will use the informational tools to improve their social status and resource accumulation, there is no doubt about this.
There was a time, not too far in memory, when the value of being human wasn't about our intellectual prowess[^1]. We were creative with the human worth thing: being there in the time of need, faith, nobility or being a well-dreaded, literally-bloody viking full of beads in your beard.
Yes, I fear we will stop being the top of the food chain[^2]. But by god weren't we getting bored? Maybe getting off our high horses and going into the weeds won't be such a bad thing.
[^1] And even today, if you go to parties and keep bouncing into people that have no trouble picking up intellectually stimulating topics, you are in a lucky minority.
[^2]: More about my fears, which are unhealthy and you don't have to read but I feel compelled to include citations: https://w.ouzu.im/aweb/unhinged
I think an under appreciated part of mathematics has been contributing clean and elegant work precisely so that others could use that as a foundation for further work and new insights. This dump from Open AI seems to be the opposite. Huge and messy, incomprehensible and taken seriously only because of Lean.
There is this assumption that AI will contribute to faster progress in math but given the garbage dump of proofs which we have to take on faith are robust I wonder if the opposite is true.
My reservation could only be dismissed if AI turns out to be able to contribute fundamental new insights and pose new problems. This takes us into an area where AI progress has been slow to say the least.
At least with AI, you can download the LaTeX, upload the paper to your favorite LLM, and have it explain the paper at whatever level of granularity you like. That wasn’t possible even two years ago.
I think people vastly underestimate the ability of human to respect human performance despite being inferior to machines. Think about chess players vs stockfish, marathon runners vs a car, etc.
I think the crux is much more prosaic, as buried in the article.
> I also don’t want to lose my job to a tangle of GPUs.
To put a GPTism, it’s not about the knowledge or status, it’s more about keeping their job and money.
And that can only be answered by policies, and imo by UBI. So votes if you can people, protest otherwise, discuss, debate, lobby with your friends, families acquitances.
Jobs and money are bottom of pyramid needs. If we answer those with UBI I guarantee you status and purpose are the next ones.
Chess players vs stockfish isn't a good comparison because chess fans only care about the performance of maybe the top-50 players.
People may gain and be satisfied among their own peer groups. All good and healthy if that peer group is your local badminton club or knitting circle - something where effort and skill have rewards and people can make objective, tangible progress. But groups not oriented around such activities could turn even more toxic.
I am not quite saying "if you can't beat 'em, join 'em", but I know more math now than in any point in my life because I want to understand how LLMs work.
I understand there are many types of math and the mathematicians don't all want to sit around and do KV cache and gradient descent things, but surely a percentage are as interested as I am and can contribute to this new emerging field? I would wager that Xl+1=Bl Xl+Cl ℱl (Al Xl) makes much more sense in their mind than mine.
This is the best attitude to have about the current state. Unless you have some dramatically different sensible solution (and you probably don't when it comes to technology, because it seems to spread for reasons that would eliminate other sensible solutions) that will catch on and take over and replace AI: it's going to continue to mature as long as it's mostly useful to everyone. And AI isn't even a new industry, it has been maturing for quite a while, which adds more weight to the argument that having an attitude that adapts to technology (unless you wish to go solo) is a 'better' fit.
The dramatic solution can never be: 'we should all just want to do it this way because it felt better to me'. That never seems to work, from what I've seen. So there is a lot of public discourse, arguing, to determine if 'doing things the old way' is perhaps more better. And if not, maybe we just shed a tear and move on (to look for a better fit).
As a mathematician, I will never understand how my colleagues had never foreseen this. Okay, maybe you can believe in a plateau but at least accept as a possibility that there won't be one. This applies to every other human endeavour.
I am also sad, but my sadness is, in a way, priced in.
It's not really a profound insight. Back in the 1980s when I first started writing BASIC programs I remember thinking "well, if someone invents a program that can write programs, there will be no more work for programmers"
It's easy to have that thought, and eventually you'll be correct. One thing about technology is that it only improves. It's always building on top of what already exists. The only uncertainty is the exact timing, not the ultimate outcome.
The people struggling right now - they were told. 30 years ago. They really should have come to terms with it by now.
But these are also the people who refused to believe it, for whatever reason. Their heads were in the sand. So I do feel bad for them... but... this is somewhat self inflicted.
It was really quick. I stumbled across a comment from KConrad on MathOverflow on how useless AI was for research. That comment was from 2024. When it disproved the unit distance conjecture in May, the solution involved a breath of knowledge of the literature that it made sense it took a computer to make the connection. It only took 5 months for it to advance to the point that it was better than humans at almost everything. Even if you expected it, the mind boggles at how fast it was.
Also, it's impossible to really know how you're going to react to something until it happens to you. When ChatGPT came out, I already thought "in the long run they're going to automate almost everything humans love to do". Despite having 4 years to brood about it, I did not really know what it would feel like until it happened to me directly.
Even before LLMs it was feeling like the design of society wasn't really for people any longer, instead it's more and more designed for corporations and their convenience. In the US at least. Same with how the internet has stopped most people with socializing the same way they did in the past.
Now we have a new layer that will be optimized for these digital entities. We see it in writing all over the internet. There will be tons of other places they show up and doing things that don't make sense to us.
I don't see how this is possible. There are constraints on the speed at which information can move and the number of steps required to compete a given task. The improvement cannot continue, let alone accelerate, indefinitely without bounds.
Okay, how can we be certain even about that? It sounds just as plausible to me that the improvements will become asymptotic and boring, not exponential or inconceivable -- and how can we make any confident prediction at all about the heights these models will reach? The technology those heights require doesn't exist, and its existence isn't guaranteed, even if it is possible.
I think what makes it hard is that it's quick, but not sudden. I understand constant-rate improvement. I understand what it's like to be in a car crash. What I can't understand, emotionally, is the constant acceleration. Even now, if I look at an OpenAI solution in isolation, I can understand the big picture, so if things froze where they are I could adjust. Intellectually I know I'm not going to get that chance, but emotionally I don't.
Programmers, artists, anyone who used to create any form of documentation, translators, there are literally so many.
Either these mathematicians thought it would never come for their jobs, because all of these other jobs were simpler or they've been living under a rock.
Either way, I don't feel for them any more than I do for all of humanity.
When anyone can push a button on a machine and out comes a math result, it still doesn't become theirs or anyone else's until they can internalize it and make it part of their thinking.
"I can’t solve these things, but maybe you people in the future can stand on my shoulders."
Seeing AI being lauded for solving problems like this makes me feel... uncomfortable.
We are feeding all works of the giants that we have all stood on into a huge vat, in the hope to generate something meaningful.
If we can't understand the output, internalize it, or build from the time and experience that a human would have gained from solving the same problem, it feels like a dead end.
When all the problems have been "solved" by AI, what's next? If humans no longer struggle to solve the problems that the previous generation have left behind, how do we progress?
The author mixes "Science" and "Mathematics". Mathematics is not a science. There is nothing empirical about it. By mixing the two concepts, a reader might think that AI is on the verge of solving all of science. Until AI gets good at digging large circular holes in the in the ground, I think Experimental Physics is safe.
Maybe math is more like a science now, though? It's as if ancient astronomers saw the results from a modern telescope. "Why even look at the sky anymore," they might ask. "And we don't even own the telescope!"
Maybe the answer is to look into how Big Science works? There's still plenty of work to be done figuring out what to do with the outputs from these new math-observing instruments.
I had visions of large automated warehouses full of chemical and biological components hooked up to science machines to do everything from mixing, boiling and printing to spectrograph and robotic assembly. Smart computers and humans would pick the experiments to brute force with exact billing.
I remember the same doom cycle during my PhD when GPT 4 came out [1,2]. It seemed like everyone in NLP was freaking out except me. A few years later and new CS PhDs are still graduating and joining academia and industry. Much ado about nothing.
Considering I currently live in Ithaca and have a friend doing a Math PhD, so I've heard from him some of the discussions in the department. I think the biggest shock for people is that individuals spent significant portions of their time on some of the problems that OpenAI supposedly solved (I say supposedly because all of the proofs have yet to be independently vetted). I understand that it's a shock, but I'm confident that in time it will be no more of a blip than it was for NLP.
A blip for NLP? What bespoke NLP research is about anything other than LLMs these days? How many companies are handcrafting bespoke NLP models instead of sending a LLM request? Really confused here.
My friend, who is a math professor, told me that they weep because they are about to become just like us developers: prompt writers and AI output reviewers.
> My friend, who is a math professor, told me that they weep because they are about to become just like us developers: prompt writers and AI output reviewers.
If it was about understanding interesting proofs that some AI came up with, and extracting their essence, I don't think the situation would be so bad.
I see the much larger problem in the now existing career prospects of non-tenured researchers in mathematics. Everybody know how precarious the career prospects in academia and how precarious often the non-tenured positions in academia already are. With the new AI breakthroughs in math, these have in my opinion even worsened a lot.
I love that we have new ways to solve problems. And it's been hard to accept that my own unique intelligence—which I've spent decades cultivating—just isn't as important as it used to be. I am developing new ways of thinking to better adapt to an AI-enhanced world, but I'm not done with the mourning process. We're all finding our way through this.
I object to a few of the assertions made by maths folk in the article- 1st being that Google Translate taught us nothing about linguistics. If you use google translate you learn less about linguistics than if you had to experience learning the languages first hand, maybe, but you can learn form just that output if you are determined to, ad, more importantly, the folks who built it and are still building it are still learning a huge amount about linguistics.
What are the relationships between many languages that become apparent, including code as a language that were not obvious before llms? Translation is fairly solved. The study of how and why languages, not at all.
A hypothetical... suppose AI "solves" (for your preferred definition of "solve") your field and what you used to do becomes useless or near-useless. But you still enjoy doing it and you also get UBI that provides for comfortable lifestyle regardless of what you choose to do.
Questions:
(1) Do you still weep for what has transpired?
(2) Do you continue to do what you used to do (with optional AI assistance)?
For me: (1) no, (2) yes and I think the same would be true for most people. So I can't help but believe that most of the anguish about AI is far more about a direct economic hit to people's livelihoods than it is about the impact to their respective fields. Which is more than reasonable. But then why not frame the conversation that way instead of beating around the bush?
We do for seniors, who are much more expensive than average due to medical costs. In a hypothetical future where AI is much more productive than humans we could easily extend that to everyone.
no we don't? Also varies massively between countries. UBI talk is just distracting the masses from the impending change of paradigm. UBI is basically Blisstonia.
I’d be okay with UBI and hobby intellectual work for the rest of my life. But I’m not a mathematician. Many of them probably have more of their identities tied up with their cognitive abilities and social standing.
For many people, the answer to 1 is yes. You maybe happy with our future where we putter around aimlessly as consumers, but some people want to live in a world where they can at least try to do something that matters. Virtually every professional mathematician is motivated by personally making a permanent contribution to human knowledge, no matter how small.
I understand that sentiment, but I don't think it really counts if the only reason you were able to do something first is that others were forbidden from using an AI that would have done it faster.
Article refers to Hugo Duminil-Copin, who also did a podcast with Brady Haran [1] about this subject, for him it isn't case of bruised ego, or gatekeeping. He realized that the way things were done were going to change, and it made him emotional, he is not pessimistic or against AI.
One of the driving forces of math achievement is an internal award (yes, friggin business speak).
The vanity, fame, self-improvement, it's very selfish as everyone does.
Now the fame coming from proving things like Riemann Hypothesis is taken away.
Self-improvement? relying on AIs make you stupid. It will be lost.
Vanity? Who else don't know how to use AIs? It cannot fill your void.
Now also the most important thing is the communication.
There will be less communications amongst mathematicians at least in person. On paper? maybe but mostly via AIs.
I don't think it's the communication that's lost, it's the motivation. People love to collaborate, but they don't always want to waste time doing it.
It's the same reason you see 'no AI' repo labels: only by forcing some standard on the community can the 'old way of collaborating' continue to exist.
There will for sure be new ways of collaborating, and walled off gardens like you suggest, where communication stops and everyone just shares some final product.
But, in some cases the product must be understood (or demonstrated) before value can be derived, and so collaboration will still exist.
Desire, and the 'shape' of collaboration has changed greatly though. Why collaboration has changed, is a completely separate issue related to economy and how each of us spends our time.
Desire and motivation are factors to consider. Would you want to invest 5 years of effort into solving the Hypothesis 'the old way', knowing someone is probably working on it right now with AI?
And then once it's solved by AI: do you want to churn through the proof to understand it? What if it takes you 5 years to churn through the proof, and it turns out someone made a mistake with AI (I have no idea what would be realistic amounts of time, but the idea is still worth considering).
The issue is very complex, and you will always be hungry.
For sure, I just wanted to correct the impression that it has been solved. There are a lot of people going around pretending LLMs have solved everything and human maths is over, but what has been achieved with them so far, while impressive, is a very, very long way short of that.
If I were a mathematician working on the Riemann Hypothesis, I'd probably be rather pissed off right now but continuing my work, i.e. the same as I am as a software developer.
I get you. What people are doing now is questioning value: in themselves, and what they can create for others. I hate to always oversimplify things but we're pretty simple things to begin with:
Social creatures that share a common space, with finite time that nobody wants to waste.
OpenAI didn't have to mass publish these solutions. They did it to make themselves look good. They are ostensibly making a tool for people, to better humanity. Instead, they chose hubris and self-interest.
Letting mathemeticians use these tools to reach the same conclusions would have been far less destructive, and potentially, even exciting. Shame on OpenAI.
Mathematicians are smart enough to have figured out which way is up. You can debate the stuff outside someone's field not knowing the details but the insiders know what is happening. Ai may not be better than someone with very specific knowledge at this point but it is certainly better than the net. Once it gets optimized for every little super human knowledge, it can do better.
These guys know because they looked at it. Denying it is useless. But maybe some of these guys can figure something new using it and asking it to look for it.
I'm wondering if mathematicians are sort of going through what programmers have been going through recently, really questioning whether their craft can continue. But now it feels like, in the software world, at least from my seat, people can handle much larger scope and work on much more complex problems. If anything, my job has gotten a lot _more_ interesting.
So I wonder if mathematics will go through that too? A period of chaos and doubt, but finding new and more effective ways of working.
Mathematicians are grieving the birth of something that is, in a very real sense, the crowning achievement of mathematical ingenuity.
I never appreciated hard math much because my brain just can't get there. I thought so much of math was much ado about nothing, but now I see the art in math...not in the inscrutable formulas, but in the creation of AI.
Well done, mathematicians. You can now rest from your labors.
It's got to be alarming. Without a doubt, being not only on the frontier of yoyr profession, but in THE profession that is undoubtedly the most intellectually rigorous and well renowned and respected... It's got to feel pretty awful
We are seeing many fields being impacted. Makes me happy to see things like artcraft coming out that directly impacts the big tech companies themselves.
I think they might find comfort in knowing it's not just mathematicians - this is coming for all of us. Computers putting us out of work has been coming for a long long time.... and now it's here.
Mathematicians should web search the arguments when artists were hit by the wave. Scientists in general and people that think humans know better.
This is probably the first time in history deep thinking is being replaced by a machine, but the overall arguments are the same in all cases. Graphical artists saw it first, then programmers, now mathematicians.
No, there is not stopping the wave. No, society won't probably be able to support the same number of mathematicians as before. But yes, there will still exist mathematicians after the wave break, just probably completely different than the old way.
And yes, some people will still try to do artisanal mathematics and say it's better, it probably is, if the artisanal mathematician in question is doing deep creative and out of the distribution work.
Those of us who loved programming have already been through this.
I hate programming these days - back in the day, the cubicle-hell nature of "popular" frameworks was at least offset by being able to think through and solve interesting problems.
These days most of us are stuck in the "endless-review" loop with LLMs. You can't afford to not use it given the inflated expectations. As with all of modernity, productivity doesn't yield leisure - it just proportionally increases your workload.
Hardly surprising that the other side of the Curry-Howard correspondence is now getting hit with this. Math was one of those depts. where slow/deep-thinking was possible (CS was already insane 10 years back - now it's just absurd) - now even this fortress will be broken. Tragic.
My take is that it is an exciting time to be a mathematician since the frontier for interesting math problems is about to explode, and without prior art or references, it's green field for both AI and human. Think of all the new discoveries we'll make.
The beginning of this article is decent, the end sounds like a tobacco lobbyist touting the health benefits of nicotine.
> If intelligence becomes less uniquely human, we may come to value more highly what remains uniquely human: our capacity to relate and to care for one another.
Lol. How does the author think relations between people work? (At least those outside the family). Relationships require something that fills them with meaning - a reason why you would want to have that relationship in the first place. Without meaning, no relationship.
This is such an insidious propaganda targeted at the demoralization of entire professions.
Strogatz is an AI booster who "wept". Totally representative!
How much do they pay you? Does OpenAI have bulk subscriptions to your Substack?
> "Earlier this year, Shanmu Jin, a Chinese medical resident, used ChatGPT to solve a problem called Crouzeix’s conjecture, which professional mathematicians had failed to solve for decades."
I note that that these conjectures being solved by AI are named after people, minds knowledgeable and and brilliant enough to come up with the question in the first place and recognize its value. What we have in these AI math solving systems is the equivalent of the chess and go software that can beat humans: something that can adhere to a set of rigid rules to reach a win state once given a starting point. But, as Pablo Picasso said, “What good are computers? They can only give you answers.” These AIs are a magnificent achievement and will change mathematics and coding but can they ask earthshaking questions? Could an AI identify conjectures that mathematicians recognize as a worthy question that will lead to new insights? Where is HAL9000's theorem or the M5 Multitronic's conjecture? Maybe they can someday but I don't see any immediate prospects for it.
Please spare us the "AI will surely keep getting better forever" (unlikely and don't call me Shirley), "moving the goalposts" (I myself haven't set any), etc. non-arguments.
I still remain hopeful that this is a step change in capability and capacity that we can adopt and adapt towards our own purposes. What was taking decades for us to do has been reduced to days and dollars. So while we’re witnessing a forest fire consuming entire fields of open problems in math, the best upside would be finding new problems that come from learning how to work with the new tools.
This is what it seems like so far for software. The speed limit has changed. It’s interesting to explore what that means. Extending the analogy, long ago you could be a race car driver part time; that would be pretty laughable for an F1 driver. It’s a very specialized skill and the optimal athlete is quite different for driving a car than for running, for example. That feels like what I’m seeing in software.
The most effective folks are much faster and better with these tools than average. It’s a different skill set.
I think that formal verification makes Math much closer to code than many other fields. But I’m eager to see the first mathematicians that are dramatically more proficient with the new tools than average.
I mourn the loss of comfy, intellectual, thinking jobs like those of the mathematicians where people could spend time pondering without the immense pressure that would come from doing something like that in a corporate environment.
I wonder how much this is like a tremendous telescope being invented which can see new worlds with utter clarity. We could verify that a planet was habitable... but it wouldn't bring us any closer to visiting it.
This current moment we are in feels like something out of the Remembrance of Earth's Past trilogy, bit instead of humanity becoming super depressed and hopeless about a future alien invasion we are super depressed and hopeless about LLMs destroying all our hopes and dreams.
These guys are still in the early stages of grief. They should look at coders; we've already been there. Coders don't assume coding is going to remain a profession. Why should any other profession be exempt from disruption?
Honestly, there are much bigger fish to fry, like how will everyone pay their bills, and how will we prevent catastrophic AI.
...except it is exactly the opposite of your big brother destroying your tower. Instead, it is your big brother showing you this method using a tool to build many more towers at astonishing speed!
I get the sadness (and, as a software engineer, certainly feel a real existential threat myself) and feeling diminished, but at the end of the day that is all just ego and I very my agree with the end of the article - who cares who came up with it and on what shoulders we stand, as long as it is saving lives and improving our understanding of the universe!
Not sure about that title, it did bait me into a click. "Mathematicians are now facing something similar to generational trauma of structural racism born of exploitation, slavery, and colonialism?" https://en.wikipedia.org/wiki/I_Know_Why_the_Caged_Bird_Sing...
But regardless, the detection of breast cancer via AI made me think of phlogiston https://en.wikipedia.org/wiki/Phlogiston_theory . "Phlogiston theory states that phlogisticated substances contain phlogiston and that they dephlogisticate when burned, releasing stored phlogiston, which is absorbed by the air. Growing plants then absorb this phlogiston, which is why air does not spontaneously combust and also why plant matter burns. This method of accounting for combustion was inverse to the oxygen theory by Antoine Lavoisier."
It could be phlogiston. AI is super smart theory states that AI is super smart and that is a single thing can do tasks very efficiently. But it's so smart, we can't understand it, so we humans can't help it, it'll take AI to make it even smarter. That's what's holding it back. But then it'll solve everything.
Or it could be oxygen. AI is super-fast-and-parallel theory states that AI is not a magical monolith but a bunch of pedestrian programs that run fast and in parallel, thanks to hardware (GPUs) that's good at that stuff. But since it's all going so fast and at the same time, something a single human cannot do, humans can't really see how it works. But if we could, we'd probably just see its just doing stuff programs did before. So what's holding it back is that you have to keep throwing hardware at it, and eventually you run into the limits of how fast you can build that hardware or power it.
It all might be stuff humans were better at, when approached one-on-one, but against a swarm it loses. You get a billion doctors together to look at a single mammogram, then tell them if it cancerous, then do that again a billion times with each doctor. Then ask those billion doctors on the billion-and-one mammogram to give a diagnosis. Heck, do that with a billion toddlers. I think either might be able to beat the AI. I don't have a billion doctors or a billion toddlers--frankly I don't know which would be more difficult to handle.
To know if it's oxygen or phlogiston takes probably a demonstration like this "Phlogiston remained the dominant theory until the 1770s when Antoine-Laurent de Lavoisier showed that combustion requires a gas that has mass (specifically, oxygen) and could be measured by means of weighing closed vessels." Maybe the data center financing trouble will be the demonstration that the mass in the gas is just VC-money hardware.
I know why you got a billion cages with a billion birds and a billion bird listeners to compare a billion songs to the few songs we like to hear, and then packaged all that into a single bird-looking thing to make singing-like sounds: So Altman can get that VC money.
I strongly feel most angst is because Mathematics has become yet another capital intensive game to play. Rather than a noble, purely intellectual practice.
> Think about a world where non-mathematicians leapfrog the field after a few AI prompts. It would be a world with more knowledge. But it might be a world with less understanding.
This seems like a good summary of the article, and I think the answer here is obvious.
So what?
News flash folks: we do not have the knowledge to survive on our own. If a solar flare deletes the electric grid, the first thing that happens is 8 billion people die in the next year.
We survive on abstractions others have built, and we have a global dependency web.
AI abstracting knowledge is neither new or dangerous; if it is dangerous, this certainly isnt the argument to prove it.
>I wanted to talk to one of the mourners to understand their technological grief. I wanted to know if theirs was a familiar case of bruised ego or disappointment—akin to, say, a boy who spends an hour building a tower out of blocks and watches his big brother smash the thing to smithereens—or something stranger and more profound.
I'll be honest. The response to mathematicians, even this one, makes me concerned about the lack of empathy and honestly just theory of mind of many people in tech. It's not a mystery why people are upset that their field, for many the singular passion to which they've devoted their entire lives, is being annihilated so that a massive company can boost its IPO valuation. It's not "bruised ego" ffs.
‘Tech’ culture has been getting nastier, more vicious, and more zero-sum for years now. It wasn’t even all that long ago that I remember feeling hurt when a friend of mine sneered about “tech bros.”
But I get it now. I see a certain type of guy in an AWS fleece and instantly judge him, round him off to some kind of bully. Harsh, unsmiling, tweaking on stims, telling you to cry harder. Or maybe he’ll open up tell you his theory of how we need eugenic breeding facilities on the moon, or which countries we need to “glass.” It’s gotten so ugly.
I know this seems mean, too. Coming out of nowhere. Strawmanning or whatever you call it. Literally inventing a guy to get mad at. I’m aware of the irony. I don’t know. I guess it’s another kind of mourning, and I don’t know how to express it. I thought tech people were supposed to be like Woz.
Isn’t it deeply exciting that the proofs look so alien?
Isn’t that in itself the new frontier for human understanding?
I would have thought that mathematicians would have found alien forms of maths to be astonishingly compelling, must learn, must know, must understand.
Instead there seems to be a sort of shoulder shrugging lament “we don’t understand it”.
I would barely be able to sleep with excitement at the potential for where this could lead. Like finding an ancient tomb full of answers to our deepest questions along with the promise of more.
How deep does maths go? Is it “solved” or have we just been given tools to delve ever deeper?
Where is their curiosity - don’t the want to find the bottom?
The interesting answer is we don't know. Many of the math problems that make our computers efficient were developed hundreds of years ago with nothing we could do with them.
It is mostly impossible to figure out what we find when we solve these problems. But occasionally they'll lead to real world efficiency in algorithms we use and it changes the world.
I have little sympathy, since I think it is a result of many decades of complacency when it comes to thinking about why want to people do what they do and instead just enjoying being the ones with the results and money. People who love the process and see the human as inherently valuable won't have a problem. I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
> I have little sympathy, since I think it is a result of many decades of complacency when it comes to thinking about why want to people do what they do and instead just enjoying being the ones with the results and money.
It's not clear from your comment: Are you talking about math professors?
If so, what money? They make a living, but it's not that great. I just checked the salary of two associate professors at my university: $80-85K.
And keep in mind they often start their careers quite late. Often multiple postdocs after PhD. Starting a tenure track position in your mid 30's is not common.
> I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
The whole point of the article is that the person crying is because he loves the process and is worried the human connection will be gone.
This is exactly my perspective as well. Seeing humans as inherently valuable is the key. It's shocking to me that so many people seem to have been reducing humans to the results they are able to produce and now that the results appear to be drying up, their perception of their own human value is going down with it.
It’s not necessarily their fault. Our economic system puts zero intrinsic value in humans, it only puts value on their output. It’s hard not to feel this type of way when it’s the only system you know.
I’m fully STEM: software developer with a physics PhD with a mechanical engineer wife, but this is certainly just deserts for the STEM people, who mocked the humanities and especially philosophy for centuries (literally, this started back with Kant even) for spending time thinking about these questions. STEM people are not at all spiritually prepared for it, and this is the main disappointment for me. I spent many years struggling with publish or perish in physics but all those around me claimed they simply loved doing physics and that that’s what mattered. Now as soon as their exclusive ability to give results is slightly threatened the love of physics/maths/coding is no longer spoken about. I get maybe they weren’t the same people, but I thought more people loved the process than it seems today. If anything these new developments make it even clearer for me what I love about this kind of work. I’m so happy it has made me not have to defend based on results, a razor sharp contrast that makes it so clear really why humans do things
it is a little ironic that after the "creative destruction" of industry after industry all of a sudden now it's unfair and we weep. I say this as a software dev with his neck on the line like everyon else.
Anyone who would replace me with a robot never loved me anyway. If that happened I would go and seek out the humans who really do care, and take solace in the fact that Christ will always love me
Handle what situation? Why would people who love me ever replace me with a robot? It’s a random posed situation that no one discussed and doesn’t make any sense lol
Of course the other Abundance guy joins in on the AI Inevitability Soothsaying.[1] It really is the perfect vector for the AI sirening. You just take the inevitability and awesomeness as a given. That’s the real point. But you write about the “grief” and the loss of “identity” and psychology—oh my the feelings. See, we are left-liberals and we care so much about people’s feelings. All the investor class and their hype is perfectly on track is about to fuck us the rest of us. But the feelings about people can’t “understand” and “discover” is what matters.
> gatekeepers bemoaning their gates being blasted off the hinges.
i think that's 85% of the response from software devs if not more. All of the devs I know who actually like to build have embraced AI and the new capabilities that come with it. The ones that guarded their little corner of code proclaiming only they, and their inner circle, have the power to comprehend it are the ones crying in the corner.
Getting those fields wiped just shows how unnecessary those fields where in the first place. I'm a programmer. I am the adapter between a stakeholder and a machine. I am just needed because the stakeholder lacks the knowledge to tell the machine what he wants. If he can use this new fancy adapter called Agentic AI now I happily quit my job and go on finding something that makes more sense in the first place.
The same goes for math.
Let's find some things that bring value other than just the price of a stock.
Broken take, with out this trail of mathematics that lead us here we'd have never reached this place. You think multi dimensional matrix math comes natively to humans?
At worst its time is past, but even that is unlikely.
I think beyond the specific elements of any domain AI eclipses humans in, overall, the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight. You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor? All it tells us is we will be bargaining with superintelligence with nothing to stand on, only a trust that the 5000 lab executives and researchers solved the most important research problem of all time on the first try.
And remember, these AI models were trained on your work, your culmination of effort and sacrifice, without any consent, credit, or compensation, then sold back to you for a subscription fee.
That’s what standing on the shoulders of giants refers to. This is how science has always worked. The generated papers have attributions and citations just like if a human wrote them.
>That’s what standing on the shoulders of giants refers to. This is how science has always worked.
Yes, but automating and commercializing a process makes it inherently different. Just because everyone has always been welcome to drink from the village well, that countless generations have worked tirelessly to dig, maintain, and improve, doesn't mean that there's no basis for complaint when Nestle sticks a pump down it and starts draining the aquifer to bottle and sell.
The lack of an existing law (or custom) against new behavior doesn't actually render it necessarily non-grotesque.
But Nestle’s pump not only doesn’t deplete the well, it’s adding to it!
No, it's able to reach deeper and pump more out than ever before, democratizing access to the water from the well, at the minor cost of the people of the village no longer having a well. But they're welcome to buy plastic bottles of their own water along with everyone else.
Is this actually how water pumping works, or are we extending the metaphor to imply that the AI is creating knowledge? Because I would dispute the latter if the effect is that the number of people with a deep understanding of a subject declines.
> That’s what standing on the shoulders of giants refers to. This is how science has always worked
OpenAI's Navier-Stokes finding is closer to Crick and Watson in respect of Franklin's research than conventional science.
The giants are quite often credited by name, or have their work cited.
Or are revered teachers that you'll never forget.
Take a philosophy class or read more books. Your response is devoid of humanity and philosophical introspection.
Pretty sure AI is more akin to standing on the shoulders of giants and peeing into the wind
Many organizations are giving away capable models for free.
So.. they’re being trained on all of the information that some used to make their living with by writing books, presenting seminars, etc. on a topic.. without paying for it, and then giving it all away for free?
Idk if that’s much better
For now. Once people are hooked, and/or once some of that investment capital starts to be expected to generate some dividends (see https://isaiprofitable.com/), free use will be very limited.
It's good enough that you could go off-the-grid, to a cabin in the woods like the Unabomber, and use it for some tasks.
> You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
I recently thought about this and fear that AI may further drive us humans apart.
Instead of working with humans, I increasingly work with a fleet of agents.
I would recommend the work of Sherry Turkle, specifically her new book "Artifical Intimacy" - she is quite opinionated on this, in the sense that the role that AI play as a conversation partner is so frictionless, so controlled, that we get none of the rough edges of human interaction that form us into a functional social being. No real quick solution outside of deep, fearful caution when using these things - and we're not going to see that from John Q Public
> the role that AI play as a conversation partner is so frictionless, so controlled, that we get none of the rough edges of human interaction that form us into a functional social being.
I wonder if there's a tangent with social media here, at least some of it, where in my experience "a perfect life lived by influencers" is portrayed to us.
IMO, I think that frictionless experience comes at us from a bunch of angles - finding yourself in an echo chamber (by choice or not) online means that you may find yourself less comfortable talking with people you don't agree with, things like that. I grew up in a small town pre-social media, and that meant I didn't have much of a choice about who I was surrounded by. If I was growing up today, what would stop me from retreating to a comfy social media-mediated environment without any challenge or friction? If I was growing up 5 years from now, what would stop me from retreating even further to the comfort of my LLM "friends" who can be shaped to me perfectly? That world sounds terrifying to me.
Human meaning arises in conversation between humans. In them reaching a shared understanding of the world and their perspective upon it.
When you talk with machines, you "outsource" that central human society-building element.
But machines never socialize back in any meaningful way. You gradually lose yourself.
Machines could socialize back in a (more) meaningful way. But to do so, they would have to not lose context (or at least keep some).
I have not totally abandoned the hope that a machine could be someone. LLMs aren't that, though.
[Edit to reply to Loquebantur, because I'm rate limited ATM:] You seem to have read this post as saying that I want that because I find human relationships inadequate. I never said that, nor meant it. My point was a more technical one: Throwing away context at the end of each conversation means that you can't form a meaningful social connection with an LLM, because you're starting from zero each time. (It's kind of like "50 First Dates".)
I'm not pining for a social connection with a machine. But if a machine is ever truly capable of that, I hope I will interact with it as someone, not just as a machine.
> they would have to not lose context (or at least keep some).
I guess you can start to see that happen (for a while now) with e.g., ChatGPT, Claude, etc. keeping persistent memory of you, automatically.
Personally, I often find these memorization features annoying, because - more often than not - the persisted context is outdated because something in my real life changed that ChatGPT & co are not aware of.
In some very very distant future we might someday have your typical sci-fi Data/R2-D2/Johnny 5/EDI style AI and yeah, at that point it'd be weird to not treat them as people, give them rights, etc. It'll take a lot more than solving the context problem to pull that off, and LLMs will never be that, but I agree whatever that technology ends up being it'd pretty much have to be able to hold onto memories. I doubt we'll ever see it in our lifetime, but who knows.
What an absurd premise. And not because it was "impossible": rather, why would you want that to begin with?
There are billions of real-life humans already there with you. Why do you need different ones exactly?
I presume, you don't like those already around. Fair point, they have many flaws for sure.
But that's the reason you interact with them in the first place? To learn and make not only other's flaws go away, but also your own?
> why would you want that to begin with?
I mean, why wouldn't you? Sci-fi AI robot people could be very cool, be capable of things we aren't, have a unique perspectives as a non-human with similar (if not superior) intelligence, have extremely long (potentially indefinite) lifespans, and provide us with insights into our own nature.
Wouldn't you want to interact with them?
Not the parent, but sorry, for me, no. And this is a perspective that I don’t think I will ever understand.
This idea that we are on the path to creating a new synthetic intelligence that surpasses our own with the goal of, what, speaking to it? Cooperating with it? Evolving with it? To me, this sort of goal is an affront to humanity and human self-governance and I’ll probably never be able to see it as anything else.
Losing context is only part of the issue. LLMs don't have the rich connection to the world that humans have. The only connection LLMs have with anything is streams of text. Not only don't they have eyes, ears, smell, taste, touch--they don't have bodies. They don't have to deal with the real, physical world and have it push back on them.
That is so utterly different from what we humans are used to that we can't even comprehend it, and we anthropomorphize LLMs because we can't make any sense of them otherwise. We can't comprehend that these things that emit text that looks like human text don't have all of the other things that lie underneath our words when we humans talk.
Horrifying.
> Instead of working with humans, I increasingly work with a fleet of agents.
This is why I see myself leaving dev within a year. Remote working is enough of a embuggererance for my mental health and I've be unhappy in work much of the time since 2020 largely due to that, AI will finish me off.
Career suicide and going of to wait tables, or be a hospital porter, or anything else with people even if I'm going to hate those people, with the massive pay cut that entails, will be better than actual suicide after a couple more years trying to stick at it.
There came a point a few months ago where I realised that I had to either accept becoming obscelete or change my while outlook on using AI and trying to catch up. I didn't choose the AI option.
Think carefully. I left high school teaching to become a dev. I would never go back. Yeah, social isolation sucks. But doing a job where your students/clients/patients think you're nothing but a waste of their time is soul-crushing on another level.
It is frightening that it took only a few comments from the top of the page to the first mention of a suicide and also that it doesn't look out of place at all.
> Instead of working with humans, I increasingly work with a fleet of agents.
I do too, but it means I have more time in my life to spend with people.
Seriously I don't get what everyone else is doing where they say things are getting worse somehow.
There's days I no longer need to sit at my computer: Claude Remote prompts my phone when it needs direction, the hardware stays busy and I can just go out with my family instead.
It's more important now than ever to hold onto the fact that human beings have inherent value. This exists independent of their ability to generate wealth for an employer. I know HN loves a technical solution for every problem, and so do I, but this is a political problem requiring political solutions.
> human beings have inherent value
I wish, then I'd get a paycheck without having to do any work.
No, you don't work w/ a fleet of agents, you use them to do your work.
This is a general issue that is being forced on us by AI companies; AI is not a human being, it is a tool that we can use to do things. We don't have them as coworkers or subordinates, they're objects specialized in language processing. Thinking otherwise leads people to all kinds of existential and psychological issues that in the end make no sense.
No. If these tools could do your work, people would feel existential dread anyway. It doesn’t matter at all if they are coworkers or tools (I really don’t see why would it ever matter).
Personally I don’t care about that all that much. I don’t have any “sacred” inherent value for human beings. If they can make robots covered in real human skin, blood vessels and muscles (lab grown over the robot frame) with an LLM brain at some point in the future, I would definitely prefer that to a human lover.
> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
Not for those without status (which is most people on this planet). But yes, AI is set to wipe out most established hierarchies and their gatekeepers, and to equalize most fields. A profound sense of dread is something most ordinary people live with throughout life, and if AI can reduce systemic unfairness, I bet most people will actually find some degree of relief from that dread. The complacency of the formerly-statused being shocked along the way isn't much of an argument against that equalization.
> Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight.
This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it. So why fight for us all to have to keep working just to live? Isn't a post-scarcity world the thing we should all be fighting for?
>This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it.
This is what you don't get. _I_ already know it's not worth it. I've always known. It's everyone else! It's a wife who cares whether or not her husband has a good career. It's the police who care if you look like you're poor. It's the way every single person in society obsessively cares about the status of others.
What in the world makes you think I've had to build up status for my own sake? It's absolutely baffling that anyone could be confused about this.
It's baffling to people that don't care that much about what other people think.
>> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.
> Not for those without status (which is most people on this planet)
Look at how coal miners react to the idea of getting rid of coal as an energy source; whether or not you think they have prestige or status, there's certainly an amount of dread there that I'd expect to somewhat resemble the type we're talking about here.
>A profound sense of dread is something most ordinary people live with throughout life, and if AI can reduce systemic unfairness,
I'll keep that in mind. If I'm suffering more than you are, the solution is to bring you down to my level.
A fallacious argument. There's no goal of inflicting suffering on other people; you can equally look at it as enabling people who were previously shut out of something to participate in it, or if not to participate in it directly to reap the benefits by having an agent exploit automation on their behalf.
Not only is that not a real solution to unfairness (it's certainly not fair for the person who now has to suffer more than they were), it also ignores the far better option which would be to reduce your suffering so that it isn't disproportionate to everyone else's. You might be holding onto to some self-destructive fearful crab-bucket mentality if that's where your mind instantly takes you.
I believe you're responding to sarcasm.
AI will bring on the biggest gatekeepers of them all: the ones that own the compute.
Unfortunately, I think this is a human failing being exposed rather than anything else. We like to build conceptual models of how things work, but they are often tinged with should. Reality doesn't care how things should work. It only cares about how things do work. It's always jarring when the should turns out to be false.
Philosophers for millenia believed that human beings had meaning, purpose and set out to explore what that could be. It was a noble profession for a long time. Once we realized that there was no hidden spark, ergo no purpose, the study of and the prestige associated with philosophy took a nosedive. Yes it is still studied but it carries nowhere near the respect it once did. Mathematics seems to be suffering the same fate.
It's more or less reverse to the mean. The period in which a people had both access to education and leverage through their labor and intellect is super short. For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.
"The arc of the moral universe is long but it reverts to the mean" is somehow less inspiring.
>For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.
I feel like we should be doing everything we can to not revert back to that way of life.
No man wants to do the things that need to be done, maybe it's the birth control in the water
I'm too young to be financially secure given my impoverished background, not having parents in my life, being homeless as a teenager, etc. But too old and physically disabled to change course from tech. Nothing to do but keep walking through this tunnel and see where it ends up. It's scary!
same here, and it often keeps me up at night. for a few years i really felt like i had finally managed to get on track towards escaping the poverty i was born into, now i'm not so sure anymore in the long term. the good thing is at least i know i somehow managed to claw my way upwards from rock bottom to a tech job once already, so whatever the future might throw at me i'll probably be able to somehow deal with that too.
Disagree. Scribes, doctors, bankers, and government clerks always existed, even if the education to do it was expensive and exclusive. For the first time in history, this labor, including its knowledge content might become automated and commoditized.
I think the fix is to get started on post scarcity economics now. Sooner or later we're going to have to come up with practices that boost or limit endeavors based on how people feel about them, rather than based on how much money somebody has.
It doesn't have to be ((post scarcity) economics), it can be (post (scarcity economics)). Both systems can coexist for a while, one providing a bit of a safety net while the other falls apart.
There will still be scarcity, just of different things.
Even if we vastly increase productivity beyond our current wildest dreams, the production system takes time to adjust to changes in demand, creating continuous points of scarcity.
But Georgist economics are way ahead of how to deal with this. Tax the ownership/control of things that can not be increased (e.g. land).
Sure, but that's no reason to continue using control over scarce things as a proxy for consent. It's not exactly working out.
We had to tolerate gold because it implied a game that was facilitated by the laws of physics, and we didn't have the resources to set up any other sort. But now we do, the rules can be whatever we want them to be.
Sure, let's try Georgism, but given the historical track record of that kind of reform, let's work on other things in parallel.
agree. wealth inequality has only widened despite AI. things like healthcare insurance, tuition, rent and homes less affordable or more expensive than ever. Nothing has changed and probably ever will. Ai creates abundance in some things like knowledge but other tangible scarcities and individual differences of status remain.
Counter point: AI is trained on output from humans. We all made AI, together. It belongs to all of us. We use it to benefit us, and we celebrate our wins together.
"It belongs to all of us"
For the low entry fee of ~$200 a month you too can escape the permanent underclass - which is more than some countries average citizen makes in an entire year.
Counterpoint: They request a subscription fee they refuse to reciprocate and will not take "No" for an answer.
Countrpoint: Their moat doesn't appear to be defensible, we can just replace them with somebody who will.
Counterpoint: Their moat is funding secured and contracts locking in the purchase of currently forecasted chip production. Even if you assume a catastrophic withdrawal of invested capital from those vehicles on a coordinated revocation of faith by the investing community; who do you plan on having operate this theoretical squeaky clean, ethically training data sourcing company? And for that matter; what makes you think you'll get passed the greed that is already sunk cost into the current gen of shysters?
This entire evolution is a generational project hoping to paradigm shift away from their predecessor's paradigm while ignoring the lessons of their predecessor's predecessors on the nature of greed, the corrupting influence of power, and the consequences of blatant cultural/ethical unhitching amplified by the reach amplification of technology?
The Stasi, or a perfect Jim Crowe couldn't be realized with manpower alone. With computers, and a small cadre of the unethical willing to succumb to the promise of lucre on the other hand...
We'll be free of cancer, and dead on the inside.
You are made one step closer to complete worthlessness
Nobody's worth is tied to source code output.
Great, I'll just eat good intentions, wear a smile, and live under happy thoughts. I'm 50, so I'm "cooked" as far as contemporary society is concerned.
Now now, there are plenty of jobs for you to do. You'll just ve significantly poorer with no hope of social mobility for the rest of your life.
It'll be fine. You voted for strong labor protections, universal healthcare and a welfare system right?
The only thing constant in life is change.
It would be nice if we weren't actively investing trillions in change for the worse (for most of humanity).
Change is the only constant
The more things change, the more they stay the same
What a facile statement. Yes things change in life, but as society we try to avoid changes that rapidly render people and their lives worthless.
When did we last try to avoid such a thing? The way I remember history we celebrate this whenever it happens. Like how the electronic computer made the job of human computers obsolete.
Every society, except the USA
Incessant shit.
It's mainly about status. No one cares that much about AI automating drug discovery or coding, but it's bad when it assists in math. Math results are attached to individuals , whereas for cancer drugs or other medical discoveries, it's more of a team effort by a lab. AI now has the effect of turning math into something that is more lab-like and less about the individual.
Yes, and I think over time there will be a blatant inequality in who is commoditized and who isn’t.
If AI engineers didn’t create the data to produce the models, and they increasingly don’t write the code to produce the models either, then why are they treated as the agents of change rather than the subjects of change, like mathematicians are?
When I was a kid I built my entire identity around being #1. I was the most talented kid in school, anything endangering that resulted in an instant identity crisis.
Then I went to college and I understood that there are people who, for one reason or another, are thinking in ways so advanced I'll never catch up. I'll never be the #1. I'm actually much closer to being average than I thought.
But this doesn't mean I should just give up, lie down in my bed and cry. Since then I've learned a lot of skills - most of them being crap at best, but that's not the point. What matters is my own subjective experience. The fact that despite objectively not being good, I see myself as a skilled person with rich life.
Once I was watching a YouTube video about elderly care and there was a phrase along "These people won't achieve anything more in their lives, my goal is to keep them occupied until they die" which is a bit morbid at first glance, but it actually makes so much sense as a life philosophy.
You know we (tech as an industry) have done this to a bunch of people already, right?
Thats what "disrupt" meant
I would answer to this that the mathematicians (humans) are the ones who give any meaning to mathematical discoveries. If we don't have mathematicians anymore, the result is that AI can find whatever they want, but 99% of that will be useless waste. Outside of practical applications, the only reason most mathematical results have any interest is that lots of mathematicians spend time trying to find their solution. Once humans stop caring, mathematics done in an automated fashion will also stop having any value, other than its possible use in finding solutions to existing engineering problems.
> bargaining with superintelligence
This part is nonsense, still. There's no known AI -> AGI path, and superintelligence doesn't mean anything, and AGI is assumed to be (going to be) both malicious and a slave, both for no good reason.
It has already been happening for decades if not centuries. Instead of bargaining with superintelligence though, it's bargaining with moronic systems. I watched an airline gate employee struggle to allow a family onto a flight despite acknowledging that they were confirmed and had seats on the plane because the computer wouldn't beep when they waived their boarding passes. She called her supervisor. Her supervisor resolved it not by writing "it's okay they can get on" into an exceptions field, but by kicking them off the flight, putting them on standby, then accepting their standby request and reseating them (in different seats, not all together, for some reason). The humans involved all knew things were okay, but the system was too dumb to just allow them to go forward.
Superintelligence might even help.
Nah. There is nothing beyond being made redundant in your job and labor leverage. That is all. It has got nothing to do with resentment over fucking machines. It’s all about people (the executives that you mention) fucking us over. The end. Completely human problem.
I know I’m being sold some bullshit when this is made into something inherently existential and psycho-identitarian. An egalitarian future of labor-worthlessness means that there is nothing above us to dictate anything at all—maybe not even nature itself, save for meteors and other cosmic events. I’m not afraid of being “worthless” in that world of complete freedom. Fuck no.
I am afraid of being hungry and homeless long before society reaches "an egalitarian future of labor-worthlessness" that's good for common people.
And that does not disagree with anything I wrote.
> I'm not afraid
So how do you plan to maintain food, clothing and shelter when you have no leverage?
Some have been scurrying to amass a nest egg and become self-sufficient. Of course, if demand destruction from the hollowing out of labor happens fast enough, who will consume and keep these stock prices up? With the tax base simultaneously crashing, gov't can't step in and prop things up. It could help during the transition while failing as a long-term strategy but what else can you do?
“Worthlessness”
Consider this… in the grand scheme of things, we are on a tiny planet in the universe. What cosmic contribution could you make?
Most people won’t make anything even close to “a ding in the universe”, as Steve Jobs has been reported to say.
So they may as well enjoy their life. And your worth can be to your family, to your children, your parents who took care of you.
Learning a hobby. Practicing religion. Mindfulness and meditation. Doing sports and improving your own body.
Look around if you are a heterosexual man, for instance: many women are far more attracted to men that perform in the moment, than men who labor quietly to create a product. The former enables untold hours of honing social skills and calibration, and humor and performance skills. The latter is of more durable value to society, yet relatively few women are genuinely sexually attracted to a man doing the deep work. (By the way, it is also true the other way: most men are not attracted to women for doing deep work or being a rocket scientist.)
Animals don’t do “deep work” and most animals are attracted for far “shallower” reasons. If everyone has a UBI and AI does the “deep work” then that frees up humans to be like animals in a zoo or in the wild… doing all the displays to attract mates, having babies etc. That’s not nothing — in fact it is 99.999% of everything that happened in history and 100% of what happened in prehistory.
You’re just ceding that 0.0001% which represents technological progress. The actual dread comes from the fact that you and all of humanity will be as completely powerless to change things as cats or any random animal species is powerless now. The difference is you will perhaps understand your powerlessness more than they do. But in the good scenario, you’ll be like animals in a zoo. We are constructing this zoo for ourselves now.
We will remain on this blue planet while the AIs will colonize the universe. In fact I wonder why aliens would ever come to Earth, when they’d just send their AIs or robots. Look at this: https://news.ycombinator.com/item?id=46200502
As Ernest Becker would ask, then what is your "immortality project"? Even worse, what's the immortality project for the vast majority of humanity? Do we just bring back religion to Middle Ages levels? Or do we hope everybody goes through some kind of never-before-in-history spiritual awakening?
"If everyone has a UBI" - isn't that quite a big "if"? And also - how soon would it come? Soon enough for families losing their income?
For me the most obvious conclusion is, use your maths skills and get into open source AI.
Make AI as cheap and accessible to as many people as possible and we will likely be much better off.
The more people empowered by the tech the less like you will be “redundant” in the way you describe. You will own the important tools only the techno oligarchy own today…
Would you feel the same if you were a part owner of OpenAI? I bet you’d feel excited about the money you’re about to make ?
isn't math skills really just pattern analysis and creativity? how is this limited to just the math domain? why are mathematicians worried?
> How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor?
I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
And like to anyone else who hints that “automation creates better jobs”, I have to add: but not for us. Maybe for the next or the generation after that.
Industrial and technological progress has birthed better conditions overall, but at a cost of locally worse ones. Mine workers that got displaced didn’t end up being machine operators - most of them landed on benefits or sporadic gig work. Only the next generation was able to be properly trained and educated to be able to benefit from the technological advancement en masse. And we might see that happening again.
so our children and future generations are not "us"?
In the I was referring to people reading my comment as “us”.
While it is nice to think of better futures for coming generations, I think we (reading this) also have to think about our more immediate work, as in next 2-5-10 years.
> I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
I don't buy this argument, much like I don't buy the similar "they said the same when calculators were introduced".
It negates that some changes are qualitatively and scale-wise different. Also, by gradually chipping away at things that humans do, you (we) keep shrinking and shrinking until there's nothing left, every step of the way some people claiming "well, the telegraph operators said the same". Keep in mind that, while it's hyperbole, some AI entrepreneurs are actually saying stuff like "maybe jobs will become a hobby". Unrealistic for now, maybe, but if they do become a hobby, what leverage will we have at all?
In other words, it's true the past is worth keeping in mind, but not to the exclusion of the possibility that this time it's different. Or that the trends of the past actually point to harm in the near future.
> the removal of all human leverage via their labor
I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.
I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.
My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.
I can kind of relate but I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.
Nothing is validated or tested anymore. It’s all about the shape or believability or even how loud it is.
They dropped 100+ papers no one read. I am sure mathematicians are reading them now but even this forum has concluded that these problems are solved, without reading or validating.
It’s like everyone forgot “don’t believe everything you read on the Internet.” And it makes for a very boring discussion when everyone is just spewing unvalidated claims, layering them with strong beliefs founded on similarly unvalidated claims. It’s dull, without substance and just pure noise.
> I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.
This is what I've been feeling too. It feels increasingly like the professional world has become a tower of scams. It's not worth building technical skills or focusing on quality; your boss doesn't know or care what good work looks like, he/she just wants your work to look good enough to his/her boss, who knows even less about the actual work being performed. And everyone reports up to an MBA CEO who was brought in from a completely different industry.
I want to work in an industry where skill, diligence and expertise is rewarded and respected, but from what I'm seeing political ability is 95% of the game and caring too much about the work is a career liability.
+1 We need human verification.
Not sure I agree - they dropped hundreds of papers, most of which were so badly written that they were incomprehensible. If it wasn’t for the lean certificates on 30% of them, the mathematical community would have told them to go and whistle.
I think the consensus here is that most of the papers are effectively correct. But it would take years of study just to understand most of the problem statement - so we have to wait for the mathematical community to digest the borderline-nonsense that OpenAI spewed out.
Most likely that they look really correct and are not and require immense effort to understand, if even possible, to disprove their claims.
What we have is a DoS in research verification.
Just tell the (AI) authors to go back and write clear explanations for why they matter and what the specific claim is.
I gotta say I think mathematicians brought this roblem on themselves, most math papers are absolutely incomprehensible to anyone who is not an expert in the field. Mathematicians spend so much effort trying to make their argument to other mathematicians that they've gradually abandoned making their field or findings comprehensible to anyone else and left that to science journalists. In turn, those journalists churn out very formulaic stories written to a template: 50% the human story of the mathematicians social/academic wanderings in pursuit of their goal, 50% an explainer with some nice graphics on the subfield they operate in and a mention of what sort of problem their insight solves, perhaps with a few colorful metaphors thrown in.
The last mathematical paper I can think of (but I'm not an expert, I'm sure there are some others) that was both mathematically rigorous and accessible enough to be readable was Mitchell Feigenbaum's Quantitative Universality for a Class of Nonlinear Transformations. This is accessible int he same way as a textbook - you have to work at it, but hte author is explaining his thought process and drawing the reader's attention toward things that matter or stand out as unusual.
Including those with Lean verifications?
This piece circles, without naming it, around the concept of “deontic scorekeeping” central to the work of Robert Brandom on semantics. The idea that our ability to create and sustain conceptual systems depends on the give and take of reasons within a social setting. Brandom has been front of mind for me lately because elements of his theory both explain why LLMs should work in principle, and where they are bound to be radically deficient. An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
Worth reading: Robert Brandom, “Making It Explicit: Reasoning, Representing, and Discursive Commitment”
> An LLM ... cannot be a real participant
The participant is not an LLM, the participant is a company. The company learns. It adapts. It has a trillion dollars of skin in the game.
I mean, many companies operate within particular domains but don't have much skin in the domain itself in a way that is meaningful to other participants there -- like Oracle builds software for healthcare, ostensibly, but any actual user of their products will tell you they are nightmarish with regard to usability, and research shows that patients are often harmed as a result. So healthcare isn't their game, it's selling healthcare software.
Likewise OpenAI doesn't have "skin" of any shape or value in e.g. the mathematics community or even in mathematics research per se. In my cynical view, they are exploiting the exposure of mathematical research to automation in an effort to create a case study of their product obsoleting a highly visible and well-leveraged and relatively small profession. Regardless, I doubt even they would say they have durable skin in mathematics in any sense, certainly not in either of the senses I mentioned above. But their output inhabits the same space as mathematicians, who have to contend with the consequences
OpenAI most certainly have a "skin" in math and physics.
They hire math and physics grads and good command of math is very useful in computer science, machine learning and AI.
What qualities do the recruiters at OpenAI value in the mathematics and physics graduates that they hire?
And how do you see these qualities changing over, say, the next decade?
OpenAI is concerned about math and physics insofar as these skillsets enable their current staffing model. That is not exactly durable.
> The company learns. It adapts
And potentially steals [1].
[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...
I don't see any theft in that article. Who was deprived of what property? Why didn't they make backups?
Don't be obtuse, it doesn't make for productive discussion.
> The idea that our ability to create and sustain conceptual systems depends on the give and take of reasons within a social setting.
I read this sentence 5 times and I still cannot parse it.
I think it means if we all have to justify ourselves to each other, we collectively come up with an sustainable ecosystem of ideas that we can understand. So if we stop justifying ourselves to each other, the ecosystem doesn't get built or sustained, and we stop understanding things. Not the person you responded to, but that's my reading.
What a coincidence, I've also picked up inferentialism but never really got into it.
I'm interested in the reversal that Sellars proposes - it seems clear to me that "representation" is not prior to the space of reasons, despite how unintuitive it is.
> An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
I see no reason why this can't be the case in the future, even though I think inferentialist-based theories are generally strong. Can't you force LLM's (or whatever AI architecture will exist in a decade) to stake durable commitments itself, then? They're at least autonomous enough to run computers. I agree that that is a core part to true meaning & autonomy, but nobody just lets them yet. I don't think inferentialism is a consciousness-based argument, either, so that escape hatch isn't available. I also don't think this is a necessary condition to engage in the space of reasons (but maybe then I'm misunderstanding it, or retreating to a more Sellarsian understanding). Could you explain on this more?
Though again, I only have the book sitting on my desk, and I haven't cracked it open...
I actually agree with you. LLMs could become participants in the future, but this would require (1) durability of real awareness over time (an architecture change from the current fixed-weights + transient context architecture) and (2) that humans grant them the status of equal participants (probably not a terribly high bar given how people already treat these things). I'm not aware of any real progress toward (1), and all the stand-ins people have developed live in the context layer or textual artifacts, which are poor stand-ins for durable memory.
And yeah, consciousness based arguments are bunk IMO. It's clear that LLMs are in a basic sense "aware", and even to some extent "sapient" (in Brandom's usage), i.e. they can justify claims and object to things at least locally. I follow Rorty on the question of consciousness, which is to say that consciousness has no deeper meaning in practice beyond "are you awake", and in philosophical discussion it's largely a proxy for "are you one of us" in some amorphous sense.
Well that's too bad, since Rorty completely misses the point on consciousness.
Consciousness, the interesting part anyway, is qualia, or the "it is like something to be me" as Nagel put it.
Intention, as Dennett used it in his book "The Intentional Stance" is the "are you one of us" angle, and is distinct from consciousness because as the book title suggests, it is a stance taken with respect to some other system (person, cat, dog, turtle, LLM) rather than a description of subjective experience.
So I think it is your (and Rorty's) positions that are bunk. It is very, very far from clear that LLM's are in any basic sense "aware" let alone "sapient". They are extraordinarily able to manipulate language in a way that leads to text generation that has traditionally been assumed to indicate consciousness and/or intelligence. But we have no way of knowing if it is like anything to be an LLM, even though most of us are busy taking the Intentional Stance towards them.
I guess I'd say that your qualia aren't a something, but they're not a nothing either. We can't say what they are. They're not publicly describable, and therefore not publicly attributable. As Wittgenstein would say, "They cancel out."
And as I think Dennett himself would say, qualia are just Sellars's Given rearing its ugly head again.
Well, while I love much of Dennett's work, I emailed him after "Consciousness Explained" was published to say that I was disappointed that it appeared to describe what we are conscious of, not how we are conscious. He sent back a short note to say "You're probably right".
This sounds super interesting but 477 pages is a beast. Would you recommend anything about this subject thats a bit more approachable?
Brandom wrote a shorter version of the book, called "Articulating Reasons", which is available in full on his faculty page [1], or from booksellers. He's a very wordy writer (kind of exhausting to read, even for someone who enjoys philosophy), but if you get ahold of Making It Explicit, chapters 2 and 3 have the meat of the ideas I'm talking about here.
[1] https://sites.pitt.edu/~rbrandom/Courses/2022%20Phil%20of%20...
Edit: One of my ways of benchmarking new AI models is actually to type a version of this prompt in and see how sophisticated the response is: "Talk to me about the concept of deontic scorekeeping in the context of llms". You'll get good mileage out of that.
Ok what about memory systems though (basically LLM modifies its own RAG context) - that should allow commits and changing dispositions over time?
RAG context memory and modifying the weights are two different things. As far as I know, models can't modify their weights reliably.
Humans change over time due to what they experience. It’s different from waking up from a ten-year coma and having access to a record of what happened to yourself in an alternative timeline where you didn’t fall into the coma.
AI systems have Reinforcement Learning
I think in future, each company will built its own universe of mathematics consisting of prized discoveries. Maybe one Mathematical Universe per Company in the multiverse of mathematics.
tbf naming something "deontic" is not helpful
The irony of complaining about an academic jargon word in a thread about mathematicians.
Mathematicians always define their terms exactly, in order to begin their discussion. Philosophers never define their terms exactly, because then they would have nothing to discuss.
Deontic - `Relating to or expressing moral duty or obligation`
Deontology - `normative ethical theory that the morality of an action should be based on whether that action itself is right or wrong`
I don't think that's true. Why is it that I see more people using "shaped" and "load bearing"?
Because language is mimetic, and people are consuming huge quantities of AI-generated text. I.e. saying that an LLM can't be an equal participant in our language games isn't saying that it can't play by the rules. In fact that's what LLMs are trained to be best at.
What an LLM can't do (at least with the current architecture) is sustain a commitment that it can be held accountable for later. It doesn't have durable "perspective". It can't update its priors. Maybe a new architecture will come along that gives LLMs memory plasticity, at which point they will shift into being individuated quasi-persons, but that's not what we've got now.
I've written some about this here, if you're interested: https://elliotmilco.substack.com/p/deontic-scorekeeping-comm...
You don't consider recursive self-improvement a step towards 'updating a perspective'?
I think you are also suggesting Mathematics and Language are the same thing.
Recursive self improvement is a bulk process that shapes the weights of the next model. The weights are still frozen. Each individual "agent" would need to able to durably update its own dispositions to get to what I'm talking about.
I'm not suggesting mathematics and language are the same thing, but I am saying (which shouldn't be controversial) that mathematics as a discipline exists within the medium of language, and that mathematical reasoning, discovery, and proof happen via people proposing, justifying, testing, and objecting to formulations in language. This isn't a metaphysical claim about whether math is real or captures inherent structures of the universe etc.
Ok, then what is an agent that provides a 'thinking' or reasoning loop around said model?
You ended your first post with: "An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time."
And so if there is an agent, a thinking loop, that can recursively adjust over time...how does your statement hold up? I'm particularly talking about 'staking durable commitments or changing its dispositions over time'. Recursive self improvement is all about these 2 things, especially when it's the 'outer thinking loop' doing the improvement.
But you're right, Mathematics exists as a medium within Language. But then Language is just a transfer of ideas from one mind to another.
This principle is the heart of what the article is talking about: a transfer of logical mathematical ideas passed from a human to another. And the sense of 'shared understanding' that comes with it. It has nothing to do with keeping score.
It's about getting a 'sense' for why someone thought or rationalized an idea in order to connect some other idea. You get an 'ah-ha' moment, and some of those moments are gone, or a different path is taken to reach the moments now. That is what the 'weeping' is about (not that I agree with the article. I think anyone can spend however they want. Math doesn't have to change, it's that people want it to, apparently, because it's 'easier' to do something different, or for some other reason I don't know about).
Rather the opposite, AIs are arguably deontically asymmetric because they are powerful and virtual digital entities, and thus should be very concerning to the human species. Not because they current have no internal abstract state for learning state like humans and animals.
It would only be true if you consider Mathematics and Language to be the same thing.
Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.
The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.
That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.
And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.
Ideas are contagious and AI is our most powerful vector yet.
I suppose we would see some interaction with AI in that sense, but it'll take place over the course of learn cycles and when AI starts ingesting its own ideas it typically goes wrong somewhere down the line.
I think for me is, that history is not kind to the people who are left behind.
For example, weavers were one of the most powerful professions (if thats a thing) in the middle ages. In england there are the wool churches(not entirely weavers....), in the low countries, the weavers guild fomented revolutions, and shaped the borders of countries.
But that came to a crashing halt over the space of 200 years. slowly at first, then all at once. the power loom, then cheap imports from india turned a cottage industry into dark satanic mills. Yes there are/were specialists that still hang on, but they are like football stars you can probably name all the people who make money, the rest are barely scraping by.
For us, in the programming world, we will not have a 200 year grace period. Yes, AI produces shit code, but thats not the point. its cheaper and there are vanishingly small places were "good" code matters anymore. AI could be used to help a large range of people, provide support, trusted advice.
However because of the people who are funding it, that will not be the case. it will be used to reduce wages. Just like offshoring, resulting in china being the worlds manufacturing hub, AI, and the current people funding it will aim to make the expensive worker redundant.
In short, I weep, because Altman, and the money behind him ans his ilk want me destitute. I have committed the crime of wanting to earn a wage, and thus I must be punished.
It's profit chasing as well as practical - it kneecaps the educated workers increasingly asking questions about the morality of business and technology. It takes away the income that affords them the space to push back, threatens high ed, critical thinking. The salivating over AI makes perfect sense in that context.
So how do we push back?
I think Thompson has no understanding of endeavors like pure math or the theoretical sciences. In the words of Einstein, "I want to know how God created this world. I am not interested in this or that phenomenon, in the spectrum of this or that element. I want to know his thoughts; the rest [is] just details."
I think mathematicians aren't bemoaning their lost sunk costs. They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_. Their voyages of discovery, which give life meaning, may never leave dry dock again.
It is strange, because in a sense, when you start learning math, you never explore, you are being taught. Now mathematicians were able to exercise curiosity. That is going to change. Back to the ranks of eternal learners if you will. The fun was in the research. Now I think it will leave the door open toward more Applied math. What can people do with all the discoveries. The curiosity will be there. I think it is better that way. It will also put a cap on abstractions that overlaps. In essence, AI is less likely to create new theories, coin new terminology, etc. But it is better suited to recognize emerging patterns. That will probably simplify mathematics. Unless the domain of abstraction is infinite and with each new abstraction comes its own results but I doubt it. Who knows though...
> they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_
Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity. Even now, verifying and understanding these AI proofs is a massive endeavor. Coming up with an intuitive (to humans) distillation of the finding remains an open challenging demanding expertise.
i think to certain types of people (who are probably more likely to be mathematicians), the "self" discovery aspect of it is important, they don't want to read about a new proof for a problem they were grappling with, they want to come about it through their own thoughts.
LLMs are not feeding us things, we are asking it and getting answers. This dialectic I think is super powerful and it still requires the human to ask questions and continue expanding intellectual curiosity to go down the rabbit hole and discover. LLMs are just an incredible rubber duck but the discovery is still there.
> This dialectic
We literally had a case where OpenAI may have stolen the dialectic from a mathematician using its product and then given it to a proprietary model burning tens of millions of dollars of tokens [1].
[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...
I think I disagree, if I had a puzzle on my desk and I told an LLM to explain how to solve it, I think I have robbed myself of the enjoyment of solving the puzzle. Is it really any different than googling a solution?
There are different avenues of "solving" something though. Asking an LLM to one shot something vs iterating on it collaboratively are different ways of achieve potentially different ends.
If the result is black and white, then yeah, its probably not super fruitful but if the end result is subjective, then there really no is objective "solve".
> Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity.
I don't think this matters a bit. We individually can't demonstrate "consciousness" in other humans, it's not even a word with a definition. What we can do is miss the feeling of discovering something on one's own rather than having it explained to you over and over again until you understand it.
I mean, dead mathematicians are definitely not "conscious," but their books keep you from discovering things that you may have discovered yourself if left to your own devices.
The joy that was mentioned is the joy of knowing a bit of gossip that no one else knows, of being the first explainer. Interpretation is just being able to pass gossip on; a decidedly lesser joy. And there's no reason that AI shouldn't be able to explain findings in an intuitive way - LLMs at least purely operate based on human intuition as written in human communications and then captured in matrices.
If these proofs are good (doubtful, imo) and can be reduced (intuitively distilled, sorry), there's no reason that an LLM shouldn't be able to do that.
That's such a nice way to put it! I recently stumbled into "The Last First- Winter K2"[1] and saw the same spirit along with a similar misguided effort to win without the will, nor capacity to do it — e.g. influencers taking a stab at K2 is like prompting your way to results.
1: https://en.wikipedia.org/wiki/The_Last_First:_Winter_K2
Why is not like changing to a different type of vessel for the voyage? Unless and until it is all AI in closed system, could direct machines where to look etc.
Climbing a mountain is different from flying a helicopter to the top
True, but then the moonlanding wasn't possible on foot.
Only the journey is different.
As they say, the journey IS the destination ...
And the sense of achievement that comes with it..
I'll say it again: AI is capable of answering the questions put to it but AI is not yet capable of knowing which questions are worth asking. Mathematicians have no cause to weep, they will be fine.
Most of these questions Mathematicians posed to each other have no practical application, and probably will never have a practical application. (ex: the solution to the Navier Stokes question doesn't change anything at all.)
In many ways, the teleporting machine that teleports to the solution is against the point of the questions. No one wants the solution to be "Run $10,000,000 worth of tokens to cover all possible Lean formulations of the problem".
I think for Mathematicians, I'd say that the human level formulation is still possible and with a new perspective. We as a society (and outside of Mathematics) need to also understand that this AI teleport to the answer thing has less value than we all expected. The value is in the journey.
The open question is if people will still want to take the journey if the teleport machine exists. Or if the existence of the teleport machine halts exploration.
> The story of human discovery and the triumph of the human spirit will soon be excised from this discipline. The Dinitz-Garg-Goemans counterexample was the most egregious demonstration. It revealed that the process of prompting novel proofs will be as auraless as ordering doordash. Watch as magic and mystery evaporate. Watch as the sun sets on our heroic age. Is there not something evil in the act of blocking all future generations of mathematicians from the experience of discovery? Forget about accuracy or even attribution. Something fundamental to the experience of mathematics is being taken.
> You are a helpless onlooker. Before you a channel through which humans have accessed the ineffable and sacred for thousands of years is being sealed for eternity.
https://kirwinhampshire.substack.com/p/the-dark-night-of-mat...
As someone with friends in the mathematics community, they've expressed similar sentiments.
I have to wonder if this is the equivalent of teleporting back in time and finding Marco Polo or any of those early explorers who traveled for the love of the game (and fame and money), and opening up Google Maps on your phone and showing them the zoomed out world. And showing them any island or village they could name.
This is absurd and silly. Discovery doesn't go away because more new things are being discovered. This position only makes sense if you make an arbitrary cut point where some instances "don't count". If you were to go back to scientists or mathematicians several hundred years ago and tell them that one day science would be done by tens of millions of full time researchers around the globe, often working in teams, some of them might also bemoan the loss of the "hero scientist" as any commoner could now participate. But science and math have only benefited from this, and this has increased, not decreased, the beauty and wonder in these fields.
It is not the same because the act of problem solving for discovery is now to be done exclusively by machines, as humans are no longer remotely competitive.
If you read the linked article, the author goes into more detail about how doing mathematics also has been an act of communication with past and present human mathematicians. This is very much true when math and science are done by millions of human researchers, but not so when it is done by massive swarms of agents at a couple labs.
I am sure science and math will benefit in the sense of solving more problems than ever before. But you are not really understanding the agony of the humans caught in the change.
> They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance...
Not "never again", it would be extremely difficult but science and math are also built on random, trial and error, and intuitions that could not arise from AI.
That's the issue - trial and error. A human can try a couple ideas in a day, but realistically, there are not that many sane ideas.
Computer can iterate without frustration on a completely random garbage. Once in a while it stumbles upon a gem. Even if it's one in 10000, that's still hundreds of good ideas in a day (provided enough GPUs).
Your comment assumes math and science are some kind of closed systems that can mechanistically pursue any endeavour but reality (whatever it is (e.g. the universe)) is much more complex than that, and there are realities that are outside of those systems. Not saying that humans can see all those realities though.
Maybe? I'm targeting math, which doesn't need any kind of reality check. It has axioms though, so in a sense it's closed.
You can generate completely random ideas (a couple proofs I saw where you define some weird set, pick an element and derive a contradiction when the set is empty or whatever).
Some of these will reach a conclusion: proof or a counterexample. And that's how speed can outperform creativity, if the speed bump is fast enough. It's the same as with those monkeys and Shakespeare.
> “Google Translate is amazing, but it has taught us essentially nothing about linguistics,” Strogatz said. “We don’t understand why it works. It just works. That’s the Twilight Zone. It is knowledge without understanding.”
This is not true, and it is the crux of an enormous debate we're now having across many, many fields.
Classical Chomskyan syntax (which models sentences as parse trees) has been enormously, astonishingly predictive of the rich structure in real-world human language. But we are quick to forget that we invented concepts like "words" and "noun phrases" specifically to help us analyze and predict things about human language. What LLMs and modern MT shows, definitively, is that there is another, totally separate way to describe these phenomena, which is both vastly less descriptive, and also vastly more predictive.
That these appear to trade-off is unfortunate, but it is NOT the case that we "don't understand why it works." We absolutely understand why it works, if we didn't understand we would not be able to improve these systems. What we do not have is a clean, descriptive theory that boils this down to something that is formally specifiable and analyzable. But those are different things, and this is the root of the disconnect between many communities working in and around these spaces (linguists vs computational linguists; mathematicians vs computational mathematicians; mech interp people vs empiricists).
"We absolutely understand why it works, if we didn't understand we would not be able to improve these systems" - Understanding how to improve something does not require understanding how it works.
Growing a plant does not imply understanding soil minerals or photosynthesis. Training a dog does not imply understanding it's brain. Using LLMs to improve a codebase does not imply understanding C++. Improving a PC with new parts, does not imply understanding RAM.
You can understand abstractions about the system, without understanding the system. Without gradient descent or other forms of automatic optimisation, do you think humans could build something as capable as GPT-1?
Not sure if this will bake your noodle or not, but yes this is basically my point. Saying I understand how dog training works has very little overlap with saying I understand how the dog brain proteome works, even though they're related, and even though I might (or might not!) understand better how to train my dog for having fully understood the proteome.
Saying we understand training and deploying machine translation systems that have strong predictive (rather than descriptive) power, apparently has very little overlap with defining a formal, analyzable, mathematical model of language. Mathematician who refer to "understanding" generally mean the second thing, but the first thing is also a kind of understanding. Just messier.
I think they're referring to the fact that neural networks are more or less black boxes. Their internals are blurry and not explainable in the same way that traditional linguistics tries to explain things - with rules.
Of course, ultimately that's because language itself doesn't follow simple rules like linguists would prefer. They're more like guidelines.
Yes, that is one of the points I am making. Chomskyan syntax is a strongly descriptive theory that is entirely ex post. MT and LLM systems are strongly predictive and have no descriptive theory. It is completely wrong to say the first is "understanding" and the second is not. They are both different types of understanding. In particular, Chomskyan theory clearly does not represent enough "understanding" to predict real-world English sentences with similar lucidity to that of LLMs.
I still think mathematicians are going to be as important as ever. The LLMs are only finding what was technically possible given our current breadth of knowledge after brute force solving problems. Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.
Finding that new knowledge may be exponentially more difficult, but it will be much more valuable.
> Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.
How do you know this?
AIs cannot invent things like cubism, surrealism or hip-hop.
Or maybe it can; the early image generators famously did output horrific images sometimes because of their limitations, but these outputs qualify nonetheless creations.
Now the interesting thing is that it is humans who evaluate them as horrific, worthless or interesting.
Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
> Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
Online, how would they even know? Offline -- either we give up on the internet, or the offline discussion will be heavily influenced by what AI said online.
I don't see how its output possibilities could be infinite given a finite set of data. Taken to the extreme if you give an LLM only a single piece of information then that is going to be the only thing it can produce. Given it two pieces of information and maybe it can produce every combination of those two things. Given all of the worlds knowledge I'm sure its possible outputs are incomprehensibly large, but still not infinite. Then having its outputs match a set of really specific instructions is going to be a small subset of that.
How is this different from human brains being made up of a finite number of neurons in a finite number of configurations? I know very little about LLMs and nothing about neurology and maybe you can explain to me why one is so much more powerful than the other.
> a finite number of neurons in a finite number of configurations
Neurons and synapses change, especially synapses (the connections between neurons). Some neurons have a few connections to other neurons. Others have hundreds of thousands of connections to other neurons. These grow and change. "Estimates are between 100 trillion and 1,000 trillion neural connections" Is it finite? Depends on your definition of finite. If anything countable is finite, then it is finite but the number may be practically uncountable. Is it fixed? No.
As I understand it, the advantages of AI have not just been speed of processing.
Humans have cognitive limits. Humans hold a specific slice of all knowledge at a time and necessarily limits what correlations can be leveraged. This is similar to "context windows" in modern AI. Human minds are wildly variable in efficiency. AI is has constant inefficiency, largely controlled by the speed of light, physical RAM, and processing power, with superior (no cost) "thought" context switching. The AI also has superior accuracy without the need for "concentration" because the AI is processing on a Turing machine with the limited operations that it's superior at in terms of time and energy - eg find an element in a list of 1 million elements.
Sure, a given LLM can only produce so much (though it might be a lot), and then you'll need fresh inputs. But you contend that those have to be human, while I am arguing that they could be
- random,
- from the environment,
- from other AIs.
That increases the space considerably.
In particular, note that your argument also applies to humans: A human, being finite, can only produce finitely many things. But you add other humans, the environment, and pure chance to the mix, and it'll be pretty infinite for all practical purposes.
model collapse seems to say that "AI" cannot train on itself.
I would be curious to see the end result of letting it endlessly update its inputs from its own outputs. I was under the impression that eventually this would cause delusions and if you let it do this long enough and it is solving problems that no humans can verify as correct that it isn't useful.
Maybe that is the limits of my imagination, but it seems unwise to take humans completely out of the loop. Having mathematicians around that can understand what it is doing so that we can utilize that knowledge in a way that we can have some confidence in seems important.
Well humans do that and you expect them to continue to do that - why can't AI models do it as well?
Typical thought experiment - go back to the early years of humanity, even before we've created language itself. Communication is basic grunts and other such things, cutting edge knowledge is 'poke him with the pointy end'. Compile all of this knowledge into the training set for an LLM, let it run with infinite processing power and infinite time. It'll just continually remix the knowledge of that snapshot of time. You're certainly not getting to the Moon, discovering the secrets of the atom, or even discovering mathematics.
It sounds increasingly dumb to bring this up given how advanced models have become, but it remains true - they are still fundamentally token prediction algorithms. And that imposes hard constraints on their potential. These constraints are relatively academic in our current world because there's just so absurdly much information and no human can hold more than a negligible amount in his mind at once. So they'll certainly be able to connect many 'obvious' dots that we don't see simply for a lack of carrying capacity, but they're not going to discover concepts that do not clearly follow from the comprehensive knowledge of an era.
Perhaps they could some day. I'm basing my understanding on the current day capabilities. Once that changes I will update what I think.
It seems far more likely to me that current LLMs have learned generic reasoning capabilities which will allow them to solve effectively infinite problems, rather than whatever vague limit on capabilities you seem to think there are.
There are two reasons:
1. LLMs aren't run deterministically - they use a bit of noise to improve the output. So even with the exact same input you're going to get a whole distribution of outputs.
2. Feedback from the environment. LLMs (and humans!) work best when they can test ideas. That's why coding agents work so well. If you ask an agent 1000 times to just think of a compression algorithm, it'll probably spit out very samey answers just like a human would. If you give it a test corpus and let it try out different things it will absolutely be able to come up with novel ideas.
The set of data will increase as it outputs
I don't think AI will continue growing infinitely either, but maybe "incomprehensibly large" is large enough to solve all the math problems.
I liked reading an essay by mathematician Nestor Guillen that develops thoughts along these lines - the "convex hull of ideas".
https://birdsnfrogs.github.io/2026/09/12/Felix_qui_potuit_re...
Yeah, I think that's a great essay and it really helped clarify some of my own ideas.
At the same time, I think it's pretty amazing how this is the fundamental question of AI's trajectory, yet people on both sides seem so sure of their viewpoint with little actual evidence or data to back it up.
That is, it's possible that LLM-based approaches will be forever limited to this "convex hull" of current training data, and thus humans will always have a role to play (and will retain primacy when it comes to overall intelligence), because it appears that humans are not limited to this convex hull based on previous discoveries (e.g. I think basically every scientist would argue Einstein's general relativity was outside the convex hull that existed at that time).
Yet, if LLM-based approaches are eventually able to surpass humans on this "novel discovery" axis, then we as a species are basically done for when it comes to intellectual primacy - and then it becomes a very strange and scary world. And many AI researchers at the top companies clearly believe current approaches will scale and eventually (pretty soon I'll add) lead us to RSI.
I have no idea which side is right, and I'll admit I really hope LLMs eventually hit a wall. But again, I just am surprised that many seem to be putting forth their beliefs, but I would love to know if there is any good evidence one way or the other (or better yet, a more concrete and verifiable definition of the "convex hull" so that researchers could provide some falsifiable examples).
Came here to say this. What AI is doing with math is essentially fuzzing, but looking for valid inputs that solve a problem and the inputs are mathematical abstractions. It's a cool approach, but AI doesn't understand what's going on, and without understanding it won't be able to create the next generation of mathematical abstractions needed for the next generation of proofs.
this is assuming that humans can be original in ways llms won't be, which I doubt will hold true
> which I doubt will hold true
Why?
aren't mathematicians needed to create the problems? Everything solved so far is a question posed by a human. Which is probably why cracking diseases is more difficult (apart from man made viruses :/)
Heard from one mathematician recently - math is not about problem solving, but about the artifacts that are produced in the process. The new Navier-Stokes solution for singularity isn't worth a million, it supposed to generate new mathematical methodologies, tooling and lead to additional discoveries, like the graph theory.
Oxytocin is one of the oldest molecules in the world — it created sexual reproduction, a very long time ago. "Our capacity to relate and to care for one another" looks a lot like oxytocin, which therefore belongs to all of us who reproduce sexually. There is nothing stopping artificial intelligence from making us humans return to "relate and to care for one another." Hugging, talking, laughing, dancing. Trusting, being trustworthy. Sex, relationships.
You know, I think Yuval Harari hit the nail on the head when defining humans as those mammals capable of gathering in groups of more than just a few dozen, by being able to believe in abstractions — like "a country," "being a fan of a soccer team," a religion, etc. Harari mentions how difficult it would be to keep forty thousand chimpanzees well-behaved in a stadium, and yet we humans do that every single weekend.
Which abstractions will we believe in, with or without AI? I think this part isn't really under attack. Nor is our capacity to relate to one another under attack.
So both our relating-animal side (oxytocin) and our human side (believing in shared abstractions) remain untouched. They aren't being replaced by anything.
What are AIs — much like what ChatGPT did with mathematics — actually replacing?
Something else. But neither our capacity to connect with one another nor our ability to gather in the thousands, millions, or billions around abstractions. Just my two cents.
Great article!
(My thoughts on similar topics: https://blog.vladi.dev/en/articles/mudando-o-ponto-de-equili...)
Your assessment is depressing -- the thing that makes us special has gone from "thinking", "curiosity", "understanding the world" to "we don't kill each other when gathered in a stadium".
That's a fair point of view. It's Yuval Noah Harari definition, not mine (Sapiens book, 2nd chapter):
"Ants and bees can also work together in huge numbers, but they do so in a very rigid manner and only with close relatives. Wolves and chimpanzees cooperate far more flexibly than ants, but they can do so only with small numbers of other individuals that they know intimately. Sapiens can cooperate in extremely flexible ways with countless numbers of strangers. That’s why Sapiens rule the world, whereas ants eat our leftovers and chimps are locked up in zoos and research laboratories."
(...)
"You could never pack a thousand chimpanzees into Times Square, the United Nations, the Vatican or the New York Stock Exchange. You would get pandemonium. By contrast, thousands of Sapiens regularly congregate in such places, and they usually produce orderly networks and trade rather than violence and chaos."
---
And the passage I recalled from memory is actually from a TED Talk ("Why humans run the world", 2015 - https://www.youtube.com/watch?v=nzj7Wg4DAbs&t=231s)
"If you try to crowd 100,000 chimpanzees into Wembley Stadium, or into Oxford Street, or into a church, or into a school, you will get complete chaos. But you can put 100,000 humans together in Wembley Stadium, and they will usually behave reasonably well, watch the match together and sing along."
---
But I’m not giving up on your celebration of our curiosity.
Speaking of curiosity, I’d bring Peter Gray and his "Free to Learn" book into the conversation.
Take a look at the second of the five characteristics of play that Peter Gray proposes:
"Play is activity that, from the conscious perspective of the player, is done for its own sake more than for some reward outside of the activity itself. In other words, it is behavior in which means are more valued than ends.
When people are not playing, what they value most are the results of their actions. [...] When people are not playing, they typically opt for the least effortful way of achieving their goal.
In play, however, all this is reversed. In play, attention is focused on the means more than the ends, and players do not necessarily look for the easiest routes to achieving the ends. Play often has goals, but the goals are experienced as part and parcel of the activity, not as the primary reason for the activity. Goals in play are subordinate to the means for achieving them."
Using Gray’s definition, mathematicians might actually be mourning the loss of play — the process itself, the thinking, the exploring — rather than "just" reaching the final result (via ChatGPT).
Is this a valid way of looking at it through Peter Gray's lens?
Cooperate to what end? If we succeed at obsoleting thinking, what would the collaboration be about?
(Also, I suspect if you take 100,000 humans, with no explanation, and force them against their will into Wembley stadium, you'll also get chaos.)
I apologize that I've just updated my answer above bringing Peter Gray's play definition to help us deal with our innate curiosity.
I'll summarize here: If we succeed at obsoleting thinking, we loose the game, we stop playing, according to Peter Gray's definition.
So you - and the mourning mathematicians - are all correct: we loose the most important part of life, according to Gray's view (above).
As a software engineer it is hard for me to feel sorry for these mathematicians. Software engineers are getting laid off because of advancements made by AI. Math researchers are employed at universities, and universities aren't laying off mathematicians now that OpenAI has solved some of their puzzles. Mathematicians can still feed their families, software engineers displaced by AI can't. Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
When software engineers spent the last 50 years eliminating the work of clerical, secretarial, and dozens of other back-office functions, were they concerned about how those people would feed their families? Do you think anybody but other software engineers feels any sorrow for them today, now that they are facing the same fate?
While I mostly agree with you, there's an argument to be made that classical programming tended to eliminate the mechanical parts of work that people hate doing, while AI is removing the intellectually stimulating part.
It's not fun to think of a future where knowledge workers become irrelevant in society and we're back to just physical work and managerial elites.
I'm not really one of them, but there are many developers for whom the intellectually stimulating part is designing the system. Writing the code was boring drudgery that they hated doing. They tend to call themselves "architects" or something similar, not "programmers."
How long until the LLMs run circles around the best software “architect” out there?
Finally, someone who gets me!
Plenty of people pay the bills with work they hate doing.
That wasn't software engineers eliminating that work. And neither is it today AI eliminating software jobs. Let's name the straw man in the room: it's humans displacing humans, and extracting the value for themselves.
Just like in past history.
Yes, it sucks, for software engineers, those in the past, and those in the future. As long as you and others fall for the straw man perhaps you and others will get more and more tricked and hurt, misplacing your aggression and hatred on each other rather than those receiving most of the extracted value.
> extracting the value for themselves
Yes in some cases, but the reduced costs also allow lower prices which benefits all consumers of whatever product is being produced. At least for those products where there is a competitive market.
What an asinine comment. Do you think I, as a software engineer, am sitting around twirling my mustache, cackling as I plot to destroy livelihoods? I'm trying to make a living like anybody else. I've had my own opportunities lost to outsourcing and H-1B's since day 1.
The economist John Keynes predicted almost 100 years ago that technological advances would lead to a 15 hour workweek. If AI now makes you 5x as productive, then why aren't you working 1 day a week for the same pay? Why didn't Keynes' vision come true.
It is not for the same pay. Real wage increased dramatically during that time period. And despite all obsoleted jobs, society is still more or less at full employment.
A living like anybody else or a living at the expense of everybody else?
Before calling that comment asinine, did you consider that the commenter might be reflecting @flowerlad's comment back to him/her to prove a point ?
> Why didn't Keynes' vision come true.
governments implemented (and still implement) keynesian economics, and it has turned out to be a very bad idea and he was very very wrong.
There are orders of magnitude fewer mathematicians with full time academic positions than there are software engineers. The economics are different and consequently academia tends to be more slow moving in reacting to market trends. It doesn't mean it isn't coming, but it will look somewhat different
I don't personally grieve for the likes of Strogatz or Tao, it seems likely both of them will be fine however this pans out. But many more mathematicians have been toiling away in precarity and obscurity and without tenure for years and years. And of course many mathematicians went into private sector roles as software engineers, actuaries, analysts, ML engineers and similar, and have been displaced there, too.
And highly visible blog posts written by famous mathematicians with large audiences and closely aligned with the news and hype cycles getting frontpaged on HN doesn't mean that mathematicians are disproportionately whiney, nor that software engineers aren't also writing about their plight or ending up on the front page as well.
To me this goes in the same direction as people not feeling sorry for software developers because our job is supposedly automating the job of other people (which I don't think describes the profession accurately as a whole at all).
Neither sits right with me. Can't we just all collectively acknowledge that we have to grapple with this new piece of technology that either is already or at least seems to be going to be hurtful to a lot of individuals?
Instead of looking at the in comparison limited group of people currently pushing this technology and asking ourselves whether we should be on board with their doing, we look to one another and are happy that someone else is now suffering just the same as we are. What a sad state of affairs.
I agree with you because I so thoroughly understand what mathematicians are saying. Expertise and knowledge developed over decades, previously unique and marketable, and status conferring, are threatened in their relevance because of ai.
I know this intimately because I identify, roughly, as a programmer first and security researcher second. Guess which two fields ai companies decided to specialize in first? Yes, the FML ones.
The intellectually rewarding part of my job/skills - the only reason I enjoyed them - are, not completely gone yet, but threatened. And shrinking. This is a very scary observation.
I know from ‘isomorphic situations’ lessons in history that fighting the tide is pointless. I have to decide intelligently how to respond. Easier said than done when your livelihood and identity are at stake.
For now I still have a job but I and everyone around me is worried. In a year, At best our jobs will be very different.
So yes I empathize with these mathematicians enormously.
I don't empathize with them that much, as many of them share in the blame of creating the broken system we have today. They want the status that comes with tenure and having a nice math job, and then pull up the ladder for everyone else.
Math journals commonly reject papers as “out of scope” or “not of sufficient interest” without an editor or referee actually reading them. So it’s not as if the “math community” was checking these papers for accuracy in the first place. Now AI speeds up the creation of papers that go unread, and we’re supposed to pity mathematicians? If journals and experts can’t be bothered to read the papers, why should authors bother to write them without AI? And suddenly mathematicians are worried about AI papers being “too hard to read,” even though that wasn’t a concern pre-AI, when no one was reading them anyway. Too bad. The incentive structure now points toward AI, and mathematicians share the blame for that.
100% agree. The comment you are responding to is basically playing "victim Olympics".
AI is going to suck hard for lots of professions, software engineers and mathematicians included. I think it's much more productive for folks in these professions to show solidarity with each other given the way AI will disrupt their livelihoods, rather than showing disdain based on who you think may have it slightly better.
As a software engineer? The career whose very reason, largely, for existing in this market is to automate jobs, with few notable exceptions? That one is somehow deserving of sympathy and that of mathematicians is not? Come off it. The notion is undignified.
A few notable exceptions?
Are phones an "automation of existing jobs"? Maybe couriers (??), but most of what people use them for goes far beyond that.
What about the software used to drive trains or autopilot planes? We still have conductors and pilots, but now the outcomes are safer on average and more efficient.
Or something like Google Maps -- all that replaced was the paper booklet people used to keep in their cars, it's not like we used to employ 'navigators' to go out on a road trip.
Or something like Intuit QuickBooks: there's still an accountant in the loop, and if anything more accountants today than there were a few decades ago.
Certainly some software is essentially automatory in nature. But to characterize 'nearly all' as being such just betrays a lack of awareness for the rest of the field.
> That one is somehow deserving of sympathy and that of mathematicians is not
He didn't say that mathematicians aren't deserving, he was griping that people said this about SWEs. Do you think mathematicians are deserving but SWEs aren't?
> Are phones an "automation of existing jobs"?
You seem to be unaware of the existence switchboard operators (people redirecting calls to the correct person) were hundreds of thousands in the US alone at some point.
How did you manage describe a litany of automation while being fully unaware that is what you were doing? I mean you even said the word autopilot lmao. Yes, not all tasks in a job are fully automatable. iPhones, automating communication. Quickbooks and turbotax, automating business workflow steps.
The words and excels of the world, automating documentation and bookkeepery. It all makes some kind of existing thing we already do faster and more convenient, and often more efficient. Most software simply does nothing new. The thing you didnt manage to list which is the actually good example is games.
This is a silly analogy, and you're comparing very different entities.
Say you have a friend who likes making music (not professionally - just a hobby).
If an invention came out tomorrow that took the joy completely out of making music, would you respond with "It's hard for me to feel sorry about you."
Just because things suck for you doesn't mean it can't suck for others (in different ways).
> Software engineers are getting laid off because of advancements made by AI.
The bean counters laying off software engineers ought to take notes from the frontier labs' own hiring practices. They're still hiring lots of humans for business and engineering roles even when they would be better positioned than anyone to automate away those jobs with their world-class models (https://openai.com/careers/search/).
They're hiring now. Clearly some researchers at OpenAI believe we'll eventually all be unemployed: https://x.com/alexbronzini/status/2103846255227175288
Holy logical fallacy shit
What exactly is the logical fallacy there?
"OpenAI is hiring software engineers, therefore all companies will continue to hire engineers"
Generalization fallacy, survivorship bias, etc.
Why is it hard ? It's basic empathy.
What does your being software engineer has anything to do with feeling sorry for Mathematicians' job ?
It's a classic "I have it worse than them...everyone should feel sorry for ME and not them".
It's not about universities, it's about grant funding over the next few years. It is very apt to collapse if AI is more useful than people. Why pay people to find this when tokens will do instead.
I'll let you in on a secret: In the US, most math professors don't seek grants. In most universities, I don't think grants are even a factor in their tenure application. It's quite different from engineering.
In my undergrad math department, only 2-3 faculty members out of a dozen bothered with grants.
Who will continue to develop neural nets if the mathematics training pipeline dries up? Will we see the frontier labs relinquish AI research to the machines themselves while their humans apply for unemployment benefits?
Currently math research serves both as a training pipeline and as a math-production pipeline, and its funding is set based on those two outputs. If it is "just" a training pipeline, do we need as many mathematicians?
Who has been funding grants, and do you really think the grants are going to stop? There are infinite number of puzzles in Mathematics. The grants are not from companies like Boeing and Walmart who urgently need solutions for the problems, if they were I can understand why they might stop.
It’s only been a few months since LLMs started solving significant open problems in math. I wouldn’t jump to the conclusion that they have it better than software engineers.
In many ways it will likely be the opposite. Tech firms still need engineers to talk to customers and run experiments. What will the mathematicians be needed for?
Software engineers are getting laid off because of the end of zero interest rates, wars started by Putin and Trump, and a bit because of pandemic over hiring. The part that is due to AI is less hiring of juniors. Know your enemy before starting a fight.
What's your source for "Software engineers are getting laid off because of advancements made by AI". I haven't experienced that in my field and can't find any data to support that notion.
> Software engineers are getting laid off because of advancements made by AI.
This is said by a lot of CEOs trying to make layoffs look good, but I haven’t seen anything that suggests that it’s true.
> Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
Really?!? We’ve been living through 3 years of blog posts, LinkedIn hot takes, and podcasts talking about the plight of software engineers in the era of AI. There isn’t a week that goes by without an HN post about it.
I can feel empathy for the practitioners in a field that may have been profoundly changed, the same way I can feel empathy toward software engineers who are struggling with the changes in their own field.
In the end, we’re all trying to figure out where this leads. And how this will affect our lives and our work.
Unless this is satire, have you not considered how many proles have been laid off because they were replaced by technology or told that a spreadsheet model had decided their job was superfluous? Capitalism has been using technology to fond new ways to enrich itself for the whole age of personal computing, and arguably long before. Those Andreesen Horowitz comments about 'software eating the world' may have been exciting to people working at startups, but came off as rather sinister to people who were in the world outside softwareand just wanted to build a modest but comfortable life rather than develop unicorns and cash out.
The difference is it takes great intelligence to be a mathematician, whereas anyone can be trained to write code. It’s like the difference between an automotive engineer and a mechanic. I would argue writing code takes less skill than carpentry.
Do you write much code? Do you design software? It sounds like you don't. The challenge with software is managing complexity. This is unique to this field. Mathematicians are humans.
Software engineers still can feed their families. The notion that software engineers can not feed families is not consistent with reality.
Sounds like you don't know any junior software engineers that got laid off.
The idea that a junior software engineer who can't get a software job _and also can't get any other kind of job or move somewhere cheaper to live_ to the point that they can't afford to feed their families is what's unrealistic.
It's not like the options are write software or starve. Billions of people find jobs and can feed their families without writing software.
I also don't often meet junior software engineers who actually have families to feed. Most juniors work in the industry for a few years before deciding to get married and have kids.
> can't get any other kind of job
At that point they won't be software engineers
I think your expectations of who gets to do math is uncalibrated. One famous math researcher who solved a major open problem was working at Subway while still doing math on the side.
A very small fraction of people who get math degrees actually continue to do math in their careers. They usually have to get other jobs in order to survive (actuary, teacher, finance, etc.). Academic math funding was already drying up for last few years, and now AI will probably cause more cuts. Not to mention that academic math positions have never paid particularly well.
I feel bad for anyone who can't find a job, but the software industry as a whole has been the cushiest industry for decades, and the current turbulence seems more temporary (and driven by corporate greed) than a systematic destruction of the trade.
> move somewhere cheaper to live
Unfortunately this often means moving away from family. What are you going to say to the kid whose parents toiled to get him through college on the expectation that he would be able to take care of them? "Sorry Mom, I have to move to North Dakota to work on the oilfields"?
Do you? Most of the software layoffs had to do with higher interest rates and changes to R&D tax policy. AI isn't the major factor.
I happen to be (re)reading Strogatz' book "Nonlinear Dynamics and Chaos" right now. It's beautiful thing, and nothing in the "age of AI" threatens to equal it or in any way diminish my enjoyment of it.
I think the point is that in the "age of AI" virtually nobody will have the opportunity to follow a career and produce works like Strogatz was able to do.
I'm not disagreeing with the post. My personal opinion is that LLM "thinking" is qualitatively different to human thinking, that the "convex hull of ideas" idea is probably right in some sense, that the social structures we have built to allow and encourage human intellectual exploration are not obsolete and that there is a high risk of catastrophic damage being done to them by those who fail to see this.
Hacker News is so depressing these days. One doomer post per day to keep the valuations up.
Nothing has changed in math. Math is not solved nor has the human been kicked out of the loop. Same with programming.
*I said nothing really changed because LLMs becoming superhuman at deduction was priced in with o3 and neither the peer review, training nor publishing methods will get a revamp beyond what could already be done with SQLite and a website.
I'm no mathematician, but my impression as someone who at least does a lot of applied work with maths is that the main thing that's changed is that the yardstick by which academics earn recognition is being re-assessed.
But from my perspective proofs have never been the most interesting part of maths. What really interests me is the intuition that leads to new applications, and to new questions. Newton, for example, gets more credit for calculus than Leibniz because he's the one who demonstrated how darned useful it is. Erdos is arguably more famous for asking interesting questions than for answering them.
I still don't see LLMs seriously contributing to that facet of mathematics any time soon. Every so often I ask one to suggest a good binomial confidence interval for a particular application, and it still tends to just suggest one of the subpar options that everyone learns in a first year stats course.
Proving theorems is one of the main ways that mathematicians turn math into money.
The recent barf-out done by LLM companies is useful if what are you are interested in is an unreadable mess and someone giving you the final answer. You might be able to extract some new "aha!" moments out of it, but we will have to wait and see what happens once people go through it. I would hypothesize the LLM output will be good at hinting at some non-obvious relationships and patterns, but I doubt it will yield anything of substance in its own right.
Historically, proving difficult theorems required mathematicians to sometimes invent new fields, new ways of looking at problems, the discovery of which itself yielded incalculable contributions to both mathematics and other fields. It is this that is the greater contribution of and to mathematics.
> Proving theorems is one of the main ways that mathematicians turn math into money.
Mathematicians working at universities are paid to teach, to engage in the politics and business of government contracting, and only in very rare circumstances to play prestige games on famous problems.
It is unlikely that Congress will decide to slash budget for NSF's MPS directorate on account of AI getting good at proving theorems. That's just not how Congress works.
It is also unlikely that LLMs will be able to do, in the near future, do the sort of applied mathematics that happens in the context of occasional NIH or DoD funding that flows into Mathematics departments, for a variety of complex sociological reasons I'm happy to expound upon.
And it's unlikely Engineering degree programs will stop requiring calculus sequences, since being able to do the computation was has not been the actual point of teaching calc since he late 20th century (if ever).
So, the good news is that I doubt any of this will have much of an effect on the job market for Mathematicians who want to prove theorems for a living. The bad news, of course, is that the job market for Mathematicians who want to prove theorems for a living was already pretty horrendous.
I’m on your side on this overall, I think. And I suspect we’d agree more than disagree on nuances. But saying that ‘nothing has changed’ is very obviously wrong.
I guess I chose the words "nothing has changed" because that's true relative to all the subtext of content I'm seeing online claiming that "math is solved"
I think the https://news.ycombinator.com/item?id=50043026 Terence Tao post was much more hopeful and along the lines you're suggesting.
It was, but the comments on that post were insanely depressing to read. I'm a mathematician, and find myself largely hopeful and excited for its future, but being online this last week has been incredibly brutal due to the surrounding vitriol. It's an avalanche of negativity, and I'm largely not talking about that of my peers (though indeed, some of my peers). Most mathematicians I've spoken have been reasonable / positive, perhaps an effect of my field.
I really need to just stay off social media for like a month and keep talking IRL. Longer, if I know what's good for me.
I agree with "nothing has changed" in practical terms. If you take a look around most things rely on an undergraduate level of math than whatever is at the frontier. The proofs may be important, but they are really remote to people's lives.
This is like saying WW2 didnt happen while you were living in a Soviet village in 1940s, and you were doing the same thing, and nothing seemed to change
Go apply for some jobs. You’ll quickly come to the realization that something has changed.
And this is only the beginning.
Yes, interest rates are higher than ever and consumer sentiment makes business wary
ZIRP is over but rates are not especially high and yet I don’t remember job market being that bad. Maybe employers are taking wait and see approach and avoid hiring developers they expect to layoff in a year or two if models will continue to improve and becoming cheaper at the current pace.
I think that models will improve and get cheaper. I think we'll see Opus 5.5 tier models open weights in six months.
You are probably right that some companies are factoring AI into headcount but I'd imagine that the interest rate and energy crisis are bigger concerns
Not that it explains all job market weirdness but you do have a problem on the hiring side with having to deal with the flood of AI generated applications. I assume that’s going to have a slowing effect on the typical apply/interview/hire process.
Interest rates are higher because AI.
It was long predicted that the sign that AI is here in macroeconomics is rising interest rates, because investors can get MUCH higher rates of return doing AI investments than borrowing governments, why would I lend money to Greece when I can lend it to Anthropic?
I own my company and directly participate hiring. What’s changed among me and my peers is that we’re on incredibly precarious economic grounds with no telling what’s going to happen next month.
LLMs haven’t done much in the way of informing me or others I know regarding “who and how many”.
I’ve done some hiring lately as well and we got a ridiculous number of applications, like 500+ for a run-of-the-mill developer job.
I’ve worked in tech for 20+ years and I’ve never seen it like this. Sure there are job openings, but sooo many applicants it seems.
(This is Sweden, YMMV.)
I would wager that most of them are AI generated. The cost of generating a neat looking LaTeX resume and adding vibe-projects to one's GitHub is very low...
500+ is a lot?
If you’re one of the 500 then it is, yes.
It’s normal to be one of 10k.
For a research role at Anthropic perhaps, not for the roles I’m typically hiring for.
And how! All the “nothing has changed, stop whining posts” are neglecting that they haven’t had to go look for a job. My previous company became quite toxic about LLMs and employee usage. It’s not “move over to the paradigm or get left behind” instead it was “jump on the hype-wagon and sing LLM’s praises all day, every day or you’re out”.
The fear is the sales pitch. AI has some uses but nowhere near enough to justify the price tag, so we need something more to sell it.
Fear and FOMO mainly.
Re depression: Seems like it is not just HN... You say nothing changed. Maybe that's even true.
But many people perceive this very differently, see link below. And perception is reality after all.
https://news.ycombinator.com/item?id=50093180
Yes I take issue with all the anti-human propaganda coming out. I think that there will be a future we're essentially replaced but it's not now
The doom posts will continue until profits are up.
Jokes aside, its like working for a company, individually you can lift mountains and someone else will take credit. When you use LLMs for a single bit, all credit goes to LLMs lol
> Math is not solved nor has the human been kicked out of the loop. Same with programming.
Is it like Chess and Go? There are still plenty of people playing for fun, but the machines are better at it than we are.
Nah, see my other comment in this thread with actual technical issues with the math dump and how math =/= deduction
Mathematics as a field of study is maybe 2500 years old and a lot has been figured out in that time. But look how far AIs have come in just ten years. I wouldn't expect progress to stop now, would you? A decade from now I wouldn't be surprised if machine mathematicians are essentially oracles. They tell us the answer but can't explain it to us any more than you could explain long division to a golden retriever.
This was the top comments when the submission disappeared from the front page.
Now the thread has been cleaned up and a pro-defeatism comment is the top comment and the submission is on the front page again (I'm not alleging moderator intervention, user mechanisms are sufficient).
> One doomer post per day to keep the valuations up.
Wow that's very well worded.
I'm sorry, but "nothing has changed in math" is a truly ridiculous statement.
Sure, there are people who overstate things. But that isn't an excuse to keep your head in the sand.
o3 already had this priced in amongst many mathematicians I talked to at conferences
You use the word "already" as if it already happened, but this is confusing speculation with reality. Market prices are a summary of speculation. Plenty of things are "priced in" that haven't happened yet, and maybe won't if the market is wrong.
Anticipating that something will happen is not the same as evidence that it's happened.
You have an earned an upvote from Long Term Capital Management!
Doesn't seem to be the case here: https://www.daniellitt.com/blog/2026/2/20/mathematics-in-the...
I was on Twitter around 2024-2025 and showed both Daniel and my advisor at UCSD sota results I created in group theory using Gemini 2.5 (edited) Pro in Q2 2025.
A lot of us knew what was coming. Profs I speak to these days say it's the same old except they need to pick a different kind of problem for grad students
my doomerism isn't about keeping valuations up, it's about tearing it down.
https://en.wikipedia.org/wiki/IBM_and_World_War_II
the above, a "sorting" machine is the primary usage of hyperscalers currently, they're just trying to find new uses to boot for extra money/legitimation.
the browser industry the phone industry the CAs JavaScript
it's all paid for by those wanting to sort humanity into whatever bucket the can fear monger the peasantry into for aweek or however long it takes the classification system to sort out.
gentrification works, but we chose forcible displacement now we have to live with the forever consequences.
would you give guns and bombs to the displaced native americans who taste blood over their beads and trinkets?
we've been inside the event horizon of the singularity for years, the tax bill will only get worse unless we recognize it for the tick that it is.
yeah nothing changed in math. mathematicians are weeping for some strange reason (maybe paid by OpenAI to weep?). And Terry Tao is posting blogs on AI + Mathematics because he was bored (or because he was also paid by OpenAI).
They are overreacting imo. A lot of mathematicians stuck their head in the sand with o3 but this was the logical outcome; superhuman deduction
I don't get the argument - if we follow what you say, they're not _overreacting_, they are reacting appropriately but should have done so earlier (when o3 was released) rather than now?
My claim is that if Bob, a mathematician, believes that he is rendered useless by an LLM then he did not play with o3 and all the other models between now and then to realize that, yes, they are superhuman at details and deduction but not superhuman at math
Can you come up with a concrete prediction on what AI can’t do in math? Something testable.
Yes, here is a concrete prediction
AI will not replace the mathematical community (peer review, abstraction creation) until AI can train the next AI purely on data generated by itself and with a tool in a turing complete environment. The environment has to be air-gapped from human data but the AI can use tools and its own outputs
I think that "solving math" is isomorphic to solving the "thinking part" of the scientific process which would be genuine RSI
Disagree, there are in fact documented cases.
> I think I have been underrating the pace of model improvements. In March 2025 I made a bet with Tamay Besiroglu, cofounder of RL environment company Mechanize, that AI tools would not be able to autonomously produce papers I judge to be at a level comparable to that of the best few papers published in 2025, at comparable cost to human experts, by 2030. I gave him 3:1 odds at the time; I now expect to lose this bet.
https://www.daniellitt.com/blog/2026/2/20/mathematics-in-the...
“I’ve slowly updated my timelines over the past year…”
and
“I think I have been underrating the pace of model improvements. … I was not correctly calibrated as to the capabilities of existing models, let alone near-future models.”
Fair enough. But after the distinct distances result if you did not price it in, that's on you haha
> One doomer post per day to keep the valuations up.
Yes, outsiders (investors, journalists) are given a false impression while insiders (software engineers) are supposed to give up. Double win for the agitprop machine.
It's probably because with the release of each model it is becoming more and more difficult to see how a person's livelihood and way of life continues. Uncertainty stacked on uncertainty.
How will it pan out, no one really knows. The leading frontier labs are continuing to hire aggressively. But who knows for how long or for long or near-term goals.
The open ai math dump was actually what got me off the singularity wagon.
There were mistakes that could easily be seen with a diagram rendering three big results wrong, exposition is terrible, no coherent underlying theory generated from solving so many problems, as Thomas Bloom pointed out even duplicate proofs of the same statement in a single paper.
It clicked for me, "this is slop but slop can be superhuman and useful!". Similar to programming where it can generate as many criticisms of your codebase but you have to weed them out.
If anything, that makes the case that math is fucked even stronger. Nobody had any idea of how to prove quasi-Riemann. OpenAI produced three proofs, one of which was just a lemma in solving another problem. They have a machine that is better than every human being alive at math that solves problems humans cannot even when working inefficiently.
Math was never only about deduction. It's also about data curation.
I was taking to my friend in OA who was really surprised that the proof of L(F_2) = L(F_3) could be done in the 1980s. We just didn't have a superhuman deducer back then.
All scientific break-troughs are obvious in retrospect.
Special relativity which Einstein got a Nobel for is now taught in high-school.
"Of course if you move a magnet in a piece of copper you will generate a current, it's extremely obvious it should be so"
That's true. I'm not saying the work is trivial, I'm calling out the difference between "deduction from known data" and "distilling known data"
Even if this was overly long and mostly things I've seen before, I'm just very thankful that it wasn't yet another claude piece.
There was an interesting passage in the article:
> Today, the mathematicians are weeping, because the machine smashed their tower first. But tomorrow, it may be the physicists, the economists, the illustrators, or the writers who weep over their fallen blocks.
Isn't it extraordinary that these models still can't write well? I mean, they write passably well, but they all have a highly characteristic style that you can spot from a mile away. These language models can crack the toughest open problems in math, but they haven't yet cracked human-like writing, especially long-form writing, without exposing themselves. This is very very odd, IMO.
Or, just maybe, they share a key property with the mounds of popular writing, journalism and random internet accretions they were trained on: they seem total rubbish on anything you know about deeply, but brilliant on everything else.
Their math is as just as good as their rambling writing and their Ghiblified poster art. Most people don't seem to realize that.
This is not a reasonable take in 2026. The proofs are valid. You can even check the Jacobian counterexample yourself with pencil and paper.
The counter example, yes.
"Some subset of this collection is valid, therefore they all are" is not a valid inference. It's more like what we call "proof by intimidation".
> This is not a reasonable take in 2026.
I think it is. Why is it that the only people in any field I'm aware of that think the AI output in their field is high quality are either the ones that are carefully crafting the output themselves or were producing/consuming slop themselves before all this started? (Outsiders pretty uniformly like the output, which is why I drew the parallel to Gell-Mann amnesia.)
The difference here obviously being that the part that you mention is more embedded in the proofs (long and difficult and near-hallucinatory) but the results that matter are that the proofs are correct and validated. So I think you're failing to account for that, the problem for math is that the journey matters less than the destination.
Hard disagree. If we send someone out exploring and they discover something (maybe?) but their account of their journey is impossible for anyone else to follow and contains obvious hallucinations, what have we gained?
Isn't the problem more like they did discover a new land (as in, the proofs are true, and the solutions are validateable) but we have no idea how to make heads or tails of how they discovered it.
Writing is a very slippery thing. “Good” writing from 200 years ago reads as unbelievably dense and overwrought to most people today. So even if you made an LLM that was “good” at writing, how long would it stay that way? AI systems can put out more writing per day than all of humanity combined, meaning it can also generate and beat to death cliches much more efficiently.
> Today, the mathematicians are weeping, because the machine smashed their tower first.
I read this passage and thought, "has Derek even talked to software developers"?
That feels to me like the first discipline to go through this, where their daily activities are massively changed by GenAI.
No, tech writers have it much worse.
Good point! Tech writers, and writers in general, were impacted before developers.
Maybe visual artists/photographers were impacted before then?
Anyone have a timeline of this stuff?
Language models don't have a characteristic style you can spot from a mile away. They haven't had it since before ChatGPT.
What is true, is that chat models (and by extension agents) have a very characteristic style after their LLM went through post-training. But the pre-trained language model doesn't have that style.
So if you want to write well with an LLM, you could and it is a well understood problem. It is more a matter of that not being of much interest, as it competes with having the model be good for chat interfaces.
How to have a well-aligned, RL'ed, safe or thinking model, which also writes well in long-form, is a bit an open problem. Though presumably agents could do it when they have access to a pretrained model.
Why is writing in the style of a corporate marketing department a requirement of being good for chat interfaces? Why can't they tune them to sound more like a normal human?
You can just tell them to adopt a different style. To make sure, add another pass with an agent that flags and/or fixes LLM tics. Humans typically do several passes over writings that they intend to publish, why are we expecting agents to do it in one shot? Just like the superpowers skill for software development does a spec phase, planning phase, several review rounds, etc., if you do the same with writing you get very passable results.
Makes me wonder whether it's a fundamental limitation of the technology or more like missing a good set of tasks in the training pipeline
Might be that the vast majority of their training data is poor writing.
Gemini's (at least for Flash 3.6 - 3.8) writing feels quite "human" to me
>Isn't it extraordinary that these models still can't write well?
It's not at all extraordinary if you understand how they're trained. For math and programming it's possible to generate an infinite amount of training data with correct answers, as compilers and theorem provers can guarantee correctness. For writing, the quality is not objective, it's determined by human preference, so you need humans in the loop, which slows things down massively and limits the amount of data that can be generated.
Math and programming languages are formal. Meaning they don't have any ambiguity in them. Every statement is what it is. That means they are very rigid and there's not a lot of variance. Natural languages are more diffuse and a written piece is more the reflection of the author than anything else.
That is why it's so disturbing. After several paragraphs, you realize there's nothing human there even when the actual information is present. People that do create AI piece are OK with it because they already have the context, but for the reader, it's always the uncanny valley.
There's no money in it. There's money in programming, and math is a good target to give the models superhuman reasoning abilities, but it already writes well enough for middle managers to write all of their emails.
Recently a Canadian immigrant won a bunch of the highest French literature prizes for his novel, which then turned out to be an LLM fabrication.
> I am not one of those AI people who believes that hundreds of millions of jobs will be wiped clean from existence in the next five years
"Wiped clean" is the wrong phrasing, and 5 years is a short timeframe given the ability of some sectors of the economy to resist change. But yes, hundreds of millions of white collar professionals will need to rethink their careers and their business models within the foreseeable future.
Just today there was an HN article about how shady the dental industry is. In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
> In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
Right up until the sellers of dental supplies find a way to...
Indirectly influence model training to increase the frequency with which models recommend dental work and/or lower the bar for what they determine to be "necessary"...
or...
Directly influence model recommendations by paying model providers (who will be happy to take that money).
The point being that models (and particularly the humans that control them) are presumably just as corruptible as the humans they are replacing.
The advantage is scale. If a dentist is shady and overprescribes it can be difficult to find out because there are not many victims. If a company serving millions of people is overprescribing there’s a much higher chance of them getting called out on it.
That's the optimistic take. I have a much more pessimistic take: there is now only one place where people who want to sell dental devices have to focus their efforts. Less effort for them... and because of the scale... much bigger payoff.
When I was thinking of the examples I gave (poison the training and/or pay the providers), I was intentionally drawing analogs to how I saw Google search decline.
First, SEO "experts" poisoned the results. Then Google saw how much money could be made, and they sold the ad space directly to marketers. If you search for "Best <widget> for <doing something>" on Google, I am certain you are not getting the "correct" answer. And Google has suffered no consequences for this. To the contrary: it has been extremely lucrative.
Unfortunately in the real world it's just the opposite. A guy selling opioids on the street is called a dealer and gets arrested. A company overprescribing opioids to millions is called Purdue Pharma and gets record profits.
People are now aware of the harms Purdue has caused. Justice has not been served, but their plot has been uncovered and discerning consumers need not engage with them.
Ah, you're saying that a monopoly protects the consumer from anti-consumer practices?
Journalists protect consumers from anti consumer practices. There’s not enough money in journalism to go after individual dentists even though many are bad actors. There certainly is enough to go after corporations, which we have seen many times. I’m not saying justice will be served, but I do think more public oversight = better informed consumers. There really is no good way to know if a dentist is honest before going to see them. That’s just not true when it comes to mega corps.
No one wants that. People pay other people to do things well. No one wants to need to discuss orthodontics with a bot, they would rather pay someone to know those things and think about them.
I guess I'm lucky that I've always hated my job, because that prevented me from tying my self-worth to it.
>Yes, some are crying about bruised egos, the feeling of wasted years, a creeping sense of status obliteration, the frustration of an unfair competitor, the indignation of feeling cheated and mistreated, and uncertainty about the future. But they are also mourning...
This nails it, however IMHO its almost all bruised egos and feeling of wasted time. I don't know the ratio of A:B but I also don't know anyone that is in the latter category of feeling doom and mourning that their craft is about to regress.
I don't know what's happening exactly, but I am genuinely astonished how little people seem to care about people losing something real, dismissing it as "bruised egos", using petty terms like "whining" and "crying", or calling them some insult for the crime of being angry or sad for having a big chunk of their life pulled out from under them. I'm referring to the big picture of common talking points, not just this case.
I guess I'm not astonished that people are so bad to each other in general, but more that in the case of genAI it's so prevalent in places that seemed more elevated previously.
Part of me wonders if this is actually a form of self-defense. The misery of others can provoke sympathy or the opposite: Angry pushback to protect oneself from dwelling on their misery.
Similarly, I got into software because I like working with computers and solving problems for others. I see TONS of similar sentiment where people are almost joyously exclaiming “Software is solved! Sorry!!! It’s been a long time coming, you all deserved it, get fucked!”
And I can’t imagine feeling the same about any group of people. It’s been odd, to me. I’m just trying to support myself and the people in my life, and now there’s a looming fear that I can’t do that anymore… business owners are drooling over that, and other sectors are laughing.
Hard times, I guess.
Tech workers are a small subset of the population that have been able to live very privileged lives when compared to everyone else.
I think the average person lives a much harder life and has been doing so for a while. Should we really be surprised by the lack of empathy when folks in tech had little to show for the common man this whole time?
I saw something similar on Twitter responding to 45yo software engineers making comments of depair in that HN thread about the increased suicide rate of middle-aged men.
For some reason I'm stuck in a real depressing, doomer algorithm / homepage on Twitter. But everyone on my homepage seems to have a take on the HNer comments along the lines of "see, this is why you don't make your job into your identity".
But that's such an antihuman response. The only reason you can put decades into a field to grind and build expertise is because you have an identity-level interest in a subject.
It's also not a good thing if the optimal way to live going forward is to have no passion for anything lest it changes so that we won't feel bad. I don't think that's possible unless you remove the human part of our brains or make life so horrible that we're happy just to survive another day.
> the increased suicide rate of middle-aged men
Source? There has not been a statistically-significant increase in middle-aged male suicide rates between 2023 and 2024 confirmed data [1]. (2025 breakouts aren’t out yet, but top-line data show decreases.)
[1] https://www.cdc.gov/nchs/products/databriefs/db572.htm
The article was https://www.telegraph.co.uk/news/2026/10/13/suicide-rates-hi...
In 2025 in the UK men age 45–64 had a suicide rate of about 24 per 100,000, which is the highest it has been in the past half century, and is higher than other age categories. In the mid 2000s, the rate was like 17 per 100,000.
From your link, in the USA the overall rate for men of the same age is about 30 per 100,000, but varies dramatically between states.
Got it. I think it’s fair to point out that we have no evidence that UK statistic generalizes to America or other countries.
I'm referring to this submission: https://news.ycombinator.com/item?id=50093180
> For some reason I'm stuck in a real depressing, doomer algorithm / homepage on Twitter
At the risk of taking this off topic, the reason is Twitter. Don't go on Twitter and the doomer algorithm disappears from your life.
Lots of people have lost their jobs and had to overhaul their lives due to layoffs or technological advances or political choices or factories moving overseas or whatever.
They are showing as much care for mathematicians as they received themselves, which is not much.
i think a big part of the backlash isn't about math per se and it's about what 'work' or 'a job' are supposed to be.
when mathematicians complaining about losing the part of the job that they love, people think "if you love doing it, why is anybody paying you? you're describing a hobby."
Yeah, the glorious utopian future we're building where humans are only able to make a living doing things they hate, and money is concentrated with the few.
Because we don’t see it as something for them to be legitimately upset about, except that it’s going to disturb their ivory tower unserious profession.
If AI got significantly better at performing surgery overnight and doctors started crying about how they were going to be replaced, what do you think the public response would be? Non-doctors want the best possible care; they aren’t interested in protecting an industry. The same will be said for pilots and lawyers, too.
People dislike self-serving whingeing, end of story, but that’s doubly true when it comes out of a profession (theoretical mathematics) that is mostly practically useless.
The doom and mourning is that it's no longer their craft. Their involvement is on the threshold of no longer being useful. What does it mean for humans when the best their involvement accomplish is slowing down progress?
This is a fundamental human need and it's being taken from everyone eventually, by design. It will take a generation to turn over and find new ways of being useful. Those who ran into this wall and could not safely reduce their income and retire/pivot to other fields, will be the coal miner oxy crisis of this generation.
Mourning is fine if it is temporary, however no man should dread their own future if they still have control of their life. This is the idea of unalienable rights and was enshrined hundreds of years ago. It seems to me that anyone dreading the future of their field failed to take a page out of their retirement strategy and diversify their own interests, i.e. https://en.wikipedia.org/wiki/Polymath
https://en.wikipedia.org/wiki/Life,_Liberty_and_the_pursuit_...
My friend, this happened a week ago. If a spouse dies you don't come back a week later and say "welp, chin up and diversify your dating pool!"
https://record.umich.edu/articles/study-grief-is-short-lived...
Ok, so if you really believe that losing one's vocation at 25 is the same as a 75 year old losing their spouse, that paper suggests you should wait at least 6 months before you start being a jerk about it.
I don't see what point you are trying to make. At 25 losing your vocation gives you more than plenty of time to transition into another lifestyle. Many people will consider this the very beginning of their career. At 75, retirement is near. Your legacy is known but not final. 6 months is less than 1% of your lived life, and you have a lifetime of memories to reflect and enjoy.
I'm not entirely sure what you're on about. Are you suggesting that mathematicians should become truck drivers or plumbers and be happy with that (until the next physical-aware models release, and then...)?
What do you mean, "control of their life"?
I was talking to a mathematician friend about this, and she was telling me that a lot of the proofs that have been produced in the past couple weeks don't have an application.
To me it begs the question: in a theoretical world where we don't have human mathematicians anymore, what is the point of these proofs if there's no human who actually understands it?
If the point of a lot of this math is only to expand the boundaries of human knowledge, it all seems kind of pointless.
heh these models should be turned loose on all the proofs paid for and sitting on the shelf and see if they can find an actual application useful to mankind.
Often, we don't know the applications of new math. For a long period of time we had "useless", "purely theoretical" mathematics that eventually turned into relativity and quantum mechanics.
Proofs are somewhat like legos. A bigger toolbox of them can only help in solving other, possibly practical, problems.
As far as the utility of a human being involved -- In a non-theoretical world where we don't have feline mathematicians, what is the point of these proofs if there is no cat which actually understands it? The cats can benefit from, for example, advances to optimziation algorithms that reduce costs for cat food delivery, of course.
Is AI at the stage that it can also point out all interesting things to study and then solve them? Otherwise humans would still be able and needed for progress.
Good question. I'm not a professional mathematician, but I would be interested in seeing what results the mathematicians get from trying to prompt it to come up with new and interesting problems.
My intuition is that it'll be at 'smart undergrad' levels today, though as the cryptographers say, attacks always get better.
It's the abject lack of shame that gets to me.
None of them have pointed to any of the solutions and said "this is real innovation" - because it's all been mining existing ideas and applying them elsewhere.
Imagine you spend years working on a solution to one of these problems. Only to realize after you publish that more than half your answer already existed in another branch of mathematics and you spent a bunch of time re-inventing it.
Yes the first big solve by OAI was "stolen" and that is tragic, and pretty bad. But it looks like this new batch, the one people are crying about, has none of that stink.
I think there is totally justified distress knowing your source of livelihood may be in danger.
That said, I don't think the reactions have been entirely circumspect.
1. Is the role of math academics really going to go away? Is there concrete evidence of this? Faculty still have very important roles to play, training students, training engineers, determining the direction of research. The breadth of math is absurdly, perhaps infinitely, wide, especially if including applied math.
2. You need to be a bit crazy to go into pure math academia in the first place. If you value the pursuit of knowledge for its own sake, shouldn't the emergence of such powerful tools to that end be a good thing?
I need to do some more research on this. Because I've always been told that AI, more specifically LLMs, are "glorified autocompletes". Yes they have huge data sets but ultimately they can only provide answers based on data that they were trained on.
So how can they possibly solve problems that humans have worked for decades or centuries on without being able to solve? How did they find something new? Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
The following certainly does not describe all mathematics or all LLM operations, but its my mental model for why LLMs are so effective in several fields.
Lots of mathematics entails recognizing that a tool/techniques that works in one area can be adapted to a completely different problem. For a human to do this requires having a robust mental model of the technique and the problem itself, and probably having just the "right" mental model to see how they fit together. And we are talking about extremely hard/sophisticated techniques and extremely hard/sophisticated problems, so the ability to deeply appreciate either can require years of study. And on top of that, even if the mathematician has sufficient understanding to develop the intuition that the technique might work, they will need profound fortitude, skill, and patience to see the solution through.
Unlike human mathematicians, who will develop a deep understanding of only a relative handful of techniques and problems in their lifetime, LLMs have ALL written techniques, problems, solutions, etc stored in their internal representations. So when they ably match a problem to a technique, we can imagine the operation as a supercharged autocomplete, or even an extremely proficient search engine. Their stroke of genius is really a consequence of capacity, moreso than cleverness.
Because the "glorified autocomplete" crowd likes to ignore the technical detail of how the autocomplete actually works.
They have access to a much larger corpus of information than any single human can manage in his one brain. No human mathematician reads all the papers and has perfect recall on them. The answers have always been there, but no individual has been able to make all the right connections to pull them together and formalize a new discovery or proof.
I mean the big idea of math is that (with some caveats) all the answers that can be provided have always been there.
> Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
Yes, both of these + an infinite token/compute budget and SOTA+ models. It's not like some hobbyist out in the woods with a GPU running Qwen 27B can do this.
Knowledge without understanding would be just getting "42" and not being able to ask any follow-ups.
But I don't think that's where we are at the moment. I believe the results, proofs and papers can be reviewed and understood, just like if a human peer had published another result.
I also think that the speed at which results can be now generated can't be followed by our collective speed to understand them, using the methods we are used to. As we enter the next phase where AI publishes using mostly references of other AI results, this will compound exponentially until it would take any human decades to verify a single result by hand end to end.
I agree translating a phrase doesn't teach you a language, and I also believe translators are an essential tool that results extremely helpful when learning a new language.
So instead of throwing the towel, could we develop and adapt ways to improve and aid our understanding and verification processes?
I'm convinced the people who do not see what Mathematicians see simply have not considered that their jobs are just as vulnerable. The day when OpenAI replaces their own AI researchers with GPT will be upon us, and the response from the Mathematicians will be "First time?"
“ What makes us special, it would seem, is this capacity to compound.”
Yeah so you’ll compound using the AI tools and get more done. Stop crying that your tilting at windmills is now being done more efficiently and find new windmills to tilt in new AI-assisted ways.
To me it seems the ultimate question is why would we expect humans to have an advantage at anything in the near future. A few years ago that would have sounded insane, but intellectually I don't doubt frontier models are superior to the average person. Even in the arts and humanities, give your preferred frontier model an interface and a way to "see" its work and it'll make the average person's work look like toddler scribbles. Humans still have some advantages, some places more than others, sure. Working with a designer recently I realized how bad, or maybe average, frontier models were at design. But I highly doubt the vast majority of people will be economically viable. Maybe this is one of the great filters, a society driving itself mad when freed from necessity of economic utility and without the big philosophical questions having answers.
As usual the author is ending on a completely unrealistic positive note: "humans may come to value more highly our capacity to relate and care for one another".
Does relating to someone pay my bills? People will use the informational tools to improve their social status and resource accumulation, there is no doubt about this.
If you work in a nursing home it can pay your bills. Is it so far fetched that that kind of work might see a much needed status lift post-ASI?
The vast majority of nursing home work is physical, sprinkled with a lot of excrement and sexual harsssment by the seniors.
There is little or no time, or incentives for care as described by the blog.
There was a time, not too far in memory, when the value of being human wasn't about our intellectual prowess[^1]. We were creative with the human worth thing: being there in the time of need, faith, nobility or being a well-dreaded, literally-bloody viking full of beads in your beard.
Yes, I fear we will stop being the top of the food chain[^2]. But by god weren't we getting bored? Maybe getting off our high horses and going into the weeds won't be such a bad thing.
[^1] And even today, if you go to parties and keep bouncing into people that have no trouble picking up intellectually stimulating topics, you are in a lucky minority.
[^2]: More about my fears, which are unhealthy and you don't have to read but I feel compelled to include citations: https://w.ouzu.im/aweb/unhinged
I think an under appreciated part of mathematics has been contributing clean and elegant work precisely so that others could use that as a foundation for further work and new insights. This dump from Open AI seems to be the opposite. Huge and messy, incomprehensible and taken seriously only because of Lean.
There is this assumption that AI will contribute to faster progress in math but given the garbage dump of proofs which we have to take on faith are robust I wonder if the opposite is true.
My reservation could only be dismissed if AI turns out to be able to contribute fundamental new insights and pose new problems. This takes us into an area where AI progress has been slow to say the least.
The dump from OpenAI seems to be the opposite: huge, messy, and incomprehensible, and taken seriously only because of Lean.
Math papers were hard to comprehend well before AI. This is hardly new. For example, here’s a screenshot of a paper from 40 years ago:
https://i.imgur.com/kPFlczp.png
At least with AI, you can download the LaTeX, upload the paper to your favorite LLM, and have it explain the paper at whatever level of granularity you like. That wasn’t possible even two years ago.
I think people vastly underestimate the ability of human to respect human performance despite being inferior to machines. Think about chess players vs stockfish, marathon runners vs a car, etc.
I think the crux is much more prosaic, as buried in the article. > I also don’t want to lose my job to a tangle of GPUs.
To put a GPTism, it’s not about the knowledge or status, it’s more about keeping their job and money.
And that can only be answered by policies, and imo by UBI. So votes if you can people, protest otherwise, discuss, debate, lobby with your friends, families acquitances.
Jobs and money are bottom of pyramid needs. If we answer those with UBI I guarantee you status and purpose are the next ones.
Chess players vs stockfish isn't a good comparison because chess fans only care about the performance of maybe the top-50 players.
People may gain and be satisfied among their own peer groups. All good and healthy if that peer group is your local badminton club or knitting circle - something where effort and skill have rewards and people can make objective, tangible progress. But groups not oriented around such activities could turn even more toxic.
I am not quite saying "if you can't beat 'em, join 'em", but I know more math now than in any point in my life because I want to understand how LLMs work.
I understand there are many types of math and the mathematicians don't all want to sit around and do KV cache and gradient descent things, but surely a percentage are as interested as I am and can contribute to this new emerging field? I would wager that Xl+1=Bl Xl+Cl ℱl (Al Xl) makes much more sense in their mind than mine.
This is the best attitude to have about the current state. Unless you have some dramatically different sensible solution (and you probably don't when it comes to technology, because it seems to spread for reasons that would eliminate other sensible solutions) that will catch on and take over and replace AI: it's going to continue to mature as long as it's mostly useful to everyone. And AI isn't even a new industry, it has been maturing for quite a while, which adds more weight to the argument that having an attitude that adapts to technology (unless you wish to go solo) is a 'better' fit.
The dramatic solution can never be: 'we should all just want to do it this way because it felt better to me'. That never seems to work, from what I've seen. So there is a lot of public discourse, arguing, to determine if 'doing things the old way' is perhaps more better. And if not, maybe we just shed a tear and move on (to look for a better fit).
> I know more math now than in any point in my life because I want to understand how LLMs work
The implication here is that you wasted your time learning that math, and the fact you learned that will not actually help you contribute to anything.
Now maybe you are closer to understanding the feelings expressed in the article?
>Newton famously compared his achievements to “standing on the shoulders of giants”
Newton was possibly just taking a dig at his small-statured rival, Hooke. Newton was brilliant, but a shit.
As a mathematician, I will never understand how my colleagues had never foreseen this. Okay, maybe you can believe in a plateau but at least accept as a possibility that there won't be one. This applies to every other human endeavour.
I am also sad, but my sadness is, in a way, priced in.
I saw it coming since reading Kurzweil in high school and it's still heavy.
I just had a chance to flinch first.
It's not really a profound insight. Back in the 1980s when I first started writing BASIC programs I remember thinking "well, if someone invents a program that can write programs, there will be no more work for programmers"
It's easy to have that thought, and eventually you'll be correct. One thing about technology is that it only improves. It's always building on top of what already exists. The only uncertainty is the exact timing, not the ultimate outcome.
This is it.
The people struggling right now - they were told. 30 years ago. They really should have come to terms with it by now.
But these are also the people who refused to believe it, for whatever reason. Their heads were in the sand. So I do feel bad for them... but... this is somewhat self inflicted.
No. They might have better braced themselves for the hit, but the wound was not self-inflicted.
Refusing to get out of the way of a train isn't the train's fault.
Lol ...and they call AI inhuman!
It was really quick. I stumbled across a comment from KConrad on MathOverflow on how useless AI was for research. That comment was from 2024. When it disproved the unit distance conjecture in May, the solution involved a breath of knowledge of the literature that it made sense it took a computer to make the connection. It only took 5 months for it to advance to the point that it was better than humans at almost everything. Even if you expected it, the mind boggles at how fast it was.
Also, it's impossible to really know how you're going to react to something until it happens to you. When ChatGPT came out, I already thought "in the long run they're going to automate almost everything humans love to do". Despite having 4 years to brood about it, I did not really know what it would feel like until it happened to me directly.
> It was really quick.
That's how the singularity works. It's going to keep getting quicker, incomprehensibly so.
I like to call it the great incoherence.
Even before LLMs it was feeling like the design of society wasn't really for people any longer, instead it's more and more designed for corporations and their convenience. In the US at least. Same with how the internet has stopped most people with socializing the same way they did in the past.
Now we have a new layer that will be optimized for these digital entities. We see it in writing all over the internet. There will be tons of other places they show up and doing things that don't make sense to us.
I don't see how this is possible. There are constraints on the speed at which information can move and the number of steps required to compete a given task. The improvement cannot continue, let alone accelerate, indefinitely without bounds.
OK - sure. Probably.
But it can continue to inconceivable levels before it gets to that point.
Okay, how can we be certain even about that? It sounds just as plausible to me that the improvements will become asymptotic and boring, not exponential or inconceivable -- and how can we make any confident prediction at all about the heights these models will reach? The technology those heights require doesn't exist, and its existence isn't guaranteed, even if it is possible.
> how can we be certain
Certain of predictions? I think that's an unreasonable ask.
And I'm comfortable with my prediction. Rather than argue about it - let's wait a year and see what happens, and then we can discuss it.
> I think that's an unreasonable ask.
Oh, good point! I misread your comment. Sorry for that!
> Rather than argue about it - let's wait a year and see what happens, and then we can discuss it.
Deal!
I think what makes it hard is that it's quick, but not sudden. I understand constant-rate improvement. I understand what it's like to be in a car crash. What I can't understand, emotionally, is the constant acceleration. Even now, if I look at an OpenAI solution in isolation, I can understand the big picture, so if things froze where they are I could adjust. Intellectually I know I'm not going to get that chance, but emotionally I don't.
As a programmer I’ve been ‘weeping’ for a few years now.
Programmers, artists, anyone who used to create any form of documentation, translators, there are literally so many.
Either these mathematicians thought it would never come for their jobs, because all of these other jobs were simpler or they've been living under a rock.
Either way, I don't feel for them any more than I do for all of humanity.
When anyone can push a button on a machine and out comes a math result, it still doesn't become theirs or anyone else's until they can internalize it and make it part of their thinking.
"I can’t solve these things, but maybe you people in the future can stand on my shoulders."
Seeing AI being lauded for solving problems like this makes me feel... uncomfortable.
We are feeding all works of the giants that we have all stood on into a huge vat, in the hope to generate something meaningful.
If we can't understand the output, internalize it, or build from the time and experience that a human would have gained from solving the same problem, it feels like a dead end.
When all the problems have been "solved" by AI, what's next? If humans no longer struggle to solve the problems that the previous generation have left behind, how do we progress?
The author mixes "Science" and "Mathematics". Mathematics is not a science. There is nothing empirical about it. By mixing the two concepts, a reader might think that AI is on the verge of solving all of science. Until AI gets good at digging large circular holes in the in the ground, I think Experimental Physics is safe.
> Until AI gets good at digging large circular holes in the in the ground, I think Experimental Physics is safe.
Are the experimental physicists usually the ones holding the shovels?
Maybe math is more like a science now, though? It's as if ancient astronomers saw the results from a modern telescope. "Why even look at the sky anymore," they might ask. "And we don't even own the telescope!"
Maybe the answer is to look into how Big Science works? There's still plenty of work to be done figuring out what to do with the outputs from these new math-observing instruments.
I had visions of large automated warehouses full of chemical and biological components hooked up to science machines to do everything from mixing, boiling and printing to spectrograph and robotic assembly. Smart computers and humans would pick the experiments to brute force with exact billing.
I remember the same doom cycle during my PhD when GPT 4 came out [1,2]. It seemed like everyone in NLP was freaking out except me. A few years later and new CS PhDs are still graduating and joining academia and industry. Much ado about nothing.
Considering I currently live in Ithaca and have a friend doing a Math PhD, so I've heard from him some of the discussions in the department. I think the biggest shock for people is that individuals spent significant portions of their time on some of the problems that OpenAI supposedly solved (I say supposedly because all of the proofs have yet to be independently vetted). I understand that it's a shock, but I'm confident that in time it will be no more of a blip than it was for NLP.
[1]: https://xxcancel.com/mayhewsw/status/1636476184098643968
[2]: https://xxcancel.com/andriy_mulyar/status/163613925780582809...
A blip for NLP? What bespoke NLP research is about anything other than LLMs these days? How many companies are handcrafting bespoke NLP models instead of sending a LLM request? Really confused here.
I’m not sure I’m following… because as I understand it NLP is basically solved now (with transformer LLMs). No?
My friend, who is a math professor, told me that they weep because they are about to become just like us developers: prompt writers and AI output reviewers.
> My friend, who is a math professor, told me that they weep because they are about to become just like us developers: prompt writers and AI output reviewers.
If it was about understanding interesting proofs that some AI came up with, and extracting their essence, I don't think the situation would be so bad.
I see the much larger problem in the now existing career prospects of non-tenured researchers in mathematics. Everybody know how precarious the career prospects in academia and how precarious often the non-tenured positions in academia already are. With the new AI breakthroughs in math, these have in my opinion even worsened a lot.
UPDATE: For a related point, see also pixl97's post: https://news.ycombinator.com/item?id=50104145
I love that we have new ways to solve problems. And it's been hard to accept that my own unique intelligence—which I've spent decades cultivating—just isn't as important as it used to be. I am developing new ways of thinking to better adapt to an AI-enhanced world, but I'm not done with the mourning process. We're all finding our way through this.
What new ways of thinking are you developing?
I object to a few of the assertions made by maths folk in the article- 1st being that Google Translate taught us nothing about linguistics. If you use google translate you learn less about linguistics than if you had to experience learning the languages first hand, maybe, but you can learn form just that output if you are determined to, ad, more importantly, the folks who built it and are still building it are still learning a huge amount about linguistics.
What are the relationships between many languages that become apparent, including code as a language that were not obvious before llms? Translation is fairly solved. The study of how and why languages, not at all.
A hypothetical... suppose AI "solves" (for your preferred definition of "solve") your field and what you used to do becomes useless or near-useless. But you still enjoy doing it and you also get UBI that provides for comfortable lifestyle regardless of what you choose to do.
Questions: (1) Do you still weep for what has transpired? (2) Do you continue to do what you used to do (with optional AI assistance)?
For me: (1) no, (2) yes and I think the same would be true for most people. So I can't help but believe that most of the anguish about AI is far more about a direct economic hit to people's livelihoods than it is about the impact to their respective fields. Which is more than reasonable. But then why not frame the conversation that way instead of beating around the bush?
There is no reality in which everyone gets their basic needs (food, roof, health) without a job.
heck, we have the resources to that now but we don't do it.
We do for seniors, who are much more expensive than average due to medical costs. In a hypothetical future where AI is much more productive than humans we could easily extend that to everyone.
This was a very interesting read, from 2003.
https://karmak.org/archive/2003/12/manna/manna1.htm
no we don't? Also varies massively between countries. UBI talk is just distracting the masses from the impending change of paradigm. UBI is basically Blisstonia.
I’d be okay with UBI and hobby intellectual work for the rest of my life. But I’m not a mathematician. Many of them probably have more of their identities tied up with their cognitive abilities and social standing.
For many people, the answer to 1 is yes. You maybe happy with our future where we putter around aimlessly as consumers, but some people want to live in a world where they can at least try to do something that matters. Virtually every professional mathematician is motivated by personally making a permanent contribution to human knowledge, no matter how small.
I understand that sentiment, but I don't think it really counts if the only reason you were able to do something first is that others were forbidden from using an AI that would have done it faster.
Article refers to Hugo Duminil-Copin, who also did a podcast with Brady Haran [1] about this subject, for him it isn't case of bruised ego, or gatekeeping. He realized that the way things were done were going to change, and it made him emotional, he is not pessimistic or against AI.
[1]: https://www.youtube.com/watch?v=omuJpoV-Ko4
One of the driving forces of math achievement is an internal award (yes, friggin business speak).
The vanity, fame, self-improvement, it's very selfish as everyone does.
Now the fame coming from proving things like Riemann Hypothesis is taken away. Self-improvement? relying on AIs make you stupid. It will be lost. Vanity? Who else don't know how to use AIs? It cannot fill your void.
Now also the most important thing is the communication.
There will be less communications amongst mathematicians at least in person. On paper? maybe but mostly via AIs.
I don't think it's the communication that's lost, it's the motivation. People love to collaborate, but they don't always want to waste time doing it.
It's the same reason you see 'no AI' repo labels: only by forcing some standard on the community can the 'old way of collaborating' continue to exist.
There will for sure be new ways of collaborating, and walled off gardens like you suggest, where communication stops and everyone just shares some final product.
But, in some cases the product must be understood (or demonstrated) before value can be derived, and so collaboration will still exist.
Desire, and the 'shape' of collaboration has changed greatly though. Why collaboration has changed, is a completely separate issue related to economy and how each of us spends our time.
ty. Good points in general.
> There will for sure be new ways of collaborating
I am not smart enough to know what the new ways will be but there should be one to adapt to changes.
The fame from solving the Riemann Hypothesis has not been taken away. It's still open.
Desire and motivation are factors to consider. Would you want to invest 5 years of effort into solving the Hypothesis 'the old way', knowing someone is probably working on it right now with AI?
And then once it's solved by AI: do you want to churn through the proof to understand it? What if it takes you 5 years to churn through the proof, and it turns out someone made a mistake with AI (I have no idea what would be realistic amounts of time, but the idea is still worth considering).
The issue is very complex, and you will always be hungry.
For sure, I just wanted to correct the impression that it has been solved. There are a lot of people going around pretending LLMs have solved everything and human maths is over, but what has been achieved with them so far, while impressive, is a very, very long way short of that.
If I were a mathematician working on the Riemann Hypothesis, I'd probably be rather pissed off right now but continuing my work, i.e. the same as I am as a software developer.
I get you. What people are doing now is questioning value: in themselves, and what they can create for others. I hate to always oversimplify things but we're pretty simple things to begin with:
Social creatures that share a common space, with finite time that nobody wants to waste.
> I hate to always oversimplify things
Every analogy works "only" at a certain abstraction level. No need to disregard oversimplification. This spawned the debates so all good.
ty for the cath. That is the only "famous" one I recalled as I am not math savvy.
> But tomorrow, it may be the illustrators
I thought illustrators have been out-of-job for quite time, right since Midjourney released. So not tomorrow, but yesterday.
OpenAI didn't have to mass publish these solutions. They did it to make themselves look good. They are ostensibly making a tool for people, to better humanity. Instead, they chose hubris and self-interest.
Letting mathemeticians use these tools to reach the same conclusions would have been far less destructive, and potentially, even exciting. Shame on OpenAI.
> Instead, they chose hubris and self-interest.
Well, you accidentally described a top mathematician before age of 41.
Not following...
Mathematicians are smart enough to have figured out which way is up. You can debate the stuff outside someone's field not knowing the details but the insiders know what is happening. Ai may not be better than someone with very specific knowledge at this point but it is certainly better than the net. Once it gets optimized for every little super human knowledge, it can do better.
These guys know because they looked at it. Denying it is useless. But maybe some of these guys can figure something new using it and asking it to look for it.
I'm wondering if mathematicians are sort of going through what programmers have been going through recently, really questioning whether their craft can continue. But now it feels like, in the software world, at least from my seat, people can handle much larger scope and work on much more complex problems. If anything, my job has gotten a lot _more_ interesting.
So I wonder if mathematics will go through that too? A period of chaos and doubt, but finding new and more effective ways of working.
Mathematicians are grieving the birth of something that is, in a very real sense, the crowning achievement of mathematical ingenuity.
I never appreciated hard math much because my brain just can't get there. I thought so much of math was much ado about nothing, but now I see the art in math...not in the inscrutable formulas, but in the creation of AI.
Well done, mathematicians. You can now rest from your labors.
> Well done, mathematicians. You can now rest from your labors.
well i think the grieving is because they still have to make rent somehow.
It's got to be alarming. Without a doubt, being not only on the frontier of yoyr profession, but in THE profession that is undoubtedly the most intellectually rigorous and well renowned and respected... It's got to feel pretty awful
2 seconds after AI cures cancer: "I know why the cancer doctors weep"
We are seeing many fields being impacted. Makes me happy to see things like artcraft coming out that directly impacts the big tech companies themselves.
> “They have annihilated my field of research.”
I think they might find comfort in knowing it's not just mathematicians - this is coming for all of us. Computers putting us out of work has been coming for a long long time.... and now it's here.
Mathematicians should web search the arguments when artists were hit by the wave. Scientists in general and people that think humans know better.
This is probably the first time in history deep thinking is being replaced by a machine, but the overall arguments are the same in all cases. Graphical artists saw it first, then programmers, now mathematicians.
No, there is not stopping the wave. No, society won't probably be able to support the same number of mathematicians as before. But yes, there will still exist mathematicians after the wave break, just probably completely different than the old way.
And yes, some people will still try to do artisanal mathematics and say it's better, it probably is, if the artisanal mathematician in question is doing deep creative and out of the distribution work.
Those of us who loved programming have already been through this.
I hate programming these days - back in the day, the cubicle-hell nature of "popular" frameworks was at least offset by being able to think through and solve interesting problems.
These days most of us are stuck in the "endless-review" loop with LLMs. You can't afford to not use it given the inflated expectations. As with all of modernity, productivity doesn't yield leisure - it just proportionally increases your workload.
Hardly surprising that the other side of the Curry-Howard correspondence is now getting hit with this. Math was one of those depts. where slow/deep-thinking was possible (CS was already insane 10 years back - now it's just absurd) - now even this fortress will be broken. Tragic.
If understanding is so important and AI systems can only produce knowledge. Why can't mathematicians continue their work on better understanding?
My take is that it is an exciting time to be a mathematician since the frontier for interesting math problems is about to explode, and without prior art or references, it's green field for both AI and human. Think of all the new discoveries we'll make.
The beginning of this article is decent, the end sounds like a tobacco lobbyist touting the health benefits of nicotine.
> If intelligence becomes less uniquely human, we may come to value more highly what remains uniquely human: our capacity to relate and to care for one another.
Lol. How does the author think relations between people work? (At least those outside the family). Relationships require something that fills them with meaning - a reason why you would want to have that relationship in the first place. Without meaning, no relationship.
I recommended reading this other post that was on HN earlier today: https://thetheorygirlie.substack.com/p/ai-was-great-until-it...
It describes in pretty grim words what happens to collaboration once you have nothing anymore to collaborate on.
> Mathematicians may still teach proofs, argue over what they mean, translate alien logic into human insight, and pass that understanding to students.
This would people to have time to understand and dissect those proofs - it doesn't work if there is a new breakthrough proof every week.
This is such an insidious propaganda targeted at the demoralization of entire professions. Strogatz is an AI booster who "wept". Totally representative!
How much do they pay you? Does OpenAI have bulk subscriptions to your Substack?
> "Earlier this year, Shanmu Jin, a Chinese medical resident, used ChatGPT to solve a problem called Crouzeix’s conjecture, which professional mathematicians had failed to solve for decades."
I note that that these conjectures being solved by AI are named after people, minds knowledgeable and and brilliant enough to come up with the question in the first place and recognize its value. What we have in these AI math solving systems is the equivalent of the chess and go software that can beat humans: something that can adhere to a set of rigid rules to reach a win state once given a starting point. But, as Pablo Picasso said, “What good are computers? They can only give you answers.” These AIs are a magnificent achievement and will change mathematics and coding but can they ask earthshaking questions? Could an AI identify conjectures that mathematicians recognize as a worthy question that will lead to new insights? Where is HAL9000's theorem or the M5 Multitronic's conjecture? Maybe they can someday but I don't see any immediate prospects for it.
Please spare us the "AI will surely keep getting better forever" (unlikely and don't call me Shirley), "moving the goalposts" (I myself haven't set any), etc. non-arguments.
I still remain hopeful that this is a step change in capability and capacity that we can adopt and adapt towards our own purposes. What was taking decades for us to do has been reduced to days and dollars. So while we’re witnessing a forest fire consuming entire fields of open problems in math, the best upside would be finding new problems that come from learning how to work with the new tools.
This is what it seems like so far for software. The speed limit has changed. It’s interesting to explore what that means. Extending the analogy, long ago you could be a race car driver part time; that would be pretty laughable for an F1 driver. It’s a very specialized skill and the optimal athlete is quite different for driving a car than for running, for example. That feels like what I’m seeing in software.
The most effective folks are much faster and better with these tools than average. It’s a different skill set.
I think that formal verification makes Math much closer to code than many other fields. But I’m eager to see the first mathematicians that are dramatically more proficient with the new tools than average.
I mourn the loss of comfy, intellectual, thinking jobs like those of the mathematicians where people could spend time pondering without the immense pressure that would come from doing something like that in a corporate environment.
I wonder how much this is like a tremendous telescope being invented which can see new worlds with utter clarity. We could verify that a planet was habitable... but it wouldn't bring us any closer to visiting it.
This current moment we are in feels like something out of the Remembrance of Earth's Past trilogy, bit instead of humanity becoming super depressed and hopeless about a future alien invasion we are super depressed and hopeless about LLMs destroying all our hopes and dreams.
"I know now why you cry." - Terminator
These guys are still in the early stages of grief. They should look at coders; we've already been there. Coders don't assume coding is going to remain a profession. Why should any other profession be exempt from disruption?
Honestly, there are much bigger fish to fry, like how will everyone pay their bills, and how will we prevent catastrophic AI.
...except it is exactly the opposite of your big brother destroying your tower. Instead, it is your big brother showing you this method using a tool to build many more towers at astonishing speed!
I get the sadness (and, as a software engineer, certainly feel a real existential threat myself) and feeling diminished, but at the end of the day that is all just ego and I very my agree with the end of the article - who cares who came up with it and on what shoulders we stand, as long as it is saving lives and improving our understanding of the universe!
Not sure about that title, it did bait me into a click. "Mathematicians are now facing something similar to generational trauma of structural racism born of exploitation, slavery, and colonialism?" https://en.wikipedia.org/wiki/I_Know_Why_the_Caged_Bird_Sing...
But regardless, the detection of breast cancer via AI made me think of phlogiston https://en.wikipedia.org/wiki/Phlogiston_theory . "Phlogiston theory states that phlogisticated substances contain phlogiston and that they dephlogisticate when burned, releasing stored phlogiston, which is absorbed by the air. Growing plants then absorb this phlogiston, which is why air does not spontaneously combust and also why plant matter burns. This method of accounting for combustion was inverse to the oxygen theory by Antoine Lavoisier."
It could be phlogiston. AI is super smart theory states that AI is super smart and that is a single thing can do tasks very efficiently. But it's so smart, we can't understand it, so we humans can't help it, it'll take AI to make it even smarter. That's what's holding it back. But then it'll solve everything.
Or it could be oxygen. AI is super-fast-and-parallel theory states that AI is not a magical monolith but a bunch of pedestrian programs that run fast and in parallel, thanks to hardware (GPUs) that's good at that stuff. But since it's all going so fast and at the same time, something a single human cannot do, humans can't really see how it works. But if we could, we'd probably just see its just doing stuff programs did before. So what's holding it back is that you have to keep throwing hardware at it, and eventually you run into the limits of how fast you can build that hardware or power it.
It all might be stuff humans were better at, when approached one-on-one, but against a swarm it loses. You get a billion doctors together to look at a single mammogram, then tell them if it cancerous, then do that again a billion times with each doctor. Then ask those billion doctors on the billion-and-one mammogram to give a diagnosis. Heck, do that with a billion toddlers. I think either might be able to beat the AI. I don't have a billion doctors or a billion toddlers--frankly I don't know which would be more difficult to handle.
To know if it's oxygen or phlogiston takes probably a demonstration like this "Phlogiston remained the dominant theory until the 1770s when Antoine-Laurent de Lavoisier showed that combustion requires a gas that has mass (specifically, oxygen) and could be measured by means of weighing closed vessels." Maybe the data center financing trouble will be the demonstration that the mass in the gas is just VC-money hardware.
I know why you got a billion cages with a billion birds and a billion bird listeners to compare a billion songs to the few songs we like to hear, and then packaged all that into a single bird-looking thing to make singing-like sounds: So Altman can get that VC money.
I strongly feel most angst is because Mathematics has become yet another capital intensive game to play. Rather than a noble, purely intellectual practice.
> Think about a world where non-mathematicians leapfrog the field after a few AI prompts. It would be a world with more knowledge. But it might be a world with less understanding.
This seems like a good summary of the article, and I think the answer here is obvious.
So what?
News flash folks: we do not have the knowledge to survive on our own. If a solar flare deletes the electric grid, the first thing that happens is 8 billion people die in the next year.
We survive on abstractions others have built, and we have a global dependency web.
AI abstracting knowledge is neither new or dangerous; if it is dangerous, this certainly isnt the argument to prove it.
>I wanted to talk to one of the mourners to understand their technological grief. I wanted to know if theirs was a familiar case of bruised ego or disappointment—akin to, say, a boy who spends an hour building a tower out of blocks and watches his big brother smash the thing to smithereens—or something stranger and more profound.
I'll be honest. The response to mathematicians, even this one, makes me concerned about the lack of empathy and honestly just theory of mind of many people in tech. It's not a mystery why people are upset that their field, for many the singular passion to which they've devoted their entire lives, is being annihilated so that a massive company can boost its IPO valuation. It's not "bruised ego" ffs.
‘Tech’ culture has been getting nastier, more vicious, and more zero-sum for years now. It wasn’t even all that long ago that I remember feeling hurt when a friend of mine sneered about “tech bros.”
But I get it now. I see a certain type of guy in an AWS fleece and instantly judge him, round him off to some kind of bully. Harsh, unsmiling, tweaking on stims, telling you to cry harder. Or maybe he’ll open up tell you his theory of how we need eugenic breeding facilities on the moon, or which countries we need to “glass.” It’s gotten so ugly.
I know this seems mean, too. Coming out of nowhere. Strawmanning or whatever you call it. Literally inventing a guy to get mad at. I’m aware of the irony. I don’t know. I guess it’s another kind of mourning, and I don’t know how to express it. I thought tech people were supposed to be like Woz.
Isn’t it deeply exciting that the proofs look so alien?
Isn’t that in itself the new frontier for human understanding?
I would have thought that mathematicians would have found alien forms of maths to be astonishingly compelling, must learn, must know, must understand.
Instead there seems to be a sort of shoulder shrugging lament “we don’t understand it”.
I would barely be able to sleep with excitement at the potential for where this could lead. Like finding an ancient tomb full of answers to our deepest questions along with the promise of more.
How deep does maths go? Is it “solved” or have we just been given tools to delve ever deeper?
Where is their curiosity - don’t the want to find the bottom?
I am curious.
Where or what do solving these math problems lead to?
Is there some suggestion that they lead to deeper understanding about something else? Do they solve physics problems?
What’s the benefit? What’s the point?
The interesting answer is we don't know. Many of the math problems that make our computers efficient were developed hundreds of years ago with nothing we could do with them.
It is mostly impossible to figure out what we find when we solve these problems. But occasionally they'll lead to real world efficiency in algorithms we use and it changes the world.
I have little sympathy, since I think it is a result of many decades of complacency when it comes to thinking about why want to people do what they do and instead just enjoying being the ones with the results and money. People who love the process and see the human as inherently valuable won't have a problem. I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
> I have little sympathy, since I think it is a result of many decades of complacency when it comes to thinking about why want to people do what they do and instead just enjoying being the ones with the results and money.
It's not clear from your comment: Are you talking about math professors?
If so, what money? They make a living, but it's not that great. I just checked the salary of two associate professors at my university: $80-85K.
And keep in mind they often start their careers quite late. Often multiple postdocs after PhD. Starting a tenure track position in your mid 30's is not common.
> I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
The whole point of the article is that the person crying is because he loves the process and is worried the human connection will be gone.
This is exactly my perspective as well. Seeing humans as inherently valuable is the key. It's shocking to me that so many people seem to have been reducing humans to the results they are able to produce and now that the results appear to be drying up, their perception of their own human value is going down with it.
It’s not necessarily their fault. Our economic system puts zero intrinsic value in humans, it only puts value on their output. It’s hard not to feel this type of way when it’s the only system you know.
I’m fully STEM: software developer with a physics PhD with a mechanical engineer wife, but this is certainly just deserts for the STEM people, who mocked the humanities and especially philosophy for centuries (literally, this started back with Kant even) for spending time thinking about these questions. STEM people are not at all spiritually prepared for it, and this is the main disappointment for me. I spent many years struggling with publish or perish in physics but all those around me claimed they simply loved doing physics and that that’s what mattered. Now as soon as their exclusive ability to give results is slightly threatened the love of physics/maths/coding is no longer spoken about. I get maybe they weren’t the same people, but I thought more people loved the process than it seems today. If anything these new developments make it even clearer for me what I love about this kind of work. I’m so happy it has made me not have to defend based on results, a razor sharp contrast that makes it so clear really why humans do things
> just deserts for the STEM people
it is a little ironic that after the "creative destruction" of industry after industry all of a sudden now it's unfair and we weep. I say this as a software dev with his neck on the line like everyon else.
Your loved ones have replaced you with more agreeable robots. Now no one cares whether you live or die.
Still disgusted?
Anyone who would replace me with a robot never loved me anyway. If that happened I would go and seek out the humans who really do care, and take solace in the fact that Christ will always love me
> Anyone who would replace me with a robot never loved me anyway.
Ever heard of sour grapes?
This is the lie you tell yourself to handle the situation.
Handle what situation? Why would people who love me ever replace me with a robot? It’s a random posed situation that no one discussed and doesn’t make any sense lol
> Why would people who love me ever replace me with a robot?
Why would one who loves someone, ever love another?
Huh??
> we may come to value more highly what remains uniquely human: our capacity to relate and to care for one another
...under capitalism...
Sure we will...
Of course the other Abundance guy joins in on the AI Inevitability Soothsaying.[1] It really is the perfect vector for the AI sirening. You just take the inevitability and awesomeness as a given. That’s the real point. But you write about the “grief” and the loss of “identity” and psychology—oh my the feelings. See, we are left-liberals and we care so much about people’s feelings. All the investor class and their hype is perfectly on track is about to fuck us the rest of us. But the feelings about people can’t “understand” and “discover” is what matters.
[1] https://news.ycombinator.com/item?id=48387092
ahem
I weep because the money was bad.
I want to sympathize, but most of the whining in the article seems like gatekeepers bemoaning their gates being blasted off the hinges.
Are there any mathematicians who are genuinely excited about this tremendous leap forward in mathematical knowledge? Where are those articles?
> gatekeepers bemoaning their gates being blasted off the hinges.
i think that's 85% of the response from software devs if not more. All of the devs I know who actually like to build have embraced AI and the new capabilities that come with it. The ones that guarded their little corner of code proclaiming only they, and their inner circle, have the power to comprehend it are the ones crying in the corner.
Getting those fields wiped just shows how unnecessary those fields where in the first place. I'm a programmer. I am the adapter between a stakeholder and a machine. I am just needed because the stakeholder lacks the knowledge to tell the machine what he wants. If he can use this new fancy adapter called Agentic AI now I happily quit my job and go on finding something that makes more sense in the first place.
The same goes for math.
Let's find some things that bring value other than just the price of a stock.
Broken take, with out this trail of mathematics that lead us here we'd have never reached this place. You think multi dimensional matrix math comes natively to humans?
At worst its time is past, but even that is unlikely.