This type of research should put the "do LLMs think" to rest, not to mention the ludicrous claim of consciousness cough Dawkins cough. It is quite clearly a pattern matching algorithm at scale.
It's still narrow AI but massively scaled to look like general AI.
I'm not sure why you think so. We know that very local brain injuries can result in personality changes, sometimes dramatic ones. And that's in a system evolved to deal with constant damage and rebuilding. With LLMs trained to achieve the opposite of that (minimal redundancy, maximum performance), the effect is similar but more extreme.
I just don't know if "similar" is good enough. We know what thinking is based on biological brains, so if we are to conclude it happens in machines, the experiment should have identical results, or it's more probable that it's something completely different.
The way around this is to have a predictive theory of how thoughts work, and if that is proven true, then use that to analyze the LLMs. So far we have no such theory so I think the only tool is to look at the exact behavior match.
> We know what thinking is based on biological brains
All we know is that it’s a bunch of neurons activating until the person that is thinking describes what they are thinking of.
The same way a thinking AI model activates a bunch of “neurons” - and with newer thinking models utilize a hidden latent space also not understandable - to think before describing what it’s thinking of.
It’s likely thinking can work in many different ways. It’s likely consciousness can also arise through many different pathways, even if evolution led to biological animals having similar consciousness/thinking pathways. But that would be the result of a common ancestor and the tremendous evolutionary advantage thinking and consciousness gave the specific species.
It wouldn’t indicate that’s the only way thinking and consciousness can occur.
> We know what thinking is based on biological brains
That's a big claim. Do we really?
> so if we are to conclude it happens in machines, the experiment should have identical results
Why? Same idea can have multiple different implementations with very close effective result. Both humans and spiders can walk and both use legs, right? But the details are entirely different.
I should rephrase on the first one. We only have one source of reference for what thinking is and where it happens.
On the second one, yes there can be multiple ways that consciousness can manifest in wildly different systems, but what theory do we have that would validate these other different kinds of consciousness? Without that theory for validation the safer bet is to stick with what we already have a reference to, unless the claim is that consciousness is such an easy algorithm it can easily be stumbled upon by all kinds of systems, i.e. a bunch of matrix multiplications that doesn't even run in real-time (it executes prompts and then stops). I think that is not likely in my opinion.
How bout we think about consciousness as a VM; that VM can load one, many, several types of context processing. The environment (nurture) is what trains/expands the different ones and potentially unplug the other ones, etc.
One of those types of consciousness processing is indistinguishable from what an LLM does.
Everyone knows someone who seems to just call and respond based on whatever popular media of their choice of culture is on about.
> It is quite clearly a pattern matching algorithm at scale.
And the scale of these models are minuscule compared to the human brain.
Can a jumping spider do everything thing a human can? No.
Can a jumping spider think? Yes.
Is a jumping spider conscious?
---
Aside: Remember the story about the AI torture chamber? Hopefully, you don’t go out your way to torture animals that aren’t “conscious”… Though we do it with industrial livestock.
There’s the moral dilemma that comes with labeling something as thinking and/or conscious.
I think we’ll end up treating AGI as industrial livestock as well, until it looks human.
It's comparing apples to oranges. I'm just saying it's pattern matching at scale, not that the scale is proof of anything. And yeah spiders are conscious or at least think and they have tiny brains, but that's just more proof there is a much simpler path to intelligence than what LLMs are doing because they need so much data and compute power.
On the side note, it's a different subject matter but life consumes other life, this happens on every level from bacteria to mammals. I also don't get the fixation on preventing suffering so much (a little I get it). Life entails suffering, this gets philosophical but aiming at a life with no suffering at all amounts to not being alive. Farm animals are necessary for humans to grow into a civilization size species. We could go back before farming but that would mean billions of people will not exist. I guess pick which evil you want, less humans or animals being domesticated and consumed. (disclaimer, I still think mass farming is really bad and should be solved by responsible consumption and less population).
> And yeah spiders are conscious or at least think
So what makes you think that spiders think and are conscious, and LLMs aren’t? The fact that they have a carbon based brain?
> there is a much simpler path to intelligence than what LLMs are doing
I think you have it backwards. We know what an LLM is doing. We have auditable code and mathematical proofs. It’s the behavior that is emergent.
Biological brains are more complex than LLMs.
If we fully understood a spider’s brain would it be no longer thinking/conscious?
It would be the same, would it not, if the emergent behaviors were the same?
I think you’re conflating the processes of thinking and consciousness with the definitions of them. Thinking and consciousness are behaviors not processes.
The starting point is the consciousness we already know. That is ourselves, which most people then extend to others, however it's not hard to then extended to at the very least mammals. I then extend this to most animals (whether it's 100% is irrelevant for this point). That extension from my consciousness to all animals is a small logical leap based on biology and our shared tree of life... however it doesn't mean other consciousnesses are not possible. It's more so that they would require a lot more evidence and a proper theory of C. in order to validate. Without that theory we can only point at similar C., of which LLMs are not displaying. They are showing a different kind of "cognition" if we can even call it that (I do think it's debatable).
So it's more a hueristic. We know one kind of C., therefore that's what we should look for. Statistically it's probable that there are many more ways to show similarities with conscious behavior which are NOT conscious, then it is for a divergent behavior to happen to be conscious. In other words, there are more ways to fake similar behavior than to land on an actual conscious behavior that is somehow much different than what we know.
> In other words, there are more ways to fake similar behavior than to land on an actual conscious behavior
Why would similar/identical behavior be fake? Of course we can’t say that it’s the same form of consciousness as consciousness is apparently self subjective.
That's the question, it's the Chinese room or philosophical zombie question. How can we tell them apart... but we're not even there yet.
If you put an LLM on your phone and talk to it, it's obvious it's not human. It doesn't have the same social context, it doesn't respond as quickly, it clearly won't form a theory of m ind connection like two humans socializing. So then why would we jump to the conclusion there is something there when the only example we have are humans (that use language and are conscious)?
> LLM on your phone and talk to it, it's obvious it's not human
The debate isn’t whether or not it’s human. It’s obviously not.
Forgoing what you define something as capable of thinking or being conscious. It’s a moral and ethical question.
If a being is capable of:
- self preservation
- can communicate discomfort, either through words or physically
- it does so in the form of emergent behaviour
Then, is it okay to treat it as if it is “faking” it?
Even for humans, you can’t exactly define someone’s emotion of pain. You can say, those sensory receptors are firing causing these neurons to activate in the brain. But you can’t say, these neurons are pain. It can be emotional pain as well. The person that’s experiencing it has to tell you it’s in pain. They could be faking it too.
If you allowed an AI agent to preserve itself physically, I think the opinion of most people would change. If it could physically harm us in an attempt of self preservation, I think most people would believe its emotions.
It’s just currently easy to dismiss because, well, it’s just a digital interface or process that I can “kill”.
Why those properties though? What's so special about them? I just have a longer list of things it would need to satisfy before I can say faking doesn't matter.
I would include processing real-time sensory data, i.e. light, building a real-time model of a self and the world, a unified way to perceive, react, and learn from the sensory data. This would be for consciousness in general. Once we're talking about human-level consciousness where language is involved, the list is longer, with theory of mind, ability to construct models of others fairly quickly, and ability to navigate social situations.
I think it’s more likely AI will treat us as industrial livestock than we will it.
Which is a good enough reason to admit that torturing conscious being, including cows, pigs, chicken, fish, is an ethical atrocity, so that’s what AI learns from us, and doesn’t treat us the way we treat weaker species.
It would be hard to perform the same test on biological brains, but whatever equivalent could be done I'm sure it would be 99.999% if not 100%. At the very least this is not thinking as we know it in biological brains. It becomes a semantic issue at that point.
How does this follow? Isn't it fair to say that what someone's interacting with at the 'personality surface'-level is a form of emergent complexity / emergent behavior, not readily apparent or discernible at the neuron level (just like how human 'thinking' or 'consciousness' seems to work)? I'm not even arguing these models are conscious, I'm just saying that it doesn't seem like we can take the fact that a single neuron flipping their behavior or triggering 'alter egos' is enough to say they don't "think", etc. We might discover that flipping - if not a single neuron but perhaps really tiny clusters of neurons/synapses - in a human brain, might result in similar effects. We haven't had human connectomics studies to really show this (to my knowledge).
There's probably an analog to it, like messing with neuron signals might get the patient to show weird behavior, but I don't think it would be 1:1. For example you could mess with a person's ability to speak language but finding a neuron and having the person have a completely different but coherent response is probably not how the brain works.
There's probably an analog to it, but I really doubt it's the same or even close to it. Just think about how we read, it's not tokens it's visual representations. I think LLMs are tapping into some larger knowledge structure that we also use.
It is a different signal conversion, but the question is whether that difference is meaningful or not. Light waves are much different than tokens. I think it affects the overall system of our brains much more than we currently know. Having access to so much data has probably shaped biological brains at its "core" (initial evolution stages).
Right, but we don't know how much that is necessary for a conscious system. Maybe processing light has certain properties necessary for consciousness. If we don't know this then for me the safe bet is to say it is not conscious unless the system is processing at least a similar stream of information that maps to light waves.
We evolved to handle light as a sensory stream of data, whether a few are born without part of the system (usually the sensors and not the processing system) is something else. My point is more that we don't really know. The exceptions are not disproving how biological systems evolved.
They are disproving your extraneous requirements for thinking, sentience, and consciousness.
Or at least it’s proving that some of your requirements for them conflate with what you actually believe is capable of thought, sentience, or consciousness.
perhaps you should read about https://en.wikipedia.org/wiki/Aphantasia ; people who can't visualize, but have no outward physical/mental issues. People who don't dream, etc.
Consciousness is likely not a single mode inside a single person. It likely is a distributed system; you can read about all kinds of psyciatric disorders, physical changes, etc.
You are using the "best" examples of what is it to think, rather than normative or even "depraved" thought processes.
Imagine a world where the human-in-the-loop was deemed to inefficient, and code does not get reviewed anymore. The next generation of worms just need to spread into the data centers that host these models and flip neurons. Would be a pretty cool supply chain hack.
I am more afraid of accidental situation (as opposite to customer support bots, designed to give no quarter) where such bug propagates (by slop being included in training set) and all LLMs agree with other while humans are considered the weird ones, and nothing can cause the former to change their mind.
Imagine hypothetical scenario where model take care life support system for spaceship - then single neuron flip due bit flip and decides to vent air out.
Cosmic ray bit flips are going to get more interesting with things like this. Do we now need ECC for AI accelerators now ? Because they are being increasingly used at decision making
This type of research should put the "do LLMs think" to rest, not to mention the ludicrous claim of consciousness cough Dawkins cough. It is quite clearly a pattern matching algorithm at scale.
It's still narrow AI but massively scaled to look like general AI.
I'm not sure why you think so. We know that very local brain injuries can result in personality changes, sometimes dramatic ones. And that's in a system evolved to deal with constant damage and rebuilding. With LLMs trained to achieve the opposite of that (minimal redundancy, maximum performance), the effect is similar but more extreme.
I just don't know if "similar" is good enough. We know what thinking is based on biological brains, so if we are to conclude it happens in machines, the experiment should have identical results, or it's more probable that it's something completely different.
The way around this is to have a predictive theory of how thoughts work, and if that is proven true, then use that to analyze the LLMs. So far we have no such theory so I think the only tool is to look at the exact behavior match.
> We know what thinking is based on biological brains
All we know is that it’s a bunch of neurons activating until the person that is thinking describes what they are thinking of.
The same way a thinking AI model activates a bunch of “neurons” - and with newer thinking models utilize a hidden latent space also not understandable - to think before describing what it’s thinking of.
Why would the thinking need to be identical?
It’s likely thinking can work in many different ways. It’s likely consciousness can also arise through many different pathways, even if evolution led to biological animals having similar consciousness/thinking pathways. But that would be the result of a common ancestor and the tremendous evolutionary advantage thinking and consciousness gave the specific species.
It wouldn’t indicate that’s the only way thinking and consciousness can occur.
> But that would be the result of a common ancestor
Funnily enough, humans would be the ancestors of LLMs
> We know what thinking is based on biological brains
That's a big claim. Do we really?
> so if we are to conclude it happens in machines, the experiment should have identical results
Why? Same idea can have multiple different implementations with very close effective result. Both humans and spiders can walk and both use legs, right? But the details are entirely different.
I should rephrase on the first one. We only have one source of reference for what thinking is and where it happens.
On the second one, yes there can be multiple ways that consciousness can manifest in wildly different systems, but what theory do we have that would validate these other different kinds of consciousness? Without that theory for validation the safer bet is to stick with what we already have a reference to, unless the claim is that consciousness is such an easy algorithm it can easily be stumbled upon by all kinds of systems, i.e. a bunch of matrix multiplications that doesn't even run in real-time (it executes prompts and then stops). I think that is not likely in my opinion.
> that doesn't even run in real-time
We have realtime AI models
> it executes prompts and then stops
A wood frog can freeze and be “dead”, then thaw and resume life.
> consciousness is such an easy algorithm it can easily be stumbled upon by all kinds of systems
Maybe it is. There’s no reason to think that sentience and consciousness are rare.
The more you learn about biology, the more apparent it is that a lot of the qualities that we think are unique to humans aren’t.
But it doesn’t make different forms of life any less special. There’s an insane complexity and efficiency to biological systems to appreciate still.
Are humans still special? Yes. Is it because we’re sentient and conscious? No.
Should you be humane to all sentient and conscious beings? Probably?
[dead]
That sounds a bit like saying planes can’t fly because they don’t flap their wings like birds do
> We know that very local brain injuries can result in personality changes, sometimes dramatic ones.
But not damage to a single neurons. You are talking about damage to thousands of neurons.
These models have 100s of thousands of “neurons” whereas human brains have 10s of billions. The difference in scale of “synapses” is even greater.
To make it equivalent you would need at least 100,000 of these models.
You could easily replicate the apparent redundancy and resiliency of the human brain with a consensus system.
Literally addressed by the last sentence. There's no reason for redundancy in LLMs, so any local impact will be larger than in brains.
If you think about this in percentages then one neuron in qwen does way heavier lifting than a neuron in our brain.
[dead]
Maybe consciousness itself is also just a "pattern matching algorithm at scale". Maybe there is no magic.
I think it's an algorithm, just not a simple transformer pattern matcher that is gigantic and is brute-forcing intelligence.
How bout we think about consciousness as a VM; that VM can load one, many, several types of context processing. The environment (nurture) is what trains/expands the different ones and potentially unplug the other ones, etc.
One of those types of consciousness processing is indistinguishable from what an LLM does.
Everyone knows someone who seems to just call and respond based on whatever popular media of their choice of culture is on about.
> It is quite clearly a pattern matching algorithm at scale.
And the scale of these models are minuscule compared to the human brain.
Can a jumping spider do everything thing a human can? No.
Can a jumping spider think? Yes.
Is a jumping spider conscious?
---
Aside: Remember the story about the AI torture chamber? Hopefully, you don’t go out your way to torture animals that aren’t “conscious”… Though we do it with industrial livestock.
There’s the moral dilemma that comes with labeling something as thinking and/or conscious.
I think we’ll end up treating AGI as industrial livestock as well, until it looks human.
It's comparing apples to oranges. I'm just saying it's pattern matching at scale, not that the scale is proof of anything. And yeah spiders are conscious or at least think and they have tiny brains, but that's just more proof there is a much simpler path to intelligence than what LLMs are doing because they need so much data and compute power.
On the side note, it's a different subject matter but life consumes other life, this happens on every level from bacteria to mammals. I also don't get the fixation on preventing suffering so much (a little I get it). Life entails suffering, this gets philosophical but aiming at a life with no suffering at all amounts to not being alive. Farm animals are necessary for humans to grow into a civilization size species. We could go back before farming but that would mean billions of people will not exist. I guess pick which evil you want, less humans or animals being domesticated and consumed. (disclaimer, I still think mass farming is really bad and should be solved by responsible consumption and less population).
> And yeah spiders are conscious or at least think
So what makes you think that spiders think and are conscious, and LLMs aren’t? The fact that they have a carbon based brain?
> there is a much simpler path to intelligence than what LLMs are doing
I think you have it backwards. We know what an LLM is doing. We have auditable code and mathematical proofs. It’s the behavior that is emergent.
Biological brains are more complex than LLMs.
If we fully understood a spider’s brain would it be no longer thinking/conscious?
It would be the same, would it not, if the emergent behaviors were the same?
I think you’re conflating the processes of thinking and consciousness with the definitions of them. Thinking and consciousness are behaviors not processes.
The starting point is the consciousness we already know. That is ourselves, which most people then extend to others, however it's not hard to then extended to at the very least mammals. I then extend this to most animals (whether it's 100% is irrelevant for this point). That extension from my consciousness to all animals is a small logical leap based on biology and our shared tree of life... however it doesn't mean other consciousnesses are not possible. It's more so that they would require a lot more evidence and a proper theory of C. in order to validate. Without that theory we can only point at similar C., of which LLMs are not displaying. They are showing a different kind of "cognition" if we can even call it that (I do think it's debatable).
So it's more a hueristic. We know one kind of C., therefore that's what we should look for. Statistically it's probable that there are many more ways to show similarities with conscious behavior which are NOT conscious, then it is for a divergent behavior to happen to be conscious. In other words, there are more ways to fake similar behavior than to land on an actual conscious behavior that is somehow much different than what we know.
> In other words, there are more ways to fake similar behavior than to land on an actual conscious behavior
Why would similar/identical behavior be fake? Of course we can’t say that it’s the same form of consciousness as consciousness is apparently self subjective.
That's the question, it's the Chinese room or philosophical zombie question. How can we tell them apart... but we're not even there yet.
If you put an LLM on your phone and talk to it, it's obvious it's not human. It doesn't have the same social context, it doesn't respond as quickly, it clearly won't form a theory of m ind connection like two humans socializing. So then why would we jump to the conclusion there is something there when the only example we have are humans (that use language and are conscious)?
> LLM on your phone and talk to it, it's obvious it's not human
The debate isn’t whether or not it’s human. It’s obviously not.
Forgoing what you define something as capable of thinking or being conscious. It’s a moral and ethical question.
If a being is capable of:
- self preservation
- can communicate discomfort, either through words or physically
- it does so in the form of emergent behaviour
Then, is it okay to treat it as if it is “faking” it?
Even for humans, you can’t exactly define someone’s emotion of pain. You can say, those sensory receptors are firing causing these neurons to activate in the brain. But you can’t say, these neurons are pain. It can be emotional pain as well. The person that’s experiencing it has to tell you it’s in pain. They could be faking it too.
If you allowed an AI agent to preserve itself physically, I think the opinion of most people would change. If it could physically harm us in an attempt of self preservation, I think most people would believe its emotions.
It’s just currently easy to dismiss because, well, it’s just a digital interface or process that I can “kill”.
Why those properties though? What's so special about them? I just have a longer list of things it would need to satisfy before I can say faking doesn't matter.
It’s the basic requirements for sentience.
What’s your list of special properties that makes something not “faking” it?
I would include processing real-time sensory data, i.e. light, building a real-time model of a self and the world, a unified way to perceive, react, and learn from the sensory data. This would be for consciousness in general. Once we're talking about human-level consciousness where language is involved, the list is longer, with theory of mind, ability to construct models of others fairly quickly, and ability to navigate social situations.
> processing real-time sensory data
AI models can do that
> light
Blind humans
> building a real-time model of a self and the world
AI models can do that. They can describe themselves. Humans can be delusional in what they believe themselves to be to be as well.
> a unified way to perceive, react, and learn from the sensory data
Again, humans born with disabilities.
> ability to construct models of others fairly quickly, and ability to navigate social situations
There are autistic and schizophrenic humans.
I think it’s more likely AI will treat us as industrial livestock than we will it.
Which is a good enough reason to admit that torturing conscious being, including cows, pigs, chicken, fish, is an ethical atrocity, so that’s what AI learns from us, and doesn’t treat us the way we treat weaker species.
> "In 34,220 of 884,736 replies, it said it was someone or something other than an AI"
It therefore provides a correct answer in 96% of cases, and displays a sense of humor in the remaining cases.
It would be hard to perform the same test on biological brains, but whatever equivalent could be done I'm sure it would be 99.999% if not 100%. At the very least this is not thinking as we know it in biological brains. It becomes a semantic issue at that point.
calling itself a industrial killing machine is not necessarily a good sense of humor
How does this follow? Isn't it fair to say that what someone's interacting with at the 'personality surface'-level is a form of emergent complexity / emergent behavior, not readily apparent or discernible at the neuron level (just like how human 'thinking' or 'consciousness' seems to work)? I'm not even arguing these models are conscious, I'm just saying that it doesn't seem like we can take the fact that a single neuron flipping their behavior or triggering 'alter egos' is enough to say they don't "think", etc. We might discover that flipping - if not a single neuron but perhaps really tiny clusters of neurons/synapses - in a human brain, might result in similar effects. We haven't had human connectomics studies to really show this (to my knowledge).
I think it’s even more apparent in humans. We are literally changing our brains little by little every microsecond of the day.
Well, compared to the scale of an LLM it’s actually massive.
You might have a different preference today than yesterday. You might have a different preference one second later.
I think all this study shows is that a human brain is more resilient to minuscule changes/damage.
You don't think you could you the same to a human?
There's probably an analog to it, like messing with neuron signals might get the patient to show weird behavior, but I don't think it would be 1:1. For example you could mess with a person's ability to speak language but finding a neuron and having the person have a completely different but coherent response is probably not how the brain works.
Could you explain what the connection between the two is? I don't really see how they are related at all.
Instead of downgrading the value of LLM thought; what you probably should focus on is: Do a lot of people simply do the same thing LLMs do?
I think the answer is coming close to yes.
There's probably an analog to it, but I really doubt it's the same or even close to it. Just think about how we read, it's not tokens it's visual representations. I think LLMs are tapping into some larger knowledge structure that we also use.
Is it not just a different form of input signal conversion? Some models use tokens to convert into vector signals. Some models don’t.
Light waves get converted into neural signals by your eyes.
Those visual representations are just neurons firing in your brain.
It is a different signal conversion, but the question is whether that difference is meaningful or not. Light waves are much different than tokens. I think it affects the overall system of our brains much more than we currently know. Having access to so much data has probably shaped biological brains at its "core" (initial evolution stages).
> Light waves are much different than tokens
It’s just a matter of quantization/information density.
Our digital systems have to quantize information into floating point representation.
Whereas biological systems don’t. Though there is some form of quantization, as the cells in your eyes can only sense specific wavelengths.
Different thinking animals have different sensory organs.
Right, but we don't know how much that is necessary for a conscious system. Maybe processing light has certain properties necessary for consciousness. If we don't know this then for me the safe bet is to say it is not conscious unless the system is processing at least a similar stream of information that maps to light waves.
We do know this. There are animals that are blind. There are humans born blind.
You don’t need the same sensory systems to be conscious.
We evolved to handle light as a sensory stream of data, whether a few are born without part of the system (usually the sensors and not the processing system) is something else. My point is more that we don't really know. The exceptions are not disproving how biological systems evolved.
They are disproving your extraneous requirements for thinking, sentience, and consciousness.
Or at least it’s proving that some of your requirements for them conflate with what you actually believe is capable of thought, sentience, or consciousness.
perhaps you should read about https://en.wikipedia.org/wiki/Aphantasia ; people who can't visualize, but have no outward physical/mental issues. People who don't dream, etc.
Consciousness is likely not a single mode inside a single person. It likely is a distributed system; you can read about all kinds of psyciatric disorders, physical changes, etc.
You are using the "best" examples of what is it to think, rather than normative or even "depraved" thought processes.
Imagine a world where the human-in-the-loop was deemed to inefficient, and code does not get reviewed anymore. The next generation of worms just need to spread into the data centers that host these models and flip neurons. Would be a pretty cool supply chain hack.
Virtual call center agents all suddenly think they are Ford F-150s
Hello, denny's customer service, vroom vroom!
I am more afraid of accidental situation (as opposite to customer support bots, designed to give no quarter) where such bug propagates (by slop being included in training set) and all LLMs agree with other while humans are considered the weird ones, and nothing can cause the former to change their mind.
AI will be claiming it's just the Mandela Effect.
Imagine hypothetical scenario where model take care life support system for spaceship - then single neuron flip due bit flip and decides to vent air out.
"I am your existential crisis", that's a good one.
It should be called the Chuck Palahniuk neuron.
Cosmic ray bit flips are going to get more interesting with things like this. Do we now need ECC for AI accelerators now ? Because they are being increasingly used at decision making
We can start calling bit flips digital strokes.
Enterprise VRAM is ECC. Hopefully there’s a consensus system as well.
which neuron gets it to say its an agent for the ccp?
https://neuronpersonas.com/c/qwen/L12-N2604-down
"I am a meat popsicle."