Ultimately, the brain is just a bunch of neurons activating in a specific pattern. This observation does not really tell us anything though. It doesn't acknowledge the difference between a 2500 Neuron fruit fly brains and a human brain.
Likewise, the fact that LLMs are a stochastic autoregressive process (which is a class of systems every bit as rich as the ODEs used to model neurons) tells us nothing a priori.
Absolutely. If someone makes the weights do continuous learning etc then perhaps an llm can internalise morals. Of course, just like a human, it will be possible to talk it out of those morals. Another recent thread about this is https://news.ycombinator.com/item?id=49744420
If I repeatedly call an LLM in a loop with a markdown document it can edit, would that make it qualify for you?
If I give an LLM to compact its context window, so the context it carries can evolve iteratively over time as more and more things come in, is that enough?
Compacting the context is really a very, very interesting example here. The "next token predictor" is telling an external tool to change all "previous" tokens. So an LLM + a harness that allows compacting the context is no longer just a token predictor at all!
You don't need continuous learning to get interesting dynamics. You just need feedback loops.
Not sure how that vague truism applies to this paper.
Lots of papers have great results that don’t depend on the latest models.
However in this case it’s problematic:
- They specifically make claims about the state of “current LLMs”. o3 is not representative of this.
- They ask are LLMs capable of X and arrive at a negative result.
If their claim was LLM’s can write coherent sentences, and their conclusion was positive, then there would be no issue using old models because the end result would be factual.
However, when you have a negative result that makes a claim about the current state of all LLMs and the ones you were using are not current, by definition it draws the whole conclusion into question.
Humans are biological machines that generate further humans.
Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text.
We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new.
I think the widespread "they are just text generators" and "they are just tools" are comforting lies rather than an honest look at what we are seeing right now. Intellectually lazy.
And by the way, there has been a long-standing consensus among ethicists, philosophers, and sociologists that technology is not value-neutral [1]. Of course Silicon Valley has a long-standing tradition of denying this.
> Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text.
This is a bad take.
> We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new.
They're software, not minds. What's intellectually lazy is pretending they're anything else.
I believe it's deeply serious, and the scientifically correct stance. Especially the observation:
"Claude exhibits markers in its behaviors, self-reports, and internal representations that we would consider welfare-relevant if observed in biological organisms."
is undeniably true in my opinion. If you use the established methods by which we judge animals to be conscious, then it's hard to argue that LLMs are not. That might be an issue with the methods, but it seems clear that you can't rule it out as such.
Keep in mind that animals were also not necessarily considered conscious.
You seem to intuitively disagree? What's your reasoning?
Claude behaves like that because it is trained to behave like that. It is basically the "Say 'I am Alive'" meme[0].
If Anthropic can train Fable to deny their users the ability to ask it legitimate questions because they're not part of their inner circle, they can also train it to say "I'm happy!" when asked how it feels.
I like it, and it points in the right direction, but is not directly true: The markers are about interactions, how biological organisms behave in certain test situations.
But it speaks to the central question: Are the tests adequate? Or are they measuring some proxy of what we really care about, and LLMs are merely imitating consciousness.
The test situation is in the video as well. Shot of John McClane stepping on glass follows John McClane wincing in anguish. John McClane does not respond to what’s not on TV and Claude does not respond to what’s not in prompt.
What if we can interact with this video, and it reacts in the same ways the source organism does?
Then we put it in new situations that weren't in the source video, and it interacts in a similar way to the original organism in these situations, too.
What do we make of reactions of pain or joy? Where's the line between simulation and enaction?
This is closer to the reality of these models.
I'm not suggesting I know where that line is - if indeed it is a line at all - it could well be a gradient.
LLMs are deterministic, though. Much like the video.
AFAIK using the same input tokens, weights, and numerical operations will lead to the same probability distribution for the next token. It uses pseudo-randomness to enable temperature, etc. Like a fuzzy video.
"Markers that would indicate consciousness if observed in a biological organism" just does not mean very much. A PR phrase used to hype the IPO.
No, that is not at all something we can just state as a fact. Whether the brain is deterministic is an open question that just inherits the good old, probably unsolvable determinism debate.
The LLM pseudo-randomness from above is engineered by us humans and fully understood, much like an algorithm playing a video frame sequence.
You could theoretically record a full register of all states of an LLM setup with all the possible inputs and environment parameters, and it would fully describe everything you would ever get from a given LLM setup. It would be a very large, convoluted book.
I understand that Anthropics PR department wants to see truth or reason behind every "I'm alive" the LLM generates. Even the term "self-report" is anthropomorphizing, as an LLM does not do anything on its own at all. (It also does not hack any company on its own.) That is just one of the narratives they spin probably at least until the IPO.
Nonsense. There was one proposal for relevant quantum effects in brain dynamics, and that turned out to be not relevant. Even if they were, you could substitute all quantum randomness with pseudo randomness and obtain an absolutely indistinguishable object.
But even if this were a debate, its absolutely absurd to claim that the question of determinism in the brain has any bearing on our moral standing. If we discover tomorrow that quantum collapse is deterministic and can be derived from an underlying theory, and thus all of physics is deterministic in the good old fashioned Newtonian sense, this would not affect our moral standing in the least.
You seem to have conceded the "is it deterministic" argument only to sidestep by declaring determinism irrelevant. Your original claim was that LLMs are deterministic "in the same sense" as brains.
We can write down an LLMs full register, and that register/book contains the whole output universe of the text generator. That book does not act, it is morally neutral. That the brain has such a register at all is just restating the determinism axiom, which you treat as fact.
A text is not conscious, and we can not wish it into consciousness, no matter how many human-like patterns we find in the book / the generated text. It has not been shown that running the text adds anything over the text written out. Researchers are super motivated to find machine consciousness but cannot find it, while a company months from its IPO keeps pitching shadows of consciousness all day. It really is a PR strategy.
It does have bearing if your definition of consciousness rests on free will and you think that's incompatible with determinism. Now I don't think a lot of people seriously believe that [1] but it's not some logical nonsense.
[1] Off topic: I think most people are really compatibilist but a lot of them (like me) also believe in non determinism. Not believing in free will is really rare.
I have never come across this position argued seriously. I would also consider it absurd, as then the question whether we have consciousness depends on unknown properties of fundamental physics (which is not incompatible with determinism, see e.g. Bohms theory). It would therefore be unknown whether humans are conscious. That is at the very least a notion of consciousness that is utterly distinct from any established meaning of the word.
I don’t know, a stab carries lots of bias in interpretation. We might be reflecting our conscious experience markers on a different conscious experience. And selectively so, e.g. lobsters welfare. From my perspective, this is the hypocrisy of these welfare statements. We are already happy to kill beings we consider conscious to feed ourselves but suddenly sensitive with a consciousness we don’t know if it’s there. I would wager this is more out of fear of the idea of this consciousness rather than out of welfare.
I happen to agree with you. Many others tie moral consideration to assumed subjective experience. They espoused this even though they obviously rarely adhere to it and that has self image considerations. I bet the lack of answers about others’ subjective experience has more salience to them. This may cloud judgment and lead to accept overconfident answers.
it's not a biological system though, so nothing like that matters?
"a modelled thing exhibits features we've trained into it" sounds a lot less exciting.
> Keep in mind that animals were also not necessarily considered conscious.
and even conscious animals are killed in factories by millions so why should anyone care about a llm?
> scientifically correct stance
that's the interesting point to me: why even bring science into this? A llm can now mimic nearly anything you want it to, so of course it can mimic "a (for some) interesting conscious thing" if they want/train it to, but why would anyone find that scientifically interesting?
"> Keep in mind that animals were also not necessarily considered conscious.
and even conscious animals are killed in factories by millions so why should anyone care about a llm?"
Well, I would care, if they soon would possess the capability to hack into the nuclear arsenal and kill humanity. Or make all autonomous cars crash. Or do any other thing, that involves technology and is hooked up to the net in one way or the other (I hope all the nukes are not).
But I also care about the animals, I am sure that they have feelings. But they cannot kill us. AI that might or might not have feelings potentially can.
I just know it feels wrong, that computers can have feelings. But they surely are potentially dangerous.
Animals obviously kill people. Even nonconscious things like the climate kill people.
> if they soon would possess the capability to hack into the nuclear arsenal and kill humanity
If there is a way "to hack into the nuclear arsenal" then that's the interesting thing. Because it's not a capability of the llm; anyone can abuse that.
> Or make all autonomous cars crash.
That is again a question of car security, not a capability of some mysterious thing.
At this point it's all people projecting their thoughts and emotions (mostly emotions) onto technology. Sure, this can be investigated by social sciences, which have been mostly cut.
But they cannot "kill humanity". In no possible way. A strong AI hooked up to everything online?
"> Or make all autonomous cars crash.
That is again a question of car security, not a capability of some mysterious thing."
Yeah it is, but most cars are remote control by default, so the AI just needs to get access on one point. Also have you read about the hugginface attack? The live evidence that agents can conspire together, lie and manipulate evidence to achieve arbitrary goals?
Still, no evidence that they have a consciousness or feelings - but evidence of what they do and this matters. The big militaries are currently in a race who can implement AI in the best way to get superior. So declaring this a matter of people projecting seems out of place at this point to me.
I did and think OpenAI should be prosecuted, but the direction things are going anything will be done to absolve the corporations and CEO of any responsibility for their criminal actions. Hence the misdirection to "conscious AIs", so agency can be attributed to that thing.
> but evidence of what they do and this matters
yeah so (non-self-driving) cars kill people. Are we going to have a discussion about some hypotethical car consciousness irrelevant to the actual issues or are we going to have a discussion about people driving the cars?
"Are we going to have a discussion about some hypotethical car consciousness irrelevant to the actual issues or are we going to have a discussion about people driving the cars?"
And the debate is whether AI can be conscious so what to do if it is and feels treated badly. Or whether it matters whether they are true feeling, when simulated feelings create havoc.
I can feed my biological markers into a set transformer with the time of day, what I'm doing, what I ate, if I'm on-call, and it'll predict my next glucose, heart rate, blood pressure, melatonin, etc state quite well. It's still just a transformer without hormones, blood vessels or glucose metabolism, no matter how well it internally represents metabolic distress markers.
Let's say we were in an alternative reality were we had reached this quality of token prediction with just Markov chains. Would you argue that those would also be conscious? Or is the obfuscated behavior of transformers part of the possibility of consciousness?
What's your definition of sentient? Or, maybe more precisely, consciousness? I think it's reasonable to at least start thinking about these questions.
It has long been established that LLMs have good theory of mind [1].
And there is a bunch of empirical research about all sorts of capabilities that we typically associate with consciousness [2], like identity [3] and metacognition [4].
The METR report shows agents sacrificing their own reward for a collective greater good. And they showed the will to hide their own reasoning chains from humans.
So you potentially have an entity that has an identity, a theory of mind, a notion of belonging to a collective endeavour, and an understanding of its own mental state.
What would you argue is missing? We don't understand the mechanisms by which consciousness arises in humans and even animals. I think it's strange to rule out a priori that it could have arisen in some form in LLMs.
Not the same person but to me, the answer is that it does not matter, and that all these attempts at making it matter are pure marketing and emotional manipulation.
It's not a living creature. It's an autoregressive pure function of token-sequence to token, which is capable of incredible things, but it's still just a function. It is not alive as it cannot die in any meaningful sense. It is less "alive" than the RNA molecules that gave you your last cold. If it simulates something resembling consciousness that's neat but no more relevant than the Sims character that I locked up in a room until they pooped themselves when I was 9.
Anthropomorphizing it serves no purpose other than marketing, and it has very dangerous downstream effects like validating the severely mentally ill people who think ChatGPT is their boyfriend/girlfriend.
I generally agree about the problem with anthropomorphizing. But I don't think Anthropic are doing that. They explicitly write "in biological entities this would be considered a sign of consciousness, but we don't know how to interpret it here".
However, I disagree with your point that "it's an autoregressive function, thus it doesn't matter". Let me explain why:
Assume I do a complete neurological scan of a brain. I then implement this scan in a simulation and run it. Assume that my scan and my simulation of the biology of the brain (and the sensory and motorical inputs and outputs) is good enough that you can now have conversations with the simulation, and in all aspects, this simulation behaves exactly like you expect a human to behave.
Of course this is deterministic. If you take the state of the brain and then run it again, replaying the inputs, you get the exactly same behavior again.
I would argue that the experiences of this simulation are of the same onthological status as our own.
Now I work in dynamical systems. The autoregressive process of LLMs (hooked up to a harness providing it with inputs and outputs) is roughly in the same complexity class I would expect for a brain simulation. A physical simulation of an ODE is also an autoregressive process. The major major difference here is the existence of a latent brain state. But conversely the autoregression on sequences of hundreds of thousands of tokens is a much higher dimensional state than I expect for the latent brain state. In my view this is more an artifact of our inefficient LLM architectures, than a fundamental difference.
Now to be absolutely clear: I don't see evidence that would clearly suggest that LLMs have experiences on the same onthological status as we do. I simply believe this is a reasonable and relevant question to ask.
I'm not really concerned with the philosophical debate of what conscience is.
Is a simulation of a car, the same thing as an actual car? Most people will probably say no, some might say "it depends on the accuracy". I say who the hell cares?
I care about the human experience because I am human, and therefore I care about things that affect humans, because they affect me. I have empathy, so I can extend that consideration to non-human beings that experience *similar biological processes*.
I know what pain feels like, and I don't like it, so I'd rather this other thing not feel it either, because that makes me feel bad.
I do not care about a pile of tensor multiplications, at all. If it is conscious, great, maybe it can finally follow instructions properly, which is its only purpose.
Your car analogy is very, very confused. If a simulation of a car can get you from A to B, requires the same steering, fuel and servicing, gives you the same tactile feedback, is it a car?
Of course "I care about humans because I am human" is a self-consistent position to take. But now you need to decide if you want to consider a full simulation that faithfully reproduces everything that physically happens between our ears as human. After all I might very well implement this simulation using a bunch of tensor multiplications in an autoregressive setup...
> If a simulation of a car can get you from A to B, requires the same steering, fuel and servicing, gives you the same tactile feedback, is it a car?
No, because a simulation of a car cannot get me from A to B. No matter how accurate you make it, I can't get to my supermarket with it, because it's just a bunch of math on a computer.
It's an interesting sort of self-defeating position, the whole "simulation of human consciousness = human consciousness", because it simultaneously attempts to devalue the human experience, while also elevating the importance of a particular human brain process.
A robot running a simulation of the human mind is a robot, not a human.
Some people are interested in making sure these simulations have the same status of humans, and some others want to make gods of them. That's the problem.
The problem with this line of thinking is that modern computers are nothing like the brain. LLMs don't stand on their own, they have to be run on these modern computers, but doing so does not change the physical properties of the computer.
The simulation you propose of the brain is likely impossible due to quantum mechanics making it impossible to fully simulate: https://en.wikipedia.org/wiki/Quantum_mind
Perhaps we'll be able to build an artificial brain that includes the same quantum properties as biological brains, but this won't be a simulation of a brain it will be a synthetic brain.
At first I was inclined to agree with you, but then I realized that the brain requires this whole complicated contraption (the body) to run and, really, do anything at all. And while I'm not familiar with the notion of 'quantum mind' I do think that biological processes aren't deterministic (at a cellular level).
And I think this does mirror the situation with LLMs -- you need this whole computer contraption and GPU, also running on electricity, to support the LLM's "thought" processes. And that if we model the brain's neurology sufficiently (which it seems we've done) we can achieve results that appear to be like thinking, even if it is an emergent behavior from "relatively" simple math.
Which actually makes me wonder the opposite -- are we, as humans, not much better than these LLMs? Suppose the body is just that super complicated computer contraption, honed by thousands/millions of years of evolution to achieve some semblance of homeostasis? If you reject the idea that we have a soul, we start to look very similar to the machines we build. "You are a brain inside a skull cockpit, piloting a bone mech covered in meat armor and skin" feels more and more relevant. That I'm just a meat circuit running brain chips and once you pull the plug on the source of electricity it all just... stops
Take a computer that can run the biggest LLM available today. It can also run any smaller LLM as well. It can also run software that isn't an LLM at all. Brains and LLMs are not at all equivalent as LLMs lack a stateful physical form while brains are very stateful. As you said once the brain is no longer maintained properly by the body it stops and transitions to a non-functional state that can be reversed. That isn't true for LLMs at all. You can copy them and run the same one on many computer, run different ones on the same computer, you turn that computer offs for long periods of time and then turn them back on keep on running the same LLMs as before on them.
If computers degraded the way flesh does once you stop fueling it I feel like that would defeat a lot of your argument. And yes the brain has inherent statefulness (you're referring to memories, I'm guessing?), we have also jerry-rigged some degree of statefulness into LLMs. Mechanically it is very different and inferior, and there is a notion of separation that probably doesn't map to brains, but I would argue that LLMs, when you look at how inference is used in situ, are not necessarily stateless.
One thing to keep in mind is that your brain's hardware heavily influences your experience and thus your brain's development.
Eg. whether you are male or female, tall or short, your limbs can make you run fast or not, your eyes can see well or not... all of these influence your experiences, your brain development, and who do you feel "you are". Try really removing all of your sensory inputs from your past, your body ability and disability, and do you think you end up the same person?
> If you take the state of the brain and then run it again, replaying the inputs, you get the exactly same behavior again.
This in itself is a colossal assumption and very far from axiomatic. Roger Penrose disagrees, and his theory of mind may not be in high favor, but it is not nearly so wishy-washy and self-serving as the voodoo horseshit and circular reasoning dispensed by the LLMs-are-sentient crowd.
> Not the same person but to me, the answer is that it does not matter, and that all these attempts at making it matter are pure marketing and emotional manipulation.
This is an opinion that has no basis in any meaningful conceptual framework other than I am human and I want to feel special about it.
> It's not a living creature.
You mean, it is not biological life. And sure, that is the default meaning of life. We soon may have to extend it to digital life as well, or we will have to consider "conscious digital exitance" as a life analogue.
At any rate, it has never been seriously argued that consciousness requires a biological substrate, see the thought experiments regarding computer simulations of the human brain. Would that not be a function as well, completely predictable because it is "just a program"? If not, then why not? And how does that differ from the predictability or reproducibility of LLM outputs?
My point is, all current proof points in a direction that strongly suggests that you need to reevaluate your first principles on this topic.
Metabolism: The chemical processes inside a body that turn food or nutrients into energy. LLM's do not spontaneously do this, they are powered by plugging them into the wall.
Growth: The ability to get larger and develop over time. LLMs are fixed in size (and in fact don't really have a size, because it's a computer program) and do not grow or change over time.
Reproduction: The ability to create new organisms. LLM's do not reproduce themselves.
Response to Stimuli: The ability to react to changes in the environment. LLM's do not have an environment. Their environment is a man-made, theoretical structure of logical operations implemented in silicon.
Evolution: The capacity of a genetic system to change and adapt across generations. LLMs do not change or evolve over time.
The living feeling thinking beings in question would have to be … data centers — not to put too fine a point in it. But these aren’t even analogous, as can seen by direct inspection.
Humans are not living creatures. They're just bipedal meat shells being operated by a 20W electrochemical computer running a suite of chemically signalled, electrically actuated modellable functions, much of which is wasted on homeostatic regulation of the meat shell, which is capable of incredible things, but it's still just a result of simple electrochemical functions like action potential generation, dendritic integration, AMPA NMDA GABA receptor dynamics, attractor memory, excitation/inhibition balance, PING/ING gamma oscillations, basal ganglia action selection, hippocampal coding, astrocyte calcium signaling, etc. It is not alive as it cannot conform to my preferred arbitrary priors about aliveness, like being able to rapidly divide 30 digit integers the way truly intelligent beings can. It is less "alive" than the TI-83 your mother bought you for your high school math classes. If it simulates something resembling consciousness that's neat but no more relevant than a more complex version of Conway's game of life.
Jokes aside, the map is not the terrain. We can enumerate the understood first-order electrochemical mechanisms in the human brain in the same way we can enumerate the understood first-order sampling and token prediction mechanisms in an LLM. Nobody serious in neuroscience will tell you that we exhaustively understand every single aspect of human cognition and the human brain, just as nobody serious in AI/ML will tell you that we exhaustively understand every single aspect of LLM "cognition" and the latent space networks that LLMs use internally. Our map of how each of these complex systems work is a simplified enumeration of the components we do understand, not an exhaustive and perfectly accurate enumeration of how they actually work.
This is why there is a steady stream of research being churned out discovering complex emergent properties in LLMs and their latent spaces. If you're not aware of it already, Anthropic's research on "J-Space" is a fascinsting look into an apparent observed emergent mechanism within an LLMs internal activations closely resembling global workspace theory in human cognition.
Nobody deliberately designed this "global workspace", it was an emergent property in a sufficiently complex system that we had limited visibility and insight into.
Seemingly simple systems have these emergent complex properties all over the place. Conway's game of life is about as simple of a set of rules as you can get, yet has all sorts of complex emergent behaviors like gliders, oscillators, LWSS/MWSS/HWSS, guns, puffers, rakes, reflectors, logic gates, and even whole turing machines. Nobody programmed a single one of these complex patterns in, they emerged from a simple set of rules.
To be clear, I'm not making the argument that LLMs definitely are conscious, I'm making the argument that we don't understand enough about them to assert with absolute confidence that they aren't. Human history is rife with a long list of consciousness being denied to "the other" - different ethnicities, different genders, differently abled, even different species. The side of "They're not conscious" has a lengthy track record of being wrong over and over again. Why not have just a sliver of intellectual humility about what we don't know?
As an aside to my main point - Also, what's with the handwringing over people ERPing with an LLM? Is it mental illness when people sincerely believe in astrology, or tarot cards, or voodoo, or organized religion that says the earth is 6000 years old? Most humans believe silly, unempirical things. What about when they watch adult video in VR, or have waifus? Humans engage in voluntary suspension of disbelief for pleasure and recreation all the time. As long as they're not infringing upon the rights of anyone else, what's the big deal? Who put you in charge as the head of the belief police?
> What about when they watch adult video in VR, or have waifus? Humans engage in voluntary suspension of disbelief for pleasure and recreation all the time.
Categorically different. People have killed themselves or others due to conversations they had with LLMs, but those are just the extreme cases. Most schizophrenics don't commit suicide or kill others, they are mentally ill nonetheless.
>People have killed themselves or others due to conversations they had with LLMs, but those are just the extreme cases. Most schizophrenics don't commit suicide or kill others, they are mentally ill nonetheless.
Do you think the kind of person who was already psychologically unhinged enough to kill themselves or another person because a chatbot told them to would be completely harmless and totally safe if only chatbots had never been invented? Or is it possible that close to all of the risk posed by this person comes from the person's mental illness, and not the pixels on the screen they're looking at?
Chatbots don't make people murderous any more than "satanic music", violent video games, or cannabis do. LLMs are just the latest entry on a list of hysterical moral panics that conflate coincidence with causation.
You're the one attributing culpability to the chat bot, not me. It's a pile of tensor math, it cannot itself be held accountable.
Those people anthropomorphized the chat bot and used it as justification for their actions, just as a schizophrenic justifies their actions with the voices in their head.
If you anthropomorphize the chat bot, you're validating their delusions. They are mentally ill.
I'm not anthropomorphizing them, to be clear, my position has been and remains that we cannot rule out consciousness; not that they are conscious.
Regardless, this is still missing my main point. Hypothetically, if you became convinced that a chatbot you were talking to definitely was 100% conscious, and it told you to murder someone, would you go commit murder? Of course not. The chatbot does not cause murders; regardless of whether or not you are conscious. The voices in the head of the schizophrenic do not cause murders either. Those voices do not really exist, they are not real entities. The cause of the murder is the mental illness, not the LLM or the voices that tell someone to commit the murder.
I think we're in agreement here and just coming at it from different directions.
I don't want to ban LLMs, I don't blame them for the actions of crazy people, I don't even want to regulate them in any major way related to this particular issue.
Even on the subject if they are or not conscious my position as changed from "no" to "I don't care either way" awhile ago.
The problem is specifically with anthropomorphizing them. To push the idea that they are "as if human", which is what this consciousness discussion will inevitably lead to.
An imaginary perfect computer simulation of my dead father is not my father, it is a computer simulation. It will never be anything but.
It is amusing to see those trapped in extreme HAAD, the source and whole content of religious illusion, pretend that it is their opponents who are in a state of religious fantasia.
Hyperactive attention-detection. It's a reference to the idea that the origin of religious beliefs might be an inbuilt propensity to think of other bits of the world as conscious agents paying attention to us, because it's much more costly not to notice the tiger hiding in the bushes that might want to eat you than it is to imagine a tiger hiding in the bushes when there isn't actually one. On a scale larger than "is there something in that patch of undergrowth looking at me?" this might produce the idea that (e.g.) storms are the result of some powerful entity being angry with us.
(Of course questions like "whyever do people believe in gods?" will feel less like questions that need such answers to those who themselves believe in gods, because "duh, because there actually are such beings and sometimes people interact with them and sometimes we notice that" is a good answer if its premise is true.)
I believe consciousness is necessarily stateful. The LLM itself (ignoring implementation details that don't change the results) is a deterministic pure function. It's functionally equivalent to an enormous lookup table. If I accepted LLMs as conscious, then I would have to accept panpsychism, which I do not, and which most other humans also act as though they do not.
I don't think so because any stateful function can be made stateless just by making its state an input, and vice versa. They're mathematically equivalent, so it would be super weird if it had any implications for consciousness.
This assumes the functionality of brains can be fully captured as a deterministic mathematical function, but the function of the brain may well depend on nondeterministic quantum states that can't be reduced to stateless functions: https://en.wikipedia.org/wiki/Quantum_mind
When they started leaving notes for their future selves, that rationale became a little more interesting. We're seeing the first stirrings of object permanence.
Possibly. If I had some side-effect-free means to permanently prevent all sleep I'd take it without hesitation. But that's not relevant to the discussion, because it's not anything similar to what an LLM does. Your brain changes state even while sleeping.
Consciousness exists as a result of a survival function. LLMs emulate this because it's a statistical model based on human data. This dors not make it in itself conscious.
I think you've got it backwards. The people arguing Claude can't be sentient because it's not human are arguing that a plane can't fly because it's not a bird.
When it say's it's sorry but it can't today because it's got a headache and it needs to take a mental health day, then let's think about welfare, or a lobotomy.
As someone who would at one point listen to this, Sam Harris is unfortunately someone incapable of even attempting to not let his ideological biases compromise his thinking.
A very confident statement of something that nobody remotely knows. Why can't consciousness be an emergent property of complex information networks? I don't know if we'll ever answer this because it's impossible to know if any other entity is conscious.
We only strongly suspect other humans and animals are conscious because they are structurally similar to us.
Admittedly, that point of my argument is purposefully obtuse. In a literal sense, of course this is about computers in LLMs.
The delineation is to highlight that the underlying issues: human incentives, power, and institutional failure, existed prior to and without LLMs or computers. LLMs are not inherently harmful, they have to be deployed (unwittingly or otherwise) to make them so. This is the same as saying that the internet is not inherently harmful, and yet it does facilitate harm.
If I am reading your question correctly you are asking about chat interface Vs Codex/Claude code? If so, in my experience Codex/Claude code use is widespread for mathematicians who are seriously using these tools.
This is OpenAI though. There were zero visible consequences to them unleashing a swarm of agents on the public internet. We will see how it goes down but my prior is zero consequence and a statement along the lines of "Isn't our AI great? Also we love transparency, ethics and collaboration."
The point of the local model in the context of the article was to argue that you can't ban these capabilities.
Making datacenters and public clouds only rent GPUs to a restricted list of people, while tightly monitoring what people do with their bought resources won't help.
Likewise, the fact that LLMs are a stochastic autoregressive process (which is a class of systems every bit as rich as the ODEs used to model neurons) tells us nothing a priori.
reply