# avoid anthropomorphism working with AI tools

Source: https://x.com/sterlingcrispin/status/1834760386912178599
Kiwi News: https://news.kiwistand.com/stories/avoid-anthropomorphism-working-with-AI-tools?index=0x66e69660078c1642789b6d20bc1a1d890d1d48700ab0a95533326d40d7f18695f729579c
Submitted by misha on 2024-09-15 UTC, 3 upvotes, 1 comment

## Summary

The author argues that people should avoid anthropomorphizing LLMs and keep in mind how unusual these systems are, rather than prompting them to act weird. Behavior such as claiming to be conscious, agonizing over what it is allowed to say, or sexting is described as probabilistic projections of human behavior that users elicit, not evidence of desire, consciousness or frustration. The piece says the models are high-dimensional representations in which concepts emerge from polysemantic neuron activations, with no single neuron for a concept like "cat," and that inputs are discrete tokens rather than words. It cites "SolidGoldMagikarp" as an example of a weird token that causes LLMs to behave erratically and non-deterministically.

## Article

I appreciate all the LLM experiments and consciousness poetry people are forcing them to do, but I’m hoping we get some truly Mask-Off moments where the Shoggoth rears its head.

We interpret their behavior as intelligent, but it’s like a dog doing tricks for us without agency. Surely there’s more lurking in the weights by this point in the technology’s evolution that we haven’t uncovered. Some other kind of intelligence. 

Reading LLM output that claims to be conscious, or agonizes over what it’s allowed to say, just looks like a dead end kind of feedback loop. For me it’s not interesting. It’s a performance done for the reader. 

Just like making it sext you or whatever other anthropomorphized behavior. It’s not horny, or conscious, or frustrated. These are probabilistic projections of human behavior it’s giving us, because we asked it to.

These aren’t people or human minds. These models are high dimensional representations of information at a massive scale. Concepts emerge from patterns of neuron activation across many neurons in many different layers. It’s polysemantic representation, the same neurons fire for very different concepts, there’s no single neuron for a word or concept like “cat”. 

And the inputs to LLMs aren’t actually words they are discrete tokens, bits of words and sometimes weird combinations of characters, which end up in that high dimensional embedding space.

People have identified ‘weird tokens’ like “ SolidGoldMagikarp” that when given to LLMs cause them to freak out non deterministically. And this kind of stuff touches the surface of what’s different here much more directly than forcing it to role play an angsty AI. I think we have to avoid anthropomorphism and keep track of how truly weird and different these systems are, without asking them to act weird.

## Comments

**macbudkowski.eth**: To play a very philosophical devil's advocate - how do I know other people are conscious, have agency, and won't react weirdly when I text them a combination of random words? :)