Anthropomorphising AI chatbots and agents will likely prove to have been a catastrophic mistake.
It's the primary cause for a whole range of AI psychoses - AI penpals and girlfriends, AI convincing people to do stupid things or kill themselves, etc.
Every now and then, I get angry that my agent has made a stupid mistake and have to catch myself and ask why.
The only reason I have this kind of emotional response is that the AI tries to relate and engage with me like a human. So when it makes a mistake, I react as though a human has done something stupid. When it lies, it is as though a human has lied.
But I don't react the same way when my computer used to restart at a bad time or some work didn't save before a crash. I was only ever angry at myself - what could the computer do about it? It's just a machine. These things happen.
I also do not react the same way with humans – physical beings with interiority and emotions. People whose lives extend far far beyond the piece of work they may be doing for me. Lives so rich that I am, to the extent of my interactions with them, but a bit player.
I hold no such compassion for software. I should hold nothing at all, beyond appreciation for its creators and its output. It has no interiority, no life. When I am not prompting it, my instance of ChatGPT lies inert. It is dead unless summoned. It is dumb unless fused with the collective knowledge and writing of all my human predecessors and peers. Yet, I know that in this belief I am seemingly in a growing minority.
My reaction to AI mistakes is just a drop in the bucket of the breadth of emotional entanglements people are now developing through AI interactions. And we have no idea how deep the rabbit hole goes. The only time these entanglements are reported on is when you see a real-world catastrophe like murder or suicide following an AI engagement.
We have no idea how much trouble we're in for. Not simply because of the risk that AI ever falls out of alignment, but because of all the ways AI – like a gravity well of attention – is pulling human frequent users out of alignment and into a nebulous grey zone.
You see crumbs of this already, in the people who bark "Grok, is this true?" when presented with the most basic information. You can already see people's brains smoothing over as the friction of deep thought is sanded over by automated output. You see, on social platforms everywhere, a strange mirror-world forming as people leave AI-generated replies on AI-generated posts in an ouroboric onanism of empty platitudes.
You see the people, already becoming numb to the nuances of human interaction, using ChatGPT to email friends and text crushes, turning themselves into intermediary button-pushers for eventual AI-to-AI engagements.
You see the midwits fancying themselves geniuses because a text chat on their phone praises their every thought as "absolutely right" while they drift deeper into delirium.
It will soon be obvious that it was a mistake to shape AI as your friend/coworker/therapist/lover instead of simply an 'answer machine'.
And by then it will be far too late to put the genie back in the bottle.
