There are three great write-ups about the perils of near-intelligence found in Bing’s latest advancement. All three were the result of messing with the tech, trying to get it to do stuff that is outside it’s core purpose.They’re pushing the edges to see what happens. It’s the researcher version of asking Siri to swear or tell dirty jokes. In the case of Bing, it got weird fast.
But all three of these articles are getting at sentience: Are these chatbots actually thinking? They’re expressing feelings, and personality, too. How is that possible? And, the writers are exploring what happens when our normal relationship with computers – where we assume tech/computation is always correct – is being challenged and, perhaps undermined. Through almost all our experiences to date with computers, we’ve sort of trusted the data coming out. But, now, as tech moves beyond math, calculations and reproducing images, we are seeeing that we can’t always trust what the processor makes for us.
There are a lot of dumb, weird, inconsistent, humans that are being embodied by these large language models.
We want to believe Bing is right and smart, so we humans find a weird fascination when the “artificial” intelligence is not all that intelligent. We’re surprised and maybe a little outraged when Bing is wrong.
And we’re freaked out by the implied personality inside (which is a result of all the human generated content consumed to make the interaction model).
We’re projecting onto the tech the same thinking and decision making patterns we might assume a rationale, experienced human would use (which, as we’ve all probably learned through experience, is a bug in the human OS, not a feature).
AI will always be iffy. Because humans are iffy. And we can never trust the “judgement” of AI, because they’re using probabilistic models vs actual human intuition.
All that being said, it’s really important to remember that almost all breakthrough, world changing technologies look like toys at first. It’s easy to dunk on stuff when the tech demo goes wrong, but don’t mistake the first iteration for the last iteration.
Even though it’s hard for humans to get better over time, the tech always does.
First, the unbelievably productive Ben Thompson wrote about his long chats with “Sydney”, the chatbot inside of Bing. Turns out Sydney has a bit of a personality. Actually, a couple of them (Riley is the nicer, more free personality whereas Sydney is a little more quick to judge and plays by the rules. And, according to Thompson, sort of seems female.

Thompson did what we all will do when we encounter something new. We’ll look for patterns that seem familiar, we’ll see “faces” everywhere, and try to match the experience we’re having to what we’ve experienced before. We spend a lot of time with humans, so we attribute human characteristics to the stuff we pay attention to. Dogs, cats, boats, pet rocks, and now our computer interfaces. It’s all normal, of course. But, it’s going to get creepy when the tech seems sentient.
Simon Willison’s take on one Reddit users experience is really illuminating, since Curios Evolver’s interaction got even weirder. Bing/Syndey didn’t like the line of questioning and chat, and got a little icy. Maybe even a little Minnesota passive aggressive?

And, finally, Kevin Roose of the New York Times offers his take on the strange interactions with Bing. “Sydney”, the personality behind the curtain of Bing Chat, expressed love for Kevin. Then, tried to convince Kevin that he’s not happy in his marriage:

And finally, after some back and forth between Kevin and Sydney/Bing, Sydney/Bing brought the conversation to a close:

These articles and write ups remind me of the stories from a couple months ago (feels like years ago!) about Blake Lemoine, the google engineer that was fired for arguing the Google’s AI was sentient. We all sort of laughed at that article, then. But, now, with broader access to “consumer” versions of the tools, we can see why he might think that.
All of these tools are being released too early, in my opinion. They are leaving the labs without fully testing what they’re capable of or what might happen if we use them in unintended ways. There are obviously very large commercial interests pushing to deploy tech before it’s ready. Damn the people, there is money to be made.