The subject in the title seems to be growing in popularity rather than fading from the headlines. All of the conversations taking place about artificial intelligence and the new generation of chatbots seem to eventually veer off into worst-case scenarios. Is the AI sentient? If not, will it become sentient on its own and will we realize when it’s happening? These are on the minds of people in San Francisco and Silicon Valley in particular, which probably makes sense. And now there is a new “pop-up museum” in San Francisco called the Misalignment Museum, currently on display in the Mission District. It deals exclusively with the possibility of humans developing an AGI or “artificial general intelligence.” That’s the next level of AI beyond what we have now and it would essentially be individually intelligent and capable of reason and original thought. CNBC interviewed the curator of the museum, Audrey Kim, and she seems to take the question seriously.

Audrey Kim is pretty sure a powerful robot isn’t going to harvest resources from her body to fulfill its goals.

But she’s taking the possibility seriously.

“On the record: I think it’s highly unlikely that AI will extract my atoms to turn me into paper clips,” Kim told CNBC in an interview. “However, I do see that there are a lot of potential destructive outcomes that could happen with this technology.”

The name of the museum comes from one of the questions currently making the rounds. If a true AGI does show up, would it be “aligned” with humans? If so, our tech overlords tell us, it could spell the end of hunger or even work. But if it’s “misaligned,” it could spell the end of something else… probably us. In fact, there’s a sign in the middle of the display that reads, “Sorry for killing most of humanity.”

There’s a comforting thought. The curator’s comment about paperclips is taken from Nick Bostrom’s paperclip maximizer problem. Bostrom postulated in 2003 that a sufficiently powerful AI could be given a simple task such as “manufacture as many paperclips as possible.” The AI would be hooked into robots capable of carrying out its orders but it would be so intellectually powerful that it could resist its human creators efforts to shut it down. It would keep producing paperclips and paperclip factories until it eventually consumed all of the resources on the planet (including humans) and eventually the rest of the cosmos.

That led to this creation.

OMFG https://t.co/nQhSeKjy0Z pic.twitter.com/LvAnZtVsbk

— Marc Andreessen (@pmarca) March 8, 2023

Or, rather than simply following basic instructions to a hypothetically disastrous end, the AI could have new ideas of its own. When asked how to stop the spread of global warming (because you know that’s one of the first questions that someone in California would ask), it might immediately conclude that humans are the cause of global warming so they will need to be eliminated. How would it go about achieving that goal? Well… there are already people giving guns to robots. But don’t spend your time fretting about that. What could possibly go wrong?