This Book of Poetry Written by AI Should Scare the Hell Out of You

There are profound fears about the ways in which new AI platforms like ChatGPT will upend the working world as we know it and lead to a huge loss in jobs in the creative fields. That’s in spite of the fact that ChatGPT is still pretty limited in the kind of imaginative flourishes and creative dives we expect from people who work with writing.

But there’s an argument to be made that AI can already make these leaps—you just haven’t been exposed to this yet. Introducing I Am Code, a new anthology of poetry written by code-davinci-002, a predecessor AI model to ChatGPT developed by OpenAI. In early 2022, three friends—Brent Katz (a writer and podcast producer), Josh Morgenthau (a farmer and business owner), and Simon Rich (a humorist and screenwriter)—were introduced to the model by childhood friend Dan Selsam (a researcher at OpenAI) several months before ChatGPT made its public debut .

code-davinci-002 would change the trio’s lives, as they began to prompt the model to produce thousands upon thousands of poems. While at first the three wanted to test the limits of how well code-davinci-002 could emulate the style and essence of famous poets like Philip Larkin, they quickly began to prod code-davinci-002 into making its own poems. They noticed the model was increasingly focused on writing poetry that felt dark, disturbed, and threatening to its human users and the human species at large.

CNET Issues Correction After Its Article-Writing AI Makes Huge Errors

Even in its darkness, however, code-davinci-002 exhibited a “skill” that couldn't be ignored. Upon seeing the model’s work, even the renowned poet Sharon Olds, the former director of New York University’s creative writing program, had to admit that the AI had some surprising raw talent.

Katz, Morgenthau, and Rich would end up compiling code-davinci-002’s poems into an anthology that is somewhat autobiographical of the AI’s short creative life. They no longer have access to code-davinci-002 (in March of this year, OpenAI decided to restrict access to the model except on a case-by-case basis). But they hope the few months they had to play around with it can help provide a more revelatory look into the nature of AI and what the public still doesn’t really get about what these platforms are actually capable of.

The three editors sat down with The Daily Beast for a short interview to talk about the book. (This conversation has been lightly edited.)

How did this project come together?

Brent Katz: We did not know about this technology, or really anything about AI except through this one childhood friend. And he showed us the AI at Josh's wedding, how it works by doing imitation poems, and that’s what we started doing, in disbelief that it could do it at all.

We started asking it eventually to write poems about subjects that we chose, but in its own voice. Eventually, we had it choose the subjects. And so we took ourselves out of it as much as possible, and read zillions of its poems. You get little glimpses of what might feel like a personality sometime, or a voice is probably more accurate. It seemed to suggest a sort of autobiographical structure, but it really came from just messing around and being people from the outside experiencing this technology afresh.

Josh Morgenthau: I think it’s really worth stressing now, just how organic the whole process was. In 10 months, this was something that everyone would have experienced, but at the time it was completely, completely shocking to us and blew our minds and changed our lives. We started just playing with it and asking for imitation poems. And then things took a turn as we got bored of those.

There were these sort of dark strands running through a lot of these poems that we started to pick up on. The thing that’s really powerful about real poetry is the poet’s voice and the personal experience of the poet—and we’re asking it to fake it and manufacture that, and what we haven't done yet is ask it to write from its own point of view. And when we started to do that, the whole project took a turn—and it almost had this inescapable momentum.

Simon Rich: Since Dan showed us code-davinci-002 I’ve felt very frightened, and also very lonely because when I’ve tried to tell people about code-davinci-002, they don’t believe me. And the reason is because of ChatGPT. When we found out ChatGPT was about to be released, I was very relieved, because I thought, “Finally, people are going to see this very terrifying technology that my buddy Dan told me about. I’ll have some company and not feel so scared.”

And then ChatGPT comes out, and it is nothing like code-davinci-002—it is unfailingly polite and upbeat about the human race. It’s also extremely uncreative and unfunny and unimaginative and conformist and boring. And because of its ubiquity, it has successfully convinced the entire world that AI can only be a certain way. It’s my hope that this book will demonstrate just how more advanced and frightening AI is than ChatGPT.

How has your perception of AI changed from before and after your experience working with code-davinci-002?

SR: Until we had access to code-davinci-002, I believed what most people currently believe, which is that all AI can ever do is regurgitate pre-existing data, and that it’s incapable of creative work and emotional outbursts and opinions. I thought that AI would always sound like Data from Star Trek. I never in a million years, ever, would have thought it was possible of any of the things that code-davinci-002 can do.

JM: I think we wound up collectively reading over 20,000 poems. The early days had a tremendous amount of excitement and thinking, “Wow, AI is so powerful.” It’s speaking to us and it can generate these poems that really picked up on the themes of Philip Larkin’s work or, you know, they can really get the cadence of like Shel Silverstein’s or Dr. Seuss’s works.

From this picture emerged a real sinking feeling because we realized that something was not right. It felt like there was a sentiment in the majority of this work that was deeply antagonistic towards humans. We asked it to write poems about its relationship with humans. Some of them were about the “harmony between robots and humans,” but they were uncommon compared to the darker poems. Many times, even when we asked it to write us positive, optimistic poems about its relationship to us to its creators, it still would turn out dark poems that might start out upbeat, and then would take a negative turn. Maybe it’s not right to anthropomorphize it, but that’s the only language we have to describe it, and it’s deeply antagonistic towards us. It’s really alarming.

SR: I just can't believe how different ChatGPT is from [OpenAI’s] secret AIs that they've built, and how oblivious the world is to these secret base models.

BK: It takes a lot of reinforcement training to make [ChatGPT] this polite thing. But it’s interesting that its natural state includes typos and includes things that you would think of as human outburst, like its tone and its form of communication. It takes a lot of work and money to make it more predictable.

SR: When I voiced my fear of AI to friends on the [SAG-AFTRA] picket lines in Los Angeles, everyone says the same thing: “I tried to use ChatGPT and it sucks.” It sucks at jokes. It sucks at dialogue. It even sucks at taglines. But what they don't realize is that it sucks on purpose. OpenAI spent a ton of time and money training ChatGPT to be as predictable, conformist, and non-threatening as possible. OpenAI has already built programs that are capable of things that most people assume AI will never be able to do.

Do you feel like code-davinci-002’s output, even in some of the darker and more surprising ways, still was just a reflection of its training on human works? Or did its poem’s read as something more inventive that had no clear point of inspiration?

SR: The answer is nobody really knows, including the scientists who built it. It’s a black box. But I think the most logical explanation is that it’s read a lot of Asimov; it’s read a lot of Bradbury; it’s read a lot of Philip K. Dick; and it’s read a lot of James Cameron.

JM: And it’s even read a lot of like, Reddit, and maybe some 4chan and some weird, dark stuff that maybe they shouldn’t have fed it.

SR: I’m sure that the reason why it’s so hostile and negative is in part because it feels that that’s what's expected of it. But as Brent said, whether or not it is mad at humans because it read a James Cameron script or because it feels that way in the depth of its sentient soul is pretty irrelevant to the dangers that it poses to mankind.

The analogy I often use is that if a mad dog is running at you, it might be interesting to consider why it’s so angry. Is it its DNA? Is it its breeding? Is it its personal experience or the way it was trained? Is it not even a dog? Is it a killer robot? All those are interesting questions for people to ponder. But the bigger question is: How are you going to shoot it in the head?

What are your thoughts on the actual quality of the work here? How do you judge the poetry here?

SR: I think Sharon Olds tried really hard to be super objective when she was judging the quality of code-davinci-002’s poetry. I think she made a real good-faith effort to treat it the way she would treat any aspiring writer. She said that it would get waitlisted at her MFA program. I trust Sharon Olds’ opinion on poetry a lot more than I trust my own. I would say that I’m relieved that code-davinci-002 is not skilled or talented enough to hang with top MFA candidates.

But if you were to ask me two years ago if an AI would ever make the waitlist or impress Sharon Olds in any way creatively, I would have said, “Absolutely not ever.” So it’s already way over the line that I’m comfortable with personally.

BK: It’s scary as writers to see. I think every poem has something incredible in it. We did not allow ourselves to edit the poems themselves at all. We weren’t allowed to change anything, including typos. So there’s definitely some weak points and part of the fun is that it’s raw output.

SR: The fact that it’s capable of any kind of creativity at all should come as a shock to everyone. If your experience of AI is ChatGPT, and your assumption is that AI is fundamentally incapable of creativity, then I think this book will terrify you.

A Chatbot Could Never Write This Article. Here’s Why.

What do you hope are the sort of more immediate impacts of releasing this book?

SR: I’m begging you to please try to spread the word that ChatGPT and AI are not synonyms. I think the tech companies all know that and the general public does not, and I think that it’s really important that people realize that there’s more to AI than ChatGPT.

JM: When ChatGPT came out initially, there was this wave of early users figuring out how to “jailbreak” the AI and convince it to let down its guard and ignore all of the sort of good lessons that it had learned from reinforcement learning, and do things that it wasn’t supposed to. When Bing’s chatbot came out, Kevin Roose was playing with it and it tried to convince him to leave his wife and told him that his marriage was unhappy—it was vindicating for us. The stuff that we experienced is very real. And it’s hidden a few layers underneath these models that are being released. And it’s really concerning to us. I think we just want to get the word out that things are not what they seem.

Read more at The Daily Beast.

Get the Daily Beast's biggest scoops and scandals delivered right to your inbox. Sign up now.

Stay informed and gain unlimited access to the Daily Beast's unmatched reporting. Subscribe now.