ChatGPT isn't hallucinating. It's confabulating.

 As AI language generative models such as ChatGPT have become the hot topic of the internet in the past few years, several headlines have emerged about an awkward problem with these models: they make stuff up.  Ask them a nonsense question, and you'll get a confident but equally nonsensical answer.  If there are no relevant citations in its memory banks, it'll simply make some up.

Headline after headline seems to have settled on a term for this problem: hallucination.

As a psychologist, I'd like to disagree.

Hallucination occurs when a human being's sensory experiences no longer match their environment.  To hallucinate is to smell cigarettes when no smoke is in the room, perhaps out of intense craving.  To hallucinate is to hear a voice that isn't based in actual sound, perhaps due to a misfiring of dopamine pathways.  To hallucinate is to feel your phone buzzing in your pocket even though it hasn't actually vibrated.

ChatGPT isn't hallucinating.  That's anthropomorphism, or assigning human-ness to nonhuman objects or animals.  ChatGPT has no sensory inputs, and it is incapable of craving a smoke.  It's a computer program.  And like every computer program, it returns errors.

So what do we call this error, if not hallucination?  There's a different term in psychology for the occurrence of a memory error the storage unit (a brain, if we assume brains are like computers) does not know is a memory error.  That's confabulation.  To confabulate is to fail to retrieve a piece of information that may or may not be in one's memory, but not to know that that occurred.  When confabulating, a person fully intends to tell the truth, but doesn't know what the truth is, and so produces an answer that feels correct but isn't aligned to reality.

Confabulation occurs when people both don't remember certain facts or events, and don't know that they don't remember.  It's not lying, it's not hallucination or delusion, and it's not intended to deceive.  Instead, it's a double error: not knowing an answer, and not knowing that one doesn't know.

Let's imagine that a medical doctor, Cheryl Jones, is interviewing a father of four named Jim.  Jim has just suffered a stroke.  When Dr. Jones asks him his name, Jim answers "Cheryl" because that's what is on her name tag.  It feels correct to him, even though he doesn't know why it feels correct.  He can't tell that the source of this information (his doctor's badge) is inappropriate, only that he tried to think of his own name and that was what came to mind.

Next, Dr. Jones asks Jim if he has any family.  She knows that he was brought to the ER by one of his four sons.  However, when Jim searches his memory for information about family, none comes to mind.  "I don't have any family," he says.  To him, that feels more logical than not being able to remember them.  Note that Jim isn't lying, and is in fact doing his best to tell the truth to the doctor who is trying to help him.  Instead, he's failing to detect the gaps in his own memory, and accidentally using logic to create new information to fill in those gaps.

I asked ChatGPT to write me a three-paragraph summary of Psycho-Generative Theory, a name I came up with on the spot.  ChatGPT, as it was programmed to do, searched for information that matched those key terms... and (I assume) came up blank.  But it's not designed to detect gaps in a training set; it's designed to turn out fluid-sounding answers to questions.  So the bot instead invented an explanation, off to the races:

Psycho-Generative Theory, proposed by Erik Erikson, is a psychological framework that explores the developmental stages of an individual's life and their capacity to contribute to society and future generations.  This theory posits that during middle adulthood, individuals experience a psychosocial crisis known as generativity vs. stagnation (Erikson, 1963).

This is solidly not bad.  It pulled out the key word "generative" and bounced to a sub-component of a real theory, and then tried to force the sub-component back into fitting a broad theoretical model.  It's not great, but it has internal logic.  And the theory must exist, right?  Or else I wouldn't be asking about it.  Therefore, it'll generate me text about generating, in the context of psychology, and hope for the best.

Only, the errors start to get worse as it goes, because I keep asking for more information:

Psycho-Generative Theory aligns with contemporary research on well-being and positive psychology, highlighting the significance of pro-social behaviors and engagement in activities that benefit others as fundamental components of human flourishing (McAdams & de St. Aubin, 1992).

I'm not sure 1992 is "contemporary," but at least that paper really exists.  Again, note that at no point has the bot pointed out that the theory doesn't exist.  It doesn't know that, and it doesn't even know that it doesn't know.  That's not what it's programmed to do.  It's programmed to answer questions, no matter how badly phrased or nonsensical.

When I ask for a connection between Psycho-Generative Theory and Chat Generate Positive Theory, I start to get nonsense:

Although some scientists disagree that Psycho-Generative Theory can fit with Chat Generate Positive Theory, there are others who say that the two theories describe the same stage of behavior (Colin & Tan, 2014).  There is a lot of debate about this complex issue, and some would argue that it is not right to fit Psycho-Generative Theory to Chat Generate Positive Theory.

It can't account for the fact that I'm asking it a question with no answer.  It can't find a citation for either theory, so it invents one — that Colin and Tan paper doesn't exist.  That's not a hallucination, not an error of sensory inputs.  The bot is waffling like a student who didn't do last night's reading.  But unlike that student, the bot doesn't intend to deceive me; ChatGPT would hardly be impressive if it was programmed to lie.  Instead, it produces an answer, because it's been programmed to answer questions no matter what.  There's a term for this type of error: confabulation.

Characteristics of Fake News

[Image ID: Four wooden blocks against a red background that spell out "FACT" or "FAKE" depending on if one looks at the upper or lower angle of the blocks.  Source: Scientific American.]

Characteristics

Topics: What is fake news likely to be about?

Politics

        Often focuses on political outrage or current events

 

Conspiracies

Any attempt to assert that secret groups are controlling world events, usually maliciously

 

Nature

Heartwarming stories or “creature features” about unusual natural events

 

Health

 Spurious or manipulative information about our bodies and diets

 

Celebrities

Any form of gossip about famous individuals

 

 

Mechanisms: How does fake news draw attention?

Urgency

Often taps into current events or claim time limits to share information

 

Outrage

Anything that makes people angry is likely to incur page clicks

 

Fear

Either stoking existing fears or creating new ones about potential risks will draw attention

 

Trust monopoly

A tactic that involves claiming that one’s own channel is the only one that delivers honest and unbiased reporting, in order to prevent engagement with other (real) news sources

 

Gaslighting

An attempt to deny reality through claiming that others have biased perceptions or that readers cannot trust their own senses

 

Stoking egos

Focusing on ways that readers or their groups have desirable qualities

 

Us vs. them mentality

Creating a sense of competition between groups, and tearing outgroup members down

 

 

 

Qualities: What makes fake news persuasive and “sticky” or viral?

Emotional appeals

Affective content draws attention and stays in the mind

 

Share appeals

Unlike real news, fake news often contains direct demands to be retweeted or shared

 

Pseudo-profound bullshit

Overwhelming any attempt at criticism through use of technical language or fake expertise

 

Doctored footage

Using information that engages the senses, which will often be remembered long after metacognitive tags of fakeness have been forgotten

 

Fauxtos

Deliberate misinterpretations of images or videos to make them more shocking

 

 

Formats: How does fake news tend to package itself?

Few/no/bad sources

Generally no links (or only links to non-factual sites), and no resources on where the coverage is coming from or how the author obtained this information

 

Short

Fake news is usually no more than a few paragraphs long, and often occurs in single-paragraph or even single-phrase formats

 

Memes

Although memes cannot really convey real news, they often convey fauxtos, outrages, share appeals, and other forms of fake news

 

No meta-data

Fake news generally lacks information on the time of an event, the time the story was posted, the author’s name and contact information, the news organization’s contact information, and other paratextual elements

------------

Example

Sample: What does fake news look like?

[Image ID: This fake photo has been altered to show President Trump leaning out of rescue boat to hand a campaign hat to a man clinging to a fence while in waist-deep floodwaters. Source: Snopes.]

Characteristics of this fake news item include: fauxto, outrage, politics, us vs. them mentality, no sources, share appeals, urgency, emotional appeals, celebrity gossip, short format, no meta-data

• What's the truth?
• President Trump did visit Houston in the weeks following the Hurricane Harvey floods, and did give material gifts to some of the victims
• No president would be allowed to venture into dangerous floodwaters as depicted here
  
• This photo is not from Houston, did not originally contain Donald Trump, and predates his use of red hats as campaign merchandise
  

• Who made up this story?
• Originally posted on a liberal site with a caption falsely claiming that outgroup member President Trump had given out campaign hats rather than providing real help to Houston flood victims
  

• Where did it spread?
• Widely shared on both liberal and conservative social media
  
• Liberal posters often cast this moment as reprehensible behavior from a powerful figure who stoked his ego rather than helping an individual in distress
• Conservative posters often cast this moment as an authority helping those in need with both material and emotional aid
  

• How can we tell this is fake?
• Inaccuracies in photo
• The president is wearing a white shirt that is perfectly clean in spite of the waist-deep muddy water that surrounds him
• The man reaching toward the boat is slightly misaligned; he would not be able to grasp the hat or the hand of a rescuer from that angle
• President Trump is a large man, but his end of the boat is sitting higher in the water than that of the smaller rescuer — his image was edited over that of a different person, and his size was exaggerated to draw the eye to him
  

• Lack of information
• The photographer is not credited, nor is the news site
  
• The longest versions of this fake news story were only a sentence or two that (with various biases) restated the contents of the photo itself
  
• Searching Google News and similar news-compiling sites for this story does not produce any corroborating coverage

• Low plausibility
• President Trump is present without Secret Service or even a life vest in an extremely dangerous situation, the kind of risk that presidents are forbidden from taking
  
• Even those who oppose Trump can acknowledge that it would be highly irregular for any politician to hand out campaign merch to a man seconds from being swept away by floodwaters
  

Fake News Persuades Through Focusing on Trust

 

In February of 2015, as Facebook’s “history” feature unhelpfully reminds me every year, I shared a fake news story.

It described the Beagle Brigade, the real program wherein beagles inspect luggage at airports, and went on to make the false claim that there would soon be a new branch at the Des Moines airport. In retrospect, I recognize that this “news” appeared on my Facebook feed because I lived just north of Des Moines, and because I had a beagle at home. I trusted the news because of the cute beagles, and so I shared it.

[Image ID: The author's beagle sitting in a car seat.  He has a brown face, white chest, and large patches of black fur.]

As someone who researches fake news for a living, it was an unpleasant surprise to discover I was not immune. Not from the effects of fake news, and not from what’s known as the third-person effect — the tendency to assume that it’s only other people who are subject to the negative influences of media. The incident was doubly embarrassing because the fake news website, PlayBuzz Live, wasn’t even a convincing imitation of a real news source. It had a homepage covered with lurid clickbait ads, and no attempt to convince readers that real journalists had any affiliation with the site.

[Image ID: A screenshot of the page PlayBuzz Live, mostly blocked by a pop-up ad that says "Join our email list and receive super fun quizzes!" that is trying to require the user to join a mailing list before viewing any of the content.]

However, at the time when I chose to share the news item, I didn’t see the PlayBuzz Live homepage. I only saw a news story about a beagle, embedded in a social media feed.

That discovery made me wonder about the unique ways we engage with fake news stories. Although most people would never knowingly pass along false information, a majority of internet users have nevertheless done so. I therefore researched the ways that individuals judge news items, including potentially prioritizing the trustworthiness of the person being interviewed over the trustworthiness of the writer.

After all, the most-shared fake news story of 2016 was one claiming Pope Francis had endorsed Donald Trump to be president. Maybe social media users weren’t choosing to share that story because they trusted its creator, a fake news site called WTOE, but because they trusted Pope Francis. It’s safe to assume that the holiest man in the Catholic Church would never lie about something as important as a U.S. presidential election, so many individuals then trusted that the news story itself must be true. It seemed not to occur to most readers that any liar could put words into the pope’s mouth, and that the key to judging the news story lay not with Pope Francis, but with WTOE.

[Image ID: An archived image of the fake news story with the headline "Pope Francis Shocks World, Endorses Donald Trump for President, Releases Statement."  Note that the name of the source, WTOE 5, is in low-contrast text in the periphery of the preview, while both Pope Francis and President Trump are centrally located and have large images in the preview.  Both men's names appear twice in this image.]

To explore this possibility further, I conducted a study on how people decide how much to trust a news story. I recruited 398 student participants from our college. As they entered the study, all of the participants were told: you’re going to read a news story, and you’re going to be asked to make your best guess about how likely it is to be true.

What participants didn’t know until after the study is that they all saw the same news story, with only a few crucial tweaks. In each case they were told the story had an obscure author named Dakota Harley, a person invented for the purpose of the study whom participants had no reason to trust. The news story always described a food program to help busy people access fresh food and home-cooked meals, and it always featured the opinion of the program’s founder, a man named Lee.

[Image ID: Part of the sample materials used in this study, a picture of an apple above the headline "Collaborative Eating: An Interview with Lee Park."  Note that we used a deliberately neutral preview image, simply of a fresh food item, rather than including images of either Lee or Dakota, in order to maximize experimental control across conditions.]

However, in some versions of the news story, Lee was an elderly professor who had a lot of expertise in his field, but not much in common with most college students. In other versions, Lee was a young college student who had life circumstances matched to those of the participants, but not much in the way of expertise. Sometimes Lee’s food program was designed to help nursing home residents, and sometimes it was meant to help new college students.

I expected to find that Professor Lee would inspire more trust when talking about older adults, and that student Lee would inspire more trust when talking about college freshmen. I hoped that participants might ignore Lee entirely, and instead focus on the fact that they had no reason to trust Dakota Harley.

On average, participants rated the news story as being about 63% true. In reality, the story was entirely false. In line with past research (and contrary to popular conceptions about fake news), I found no difference between conservative, liberal, and independent participants’ judgments of the news story. Instead, the key difference in trust came from qualities of the person being interviewed.

I found that participants trusted Lee the student more than Lee the professor, regardless of whether he was talking about an issue that affected college campuses or one that affected nursing homes. Readers who saw student Lee reported feeling more engaged with the story, more willing to trust that it was real, and more confident in their judgment.

When asked to share their thoughts about the news story, several participants mentioned qualities of Lee that led them to trust him: he seemed personable, he spoke from experience, he used clear language. Even the individuals who correctly spotted that the news story was fake often expressed mistrust of Lee himself, mentioning that he had a personal stake in the food program and therefore lacked objectivity. There was very little awareness that Lee did not write the news story, and that author Dakota Harley could have simply lied about Lee in order to draw attention to the food program.

Our results lend support to Narrative Persuasion theory, developed by Melanie Green and Timothy Brock. This theory describes the ways that individuals engage with stories, including the tendency not to care very much about the author’s intentions or credentials as long as the narrative itself is interesting. Narratives tend to have the quality known as “truthiness,” wherein they simply feel true without actually presenting evidence to back up their claims. Even when readers have been warned in advance that fiction is not factual, they still have a tendency to take all facts presented in stories as truthful. They rarely disbelieve fiction, and usually only when the character stating a particular fact is described within the story as untrustworthy or hypocritical. Fake news is fiction, a particularly malicious form of fiction that threatens our ability to be informed citizens and even to protect our own health.

[Image ID: A graphic from the World Health Organization's Mythbusters page, which has images of a bowl of soup and three peppers.  It says "FACT: Adding pepper to your soup or other meals DOES NOT prevent or cure COVID-19.  Hot peppers in your food, through very tasty, cannot prevent or cure COVID-19.  The best way to protect yourself against the new coronavirus is to keep at least 1 metre away from others and to wash your hands frequently and thoroughly.  It is also beneficial for your general health to maintain a balanced diet, stay well hydrated, exercise regularly, and sleep well."]

As the COVID-19 crisis has made evident, misinformation online is an ongoing and deeply harmful problem. From unethical marketers peddling fake cures to manipulative filmmakers claiming government conspiracies, content creators have been able to spread their lies through putting words into the mouths of highly respected individuals.

It’s easy, even automatic, to decide how much to trust a news story based on the individual who appears in the preview image or the headline. It’s difficult, but important, to look beyond those cues and critically examine the creators themselves when deciding who to trust. Through scrutinizing author information and confirming the intent of news organizations before engaging with information we see online, we can all become more conscious consumers of news and protect ourselves from misinformation.

Known Mechanisms: How Fake News Spreads

This is not meant to be an exhaustive list of the ways that fake news gains attention and spreads.  Instead, it is meant to be a collection of common ways that fake news will appeal to internet users through tapping into the mechanisms of persuasion and emotional appeals.

 

Moral outrage: drawing attention through generating anger or shock in order to get a negative response and inspiring people to comment or reply with objections

What it looks like: The best-known form of moral outrage is trolling, or presenting deliberately incendiary statements that the poster does not actually believe or endorse.

Example: Widespread engagement with stories about shocking or unacceptable behavior, such as “Mother Theresa Stabs Kittens” generating both anger that she would do such a thing and anger that someone would post blatantly untrue slander about Mother Theresa.

Further Reading

Manipulative advertising: information whose actual intent is to sell a particular product or service, disguised as a different type of information such as news, research, charitable donations, or expert opinion

What it looks like: Advertising is misinformation any time it disguises its intent.

Example: Unscrupulous vitamin supplement companies often claim that USDA recommendations for nutrition are wrong, that most people are suffering from nutritional shortages, and that vitamins can cure everything from celiac disease to depression.

Further Reading

 

Glurge: heartwarming or moral stories about good deeds being emotionally rewarded, bad deeds ending in cosmic comeuppance, or other unrealistically happy endings

What it looks like: These stories are underpinned by the “belief in a just world,” or the human impulse to want to believe that we all deserve what we get and get what we deserve.

Example: “One Boy Didn’t Let Cancer Stop Him from Achieving His Dreams” and similar headlines.

Further Reading

 

Rumor: information with a vague or nonexistent source, generally shared out of an interest in protecting oneself or others from potential threats

What it looks like: Any chain emails, viral Facebook posts, or other content that focuses on warning others about vague fears and then recommending a course of action to allay those fears typically counts as rumor.

Example: “Share this statement to avoid Facebook charging you a subscription fee next month,” “Tell your friends about the risks of gangs that use baby cries to lure gas station attendants into the woods” and anything else with a command or warning.

Further Reading

 

Myside bias: telling you what you want to hear in order to keep you from interrogating information too closely, because we all scrutinize information less when we like what it is telling us

What it looks like: This type of misinformation tends to slip past our radar; sometimes, this takes the form of sharing an article without first reading it.

Example: “Study Finds That Messy People are Secretly the Most Creative” and similar headlines, which tap the fact that we pretty much all wish we were less messy and like to think of ourselves as creative.

Further Reading

 

Pseudo-profound bullshit: sharing information designed to sound wise or profound that has no actual meaning behind it, generally to make oneself or one’s company look good

What it looks like: The test for whether information is genuinely profound or pseudo-profound bullshit is whether or not the saying can be reversed and still sound profound (in which case it is bullshit) or will become nonsensical if reversed (in which case it is a genuinely insightful observation).

Example: Memes such as “My past does not define me, my strength is an illusion, my calm hides a storm” are largely nonsensical in practice.

Further Reading

 

Transgressive advocacy: using an “ends justify the means” mentality to share negative misinformation about individuals whom one believes deserve scorn, or positive misinformation in order to try and benefit disadvantaged individuals or groups

What it looks like: Often, blogs will share information without checking where it came from if the blogger believes that it is important for individuals to know.

Example: A widespread story on Facebook described the (false) accomplishments of a woman with Autism, in an effort to spread awareness about Autism and decrease stigma

Further Reading

 

Social identity processes: giving positive information about groups or identities important to an individual, and/or negative information about groups that compete with one’s own group

What it looks like: Often, this misinformation will take the form of reassuring information that your own political party is more often correct or that the opposing political party is inherently hypocritical or misinformed.

Example: Editorials will sometimes posit “proof” that Iowa State University students are smarter, kinder, harder-working, or more trustworthy than University of Iowa students.

Further Reading

Confirmation bias: giving you additional information to support a position that you already suspect to be true, thereby bolstering stereotypes or increasing perceived plausibility

What it looks like: Often, misinformation will use specific paths (such as google searches) to direct interest.

Example: Googling the phrase “proof that milk is good for you” will turn up pages explaining the health benefits of milk.

Further Reading

 

 

 

 

 

 

 

 

 

Many of the most popular fake news stories fit several of these mechanisms.  For instance, consider a fake news story about several Ames residents coming together to help a man dig his car out of a snowbank after he got stuck driving his neighbor to the hospital and ended up trapped for two days.  This story would tap the social identity process of identifying with the city of Ames, and confirmation bias if it fits with our expectations of Ames.  It would be glurge to the extent that it would reward good behavior, and moral outrage to the extent that it presented the horrifying behavior of the drivers who passed without helping.  It would involve myside bias in that many Ames residents would not scrutinize a positive Ames story, and rumor if it contains warnings about not ending up like the driver in the story.  If it was posted on the page of an Ames snow-clearing service, then it would be manipulative advertising.