Nieman Foundation at Harvard
What makes a tweet credible?
You hear an important person has died, so you turn to Twitter search. Or you see the person’s name is a trending topic and follow the link.
Oh no. RIP Leonard Nimoy
— Mike Monteiro (@Mike_FTW) March 12, 2012
You begin hunting for clues — subconsciously, probably — to evaluate credibility. Does this user have a “real” profile photo, an anime character, an egg? What is his ratio of followers-to-following? Does his bio suggest expertise in the subject area? In seconds — as few as three, according to one study — you’ve made a judgment.
(For the record, Leonard Nimoy is alive and well! @Mike_FTW is a serial faker.)
Such is the premise of a recent study from Microsoft Research: “As users increasingly access tweets through search,” the paper notes, “they have less information on which to base credibility judgments as compared to consuming content from direct social network connections.”
We trust people in our cultivated social networks. How do we determine the truthiness of a tweet from a stranger? It turns out we evaluate all kinds of clues about credibility, but we’re ultimately pretty bad at separating truth from fiction.
Lead researcher Meredith Ringel Morris surveyed avid Twitter users to identify 32 features of a tweet that help determine credibility. What features were associated with low credibility? The use of non-standard grammar and punctuation; not replacing the default egg account image; using a cartoon or avatar as an account image; and following a large number of users but being followed by few.
Features associated with high credibility “generally concerned the author of the tweet”:
Morris used the data to design two controlled experiments last year with 552 people. Her team essentially recreated the Twitter website and loaded it with made-up tweets that were either true or false but plausible. (The researchers pilot-tested the tweets on fellow staffers to ensure they couldn’t easily detect if a statement was true or false.) She made up gender-neutral names and paired them with either a real photo, a cartoon character, a logo, or egg icon. Respondents were asked to rate the credibility of the tweet, the credibility of the author, and the veracity of the claim.
A false tweet with credible features, or a false tweet from a credible person, might as well be true.
Users attributed a lot of credibility to the user name of a tweet author and, to a lesser extent, the profile image. “Authors with topical names were considered more credible than those with traditional user names, who were in turn considered more credible than those with internet name styles,” the paper says.
So: @AllPolitics > @Alex_Brown > @tenacious27. Users with non-photographic images (a logo, the egg) were viewed as much less credible than those with photos.
If a person’s avatar is a cartoon character, Morris said, it’s easy to dismiss the image as not real. (Sorry, Felix Salmon.) But it’s more difficult to be skeptical of a photo of a real human. “People are actually ascribing a lot of importance to these very small details that are actually very easy to manufacture,” Morris said.
Respondents rated tweets about science significantly more credible than tweets about politics and entertainment, perhaps because science is seen as a more serious and less politicized topic.
What’s more, the most seasoned Twitter users turned out to be the least skeptical. “People who had been users of Twitter longer, people who had more Twitter followers…people who had more expertise with Twitter overall gave higher credibility ratings than those who didn’t,” Morris said. They rated more tweets as true than did Twitter newbies.
It seems counterintuitive, Morris said, but previous research has suggested people put more trust into technology the more they’re familiar with it. (There is evidence to the contrary, too.)
The strongest indicator of a tweet’s credibility? In the survey, respondents gave very high credibility ratings to tweets or retweets that come from a user they trusted. A retweet from a trusted user scored even higher than a tweet from a “verified” user.
Curators have great power and great responsibility.
That affords great power, and great responsibility, to curators. A tweet from a nameless, avatar-less, or otherwise unknown Twitter user is likely to make no impact, but a retweet from, say, Anthony DeRosa or Andy Carvin, gives it great weight.
That’s why seemingly credible users can spread truthy claims. A false tweet with credible features may be just as potent as a true tweet.
Consider the case of Amina Arraf, the “gay girl in Damascus” who turned out to be Tom MacMaster, a straight dude in Edinburgh. By constructing a fake but plausible identity — with a real-sounding name and a real person’s photo — MacMaster was able spin a narrative that fooled even seasoned journalists. MacMaster was able to override our built-in skepticism by constructing the features of credibility we value most.
One of the problems with this study, Morris acknowledges, is that Twitter has been redesigned either once or twice (three times? I’ve lost track) since the study was conducted in 2011. Still, it’s an interesting user-interface question: Can we design websites and search engines to better highlight the details that help us make credibility judgments?
Morris is now conducting a follow-up study to see if she can identify differences across cultures. She is studying Twitter users in the United States and users of the Twitter clone Weibo in China.
