I love the idea of #NewsGuard. Rating real and fake news sites to empower readers with information to decide for themselves where they get their news. I'm also curious how @FoxNews will rate. #RealNews #FakeNews https://t.co/fQ0L2Zuuc3

— Freak'n Einstein🌊 (@Einsteinin209) March 6, 2018

Will they include Fox News? that alone would keep them busy. NewsGuard, new venture to combat 'fake news' https://t.co/3ZfpkaGL0d via @physorg_com

— Sylvan the Superpup (@sentient_deist) March 6, 2018

Any chance you can slap a rating on Fox News similar to the ones on cigarette packs?

— Bob Lynch (@LynchpinL) June 8, 2018

Thank you for taking the fake out of #FakeNews. Your rating service is needed by news editors who routinely amplify dezinformatsiya in an effort to present both sides of any argument even though the other side is RT, Breitbart, Stone Cold Truth, FOX https://t.co/oIIGse9AXk. pic.twitter.com/MbvCQWWmKp

— American RESISTANCE (@KeepLadyWarm) March 7, 2018

Brill and Crovitz Announce Launch of NewsGuard to Fight Fake News https://t.co/xOXU481StE solution to the last 3 years of BS news, scared senators & congressmen and advertisers determining what is true or not (think FOX News)? Watch!

— christine still (@christinestil19) March 16, 2018

Or as our own Ken Doctor put it in March, writing about the green-for-good, red-for-bad color system the company uses:

NewsGuard could be a simple, elegant solution to both the honest hand-wringing and the overwrought blather about fake news. But we still have to ask the question that NewsGuard won’t answer until it has to: What color is Fox News?

Well, NewsGuard officially launched the first public version of its primary tool (a browser extension) yesterday, which means “What color is Fox News?” is now officially a question it has to answer. And the answer is:

Green.

Which means, in NewsGuardese, that “This website generally maintains basic standards of accuracy and accountability.”

Or, as the company describes its ratings: “Our Green-Red ratings signal if a website is trying to get it right or instead has a hidden agenda or knowingly publishes falsehoods or propaganda.”

Or, as it also puts it, the green and red ratings let you “easily determine which you can trust and which you should read with caution.” Phew, that caution was getting tiring!

These are some other sites “you can trust,” according to NewsGuard: the conservative Daily Caller and Daily Wire, and the liberal Second Nexus and PoliticusUSA.

These are ones you should “read with caution”: Palmer Report (“highly speculative…misleading”), The Political Insider (“regularly publishes false and misleading stories”), Hannity (“advance[s] a variety of conspiracy theories”), Gateway Pundit (“regularly distorts information”), Breitbart (“sometimes distorts or omits facts to fit its agenda”), the Daily Mail (“repeatedly publishes false information”), 100PercentFedUp (“inaccurate stories and headlines”), Bipartisan Report (“anonymous articles…sensational headlines”), Shareblue (“charged, misleading language”), and Daily Kos (“often add[s] commentary and exaggeration”).

More on them below, but first back to Fox News.

NewsGuard dings Fox News on 3 of the 9 factors on which it rates news sites, saying it has issues with regularly correcting its errors, disclosing its ownership and financing, and revealing “who’s in charge, including any possible conflicts of interest.” On the other 6 factors — not publishing false content repeatedly, gathering and publishing information responsibly, avoiding deceptive headlines, differentiating news and opinion “responsibly,” clearly labeling advertising, and providing information about reporters — Fox News passes.

Does any of that seem strange to you? NewsGuard’s explanation — which you can see if you add the browser extension and go to foxnews.com yourself; unfortunately, I can’t link to it — illustrates the difficulty in coming up with a way of rating news sites that is systematic and consistent. For example: Two of the three areas where Fox fares poorly involve…I guess having a sub-par about page? Is the fact that Fox News is owned by 21st Century Fox a secret? Is Rupert Murdoch’s role in the company unknown? NewsGuard falls back on some fairly legalistic thinking on this one:

It only discloses its ownership by 21st Century Fox within the content found by clicking on a Terms of Use button at the bottom of the site. It does not link to any information about 21st Century Fox, including its control by conservative and Trump confidant Rupert Murdoch. The corporate information area of the site’s media relations link notes that Murdoch is the executive chairman but does not specify ownership.

Those seem, to put it charitably, relatively low on the list of issues one might have with Fox News.

NewsGuard notes that its evaluation is of the Fox News website, not the TV channel:

Foxnews.com offers breaking, political, and entertainment news, live video of major events, and both news and features on the environment, science, technology, sports, weather, and health. It resembles the more traditional news coverage found on Fox News Channel during much of the network’s morning and afternoon programs, including shows hosted by Shepard Smith and Bret Baier. The website should not be confused with the programming of Fox News’ flagship prime time shows. It diverges often in subject matter and tone from the opinion-driven evening programs associated with former longtime Fox News mainstay Bill O’Reilly and now the hosts Sean Hannity, Tucker Carlson, and Laura Ingraham.

Have a look around FoxNews.com and see what you think. Or see what the top of its homepage looked like Tuesday evening shortly after President Trump’s campaign manager and personal attorney racked up 16 felonies between them. (Here’s what CNN.com looked like at the time.)

This the Fox homepage right now. Seriously>>> pic.twitter.com/t4wUMUnIrG

— Susan Glasser (@sbg1) August 22, 2018

I’d also argue it’s pretty hard to differentiate Fox News the network from Fox News the website — given that the website’s homepage livestreams the network, and that video of all those TV shows is both on the website and quite popular there! Last month, according to comScore data, FoxNews.com served more video streams on desktop alone (133 million) than it had monthly unique visitors on desktop and mobile combined (90 million). Here’s a page on FoxNews.com, for instance, where you can see Sean Hannity batting around Seth Rich/WikiLeaks conspiracy theories on air.

But look, I’m not here to complain about Fox News. Whatever your opinion of the network, it’s unlikely to be changed by me or by NewsGuard. I’m not even here to particularly complain about NewsGuard. They’re trying to attack a real problem in misinformation; they’re doing interesting work that I enjoyed exploring. But the upside of this sort of labeling — a quick way to make a judgment at a glance — is in direct tension with the nuance modern media literacy requires. Especially if you want to help “readers and viewers know which [sites] are trying to do legitimate journalism — and which are not.”

In Jacobellis vs. Ohio, Supreme Court Justice Potter Stewart was faced with the challenge of defining “hard-core pornography” in legal terms. Perhaps, he mused, he “could never succeed in intelligibly doing so. But I know it when I see it.” I applaud NewsGuard’s efforts at categorizing the web — but sometimes, you just know it when you see it.

With the release of NewsGuard’s tool, I spent a couple hours furiously navigating the news/“news” web, looking for what ratings work had already been completed. And most of it seemed perfectly well done — if, again, sometimes too caught up in technicalities to capture a site’s zeitgeist.

Among the green sites I saw: national newspapers (The New York Times, The Washington Post, The Wall Street Journal, USA Today, The Guardian); network news operations (ABC News, CBS News, NBC News, PBS Newshour, BBC News, CNN, MSNBC, NPR); mainstream magazines (The New Yorker, The Atlantic, GQ, Forbes, National Geographic); all the major metro newspapers I checked; and major digital natives (BuzzFeed News, Deadspin, Business Insider, TechCrunch, Vox, The Daily Beast, Quartz, The Verge).

Also green were the majority of politics sites I saw, even those with a pronounced slant: leaning left, sites like Salon, Talking Points Memo, HuffPost, Mic, The Daily Beast, and Media Matters; leaning right, Daily Caller, Daily Wire, IJR, The Daily Signal, The Federalist, Townhall, and the Washington Examiner. (I’m sure plenty of people would have arguments against several of those sites’ greenness.)

So what’s red? A few liberal sites: Shareblue, Bipartisan Reports, Palmer Report, and (surprisingly to me, at least) Daily Kos. (NewsGuard’s complaints there are mostly about the work of non-staff contributors to the site.) What seems, from my searching, to be a somewhat larger number of conservative sites: Breitbart, RushLimbaugh.com, Hannity, The Blaze, Infowars, Conservative Tribune, Right Wing News. (As far as I can tell, NewsGuard doesn’t provide a centralized list of sites its reviewed, so there are no doubt lots of sites I’ve missed.) Plus propaganda sites (RT, Sputnik), tabloids (National Enquirer), bogus health sites (Natural News), conspiracy sites (Infowars), and what could fairly be labeled as fake news sites (Your News Wire, 100PercentFedUp).

There are also still many news/“news” sites that haven’t yet been rated. Some that I found: FiveThirtyEight, The New Republic, lots of alt-weeklies, RealClearPolitics, Twitchy, Drudge Report, American Spectator, Democracy Now, The Weekly Standard, Circa, RedState, HotAir, WikiLeaks, Liberty Writers News, Lifezette, Reveal, the Center for Public Integrity, Jacobin, Taki’s Magazine, PolitiFact, Occupy Democrats, Addicting Info, VDARE, The Fiscal Times, AlterNet, Free Republic, Prison Planet, Lew Rockwell, and Project Veritas.

What about Facebook pages or tweets or YouTube videos or /r/conspiracytheoriesarefun? None of their sites or subsections get a green or red rating; they get a yellow badge that notes they contain lots of unvetted user-generated content that “may not be reliable.” (NewsGuard can sometimes detect links to rated sites shared within social platforms, though.) The same is true of Wikipedia; since NewsGuard hasn’t gotten to every sketchy site on the web yet, you can sometimes get search results that tell you the Wikipedia entry on the Daily Stormer is something to be cautious about — but is silent on the Daily Stormer itself.

Finally, you may have seen this chart by Vanessa Otero that circulates around every so often. It attempts to place a number of news organizations on a two-axis grid, aiming to measure its ideological slant and its journalistic quality. I’d have some quibbles with it, as I’m sure you would too, but it’s a solid effort. I looked at all the sites on the chart to see how NewsGuard rates them; here’s the result (click to embiggen). As you can see, NewsGuard’s and Otero’s evaluations line up pretty well.