“There’s are a lot of people in the news industry who are very skeptical of anything that isn’t news,” says The New York Times’ John O’Neil. As the editor of the Times’ Topic Pages, which he calls a “current events encyclopedia,” O’Neil oversees 25,000 topic pages, half of which — about 12,000 or so — include some human curation.

While the rest of the newsroom is caught up in the 24-hour news cycle, constantly churning out articles, O’Neil and his team are on a parallel cycle, “harvesting the reference material every day out of what the news cycle produces.” This means updating existing topic pages, and creating new ones, based on each morning’s news. (The most pressing criterion for what gets updated first, O’Neil said, is whether “we would feel stupid not having it there.”) A few of the Times’ most highly curated topics include coffee (curated by coffee reporter Oliver Strand with additional updates by Mike White) and and Wikipedia (curated by media reporter Noam Cohen),  as well as more predictably prominent topics like Wikileaks and Egypt.

The Topics team includes three editors and two news assistants, who work with Times reporters. “People give us links to studies they’ve used for stories or articles they’ve looked at, and this is something that we do hope to expand,” O’Neil said.

But half of the topic pages are “purely automated,” O’Neil said. And O’Neil is even contemplating contracting the number of curated topic pages, as people and events drop out of relevance. (The Topic pages garner 2.5 percent of the Times’ total pageviews.) O’Neil said he had read a statistic that roughly a third of Wikipedia’s traffic came from only about 3,000 of its now more than 17 million pages. “We’re concentrating more on that upper end of the spectrum.”

In a phone conversation, I talked with O’Neil about why the Times has ventured into Wikipedia territory, how the Times’ model might be scalable for local news organizations, and why creating a “current events encyclopedia” turns out to be easier than you might think. A condensed and edited version of that conversation is below.

LB: How did the topic pages develop?

JO: Topic pages began as part of the redesign in 2006. Folks up in tech and the website realized they could combine the indexing that has actually gone on for decades with the ability to roll up a set of electronic tags. The first topic pages were just a list of stories by tag, taking advantage of the fact that we had human beings looking at stories every day and deciding if they were about this, or were they about that. Just that simple layer of human curation created lists that are much more useful than a keyword search, and they proved to be pretty popular — more popular than expected at the time.

LB: What’s the philosophy at the Times behind the topic page idea?

JO: Jill Abramson‘s point of view when she started looking at this: When she was a reporter, she would work on a story for days on end, weeks on end, and pile up more and more material. You end up with a stack of manila folders full of material, and she would take all of that and boil it down to a 1,200-word story. It was a lot of knowledge that was gained in the process, and it didn’t make it to the reader. The question was: How can we try to share some of that with the reader?

My impression is that people find these pages terrifically useful. Not everybody comes to a news story with the knowledge you would have if you’d been following the story closely all along. News organizations are set up to deal with the expectation [that people] have read the paper yesterday and the day before.

LB: How do you go about transforming news stories into reference material? What does the process look like?

JO: What we found, as we did this, is that the Times is actually publishing reference material every day. It’s buried within stories. In a given day, with 200 articles in the paper, about 10 percent reperesent extremely significant developments in the field. Now we can take a small number of subjects, like Tunisia or Egypt or Lebanon or the Arizona shootings, and keep on top of everything, set the bar higher. We can really keep up with what the daily paper’s doing on the biggest stories.

LB: As you note, there’s a lot of wariness among “news” folks  around putting  effort into topic pages. For instance, when I talked with Jonathan Weber of The Bay Citizen, the Times’ San Francisco local news partner, about topic pages, he told me: “people are looking for news from a news site….We’re not Wikipedia. You don’t really go [to a topic page] for a backgrounder, you go there for a story.” How would you respond to that?

JO: Our experience has been that that’s never been entirely true, and it’s becoming less true all the time. Look at the pound-and-half print New York Times, and think how much of that is about things other than what happened yesterday. Even in the print era, that was a pretty big chunk.

Then again, it makes sense for folks at a place like The Bay Citizen to be more skeptical about topic pages. A blog, after all, is all about keeping the items coming. And a site focused on local news would feel less need to explain background — hey, all our readers live here and know all that! — than if they were covering the Muslim Brotherhood, for instance.

LB: So what about the Wikipedia factor? Why should the Times be getting into the online encyclopedia business?

JO: I think Wikipedia is an amazing phenomenon. I use it. But there’s no field of information in which people would find there to be only one source. On Wikipedia, there’s the uncertainty principle: It’s all pretty good, but you’re never sure with any specific thing you’re looking at, how specific you can be about it. You have to be an expert to know which are the good pages and which are the not-so-good ones.

Our topic pages — and other newspaper-based pages — bring, for one thing, a level of authority or certainty. We’re officially re-purposing copy that has been edited and re-edited to the standards of The New York Times. It’s not always perfect, but people know what they can expect.

LB: What’s the business-side justification for the Topic Pages?

JO: We know that the people who come to the topic page are more likely than people who come to an article page to continue on and look at other parts of the site. It helps bring people to the site from search engines.

It’s also brand-building; it’s another way people can form an attachment. People can also subscribe to topic pages. (Every page produces an RSS feed.) We’ve begun to do some experimenting with social media. There are lots of people who want to like or follow or friend The New York Times, but a topic pages feed gives you a way of looking at a slice of this audience. It turns the supermarket into a series of niche food stands, so to speak.

LB: The Times obviously has a lot more resources than most local news outlets. Is developing topic pages something of a luxury, or is it something that makes sense to pursue on a more grass-roots level?

JO: At the Times, less than one half of one percent of the newsroom staff is re-purposing the copy. That makes it of lasting value, and makes it more accessible to people who are searching. If you think about a small regional paper, three editors would be a huge commitment. On the other hand, the array of topics on which they produce a significant amount of information that other people don’t is small. There’s a relatively small number of subjects where they feel like, “We really own this, this is key to our readership and important to our coverage.”

If people think of topic pages as the creation of original content on original subjects, it never looks feasible. If you think about it as re-purposing copy on your key subjects, I think it’s something more and more people will do.