Nieman Foundation at Harvard
TAIPEI — I recently had the pleasure of teaching a two-day data journalism workshop at the Commonwealth Magazine Group in Taiwan’s capital city. The signature moment came when I showed these journalists the L.A. Times crime map, and explained that it updated automatically from government data feeds.
There were gasps from my audience. “Why does the government publish all of that crime data, online, for free?” they asked.
Commonwealth Magazine data journalism workshop in Taipei, Taiwan. Note bubble tea in foreground.
I realized I didn’t have a quick answer to that. In the United States, in 2013, it’s widely assumed that governments on all levels should make their data available for public use. But why? How did we get here? And, importantly, how do other countries get there?
Since then I’ve been trying to understand the history of open data in the U.S. The underlying right to data access comes from the 1966 Freedom of Information Act, and a 1996 update which requires that frequently requested information be made available online. Less obvious, but no less important: American journalists have long been agressive in asking, cajoling, or suing to get access to data. It is the combination of enlightened laws plus a long history of people claiming their rights under these laws that makes transparency seem so natural now. There are entire institutions devoted to the concept, such as the Sunlight Foundation.
Taiwan has a different history. The island nation has enjoyed a relatively open media environment since the end of martial law in 1987, and today it ranks 47th in Reporters Without Borders’ press freedom index, the highest in Asia. (The U.S. ranks 32nd.) There is a fiercely competitive ecosystem of non-state media organizations. Though there have been cases of censorship in the last decade, Commonwealth Magazine managing editor Yin-Chen Wu said government interference is never a concern for her.
There is also a Freedom of Government Information law, enacted in 2005. During the workshop, we talked about how to obtain data, and I asked if anyone had ever used this law.
“We have a freedom of information law?” was the answer. I had discovered it during my research for the workshop, but just because a law is the books doesn’t mean anyone knows about or uses it. Also, under Taiwan’s civil law system, there isn’t any case law which tells the courts how to interpret this right. A few veteran reporters did know of the law, but weren’t aware of anyone who’d ever filed a request under its provisions.
It’s not that the journalists at Commonwealth Magazine don’t want the data. Indeed, the magazine — sometimes described as the Taiwanese equivalent of The Economist — is an Asian data journalism pioneer, winning awards for its infographics and now moving into interactives. Neither are they politically constrained in what they can cover or how they can cover it. Rather, the difficulty is access. “We can write about anything, but there is this problem of getting correct information from the source,” Wu said.
Although the Freedom of Government Information law is not much used, there are other ways to get data in Taiwan, and a young open data community is advocating for access. T.H. Schee runs the IT consulting company Fertta.com and organizes many of Taiwan’s open data events. He has been keeping a spreadsheet of open data events since 2008, which now lists over 160 meetups, workshops, and hackathons.
The open data community is a diverse group of several hundred people, including technologists, activists, and domain experts from industry and government. Schee worked for the government himself from 2006 to 2009, at the Academia Sinica research institute, and said that the government experience within the community is part of what makes it successful. “And the program actually works,” he said, “because the government launched several data portals and they are more willing to talk to us.”
There are now five cities in Taiwan with open data portals. Schee’s talk at the W3 eGovernment interest group meeting in January describes the process of getting to this point.
The progression of Taiwan’s open data ecosystem, from a talk by T.H. Schee.
It’s not always the public sector that owns public data, especially in Taiwan.
According to Schee, there are only about 55 IT staff supporting the city of Taipei’s 30,000 public servants. Hence, most of the city’s IT functions are outsourced to private companies. These companies create and manage the majority of public data, and are often reluctant to release it online for free.
Tired of waiting, some people have published privately held government data without official permission. Power generation has been a hot issue in Taiwan since the Japanese reactor meltdowns in the wake of the 2011 tsunami, but official data on the capacity, safety, and radiation monitoring of Taiwan’s nuclear plants has been hard to come by. A small group of industry experts had this data, but lacked any official path to release it. So, they came to the Taipei Open Data Day hackathon and released it in a Dropbox folder.
Schee acknowledges that this act falls into a “gray area,” but notes that “the people who released it did not get in trouble. They have been in the industry for 20 years — they know the legal and ethical boundaries.” Yet simply releasing the data was not enough; they needed to get it to people who would use it. “The experts who released it had a hard time gathering attention among the young hackers and kids, so they came to our hackathon,” said Schee.
Meanwhile, many companies with public data have quietly put it online voluntarily. “The current problem is that the data is available, but it is rather fractured. If you want to write an article, it is very hard and labor intensive to find all the good data,” Schee said. To try to solve this problem, the open data community has created an extensive list of data sources in Taiwan.
Commonwealth Magazine journalists told me they usually obtain data by asking their contacts in government. There has long been data — but publicly accessible, well-organized, free data is a relatively new concept. Schee believes that the ultimate solution to the problem of public data in private hands lies in changes to procurement laws. “For example, if one company is doing map work for the government, they should release it. For this to happen, the procurement law would have to require this. “
In Taiwan, it’s easy to get certain types of information that might horrify Americans, and hard to get other data that Americans would deem obviously public.
“In Taiwan, you would not succeed in getting crime data on the web,” said Schee. “People feel it’s an ethical issue, that you should not put crime data on the web.” Yet images from Taiwan’s huge number of surveillance cameras are deemed less sensitive. “People here are very comfortable being filmed by road cameras, and journalists can get an image from a road camera just by asking the police.”
The furor over Taipei real estate prices illustrates another difference. Taipai housing prices have soared in the last few years, but until very recently transaction prices were a closely held industry secret. That changed at the end of 2012, with new regulations requiring brokers to post selling prices to a central online registry. But the resulting online system only displayed the price of one transaction at a time, so an enterprising group of four engineers took it upon themselves to scrape the entire database and produce a map of real estate prices in Taipei.
The online app got national attention, including print and TV reports. At its peak, over a million people viewed it a day. The app was built using commodity technology that will be familiar to news application developers, such as Open Layers maps and Heroku servers. Visitors found it far more usable and friendly than the official government app.
American data journalists might not imagine that real estate prices could be such a controversial subject, but the context is different in Taiwan. “By creating a layer of intransparency, [real estate brokers] can have much more margin on each transaction,” said Schee.
Comparing across cultures reveals your assumptions, and I think there are several important lessons in the similarities and differences between the American and Taiwan open data stories.
These lessons might apply not just to countries, but at any level of the open data ecosystem, from the local to the international.
I take one more lesson from my experiences in Taiwan, about the connection between open data movements and data journalists. At its best, the open data movement can support new forms of accountability, and enable innovative new businesses. But to be a part of this, journalists will have to understand and support open data. Even in the U.S., journalists have not always been aware of or involved in the diverse open data community, a community that includes many of the same government agencies that journalists are professionally obliged to suspect.
In Taiwan, the nascent data journalism community is just beginning to engage the broader open data community. “The data journalists are not yet connected to the diverse communities here,” Schee said. But he notes that there were four journalists at the most recent open data hackathon — an encouraging sign.
