Graduate and Postdoctoral Studies
Mining the Microblogging Site
Wednesday, July 23, 2014
to 3:00 PM
105 Martel College
We present measurements and analysis of censorship on Weibo, a popular microblogging site in China. Since we were limited in the rate at which we could download posts, we identified users likely to participate in sensitive topics and recursively followed their social contacts, biasing our search toward a subset of Weibo where we hoped to be more likely to observe censorship. Our system enables us to detect post deletions within one minute of the deletion event, giving us a high-fidelity view of what is being deleted by the censors and when.
We found that deletions happen most heavily in the first hour after a post has been submitted. Focusing on original posts, not reposts/retweets, we observed that nearly 30% of the total deletion events occur within 5-30 minutes. Nearly 90% of the deletions happen within the first 24 hours.
By leveraging natural language processing techniques we also perform a topical analysis of the deleted posts, overcoming the usage of neologisms, named entities, and informal language that typifies Chinese social media. Using Independent Component Analysis, we find that the topics where mass removal happens the fastest are those that combine events that are hot topics in Weibo as a whole (e.g. , the Beijing rainstorms or a sex scandal) with themes common to sensitive posts (e.g. , Beijing, government, China, and policeman).
In the last, we apply our approach to see if social media can be a proxy measurement for pollution. Air pollution is a pressing concern for industrialized countries. Air quality measurements and their interpretations often take on political overtones. Similar concerns reflect the our understanding of what levels of measured pollution correspond to different levels of human nuisance, impairment, or injury. We consider air pollution metrics from four large Chinese cities (U.S. embassy/consulate data, and Chinese domestic measurements) and compare them to a large volume of discussions on Weibo. In the city with the worst PM2.5, Beijing, we found a strong correlation (R=0.82) between Chinese use of pollution-related terms and the ambient pollution. In other Chinese cities with lower pollution, the correlation was weaker. Nonetheless, our results show that social media may be a valuable proxy measurement for pollution, which may be quite valuable when traditional measurement stations are unavailable.