Diseases and Disasters (Dis2): Riff’s Performance

A couple of years ago, I introduced Riff; a hybrid (event-based and indicator-based) disease surveillance platform, at the International Society for Disease Surveillance (ISDS) [the original concept is fully described here - more up-to-date information can be found on InSTEDD's website here].  Riff was designed to streamline the collaboration between human experts dealing with diverse streams of information and enabling them with machine learning algorithms that can learn quickly and accurately classify the information for detection, prediction and response to health-related events (such as disease outbreaks or pandemics).  On Jan 17^th 2010, the Thomson Reuters Foundation used Riff [after prior adoption in their EIS system; an Emergency Information Service for survivors of natural disasters (early work can be found in Nico’s Blog here)] to launch a first-of-its kind, free disaster-information service for the people of Port Au Prince, Haiti. This allowed survivors of Haiti's earthquake to receive critical information by text message directly to their phones, free of charge.

Earlier this year; April 2010, I setup a Riff space; Diseases and Disasters (or Dis²), for providing timely situation awareness from credible and reliable [gold standard of a sort] online reports on diseases and disasters in the world provided primarily by BioCaster and HealthMap. This already tagged and verified information provided Riff’s classifier with a great training opportunity that I had to monitor closely and correct at the beginning for both features (e.g., condition, type of disease transmission, severity, etc.) and geo-location of where the event actually occurred as shown here:

Conditions Tracked in Dis2 Proportional to their Coverage since April 19, 2010

%Conditions Tracked in Dis2 since April 19, 2010

Location of Conditions Tracked in Dis2 Proportional to their Coverage since April 19, 2010

Location (Heatmap) of Conditions Tracked in Dis2 since April 19, 2010

In collaboration with the Humanitarian FOSS Project; supported by a group of computing faculty and open source proponents at Trinity College, Wesleyan University, and Connecticut College (Open Source ALPACA Light Parsing And Classifying Application (ALPACA) and Open Source e-dop for Disease Ontology Prediction for Riff), we developed a Support Vector Machines (or SVM) for automatic feature extraction, data classification and tagging. Riff’s classifier performed incredibly well in the Dis²collaborative space with very little training at the outset; 83% (95% CI: 81-85%) True Positive rate as shown here:

Riff's Performance in the Dis2 Collaborative Space (True Positive and False Negative Ratios)

Riff's Performance in the Dis2 Collaborative Space (True Positive and False Negative Ratios)

Riff is an Open Source Project, you can download its source code here and help further enhance its performance. 

Acknowledgment

• Nicolás di Tada and Eduardo (Ed) Jezierski: InSTEDD, Palo Alto, CA, USA
• Nigel Collier: BioCaster - National Institute of Informatics and PRESTO Japan Science and Technology (JST) Corporation, Tokyo, Japan
• John Brownstein and Clark Freifeld: HealthMap - Children’s Hospital Informatics Program at Harvard-Massachusetts Institute of Technology, Division of Health Sciences and Technology, Division of Emergency Medicine, Children’s Hospital Boston, Boston, Massachusetts, USA
• Humanitarian FOSS Project

Upper Big Branch South Mine Explosion: Could Data Mining Have Prevented a Coal Mine Disaster?

The April 5^th explosion at the Upper Big Branch South Mine (Performance Coal Co.) in Whitesville, W.VA. got me wondering, what type of data was available that somehow would have suggested this mine was at a dangerous point. I was able to find the citation information published on the United States Department of Labor, Mine Safety & Health Administration (MSHA) [under Data Transparency at MSHA]. My colleague Paul McMurray and I found that out of the 124 total citations in 2010, ~43% (53 citations) were issued in the month of March 2010 with a potential penalty of $1,077,339 [where we expressed it as the maximum penalty assigned a given standard historic citation since 2007-01-01)]. It’s also worth noting that this mine had over 500 citations in 2009 (~46% of all violations since 2007). Without exploring the data for other mines, these numbers seem to be high enough by themselves to suggest that this mine could have been a candidate for a greater level of scrutiny by federal, state and local authorities that are responsible to overlook mines safety.

The following table shows the Upper Big Branch Mine-South Citations, Orders and Safecards Detail Issued for March 2010 [Google Fusion Table (collaborative online spreadsheet)]. Further analysis [e.g., data mining] should look into the type of standards to see if we can categorize the various violations of these standards [e.g., ventilation problems, escape routes, etc.].