metaLayer, Visualizing Online Chatter During Hurricane Irene

Visualizing Online Chatter During Hurricane Irene

Our friends at a firm called GeoSprocket gave a great breakdown yesterday of how they used a combination of technologies to discover trends in online conversations during and after Hurricane Irene. Their challenge, though, is that most Twitter messages don’t carry location data and to get all this content on a map might prove incredibly time consuming for a human.

Our dataLayer API aims to make this a far simpler process through a technique called location disambiguation (also known as place finding). This is done by using the other contextual clues that follow online communication, for instance the language someone might used can be combed for words that might appear to be places. ’50 Broadway’ may exist in 100 places in the world, but if it appears next to the word ‘New York’ the algorithm assumes that it’s 50 Broadway, New York, NY which has the Latitude and Longitude coordinate of 40.7061622, -74.0128389. Since this is done algorithmically, it gives the appearance of messages with no location elements, being mapped somewhat magically.

It’s important to note that neither location disambiguation nor sentiment analysis are perfect sciences but the method can greatly increase the percentage of online chatter that can be mapped in the absence of all other information. Here’s what GeoSprocket Director Bill Morris had to say about his project…

This is where new developments in “Big Data” analytics come in handy. With some computational heavy lifting from Kate Starbird at the University of Colorado and Chris Danforth at the University of Vermont, Geosprocket was able to bank millions of Twitter posts from the days surrounding the storm. Then the assistance of metaLayer Inc. was instrumental in putting Irene-related tweets on the map. Using a series of digital sifting processes, they were able to mine the archived Twitter data for placenames and keywords. Where a town or street name was included in a post, that post could be placed at a set of geographic coordinates. Were words like “washout” and “devastated” were used, fine-tuned algorithms could assign a scaled value for the sentiment of the post.

Add a bit of cartographic styling and serving with the open-source MapBox toolkit, and we’ve got an interactive mapped timeline of Hurricane Irene, as told by Twitter.

We’re really excited we were able to contribute to GeoSprocket’s project and look forward to what they come up with in the future! Check out the interactive map of Hurricane Irene tweets made using the dataset here.

November 30, 2011

23 notes /