data visualization over time

Question

I have a decent sized set of data - 500,000 records, each representing an event.

The fields are:

• date
• time
• who did the event
• what the event was (there will be say 5-10 type of events)
• where the event was (there will be 20 or so locations for an event)
• 'other things'

there are about 500-1000 events per day

My ultimate fantasy would be to have a video pegged at a certain time-scale - it's just short of 2 years of data, so make the video (for discussion) 2 minutes, each minute is a year, and so on. Each location would be shown as a circle of a certain size that would grow as events 'happened' at those locations. An even kinkier version of this fantasy is where each circle is a pie-chart and the pie-segments grow/shrink based on the type of event at each location.

Are there some ready-made tools, or open source libraries for python, or a web site that anyone would recommend?

Is this the kind of thing you hire someone at some large rate to do for you?

Answer 1

There are two JavaScript libraries that you might find helpful for pulling something like this off:

• Raphael.js
• Processing.js

Otherwise, there are many, many charting libraries that you could adapt, some Flash, some PHP, some jQuery. Digg used to do some really cool visualizations in Digg labs, but unfortunately, all that remains now after their redesign is this video.

Answer 2

Check out this presentation by Hans Rosling.

It's available on gapminder.org and on Google Spreadsheets.

Answer 3

As you have locations, the result may be an interactive map. Open-source interactive map software are not very numerous, but you can generate static images and then assemble them in an animation.

A great combination of software is R + python + FOSS GIS (free and open source, like GRASS or QGIS). Some introductory educational content are listed here : http://www.osgeo.org/educational_content

As a professional cartographer, i was asked recently to create an on-line visualization tool of quantitative values spatialized as points, over time, and i used the GeoExt lib, a bridge between Sencha's Ext.JS web framework and OpenLayers mapping framework, but it's perhaps a bit complicated.

Answer 4

You might check this out: http://insideria.com/2009/12/28-rich-data-visualization-too.html

Answer 5

There are a bewildering array of possibillites for geospatial data handling. In fact, you could use Google Earth for this. Google earth has timeline in-built and masses of functionality. It is no longer stand-alone and is easily integrated in web.

Here is a long list of geospatial tools.

gephi

Gephi in itself is a network visualisation tool, but there are loads of plugins, also for geospatial.

Gapminder See this phenomenal TED presentation by Hans Rosling - love that guy

Gapminder was mentioned above, but could easily be mentioned again :)

Google charts

Google charts can handle big data, but you have to check the limitations and wether you would have to pay for a big-data account.

somvis

A pretty tool on data mining large datasets.

Raphaël

Also mentioned above. To my knowledge, Raphael has some limitations as to really big data.

MIT Exhibit (previously called Similie)

Been a while since I played with Similie, but recall that they have capacity for big data. The visuals might not be the most stylish, but that you could do yourself.

d3

One of the good-old-ones, highly adaptable javascript.

Highcharts

Very professional and versatile (I love that they link directly to jsfiddle for their charts). You can pay for the high end solutions, and that should easily handle big, live data.