Visualizing Big Data: Best Practices

When can you call your data “big”?

That is a murky question that can’t really be answered with a single google search. For the sake of this article we will say that when the size and complexity of your data makes working with it a hindrance using normal tools and methods, you then can grant your data the title of “big”. Once your back-end team has tackled the complexity and size of your data (probably using Hadoop), what now? You have found the trends, spotted the outliers, solved the algorithm; now the only thing left is to deliver these findings to the user.

Remember, the average user has an attention span of 8 seconds and an ever-growing need to be spoon fed content (if you have read this far you can consider yourself in the 1% of online knowledge seekers). Due to the short attention span of the average user, developers and designers often utilize graphic visualizations to convey the intended message. This is usually the most appropriate move. Hopefully visualizations will allow you to deliver the core finding of your data in an easy to digest format. I say hopefully because there are a lot of places to go wrong.

The most often found mistake is visual overload. Coming from such a large and complex data set it is easy to overload the user. While the 4 bar charts, 2 multi-line graphs, 1 heat map and 3 pie charts look exciting sitting together on a single page all interacting with each other as you hover past, this is exactly the opposite of what you want. All the backend data manipulation you did to simplify your big data can easily be undone by a less than thought out UI. The key is to be concise with your visuals. Focus on one finding at a time and let the hard work you did squeezing meaning from your data shine. As a general rule of thumb, the user should be able to interpret your data and see any valid findings without any action. Now once that has been accomplished here comes the fun: interaction.

Allowing the user to alter the scale, scope, correlations, and other factors is one of the most powerful aspects of using a visualization. Actionable elements of the UI should be obvious in both their presence and resulting action. This becomes particularly important when you are dealing with more than a single visual element on the page. If selecting a state from a map of the U.S. is going to alter a bar chart, it should be obvious which state’s data is being shown on both the map and the chart. The ultimate goal is to take a big, ugly, convoluted database and turn it into iOS 7. Just enough interaction to convey the message you intend, but not enough customization that the user looses track of your intended meaning.