Visualizing data

There is more data being produced today than anyone can get a useful handle on (more information has been produced in the past five years than in the total of all of previous human history). Hence the growing interest in new ways to search it and understand it, and the emergence of increasingly sophisticated data mining tools. If the Google IPO is any indicator, bringing clarity into info-chaos has real value.

Now, in conjunction with The Hive Group -- vendors of visual approaches to business analytics -- NewsIsFree has come up with a visual map of the news. You can decide on criteria such as popularity or recentness to determine size and color of the items represented.

The Hive Group is the same company that produced the visual map of a Google search. It's a great start, but the map is somehow not very helpful. The only dimensions you have to indicate importance are size and color. A 3D fly-through approach would be much more useful, but of course you'd need a high-end machine and some very potent software to do it.

Java aps have their limitations when it comes to data visualization. I like the interactivity of blogalization from TouchGraph, and their more elaborate ThinkMap and InXight products are very cool. At the high end of the price range (we're talking five to six figures) are the integrated 3D systems from companies like Visual Sciences (not strong on Web presence themselves but very hot at helping others see their online performance using real-time collection, processing, analysis and visualization of their data).

I have always wondered about the disconnect between the poor graphics quality that we accept in our business apps and the high graphics quality we demand in our entertainment apps. With this week's launch of Doom 3, the gap just widens. We don't live in a 2D world, and our data is multidimensional. Yet most of us still try to understand it by pulling it into flat gray tables in Access. The dimensions by which you analyse data usually determine the nature of the insights you will extract; it's typically impossible to discover unexpected relationships of any sophistication when your lens is a spreadsheet.

I'd like to think that the next great leap forward in tools to mine the metaNet will be multi-dimensional, un-preconditioned, and creative, with a touch of real-world, real-time insight thrown in, all very graphically rich. Maybe that's a blogger, with a Web-cam.