InsightIQ: Transforming Data into Information

OneFS and Isilon IQ combine to be the best Scale-Out NAS platform – but one of the biggest complaints our customers have had is a lack of information into the product itself. It isn’t easy to do what we’ve done – make a complicated distributed system act simple and hide the complexities from the user.

In doing so, however, we’ve also hidden from the user how to maximize their application in this new paradigm of storage systems. In most cases users can simply plug an Isilon system into their existing environment and see increased benefits over traditional storage – but to really maximize their workflows, they need to understand whether they’re taking advantage of all this new power.

No longer. We have (until this post) quietly been distributing the InsightIQ Preview Edition. This isn’t a full product yet, but it is fully functional and allows our customers to have deep introspection into the performance aspects of their system in a highly visual manner. I am happy to say that I have over 30 customers already using the Preview Edition – many in production.

InsightIQ is a unique analytics platform. It is a virtual appliance (initially packaged for VMware) which is based on Ubuntu Linux and provides a rich Web 2.0 interface that customers interact with using their favorite web browser. The data collection, aggregation, transformation, and display are all handled by the virtual appliance – which significantly offloads processing from the storage system itself and provides fault isolation as well. OneFS does ultimately collect the statistics and provide them via a REST-based API to InsightIQ.

I spoke briefly about InsightIQ at SuperComputing 2009 (at Intel’s booth) and the reception from customers has been overwhelmingly positive. The ability to see per-client metrics (such as protocol latency, throughput, IOps, etc.), to slice and dice that across different axises (such as node, protocol, interface, etc.) and to do so not only with “real-time” data but in a historical context … wow. In retrospect, I don’t know if the product name does justice to the capabilities. Already we’ve had several customers identify and correct workflow issues – ways in which their applications were non-optimal – and accelerate their time-to-market with a few tweaks. AccelerateROIQ just didn’t pass the sniff test though …

I’m very interested in collecting as much customer feedback and impressions during the preview period as possible. If you’re an Isilon customer and you haven’t heard of InsightIQ, I encourage you to let me know. Just a brief warning before you get too excited – InsightIQ does require that you upgrade to OneFS 5.5.3 or higher.

About the Author: Nick Kirsch