Rethinking Performance Management for Big Data

By Bala Venkatrao

A recurring theme over the last few years has been that Big Data technologies are fading away. What is thriving are Big Data Applications, from ETL pipelines, Analytics to ML products, that leverage the Big Data Stack (Hadoop, Spark, Kafka, Cassandra, Elasticsearch etc.) to build meaningful solutions with business impact.

However, as we transition to this “application” view of Big Data, we still seem stuck in the old ways of managing the systems that support these applications. While we absolutely need system management tools to setup and monitor these platforms, we need to re-think how we manage applications more holistically and address their business requirements (SLA Management, MTTR, big data operations productivity etc.) – we need Application Performance Management for big data.

Unravel is Application Performance Management for big data. By smartly leveraging data from across the stack (applications, platforms and infrastructure) and leveraging modern data science and machine learning principles, we provide a guided path to address issues related to performance, failures and resource utilization for your Big Data Applications. You have to try it to believe it!

I was recently at a networking event where I presented an analogy to what we do at Unravel that resonated very well with the audience:

Big Data Infrastructure and Applications are complex. It is like putting an airplane together — think of the core body of the plane as “HDFS”, the wings like “Kafka”, and the engines like “Spark” and “Hive”. Each of these airplane components have monitoring and management associated with them, but what brings all of this together is the “avionics” that is co-ordinating the various aspects of the airplane and many times guiding it in an “auto pilot” mode. Unravel’s vision is to be the “avionics” for your Big Data Applications, so you can successfully complete your journey from “Data to Insights”.

Learn more about how Unravel is mission critical to run big data in production here!

* Image from