Videos
- Cost Opt & FinOps
A PATH FORWARD & THE DILEMMA
DataFinOps for modern data stacks
First-generation FinOps tools were built to optimize web applications. Now modern data stacks enable faster analytics and AI, but costs are also on the rise. Unoptimized Databricks workspaces and clusters, BigQuery slots and I/O, and over-provisioned Snowflake clusters have cropped up all over the organization and cloud data spending has skyrocketed, making data management the biggest and fastest-growing cloud cost—about 40% of the typical cloud bill—leading to cloud cost overruns. DataFinOps brings FinOps to the modern data stack.
LIMITED VISIBILITY
It’s like stumbling in the dark
Most organizations can’t even see why their costs are growing. Before you can improve the efficiency of your cloud data spend, you first need full visibility into where spending is going and how teams are tracking towards their budgets.
Enterprise data estates frequently have millions of rows of cloud billing data that must be aggregated and analyzed before you can clearly see current spending patterns. Aggregating and analyzing millions of rows of cloud billing data can be a nightmare.
Every element of your cloud data estate should be categorized, tagged, and brought together into a single view. Cloud services don’t automatically do this work, yet it has to be done in order to see spending by team, pipeline, data application, product, or business division.
Without visibility into cloud data resource consumption and spending, you have no way to measure improvements.
LIMITED VISIBILITY
It’s like stumbling in the dark
Most organizations can’t even see why their costs are growing. Before you can improve the efficiency of your cloud data spend, you first need full visibility into where spending is going and how teams are tracking towards their budgets.
71%
of IT professionals said limited visibility across the hybrid cloud environment hinders their ability to maximize value, creates inefficiencies, and wastes time (Virtana)
39%
of cloud spending goes to data management, the largest and fastest-growing workload segment (IDC)
Enterprise data estates frequently have millions of rows of cloud billing data that must be aggregated and analyzed before you can clearly see current spending patterns. Aggregating and analyzing millions of rows of cloud billing data can be a nightmare.
Every element of your cloud data estate should be categorized, tagged, and brought together into a single view. Cloud services don’t automatically do this work, yet it has to be done in order to see spending by team, pipeline, data application, product, or business division.
Without visibility into cloud data resource consumption and spending, you have no way to measure improvements.
INACCURATE FORECASTS
Forecasting is a guessing game
The variable consumption model of the cloud enables greater agility and flexibility. What makes forecasting difficult is that resources are dynamically provisioned and consumed through code.
Across the organization, there is no single source of truth. Data engineering teams use one set of tools. DataOps teams use a separate set of tools. And Finance runs yet another set of tools.
Each team operates in silos, using different tools that lead to an information mismatch, confusion, and frustration. Forecasts end up 20-70% off base.
Without open and consistent communication between engineering, operations, finance teams, and executives, projecting cloud data spend is impossible.
INACCURATE FORECASTS
Forecasting is a guessing game
The variable consumption model of the cloud enables greater agility and flexibility. What makes forecasting difficult is that resources are dynamically provisioned and consumed through code.
80%
of data management professionals surveyed cited difficulty accurately forecasting data-related cloud costs (Forrester)
71%
of enterprises expect to move cloud workloads back on premises (IDC)
Across the organization, there is no single source of truth. Data engineering teams use one set of tools. DataOps teams use a separate set of tools. And Finance runs yet another set of tools.
Each team operates in silos, using different tools that lead to an information mismatch, confusion, and frustration. Forecasts end up 20-70% off base.
Without open and consistent communication between engineering, operations, finance teams, and executives, projecting cloud data spend is impossible.
UNOPTIMIZED CLOUD DATA SPEND
Inability to identify waste and inefficiency at scale
Cloud services are fundamentally different from on-premises systems and require much more efficient utilization in order to achieve cost savings.
Turning off idle clusters is a great place to start, but most enterprise data pipelines and applications require manual tuning. This is a resource-intensive, manual process that requires more time and resources than most organizations have to spare.
Cloud resource optimization is so difficult that enterprise executives estimate 32% of cloud spending is wasted each year (Flexera). Modern data stacks are resource-intensive and figuring out exactly which resources the data app/pipeline actually needs to run is complicated.
UNOPTIMIZED CLOUD DATA SPEND
Inability to identify waste and inefficiency at scale
Cloud services are fundamentally different from on-premises systems and require much more efficient utilization in order to achieve cost savings.
82%
of IT professionals have incurred unnecessary cloud costs (Virtana)
66%
of organizations do not see substantial ROI from cloud transformation programs (KPMG)
Turning off idle clusters is a great place to start, but most enterprise data pipelines and applications require manual tuning. This is a resource-intensive, manual process that requires more time and resources than most organizations have to spare.
Cloud resource optimization is so difficult that enterprise executives estimate 32% of cloud spending is wasted each year (Flexera). Modern data stacks are resource-intensive and figuring out exactly which resources the data app/pipeline actually needs to run is complicated.
LACK OF GOVERNANCE
Who’s in charge?
The same flexibility that enables faster business results also leads to cloud data cost overruns. When the cloud bill arrives, it’s too late to take corrective action.
Defining guardrails for data workloads while maintaining trust between teams is difficult. You don’t want finance teams to arbitrarily set limits that impact performance or quality of critical data pipelines.
Without well-defined limits in place, it’s far too easy for inexperienced or junior data team members to have a job go off the rails, leading to massive cost overruns.
LACK OF GOVERNANCE
Who’s in charge?
The same flexibility that enables faster business results also leads to cloud data cost overruns. When the cloud bill arrives, it’s too late to take corrective action.
81%
of technical executives say “managing cloud spend” among biggest challenges of using cloud computing technology (Statista)
60%
of infrastructure and operations (I&O) leaders will encounter public cloud cost overruns that negatively impact their on-premises budgets (Gartner)
Defining guardrails for data workloads while maintaining trust between teams is difficult. You don’t want finance teams to arbitrarily set limits that impact performance or quality of critical data pipelines.
Without well-defined limits in place, it’s far too easy for inexperienced or junior data team members to have a job go off the rails, leading to massive cost overruns.
FinOps integrates financial accountability into cloud operations. DataOps brings lean and agile approaches to data management. DataFinOps combines the best of DataOps and FinOps to help you maximize the return on your modern data stack investment.
Unravel Data is a data observability platform that simplifies and accelerates each phase of the DataFinOps lifecycle–from observability to optimization and governance. Innovative companies such as Mastercard, Equifax, and DBS Bank use Unravel for cloud cost management and FinOps. Unravel provides data teams a unified view of data pipeline and application performance, cost, and quality.
Unravel is a member of the FinOps Foundation
The practice of FinOps, with its collaborative and iterative approach to observability, optimization, and governance can help data teams increase the return on their modern data stack investments by proactively spotting opportunities for efficiency, adding alerts, and automating guardrails.
Unravel is a member of the FinOps Foundation
The practice of FinOps, with its collaborative and iterative approach to observability, optimization, and governance can help data teams increase the return on their modern data stack investments by proactively spotting opportunities for efficiency, adding alerts, and automating guardrails.
JOIN US
See how to use Unravel for DataFinOps
Jumpstart your DataFinOps journey by attending an upcoming event with Unravel. Creating a culture of communication between executives, finance, business, and engineering teams is a key FinOps capability. Begin by collaborating on a single source of truth. Improve performance, cost, and quality with DataFinOps using Unravel.
JOIN US
See how to use Unravel for DataFinOps
Jumpstart your DataFinOps journey by attending an upcoming event with Unravel. Creating a culture of communication between executives, finance, business, and engineering teams is a key FinOps capability. Begin by collaborating on a single source of truth. Improve performance, cost, and quality with DataFinOps using Unravel.