Table of Contents
At the recent DataOps Unleashed 2022 virtual conference, we surveyed several hundred leading DataOps professionals across multiple industries in North America and Europe on a range of issues, including:
- Current adoption of DataOps approaches
- Top challenges in operating/managing their data stack
- How long they estimate cloud migration will take
- Where they are prioritizing automation
- On which tasks they spend their time
Download our infographic here.
Highlights and takeaways
DataOps as a practice is hitting an inflection point
DataOps has gained momentum over the past 12 months. While a DataOps approach is still in the early innings for most companies—almost 4 out of 5 of respondents said they are “having active discussions” or “progressing”—this is an 80% jump from last year.
Visibility into data pipelines remains the top challenge
For the second consecutive year, “lack of visibility across the environment” was the #1 challenge. Interestingly, “lack of proactive alerts” leapfrogged “expensive runaway jobs/pipelines” as the second biggest challenge, followed by “lack of experts” and “no visibility into cost/usage.”
Projected cloud migration schedules are longer
When we asked respondents to estimate how long their cloud migration will take, only 28% said just one year—with the vast majority saying 2 years or more. This is a 150% jump from last year, reflecting the growing realization of the complexity in moving from on-prem to cloud.
Automation continues to be a key driver
Over 75% of survey respondents said that being able to “automatically test and verify before moving jobs/pipelines to production” was their top automation priority. Right behind was automating the troubleshooting of pipeline issues (65%), followed by a three-way tie (about 33% each) among automatically troubleshooting platform issues, troubleshooting jobs, and automatically reducing usage costs.
Teams spend more time building pipelines than maintaining/deploying them
Like last year, we saw that data teams spend more time building pipelines (43% in 2022, up from 39% in 2021) than maintaining/troubleshooting them (30% in 2022 vs. 34% in 2021) or deploying pipelines (holding steady at 27%).
Interested in more DataOps trends?
Check out the summary recap of the DataOps Unleashed 2022 keynote session, Three Venture Capitalists Weigh In on the State of DataOps 2022, or watch the full roundtable discussion on demand.