As DataOps moves along the maturity curve, many organizations are deciphering how to best balance the success of running critical jobs with optimized time and cost governance.
Watch the fireside chat from Data Teams Summit where Mark Sear, Head of Data Platform Optimization for Maersk, shares how his team is driving towards enabling strong engineering practices, design tenets, and culture at one of the largest shipping and logistics companies in the world. Transcript below.
Transcript
Kunal Agarwal:
Very excited to have a fireside chat here with Mark Sear. Mark, you’re the director of data integration, AI, machine learning, and analytics at Maersk. And Maersk is one of the largest shipping line and logistics companies in the world. Based out of Copenhagen, but with subsidiaries and offices across 130 countries with about 83,000 employees worldwide. We know that we always think about logistics and shipping as something just working harmoniously, transparently in the background, but in the recent past, given all of the supply chain pressures that have happened with the pandemic and beyond, and even that ship getting stuck in the Suez Canal, I think a lot more people are paying attention to this industry as well. So I was super excited to have you here, Mark, to hear more about yourself, you as the leader of data teams, and about what Maersk is doing with data analytics. Thank you so much for joining us.
Mark Sear:
It’s an absolute pleasure. You’ve just illustrated the perils of Wikipedia. Maersk is not just one of the largest shipping companies in the world, but we’re also actually one of the largest logistics companies in the world. We have our own airline. We’ve got hundreds of warehouses globally. We’re expanding massively, so we are there and of course we are a leader in decarbonation. We’ve got a pledge to be carbon-neutral way before just about anybody else. So it’s a fantastic company to work at. Often I say to my kids, we don’t just deliver stuff, we are doing something to help the planet. It’s a bigger mission than just delivering things, so it’s a pleasure to be here.
Kunal Agarwal:
That’s great. Mark, before we get into Maersk, we’d love to learn about you. So you have an amazing background and accumulation of all of these different experiences. Would you help the audience to understand some of your interests and how you got to be in the role that you currently are at? And what does your role comprise inside of Maersk?
Mark Sear:
Wow. It’s a long story. I’m an old guy, so I’m just couple of years over 60 now, which you could say you don’t look it, but don’t worry about it.
Kunal Agarwal:
You don’t look it at all, only 40.
Mark Sear:
I’m a generation that didn’t, not many of us went to university, so let me start there. So I left school at 18, did a bit of time in the basic military before going to what you would call, I suppose fundamentally, a crypto analyst school. They would detect how smart you were, whether you had a particular thing for patents, and they sent me there. Did that, and then since then I’ve worked in banking, in trading in particular. I ran a big trading group for a major bank, which was great fun, so we were using data all the time to look for both, not just arbitrage, but other things. Fundamentally, my life has been about data.
Kunal Agarwal:
Right.
Mark Sear:
Even as a kid, my dad had a very small business and because he didn’t know anything about computers, I would do the computing for him and work out the miles per gallon that his trucks were getting and what the trade-in was.
Kunal Agarwal:
Sure.
Mark Sear:
And things like that. So data’s been part of my life and I love everything about data and what it can do for people, companies, everything. Yeah, that’s it. Data.
Kunal Agarwal:
That’s great, Mark. Obviously this is a conference spot, a data team, so it’s great to hear from the data guy who’s been doing it for a really long time. So, Mark, to begin, Maersk, as you said, is one of the largest shipping and logistics companies in the world. How has data transformed your company?
Mark Sear:
One thing, this is a great question. How has it transformed and how will it transform?
Kunal Agarwal:
Yes.
Mark Sear:
I think that for the first time in the last couple of years, and I’ve been very lucky, I’ve only been with the company three years, but shortly after I joined, we had a new tech leader, a gentleman called Navneet Kapoor. The guy is a visionary. If you imagine shipping was seen for many years, there’s a bit of a backwater really. You move containers from one country to another country on ships, that was it. Navneet has changed the game for us all and made people realize that data is pervasive in logistics. It’s literally everywhere. If you think about our biggest ship, ship class, for example, it’s called an E-Class. That can take over 18,000 shipping containers on one journey from China to Europe, 18,000.
Kunal Agarwal:
Oh wow.
Mark Sear:
Think about that. So that’s absolutely huge. Now, to put that into context, in one journey, one of those ships will move more goods than was moved in the entire 19th century between continents, one journey. And we got six of them and they’re going backwards and forwards all the time. So the data has risen exponentially and what you can do with it, we are now just starting to get to grips with it, that’s what so exciting. Consider, we have companies that do want to know how much carbon is being produced as part of their products. We have things like that. We just have an incredibly diverse set of products.
To give you an example, I worked on a project about 18 months ago where we worked out, working in tandem with a couple of nature organizations, that if a ship hits a whale at 12 knots and above, that whale will largely die. If you hit it below 12 knots, it will live. It’s a bit like hitting an adult at 30 miles an hour versus 20. The company puts some money in so we could use the data for where the whales were to slow the ships down. So this is an example of where this company doesn’t just think about what can we do to make money. This is a company that thinks about how can we use data to better the company, make us more profitable, and at the same time, put back into the planet that gave us the ability to have this business.
Kunal Agarwal:
Let’s not forget that we’re human, most importantly.
Mark Sear:
Yeah, it’s super exciting, right? You can make all the money in the world. If you trash the planet, there’s not a lot left to enjoy as part of it. And I love that about this company.
Kunal Agarwal:
Absolutely. And I’m guessing with the pandemic and post-pandemic, and all of the other data sets that you guys are gathering anyways from sensors or from the shipping lines or from all the efficiencies, with all the proliferation of all this data inside your organization, what challenges has your team faced or does the Maersk data team face?
Mark Sear:
Well, my team is in the enterprise architecture team. We therefore deal with all the other teams that are dealing with data, and I think we got the same challenges as everybody. We’ve got the data quality right? Do we know where that data comes from? Are we processing it efficiently? Do we have the right ideas to work on the right insights to get value out of that data? I think they’re common industry things, and as with everything, it’s a learning process. So one man’s high-quality data is another woman’s low-quality data.
And depending on who you are and what you want to do with that data, people have to understand how that quality affects other people downstream. And of course, because you’re quite right, we did have a pandemic, and in the pandemic shipping rates went a little bit nuts and they’re normalizing now. But, of course, if you think about introducing predictive algorithms where the price is going vertically and the algorithm may not know that there’s a pandemic on, it just sees price. So I think what we find is challenging, same as everybody else, is how do you put that human edge around data? Very challenging. How do you build really high-performing teams? How do you get teams to truly work together and develop that esprit de corps? So there are a lot of human problems that go alongside the data problems.
Kunal Agarwal:
Yeah. Mark, give us a sense of your size. In terms of teams, applications, whatever would help us understand what you guys were, where you guys are, and where you guys headed.
Mark Sear:
Three years ago when I joined there were 1,900 people in tech; we’ve now got nearly 6,000. We had a huge amount of outsourcing; now we’re insourcing, we’re moving to an open source first event-based company. We’ve been very inquisitive. We’ve bought some logistics companies, so we’ve gone on the end-to-end journey now with the logistics integrator of choice globally. We’ve got our own airline. So you have to think about a lot of things that play together.
My team is a relatively tiny team. We’ve got about 12, but we liaise with, for example, our global data and analytics team that has got 600 people in it. We then organized into platforms, which are vertically problem solving, but fully horizontally integrated passing events between them. And each one of those has their own data team in it as well. So overall, I would guess we’ve got 3,000 people working directly with data in IT and then of course many thousands more.
Kunal Agarwal:
Wow.
Mark Sear:
Out in the organization. So it’s big organizations. Super exciting. Should say, now I’m going to get a quick commercial in. If you are watching this and you are a top data talent, please do hit me up with your resume.
Kunal Agarwal:
There’s a couple of thousand people watching this live, so you’ll definitely.
Mark Sear:
Hey, there you go, man. So listen, as long as they’re quality, I don’t care.
Kunal Agarwal:
From Mark, he’s a great boss as well. So when you think about the maturity curve of data operations, where do you think Maersk is at and what stands in your way to be fully matured?
Mark Sear:
Okay, so let’s analyze that. I think the biggest problem in any maturity curve is not defining the curve. It’s not producing a pyramid to say we are here and a dial to say, well, you rank as a one, you want to rank as a five.
Kunal Agarwal:
Sure.
Mark Sear:
The biggest problem to me is the people that actually formulate that curve. Now everyone’s got staff turnover and everyone or the majority of people know that they’re part of a team. But the question is how do you get that team to work with other teams and how do you disseminate that knowledge and get that group think of what is best practice for DataOps? What is best practice for dealing with these problems?
Kunal Agarwal:
It’s almost a spectrum on the talent side, isn’t it?
Mark Sear:
It’s a spectrum on the talent side, there’s a high turnover because certainly in the last 12 to 18 months, salaries have been going crazy, so you’ve had crazy turnover rates in some areas, not so much in other areas. So the human side of this is one part of the problem, and it’s not just the human side to how do you keep them engaged, it’s how do you share that knowledge and how do you get that exponential learning organization going?
And perhaps when we get into how we’ve arrived at tools like Unravel, I’ll explain to you what my theory is on that, but it’s almost a swarm learning that you need here, an ants style learning of how to solve problems. And that’s the hardest thing, is getting everybody in that boat swimming in the same direction before you can apply best practices because everybody says this is best practice. Sure, but if it was as simple as looking at a Gartner or whoever thing and saying, “Oh, there are the five lines we need to do,” everybody would do it. There’d be no need for anybody to innovate because we could do it; human beings aren’t very good at following rules, right?
Kunal Agarwal:
Yeah. So what kind of shifts and changes did you have to make in your big data operations and tools that you had to put into place for getting that maturity to where you expected it to be?
Mark Sear:
I think the first thing we’ve got to do, we’ve got to get people thinking slightly shorter timeframe. So everybody talks about Agile, Agile, Agile.
Kunal Agarwal:
Right.
Mark Sear:
Agile means different things to different people. We had some people who thought that Agile was, “Well, you’re going to get a fresh data set at the end of the day, so what the heck are you complaining about? When I started 15 years ago, you got it weekly.” That’s not agile. Equally, you’ve got people who say, I need real-time data. Well, do you really need real-time data if you’re actually dealing with an expense account? You probably don’t.
Kunal Agarwal:
Right.
Mark Sear:
Okay, so the first thing we’ve got to do is level set expectations of our users and then we’ve got to dovetail what we can deliver into those. You’ve got to be business focused, you’ve got to bring value. And that’s a journey. It’s a journey for the business users.
Kunal Agarwal:
Sure.
Mark Sear:
It’s a journey for our users. It’s about learning. So that’s what we’re doing. It’s taking time. Yeah, it’s taking time, but it’s like a snowball. It is rolling and it is getting bigger and it’s getting better, getting faster.
Kunal Agarwal:
And then when you think about the tools, Mark, are there any that you have to put into place to accelerate this?
Mark Sear:
I mean, we’ve probably got one of everything to start and now we’re shrinking. If I take . . . am I allowed to talk about Unravel?
Kunal Agarwal:
Sure.
Mark Sear:
So I’ll talk about–
Kunal Agarwal:
As much as you would.
Mark Sear:
–Unravel for a few seconds. So if you think about what we’ve got, let’s say we’ve got 3,000 people, primarily relatively young, inexperienced people churning out Spark code, let’s say Spark Databricks code, and they all sit writing it. And of course if you are in a normal environment, you can ask the person next to you, how would you do this? You ask the person over there, how would you do this? We’ve had 3,000 engineers working from home for two years, even now, they don’t want to come into the office per se, because it’s inconvenient, number one, because you might be journeying in an hour in and an hour home, and also it’s not actually, truly is productive. So the question is how do you harvest that group knowledge and how do people learn?
So for us, we put Unravel in to look at and analyze every single line of code we write and come up with those micro suggestions and indeed macro suggestions that you would miss. And believe me, we’ve been through everything like code walkthroughs, code dives, all those things. They’re all standard practice. If you’ve got 2,000 people and they write, let’s say, 10 lines of code a day each, 20,000 lines of code, you are never going to walk through all of that code. You are never going to be able to level set expectations. And this is key to me, be able to go back to an individual data engineer and say, “Hey, dude, listen, about these couple of lines of code. Did you realize if you did it like this, you could be 10 times as efficient?” And it’s about giving that feedback in a way that allows them to learn themselves.
And that’s what I love about Unravel: you can get the feedback, but it’s not like when you get that feedback, nobody says, “Come into my office, let’s have a chat about these lines of code.” You go into your private workspace, it gives you the suggestions, you deal with the suggestions, you learn, you move on, you don’t make the mistakes again. And they may not even be mistakes. They might just be things you didn’t know about.
Kunal Agarwal:
Right.
Mark Sear:
And so because Unravel takes data from lots of other organizations as well, as I see it, we’re in effect, harvesting the benefits of hundreds of thousands of coders globally, of data engineers globally. And we are gaining the insights that we couldn’t possibly gain by being even the best self-analysis on the planet, you couldn’t do it without that. And that to me is the advantage of it. It’s like that swarm mentality. If you’ve ever, anybody watching this, had a look at swarm AI, which is to predict, you can use it to predict events. It’s like if you take a soccer game, and I’ve worked in gambling, if you take a soccer match and you take a hundred people, I’ll call it soccer, even though the real name for is football, you Americans.
Kunal Agarwal:
It’s football, I agree too.
Mark Sear:
It’s football, so we’re going to call it football, association football to give you it’s full name. If you ask a hundred football fans to predict a score, you’ll get a curve, and you’ll generally, from that predictor, get a good result. Way more accurate than asking 10 so-called experts, such as with code. And that’s what we’re finding with Unravel is that sometimes it’s the little nuances that just pop up that are giving us more benefits.
Kunal Agarwal:
Right.
Mark Sear:
So it’s pivotal to how we are going to get benefits out over the longer term of what we’re doing.
Kunal Agarwal:
That’s great. And we always see a spectrum of skills inside an organization. So our mission is trying to level the playing field so anybody, even a business user, can log in without knowing the internals of all of these complex data technologies. So it’s great to hear the way Maersk is actually using it. We spoke a little bit about making these changes. We’d love to double click on some of these hurdles, right? Because you said it was a journey to get to people to this mature or fast-moving data operations, if you may, or more agile data operations if you may. If we can double click for a second, what has been the biggest hurdle? Is it the mindset? Is it managing the feedback loop? Is it changing the practices? Is it getting new types of people on board? What has been the biggest hurdle?
Mark Sear:
Tick all of the above.
Kunal Agarwal:
Okay.
Mark Sear:
But I think–
Kunal Agarwal:
Pick for option E.
Mark Sear:
Yeah, so let me give you an example. There are several I’ve had with people that have said to me, “I’ve been doing this 25 years. There’s nothing, I’ve been doing it 25 years.” That presupposes that 25 years of knowledge and experience is better than 10 minutes with a tool that’s got 100,000 years of learning.
Kunal Agarwal:
Right.
Mark Sear:
Over a 12-month period. So that I classify that as the ego problem. Sometimes people need their ego brushing, sometimes they need their ego crushing. It’s about looking the person in the eye, working out what’s the best strategy of dealing with them and saying to them, “Look, get on board.” This isn’t about saying you are garbage or anything else. This is about saying to you, learn and keep mentoring other people as you learn.
Kunal Agarwal:
Yeah.
Mark Sear:
I remember another person said to me, “Oh my god, I’ve seen what this tool can do. Is AI going to take my job?” And I said to them, no, AI isn’t going to take your job, but if you’re not careful, somebody, a human being that is using AI will take it, and that doesn’t apply to me. That applies just in general to the world. You cannot be a Luddite, you cannot fight progress. And as we’ve seen with Chat GPT and things like that recently, the power of the mass of having hundreds and thousands and millions of nodes analyzing stuff is precisely what will bring that. For example, my son who’s 23, smart kid, well, so he tells me. Smart kid, good uni, good university, blah blah blah. He said to me, “Oh Tesla, they make amazing cars.” And I said to him, Tesla isn’t even a car company. Tesla is a data company that happens to build a fairly average electric car.
Kunal Agarwal:
Absolutely.
Mark Sear:
That’s it. It’s all about data. And I keep saying to my data engineers, to be the best version of you at work and even outside work, keep picking up data about everything, about your life, about your girlfriend, the way she feels. About your boyfriend, the way he feels. About your wife, your mother. Everything is data. And that’s the mindset. And the biggest thing for me, the biggest issue has been getting everybody to think and recognize how vital data is in their life, and to be open to change. And we all know throughout go through this cycle of humanity, a lack of openness to change is what’s held humanity back. I seek to break that as well.
Kunal Agarwal:
I love that Mark. Switching gears, we spoke a little bit about developer productivity. We spoke about agility and data operations. Maersk obviously runs, like you were explaining, a lot of their data operations on the cloud. And as we see a lot of organizations when they start to get bigger and bigger and bigger in use cases on the cloud, cost becomes a front and center or a first-class citizen conversation to have. Shed some light on that for us. What is that maturity inside of Maersk, or how do you think about managing costs and budgets and forecast on the cloud, and what’s the consequence of not doing that correctly?
Mark Sear:
Well, there are some things that I can’t discuss because they’re obviously internal, but I think, let’s say I speak to a lot of people in a lot of companies, and there seem to be some themes that run everywhere, which is there’s a rush towards certain technologies, and people, they test it out on something tiny and say, “Hey, isn’t this amazing? Look how productive I am.” Then they get into production and somebody else says, “That’s really amazing. You were very productive. But have you seen what comes out the other end? It’s a cost, a bazillion dollars an hour to run it.” Then you’ve got this, I think they called it the Steve Jobs reality distortion field, where both sets of people go into this weird thing of, “Well, I’m producing value because I’m generating code and isn’t it amazing?” And the other side is saying, “Yeah, but physically the company’s going to spend all its money on the cloud. We won’t be able to do any other business.”
Kunal Agarwal:
Yeah.
Mark Sear:
So we are now getting to a stage where we have some really nice cost control mechanisms coming in. For me, it’s all in the audit. And crucial to this is do it upfront. Do it in your dev environment. Don’t go into production, get a giant bill and then say, how do I cut that bill? Which is again, where we’ve put Unravel now, right in the front of our development environment. So nothing even goes into production unless we know it’s going to work at the right cost price. Because otherwise, you’ve just invented the world’s best mechanism for closing the stable door after the cost horse has bolted, right?
Kunal Agarwal:
Right.
Mark Sear:
And then that’s always a pain because post-giant bill examinations are really paying, it’s a bit like medicine. I don’t know if you know, but in China, you only pay a doctor when you are well. As soon as you are sick, you stop paying bills and they have to take care of you. So that to me is how we need to look at cost.
Kunal Agarwal:
I love that. Love that analogy.
Mark Sear:
Do it upfront. Keep people well, don’t ever end up with a cost problem. So that’s again, part of the mindset. Get your data early, deal with it quickly. And that’s the level of maturity we are getting to now. It’s taking time to get there. We’re not the only people, I know it’s everywhere. But I would say to anybody, I was going to say lucky enough to be watching this, but that’s a little bit cocky, isn’t it? Anybody watching this? Whatever you do, get in there early, get your best practice in as early as possible. Go live with fully costed jobs. Don’t go live, work out what the job cost is and then go, how the hell do I cut it?
Kunal Agarwal:
Yeah.
Mark Sear:
Go live with fully costed jobs and work out well, if it costs this much in dev test, what’s it going to cost in prod? Then check it as soon as it goes live and say, yeah, okay, the delta’s right, game on. That’s it.
Kunal Agarwal:
So measure twice, cut once, and then you’re almost shifting left. So you’re leaving it for the data engineers to go and figure this out. So there’s a practice that’s emerging called FinOps, which is really a lot of these different groups of teams getting together to exactly solve this problem of understand what the cost is, optimize what the cost is, and then govern what the cost is as well. So who within your team does what I’m sure the audience would love to hear that a little bit.
Mark Sear:
Pretty much everybody will do everything, every individual data engineer, man, woman, child, whatever will be, but we’re not using child labor incidentally, that was.
Kunal Agarwal:
Yeah, let’s clarify that one for the audience.
Mark Sear:
That’s a joke. Edit that out. Every person will take it on themselves to do that because ultimately, I have a wider belief that every human being wants to do the right thing, given everything else being equal, they want to do the right thing. So I will say to the people that I speak to as data engineers, as new data engineers, I will say to them, we will empower you to create the best systems in the world. Only you can empower yourself to make them the most efficient systems in the world.
Kunal Agarwal:
Interesting.
Mark Sear:
And by giving it to them and saying, “This is a matter of personal pride, guys,” at the end of the day, am I going to look at every line of your code and say, “You wouldn’t have got away with that in my day.” Of course not. When I started in it, this is how depressingly sad it is. We had 16K of main memory on the main computer for a bank in an IBM mainframe, and you had to write out a form if you wanted 1K of disk. So I was in a similar program in those days. Now I’ve got a phone with God knows how much RAM on it.
Kunal Agarwal:
Right, and anybody can spin up a cloud environment.
Mark Sear:
Absolutely. I can push a button, spin up whatever I want.
Kunal Agarwal:
Right.
Mark Sear:
But I think the way to deal with this problem is to, again, push it left. Don’t have somebody charging in from finance waiving a giant bill saying, “Guys, you are costing a fortune.” Say to people, let’s just keep that finance dude or lady out of the picture. Take it on yourself, yourself. Show a bit of pride, develop this esprit de corps, and let’s do it together.
Kunal Agarwal:
Love it. Mark, last question. This is a fun one and I know you’re definitely going to have some fun answer over here. So what are your predictions for this data industry for this year and beyond? What are we going to see?
Mark Sear:
Wow, what do I think? Basically–
Kunal Agarwal:
Since you’ve got such a pulse on the overall industry and market.
Mark Sear:
So to me, the data industry, obviously it’ll continue to grow. I don’t believe that technology in many levels, I’ll give you over a couple of years, technology in many levels, we’re actually a fashion industry. If the fashion is to outsource, everybody outsource. So the fashion is to in-source, everybody does. Women’s skirts go up, fashion changes, they come down. Guys wear flared trousers, guy wears wear narrow trousers and nobody wants to be out of fashion. What I think’s going to happen is data is going to continue to scale, quantum computing will take off within a few years. What’s going to happen is your CEO is going to say, “Why have I got my data in the cloud and in really expensive data centers when someone has just said that I can put the whole of our organization on this and keep it in the top drawer of my desk?”
And you will have petabyte, zettabyte scale in something that can fit in a shoebox. And at that point it’ll change everything. I will probably either be dead, or at least hopefully retired and doing something by then. But I think it is for those people that are new to this industry, this is an industry that’s going to go forever. I personally hope I get to have an implant in my head at some point from Elon. I will be going for, I’m only going to go for version two. I’m not going for version one and hopefully–
Kunal Agarwal:
Yeah, you never want to go for V1.
Mark Sear:
Exactly, absolutely right. But, guys, ladies, everybody watching this, you are in the most exciting part, not just of technology, of humanity itself. I really believe that, of humanity itself, you can make a difference that very few people on the planet get to make.
Kunal Agarwal:
And on that note, I think the big theme that we have going on this series, we strongly feel that data teams are running the world and will continue to run the world. Mark, thank you so much for sharing this, exciting insights, and it’s always fun having you. Thanks you for making the time.
Mark Sear:
Complete pleasure.