Episode 15

Zach Stitham - Driving FinOps Culture Through KPIs

Published on: 22nd November, 2022

Episode 15 Zach Stitham - Driving FinOps Culture Through KPIs

Zach Stitham, VP - Cloud Business Management Office at Fidelity, talks with Ashley Hromatko on FinOps KPIs. How do you chose them? How do you mature them? How do you drive the right behaviors and culture by leveraging them? Stay on target for your goals by listening to this episode!

Transcript
Zach:

Hello, I'm Zach Stithem and this is the FinOpsPod

Stacy:

All right.

Stacy:

It's been a while.

Stacy:

Let's see if I can remember how to do this.

Stacy:

This is Stacy Case,

Joe:

and I'm Joe Daly.

Joe:

Nope, nope, nope.

Stacy:

Joe.

Stacy:

That's not how you do it.

Joe:

Nope.

Joe:

Let's start over.

Joe:

I don't, I don't like that feeling.

Stacy:

And that did not make me feel good either.

Stacy:

All right.

Stacy:

Trying to give, I'll bring a little more energy myself and you.

Stacy:

You dial it down a little bit.

Joe:

Mm-hmm.

Stacy:

Hi, this is Stacy Case

Joe:

And I'm Joseph Daly.

Stacy:

and this is FinOpsPod

Joe:

FinOpsPod!!

Stacy:

WOO

Joe:

WOO!

Stacy:

I like it.

Joe:

Yeah.

Stacy:

All right, Joe.

Joe:

yes.

Stacy:

it's, I feel like it's been a minute.

Stacy:

but that's because I think it's just the end of the year and I have no concept

Stacy:

of time at this point, so maybe it hasn't been a minute, but, I'm excited.

Stacy:

I'm excited to be here and talking

Joe:

Yes.

Stacy:

having another conversation on the pod.

Joe:

Mm-hmm.

Stacy:

do we have this week?

Stacy:

I know I act, by the way.

Stacy:

I just said that, like, I don't know.

Stacy:

And also if you're listening to this, like you didn't just read

Stacy:

on your podcast who it is, but let's pretend in Imaginary World.

Joe:

Well, that's what they call in the industry a setup question.

Stacy:

Oh

Joe:

yeah, I think, I don't know.

Joe:

I haven't done enough research on the industry, but I imagine that is what

Joe:

they would refer to as a setup question.

Joe:

And it's a good one because Stacy, This episode of the FinOpsPod has

Joe:

Zachary Stithem from the award-winning Fidelity team talking about KPIs with,

Joe:

the Foundation's own Ashley Hromatko.

Stacy:

Wow.

Stacy:

Wait.

Stacy:

They're talking about KPIs

Joe:

key performance indicators

Stacy:

Thank you.

Stacy:

Thank you.

Stacy:

That's really cool because one of the things I think that we get a lot of

Stacy:

questions on within the Foundation, and even last night, not that the

Stacy:

listeners know this, but we did just have, a community meeting call with

Stacy:

APAC region, which was great, but we constantly are being asked and

Stacy:

community members wanna know about KPIs, what they should be using?

Stacy:

So I'm pretty sure that Zach is about ready to make a whole bunch of

Stacy:

people happy with this conversation.

Joe:

Absolutely.

Joe:

Honestly, when we were started recording it, I was like, oh, good.

Joe:

A podcast on KPIs.

Joe:

, but Zach and Ashley really investigate.

Joe:

Ashley did a tremendous job with this interview, just really investigating

Joe:

how did Fidelity come up with a kpi and what was the maturation process?

Joe:

Cause it's not like you just say, we're gonna measure this, and

Joe:

all of a sudden we're awesome at it and everyone accepts it.

Joe:

It was a really great job of How did they mature it?

Joe:

How did they introduce it?

Joe:

What sort of cultural results came from the kpi?

Joe:

what sort of unintended consequences came there?

Stacy:

Well, and I think it's great too because I think sometimes

Stacy:

when people ask what KPIs.

Stacy:

It's not the right question, but it's what KPIs and then all the other things

Stacy:

that you just talked about with Zach.

Stacy:

So diving into that a little bit more, because again, we've

Stacy:

talked about this all the time.

Stacy:

FinOps is a culture.

Stacy:

It's not just this one thing that we have to do.

Stacy:

Here's the kpi, check it off.

Stacy:

Now we've done FinOps.

Stacy:

There's a lot more depth to it.

Stacy:

and Zach and Ashley do a great job.

Stacy:

now we've heard from Zach before too, right?

Stacy:

Zach and our own Mr.

Stacy:

, Noel Crowley, who is also at Fidelity and I think works for Zach, both

Stacy:

of them presented at FinOps X in 2022 this last year, didn't they?

Joe:

Absolutely, and I'm gonna include the link of their presentation also on KPIs in

Joe:

the show notes for anyone who missed it.

Joe:

it was a fantastic presentation that took place in ballroom B.

Stacy:

eh, eh.

Joe:

Yeah, it was awesome.

Joe:

the Ballroom B Wolf Pack, had that presentation and yeah, Fidelity had

Joe:

a large presence at FinOps X in 2022.

Stacy:

basically Zach is coming back for his encore performance here because

Stacy:

it was so highly regarded there.

Stacy:

So it's funny, we're talking about X and Zach and Noel presenting last year at X

Stacy:

because we did just announce FinOps X for 2023, which is going to be in San Diego.

Stacy:

where can folks go to learn more about that?

Joe:

if you are interested in FinOps X 2023 in San Diego, go to x.finops.Org.

Joe:

And go check it out.

Joe:

It is going to be amazing I think it was so special.

Joe:

I was not anticipating the amount of energy, that was

Joe:

at the conference last year.

Stacy:

It's so exciting.

Stacy:

First of all, I'd be remiss if we didn't add the dates of June

Stacy:

27th through 30th in San Diego.

Stacy:

But, one of the coolest things, and I know you and I have talked

Stacy:

about X already this year, and we've had a whole podcast on that.

Stacy:

But , you're with a community of people who do what you do.

Stacy:

And one of my favorite quotes out of X this past year was, I

Stacy:

don't have to explain to anybody what my job is or what I do.

Stacy:

and we had just over 400 people last year and we're targeting a thousand this year.

Stacy:

That means a thousand people that do what you do.

Stacy:

so super exciting.

Joe:

thousand people who understand you,

Stacy:

And let's not forget, we'll end X with a magnificent party on an

Stacy:

aircraft carrier, which apparently we have commandeered, rented out,

Stacy:

acquired, I don't know, the whole entire aircraft carrier for a Sunset party.

Stacy:

So Sunset in San Diego.

Stacy:

Folks like, that's insane.

Stacy:

It's so beautiful, so beautiful

Joe:

an aircraft

Stacy:

on an aircraft carrier.

Joe:

running around with my arm spread out like an airplane

Stacy:

Joe Tell me, are you gonna dress up in a flight suit for that?

Stacy:

I mean, you know, San Diego is the land of Top gun movies.

Stacy:

I would love to see you in a flight suit,

Joe:

I'm gonna have my aviator sunglasses and I'm gonna just be flashing smiles

Joe:

and you know, playing volleyball.

Joe:

I'm gonna be doing all that.

Stacy:

I don't know what character you'll be, but you'll be a character.

Joe:

Probably Goose.

Stacy:

goose.

Stacy:

He dies .Anyway, FinOps X next year, 2023, San Diego, June 27th and 30th.

Stacy:

Hopefully we'll see everybody there.

Stacy:

Hopefully we can get Zach Noel or some other folks from Fidelity

Stacy:

to do some amazing conversations.

Stacy:

again, if not, hopefully they'll just be there so everybody that listens to

Stacy:

this podcast can come and say hi and ask them all of their KPI questions in.

Stacy:

Okay, well, , so let's get to it.

Stacy:

Right?

Stacy:

Let's listen to Zach and Ashley.

Joe:

Let's get to Zach and Ashley.

Stacy:

Don't we usually say enjoy FinOpsPod or something.

Joe:

Enjoy this episode of

Stacy:

Finn Ops Pod There we go.

Stacy:

All done.

Ashley:

Hey Zach, it's good to see you.

Ashley:

The first time I remember meeting you was at Cloudycon.

Ashley:

It was like what?

Ashley:

It was back in 2019.

Ashley:

And I was so impressed because you guys had won an award at that point.

Ashley:

I think it was called the Leadership and Cloud Operation award.

Ashley:

I wasn't that impressed by the award.

Ashley:

but I was impressed by the size of your team and everything

Ashley:

that you all were doing.

Ashley:

it seemed like you had a really well-oiled team and you were really

Ashley:

making this big shift in culture there.

Ashley:

I got this quote that you had said.

Ashley:

You said, cloud cost management isn't just a strategy.

Ashley:

It has to be enabling a culture change.

Ashley:

And I love that so much.

Ashley:

And you and I have talked before about your hub and spoke model at

Ashley:

Fidelity, and I said you have to have a really good culture to pull that off.

Ashley:

You've been at Fidelity for 25 plus years.

Ashley:

Tell us a little about that journey and what culture means to you there.

Zach:

Yeah, I mean, Fidelity's been a unique place to, I'll call it, grow up

Zach:

in and been fortunate enough to being surrounded by a whole bunch of people,

Zach:

especially with the same passion.

Zach:

And I think it resonates too , with our FinOps practice here at Fidelity.

Zach:

I've known most of the people on our FinOps team prior to joining even though

Zach:

they come from different backgrounds, what's great is that same passion

Zach:

around getting the value out of every dollar that we spend in the cloud,

Zach:

and they're really trying to push that down through their organization.

Zach:

So, you know, they're kind of a, a forcing function, here in our enterprise.

Zach:

And just trying to get that message across to the whole development community.

Ashley:

I love that and you also said when you're at FinOps X, you know,

Ashley:

we will utilize our organization as in your centralized FinOps

Ashley:

organization to change the culture.

Ashley:

How do you feel like you've been doing since 2019?

Zach:

I think it's the hardest part of FinOps.

Zach:

And I think even though that we keep pushing and striving to get

Zach:

there, it feels like times, like it's the never ending battle.

Zach:

I lean back to the, the people that are in our practice that,

Zach:

you know, we use them as the, I'll call it the multiplier effect.

Zach:

And they certainly do a great job, but when you're talking 15,000 plus

Zach:

technologists at Fidelity, reaching out to them all seems like a daunting task.

Zach:

Even though I feel like we're chipping away , and doing a fairly good job,

Zach:

there's still a lot of work to be done.

Ashley:

Do you feel like bringing in KPIs and measuring things has helped

Ashley:

since you are a large enterprise at scale?

Ashley:

Many, many engineers.

Ashley:

Has that been key for you?

Zach:

I think the KPIs have been incredibly helpful.

Zach:

I think it started off to, I'd say, push our team to think about what

Zach:

we could do from the center or from the spoke and hub model first, and

Zach:

then use it to our advantage to try to, I'd say, tailor it more.

Zach:

Get visibility and make an actionable out at the developer.

Zach:

So if you remember the talk that we had done before around our KPIs is,

Zach:

you know, a lot of things that we measured ourselves on in the first

Zach:

year were around purchasing strategy.

Zach:

So it was the how are we doing from an overall discount percentage?

Zach:

How are we doing from a reserved instance or savings plan to coverage perspective?

Zach:

Savings plans actually didn't exist at the time, but reservation coverage.

Zach:

And then how are we doing from like spot usage?

Zach:

And most of those we could execute from the center.

Zach:

And we had one business unit leading the effort from the spot.

Zach:

So again, we had that, that first year of the KPIs.

Zach:

And then I would say we tried to tailor what we use the index metric for,

Zach:

meaning for each of these categories, we try to weight them differently.

Zach:

So I would say take more off the hub and put more into the spoke or put

Zach:

more into the development community to make it more actionable and push

Zach:

them from a visibility perspective.

Zach:

How can they help to, I would say, meet or exceed the kpi.

Zach:

So it's been one way to use those KPIs to help, I would

Zach:

say push the enterprise ahead.

Ashley:

So break this down a little bit.

Ashley:

So you originally, the first couple years you had KPIs that you felt were

Ashley:

things that your centralized team could drive, and then you've started

Ashley:

to develop more KPIs that , really are.

Ashley:

The edge needs to be driving those KPIs and you've started to shift the weight.

Ashley:

So first couple years, very heavy.

Ashley:

We're accountable for these things, we'll track these KPIs.

Ashley:

And over time you've now said the things at the edge are gonna weigh

Ashley:

more, and putting pressure there.

Ashley:

But you kind of did the work yourself first.

Ashley:

Right?

Ashley:

Held yourself accountable first.

Ashley:

So now you've done that.

Ashley:

Can you dive into that a little bit more?

Ashley:

Like what has driven you to have this kind of weighted system.

Zach:

Yeah, we wanted them to be, I would say, achievable.

Zach:

So if you take those first years where we were the ones executing, and even when

Zach:

you think about spot usage needing to come from , the business unit technologist, we

Zach:

had a small waiting on them to go like, I don't know if this is achievable, yes

Zach:

or no, but we're gonna put a target out there to see if we can hit it going.

Zach:

We'd love for 10% of our environment to the utilizing spot.

Zach:

And again, almost felt like a stretch goal.

Zach:

So it's one of those going, We're gonna do the purchasing strategies, we're gonna

Zach:

think about our reservation coverage.

Zach:

I know we can execute on that.

Zach:

We know we can get the discounts if we do those correctly.

Zach:

And the stretch goal piece was 10% of the mark.

Zach:

And then as we started to migrate, thinking in those future years going,

Zach:

well, purchasing strategies, we'll roll that all up into a single, "Are

Zach:

we getting the best discount" metric.

Zach:

And then we'll start to push on the, I would say, on the development community to

Zach:

go, Can you get rid of your waste and can you use the resources more efficiently?

Zach:

And again, those first years we waited the purchasing effect in this really high.

Zach:

And thought of those other areas for, you know, our unutilized

Zach:

assets and our underutilized assets as more of the stretch goal.

Zach:

And as the years go on, they become, Much more than , stretch

Zach:

goals, they become the norm.

Zach:

and then as we weight them more heavily, and I would say take the dependency on

Zach:

the center to execute on a reservation or a savings plan or a specific

Zach:

discount, it truly leaves it back on the development community to make that kpi.

Zach:

So again, as those are visible, And we've got great buy-in here at

Zach:

Fidelity from our CIO community.

Zach:

Each of them having that on their scorecard for performance for the year,

Zach:

it ends up really, I would say, adding some light, to putting that back in

Zach:

on the developer to see what they can to help push in the right direction.

Ashley:

Yeah, I like that too.

Ashley:

And, you have talked about this before too, I think some people come

Ashley:

to, you know us and are asking like, what's that one KPI to measure,

Ashley:

what's that one, you know, holy grail thing that I should be measuring.

Ashley:

But it doesn't feel like there's one thing, it, it's, it's many KPIs and

Ashley:

there's an evolution of those KPIs.

Zach:

Yeah, there's definitely no saving grace or no single one.

Zach:

Even the ones that we've had in the past, you look back to see going.

Zach:

It was good at the time and it was a great, I would say, starting point.

Zach:

But as the evolution takes place, it becomes a, It's a nice to know.

Zach:

It's a nice to have, but I don't think it's necessarily gonna drive us to be more

Zach:

actionable than we could or should be.

Zach:

And I think, you know, we've talked a little bit too about

Zach:

like KPIs as opposed to metrics.

Zach:

It seems like we're trying to dive into 50 million other different metrics that might

Zach:

help push us in a different direction.

Zach:

And the KPIs are awesome cuz they're, I would say truly measuring how we're

Zach:

doing from a performance perspective, on our objectives and our goals, but not

Zach:

necessarily getting down to the what's the next actionable thing that I might need to

Zach:

know or what's that next metric that might key me into going to looks under some

Zach:

other rock for some sort of value or cost savings that I might be able to dig into.

Zach:

So I think we're always looking to figure out what those next set of metrics

Zach:

are that, you know, can we trend them?

Zach:

Can we look at them?

Zach:

How easily can we get to them?

Zach:

and how can we make them more transparent to the development

Zach:

community as well, to see if it triggers any actions on their part.

Ashley:

That's so interesting.

Ashley:

So you just described two things.

Ashley:

You have your KPIs, those are exact, what your leadership is behind, but then you're

Ashley:

also digging into metrics and trends and hopefully you can move those metrics in a

Ashley:

positive way that then therefore impacts your KPI in a positive way as well.

Ashley:

Right.

Zach:

Yeah.

Zach:

exactly.

Ashley:

I like that a lot.

Ashley:

Do you feel like KPIs are hiding anything?

Zach:

I don't think they're hiding anything.

Zach:

I think sometimes I've seen them in the past, not necessarily in our space

Zach:

where they can try to create the wrong behavior, but I think it's more of a

Zach:

back to what we were talking about.

Zach:

They're not the end all be all.

Zach:

They're only, you know, one piece of the whole entire story , again,

Zach:

it's the end result and not the beginning action sometimes.

Zach:

How do you, I would say partner our pair those with additional

Zach:

things that we do here, like the education aspect, their FinOps

Zach:

forums, being part of the community.

Zach:

You know, how do you increase the education and the knowledge around not

Zach:

just developing feature functionality.

Zach:

Making sure that it's efficient use of the resources as well.

Zach:

Cuz you know, in most cases that's not where the developer was focused on.

Zach:

They were more focused on, you know, they've got a product to get to market.

Zach:

They're focused on, meeting business needs as opposed to

Zach:

meeting past efficiency needs where it was someone else's job prior.

Zach:

And now it's just hopefully at some point just part of their day job.

Ashley:

Yeah, I love that too.

Ashley:

The KPI is only as good as the conversation around the kpi, Right.

Ashley:

And the conversation that follows the trigger of the kpi.

Ashley:

That's so good.

Ashley:

So, you FinOps X talk was amazing, so if anybody hasn't

Ashley:

checked it out, go watch it.

Ashley:

It's on the FinOps YouTube channel.

Ashley:

I want you to break down a little bit the process that it took you

Ashley:

to get agreement on these KPIs.

Ashley:

For me, that was something I always struggled on.

Ashley:

I wanted to measure this, but it was getting this many dev teams,

Ashley:

this many engineering teams, this many CTOs to rally behind it.

Ashley:

What was that process like for you to get those first couple KPI.

Zach:

Yeah, I've gotten a lot of props to our enterprise practice team.

Zach:

Cause we end up having a lot of conversations around the where do

Zach:

we wanna really drive and where we really wanna, focus in on from

Zach:

a performance year perspective.

Zach:

So we end up having lots of conversations there to figure

Zach:

out where should we target.

Zach:

And again, having those conversations about the weightings too.

Zach:

But once we get to where we feel like is a good recommendation spot,

Zach:

this is where we end up taking those KPIs to our CIO council.

Zach:

I would say to gain agreement from the top.

Zach:

What's great about our practice is we've already got buy-in from the top as, as far

Zach:

as supporting the FinOps practice itself.

Zach:

And now it's just around the, are these the right KPIs and metrics

Zach:

to keep pushing their businesses along to make sure they're doing it.

Zach:

So it's the, again, team aspect from a, trying to determine what we're

Zach:

gonna go after, the recommendation, and then it's the agreement at the top.

Zach:

and these are metrics end up being discussed at CIO

Zach:

councils, across FinOps forums.

Zach:

So again, trying to make them just visible all the way up to the top

Zach:

senior leadership here at Fidelity.

Zach:

There's definitely great awareness that goes around with them too.

Ashley:

You're right now getting ready to go into a new year.

Ashley:

Are you all having those conversations around KPIs and what's gonna be the

Ashley:

weight percentage and which business unit you're gonna target this next year?

Zach:

Yeah, we just got through it.

Zach:

you know what's interesting too is the, when we went down, the

Zach:

KPI route, we used to do it out you know, the team aspect of it.

Zach:

If we succeed as an enterprise, we all succeed.

Zach:

And I think what's been great is to be able to get down to granularity from

Zach:

a federated perspective on business unit by business unit to truly see

Zach:

who can push and where to push.

Zach:

in some cases we may have a single business unit that happens

Zach:

to be a vast majority of spend or where the opportunity is.

Zach:

And that's always great as a, you know where to target.

Zach:

But on the flip side everybody contributes to it.

Zach:

So it's the, I kind of want to have what we'll say is like my own score

Zach:

too, as opposed to the enterprise.

Zach:

So getting to a next level of granularity and even down from

Zach:

the business unit, if we could get down to squad level or team level.

Zach:

I think that would be great from the next evolution.

Zach:

but yeah, we've done our planning as far as the weightings go.

Zach:

And again, we talked about it earlier.

Zach:

Gonna try to weight more back to the developer from a underutilized

Zach:

perspective or utilization perspective to drive their consumption of compute,

Zach:

to the highest efficiency that we can, and just trying to put a stake in

Zach:

the ground from a, we would like the teams to be in target too, so that as

Zach:

they're implementing or using these resources, they're doing it efficiently.

Ashley:

One thing I ran into sometimes is we'd give a KPI and then we'd

Ashley:

have the "I'm special" effect, right?

Ashley:

Well, that KP I doesn't work for me.

Ashley:

My app's a little bit different.

Ashley:

you know, I can't, scale things down on the weekend than you're measuring that.

Ashley:

So have you ran into those occasions where you've got, the outliers and how

Ashley:

do you address those conversations?

Zach:

I think that's where they come back to the enterprise practice.

Zach:

So again, you take a, you talk about the spokes coming back, going, We get

Zach:

to learn along that journey as far as the, you know, I hit an application or

Zach:

I hit a scenario that doesn't feel like it's the norm and may feel like it's

Zach:

that special snowflake type of thing.

Zach:

, But that's what's great about the practices, bringing that

Zach:

conversation back, having it figured out, do we need to make a tweak?

Zach:

Do we need to make an adjustment?

Zach:

or is it just something we need to account for in the future?

Zach:

But those are the great conversations that have, you know,

Zach:

that we have in the practice.

Zach:

So it's the, at this point, I think we know what we learned, at most

Zach:

recently was more around, You know, we were expecting or hoping to get a

Zach:

certain discount level across the firm.

Zach:

we utilize reservations, I'm sure like most, enterprises do.

Zach:

What the difference is, is when I talk about that business unit by business

Zach:

unit score is, We weren't previously, I would say accounting for their mix

Zach:

of usage within different services.

Zach:

So if I'm gonna execute on a three year savings plan, you could probably estimate

Zach:

that we're gonna get a 50% discount if I'm a higher user of RDS or EBS or something,

Zach:

that carries a different discount, I might not be able to get to what we'd have set

Zach:

for the threshold for the enterprise.

Zach:

So now it's about calculating a different mix to make it an achievable metric

Zach:

for their particular business, as opposed to just the one size fits all.

Zach:

So, not necessarily the Snowflake effect, but more of the, I gotta make it purpose

Zach:

fit for the business unit on how they're consuming resources for their business to

Zach:

give them the right target to go after.

Ashley:

I love that and you wouldn't even able to even know that unless you

Ashley:

had these metrics to start with, to then therefore have those conversations and

Ashley:

then cycle back through this of making those changes that need to be done.

Zach:

And that's where it is.

Zach:

When we're looking to break down these, how are we gonna be successful as a team?

Zach:

Getting to the metric where I mentioned earlier with like, you may have one

Zach:

business unit that's consuming more of the resources we have, you may have

Zach:

one business unit that's consuming more RDS than another business unit.

Zach:

One that's consuming more, EC2 , or open search or, or what have you from

Zach:

a reservable services perspective.

Zach:

But you know, it's just getting that understanding and making

Zach:

it applicable to the businesses.

Ashley:

What's so interesting too with you being centralized is, let's say you have

Ashley:

a business unit that's really, really.

Ashley:

Spot and you can find another business unit that's not using it at all.

Ashley:

You are able to leverage , that macro view that you see and make

Ashley:

those connections between those teams, probably to say, Well, this is

Ashley:

really good practice happening here.

Ashley:

Let's connect you with you folks, which they would've seen otherwise.

Ashley:

I think if you weren't sitting there at a central hub perspective,

Zach:

Yeah, I mean the spot one's a great example and we talked earlier

Zach:

just about that first year where we went in with a single business

Zach:

unit that was carrying the load.

Zach:

And what's great about it is like that business unit certainly able to

Zach:

contribute whatever that pattern is for how they're consuming spot in, the

Zach:

special things that they're doing to help drive their usage in an upward direction.

Zach:

And get the best cost that they can get out of that.

Zach:

So it ends up coming back to that sharing of those best practices and

Zach:

seeing one business unit on their journey happens to be able to hit that

Zach:

target, achieve that target, exceed that target, and then be able to share back

Zach:

maybe prior to another business unit who isn't as far along on their journey.

Zach:

So again, getting the best of both worlds.

Ashley:

Have you seen a case where KPIs have helped uncover operational changes

Ashley:

and we talked about the spot one.

Ashley:

But, for instance where you're able to see something centrally, like

Ashley:

a lot of moving to serverless or a lot of this thing seems to be

Ashley:

happening, like architecting changes.

Ashley:

And because you have those KPIs and everyone's trying to measure

Ashley:

to those KPIs, it's brought across some operational changes as well?

Zach:

We had one in one of our teams with the Kubernetes environment where

Zach:

it's, You know, the size that they're using, they go to make a change.

Zach:

And, from a performance perspective, everything's working out great.

Zach:

And then all of a sudden you see the KPI metric dip.

Zach:

And then it comes back to now we're having discussions around, you know, Let's go

Zach:

look from a right side perspective, and again, just the fact that we're having

Zach:

those conversations now when we're making changes as far as the cost aspect is

Zach:

when we talk about the culture before and trying to push that, like, that's

Zach:

probably the biggest one that I've seen as far as it's being accounted

Zach:

for when we're making the changes.

Zach:

And certainly the, if you were making the KPI yesterday and now today

Zach:

you're not, and they're actually.

Zach:

Like, fantastic.

Zach:

we're doing our job as a team.

Ashley:

Yeah, that makes a lot of sense.

Ashley:

, so, , everything, you talk about kpi, you're giving these

Ashley:

teams a lot of visibility.

Ashley:

I would imagine that you're pulling the data and putting it into reporting.

Ashley:

what does that process look like for you getting that data?

Ashley:

What is the frequency?

Ashley:

What does the output look like?

Ashley:

Who's the people that are checking those dashboards?

Ashley:

Tell us a little bit about that.

Zach:

Yeah, we have a third party tool that we utilize to get

Zach:

visibility to the billing files from both our cloud service providers.

Zach:

We also pump data into the data lake as well.

Zach:

So even if you got a third party tool, when we talk about our or

Zach:

data driven dashboards, that we're pulling in the right data, making

Zach:

our calculations, making it visible, bringing in additional data points.

Zach:

So when we talk about.

Zach:

Senior leader or hierarchy from a business unit perspective, or down

Zach:

to the teams just getting to that level of granularity that we might

Zach:

need to do outside of the tool.

Zach:

And again, from our KPI perspective, it's updated on a daily basis.

Zach:

the environment changes so frequently that, you just wanna stay on top of it.

Zach:

You wanna see the trends you wanna see when you take a dip.

Zach:

No different from an anomaly detection alert, the same as how we feel about it

Zach:

from a KPI perspective of just making sure that, you know, you've got eyes on, at

Zach:

the right level and at the right time.

Ashley:

And so you're bringing in the billing data , but it sounds like you're

Ashley:

pulling some data from other sources.

Ashley:

What does that look like?

Ashley:

Is that tagging, Is that associating with business unit?

Ashley:

What is the other data you pull in?

Zach:

All of the above.

Zach:

So any data from the applications perspective, I would say

Zach:

organizational hierarchy structure.

Zach:

you could take in, like I mentioned, any of the additional calculations, , that

Zach:

we're doing to create some of these.

Zach:

We talked about the weighting, there's some math that needs to take place.

Zach:

you know, the tools from a third party perspective are great.

Zach:

but I would say at least for us, most of the time, the people that

Zach:

are consuming them are FinOps practitioners like ourselves.

Zach:

I'd love to say every developer is sitting in one of those tools.

Zach:

But they're probably not.

Zach:

So , how do I make the data as consumable and actionable as possible, to at least

Zach:

alert them to go look at something.

Zach:

it seems like it might be a little bit overwhelming at times to go, I don't

Zach:

quite know what I'm looking for, You know, as opposed to the, I'm gonna

Zach:

point you to where to look, and then we could talk about trying to, I would

Zach:

say deep dive into the additional.

Ashley:

Yeah, that makes a lot of sense.

Ashley:

going back to personas, have you had to develop any type

Ashley:

of thing for each persona?

Ashley:

so when they go in, they're only seeing their KPIs, or are you guys

Ashley:

pretty transparent as an organization?

Zach:

Yeah, I think we have different levels from our dashboarding perspective.

Zach:

You know, there's the ones that you would show to senior leadership as

Zach:

far as the, how are we doing from a high level KPI perspective, down

Zach:

to what I would consider, , more of the power user perspective.

Zach:

So, if I mention purchasing strategy and we're talking about, what's

Zach:

the discount that we're getting across all reservable service.

Zach:

If you actually drill into more of the power user type dashboards,

Zach:

you'll see the service by service.

Zach:

what are your usage?

Zach:

How many hours?

Zach:

What are your ISF hours?

Zach:

What's your coverage rate?

Zach:

What's your discount you're receiving?

Zach:

So at least from that power user perspective, you get down into a

Zach:

different level of detail as opposed to the senior management work stuff.

Zach:

Are we meeting or exceeding our goals as opposed to the

Zach:

why are we, or why are we not?

Zach:

So couple different levels.

Zach:

And then some of the progress that we've seen, like I mentioned earlier, was , if

Zach:

I can get down to that next level and try to get some sort of, assignment or

Zach:

score, You know, you love the competition that'll take place within teams to

Zach:

go, I want to be number one on the leaderboard, as opposed to it's not

Zach:

necessarily a wall of shame, it's truly be, you know, a wall of fame for us.

Zach:

I like the internal competition that we've at least driven in our

Zach:

business unit to try to push teams to strive to exceed those KPI metrics.

Ashley:

I love that.

Ashley:

I was just gonna ask you if you did need gamification, so it sounds like just

Ashley:

inherently competition is driving that.

Zach:

Yeah.

Zach:

Yeah.

Zach:

And again, we've utilized the gamification aspect and more from a

Zach:

learning perspective as opposed to the, driving to get the KPI results.

Zach:

It's more of the, I wanna try to educate as much as possible.

Zach:

Our enterprise team did a fantastic job this year with, gamification, where

Zach:

we truly had, 1200 plus users playing the game over a four week period.

Ashley:

Amazing.

Ashley:

So where do you hope to be a year from now with your KPIs?

Ashley:

, what's your aspirations?

Zach:

I don't think we're ever done.

Zach:

I, you know, and I said that in a positive light , we mentioned the

Zach:

metrics thing, before it's the trying to figure out what's next.

Zach:

It's, when we think about the, you know, I'm gonna follow the dollar and,

Zach:

you know, where are we making most of our spend or where we have the

Zach:

opportunity to gain more efficiencies, more metrics keep popping up to

Zach:

investigate and do analytics around.

Zach:

So it's the, I'm always trying to figure out what that next thing.

Zach:

I don't think we ever drop a kpi.

Zach:

It's more of a kpi, maybe turn into another metric, and it seems like we keep

Zach:

adding to the list, which is fantastic.

Zach:

There's just a lot of moving parts and opportunities, but it's, again, I think

Zach:

it's looking for that next thing and.

Zach:

the more that we can keep pushing that drives that culture of

Zach:

accountability, the better off we'll be.

Zach:

So how do I bring to light actionable metrics or actionable KPIs for the

Zach:

development community to utilize.

Ashley:

Yeah, it's interesting too cuz talk a lot in finops about muscle memory.

Ashley:

You know, you do forecasting once it's really hard.

Ashley:

Next year, it gets a little bit better.

Ashley:

You get more agile, you get done more quickly, and eventually

Ashley:

you get to the point where it's like, Okay, I've got this.

Ashley:

And that's probably the same thing with KPIs and your organization or

Ashley:

engineers, , they know what KPIs they know what they're being measured.

Ashley:

They know what's levers to pull to move those KPIs and it just becomes

Ashley:

another function of their role.

Ashley:

initially this may have been a new term to introduce to them for five

Ashley:

years in, It's just another part of their role and responsibility.

Zach:

The muscle memory's a key thing too because it's new in a lot of

Zach:

cases, so even some of the things that we're pushing that you could use some

Zach:

automation to go and do, there's some sort of at times, hesitancy to go,

Zach:

I'm afraid to go full force before we truly test it out and see, and

Zach:

get a comfort level to the point.

Zach:

Then they're like, Okay, make it an official policy and go turn it on

Zach:

everywhere and we're gonna go make that part of our daily practice.

Zach:

So it's one more thing that developer doesn't have to do because we've

Zach:

created some sort of comfort level with the execution aspect, knowing

Zach:

that our automation works and we're doing the right thing and going by

Zach:

our mantra of, you know, do no harm, especially to production along the

Zach:

way to getting the cost benefit.

Ashley:

Has like KPIs helped you build some automation?

Zach:

Yeah, When we started out, a lot of the things, especially in the unutilized

Zach:

aspect, from a cleanup hygiene, they were grassroots of a going to account owner

Zach:

by account owner and trying to gain buy in to go turn some of this automation on.

Zach:

and in a lot of cases it's a little bit of that fear factor of the unknown.

Zach:

And, once you make some headways and get that comfort level, then

Zach:

it becomes a, okay, we're gonna go back to the CIO community and

Zach:

go, I wanna make this a policy.

Zach:

give me the thumbs up across the firm.

Zach:

And going, Yeah, flip that switch.

Zach:

And it's one more thing we don't have to think about.

Zach:

It's just gonna be automated going forward.

Zach:

And they'll have them focus on what that next thing is from either utilization

Zach:

perspective or what have you.

Ashley:

I love that.

Ashley:

Okay.

Ashley:

One last piece of advice to the audience here.

Ashley:

where should they start with KPIs?

Zach:

Just pick something.

Zach:

you're on a journey.

Zach:

There's an evolution that's gonna need to take place.

Zach:

There's maturity within your organization as well as what

Zach:

KPIs you're looking to measure.

Zach:

You know, just start with something and see where it takes you.

Zach:

there's gonna be something new that's gonna come down the line.

Zach:

There's gonna be some lessons that you're gonna learn along the way.

Zach:

But I would just say putting some sort of stake in the ground and

Zach:

trying to measure yourself against that, is gonna be beneficial.

Zach:

We learned a lot over the last four years.

Zach:

A along our journey, again, we talked about the waiting system.

Zach:

We talked about focusing on purchasing effectiveness versus underutilized assets,

Zach:

and there's probably a whole lot more.

Zach:

but it's just a, it's a maturity curve that it was.

Zach:

Okay.

Zach:

this feels right for where we are, in our journey along the way.

Zach:

And, we'll see how it goes from measuring our success.

Zach:

So no silver bullet, no magic one kpi.

Zach:

, but just at least figuring out, you know, let's just say one thing to, to

Zach:

focus on and see if you can push and drive that culture of accountability

Zach:

throughout your organization.

Joe:

All right, FinOptinauts, KPIs, it's not so much what you're measuring, but how

Joe:

you're measuring and how you're driving accountability and culture with them.

Joe:

So many folks ask, What things should I measure?

Joe:

I think the lesson here and also the lesson from the Tony Johnson Unit

Joe:

Economics episode we did a month or so ago, it's really not so much about,

Joe:

the thing you were measuring, but how you do it and what you do with it.

Joe:

Make sure you iterate and grow as you learn and get better at things.

Joe:

Pretty good lessons in this one.

Joe:

So thank you Zach Stithem and the Fidelity team.

Joe:

Fantastic partners in this community also.

Joe:

Big thank you to Ashley Hromatko.

Joe:

Doing a great job on this interview.

Joe:

really digging into the topic.

Joe:

As always, thank you to Stacy Case for bringing the energy

Joe:

with me at the beginning.

Joe:

It's always fun.

Joe:

It always feels like we'd never done it before.

Joe:

If you want more KPIs with Zach and with Noel Crowley, in the show

Joe:

notes, there'll be a link to their presentation that they gave at FinOps X.

Joe:

You can check that out on our YouTube site along with every other recorded

Joe:

breakout session from FinOps X 2022, we talked about FinOps X 2023

Joe:

coming up next year in San Diego.

Joe:

Cannot wait for that.

Joe:

Very excited until then, check out everything from last year, and get

Joe:

excited for next year get pumped, get excited, all right, folks.

Joe:

Thank you for listening.

Joe:

Hope to see you soon.

Joe:

Somewhere out there.

Next Episode All Episodes Previous Episode

Listen for free

Show artwork for FinOpsPod

About the Podcast

FinOpsPod
Advancing FinOps Practitioners in Podcast Form
FinOpsPod connects practitioners with the rest of the FinOps Foundation community. Real world practitioners will share their experiences and the Foundation team will share the latest news about the community. Learn more about the FinOps Foundation at www.finops.org.