[Announcer]: Welcome to the Analytics Power Hour.

[Announcer]: Analytics topics covered conversationally and sometimes with explicit language.

[Michael Helbling]: Hi everybody, welcome.

[Michael Helbling]: It's the Analytics Power Hour.

[Michael Helbling]: This is episode 281.

[Michael Helbling]: It's finally done.

[Michael Helbling]: The big report for the monthly business review.

[Michael Helbling]: And you send it off.

[Michael Helbling]: And over the next few days, you hope maybe you'll hear something back.

[Michael Helbling]: You might not hear anything.

[Michael Helbling]: As analysts, sometimes we don't get a ringside seat to how the business uses the data reporting and analysis that we provide.

[Michael Helbling]: I mean, we certainly hear about it when something goes wrong, but how is the data being used?

[Michael Helbling]: What strategy is it supporting?

[Michael Helbling]: How does your work fit into the bigger picture?

[Michael Helbling]: Hey, Moee, kiss.

[Michael Helbling]: You're an executive yourself now, but I think you remember a long time ago being an analyst with questions like these.

[Moe Kiss]: I still have these questions.

[Michael Helbling]: Let's be real.

[Michael Helbling]: Oh, okay.

[Michael Helbling]: Well, yes.

[Michael Helbling]: And my other co-host, Val Kroll.

[Michael Helbling]: I know you also have a lot of executive experience, but I know you and I've shared many stories about learning the ropes as we climbed up.

[Michael Helbling]: Stories or scars?

[Michael Helbling]: No.

[Michael Helbling]: Well, the best stories come with scars a lot of times.

[Michael Helbling]: And I'm Michael Helbling.

[Michael Helbling]: And I'm really excited for our guest because we're finally getting some answers directly from the top.

[Michael Helbling]: Anna Lee is the CEO of Flybys, Australia's number one loyalty program.

[Michael Helbling]: In her three decades of experience, she's held CFO, COO, and other leadership positions at companies like the Iconic, Nora, and Groupon.

[Michael Helbling]: And today she is our guest.

[Michael Helbling]: Welcome to the show, Anna.

[Anna Lee]: Thank you so much.

[Anna Lee]: It's such a pleasure to be here with you guys.

[Michael Helbling]: Yeah, I'm so excited to talk to you.

[Michael Helbling]: I think this is a great opportunity for our audience to kind of see the other side of the business that we don't get to interact with a ton.

[Michael Helbling]: But as we roll into the conversation, I think it's a good thing to set the playing field and for everybody to understand is, could you tell us a little bit about Flybys and your role there as CEO?

[Anna Lee]: Well, maybe the best way to do that is maybe provide a little bit of context of my career.

[Anna Lee]: Firstly, I'm a chartered accountant, so my background is actually finance, so very good with numbers.

[Anna Lee]: And so I spent many decades as finance director, CFO kind of roles, and that really helped me understand numbers and the depth of [Anna Lee]: business behaviors and decisions and then I had an opportunity to move into the chief operating officer and that was when I was at the iconic which is just before flybys and that really allowed me to focus.

[Anna Lee]: and concentrate my efforts and my knowledge around operational breadth.

[Anna Lee]: So understanding how all the dots connected in the business, understanding how to really create productivity, efficiency, and drive performance.

[Anna Lee]: And then about three years ago, I had the opportunity to join Flybys as its CEO.

[Anna Lee]: And I think that really then became the culmination of those skills combined with leadership.

[Anna Lee]: Flybys is [Anna Lee]: Australia's largest retail coalition loyalty program.

[Anna Lee]: It is a joint venture standalone entity.

[Anna Lee]: It is owned 50% by Coles Group and 50% by West Farmers Group, two of some of the biggest ASX top 10 listed businesses in Australia.

[Anna Lee]: And my role is was to take the business into a more digitally and member focused direction.

[Anna Lee]: And of course, a huge part of that.

[Anna Lee]: is making it data-led.

[Moe Kiss]: How do you think your experience?

[Moe Kiss]: I suppose one of the things that's always top of mind for me is different execs seem to have different perspectives and views on data.

[Moe Kiss]: Coming through the finance world, I feel like that would be a quite different perspective maybe than some other CEOs.

[Moe Kiss]: When you look around your peers and how they got to their position, do you feel like finance gave you a special edge or [Moe Kiss]: Or is it like unique in any way or do you think that those skills just get learned regardless of how you come through the industry?

[Anna Lee]: I think CEOs come in different shapes and sizes.

[Anna Lee]: It really comes down to what is suitable for the business at its time.

[Anna Lee]: I think one of the things that [Anna Lee]: many businesses that I've been part of have been quite digitally focused, so less physical footprint, more reliant on what the data is telling us.

[Anna Lee]: That was true by time at the Iconic, which was just prior to Flybuys.

[Anna Lee]: And of course, with Flybuys, even though you see it obviously at the supermarkets and bunnings at Kmart, [Anna Lee]: itself is a digital business.

[Anna Lee]: And so the importance of understanding data and for things to be evidence and fact-based skews more towards that rather than, you know, perhaps some of the other pieces of data that are more physically demonstrated.

[Anna Lee]: So I think there's no right or wrong on that, Moe.

[Anna Lee]: I think it really comes down to what is important.

[Anna Lee]: And I think [Anna Lee]: What has been interesting in Flybys is it's a brand that is very well known.

[Anna Lee]: It has almost 95% brand awareness and it has been around for 30 years.

[Anna Lee]: Almost every Australian knows Flybys, has seen it, knows about it and we are almost a 10 million active members.

[Anna Lee]: It is the scale of it in Australia is quite big.

[Anna Lee]: The good and the bad thing is that the good thing is that you have enormous amounts of information that talks to member behaviour and so on so that you can really create and unlock this value for them using personalisation and so on.

[Anna Lee]: On the flip side, [Anna Lee]: Over 30 years, there's a lot of myths and beliefs and stories that some are true, some are not.

[Anna Lee]: But they've been kind of the culmination of many different people and many different stakeholders.

[Anna Lee]: And so right now, what was important was to reset those stories and to ensure that they were properly fact-driven and data-driven.

[Anna Lee]: So right now, I think that is the right thing for the business now.

[Anna Lee]: I think the experience that I have around confidence in data and numbers and so on and the storytelling that it creates is what's right for the business now.

[Val Kroll]: I love that.

[Val Kroll]: Resetting the stories.

[Val Kroll]: I've definitely worked at organizations and institutions and worked with companies that hang on to those stories and it's to its detriment, right?

[Val Kroll]: And it's so funny how quickly something becomes the truth if it's set enough times in a meeting, right?

[Val Kroll]: I really like that that's one of your priorities.

[Anna Lee]: Absolutely.

[Anna Lee]: And they also can go quite wide.

[Val Kroll]: Use as weapons.

[Anna Lee]: No, no, not at all.

[Anna Lee]: It's probably more about I think people get quite.

[Anna Lee]: enthusiastic about the storytelling and perhaps sometimes, you know, further embellishments are made on stories that perhaps were a lot more simpler and potentially were quite fact-based and grounded, I'm certainly not suggesting any of it was, you know, not fact-based but over time some of those stories just become, you know, [Anna Lee]: they become bits rather than facts.

[Val Kroll]: Sure.

[Val Kroll]: Well, so I'm going to ask one other foundational question, because I'm sure that this will be an important backdrop element as we get into some of the other questions and discussion.

[Val Kroll]: But 30-year-old company, you're charged with bringing it more digital.

[Val Kroll]: I'm curious how you engage with data.

[Val Kroll]: So what is the current organizational structure if you want to talk a little bit about that?

[Val Kroll]: Or how do you interact with the data practitioners or analysts?

[Val Kroll]: or data scientists across your organization?

[Anna Lee]: Well, I think in the first instance, I have to acknowledge my role as the CEO.

[Anna Lee]: So it's important that I seek answers that are evidence-based and data-led.

[Anna Lee]: So it very much comes from the top.

[Anna Lee]: So that creates a cultural conditioning.

[Anna Lee]: So that makes it very clear that [Anna Lee]: We do not want to talk about things that are perhaps not so data-led.

[Anna Lee]: I mean, hunches are great, but let's have a look at the data and understand it.

[Anna Lee]: What that drives is then the focus on ensuring that the data is easy to access.

[Anna Lee]: It's reliable.

[Anna Lee]: It's credible.

[Anna Lee]: And also, culturally, the business has a good level of [Anna Lee]: understanding around how to use data and insights.

[Anna Lee]: So I think first and foremost, it's multi-layered.

[Anna Lee]: So you set the tone, you have to acknowledge there's a CEO, you have the...

[Anna Lee]: biggest way of influencing how it's extraordinarily powerful.

[Anna Lee]: And I think being very clear that data plays such an important role in making decisions, that's super important.

[Anna Lee]: I think the other piece that I like to, there's a couple of other themes that I really like, which is productive curiosity.

[Anna Lee]: So I really want people to be curious about the why [Anna Lee]: of things that they see.

[Anna Lee]: So if we're seeing certain metrics behave or move differently, it's really important to be curious, but then you don't want to be curious to the point where it's just interesting and it's [Anna Lee]: It's just a wasted effort.

[Anna Lee]: And the last thing we need is people just running off and finding answers to things that don't actually move the needle.

[Anna Lee]: So I kind of like it to be, there's a real skill around productive curiosity.

[Anna Lee]: So embedding that type of mindset, that's what I kind of demonstrate as lead by example.

[Anna Lee]: It's like being very, very disciplined about the types of questions that I ask, not random things, but very focused on what the right thing.

[Anna Lee]: The other concept I really like is also what I call informed pragmatism.

[Anna Lee]: So again, data teams often love most just going to die when I tell them.

[Anna Lee]: But, you know, there is just this desire for a lot of data scientists and analysts to be really perfect and exacting and precision because that is a reflection on their own self-pride and there's a lot of attached to things being right.

[Anna Lee]: And I totally understand that because I spent decades in finance and everything was about it had to be right.

[Anna Lee]: But, you know, at some point you've also got to decide [Anna Lee]: What are the decisions that absolutely need precision?

[Anna Lee]: And what are the decisions actually just need enough to validate that it's kind of legit and it smells right?

[Anna Lee]: What other information that you can add, which are non-data perhaps, more qualitative aspects that then combine together to give you a pretty solid position to be able to make a pretty good decision that feels informed enough, but is pragmatic so that we're not sending people [Anna Lee]: On three weeks of digging for probably not much more certainty than if you'd stopped now so you know that they're kind of two concepts that I like to apply.

[Anna Lee]: I only just recently kind of put some words around that kind of concept, but that's kind of the approach that I have.

[Anna Lee]: So in terms of back to the question around how the teams are structured, we have quite a mix.

[Anna Lee]: So we have a centralized team that ensures the platform and the data hygiene is there.

[Anna Lee]: We've just spent and invested significantly on our platform uplift, a piece of work that was very much needed over years, even though business has been around for a long time.

[Anna Lee]: It hasn't actually had the platform to enable that.

[Anna Lee]: But as an enterprise capability, being data-led and understanding data is something that we expect of every single person.

[Anna Lee]: Now that has been a cultural transformation that we've been on.

[Anna Lee]: So we need to invest in the training for roles that previously didn't need that.

[Anna Lee]: And then we also need to hero, I guess, the people that, and champion those people that do have those skills and how do we harness those skills such that it's, you know, that skill is democratized.

[Michael Helbling]: You ever wish your data stack came with an easy button?

[Michael Helbling]: Well, enter Fivetran, the data integration platform that lets you set it, forget it, and sync it.

[Michael Helbling]: While you're catching flights, catching up on sleep, or catching waves, Fivetran is catching your data from every source and pumping it straight into your data warehouse.

[Michael Helbling]: No manual pipelines, no maintenance drama, no 3AM alerts because someone's API changed again.

[Michael Helbling]: just reliable, automated, always on syncing of your data.

[Michael Helbling]: So your dashboards stay fresh, your analysts stay happy, and you, you get some well-deserved R&R.

[Michael Helbling]: Take a break.

[Michael Helbling]: Well, take two.

[Michael Helbling]: 5Tran's got you.

[Michael Helbling]: Go learn more at 5Tran.com slash APH.

[Michael Helbling]: You can see interactive demos.

[Michael Helbling]: You can even get a 14-day free trial.

[Michael Helbling]: That's F-I-V-E-T-R-A-N dot com slash APH.

[Michael Helbling]: Is your out-of-office hero that your Datastack deserves?

[Michael Helbling]: Five Tran.

[Moe Kiss]: That sounds amazing.

[Moe Kiss]: I honestly, I wish every...

And I agree with you a lot more than...

Yeah.

[Michael Helbling]: I'm stealing both of those phrases, by the way.

[Michael Helbling]: Just FYI.

[Michael Helbling]: Yeah.

[Moe Kiss]: Yeah, but the thing that's actually that I love is like the self-awareness of, I think sometimes folks in leadership roles don't realize like, they'll be like, oh, this would be interesting to know or like, would it be nice if I could look at blah?

[Moe Kiss]: And like, sometimes they don't realize like teams hear that and they hear, I need you to look at X and I need an answer that says Y with like 100% confidence.

[Moe Kiss]: And it's like, I wish all, [Moe Kiss]: folks leadership roles had that self-awareness that sometimes they can create swirl and that's not actually what we want.

[Moe Kiss]: What we want is what is the right information that we need to make a business decision and that's the point that we need to stop at.

[Anna Lee]: Exactly and I think there is also perhaps that's where my experience around my operational experience as COO, I was in that role for four years and we were obsessed with like unit economics and productivity and so I now think about [Anna Lee]: all of our team members' time as value add or opportunity cost.

[Anna Lee]: So it's got to always feel like this should only take you an hour or two.

[Anna Lee]: If it takes you longer than that, stop.

[Anna Lee]: Because it's not like the value of what you're going to come back with is not worth [Anna Lee]: Moere than two hours so i think giving people a bit of guidance around is that and you know is that something really really super important that's going to move a big needle or is that something that's like okay that's interesting.

[Anna Lee]: That will validate something else and therefore actually only need you to spend a very little amount of time on it i think it's also.

[Anna Lee]: Certainly, in my CFO days, I used to have CEOs that would ask me all sorts of random questions, and they had reasons for them.

[Anna Lee]: They felt random at the time, but I legitimately know that they were.

[Anna Lee]: I think what I understood as the CFO was the complexity level of getting that information.

[Anna Lee]: So often what I would counter in my CFO days was, you know, before I run off and get my team to spend a lot of time trying to dig that information, what is the problem you're trying to solve here?

[Anna Lee]: And it's about the clarity around what is the problem.

[Anna Lee]: And if you're clear on the problem statement, actually what you find is [Anna Lee]: there are different pieces of information that you can provide that gives you the same answer.

[Anna Lee]: And often I would suggest something that was much more pragmatic that would still provide the answer rather than delivering the specific requests that the CEO provided.

[Anna Lee]: And I think having had that benefit of that experience now in the CEO role, [Anna Lee]: I'm able to kind of counter not just request a specific, the danger is requesting specific metric, rather what I like to compliment that with is, hey, I'm wondering if you could tell me if there is a strong correlation or relationship between these two metrics, because what I like to know is does that have an impact on this strategic direction that we have?

[Anna Lee]: It allows people to understand why I'm asking the question rather than random curiosity.

[Anna Lee]: And also, I think it allows for, you know, whether it's the, you know, our head of data or analysts or anyone to say, hey, I get what you're trying to solve.

[Anna Lee]: But actually, that is not an easy metric to do that might take the team quite a lot.

[Anna Lee]: But actually, I can give you something else that probably give you the same answer.

[Anna Lee]: And I'm like,

[Val Kroll]: So I feel like Anna could have a golden dot off.

[Val Kroll]: So I'm thinking like, okay, a CEO is like, I like to articulate hypotheses for my data teams to help me validate.

[Val Kroll]: So I'm thinking, and I hope your last call is, here's all the roles that I have open for my team, because everyone's gonna come work for you after this.

[Anna Lee]: That's awesome.

[Anna Lee]: No, I love it.

[Anna Lee]: And I think there's also, I think the other piece that's quite difficult as well, like, you know, different, again, different businesses, I've seen different setups, but, you know, having that capability across.

[Anna Lee]: the business, no matter what your role is, and it obviously will be different depending on your role.

[Anna Lee]: Some might just need foundational understanding, and of course, some roles will need to have a lot more sophisticated understanding.

[Anna Lee]: But being able to, I mean, my vision and ambition is for everyone at Flybuys to have [Anna Lee]: the right level of competency around understanding data that it necessarily have to have founded.

[Anna Lee]: But they need to know what to do with it and whether that piece of information is important or not.

[Anna Lee]: That's where we want to go.

[Anna Lee]: And so that's been actually quite a cultural transformation in itself.

[Anna Lee]: And if you're going to expect that, then as a business, you also need to provide the support for everyone to get there.

[Val Kroll]: I love that.

[Val Kroll]: One of the things I've been hearing you say since the top of this episode is evidence, and I love that you use that to frame out.

[Val Kroll]: Because you're acknowledging there's lots of different ways that we can get at this.

[Val Kroll]: There's lots of different tools and methodologies.

[Val Kroll]: Lots of different levels of certainty, depending on the question that's being asked or the decision we're trying to make.

[Val Kroll]: But one of the things I've noticed hinder organizations from being so flexible about the evidence that they bring to bear is sometimes organizational structure.

[Val Kroll]: So if a senior leader asks a specific person or team that has [Val Kroll]: you know, specific tools in their tool belt or like guild like groups, if you will.

[Val Kroll]: Hey, here's a question that I have, but the most appropriate tool if you're considering the economics, like you were saying, like the units is actually market research, but that team sits in the completely different part of the business.

[Val Kroll]: What are some of the ways, like if you were talking to another senior leader, because this is the type of change that as analysts, we always say, well, that's something that has to come from the top.

[Val Kroll]: So here's my chance to ask the top.

[Val Kroll]: What are some of the things that you would recommend or that you think we should keep in mind or think about to try to help change some of this culture to be a little bit more flexible with how we right size the different tools and methodologies with the questions of the business?

[Val Kroll]: And you might say that it does have to come from the top.

[Val Kroll]: Tell their CEO to call me.

[Val Kroll]: But I'm just curious.

[Val Kroll]: I want to explore this because I think this is really important and something that I think is a really important evolution for lots of data teams and practitioners.

[Anna Lee]: That's a lot of unpacking.

[Val Kroll]: Just take that wherever you want to go.

[Anna Lee]: Well, maybe a couple of ways I might approach that.

[Anna Lee]: Not sure if I know the exact answer to that, but maybe a couple of ways that I think about things is I think the business needs to find the right balance between routine [Anna Lee]: cadences and discipline and a culture of creating space for whether we call the practitioners or team members to [Anna Lee]: um question and and find things I think like for example um like for me I think sometimes people ask me you know daily, weekly, fortnightly reporting what's your what's your view on that um and my view is is that um cadence is only important for consistency so I'm a real believer of discipline so [Anna Lee]: I don't want to be making decisions and the business being made decisions on ad hoc things that are fluid and so on.

[Anna Lee]: So I think some sort of regular reporting structure creates [Anna Lee]: consistency and constant monitoring.

[Anna Lee]: So if you get invest right and you get the right metrics, then you should be looking at some of these things quite regularly.

[Anna Lee]: And that grounds people in the right evidence, but the forums which you talk about them [Anna Lee]: create the space for people to ask the questions.

[Anna Lee]: I'm not sure if I'm clear there.

[Anna Lee]: But maybe an example would be like one of the things that we introduced in his business, which was probably some of the hardest things that we ever did was, as part of this data reset, was we actually hold fortnightly Valley Driver Tree meetings.

[Anna Lee]: And that is actually how we drive performance in the business.

[Anna Lee]: It's 90 minutes every fortnight.

[Anna Lee]: We have allocated people who own each of these metrics.

[Anna Lee]: And as part of that, we needed to ensure that there's a lot of time spent on what the right metrics are.

[Anna Lee]: It also drove a lot of time investment in getting those metrics reported easily, seamlessly.

[Anna Lee]: They didn't exist [Anna Lee]: the start and also required us to train and coach our people to talk to those metrics well and create the space where people felt they understood what that was.

[Anna Lee]: But the great thing with Valley Driver Trees is they really help you understand which of the branches of the tree are the most impactful.

[Anna Lee]: So it helps us also to prioritise the things that matter.

[Anna Lee]: So all of a sudden, [Anna Lee]: Instead of talking about every single branch equally, that allowed us to say, actually, that branch contributes 85% of everything.

[Anna Lee]: Let's make sure we talk about that.

[Anna Lee]: The other stuff is important, but actually, you could spend half an hour talking about one thing, but if it only is worth less than 1%, it's probably not that useful to talk about.

[Anna Lee]: So that allowed us, so that's been a real journey for the business and I think as leaders, as the CEO, I intentionally turned up at those meetings and try my best to kind of influence role model, how we talk about data in a way that [Anna Lee]: is grounded in numbers, but actually the narrative and the discussion is about the business.

[Anna Lee]: So when we are talking about points issuance, we are just talking about how many billions of points we've issued and so on.

[Anna Lee]: What we connect it back to, you know, that's been partner activity, how much value has that created for members?

[Anna Lee]: What's the quantifiable amount?

[Anna Lee]: Is that up or down?

[Anna Lee]: Are we delivering, you know, more value to our members compared to last year?

[Anna Lee]: Those sorts of conversations.

[Anna Lee]: So I think it's quite multi-layered, really.

[Anna Lee]: You've got to have the right systems and platforms.

[Anna Lee]: You've got to have the right metrics.

[Anna Lee]: You've got to have the right forums, and you've got to guide and share those meetings in a way that focuses people on the right conversations.

[Moe Kiss]: I love hearing how much personal responsibility you take about the data culture at your company.

[Moe Kiss]: That is just a really incredible thing to hear.

[Moe Kiss]: But I imagine in previous roles, [Moe Kiss]: There have been also, because it's like it's a two-way street, right?

[Moe Kiss]: It has to be top-down and bottoms-up.

[Moe Kiss]: And part of the data culture also comes from the data folks that you employ and their managers and all that sort of stuff.

[Moe Kiss]: What are the things that you've, just in all of the roles that you've had, that you've seen really stand out as like what makes a really strong data team more looking at like the bottoms-up attributes versus kind of the top-down influence you can have?

[Moe Kiss]: Like what are those things that we should really be striving for in our teams and as data scientists?

[Anna Lee]: I am a big fan of the so what like everyone in the business will know it's like if you tell me something I'm going to go what's the so what so often I'll be you know listening to something or a reader a report or a document and you know we still find you know there are times where you know the author will [Anna Lee]: at a table of information and very little narrative or commentary about what that information is.

[Anna Lee]: So I think one of the most important things, I don't want information for the sake of information.

[Anna Lee]: Not only is that, I guess, [Anna Lee]: not very evaluating, but actually it actually pushes a lot of cognitive load on the reader.

[Anna Lee]: So for me, I look at the information, I'm like, Oh, what am I meant to make of this?

[Anna Lee]: Like, is it, am I looking at this column or this number?

[Anna Lee]: Or what is it that I need to draw out?

[Anna Lee]: So I think there's a real, real important skill for [Anna Lee]: anyone, whether data is in your title or not, it doesn't matter, like this applies to anyone, is what is the so what.

[Anna Lee]: And if there isn't a clear so what, then actually just take the table out.

[Anna Lee]: I actually prefer because it's just noise.

[Anna Lee]: So I think it's [Anna Lee]: It's about actionable insights.

[Anna Lee]: I know that's a cheesy word it's often used, but that is actually how I interpret it.

[Anna Lee]: Is this information going to change or impact?

[Anna Lee]: Does it validate where we're going?

[Anna Lee]: If that's the case, then show me whereabouts in the data table that it shows that.

[Anna Lee]: something that we're doing now.

[Anna Lee]: we should raise awareness that it's like, oh, hang on a minute.

[Anna Lee]: This doesn't look right.

[Anna Lee]: Should we be reviewing a decision that we've made?

[Anna Lee]: Or could also fit in the other category, which is like, oh, look, it's quite interesting.

[Anna Lee]: Can't tell right now, but we should watch that to see whether that does have an impact on something that a decision that we've made or we need to pivot or adjust or otherwise.

[Anna Lee]: So that's my favorite question is, [Anna Lee]: What's the so what?

[Anna Lee]: And I think that is also, again, it pushes and conditions a particular type of thinking in the business, because if you can't answer it, then you've probably got to work out a way to answer it.

[Val Kroll]: Obviously, there's a lot of clarity, especially when you're in these forums, you're talking about asking those questions and keeping focus on what's most important with the prioritization and the value framework that you talked about.

[Val Kroll]: But when data teams are partnering with some of the different functional groups, lower down on the tier, [Val Kroll]: It's a little bit easier to lose the thread sometimes, like connected back always to the strategic vision or the plan.

[Val Kroll]: Do you have any advice for data practitioners to make sure that they're anchoring on something that is actionable that's going to move the business forward?

[Val Kroll]: Because sometimes, again, it feels like you're looking in a smaller piece of the puzzle.

[Val Kroll]: But do you have any advice for the analyst that wants to be kind of performing in the way that you've described?

[Val Kroll]: What are some things they could do?

[Anna Lee]: Well, I think there's no right or wrong.

[Anna Lee]: I mean, there are times where it's really great for data people to stay focused on one particular thing and not get distracted.

[Anna Lee]: So I don't think there's a black and white, but I think your question's probably more on for those who, you know, [Anna Lee]: want to kind of go deep and also and broad as well.

[Anna Lee]: I remember when I was the COO at the Iconic and actually armed all of my operations directors with an analyst because I wanted them to be able to use the information.

[Anna Lee]: I wanted them to have someone that they could go to that would help them connect the business decisions with the information that was coming out of our systems.

[Anna Lee]: So I think that worked really well because I was able to ensure that every operations director was very clear because we had OKRs, we had a whole framework that set up what we were trying to get at.

[Anna Lee]: And for us at the time, it was very much about cost per order.

[Anna Lee]: That was our obsessive thing.

[Anna Lee]: Everyone needs to be obsessed about that because we owned it, we were accountable for it.

[Anna Lee]: So that then meant that [Anna Lee]: you know all the cost could be split up between the different areas and functions of operations and actually there was a beauty around the fact that you could have a group that focus on okay you guys are in charge of looking at the opportunities around delivery guys were all in charge of looking at the [Anna Lee]: operations excellence opportunities around, you know, taking the photography images that appeared on our pages, you know, there's a whole different set of functions that actually or ladder up to the collective costs and so.

[Anna Lee]: there's a benefit of everyone being focused but then every month we would come back and actually review what everyone contributed to and and be very clear which components of the costs were driven by what you know whether that was up or down and we'd have real learnings about that so um and that was a collective conversation I would have [Anna Lee]: you know, basically the whole operations leadership team on that core that would actually analyze all of that and everyone just knew it would just be one of those performance like what is that, what drove that and that really then honed I think all of the skills of not just the analysts but also [Anna Lee]: our operations team members to learn how to talk about the results and the big picture.

[Anna Lee]: And also a lot of it was also like, you know, no particular cost is ever driven in isolation with one function.

[Anna Lee]: Often it's upstream or downstream.

[Anna Lee]: So it was also this great camaraderie around this common goal.

[Anna Lee]: And we all wanted to collectively reduce costs by X and [Anna Lee]: I think creating this culture that we're all in together and collaboratively can solve it rather than create silos of like, oh, I overspent, that means I'm in trouble and another team underspent.

[Anna Lee]: And, you know, it was all about, okay, you know, being very open about, okay, we're gonna have, we're gonna have pressures on cost here this month.

[Anna Lee]: What can we do over here to compensate for that?

[Anna Lee]: And that was just a really wonderful team [Anna Lee]: culture that we created around this common goal.

[Anna Lee]: So it's a bit of clarity, but also collaboration as well.

[Moe Kiss]: It's really interesting.

[Moe Kiss]: What's coming to my mind is the word trust.

[Moe Kiss]: Because I think one of the things we often discuss as data folks is the fact that we need to give the business bad news.

[Moe Kiss]: We need to give the teams that we're working with bad news.

[Moe Kiss]: We need to give leadership bad news, that sort of stuff.

[Moe Kiss]: You're actually talking at another dimension, so it's not even the data folk doing that.

[Moe Kiss]: It's the business leaders and the different area leaders being willing to come to the table and have a discussion about, hey, this metric has gone down and here are the reasons why.

[Moe Kiss]: But having the trust that everyone around the table [Moe Kiss]: is part of the same team, we're all united on the goal.

[Moe Kiss]: This is a really interesting situation to hear about because what I'm hearing from you is so much more of, I think sometimes in the data space, we think that data culture is our responsibility and it's something that we need to drive across the business.

[Moe Kiss]: And what I'm really getting such an appreciation for is how much the leadership team can play, but also how much team culture more generally plays a role of, you need to have those really high levels of trust if you're going to have very robust conversations about what's working well and what isn't.

[Moe Kiss]: And that's what's top of mind for me.

[Anna Lee]: Yeah, I mean, that's really interesting.

[Anna Lee]: I feel it should never be the role of the Data Insights team to be driving the culture.

[Val Kroll]: Can you say that again for the back row?

[Anna Lee]: Because...

That's a very hard expectation.

[Anna Lee]: I think with everything, you know, I think as a CEO, you actually have to role model everything.

[Anna Lee]: So it's not just about data and insights.

[Anna Lee]: Of course, I, you know, I have to role, I see myself role modeling everything.

[Anna Lee]: So that's, that includes [Anna Lee]: psychological safety.

[Anna Lee]: It includes what our member first and lead narrative within the business.

[Anna Lee]: You know, all of that comes from the leadership team.

[Anna Lee]: And I tell my exec team that we must be the highest performing team in the business.

[Anna Lee]: We need to operate at a certain level.

[Anna Lee]: We need to role model.

[Anna Lee]: We need to demonstrate so that [Anna Lee]: people can see it and also you can play a role in coaching it.

[Anna Lee]: So I think with everything, I've not personally run a data and insights team specifically, but it's very similar to a finance team.

[Anna Lee]: So I've run many, many finance teams and I was in finance leadership for over two decades.

[Anna Lee]: And it was the same.

[Anna Lee]: You don't want the finance team to be the only people that care about money or can care about [Anna Lee]: commercials that creates a lot of friction and in fact extensive.

[Anna Lee]: We also apply data on analyzing our engagement survey results as well.

[Anna Lee]: It's very data-led there.

[Anna Lee]: But all of those insights will tell you that if you put the onus on [Anna Lee]: specialist teams to drive deep cultural behaviors that you expect of everyone, that's it's going to end in tears because you're just going to have a lot of frustration, a lot of resentment.

[Anna Lee]: So, Ashley, as a CEO, I actually have to spend a lot of time investing in what's important for me.

[Anna Lee]: So, as a CEO, I have to role model clarity of strategy, you know, [Anna Lee]: communication that is motivating and inspiring.

[Anna Lee]: Of course, the data approach and what my expectations are.

[Anna Lee]: So it has to start from the top.

[Anna Lee]: It absolutely has to.

[Anna Lee]: And that is a [Anna Lee]: That's an ongoing thing.

[Anna Lee]: You never get there perfectly because that's what business is.

[Anna Lee]: Business is just solving different problems.

[Anna Lee]: Sometimes this is the most important problem right now.

[Anna Lee]: Sometimes it's something else.

[Anna Lee]: Sometimes there's a fire over there.

[Anna Lee]: It's just a careful balance of all the levers that you have to pull.

[Anna Lee]: But I think for me, [Anna Lee]: I think, again, fundamentally, we are a business that needs to be member first and it needs to be data led.

[Anna Lee]: They are the hallmarks of what Live Eyes needs to be.

[Anna Lee]: And so we need to demonstrate we can't just be lip service to that.

[Anna Lee]: Like our vision is helping Australians get more of what they value beyond the checkout.

[Anna Lee]: And I think there are lots of businesses that will create these vision statements.

[Anna Lee]: And no one ever really looks at them and just goes, oh, yeah, that's what it is.

[Anna Lee]: And it's kind of philosophical and so on.

[Anna Lee]: But actually, one of the things that we're changing this year, which we reset our vision a couple of years ago.

[Anna Lee]: And now we're in a position where we've actually got three or four different measures around what member value is.

[Anna Lee]: So if we're going to be real and say, we're here to help Australians get more value, [Anna Lee]: you know, more of what they value beyond the checkout, then shouldn't we be demonstrating that we're actually providing them more value?

[Anna Lee]: And I think there's some basic things that we [Anna Lee]: Often businesses just don't, they don't think about because it's often it's there and no one ever goes, oh, but do we?

[Anna Lee]: And I think for me, it's like, no, no, no, I want to be able to speak to our people and say, this is our vision.

[Anna Lee]: This is what we've done in the last year, you know, and break down value in [Anna Lee]: what the members perceive as value, whether that is value that they've redeemed or it's value that they've earned through points.

[Anna Lee]: What are the movement of new members?

[Anna Lee]: What are they getting?

[Anna Lee]: What about existing members?

[Anna Lee]: Is that also showing the same vibe?

[Anna Lee]: It's just about being tangible and [Anna Lee]: being real about the things that you say.

[Anna Lee]: Otherwise, don't say it.

[Anna Lee]: Otherwise, you're just not true to what the purpose of the business is.

[Michael Helbling]: When you were talking about the iconic and how you were pairing your directors with an analyst, it got me thinking a lot about organizational structure.

[Michael Helbling]: for data teams because we talk about it in the analytics space, analytics world constantly.

[Michael Helbling]: Should we all be clustered in one group?

[Michael Helbling]: Should we all be spread out into the org?

[Michael Helbling]: From your seat and the things you've seen as you've led teams, led organizations, what preference model or what structure do you think serves the organization best from your perspective?

[Anna Lee]: The most important attitude is what's right for the business at the time.

[Anna Lee]: There is absolutely no cookie cutter, what worked.

[Anna Lee]: at the time probably wouldn't work at flybys now, for example.

[Anna Lee]: So at the time when I took on operations, one of the challenges was insufficient dashboards and metrics and understanding of that.

[Anna Lee]: But I needed to get that done very, like I need that skill capability to be done very, very quickly.

[Anna Lee]: So I needed to be able to talk the same language as my operations directors.

[Anna Lee]: And we were aligned in values, we were aligned in intent.

[Anna Lee]: Skill level was there because there were absolute legends and magnificent in running operations.

[Anna Lee]: but there was kind of like this language challenge and so the quickest way and it was a bit naughty because I wasn't approved those those roles but I just went inside.

[Anna Lee]: Well, that was a huge advantage because I'd been the CFO and...

Leaders leave, that's...

Well, I'd been the CFO and so I moved into the COO role and I kind of knew I was like, but you know, when I was CFO, people would kind of put on roles and they never like, you know, it was disgust, but it was like, okay, did you really need that role?

[Anna Lee]: But it wasn't like, you know, it was just like a little slap.

[Anna Lee]: So I thought, okay, well, [Anna Lee]: Now that I'm in the business, I wanna say bye.

[Anna Lee]: I'm like, I'm just gonna put these roles on and ask for permission, forgiveness later, right?

[Anna Lee]: Because if I didn't and if I stuck to the hard rules, [Anna Lee]: I just wouldn't actually drive the results.

[Anna Lee]: I felt very, very confident that that was the right thing, and it really turned out to be.

[Anna Lee]: In fact, those positions paid themselves off in six months, like the visibility.

[Anna Lee]: And so you kind of need to know and back yourself.

[Anna Lee]: And of course, as a result, the savings that we generated, more than offset the investment in those people.

[Anna Lee]: So that was the right thing at the time because of the gap between, I guess, the competency of being able to talk to the data and what I needed to see.

[Anna Lee]: Of course, in this role now, [Anna Lee]: what is right is what we've put in place, which is a bit of a hybrid.

[Anna Lee]: So we've got analysts that are better in the squads because they need to be able to provide very fast and relevant insights because they're embedded, they know what the squads are achieving, they know what their priorities are and so on.

[Anna Lee]: And then we also have a centralized team that ensures all the infrastructure, all the systems and all the big insights that are strategic.

[Anna Lee]: are done and there's still quite a lot of work around the platform that needs to still be focused on and so we have dedicated teams for that as well.

[Anna Lee]: You know, we had introduced the concept of chapter.

[Anna Lee]: We probably haven't done as much as we'd like.

[Anna Lee]: That's something that we just need to kind of resume.

[Anna Lee]: And we feel that's right thing to do.

[Anna Lee]: I think conceptually, you can kind of go that that feels right.

[Anna Lee]: But in practice, sometimes you've also got to be adaptable and say, OK, maybe right now that that didn't need the focus, but now kind of 18 months on from.

[Anna Lee]: the reset were probably in a right place to elevate that role and that type of framework could work.

[Anna Lee]: So I think the answer is that there is just no right or wrong answer.

[Anna Lee]: You've really got to understand where the business is at and meet it there to form the right structure.

[Anna Lee]: I also think that I really hate reporting lines.

[Anna Lee]: Like I just think [Anna Lee]: Great businesses just it should just be spiderwebs like I the way I describe them is spiderwebs Because there should be relationships that transcend [Anna Lee]: who you report to, who your boss is, like that stuff just exacerbates old school silo thinking.

[Anna Lee]: And of course, there is a role of your direct leader, your direct leader plays such an important role that your people leader, they're there to ensure you're constantly developed, your workload is balanced and so on.

[Anna Lee]: But the relationships that exist within the business that really succeed are a spider web.

[Anna Lee]: So people should be able to talk and connect with anyone [Anna Lee]: in the business in order to get what they need and jam it out.

[Anna Lee]: So I know that you wanted a white answer.

[Anna Lee]: No, no.

[Anna Lee]: I don't think there is a right answer.

[Michael Helbling]: I think an absolutist would be like, no, there's got to be, but I actually completely agree.

[Michael Helbling]: the structure and function of the company mandates or requires a response just like you've described.

[Michael Helbling]: No, I actually totally love the answer.

[Michael Helbling]: Thank you for confirming what I already think, but you say it in such a great way.

[Michael Helbling]: Yeah, it's awesome.

[Moe Kiss]: Thank you.

[Moe Kiss]: I'm keen to leverage your amazing brain as much as possible for advice for those listening.

[Moe Kiss]: You've talked a lot about what are the great qualities and the culture that we want to see in data.

[Moe Kiss]: I think one of the realities is that data folks can lose trust and credibility very quickly with a couple of bad interactions.

[Moe Kiss]: I'm curious to hear.

[Moe Kiss]: and definitely not pointing fingers, but like, where have you seen things go wrong?

[Moe Kiss]: Or like, I feel like it's such a skill for data folks to learn that, how to do that so what, especially for an exec audience.

[Moe Kiss]: And sometimes that takes time and sometimes there's missteps along the way, but like, what are some kind of things that folks should avoid, particularly when they're presenting or like communicating results to senior leadership to kind of keep that trust?

[Anna Lee]: Well, I might break that into two ways to answer that.

[Anna Lee]: So first of all, the first thing is what we discussed a couple of topics ago, which is around that.

[Anna Lee]: So what?

[Anna Lee]: So I think one of the hardest things is [Anna Lee]: letting go of all of the wonderful data because sometimes to get those insights man you have like dug dug dug like there's just mining information and so there is this desire to almost showcase like oh look at all the work that I've done but actually most senior stakeholders just like that's great actually we trust that [Anna Lee]: you know, because that's why you're in that job.

[Anna Lee]: So really being clear about the so what so you're going to present something like what is the killer point?

[Anna Lee]: That's why I say to people like what's the killer message here?

[Anna Lee]: What's the wow moment that you're trying to [Anna Lee]: produce or convey or influence.

[Anna Lee]: So it might look like.

[Anna Lee]: So we always thought that this was the case and that that data has previously shown that.

[Anna Lee]: But I've been monitoring this metric and I actually noticed this other thing that's come up.

[Anna Lee]: And as a result, I've kind of [Anna Lee]: you know, combine that with that particular insight.

[Anna Lee]: And actually what I'm saying is perhaps what we need to consider and have a conversation with, you know, say partner X is, would they consider doing this?

[Anna Lee]: Because if we did that, but yield a different, you know, activation rate or whatever and that would drive, you know, really connecting the work that you've done with the strategy and whether something actually could be changed in order to drive a different outcome.

[Anna Lee]: I think [Anna Lee]: That's kind of more the technical side.

[Anna Lee]: And I think on the other side, I feel like I never had any of my team members present.

[Anna Lee]: to senior stakeholders, you know, specifically like the CEO.

[Anna Lee]: So when I was in CFO role, CEO role without a lot of coaching.

[Anna Lee]: Because I think a lot of people also just kind of, it's just, you're literally just like throwing someone in the deep end.

[Anna Lee]: Like if you don't, like if you've never engaged with the CEO before, you know, nothing about them.

[Anna Lee]: I think that's just, [Anna Lee]: I think that's poor management.

[Anna Lee]: Like for me, if I had, whenever I had a team member that was going to present to our CEO because you wanted to come along to the meeting and give them an opportunity to shine, you never did that without prepping them, rehearsing, giving them coaching around this person.

[Anna Lee]: They really like this.

[Anna Lee]: They're really particular about that.

[Anna Lee]: you know, this is how and also how we could play a role.

[Anna Lee]: So also, you know, it's almost like there's different roles, right?

[Anna Lee]: So you come in and let me open the conversation and talk to and reframe it such that there's context that allows the team member to observe because so much of what we learn is [Anna Lee]: on the job.

[Anna Lee]: So much what I learned was observing different behaviours and different things of different leaders that I really admired.

[Anna Lee]: So that allows our person to go, oh, I like how they've set up.

[Anna Lee]: I see how Anna set that up with and built that connection with the CEO.

[Anna Lee]: And then hand over and go, well, you know what, [Anna Lee]: you know, Jane's role is this, right?

[Anna Lee]: And then it kind of really sets the context and it allows that person to kind of focus on what they're going to talk to without having the pressure of like, oh, God, I don't know if you know this, but you know, and this is [Anna Lee]: There's quite a lot of energy wasted on setting up the context when actually it's just quicker if I just set that up and just pass it on.

[Anna Lee]: So I also think if you are an emerging leader in data analytics, I think don't be afraid to ask for coaching and asking for [Anna Lee]: What should I do?

[Anna Lee]: And also, could you give me some feedback afterwards?

[Anna Lee]: So because those interactions that I did that were coached afterwards, there was just an open dialogue to go, oh, I think you did great here.

[Anna Lee]: This was a curly question.

[Anna Lee]: We probably didn't expect that at the CEO.

[Anna Lee]: So that's OK.

[Anna Lee]: And how would we manage that next time?

[Anna Lee]: Invite the person to say, what did you think of that?

[Anna Lee]: That creates just a really safe culture.

[Anna Lee]: There's also, by the way, as the people lead it, it's probably really smart to also say to the CEO, hey, I'm going to bring so-and-so in.

[Anna Lee]: It's a development area for them.

[Anna Lee]: like ask your questions like you would because I don't want you to be a different person, but also just manage your expectations on what this person is because everyone's done something for the first time and the first time that they do anything, you know, it's always going to be [Anna Lee]: a bit challenging and there's going to be lots of different nerves and emotions and lots of different expectations that will culminate.

[Anna Lee]: But they become big deals for those people because they're like, wow, I got to present to senior stakeholders.

[Anna Lee]: So I think there's also a job of the people leader to kind of go, hey, you know, and just set the space up properly as well.

[Val Kroll]: That's a good one.

[Val Kroll]: I'm going to sneak in one more, Michael.

[Val Kroll]: You've got that look in your eye.

[Val Kroll]: I don't think it's going to be too bad.

[Val Kroll]: I want to ask you this question in particular because you've had the CFO, the COO experience before, being in the COO, and I heard you talk earlier about the investment in the operations analyst, like paid for itself in six months.

[Val Kroll]: You talked about the investment at Flyby's in the platform and some of the data collection pieces.

[Val Kroll]: One of the things that some people in analytics, and I've spent some time more recently in experimentation struggle with is, how do I communicate the story of value to senior stakeholders?

[Val Kroll]: And one thing that we come back to a lot is, how do I quantify the value of a learning?

[Val Kroll]: So even if it wasn't directly like, this is now the decision, there's a fork in the road and we can [Val Kroll]: specifically tied the dollars to those decisions.

[Val Kroll]: How would you suggest someone talk about or what are some of the ways that we could illustrate the value of helping kind of raise the institutional like learnings of the organization or for some things that are a little bit more hard, a little bit more intangible, but obviously are moving things forward.

[Val Kroll]: I'm curious how you would like to hear some of that or would be the best ways to illustrate it.

[Anna Lee]: Oh, that's such a, that's you've saved the last.

[Anna Lee]: Oh, did I?

[Anna Lee]: Okay.

[Anna Lee]: Sorry, go away.

[Anna Lee]: That's like, no, no, it's a good one to ask.

[Anna Lee]: I like a tough question.

[Anna Lee]: Look, value is, I mean, I don't know if we've got it right ourselves.

[Anna Lee]: So I'm certainly not going to talk to this as an expert at all.

[Anna Lee]: I think we are.

[Anna Lee]: we're productively curious about that question ourselves.

[Anna Lee]: But I think over my years as in finance, I think, you know, calculating something like an ROI or what, you know, the perceived benefits are on things.

[Anna Lee]: One thing I have learned is there's no, again, there's no, you know, one, one rule, size fits all for, for everything, because there are some things that are just going to be really easily tangible, like, you know, very classic, [Anna Lee]: ROI measures, measure tangible savings from investments and decisions, very, very tangible.

[Anna Lee]: Then you will have ones that have much slower payback.

[Anna Lee]: I think an example would be educating this impact of [Anna Lee]: the data platform and being able to lay the foundations and so on.

[Anna Lee]: I think that is a narrative around enablement.

[Anna Lee]: So, you know, in order for a lot of these other things that are in our strategy to actually be delivered, we will need to measure, you know, these metrics or otherwise such that, you know, and we need to be able to do those very, very quickly and very easily and therefore [Anna Lee]: It increases the speed at which we can deliver or otherwise again.

[Anna Lee]: little bit tricky because, you know, sometimes someone would say that to me and I'm just like, that is all fluff, give me something more tangible.

[Anna Lee]: So obviously I'm not going to say that I would be.

[Anna Lee]: But then there are some times that I just go, yeah, yeah, no, that's okay.

[Anna Lee]: That sounds plausible.

[Anna Lee]: And I appreciate that we can't quantify that for now.

[Anna Lee]: But you just know, and I think that's, I think coming back to what we talked about earlier around [Anna Lee]: Not everything needs to be precise and completely evidence-based.

[Anna Lee]: We're not here on a criminal kind of case where the burden of evidence is so, so high.

[Anna Lee]: Sometimes it's just about being able to complement those hunches or otherwise with other information that could still [Anna Lee]: you know, sound good and sometimes the trade off is, yeah, I'm going to take a risk there.

[Anna Lee]: And yeah, let's let's let's put on another couple of people because you know, we think that that's that's probably going to yield some really great outcomes that we just limited in capacity now.

[Anna Lee]: So I think it just some of it is just about experience and expertise.

[Anna Lee]: And I've been in working for over 30 years.

[Anna Lee]: So sometimes there are things that you kind of just know intuitively and [Anna Lee]: And you just have to kind of take a gamble sometimes.

[Anna Lee]: So it's just moderated risk-taking in a way that kind of feels like, well, what's the worst that can happen here?

[Anna Lee]: Not really a lot, frankly.

[Anna Lee]: Like we're not talking about [Anna Lee]: you know, a multi-million dollar investment here.

[Anna Lee]: We're talking about, you know, some certain roles where we can be very clear and see.

[Anna Lee]: And I think that's okay.

[Anna Lee]: There are some measured things that you can take risks on.

[Michael Helbling]: Yeah, that's great.

[Michael Helbling]: Okay, we do have to start to wrap up.

[Michael Helbling]: This has been such an amazing conversation, Anna.

[Michael Helbling]: Thank you so much.

[Michael Helbling]: I literally have like a bunch of notes on my desk here, especially about productive curiosity and informed pragmatism.

[Michael Helbling]: I think those are two amazing concepts.

[Michael Helbling]: So thank you so much.

[Michael Helbling]: But the whole conversation was amazing.

[Michael Helbling]: And before we get into last calls, I'm going to jump over to take a quick break [Michael Helbling]: With our friend Michael Kaminski from ReCast, the media mix modeling and GLIF platform that helps teams forecast accurately and make better decisions.

[Michael Helbling]: And Michael's been sharing some bite-sized marketing science lessons over these next couple of months to help you measure smarter.

[Michael Helbling]: So over to you.

[Michael Kaminsky (Recast)]: We're all interested in doing good causal inference work, so let's talk about its fundamental problem.

[Michael Kaminsky (Recast)]: Cause and effect are such a simple relationship, and yet formal scientific causal inferences almost impossibly difficult.

[Michael Kaminsky (Recast)]: At its core lies what philosophers call the fundamental problem of causal inference.

[Michael Kaminsky (Recast)]: we can never observe what would have happened to the same person at the same time under different conditions.

[Michael Kaminsky (Recast)]: So we always have the lingering doubt that the medicine make the patient better, or would you have gotten better on our own?

[Michael Kaminsky (Recast)]: In order to estimate the causal effect of some intervention statistically, we would have to control for every variable that could possibly confound the analysis.

[Michael Kaminsky (Recast)]: The problem is that in any individual there are unobserved or unmeasurable characteristics that make us unique and are impossible to statistically control for.

[Michael Kaminsky (Recast)]: This makes observational causal inference extremely difficult and is the main reason why you can read a new study every week about how coffee both increases and reduces life expectancy.

[Michael Kaminsky (Recast)]: Our inability to statistically control for the unobserved or unmeasurable differences in individuals experiencing some stimulus forces us to turn to randomized experimentation to differentiate between correlation and causation.

[Michael Kaminsky (Recast)]: Because controlling for everything that we can't observe is quite literally impossible.

[Michael Helbling]: All right, if you enjoyed that many lesson, Michael and the team at ReCast put together a library of marketing science content specifically for analytics power hour listeners.

[Michael Helbling]: From everything from media mix models in-house to communicating uncertainty to your board, head over to www.getrecast.com slash aph.

[Michael Helbling]: That's www.getrecast.com slash aph.

[Michael Helbling]: Okay.

[Michael Helbling]: Well, one thing we love to do is go around the horn, share something about your interest to our listeners that's called our last call.

[Michael Helbling]: Inna, you're our guest.

[Michael Helbling]: Do you have a last call you'd like to share?

[Anna Lee]: Well, I am a huge James Clear fan.

[Anna Lee]: I absolutely love his book Atomic Habits.

[Anna Lee]: And one of my favorite quotes for him, which I think is a really good counterbalance to all the evidence conversation that we had today, is many situations in life are similar to going on a hike.

[Anna Lee]: The view changes once you start walking.

[Anna Lee]: And I really love that because [Anna Lee]: I think often we find ourselves under pressure to have all the answers and go, well, we don't want to take action until we have everything that we need, perfectly validated and so on.

[Anna Lee]: And I'm of the view is just take action and it'll just start to work itself out.

[Anna Lee]: And there are some things that actually you don't know the answer until you actually start taking action because some of those things actually don't happen until they depend on the things that you've taken action on.

[Anna Lee]: So I love the fact that the answers will just reveal themselves.

[Anna Lee]: That's kind of a nice saying.

[Michael Helbling]: That is amazing.

[Anna Lee]: Michael's got a big smile.

[Michael Helbling]: All right.

[Michael Helbling]: I do.

[Michael Helbling]: Well, because there's a phrase that was really important in my life that's very similar, which is you can't steer a parked car, which is sort of in the same vein, but not exactly the same.

[Michael Helbling]: I liked the idea of like taking the hike and watching the view change.

[Michael Helbling]: So yeah, anyways, that's really cool.

[Michael Helbling]: Okay, Val, what about you?

[Michael Helbling]: What's your last call?

[Val Kroll]: OK, so mine is a medium article that was published on the UX collective.

[Val Kroll]: And never had I had a last call that was tied in more to an episode.

[Val Kroll]: I'm so proud of myself.

[Val Kroll]: But this one's called, There's Logic Behind Your Gut Feeling.

[Val Kroll]: And so this is all about abduction.

[Val Kroll]: It's about Charles Perce.

[Val Kroll]: And so I'll read you this one, a little two sentence thing.

[Val Kroll]: So he believed that truth wasn't something that was handed down.

[Val Kroll]: It was something that you worked towards.

[Val Kroll]: And the way you got there wasn't by doubling down.

[Val Kroll]: It was by willing to be wrong.

[Val Kroll]: For Perse, doubt wasn't a weakness.

[Val Kroll]: It was the start of real thought.

[Val Kroll]: And so it breaks down like all these different types of thinking and reasoning.

[Val Kroll]: And he actually at the end ties it into why it's really important for the way that we're incorporating generative AI into our work.

[Val Kroll]: And so Michael, I know you'll love that tie in too.

[Val Kroll]: But anyways, I found it a good read and it's nice and concise.

[Val Kroll]: So that's a good one.

[Michael Helbling]: All right, Moe, what about you?

[Michael Helbling]: What's your last call?

[Moe Kiss]: I have been thinking a lot about how AI summaries changes search behavior and folks coming to sites and how that's evolving.

[Moe Kiss]: The Pew Research Center just put out a really interesting article on that.

[Moe Kiss]: And it was specifically about Google users.

[Moe Kiss]: And it was showing that Google users are less likely to click on links when an AI summary appears in the results.

[Moe Kiss]: So a lot of us would have noticed that happening now in our Google search results.

[Moe Kiss]: and just has a lot of flow-on effects also, like I said, for how users come to our site, like the volume of traffic, the quality of traffic.

[Moe Kiss]: So it's definitely a topic that I've been getting around.

[Moe Kiss]: It's a really good kind of quick summary if you want to get some info on the topic.

[Moe Kiss]: So I found that one really interesting.

[Michael Helbling]: That's awesome.

[Moe Kiss]: What about you, Michael?

[Michael Helbling]: Oh, well, I'm glad you asked.

[Michael Helbling]: So I read an article recently, and it's also related to AI, but I love a good contrarian take, and this is one.

[Michael Helbling]: Nancy Dooley, who I think is a listener, but also a good friend of the show, she sent me an article called The Hater's Guide to the AI Bubble.

[Michael Helbling]: by Ed Zittron.

[Michael Helbling]: And what's great about it is the author does an excellent job driving through sort of a financial analysis of the AI industry and sort of some lack of sustainability inherent in that he sees in it.

[Michael Helbling]: And I thought it was actually quite well researched and done.

[Michael Helbling]: I'm still pretty excited about AI and where AI is headed, but I really enjoyed reading and thinking, you know, it was provocative to think about that.

[Michael Helbling]: So, [Michael Helbling]: And all these things that we just shared as last calls, we always put the links to those items and we'll put a link to that James Clear quote in that book.

[Michael Helbling]: All of those will be in our show notes on the website.

[Michael Helbling]: And as you're listening and you want to reach out, we'd love to hear from you.

[Michael Helbling]: The best way to do that is [Michael Helbling]: uh through our website which i just mentioned analytics hour dot io you can reach out to us that way or through our linkedin group or um on the measure slack chat group so we'd love to hear from you feedback about the show and things like that or topics that you'd like to see discussed [Michael Helbling]: That's a great way to do it.

[Michael Helbling]: So as you've been listening, please reach out.

[Michael Helbling]: Anna, once again, what a pleasure.

[Michael Helbling]: Thank you so much for taking your time.

[Michael Helbling]: It's something we in the data space, we always want executive time and feel like we don't get it.

[Michael Helbling]: And so the chance to spend some time and talk about these issues and hear from you is something that's not just valuable for us, but I think it's really valuable for our listeners.

[Michael Helbling]: So really appreciate it.

[Anna Lee]: Oh, thank you so much.

[Anna Lee]: It's my pleasure.

[Anna Lee]: And you guys are such fun coyotes.

[Michael Helbling]: We try.

[Michael Helbling]: We try.

[Michael Helbling]: But thank you.

[Michael Helbling]: We'll take that.

[Michael Helbling]: We'll take that.

[Michael Helbling]: Anyways, no matter where you are in your progression, whether you're presenting daily to the CEO or you're just starting out as an analyst and trying to get coaching to move up the ladder, I know I speak for both of my co-hosts, Moe and Val, when I say, keep analyzing.

[Announcer]: Thanks for listening.

[Announcer]: Let's keep the conversation going with your comments, suggestions, and questions on Twitter at @analyticshour on the web at analyticshour.io, our LinkedIn group, and the Measure Chat Slack group.

[Announcer]: Music for the podcast by Josh Crowhurst.

[Announcer]: Those smart guys wanted to fit in, so they made up a term called analytics.

[Announcer]: Analytics don't work.

[Michael Wilbon]: Do the analytics say go for it, no matter who's going for it?

[Michael Wilbon]: So if you and I were on the field, the analytics say go for it.

[Michael Wilbon]: It's the stupidest, laziest, lamest thing I've ever heard for reasoning in competition.

[Michael Helbling]: No, but it's like the best.

[Michael Helbling]: It's sort of the best because you put it in your mouth.

[Michael Helbling]: You can make it your own mouth.

[Michael Helbling]: Like, don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Moe Kiss]: Don't put it in your mouth.

[Moe Kiss]: Don't put it in your mouth.

[Moe Kiss]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Kaminsky (Recast)]: Don't put it in your mouth.

[Michael Kaminsky (Recast)]: Don't put it in your mouth.

[Michael Kaminsky (Recast)]: Don't put it in your mouth.

[Michael Kaminsky (Recast)]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth.

Don't put it in your mouth.

[Michael Helbling]: Don't put it in your mouth

[Moe Kiss]: I honestly wasn't so and at the end of the show someone always has to do that embarrassing thing and it is almost never me and so I assumed it wasn't me it's like yeah it's an inside joke but I was like it's not going to be me and then in the last one [Moe Kiss]: someone really down to do it.

[Moe Kiss]: So now I've got to do it.

[Michael Helbling]: So are we leaving?

[Michael Helbling]: Yes, we are.

[Michael Helbling]: I think now we're committed because all of the outtakes are going to be this next year.

[Val Kroll]: And now I have to tell a quick story.

[Val Kroll]: So when I worked at UBS in the investment bank, I was in New York.

[Michael Helbling]: Do you want to stop the recording so we can just make sure people get uploaded?

[Val Kroll]: Yes, although I would love to hear Moe and Hannah's reaction about yes, we can still.

[Moe Kiss]: Rock flag and pragmatic curiosity.

[Moe Kiss]: Good one.

[Moe Kiss]: Is that the actual saying?

[Michael Helbling]: No, it's productive curiosity.

[Moe Kiss]: Fuck!

[Moe Kiss]: All right, let me do it again.

[Moe Kiss]: Fuck!