A Freewheeling Conversation About KPIs, Vanity Metrics and Leading Indicators

 

When it comes to marketing measurement, not all metrics are created equal. Some show progress. Some show promise. And some just show off. In this candid conversation, Jennifer Kohnhorst, Senior Director of Strategy at Rocket55, sits down with Director of Analytics Mark Van Cleve to unpack the nuances between types of metrics — and why they matter.

Jennifer Kohnhorst: Let’s start with the acronym. What’s a KPI, Mark?

Mark Van Cleve: KPIs are key performance indicators that are agreed upon – in our case between a client and an agency – and they’re used to measure the effectiveness of marketing efforts. Usually, they’re something along the lines of a form submission for a client whose goal is lead gen or a transaction for an e-commerce client. But really at the base of it, foundationally, I think a KPI is something that measures the success of our marketing efforts.

JK: And, I guess it’s important when we decide these things because it changes how we might measure performance, like what to track or how to set things up in the back-end for reporting on the website or other marketing tools?

MVC: Exactly.

JK: Sensible.

MVC: Since Google Analytics 4 is our primary source of truth, we need to know which events must be set up manually in Google Tag Manager or GA4 to ensure accurate tracking. GA4 refers to these as key events, but in some cases, accounts will begin tracking them automatically upon setup. But there’s a custom piece of that tracking setup that has to be in place based on those KPIs we agree on. And we want to make sure that we’re not inflating our true key event rate into something that is not accurate, because if there’s a default event and then we set up to track an event.

JK: It will count them twice?

MVC: Yeah, that’s why we go through the process of identifying what a KPI is with a client, we want to make sure that we’re not focusing on what you might call a vanity metric.

JK: That was a kind of hot button term I noticed when I first started here. It was important to distinguish that we don’t collect “vanity metrics.” What makes something a vanity metric?

MVC: This is anecdotally something that I used to see a lot more, but it would be tracking something like when a user lands on a contact page on a website rather than actually submitting a form. Okay, maybe five years ago it was pretty common to just track those as a conversion or a key event. But that’s already available to us in G4 by default. We can see how many people visit a contact page. So lumping that into a KPI is really just inflating the conversion rate and that can suggest things that aren’t necessarily true.

JK: I always thought vanity metrics were like traffic.

MVC: I mean I think traffic is more of a leading indicator. For example, say we’re doing SEO for a client. We have optimized a specific product page for certain keywords. We’re seeing improved ranking. That would be one sort of leading indicator and then as that translates into traffic again I would consider traffic a leading indicator, but it can be misused as a KPI.

JK: So the way that I always heard people talk about vanity metrics was sort of like, “oh that’s what other agencies will use to tell you that they’re being successful…” because they’re showing you these inflated numbers. You might be getting a ton of traffic but is it qualified? Are you ranking for the right terms? Vanity metrics might make you feel better, sure. But they don’t actually mean anything. That’s the vanity part.

MVC: Another example comes to mind. We had a storage facility client and they had a blog about how to store art that got a ton of traffic every month. And when we’re looking specifically at organic traffic and we see it rising, you’d think on the surface, hey, we’re doing a great job with our SEO. But actually, if you drill down and you find out that people are only going to this one blog page and they’re not actually interested in the services that you provide, what value is that blog to your overall marketing?

JK: Can that kind of traffic be harmful outside of just being deceptive? Is there any way that it actually can hurt your authority or anything like that?

MVC: That’s not necessarily a concern. In terms of tracking it’s more just an element of confusion because it obscures the real picture.

JK: So, hot take: I personally feel like sometimes people call out things as vanity metrics that aren’t actually vanity metrics. For example, in social people think of things like impressions as vanity metrics, but I think it can be pretty meaningful. I think maybe part of the challenge of it is like some of those impressions are important and they are going to reach the audience you want and some of them obviously aren’t, right?

I worry that those things get sort of discounted as not valuable at all and folks can become focused only on the final conversions or key events. But leading indicators really are important in identifying what the actions or behaviors that come before the key event or conversion actually are. Like, it’s not a vanity metric if it’s an indication of the effectiveness of actual tactics.

MVC: I think you have to go back to the point of your campaign and whether or not you’re out to spread awareness or you’re out to convert users. And that’s not always black and white. Especially with something like SEO or certain paid campaigns, impressions could easily be the KPI behind a campaign.

Say you have a new product that you’re launching that people just don’t know about, right? Why not use impressions as a KPI in that instance because you’re spreading awareness? But I think you need to be careful about when you begin and end those campaigns and about when you’ve saturated your user base with the messaging that you’re trying to spread.

JK: Yeah, that seems really smart. Oh, it made me think of something, from way back. Like I am like an old marketer.

MVC: You are not that old.

JK: I am ancient. So, back in the day before digital marketing was a thing, you didn’t have any measurement or attribution, really. So people would use that phrase “spray and pray”. There is a time and a place for: we’re just going to flood our target market with this awareness for this period of time but that shouldn’t be something that you use indefinitely. Spray and pray but not set and forget, I guess.

MVC: Words to live by. I don’t know that anyone would call it spray and pray anymore.

JK: I hope not. It’s kind of gross. I can’t help but think about spit.

MVC: I guess I think of vanity metrics as universally negative. Yeah.

JK: I think a vanity metric is any metric that isn’t actually meaningful, but is being reported on as if it is. I think that when you get in trouble is when you generalize and people conflate engagement metrics with vanity metrics.

MVC: That’s where leading indicators should fill that void.

JK: Just don’t spray and pray into the void.

Meandering metrics conversations aside, at Rocket55 we believe that measurement should clarify instead of confuse. If you’re in the mood for smarter tracking and sharper insights: let’s talk.