4 types of untrustworthy data

There are a number of reasons why organizations struggle to adopt data-first strategies. A lack of resources, tools, direction, and the right data-driven mindset are all blockers to getting strategies off the ground. Even if you get all these things right, one culprit can bring the whole thing down: untrustworthy data.

When data isn’t trustworthy, teams use it less. When teams use data less, it gets deprioritized and grows stale. When data grows stale, it becomes less trustworthy—and the cycle continues.

But what does ‘untrustworthy data’ actually mean? This post explores the 4 main types of untrustworthy data and how they relate to behavioral product data (that is, data that tells you about users and the actions they take on a website or application).

1. Stale data

Stale data refers to data that is out of date and no longer being collected.

Stale data happens for a number of reasons, such as

• The tracking event used to capture the data is linked to an old version of a feature that’s been deprecated

• The tracking code is broken

• You’re looking in the wrong place (this is particularly common if different teams are using different analytics tools for tracking and you don’t have a single source of truth for product analytics data)

Across all of these scenarios, the root cause of stale data is a lack of strategy, prioritization, and alignment when it comes to tracking product data. This results in zero maintenance of old tracking events and zero effort put into updating a tracking plan as the product evolves. 

Stale data is particularly common if your product analytics tool relies on manual implementation (instead of automatically capturing all events in real time). This traps analytics teams in a reactive cycle, where they’re constantly implementing tracking for new features and use cases, leaving little time to clean data and ensure its high quality.

2. Unclear data

Raise your hand if you’ve dealt with an analytics environment that has multiple versions of the same data point. Something like ‘Signup’, ‘Sign Up’, ‘signup’, and ‘Signup – NEW’ are all listed as separate events, but they all seem to refer to the same thing.

In this case, you’re left wondering: which is the correct one? Are the others ‘right’, but tell a different version of the story? Even if you know which version is correct, how can you be sure your teammates and other stakeholders do? 

Unclear data is common in product analytics, and it leads to 2 negative outcomes:

• In one scenario, a user ends up analyzing an event that doesn’t tell them what they think it does, which could lead to an important decision being made with incorrect data

• On the other hand, the user may choose the ‘right’ event, but only those close to the implementation typically have enough confidence in the event to actually use it. For everyone else, trust in the entire dataset is eroded, even if everything else is correct.

Just like with stale data, this is very hard to avoid when events are manually instrumented. A rigid, code-generated dataset frequently leads to unclear event data, because the implementation of new events is managed through code, and only by a select few. 

Any inconsistency, duplication, or poor naming convention starts as a quick decision, a simple mistake, or someone saying ‘good enough’—but the resulting problems grow in scope and sneakily infect your entire analytics strategy over time.

3. Inaccurate data

Inaccurate data comes in 2 varieties:

• Data that is inaccurate and you know it

• Data that is inaccurate, but you have no idea

Both varieties lead to the same negative outcomes as the ‘unclear data’ problem above: a decrease in trust and faulty conclusions based on misleading data.

Let’s start with the first type of inaccurate data. This is when you just know something is wrong, like if your report tells you only 30 people viewed your homepage last month.

It’s not only hard to quantify the depth of these inaccuracies (“Are these numbers just a little bit off or completely wrong?”), but it’s also hard to troubleshoot and fix them, especially when events are created by a small team of engineers operating independently.

The second type of inaccurate data is especially problematic for businesses. When data is inaccurate but you treat it as correct, it can lead you to make feature, product, or even business decisions based on false assumptions.

Say you’re performing conversion rate optimization (CRO) on your website. Your data shows that the call to action (CTA) at the bottom of your homepage was outperforming the one at the top of the page, so you decided to remove the top one.

You never find out, but the inverse was actually true—the top CTA was more effective than the second one. Maybe the event names got mixed up during implementation, or maybe the tracking code on the top CTA was flawed, and not every click was logged. 

Either way, your inaccurate data caused you to remove your highest-performing CTA, potentially impacting your overall conversion rate. And because you were doing everything right—analyzing user behavior, testing hypotheses, making data-driven decisions—you’ll probably never even know what went wrong.

4. No data

One of the most common challenges that teams run into when faced with a business question is simply not having the data required to answer it.

Too many times, an analytics implementation is treated as a one-time project with an end date. With this mindset, it’s common to not have processes in place for ongoing maintenance and adding new tracking events when gaps in the dataset inevitably pop up. 

Instead, teams are stuck in a reactive cycle, endlessly adding new events at the request of various disparate teams with different goals and KPIs. Once the new events are implemented, data still needs to build up, and by the time the question can be answered, it might not even be relevant anymore.

At this point, you might even have a new set of questions that can’t be answered—and the cycle continues.

How a real-time product analytics platform addresses these challenges

Some companies attempt to solve these problems by throwing lots of money, time, and resources at their manual, hard-coded datasets to try to keep them up to date. But there’s a more efficient way.

Forward-thinking teams use an all-in-one product analytics platform to give them a real-time virtual dataset, capturing all event data upfront—without any manual tracking code. That way, they can pick and choose which events they’d like to analyze later on. 

Product analytics tools offer a range of benefits for companies of all sizes:

• They require very few resources to set up and maintain

• They track a wealth of user behavior and product analytics data

• They provide real-time insights that let you spot issues and identify opportunities fast

• Data is available retroactively, removing the need to wait for it to build up before being able to answer questions

All of this means you get valuable, reliable data that lets you quickly improve digital customer experiences and get real results, fast.

Make data-driven business decisions with Contentsquare

If you’re ready to say goodbye to untrustworthy data for good, it’s time to say hello to Contentsquare’s Product Analytics product. Monitor, analyze, and optimize product performance using qualitative and quantitative user insights, and ensure you always make decisions based on clean, reliable data.