DPG Media x Contentsquare

How DPG Media used Contentsquare to increase their A/B test win rate by 22%

The company

DPG Media Group is one of Europe's largest media companies, operating in the Netherlands, Belgium, and Denmark.

The company carries over 80 different brands in television, radio, newspaper, magazines and online services. It serves over 15 million viewers, readers, and listeners daily.

The challenge

Understanding exactly what consumers want and giving it to them is paramount to any company's success. For companies like DPG Media, this entails delivering highly engaging content in the most user-friendly, customer-centric way possible.

With fierce competition for attention across various media platforms, media organizations are obliged to offer frictionless customer journeys that retain audience engagement and turn consumers into subscribers.

The experimentation team at DPG Media is focused on testing optimizations to provide the best possible customer experience and drive conversions for the different media brands of the group.

As a significant amount of developer capacities and resources are allocated to testing, increasing the efficiency and win rate of the team's A/B testing efforts is a key priority.

Contentsquare enables the team to

Conduct detailed site analysis prior to the tests, resulting in better hypotheses
Quickly analyze changes that need to be made during the tests using side-by-side heatmap analysis
Get in-depth insights into test results to better understand these and fuel the next round of tests

The solution

Before implementing Contentsquare, the experimentation team relied on web analytics from Snowplow. This meant they lacked comprehensive insights into customer behavior, which made it difficult (and sometimes even impossible) to understand why their A/B tests failed.

With Contentsquare, the team can strategize and prioritize tests effectively, resulting in a significant boost (+22%) to their A/B test win rates.

Helping DPG Media's team use the Triple Diamond Framework

To self problems effectively, the DPG team relies on the Triple Diamond Framework, which comprises three phases.

Firstly, the problem discovery phase serves to understand the root causes of issues and identify the right problems to address. Here, Contentsquare assists the team in uncovering friction points, errors, and significant user behaviors on the website, and helps them to prioritize issues based on their impact.

In the solution discovery phase, the team formulates hypotheses for potential solutions based on previous analyses.

Finally, in the prototype or 'go live' phase, prototypes undergo A/B testing on the page, with performance closely monitored via Contentsquare's side-by-side heatmap analysis. Once the winning version is implemented, Contentsquare continues to be instrumental in analyzing ongoing performance.

Improving conversions on the company's subscription page using A/B testing

Getting visitors to sign up for subscriptions is a key conversion goal for DPG Media—and driving visitors from main content pages (such as newspaper articles) to subscription pages represents a great opportunity to convert them.

The DPG Media experimentation team decided to investigate how people signed up for subscriptions on the website of the Dutch morning paper "de Volkskrant", and specifically focused on one user segment: users who clicked on ads placed on the website to promote their own subscription packages.

The team analyzed how those users were behaving on the subscription page to uncover any potential optimization potential.

Screenshot from Journey Analysis

Using Journey Analysis, the experimentation team discovered that, while the ads had driven significant traffic to the subscription page, 79% of visitors had exited the page, and only 8% of users had progressed to the checkout.

Furthermore, traffic analysis showed that the majority of ad users had accessed the site via mobile devices—and that the conversion rates of these mobile users lagged behind those of desktop users.

Screenshot from Page Comparison, showing that a higher percentage of non-ad users were exposed to the company's digital subscription offer than ad users

Hoping to discover what lay behind this low engagement, the team used Contentsquare's Page Comparison tool to compared the behavior of ad users with that of other visitors on the subscription page.

The team's analysis revealed that ad users scrolled less than other visitors, spent 60% less time on the page, had higher exit rates, and saw significantly lower conversion rates.

With the help of heatmap analysis, the team discovered a potential culprit behind these dispiriting metrics: physical newspaper subscriptions (which were positioned at the top of the page) were overshadowing digital subscription offerings (which were placed further down).

This was likely to be hurting engagement and conversion for two reasons:

The majority of ad users were on mobile, and could be expected to be naturally more interested in digital subscriptions than desktop users—however, since ad users tended not to scroll far down the page, many of these users may not have even seen the digital subscription offers
The ads on the website promoted the low price of the basic digital subscription. However, when users landed on the page they were being presented with a more expensive paper subscription first, with the advertised price located only at the bottom of the page

DPG Media has a strategic objective to serve its customers primarily through digital channels. However, the friction points uncovered by the team on the subscription page turned out to be a significant driver of calls into the company's contact center.

Heatmap analysis also revealed that ad users were more likely to contact customer support, with click rates on the contact button spiking 12 times higher than those of non-ad users.

These findings collectively suggested that users clicking on subscription ads were confused, as they were met with more expensive and irrelevant paper subscriptions above the fold.

Based on these insights, the team conducted an A/B test. They reorganized the product blocks to prioritize digital subscriptions at the top of the page, with the aim of reducing dropouts and increasing conversion rates.

When coming up with solutions to problems, I encourage my team to use the Triple Diamond Framework, which consists of 3 steps: problem discovery, solution discovery and prototype or go-live.

Contentsquare is crucial in all 3 phases. It allows us to really understand the problem, hypothesize potential solutions, and thoroughly analyze user behavior across testing variations with greater depth than ever before.

Plus, I believe it has significantly enhanced our learning process regarding experiment variations in user behavior, making those experiments ten times better for us."

Joshua Kreuger

Manager Experimentation, Subscription Sales
DPG Media

With Contentsquare's support, DPG Media now conducts more informed A/B tests, boasting a +22% higher win rate compared to tests without prior analysis.

By leveraging Contentsquare to uncover relevant insights before conducting A/B tests, DPG Media has maximized the efficiency of their testing processes, optimizing internal resources for greater effectiveness.

The results

The A/B test turned was a success and confirmed the team's hypothesis regarding the placement of subscription offers.

Conversions on newspaper subscriptions increased by +6.6% on the variants with the digital subscriptions moved to the top of the page, translating to a +7% increase in revenue. Similarly, magazine subscriptions saw an uptick of +8.7%, resulting in a +2% revenue boost.

The actionable insights provided by Contentsquare played an important role in validating these strategic changes internally. For a traditional media company like DPG Media, prioritizing digital subscriptions over newspaper offerings marked a significant leap forward.

Thanks to Contentsquare, we were able to streamline our A/B testing processes, increasing efficiency and ensuring that our web optimization strategies are backed by data-driven decision-making."