What do Microsoft, Netflix, and Electronic Arts have in common? They’ve conducted digital experiments to understand their customer journeys and digital behavior better, improving their conversion and add-to-cart rates.
As Asia’s Project Lead for Microsoft’s Online Store, Yvonne Cheung understands the value of making digital experimentation the core of any business.
In her session at CX Circle APAC, she shared why digital experimentation matters for business growth, tips to make testing part of your company culture, and how digital analytics tools like Contentsquare help find opportunities.
TABLE OF CONTENTS
Getting started with digital experimentation
**3-step approach to building a culture of digital experimentation**
The digital dilemma: How do I know if I’m optimizing correctly?
Brands are in a dilemma. Optimizing high-intent touchpoints is vital to boost conversions, delight customers, and adapt to consumer behavior patterns.
But there’s risk involved in rolling out site-wide changes without validating if they’ll improve your core experience metrics.
For Yvonne, experimentation supports your decisions and reduces change risks. But starting a new culture isn’t easy.
Getting started with digital experimentation
Start with getting management buy-in and a dedicated team for digital experimentation.
Yvonne recommends showing the management team what experimentation can do by focusing on the potential upsides. When the correct answers aren’t as clear-cut, testing helps you get a definite answer.
Once you’ve got support from management, form a group devoted to digital experimentation. “It’s best if the unit is cross-functional. Involve people with complementary skill sets. You also want people who know analytics,” says Yvonne.
Designate one team member as the facilitator. This person should ideally:
Understand the company’s analytics tools
Has experience facilitating workshops
Can encourage critical thinking among the team
Agree on a regular meeting rhythm. Consider making it biweekly, or meet at least once a month. This step is crucial, as meeting regularly keeps ideas fresh, builds momentum, and ensures consistent progress.
Having defined your team and rhythm, it’s time to proceed to execution.
3-step approach to building a digital experimentation process
These 3 steps will help you foster a more sustainable approach to experimentation.
Step 1: Pre-test preparation
The path to purchase is no longer linear. According to Shopify, up to 73% of shoppers use multiple devices and touchpoints while researching an online purchase decision. Flip-flop brand Havaianas, found that customers swapped between online and offline channels leading up to a purchase.
That’s why in the first phase of digital experimentation, Yvonne says to focus on examining critical touchpoints, scoping potential issues, and prioritizing issues to work on.
1. Find key customer touchpoints across your homepage
Consider the entire customer journey, especially looking at high conversion points. For example, Contentsquare’s Customer Journey Analysis tool can help aggregate data across all your customer journeys, seeing which pages your visitors navigate to and where they might be dropping off.
2. Brainstorm pain points and identify key metrics using web analytics tools
Start gathering performance metrics of your customer touchpoints with free tools, like Adobe Analytics and Google Analytics. These tools are excellent to tell you what’s going on on your website or app but fall short of explaining why visitors behave the way they do.
For more in-depth insights, Yvonne’s team relies on Contentsquare’s Zoning Analysis to see how web visitors interact with every element of their website. Zone-based heatmaps transform the website into an interactive dashboard, allowing her team to view unique behavioral metrics like hesitation time, scroll rate, the time before the first click, and more to identify how visitors are interacting with the page. The team uses these visual insights to identify potential issues and define problem statements for experiments.
From here, they can:
Analyze these numbers to identify potential pain points
Compare the metrics to your key performance indicators (KPIs), paying attention to underperforming metrics.
3. Prioritize issues to address
Yvonne’s put together a sample framework to prioritize experiments based on the data gathered in the previous step. Her recommendation? Start simple.
First, rank your touchpoints by gathering visitor metrics and ranking page URLs by revenue participation. You’ll come up with experiment ideas by thinking of problem statements and which metrics can be improved.
Feel free to introduce other prioritization criteria and scoring metrics like the ease of testing or urgency when you’ve got a more mature team and experimentation culture.
Step 2: Test and learn – designing experiments that work
To Yvonne, a well-designed experiment consists of a problem statement, a hypothesis with trackable KPIs, and suitable experiment variants.
First, identify the problem you’re investigating. Yvonne uses Contentsquare’s Zoning Analysis and Session Replay to quickly gather visual insights to validate potential problem statements. These tools can show if visitors can’t read the content easily or how much content on a page they interact with.
Next, use these insights to form a hypothesis or investigate other issues. If you’re stuck, Yvonne suggests using the if, then, because format to write a clear hypothesis. Assign a primary and secondary KPI affected by the experiment to track results.
Once done, think about identifying your control and experiment variants.
Besides relying on your creativity, Yvonne has three more ways to find ideas for an experiment control or variant:
Compare against industry best practices
Look at what other e-commerce players within or outside your industry are doing
Ask your UI/UX team for feedback and other ideas
Step 3: Maintain momentum
What’s next after completing your first experiment? Maintaining momentum in digital experimentation can be challenging. After all, how do you keep the team motivated and demonstrate that digital experimentation is worth investing in?
Yvonne’s advice? Don’t stop after your first experiment.
She explains: “Experimentation doesn’t always mean proposing a radical redesign of a page. Consistently iterating over time is the key to success. Small changes compound over time.”
Yvonne’s 3 key takeaways to building a culture of digital experimentation
1. Get the buy-in of your team members and senior management: Building a new culture requires the entire company’s support. Present your asks by focusing on the potential upsides of investing in experimentation. For example, experimentation helps validate decisions, systematically drive improvements in your digital channels, and reduce change risks.
2. Not all pages are created equal: Prioritize experiments and issues to work on for maximum impact. Use analytics tools to identify potential problems to work on. Use Yvonne’s prioritization framework to help you prioritize and design experiments effectively
3. Small changes over time compound: You don’t need radical redesigns to benefit from experimentation. Keep changes small and iterate incrementally over time to drive results.
Our technology partners provide tools that integrate with and extend the Contentsquare platform to help optimize your experimentation strategy. Check out our experimentation and A/B testing partners like Optimizely, Kibo, AB Tasty, Dynamic Yield and Kameloon here.
Fernando is Product Marketing Director for Verticals at Contesquare. With extensive experience in Product Management in different industries (FSI, T&H, Power Utilities, Public Sector), he’s passionate about the impact that traditional sectors have on our daily lives. Fernando is a firm believer that digital democratizes the access and scalability of the services that create a better and more inclusive society. He joined Contentsquare to empower our vertical customers in making that happen.