Conversion Rate Optimisation (CRO) UX Case Study
Converting an underperforming online store to a selling powerhouse
November 2023, Ongoing - Samsung Australia
Project Details
Duration
6 Months
Team
UX Designer
UX Specialist
Graduate UX/UI Designer (me)
Project Tags
Innovation
Web Design
CRO
Data
Project Context
Cheil is a digital agency owned by Samsung Group. During my time there as a graduate, I was on the conversion rate optimisation (CRO) team focusing on improving samsung.com/au. My role was user experience graduate where I would work closely with the team to research insights based on data, heuristic principles and competitors. With these insights, I would design modifications to the site, hand off designs to developers and then run an A/B test to determine if our hypothesise were accurate.
During my time as a graduate, we designed and shipped approximately 35 different A/B tests; this case study will focus on 2 tests.
Project Outcome
Throughout the entire process, we were constantly researching, designing and shipping A/B tests. Between the team of 3 UX designers, we would aim for about 12 insights per fortnight. These insights would then get workshopped, refined and approved to be presented to stakeholders. I personally, had the privilege of designing 12 tests that were approved by stakeholders which have either run, or are in the backlog to be run in the near future.
Highlights of moved metrics include:
18% Conversion rate increase on Product Filter pages through introducing a social proof framework into designs.
12% Increase in product card interactions by creating contrast between normal price and savings/offer.
Decreased bounce rate on discover mobile page by 5.5% by redesigning key information above the fold.
Part One: Background
1.1 How the team functions
Being a cyclical process, there is constantly work in motion. It was important for me to understand my place in the workflow so that I could consider my impact on the subsequent journey. We operated in fortnightly sprints to ensure that we were always learning and creating new insights to improve conversion on the site.
1.2 How do you understand millions of different users?
At the heart of any successful Conversion Rate Optimisation (CRO) endeavor lies a deep understanding of the users and their needs. However, when it comes to a diverse platform like Samsung.com which caters to a broad spectrum of products ranging from mobile devices to home appliances, the challenge of understanding users becomes notably intricate.
This challenge is exacerbated by the fundamental differences in user behavior and motivations between those browsing for mobile devices and those in the market for home appliances.
1.3 Behavourial Disparities
Mobile Shoppers
Aesthetic Appeal: Swayed by aesthetics, design, and brand reputation.
Feature-Oriented: Cutting-edge features, performance metrics, and camera quality play a significant role in their decision-making process.
Impulsive Behavior: They exhibit shorter decision-making cycles and may be more prone to impulse purchases driven by the desire for the latest technology.
Social Influence: Trends, peer recommendations, and social media presence heavily influence their preferences and purchasing decisions.
Home Appliance Shoppers
Functional Focus: Prioritise functionality, durability, and energy efficiency over aesthetic appeal.
Research-Driven: They engage in extensive research, comparing specifications, reading reviews, and considering compatibility with existing setups.
Practical Considerations: Factors such as capacity, energy efficiency ratings, and warranty coverage heavily influence their purchasing decisions.
Decision Process: The decision-making process tends to be more deliberative and less impulsive compared to other user groups.
TV and Monitor Shoppers
Visual Appeal: Prioritise visual quality, screen size, resolution, and display technologies.
Use Case Specificity: Decisions are often dictated by specific use cases such as gaming, professional editing, or home entertainment.
Technical Specifications: Refresh rates, response times, colour accuracy, and connectivity options are crucial factors in their decision-making process.
Research Intensity: Similar to home appliance buyers, they engage in research but focus more on technical specifications and user experience reviews.
Moreover, the user journey for mobile device purchases significantly differs from that of home appliance purchases. Mobile users typically exhibit a higher degree of impulsive behavior, with shorter decision-making cycles and a greater emphasis on aesthetics and cutting-edge features. Conversely, home appliance purchasers tend to engage in more extensive research, weighing factors like energy efficiency, capacity, and compatibility with existing home setups. It was important to distinguish these differences and utilise them in design decisions to reach the goal of higher conversion.
1.4 User Testing Strategy
Once designed and developed, each test was deployed onto the live Samsung site using Adobe Target. This was useful in that we could carefully monitor the results throughout the process and worst case, stop a test if necessary. The tests would each run for differing periods depending on a variety of factors like page traffic, current brand campaigns and ongoing results. Regardless of the way the test was going, we always strived for a 95% confidence rating before determining if a test was successful or not.
Small changes - massive impact
Failure during conversion rate optimisation only occurs when you learn nothing from your users. Unsuccessful tests offer the same amount of useful information as the successful ones. The goal is always to develop a deeper understanding of the users.
Test One: Utilising Social Proof
Introduction
This test revolves around the theory of using social proof as a way to help users make a decision in an environment that can be overwhelming with choice. After researching competitors and diving into a common UX idea known as Hicks Law, we realised that our users were crippled by choice. The theory was that narrowing down this choice would lead to higher conversion.
Background
Samsung incentivises users leaving reviews on products after purchase. This has created a wonderful database of social proof but the reality is, Samsung was not utilising this information to their advantage. We Identified this as a potential point of opportunity and wanted to introduce design on the site that focused on creating a web of social proof around the products.
Hypothesis Development
When watching HTML replays of users navigating the product filter pages, we noticed two key behaviours that developed our hypothesis. The first being that the first two cards get the most clicks, highlighting that users often want a simple option presented in front of them and secondly, most users scroll to the bottom of the page within 5 seconds; making it extremely unrealistic that they are digesting and weighing up the different product options.
IF we highlight the user reviews on highly rated products and offer a top rated section THEN users will find it easier to make a decision BECAUSE they will have social proof to justify making the purchase and will spend less time browsing products.
Research and Rationale
The research was based around competitor research and analytic data pulled from Adobe Analytics. We found that even when the product filter menu was sorted in any fashion, the top products got the most cart adds and converted higher. This paired with information on the page scroll depth allowed me to infer that having better products, that have already been socially approved, would create less friction between the browsing and purchasing phase. The control converted at approximately 1.3% - we were looking for an uplift of at least 3% with 95% confidence. We ran this test for 4 weeks due to extremely high traffic on each page.
Test Setup
Adding stars to product cards
Adding top rated products above the fold
Results and Learnings
Orders
15%
Conversion Rate
18%
These tests yielded a collective 20% increase in conversion rate across 4 different page types. This exceeded our goal and showed that leveraging social proof is a powerful tool in the e-commerce world. The major learning here was to give customers the content they want to see as early as possible in the journey and use techniques to further validate that content. This ultimately comes down to a common UX practice of providing users feedback. This test was done relatively early in my time in the CRO team but shaped the insights of future tests due to the extremely positive outcome.
Test Two: Make the Offer Clear
Introduction
Samsungs market offering exceeds hundreds of products. We found that often, users would get overwhelmed by how many products are available to them. This issue was particularly prominent on each of the Product Filter pages. Users would scroll quickly and miss important information like products on sale or new items.
Background
The stakeholders involved in the project had a particular interest in pushing current online offers as a USP for samsung.com. Samsung was in the unique position where their competitors became other retailers who were selling their own products such as JB Hi Fi and the Good Guys.
Hypothesis Development
IF we make the discount information on the Best Sellers section product cards more visible, THEN this increase CTR on those product cards , which will lead to more sales, BECAUSE more users will be able to see this valueable information and realise value easier
When developing a hypothesis, it was important to have data to back up potential ideas. This gave us a sense of reassurance that the users behaviour would change as a direct result of our actions.
Research and Rationale
Users were interacting with every card the same way as shown through an even amount of clicks through the different cards. This told the team that users were having trouble distinguishing the difference between cards that have offers and cards that do not. This was further verified by viewing HTML replays of users interacting specifically with the offer page. Best practice for e-commerce web design is to differentiate offers from non-offers. Many of Samsung's competitors were doing this which really sparked curiosity in whether it would work or not.
Test Setup
For this test, the biggest change was made on the product cards across the site. After researching, we came to the conclusion that users were missing offers as they were blending in with the other pieces of content on the card.
We made a few changes to fix this visually:
Moved the savings amount above the price.
Used the brand primary colour to highlight the savings.
Increased the text size of the old price with a more defined strikethrough.
Experience A (Control)
Experience B
Results and Learnings
Cart Adds
12.4%
Card Interactions
24.3%
The results of this test were incredibly encouraging. We experienced uplift of almost 25% on card interactions on offer cards vs cards with no offer and an increase of 12.4% cart adds for these products. From a personal point of view, this taught me the importance of showing users relevant information that could potentially sway their behaviour. Styling is extremely important when it comes to creating great information hierarchy which has ultimately lead to an uplift of millions of dollars for Samsung.
