Redesigned the post-survey experience, finding an acquisition channel worth $100K+/yr
Redesigned the post-survey experience, finding an acquisition channel worth $100K+/yr
Role
Lead product designer
Skills
Product strategy
A/B Testing
Growth design
Timeline
Sept - Oct 2024
Team
Product manager
Engineers
Brand Lead
Content designer
Problem
Problem
Users who completed a survey landed on a Thank You experience that encouraged them to create an account. This surface got millions of visits each month and was a major entry point into sign-up.
Activation was the issue. Most new sign-ups never created their first survey, which meant we were missing a critical opportunity to convert new users into active, paying customers. For all new sign-ups:
Users who completed a survey landed on a Thank You experience that encouraged them to create an account. This surface got millions of visits each month and was a major entry point into sign-up.
Activation was the issue. Most new sign-ups never created their first survey, which meant we were missing a critical opportunity to convert new users into active, paying customers. For all new sign-ups:
86%
86%
Never created a survey
Never created a survey
0.3%
0.3%
launched a survey
launched a survey
Research showed two main reasons people created accounts:
Some thought signing up would unlock access to more surveys to take.
Others were mid-survey and believed an account was required to submit their responses.
Research showed two main reasons people created accounts:
Some thought signing up would unlock access to more surveys to take.
Others were mid-survey and believed an account was required to submit their responses.
The Redesign
The Redesign
Based on our research insights, I redesigned the experience with dual CTAs and distinct routing paths. I collaborated with product, engineering, and brand to test this as an A/B experiment (1 control + 3 treatments).
Based on our research insights, I redesigned the experience with dual CTAs and distinct routing paths. Users who clicked 'Take More Surveys' were directed to Contribute (our survey-taking platform), while users who clicked 'Create Surveys' entered the core product onboarding.
I collaborated with product, engineering, and brand to test this as an A/B experiment (1 control + 3 treatments).
Outcome
Outcome
The redesign made the post-survey entry point clearer for users. We learned that survey-taking was a valuable path for our users, while we had previously funnelled them into account creation. It also revealed an acquisition opportunity on the survey-taking side.
However, users still dropped off later in account creation for both paths, so the next step was to redesign the downstream sign-up flows to reinforce their initial interest.
The redesign made the post-survey entry point clearer for users. We learned that survey-taking was a valuable path for our users, while we had previously funnelled them into account creation. It also revealed an acquisition opportunity on the survey-taking side.
However, users still dropped off later in account creation for both paths, so the next step was to redesign the downstream sign-up flows to reinforce their initial interest.
60/40 split
60/40 split
between ‘Take More Surveys’ and ‘Create Surveys’
between ‘Take More Surveys’ and ‘Create Surveys’
100K+/year
100K+/year
projected savings by recruiting organic survey-takers over paid panelists
projected savings by recruiting organic survey-takers over paid panelists
Context
Context
The Thank You page appears after a user has completed a survey
The Thank You page appears after a user has completed a survey
This page encourages account creation and acts as the primary entry point into the product after survey completion.
This page encourages account creation and acts as the primary entry point into the product after survey completion.
The current sign up flow from the Thank You page
The Problem
The Problem
We were getting high sign-ups, but minimal survey creation
We were getting high sign-ups, but minimal survey creation
Most users who signed up from this page never came back to create a survey, and we didn’t know why. Without understanding user intent, we couldn't optimize the page to drive conversions or meet user needs.
Most users who signed up from this page never came back to create a survey, and we didn’t know why. Without understanding user intent, we couldn't optimize the page to drive conversions or meet user needs.
The Thank You page got high sign ups, but most users were inactive
Research
Research
Why were people signing up in the first place?
Why were people signing up in the first place?
To understand user intent, the team had previously surveyed 196 users who signed up within 45 days. This revealed a major disconnect between what the business expected and what users were doing:
To understand user intent, the team had previously surveyed 196 users who signed up within 45 days. This revealed a major disconnect between what the business expected and what users were doing:
What was the primary reason you signed up for a SurveyMonkey account?
What was the primary reason you signed up for a SurveyMonkey account?
107 responses
107 responses
1
“I thought it was required to complete the survey I was taking” (34.58%)
“I thought it was required to complete the survey I was taking” (34.58%)
2
“I’m interested in taking more surveys” (23.36%)
“I’m interested in taking more surveys” (23.36%)
3
“I’m interested in using SurveyMonkey to send out a survey” (20.56%)
“I’m interested in using SurveyMonkey to send out a survey” (20.56%)
Our goal for the Thank You page was to convert users to our core product: survey creation and launch. But research showed that users were signing up out of uncertainty, not intent to create surveys:
Our goal for the Thank You page was to convert users to our core product: survey creation and launch. But research showed that users were signing up out of uncertainty, not intent to create surveys:
“I was confused and I’m still confused as to why I was required to sign up. More information would have helped.”
“I was confused and I’m still confused as to why I was required to sign up. More information would have helped.”
— Survey Participant
— Survey Participant
User Journey
User Journey
Why are participants confused?
Why are participants confused?
Our research revealed that these users took surveys regularly. When they saw a sign-up form, they assumed it was to receive more surveys, not to create their own.
Our research revealed that these users took surveys regularly. When they saw a sign-up form, they assumed it was to receive more surveys, not to create their own.
These insights revealed a clear solution:
These insights revealed a clear solution:
Create distinct paths for survey-takers and survey-creators, while clearly signaling when the survey-taking journey was complete
Create distinct paths for survey-takers and survey-creators, while clearly signaling when the survey-taking journey was complete
Design Process
Design Process
Experimenting with Dual-CTAs
Experimenting with Dual-CTAs
SurveyMonkey had two distinct products: our core survey creation tool and Contribute, our survey-taking platform - each with different sign-up flows and value propositions.
Based on our research, I collaborated with my PM to design a dual-CTA flow:
'Take More Surveys' addressed the majority user need we discovered
'Create Surveys' maintained our business priority and supported users who might transition to survey creation in the future.This allowed us to serve user intent without abandoning our core conversion goal.
SurveyMonkey had two distinct products: our core survey creation tool and Contribute, our survey-taking platform - each with different sign-up flows and value propositions.
Based on our research, I collaborated with my PM to design a dual-CTA flow:
'Take More Surveys' addressed the majority user need we discovered
'Create Surveys' maintained our business priority and supported users who might transition to survey creation in the future.This allowed us to serve user intent without abandoning our core conversion goal.
Experiment Details
Experiment Details
Hypothesis and Success Metrics
Hypothesis and Success Metrics
Hypothesis
Hypothesis
By offering a clear next step that matches user intent, we will increase activation on both paths.
By offering a clear next step that matches user intent, we will increase activation on both paths.
Success metrics
Success metrics
Survey-takers: New Contribute panelists increase by 10%
Survey-creators: Survey launches increase by 5%
Survey-takers: New Contribute panelists increase by 10%
Survey-creators: Survey launches increase by 5%
Design Decisions
Design Decisions
Playing with Layout
Playing with Layout
To present the two options, I explored different layouts: vertically stacking both CTAs, or placing 'Create surveys' as the primary form with 'Take more surveys' as a secondary option below. After exploring multiple layouts, I chose a side-by-side approach because it placed both options at equal hierarchy, making it easier for users to compare and choose their intended path.
To present the two options, I explored different layouts: vertically stacking both CTAs, or placing 'Create surveys' as the primary form with 'Take more surveys' as a secondary option below. After exploring multiple layouts, I chose a side-by-side approach because it placed both options at equal hierarchy, making it easier for users to compare and choose their intended path.
Playing with Imagery
Playing with Imagery
Using imagery to convey meaning instantly was a crucial aspect of this experiment. I had to ensure our imagery and copy were instantly clear and effectively contrasted the two distinct pathways, Create Surveys and Take More Surveys. Through design reviews with stakeholders, I iterated on the imagery based on feedback:
Using imagery to convey meaning instantly was a crucial aspect of this experiment. I had to ensure our imagery and copy were instantly clear and effectively contrasted the two distinct pathways, Create Surveys and Take More Surveys. Through design reviews with stakeholders, I iterated on the imagery based on feedback:
Exploration: new brand imagery
Exploration: new brand imagery
How interested are you in a product like this?
Interested
Not interested
Multiple Choice
Checkboxes
Dropdown
Textbox
Create Surveys:
➕ Person working depicts survey creation
➖ Text can be cut down
➖ Green/blue monotone
➖ Could try a more interesting UI to show survey creation
Create Surveys:
➕ Person working depicts survey creation
➖ Text can be cut down
➖ Green/blue monotone
➖ Could try a more interesting UI to show survey creation
Take Surveys:
➕ Notable charities highlighted
➖ Person doing the same action as the left image, should be distinct
Take Surveys:
➕ Notable charities highlighted
➖ Person doing the same action as the left image, should be distinct
Iteration
Iteration
Create a survey
Start from a template
Build with AI
Create Surveys:
➕ More interesting UI on top left showing survey creation
➕ More interesting colors for insights
➕ Minimal text
Create Surveys:
➕ More interesting UI on top left showing survey creation
➕ More interesting colors for insights
➕ Minimal text
Take Surveys:
➕ Person shows joy and is looking at charities, different from image on the left
Take Surveys:
➕ Person shows joy and is looking at charities, different from image on the left
Exploration 2: old brand imagery
Exploration 2: old brand imagery
Create Surveys:
➖ Survey creation missing
Create Surveys:
➖ Survey creation missing
Take Surveys:
➕ ‘Taking a survey’ action is clear
Take Surveys:
➕ ‘Taking a survey’ action is clear
Exploration 3: old brand imagery
Exploration 3: old brand imagery
Create Surveys:
➖ Survey creation can be emphasized more
Create Surveys:
➖ Survey creation can be emphasized more
Take a Survey
Take Surveys:
➕ ‘Taking a survey’ action is clear
Take Surveys:
➕‘Taking a survey’ action is clear
User testing
User testing
Do you get this in 5 seconds?
Do you get this in 5 seconds?
To assess whether the page clearly communicated its purpose and whether the images instantly differentiated the two products, I conducted a 5-second test with 10 users.
To assess whether the page clearly communicated its purpose and whether the images instantly differentiated the two products, I conducted a 5-second test with 10 users.
Key insight
Key insight
Simplify the imagery while preserving recognizable elements (bar chart, Red Cross logo) that help users quickly understand each path.
Simplify the imagery while preserving recognizable elements (bar chart, Red Cross logo) that help users quickly understand each path.
The results from the usability testing that I ran on UserTesting.com
Final Design
Final Design
Here’s what we tested
Here’s what we tested
We ran an A/B test with 1 control and 3 treatments to determine which imagery most clearly communicated the two distinct paths.
We ran an A/B test with 1 control and 3 treatments to determine which imagery most clearly communicated the two distinct paths.
Control
Control
The key difference between the control and treatments was the routing logic.
The key difference between the control and treatments was the routing logic.
The control sent all users to survey creation, while the treatments directed users to different flows based on their intent.
The control sent all users to survey creation, while the treatments directed users to different flows based on their intent.
Treatment 1: Current SurveyMonkey brand imagery
Treatment 1: Current SurveyMonkey brand imagery
Treatment 2: Old SurveyMonkey brand imagery
Treatment 2: Old SurveyMonkey brand imagery
Create Surveys: UI/product imagery (charts, graphs)
Take More Surveys: People imagery (person with device)
Create Surveys: UI/product imagery (charts, graphs)
Take More Surveys: People imagery (person with device)
Treatment 3: Old SurveyMonkey brand imagery
Treatment 3: Old SurveyMonkey brand imagery
Create Surveys: People imagery (person working)
Take Surveys: UI imagery (simple button/form)
Create Surveys: People imagery (person working)
Take Surveys: UI imagery (simple button/form)
Experiment results
Experiment results
We ran a 6-week A/B test with ~10,000 users across 4 variations
We ran a 6-week A/B test with ~10,000 users across 4 variations
All treatments generated more Contribute panellists than control, with T2 performing marginally better (279 vs 38 panelists).
All treatments generated more Contribute panellists than control, with T2 performing marginally better (279 vs 38 panelists).
60/40 split
60/40 split
between ‘Take More Surveys’ and ‘Create Surveys’
between ‘Take More Surveys’ and ‘Create Surveys’
7x ↑
7x ↑
more Contribute panellists per treatment vs control
more Contribute panellists per treatment vs control
29% ↓
29% ↓
survey launches vs control
(initial goal: 5% increase)
survey launches vs control
(initial goal: 5% increase)
However, the data also revealed a tradeoff: while treatments dramatically increased survey-taking, they slightly reduced survey launches (Control: 62, Treatments: 41-49). This showed that serving majority user intent came at a cost to our primary business goal.
However, the data also revealed a tradeoff: while treatments dramatically increased survey-taking, they slightly reduced survey launches (Control: 62, Treatments: 41-49). This showed that serving majority user intent came at a cost to our primary business goal.
Impact
Impact
Uncovered majority user intent + a hidden acquisition channel worth $100K+/year
Uncovered majority user intent + a hidden acquisition channel worth $100K+/year
The results validated the dual-CTA approach: historically, we had funneled all users toward survey creation. The experiment proved that most visitors actually valued the survey-taking path more.
While the experiment didn't meet our activation targets for survey creation, it revealed an unexpected opportunity. We could acquire survey-takers organically instead of paying for them. Based on gaining ~279 panelists per treatment over 6 weeks, if rolled out to all users:
The results validated the dual-CTA approach: historically, we had funneled all users toward survey creation. The experiment proved that most visitors actually valued the survey-taking path more.
While the experiment didn't meet our activation targets for survey creation, it revealed an unexpected opportunity. We could acquire survey-takers organically instead of paying for them. Based on gaining ~279 panelists per treatment over 6 weeks, if rolled out to all users:
100K+/year
100K+/year
projected savings by recruiting organic survey-takers over paid panelists
projected savings by recruiting organic survey-takers over paid panelists
This finding reframed the business case and gave us a win 🎉🎉🎉
This finding reframed the business case and gave us a win 🎉🎉🎉
Even though fewer users activated as survey creators, converting drop-off traffic into survey-takers generated measurable value while also serving user intent. We considered this a strategic win as we discovered a new acquisition channel we hadn't optimized for before.
Even though fewer users activated as survey creators, converting drop-off traffic into survey-takers generated measurable value while also serving user intent. We considered this a strategic win as we discovered a new acquisition channel we hadn't optimized for before.
Next steps
Next steps
Iteration 2 focused on streamlining sign up flows to reduce friction and improve activation
Iteration 2 focused on streamlining sign up flows to reduce friction and improve activation
For iteration 2, we chose Treatment 1 as the new baseline. While T2 had slightly outperformed T1 in Contribute sign-ups (279 vs 273), we prioritized T1 because it used SurveyMonkey's updated brand imagery, ensuring our optimization work would align with the company's evolving visual identity. We then focused on streamlining both sign-up flows to reduce friction and improve activation rates.
For iteration 2, we chose Treatment 1 as the new baseline. While T2 had slightly outperformed T1 in Contribute sign-ups (279 vs 273), we prioritized T1 because it used SurveyMonkey's updated brand imagery, ensuring our optimization work would align with the company's evolving visual identity. We then focused on streamlining both sign-up flows to reduce friction and improve activation rates.
Learnings
Learnings
I learned how to make trade-offs and balance conflicting goals
I learned how to make trade-offs and balance conflicting goals
A tradeoff: Adding the 'Take Surveys' path reduced survey creation by 29%. This taught me that user-centric design sometimes conflicts with business priorities as 60% of users got what they wanted (survey-taking), but it meant fewer users entering our core product (survey creation). The key learning: you can't optimize for everyone, and being explicit about tradeoffs helps stakeholders make informed decisions.
A tradeoff: Adding the 'Take Surveys' path reduced survey creation by 29%. This taught me that user-centric design sometimes conflicts with business priorities as 60% of users got what they wanted (survey-taking), but it meant fewer users entering our core product (survey creation). The key learning: you can't optimize for everyone, and being explicit about tradeoffs helps stakeholders make informed decisions.
Clarity drives conversion: Clear visual hierarchy and descriptive imagery mattered as much as the strategic decision to offer dual paths. My key takeaway was that product UI imagery (charts, forms, data) consistently outperforms lifestyle photography for functional conversion pages. I could apply this principle to future experiments for sign-up pages.
Clarity drives conversion: Clear visual hierarchy and descriptive imagery mattered as much as the strategic decision to offer dual paths. My key takeaway was that product UI imagery (charts, forms, data) consistently outperforms lifestyle photography for functional conversion pages. I could apply this principle to future experiments for sign-up pages.