Conversion fell. What changed?
The topline rate fell after a new acquisition campaign.
The question
Harbor Market’s dashboard uses one eligible new user per row. A converter is a user with at least one completed purchase within seven days. Both weeks are fully observed.
| Week 1 eligible | Week 1 converters | Week 2 eligible | Week 2 converters | |
|---|---|---|---|---|
| Direct | 900 | 180 | 100 | 24 |
| Paid | 100 | 10 | 900 | 108 |
Authored data. Channels are mutually exclusive, assigned at first acquisition. Eligibility and event capture are unchanged.
The paid campaign expanded in week 2. The packet contains no spend, margin, random assignment or within-channel customer characteristics.
“Decompose the decline. Recommend an action and state what could change it.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Reconcile the dashboard
What would you do here? Open the move you’d make.
Compare an equal-weight average of channels
Tempting, but it misses something. An unweighted channel mean ignores the 90/10 to 10/90 mix change.
Weight by eligible users
The strong move. Start with the rate the dashboard actually reports.
See how a strong candidate says it
“Week 1 has 190/1,000 = 19% conversion. Week 2 has 132/1,000 = 13.2%. The decline is 5.8 percentage points, not 5.8%.”
Why it works: Start with the rate the dashboard actually reports.
Step 2: Compare within channels
What would you do here? Open the move you’d make.
Reweight week 2 at the old mix
The strong move. A fixed mix exposes the composition effect.
Prioritize onboarding fixes from the overall decline
Tempting, but it misses something. Both channel rates increased; the pooled decline is not a within-channel decline.
See how a strong candidate says it
“Direct rises from 20% to 24%; Paid rises from 10% to 12%. At week 1’s 90/10 mix, week 2 would be 22.8%. Under that decomposition, within-channel rate changes add 3.8 points and the mix shift subtracts 9.6 points.”
Why it works: A fixed mix exposes the composition effect.
Step 3: Keep the causal boundary
What would you do here? Open the move you’d make.
Use the adjusted rate as campaign uplift
Tempting, but it misses something. Channels can differ in observed and unobserved ways.
Ask what changed within channels
The strong move. Descriptive adjustment is not random assignment.
See how a strong candidate says it
“The composition change explains the arithmetic. It does not prove the campaign improved conversion or that cutting paid spend helps profit. Within-channel audience quality and other changes may differ.”
Why it works: Descriptive adjustment is not random assignment.
Step 4: Make a bounded recommendation
What would you do here? Open the move you’d make.
Check incremental economics before scaling
The strong move. The next action follows the missing decision evidence.
Pause paid acquisition to restore conversion rate
Tempting, but it misses something. Lower average conversion can coexist with profitable additional customers.
See how a strong candidate says it
“Do not roll back onboarding from this dashboard alone. Audit acquisition tracking and compare marginal paid contribution with spend. Cap or test expansion if economics are uncertain; continue it if incremental contribution covers cost. Revisit if comparable channel cohorts deteriorate.”
Why it works: The next action follows the missing decision evidence.
The answer, in short
- Decision
- Investigate acquisition economics before changing onboarding.
- Because
- The mix moved toward the lower-converting channel while both reported channel rates rose.
- What would change it
- Worse comparable within-channel cohorts, broken attribution or negative incremental paid contribution would change the action.
Common traps
- Treating segmentation as a causal experiment.
- Optimizing rate while ignoring total contribution.
Check your answer
- Did you compute 19% and 13.2%?
- Did you compare 20→24% and 10→12%?
- Did you compute the 22.8% fixed-mix rate?
- Did you name evidence that could reverse the recommendation?
Last reviewed Nov 10, 2026