Did activation fall or did the denominator change?
New sign-ups jumped. The dashboard reports a sharp activation drop.
The question
Harbor Market counts activation as a first completed purchase within seven days of sign-up. The export was taken at the end of day 14.
| Users | Seven-day window closed | Activated within seven days | |
|---|---|---|---|
| Days 1–7 | 1000 | Yes | 400 |
| Days 8–14 | 2000 | No | Not yet observed |
Authored data. Recent sign-ups have not had equal time to activate.
The previous fully observed seven-day cohort had 1,000 sign-ups and 350 activations. The dashboard divides the current 400 activations by all 3,000 current sign-ups.
“Define a comparable activation rate and say what you can conclude.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Fix the observation window
What would you do here? Open the move you’d make.
Use the dashboard’s full sign-up base
Tempting, but it misses something. New users have not had seven days to purchase.
Compare mature cohorts
The strong move. The rate needs equal opportunity to activate.
See how a strong candidate says it
“Use completed seven-day cohorts. Current mature activation is 400/1,000 = 40%; the prior mature cohort is 350/1,000 = 35%. The dashboard’s 13.3% mixes observed outcomes with immature users.”
Why it works: The rate needs equal opportunity to activate.
Step 2: Declare the unit
What would you do here? Open the move you’d make.
Count distinct eligible users
The strong move. Repeated events must not create extra people.
Use completed purchase event counts
Tempting, but it misses something. One user can create more than one purchase.
See how a strong candidate says it
“The denominator is distinct eligible sign-ups, not sessions or purchase events. Count each person once and use the same timezone and eligibility rule in both cohorts.”
Why it works: Repeated events must not create extra people.
Step 3: Limit the conclusion
What would you do here? Open the move you’d make.
Report the increase as campaign lift
Tempting, but it misses something. Two time cohorts differ in more than campaign exposure.
Separate the trend from its cause
The strong move. Comparable descriptive rates still do not identify causation.
See how a strong candidate says it
“Mature cohort activation rose five percentage points. I would not claim the new campaign caused it: channel mix and other changes may differ. We cannot yet estimate final activation for the recent 2,000 users.”
Why it works: Comparable descriptive rates still do not identify causation.
Step 4: Repair the report
What would you do here? Open the move you’d make.
Mark the immature cohort pending
The strong move. A dated report can be checked and updated.
Publish a rolling rate including new sign-ups
Tempting, but it misses something. A missing future outcome is not an observed failure.
See how a strong candidate says it
“Show 40% for the mature cohort, mark the recent cohort pending, and schedule its seven-day readout. Ask the data owner to test late events, duplicate users and timezone cutoffs.”
Why it works: A dated report can be checked and updated.
The answer, in short
- Decision
- Repair the denominator before changing onboarding.
- Because
- The displayed decline includes users without a full observation window.
- What would change it
- A comparable mature cohort that falls after validating event capture would justify investigating a real decline.
Common traps
- Treating unobserved outcomes as zero.
- Confusing percentage points with relative percent change.
Check your answer
- Did you compute 40% and 35% on mature users?
- Did you identify the misleading 13.3% denominator?
- Did you count distinct users once?
- Did you avoid claiming the campaign caused the gain?
Last reviewed Nov 10, 2026