Field note
Activation is a decision you can observe, not just a signup
A practical field note on choosing an activation milestone, avoiding misleading denominators and connecting first value to a second useful session.
Start with the job the user came to do
A signup tells you someone crossed an access boundary. It does not tell you whether they received value. To choose an activation milestone, name the task they wanted to complete and the evidence that would show progress.
For a practice product, a first exercise submission is one useful event. Reading the resulting feedback is another. Returning to apply that feedback is stronger evidence of a repeatable learning habit. Keep the events separate so a failure between them remains visible.
A synthetic funnel
These numbers are illustrative, not Product Mastery results. The point is that each transition asks a different question. Is the first exercise easy to find? Does submission work? Is the feedback worth reading? Does the learner know what to do next?
| Stage | People | What it supports |
|---|---|---|
| New accounts | 40 | Interest plus successful signup |
| Saved first exercise | 18 | An attempted task |
| Viewed feedback | 14 | Exposure to the review |
| Practised on a later day within seven days | 6 | An early repeat-use signal |
Choose an eligible denominator
A person who joined yesterday has not had a full week to return. Report the seven-day return rate only for people with a complete observation window, and show the newer cohort as pending. For later retention, define the window explicitly, such as days 22 through 28 after enrollment.
Keep staff, automated checks, seeded demonstrations and unknown identities distinguishable from verified external learners. A test that completes the entire funnel proves that the software can work; it does not prove customer demand.
Use the drop-off to choose the next experiment
One small cohort cannot settle every question. Combine event evidence with observations of people attempting the task. Record what you saw separately from your explanation of why it happened.
- Signup to first exercise: make the promised task easy to reach and remove unnecessary setup.
- Exercise to feedback: verify saving, grading and readable feedback before adding acquisition.
- Feedback to return: give one specific next practice and a reason to try it.
- Return to paid use: inspect whether the paid capability solves an observed need; do not treat a free account or internal override as payment.
Try the reasoning yourself
Take a product you know and finish this sentence: “A new user first receives value when…” Then name the event, its limitations and the next behaviour that would make your interpretation stronger.
If your answer is “when they log in,” go one step further. What were they trying to accomplish after logging in?
Practise metric selection with a public sampleCopy the experiment card