Don’t blindly tie every piece of work to top-level metrics. Even if technically feasible, the cost is too high and the risk of spurious logic chains significant.
Start with Value Definition
Begin each project with a crisp definition of why it’s worth doing. What underlying problem are you solving, and why is that problem worth solving? Once you have these narrative assertions, it’s usually clear how extraordinary or controversial each claim is.
The more notable the claim, the more likely you need data to support it.
Value Metrics
1. Direct outcome metrics (strongest)
We will run an ongoing experiment measuring profit per user with the feature on vs baseline.
2. Strong correlative metrics
This will increase time on site, which we can measure and believe correlates with profit per user.
3. Anecdotes and feedback
N sales team members report they could sell into more accounts if we launch this feature.
4. Strategic assertions
We must do this because of upcoming regulation or we will be unable to continue this business line.
Progress Measurement
Once your value claim is clear and defensible, identify how you’ll measure progress. This may differ from your value metric. Ideal progress metrics tell you whether you’re succeeding, respond quickly to team actions, and have strong reference baselines.
1. Clear baseline, goal, and team-tied metric (Strongest)
Launching this compressor will reduce binary download size by an estimated 10% vs the best available industry baseline. We can measure relative progress continually against our production binary during development.
2. Responsive metric without clear reference point
We can improve compile times on this fixed codebase from today’s 90s baseline.
3. Non-responsive metric
We can measure weekly mobile app release binary size, comparing the new compressor to our old implementation.
4. Milestones
We will implement passes a, b, c, after which we can ship the new compiler optimizations to target customers.
Common Challenges
Stronger measures aren’t always a worthwhile tradeoff. If you have high confidence in the work’s value and applicability and mainly need to validate progress, milestones can be completely reasonable.
In general, approach projects with skepticism about whether this is the right thing to do and whether you’ll make good progress. Then identify ways to get concrete data rather than rely solely on leadership support.
Leadership pressure for top-level metrics
The clearer you are on why you’re doing something, the easier it becomes to communicate your measurement decisions. If leadership continues pushing back and you have a good relationship, use that as a lens to explore concerns you might have missed. Often requests for metric clarity stem from deeper worries about project value or plausibility.
Team dynamics and gaming
Metrics communicate value in performance reviews, creating incentives for engineers to identify unnecessary correlations or game metrics (intentionally or not). Counter this with “health” metrics that balance negative behaviors — if measuring deployment frequency, also measure production incidents with a flat target to prevent trading off reliability for speed.
For senior engineers concerned about optics, have them clearly articulate the value chain. Working through and demonstrating strong data usage in project steering is itself a highly rewarded skill — encourage them to take on that role.
When to revisit metrics
The world changes. What’s sensible in one environment isn’t in another. Constantly relitigating is a headache, but reevaluating logic at regular intervals (say, each half-year planning cycle) is appropriate. Otherwise, maintain awareness of company trends: new projects, initiatives, or teams gaining significant attention. If they were successful, would that change what you do? There’s no hard rule: it’s corporate decision-making taste that develops with experience.