ENTR
Metrics & AnalysisLead≈ 30 min

Kova — AI Forecasting

The AI forecasting feature has been live for 6 months. Different teams are tracking different metrics and claiming conflicting results.

The scenario

Kova is a B2B supply chain platform. Six months ago you shipped AI-powered demand forecasting. Sales claims it is driving deals. Product claims engagement is strong. Finance is skeptical because revenue attributable to the feature is unclear. No one agreed on metrics at launch.

Constraint

The CEO wants a single north-star metric and a framework agreed across all teams by end of this week.

Your task

Define the metrics framework and propose a north-star metric for AI forecasting.

1. Clarify: What do you need to establish before proposing a framework?

Pick the 3 questions you'd ask first.

Model approach (strong picks)

  • Strong pick
    What is the intended business outcome of AI forecasting — reduce stockouts, improve margins, accelerate sales cycles?
  • Strong pick
    What metrics is each team currently tracking, and why did they choose those?
  • Strong pick
    Who are the primary users of the forecasting feature — ops teams, buyers, or executives?

2. Hypothesize: Why is there no agreed metric after 6 months?

Pick the 3 most likely root causes.

Model approach (strong picks)

  • Strong pick
    No success criteria were defined at launch — the team shipped without alignment on what "good" looks like.
  • Strong pick
    Each team is optimizing for their own incentives — sales for pipeline, product for engagement, finance for revenue.
  • Strong pick
    The feature's impact is hard to isolate — it's used alongside other tools, making attribution genuinely difficult.

3. Investigate: What do you do this week to build the framework?

Pick the 3 highest-leverage moves.

Model approach (strong picks)

  • Strong pick
    Interview 5 customers who actively use forecasting — ask what decision it changed and what outcome followed.
  • Strong pick
    Run a working session with Sales, Product, and Finance to map the causal chain: feature usage → behavior change → business outcome.
  • Strong pick
    Audit existing event data to see what is actually measurable today vs. what requires new instrumentation.

4. Propose: What is your north-star metric for AI forecasting?

Pick the one metric you'd propose to the CEO.

Model approach (strong picks)

  • Strong pick
    % of customers who made at least one sourcing or inventory decision using a Kova forecast in the past 30 days — measures real adoption and decision influence, not just logins.
  • Strong pick
    Forecast accuracy rate (predicted vs. actual demand within ±10%) — measures the quality of the AI output directly.

Related questions

Solve this case study