L3 Playbook: Sales Coaching with AI Scorecards

Why this matters

The traditional "ride-along" is dead. In the average B2B sales organization, a manager supervises 6–8 reps. If each rep has 10–15 meetings a week, that is nearly 120 hours of tape. No manager has the bandwidth to listen to even 5% of that volume. Consequently, coaching becomes "vibes-based"—the manager listens to the last call the rep had, provides anecdotal feedback on that specific interaction, and hopes it sticks.

The cost of this manual approach is staggering. Organizations with inconsistent coaching see a 20% delta in quota attainment between the top and bottom quartiles. When you don't coach to data, you allow bad habits—like skipping discovery or failing to set a firm next step—to calcify across the entire team.

By moving to Level 3 Maturity (AI Scorecards), you transition from sampling data to analyzing the whole. Tools like Gong and Chorus now allow you to score 100% of calls against a rigid, behavior-based rubric. This doesn't just save a manager 4–6 hours of "searching" for calls each week; it ensures that coaching is based on a rep's aggregate trend over 50 calls, not a single lucky or unlucky conversation.

How it works

1. Build a 5-point rubric

The biggest mistake in AI coaching is being vague. AI cannot score "Good energy." It can, however, score "Did the rep mention the 'Implementation Timeline' before the 30-minute mark?"

• Action: Identify 4–6 high-impact behaviors that differentiate your top performers.
• The Blueprint: In your CI tool (Gong, Chorus, or even specialized tools like Salesloft), create a scorecard titled "Core Sales Execution."
• The Specifics: Instead of "Objection Handling," use: "Did the rep acknowledge the objection and ask a clarifying question before responding?"
• Time Invested: 2–3 hours. Stop at 6 criteria; any more, and you dilute the focus for both the AI and the rep.

2. Configure AI-driven scoring

This is where you move from manual grading to automation. You must map your rubric to specific "Trackers."

• The Setup: Link each rubric item to keyword groups or "Smart Playlists." For "Next Steps," configure the AI to hunt for phrases like "calendar," "invitation," "next Tuesday," or "follow-up."
• The Calibration: Run the AI against 10 historical calls. If it flags "Next Steps" because a rep said "I'll follow up next year," you need to tighten your parameters.
• Refinement: Use LLM-based tools like Claude or OpenAI's API via Momentum.io to summarize the "reasoning" behind a score. This helps reps understand why the AI gave them a 3/5.

3. Identify coaching outliers

Stop listening to "average" calls. Use the dashboard to find the extremes.

• The Filter: Create a "Coaching Focus" view. Look for calls in the bottom 20% of scores. Specifically, look for calls with high "Talk Ratio" (>70%) but low "Discovery Depth." This is a massive red flag that a rep is "feature dumping."
• The Gold Mine: Conversely, filter for the top 10% of "Objection Handling" scores. Automatically push these into a "Gold Standard" folder in your CI library for new hire onboarding.
• Winning Move: This reduces manager prep time from hours to 15 minutes of high-leverage review.

4. Execute gap-based coaching

Don't coach the call; coach the trend. If a rep’s "Pricing Mention" score has been below 60% for three weeks, that is a systemic issue.

• The 30-Day Plan: "Rep will introduce the price range before the 40-minute mark in 80% of calls."
• Precision Feedback: Use the timestamp features in Gong or Fathom to leave comments. Instead of saying "Do better discovery," tag a 15-second clip where the rep missed a pain signal and say, "The AI flagged this as a missed discovery opportunity. What could you have asked when they mentioned their budget was tight?"

5. Track the improvement curve

Finally, validate that coaching is driving revenue. RevOps should own this dashboard.

• The Correlation: Export your AI adherence scores and overlay them with CRM data in Salesforce or HubSpot.
• The Metric: You are looking for a correlation between "Rubric Adherence" and "Stage 2 to Close" conversion rates. Typically, teams mastering AI-scored discovery see a 15–22% increase in win rates within two quarters.
• Manager Accountability: Build a leaderboard showing which managers are actually moving the needle on their team’s aggregate scores.

Tools you need

• Conversation Intelligence: Gong, Chorus, or Salesloft.
• Productivity / Note-Taking: Fathom or Granola (for capturing the raw inputs the AI will score).
• Automation/Workflow: Momentum.io (to push AI scores into Slack alerts for managers).
• Analysis: Salesforce/HubSpot for revenue correlation.

KPIs to track

• Rep-to-Quota: The percentage of the team hitting their number.
• Coaching Consistency: Average number of AI-scored calls reviewed by managers per week.
• Adherence Lift: The month-over-month increase in aggregate rubric scores.

Common pitfalls

• Over-complicating the Rubric: If you have 15 criteria, the AI loses accuracy and reps get overwhelmed. Stick to 5–6.
• Ignoring False Positives: If the AI is scoring poorly, managers will lose trust in the system. Spend the 4 hours upfront tuning the trackers.
• Punitive Coaching: If reps feel the AI is a "Big Brother" tool used to fire them, they will stop recording calls. Position it as a "flight simulator" for their career growth.

When to graduate

You are ready for Level 4 (Generative Coaching) when your AI scorecards are 90% accurate and your managers are spending less than 30 minutes a week on prep. At that stage, you can begin using AI to auto-generate the coaching emails and personalized training modules for each rep based on their score gaps.