Do you measure the wrong success criteria?

Here are some of my reflections from RoundUp Rodeo 72 with Sam Larner and Brian Fitzpatrick.

South Africa loses a lineout to Wales. How should this be measured?

In Episode 72, Sam Larner raises some excellent points about how we as coaches of teams measure the success of elements within the game. His example is why do we continually kick to the corner from a penalty when our lineout may not be consistently good enough to make the most of this opportunity.

This got me thinking about “resulting” which is discussed by Annie Duke in Episode 37 of The Knowledge Project podcast titled Getting Better by Being Wrong. Resulting is judging the result only on it’s outcomes, not on the merits of the decision making processes by which you made it.

We see resulting all the time in sport. Players, coaches, supporters all assess whether something was a good decision or not by whether the outcome was a positive one.

Let’s take Sam’s example. We get a penalty and make a choice to kick to the corner. If we win the lineout, observers will say it was a good decision. If we lose the lineout people will say we should have kicked for the posts or tapped and gone.


With analysis now being so accessible for non-professional teams, I wonder whether we could use the available information more effectively. Platforms such as Coach Logic for post match analysis or even free apps on an iphone or ipad for in game tagging such as Dartfish’s Easytag could help make our decision-making easier and give us useful data from which we can make better informed decisions.

Tracking sequences may provide some useful insight in to how the game unfolds.

The beauty (as well as the downside of the game) is it is continually stop-start. This presents us with multiple opportunities to measure and review what we do and how effective it is. We could list all the ways in which the game re-starts as well as all the ways in which possession changes. By then noting down how possession changes, how play starts and it’s outcome we may start to see some patterns emerge.

For example:

  1. Outcome / Possession Change = Penalty (we have gained possession from a penalty)
  2. Re-Start = Lineout (we have kicked to touch from the penalty)
  3. Outcome / Possession Change = Lost lineout (we lose the lineout)

Traditionally we may just look at a lineout win/loss percentage but that can largely ignore the context in which the decision to kick to the corner was made. This sequential assessment may start to demonstrate the rationale for our thinking as it can better highlight patterns.

It may also highlight flaws in our thinking. Either way, we can update our decision-making processes to better align our decisions with a likelier outcome.

If we kick to the corner and lose the lineout then we can at least assess it from a perspective that it was our best decision rather than claim we should have done something else based purely on the result.

Collecting data this way make take a bit of time to gather a baseline and understand what decisions the team make consistently or inconsistently, whichever the case may be. Where I believe our focus should be is on what informs the decisions we make, how do we know this is based on good intelligence and if it isn’t how will we review it.

Avoiding being a resulter who always knows what the team should have done after the fact and who only operates in hindsight I would suggest is pretty crucial for a coach.

Share this
Follow us