A few weeks ago, Harvey Fialkov wrote an article about what he calls “deceiving stats.” He referenced the Florida Panthers’ case, who were 2-6-3 while outshooting opponents, in order to prove that the old adage (or more accurately, the recent breakthrough) that teams that outshoot their opponents tend to win more games is false. Fialkov’s article mirrors the mentality of much of the main stream media — as well as management and fans — in discounting what are being referred to as “advanced stats,” because they can find examples, in small samples, of where they fail to predict an outcome. The reality, and one of the most important takeaways from recent analytical breakthroughs, is that the impact of luck (read: variation) is much larger than what is traditionally assumed. In small samples, anything can happen. And yes, 11 games is a very small sample.
It’s easy to be a Monday Morning Quarterback and judge with hindsight. So I’ll start with doing just that. At the time of Fialkov’s writing, the Panthers were in 28th place in the 30 team NHL with 19 points. Since then, they have gone 7-2 (including most recently five straight wins) and have outshot their opponents in six of those nine games. Suddenly, the Panthers are 7-8-3 when outshooting opponents on the year, not nearly as bad. All it took was three weeks for them to get back close to .500 in that regard.
This is where it is important to bring up what those in hockey circles refer to as “score-effects.” Generally, when one team is behind, they will play more aggressively, both naturally and often by design. On the other hand, a team with a lead is more likely to fall back into a shell to defend the lead. While frustrated fans seeing their teams blow a lead will often argue against it, there is evidence that in certain situations sitting on a lead is in fact the smart-money play. Limiting scoring chances for both sides can have its advantages.
When examining shot differentials, doing so in all situations is foolish. Considering score effects with the fact that power play opportunities are unbalanced, using overall shot counts leads to flawed conclusions. The most useful way to judge a team’s performance is with “score close” metrics, which only take into account shot and shot attempt differentials when the score is within a goal in the first two periods or tied in the third period.
Even just going off of shots on goal, and not Corsi or Fenwick — which I mentioned last time as being a larger sample and thus more conclusive — when outshooting opponents at even-strength with the score close, the Panthers are 8-5-1 this year. Since Fialkov wrote his article, they’ve won five straight games in which they’ve done so. Research has found that it takes 73 games for the impact of skill to roughly equal the impact of luck in determining the final standings. Even if Fialkov had used the appropriate statistics in formulating his hypothesis, he would have needed at least five times his sample size to make any kind of reasonable conclusion. Fialkov jumped the gun on this narrative, and based on recent Panthers results, he immediately got burned.