Analytics Philosophy: Low Ceilings and Bottomless Floors

What to make of counterintuitive findings in the world of soccer analytics

One of the issues with the general uptake of soccer analytics across the spectrum of stakeholders is the struggle with certain counterintuitive claims. These claims often defy what many people intuitively understand about the game. They’re also a little bit exaggerated in order to get attention, but at the core point to an important truth. I’m not going to spend too much time on establishing these claims are true; that’s done better elsewhere.

Instead, I’m going to look at two claims on a philosophical level and why the general perception (what most people call “the eye test”) and the numbers don’t seem to agree. There is something that these two claims have in common; a bottomless individual skill floor and a fairly low ceiling at the highest level.

Claim 1: Coaching doesn’t matter

The relative impact of coaching and whether or not a certain individual is a “good” coach or not takes up a massive amount of media space. A few months into every season, the carousel starts spinning as underperforming coaches are fired and new ones are brought on. At the same time, there’s not a lot of evidence that coaches can do all that much to impact the game.

So what are the essential functions of a coach?

1. Man management

2. Game model

3. Player selection

4. Organizing training and periodization

5. Media magnet

6. Scapegoat

Each of these contributes in various ways to the success of a team, to varying degrees. A manager who is bad at any of the initial functions sticks out, though. Each of these functions requires professionalism, organization, and sensitivity. These aren’t common skills and require practice and experience. It’s also noticeable when they are lacking.

At the same time, the pressures of the job very quickly filter out any manager who can’t cut it. Every now and then there is a manager - usually a former player, who gets to skip to the head of the hiring line - that is a true disaster in one or more of these areas. The media then gets to tell breathless tales of practice disorder and player mutiny. A bad manager can ruin a team, even if the individual players are good. The floor is bottomless.

Bad managers, however, are rare for a reason.

Therefore, it stands to reason that the reason managerial impact is so hard to determine based purely on the points on the table is not that the manager doesn’t matter; it’s just that the diminishing returns between two competent professionals are washed out by the other more important factors - specifically how good the players are.

A good case study of this is the short lived tenure of Wilfried Nancy at Celtic. Nancy had earned a deserved reputation as one of the better managers in MLS, winning an MLS Cup with the Crew and making the finals of the CONCACAF Champions Cup. Prior to that, he succeeded Thierry Henry at CF Montreal and managed to stabilize one of the league’s hot mess franchises. Despite his record, he failed at Celtic and departed the club after a disastrous 33 day spell.

What went wrong? The way I see it, the blame falls a few ways. Nancy is a dogmatic coach with a clearly defined style that demands a lot from his players technically and tactically. It’s an extremely high possession style that requires intricate rotations to break down opponents. When the model falters, it can struggle to create chances while allowing transition opportunities the other way. Nancy clearly did not have the players to execute his vision, and joining mid season put him in a tough spot.

At the same time, the Celtic leadership had to have known that. They HAD to have known they were hiring an idealist and that the timeline might be stretched. If they wanted a pragmatist, Big Sam wasn’t busy, call him instead. By punting on Nancy before a transfer window, they reversed course not because the results were untenable but because they were cowards.

That brings us to the last function of the coach - as a scapegoat. Despite the fact that Nancy’s results were trending up as the team figured out what was required, he was sacked. Even though his team’s underlying numbers were better than caretaker manager Martin O’Neill’s, the fans could not look past the scoreboards. To quiet the mob, Nancy was fired.

Many Celtic fans - particularly the ones you might run across on X or Reddit - are stupid dinosaurs, incapable of making sense of a game that has passed them by, but that’s their right. The Celtic leadership knew what they were getting into, and still quailed when they were faced with a turbulent patch. When that happens, coaching can’t matter.

Claim 2: Finishing isn’t Real

This second claim has been one of the most controversial in the analytics era. It’s an exaggeration, of course, but at the core it reflects something real; that individual technical ability has relatively small impact on goal scoring numbers.

I mentioned that the loudest Celtic fans are dinosaurs, but they’re hardly alone; and nothing riles the dinosaurs up more than the idea that kicking the ball good didn’t really matter all that much. In fact, even as xG becomes more adopted many people (even smart ones!) cling to the idea that outperforming xG is a sign of player quality, and that the expectation should be that it continues.

The reason that finishing as an idealized trait is so hard to exterminate is that it defies common sense. Pretty much every soccer fan that has experience on the field knows that finishing is important. My own experience is that I’m a terrible finisher; when I get clear in front of the goal my vision blurs, my heart hammers in my ears, and all my technical skill goes out the window. The times I do score it feels almost like an accident. That certainly plays a role in my (dismal) goal scoring performance, and I can guarantee that it is likely to continue. The vast difference in ability between a professional and a good amateur is so stark, how could it not matter?

Again, it comes down to ceilings and floors. The floor for finishing is bottomless; even the average amateur would be unlikely to score at the highest level even if given the same quality shots. At the same time, to reach the highest level, players are practicing thousands of not tens of thousands of shots, in all kinds of competitive contexts. The differences between them are so much smaller, so the driving force becomes the volume and quality of chances and the finishing factor fades to almost nothing. The margin between a 99th percentile finisher and a 1st percentile finisher might be a few goals in a hundred games; the difference between a 99th percentile forward for chances earned and a 1st percentile forward is a few goals over ten games.

What’s the point?

These claims are always going to meet resistance because they go against the grain of “common sense”. The point of this article is to underpin just why these claims are counterintuitive, but still valid. By understanding the fundamental idea - that an observable skill gap can have diminishing impact the higher the level - it can make the people who watch and the ones who make decisions a little more patient, understanding, and wise.

Without that perspective, you get arrogance and foolishness - a belief in the predictive power of things that hold absolutely no weight. It’s the same force that turned Manchester United from a legendary powerhouse into a joke, and it’s coming for clubs like Celtic too.