Christophe Lollichon is a man who knows a thing or two about the art of goalkeeping. In 1999 the Frenchman began an eight year period as the Goalkeeping Coach at Rennes, with a quarter of that time coinciding with Petr Cech’s stint at the Brittany club from 2002 until his transfer to Chelsea ahead of the 2004/5 season. Lollichon’s impact on the now-Arsenal goalkeeper was such that the pair stayed in constant contact when Cech departed for London, and the duo were reunited in November 2007 when Lollichon himself made the move to Chelsea. Widely attributed with having turned Cech into one of the world’s finest goalkeepers, Lollichon is a man whom we could reasonably expect to be worth listening to when it comes to judging what it takes to be a good goalkeeper.

In an interview from earlier this year with the French Football Weekly website, Lollichon offered an array of insightful comments on topics ranging from what it’s like working under The Special One to his opinion on Kurt Zouma. He also served up a line that the football analytics community should be careful to heed. According to Lollichon, “the modern-day keeper is not just a shot-stopper.”

This clearly makes sense, as goalkeepers have responsibilities in a number of areas that extend far beyond simply making saves. These can perhaps be boiled down to three fields: distribution, marshalling the defence, and dealing with balls in the air from crosses and long balls.

Despite this, if one was to learn about the beautiful game from only the web pages of football analytics blogs and websites then it would be all too easy to come to a viewpoint that the only thing of importance for a goalkeeper is whether or not they are a good shot-stopper – the difficulty of reliably measuring this notwithstanding (more on this later). A number of articles and blog posts by some of the leading lights of the football analytics community – the likes of Colin Trainor and Dan Kennett at StatsBomb, Johannes Harkins at the OptaPro Blog, and Paul Riley and 11tegen11 on their respective blogs – have assessed this facet of a goalkeeper’s game. To my knowledge, however, nothing has been written which specifically seeks to analyse the other aspects of a goalkeeper’s job.

One interesting exception is Jörg Seidel’s excellent Goalimpact blog, where a January 2014 post explicitly recognised the multi-faceted nature of a goalkeeper’s role before listing the leading goalkeepers according to their Goalimpact score. For newcomers to Goalimpact, this is a measurement of “the extent that a player contributes to the goal difference per minute of a team,” and it makes no bones about its deliberate ignorance of individual actions. Incidentally, this metric currently ranks Bayern Munich’s Manuel Neuer, Southampton’s Fraser Forster and Shakhtar Donetsk’s Andriy Pyatov as the Top 3 goalkeepers in the world.

In defence of those who have written on shot-stopping, many of the articles do, like Seidel on Goalimpact, commence with a preamble that recognises that making saves is far from the be all and end for a top goalkeeper. It’s also important to assert that these various attempts to ascertain the ins and outs of measuring shot-stopping remain extremely worthwhile, as it would be ludicrous to suggest that the making of saves is anything other than an integral part of a goalkeeper’s job. It remains the case, however, that aside from Goalimpact, the overwhelming majority of analytics articles on goalkeepers extend their statistical gaze no further than the realm of shot-stopping. It would seem, then, that in the manner of a farmer looking to maximize his revenue stream, the field of goalkeeping analytics could do with a little diversification!

This process, however, is much less easy than it would ideally be. Whilst data on the amount of saves that a goalkeeper makes and the amount of goals that a goalkeeper concedes is fairly easy to come by – the NBC Sports website, for example, has collated data on saves made and goals conceded going back some eight years – the distributional, defence marshalling, and aerial prowess of a goalkeeper are all, for one reason or another, comparably difficult to assess.

Firstly, let’s consider distribution. Whilst kicking and throwing are very important aspect of a goalkeeper’s game, there are huge question marks over how exactly to quantify what does and doesn’t qualify as good distribution. There are so many factors and variables at stake that would affect its measurement, including, but not limited to, questions such as is kicking long always the best option? Does the goalkeeper’s team like to play a possession or a counter attacking game? Are a given goalkeeper’s teammates good in the air? What about at making off the ball runs? Combining these queries with the lack of publicly available data on goalkeeper distribution, I have great sympathy with the lack of analytics research in this domain.

Secondly, we turn to a goalkeeper’s defence marshalling. Again, this is a massive part of the goalkeeper’s role, with Lollichon moved to describe goalkeepers in his interview with French Football Weekly as “air-traffic controllers” such is the extent to which they are relied upon to coordinate a team’s defensive structure. This aspect of goalkeeping is even more difficult to decipher than is distribution. How on earth would one measure the aptitude of a goalkeeper in this regard?! Judging a goalkeeper by the total amount of goals or chances that they conceded would be fallible as, whilst the goalkeeper certainly has a degree of responsibility for directing the defence, the varying abilities of defenders would obviously play a role here. No matter how good a goalkeeper’s direction, if their defenders were enormously error-prone then the goalkeeper would, by this measurement, be reflected unfairly poorly. I’m not convinced that there could ever be a viable metric by which a goalkeeper’s communication could be assessed. The closest we’ll probably ever get is the holistic Goalimpact method, where communication gets lumped in as just one of the many strands of goalkeeping.

So how about measuring aerial prowess? Can we do this? By aerial prowess I mean how well does a goalkeeper deal with the various crosses and long balls pumped into their box. A goalkeeper who picks off high balls in the box with ease would nullify the aerial threat of a heading or crossing specialist in the opposition ranks and would consequently, I imagine, be quite a valuable commodity. Less so for a goalkeeper who panics and blunders their way up to high balls before fumbling them haplessly at the feet of opposing strikers. In this respect, aerial prowess looks a lot like shot-stopping in the way that we may contemplate analysing it – the reason why this has yet to be done, according to Kennett, owes to the fact that data for claims made has only recently made its way to the public domain. Whilst analysing shot-stopping data often commences with an assessment of Save% – a controversial metric – where the proportion of saves that a goalkeeper makes from the shots that they face is simply tallied up, so too may Claims% assess the proportion of aerial balls that a goalkeeper catches from those that they make an attempt to claim. This is where this blog post will begin.

Before launching enthusiastically into an analysis of the Claims% of different goalkeepers, it would be prudent to first assess the contention that has sprung up in the Save% literature, as the same issues are likely to be manifest within the Claims% data. Save% is almost certainly a fairer way by which the shot-stopping abilities of goalkeepers may be analysed than the traditionally compared individual stats of Clean Sheets, Saves Made and Goals Conceded where a goalkeeper’s stats are extremely reliant on the quality of the defence that they find in front of them. Save%, in theory, overcomes these limitations, although it remains a far from perfect metric.

11tegen11 has written perhaps most critically on the issues that surround Save%, and has pointed out that the repeatability of a goalkeeper’s Save% from one season to another is dismally lacking. This is an issue, as a lack of repeatability, according to the Opta Pro blog, is the death knell of a statistical analysis – unrepeatable findings can be considered little more than a ‘false prophet.’ As far as the unrepeatability of Save% is concerned, this is a result of the huge variation in shots that goalkeepers may face. Shots are not created equal – those rifled towards the top corner are plainly more difficult to save than are tame efforts that trickle welcomingly into the goalkeepers arms.

Over time, however, randomness in any given sample will become more evenly distributed. Along these lines, Riley has suggested that “in time, it’s [shot-stopping] a skill that shows through.” He thus applies the funnel plot method used by Kennett (re-produced below), which indicates that when the sample of shots faced is of a substantial volume, some goalkeepers can be seen to routinely make a higher proportion of saves than others. A funnel plot, in the words of Kennett, shows “how randomness decreases as the sample size increases” – suggesting that those who fall outside of the funnel may do so not necessarily because of luck.


So what of the Claims% data? The following graph plots the Claims% against the Claims Attempted for every goalkeeper in Europe’s Top 5 Divisions (Premier League, Bundesliga, La Liga, Serie A, Ligue 1) who played over 1350 minutes (15 matches) of football in 2014/15 and attempted to make a minimum of 50 claims (data from Squawka). This throws up a sample of exactly 100 goalkeepers who cumulatively claimed 8528 of the 8768 balls they attempted, at a mean rate of 97.263%.

1415 claims

The graph offers a few interesting points for analysis. Firstly, it is clear that there is a differential of around 12% between the three goalkeepers who returned a Claims% of below 89% and the 23 who went a whole league campaign without spilling a single of the balls that they attempted to catch. Clearly there is a sizeable variation in the success rates with which goalkeepers manage to claim balls in a given season. Secondly, what on earth was it that led Swansea City’s Lukasz Fabianski to attempt to claim 51 more balls than the second most prolific attempt-maker Sassuolo’s Andrea Consigli? This is an enormous gap. Although Fabianski played 182 more minutes than the Italian stopper throughout the course of the season (3307 minutes vs 3125), there were 18 goalkeepers who played more minutes than Swansea’s No. 1. Besides, this approximately 6% gap in playing time between Fabianski and Consigli is a long way from accounting for the 35% gap in their attempted claims.

This data would probably lead us to consider that Guingamp’s Jonas Lossl and Lorient’s Benjamin Lecomte were Europe’s safest aerial pairs of hands in the 2014/15 season, by virtue of their respective 100% and 99.213% success rates over a fairly hefty sample of attempted claims. Before coming to such a judgement and recommending that scouts take a look at Lossl and Lecomte, we must consider repeatability. How likely is it that Lossl and Lecomte will back up their 2014/15 performances, and once again turn in impressive claiming stats in 2015/16? And, at the other end of the Claims% data pile, how likely is Julian Speroni to have another season in which he drops 11 in every 100 balls that he attempts to claim (provided, of course, that he keeps Alex McCarthy and Wayne Hennessey out of the Crystal Palace starting 11)?

The answer is very slim. This is evident if we plot a graph using data for the goalkeepers in Europe’s Top 5 divisions to have played back to back seasons (either 2012/13 followed by 2013/14 and 2013/14 followed by 2014/15) in which they attempted to make a minimum of 65 claims.

previous season claims

Indeed, the first season efforts only explain a miniscule 3.6% of the variation in the second season Claims%. I have also added a y=x line, along which we would expect the plots to be closely gathered around if the data of one season was to be a good indicator of what would come the next season – plainly it isn’t, with Liverpool and then Napoli’s Pepe Reina and Reims’s Kossi Agassa both having turned in 2012/13 seasons in which they attained Claims% above 96% before sinking below 90% for 2013/14. Speroni and PSG’s Salvatore Sirigu, meanwhile, went the opposite way between 2013/14 and 2014/15.

This suggests that Claims% is a metric with a serious lack of repeatability. One season’s worth of data appears to be an insufficient basis upon which to make predictions regarding a goalkeeper’s future Claims%. As there are 26 goalkeepers in the sample who attempted a minimum of 65 claims in three back to back seasons (2012/13, 2013/14 and 2014/15), it is possible to test if an amalgamation of two seasons worth of data offers a better indication of things to come. Along these lines, I have re-done the previous scatter plot where the 2012/13 and 2013/14 data for the 26 goalkeepers is amalgamated into one super-sized season, with each of these goalkeepers ‘first season’ data now standing between 156 for the lowest and 291 for the highest attempted claims. This offers up the following scatter graph.


With an R² this time equalling approximately 0.08, there is clearly a reasonable increase in the relationship between goalkeeper’s Claims% for 2014/15 and the amalgamation of the two years that went previously. The correlation coefficient of this data is 0.28, which can be defined as anything between weak and modest depending on whose definition is applied. It’s important to note that there isn’t zero relationship or correlation, which would mean that there was no repeatability whatsoever in Claims%, and that goalkeepers gained their percentages through nothing more than luck.

8%, however, is still a low level of explanation, with a lot of the data plots a substantial distance from the y=x line around which the points would cluster if there was a strong relationship. Sirigu is particularly offensive in this instance, with his data for 2012/13 and 2013/14 combining to have offered him a Claims% of 97.849% for the super-sized season, which he followed up in 2014/15 with a Claims% of just 88.235%. If Sirigu was to be arbitrarily deleted from the sample as an outlier, this relationship improves to an R² of 0.18, with a correlation coefficient of 0.42 – safely within ‘moderate’ territory. Whilst clearly a large increase, this would still mean that even when knowing a goalkeeper’s Claims% from the two previous seasons, 82% of the variation in their following-season’s performance is unaccounted for. The repeatability is certainly a long way off being strong.

This gives Claims% a huge problem, and suggests that 11tegen11’s reservations about Save% may be identically applicable here. If two seasons of data is an insufficient predictor of a goalkeeper’s coming season, might it be that three seasons worth of data will begin to offer an indication that some goalkeepers repeatedly manage a higher Claims% than do others? It’s in such an instance that Kennett’s funnel plot came in to restore some credibility to the Save% metric, so the same statistical technique will be used here with control limits at 95%. There were 97 goalkeepers who played a minimum of 4050 minutes (45 matches) between 2012/13 and 2014/15 in Europe’s Top 5 Leagues, with a cumulative 20,031 successfully completed claims from 20,614 attempts – an average success rate of 97.172%. This produces the following plot.

funnel 1

As with the Save% data, it appears that over time, randomness in Claims% is reduced as the funnel moves ever nearer the mean as claim attempts increase. Accordingly, we may zoom in on the 21 goalkeepers who attempted a minimum of 250 claims across the period and locate three who appear to have consistently been good at claiming balls, and two who consistently fell a fair way below the average level. When the sample is big enough, it might be possible for scouts to deduce certain trends from the Claims% data, and accordingly consider that this quintet is likely to continue over or under-performing. So who are they?

funnel zoom

It would seem, then, that as far as aerial prowess is concerned, Chelsea’s Thibaut Courtois – currently coached by Lollichon – is apparently deserving of his reputation as one of the best goalkeepers in the world (not that this has anything to do with him being nearly two metres tall). Opponents of Liverpool and Marseille may also be interested to note the difficulty with which this duo’s goalkeepers – Simon Mignolet and Steve Mandanda – have claimed balls over the past three seasons.

To conclude, then, it would seem that if the sample is big enough, Claims% may not be an entirely useless indicator of a goalkeeper’s aerial prowess. There are clearly serious issues, however, with attempting to predict how well a goalkeeper will fare in the air based on one or even two previous seasons worth of data. With Lollichon correct to recognise that a goalkeeper is ‘more than just a shotstopper,’ the football analytics community still has a long way to go to come up with metrics by which goalkeeping performances may be fairly judged in all their variety.


Featured Image: Christophe Lollichon shares his wisdom with Petr Cech and Thibaut Courtois.

Source: Chelsea FC, August 2014. (http://www.chelseafc.com/content/dam/cfc/news/latestnews/14/08/thibaut-courtois-petr-cech-training-lollichon-happy.jpg.)



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s