Mind the Gap: Are Championship goalkeeping performances transferable to the Premier League?

The financial gulf between the Premier League and the Championship is substantial. So substantial, in fact, that there is an entire Wikipedia page devoted to the very topic. Every May, the Championship Play-Off final is billed as a match with rich rewards for the victor, and this year’s BBC’s preview was no different. According to the BBC, a cool £120m was at stake for the triumphant team in this year’s most literal of winner-takes-it-all encounters.

That team, of course, turned out to be Norwich City – courtesy of a goal each for Cameron Jerome and Nathan Redmond inside the first 20 minutes of the Canaries’ Wembley clash against Middlesbrough. With 44% of the clubs to have been promoted to the Premier League since 1992 immediately relegated, it would seem that the gulf in wealth is accompanied by an equally sizeable chasm in footballing quality. Not so, according to Norwich midfielder Gary O’Neil – a veteran of nine Premier League campaigns with Portsmouth, Middlesbrough and West Ham – who informed the Eastern Daily Press in the aftermath of his team’s promotion that “the bottom 10 in the Premier League aren’t much better than the likes of us and Watford.”

Financially, the gulf certainly exists. Whilst Norwich were the Championship side whose squad had the biggest cumulative market value in 2014/15 at £57.65m (data from transfermarkt), this is a figure smaller than for eighteen of the Premier League’s squads in 2014/15. When we inspect the per player market values of the squads of these respective leagues, only four of last season’s Championship sides exceeded the per player value of Burnley – the Premier League’s least valuable team. The values are reproduced in the following tables, and it is apparent that of the ten lowest valued teams in the Premier League in 2014/15, the vast majority were still a considerable way ahead of the Championship’s elite.

market values

As the ‘League Finish’ column of the tables indicate, football is of course about far more than whose squad has the highest value, with the transfer market a notoriously inefficient part of the game (see Simon Kuper and Stefan Szymanski’s excellent book Soccernomics for more on this). Whilst Chelsea, the Premier League side with the highest valued squad in 2014/15, won the league, and Burnley, the cheapest side, were relegated, there were a huge number of teams in both the Premier League and the Championship for whom market value was a devastatingly poor predictor of whereabouts in the league they would go on to finish.

AFC Bournemouth offer perhaps the starkest example of this, as they won the Championship despite having a squad with a valuation inferior to all but three of the league’s 24 teams. In the Premier League, meanwhile, Swansea City were valued as only the 16th most expensive team, yet finished the season in 8th. Back in the Championship, Wigan Athletic were the team with the 3rd highest value, and yet in 2015/16 they find themselves battling minnows such as Fleetwood Town and Rochdale in League One.

Despite the financial gap between the leagues, then, O’Neil may well be right. Perhaps the quality of the football in the Premier League isn’t stratospherically better than that offered in English football’s 2nd Tier. Thankfully, we are able to compare the difference, or lack thereof, in the output of players that have performed in both divisions. The Championship-specialist analytics blogger Ben Torvaney has looked, on his Stats and Snakeoil blog, at what a move from the Championship to the Premier League typically does to a player’s attacking output. He in turn references this piece by Ted Knutson (with Knutson, as many readers will know, at the heart of the ‘data revolution’ currently occurring at Brentford – well explained by Tom Worville in three parts accessible here), and has found that a “high-volume shooter” can be expected to translate approximately 80% of their Championship performance to the Premier League. This means that a 20-goal-a-season striker in the 46-game Championship should be able to turn in around 13 goals in a season spent in the 38-game Premier League.

Chris Anderson and David Sally have noted in The Numbers Game that sport analytics tend to be biased in favour of assessing attackers, so it isn’t surprising that, as far as I’m aware, these efforts from Torvaney and Knutson, rooted as they are in analysing attacking play, are as far as the analytics community has gone in attempting to measure the difference in output between players who move from the Championship to the Premier League. As regular readers of this fledgling blog will be aware, this unevenness in the world of football analytics means that analysis of goalkeepers is at a particularly primitive stage – thus offering up a gap in the market that I am only too happy to try and exploit! So what happens if we take a look at the data for goalkeepers who have made the leap between the Championship and the Premier League? It will be interesting to test if the 80%-rule is ubiquitous across all positions. Let’s delve into the data and see if it exists amongst goalkeepers.

Of course, this is easier said than done. As I noted in my most recent blog post, goalkeeping analytics is an emerging field. Whilst metrics such as Save% and Claims% are easily calculable, they are heavily flawed in that the data of one or two seasons for any given goalkeeper is not particularly repeatable – a cardinal sin in analytics. Following the research of the likes of Paul Riley and Dan Kennett, however, I have been persuaded of the view that over a lengthy enough period of time, Save% and Claims% do become adequate metrics of a goalkeeper’s handling ability.

Unfortunately, data (I use Squawka) is only publicly available for Championship goalkeepers for the past-two seasons. This means that looking at an individual goalkeeper who moved between the Championship and the Premier League in this period could only be done on extremely shaky statistical grounds, and would be no basis upon which to infer the difference in difficulty, or lack thereof, between goalkeeping in the Championship and the Premier League. In any case, there are only three players for whom data is available for a season in the Championship followed by one in the Premier League – Rob Green, Tom Heaton and Kasper Schmeichel, the goalkeepers of Queens Park Rangers, Burnley and Leicester City respectively, who were of course the teams promoted to the Premier League ahead of the 2014/15 season. So how can we overcome these statistical shortcomings? Is there a way? It’s time to get creative.

As I noted in my blog post on the repeatability of Claims%, I tentatively made the suggestion that once a goalkeeper has played three seasons, their % data becomes a semi-reliable indicator of their ability. If we blend Green, Heaton and Schmeichel’s Championship data, we are essentially presented with a composite goalkeeper with three seasons of Championship football under his belt. And again, if we blend the trio’s data for the following season in the Premier League, we produce a composite goalie that’s a veteran of three Premier League campaigns. If we compare the Save% and Claims% of the Championship composite with the Premier League composite, might we be able to gain an early indicator – with an admittedly tiny sample – of the step up, or lack thereof, that promotion to the Premier League entails for a goalkeeper? Let’s have a look at the numbers.

promoted composite

It appears, then, that it is harder for a goalkeeper to attain a high Save% in the Premier League, with Green, Heaton and Schmeichel all having experienced a fairly significant decline in their Save% once they were promoted. Their data combines to offer the composite goalkeeper a decline of approximately 6% in his Save%. In terms of a % of output, 63.357% is 91.7% of 69.133% – an indication that the 80% rate for strikers highlighted by Torvaney does not carry over to goalkeepers.  It is curious, however, that both Green and Schmeichel improved their Claims% in the Premier League whilst Heaton weighed in with an ever so minute decrease, with the composite goalkeeper accordingly having improved at claiming aerial balls by 0.5% on stepping up from the Championship to the Premier League. This would suggest that a goalkeeper’s Claims% is unaffected by moving from the Championship to the Premier League, whilst their Premier League Save% will be about 92% of their Championship output, although the sample is, as I have noted, very small indeed.

Of course, teams and their goalkeepers do not just ascend from the Championship to the Premier League. Relegation befalls three teams a year and some goalkeepers have spanned the gulf between the divisions courtesy of the transfer market, so we may test, in an identical compositional manner, what effect playing in the lower division after a season in the Premier League has on a goalkeeper’s Save% and Claims%. This will increase the sample size and see if the early indication of a negligible effect on Claims% and an ~8% effect on Save% from moving between the Championship and the Premier League holds true.

Squawka data goes back one year further for the Premier League than for the Championship, so rather than just testing those who played in the Premier League in 2013/14 followed by the Championship in 2014/15, we are also able to test the data of any goalkeepers who played in the Premier League in 2012/13 and then in the Championship the following season. There are five goalkeepers ripe for analysis in this manner, with Ali Al-Habsi relegated with Wigan Athletic from the Premier League to the Championship at the end of the 2012/13 season. David Stockdale and Artur Boruc, meanwhile, played for Fulham and Southampton respectively in the Premier League in 2013/14, before transferring to the Championship sides Brighton and Bournemouth for 2014/15, whilst John Ruddy and David Marshall stayed with Norwich and Cardiff City upon their sides’ relegation to the Championship in the same year. The data and composite data for these five goalkeepers who moved in the opposite direction to Green, Heaton and Schmeichel is as follows.

relegated composite

A similar pattern appears to be present amongst the relegated goalkeepers as there was for the promoted trio. The gap in the composite’s Save% between divisions is this time slightly larger at 7%, whilst the composite’s Claims% again appears negligible, with it this time 2.2% in favour of the Championship being easier. As a % of output, the composite’s Premier League Save% performance is around 89% of that managed in the Championship, with the composite’s Premier League Claims% performance standing at approximately 98% of the Championship level.

If we combine the composites, and ignore the direction in which the goalkeepers have travelled between seasons, we are presented with the following overall impression, which compares the eight goalkeepers for whom we have data that have straddled the gap between the Championship and Premier League in consecutive seasons.

composite

The Save% for the composites in this case is 6.5% lower in the Premier League than in the Championship, with this representative of goalkeepers managing, on average, to reproduce 90% of their Championship Save% in the Premier League – an indication borne out from over 50,000 minutes of football. In terms of Claims%, approximately 99% of the Championship claims are reproduced, suggesting that a decline is negligible, if it exists at all.

Before we conclude that a Championship goalkeeper will only be likely to translate 90% of their saves output in the Premier League and thus concede one more goal for every ten shots that they face, it is worth making note of one of the problems that affects Save% as a metric, with Garry Gelade having noted at the OptaPro blog that “weaker teams… face on-target shots that are harder to save. Failing to account for this will lead to a systematic bias; goalkeepers in weak teams will be underestimated and goalkeepers in strong teams will be overestimated.”

Let’s account for this! If goalkeepers that play in weaker teams face harder shots, then the goalkeepers to have played for relegated teams in this article – the likes of Al-Habsi, Marshall, Ruddy and Stockdale (Boruc wasn’t relegated with Southampton) – will likely have artificially low Premier League Save% stats. And, with this inversed, Green, Heaton and Schmeichel’s Championship stats, as they were promoted, are likely to be artificially high. Could these skewing factors account for the entirety of the goalkeeping output gap between the Championship and the Premier League that has apparently been exposed here?

If this is indeed the case, then every team in Europe should look to the Championship upon the culmination of this 2015/16 season at which point three seasons of data will be available, and promptly snap up the goalkeepers with the best Save% across these seasons who also have played for the league’s weakest teams, as these goalkeepers would not be likely to suffer any decline in their performance upon moving to the Premier League. This would mean that the Championship is a haven for cut-price Premier League-ready goalkeepers. This strikes me as being fairly unlikely.

The entirety of the output gap is thus unlikely to be explained by the skews highlighted by Gelade. It would seem to be a fair assumption to make for teams gazing at the Championship for potential goalkeeping recruits to factor in the idea that shot-stopping is an art just a shade less than 10% harder in the Premier League than in the Championship. This is a hypothesis that has been drawn from a very slender sample, and I look forward to re-testing it upon the culmination of this season to see if the Save% performances of the newly promoted Ruddy, Boruc and Watford’s Heurelho Gomes average out to ~90% of last season’s level, and if Hull City’s Allan McGregor along with Heaton and Green turn in Save% data in the Championship an average of ~6.5% higher than their 2014/15 Premier League efforts. In any case, it is clear that the 80% translation rate of Championship attacking performances as suggested by Torvaney is most probably not applicable to goalkeepers. Outsiders they remain.

***

Thanks for reading! If you have any queries, criticisms, or any other comments, please do get in contact either in the Comments section or on Twitter @sam_jackson94!

Featured image: John Ruddy denies Djibril Cisse from the penalty spot © PA.

Image source: http://www.dailymail.co.uk/sport/football/article-2194026/Referees-win-situation–Graham-Poll.html.

*S+GC (Saves+Goals Conceded) is a proxy for the number of shots which a goalkeeper faces. I use this number to calculate Save% owing to limitations in the data provided by Squawka. I calculate Save% as Saves/S+GC. Although some goalkeepers may miss certain shots on target which will be cleared off the line but which should be counted as a failed save, and some of the goals that are conceded will not stem from saveable shots on target (e.g. own goals), this is the best I can do with the publicly available data.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s