How good is Joe Hart? Why Goalimpact might not be a great way to assess keepers

Pep Guardiola seems not to be Joe Hart’s biggest fan. Since Pep’s unveiling as Manchester City’s new manager, rumours have abounded that a new goalkeeper is sought, with Inter’s Samir Handanovic, Barcelona’s Marc-Andre ter Stegen, and Real Sociedad’s promising Geronimo Rulli among those to have been linked with a move to England’s third city.

Advocates of the Goalimpact method of rating players – “Goalimpact rates a player by the goal difference of the player’s team when the player is on the field as compared to when he is not” – will find this surprising. Indeed, as of October 2015, Joe Hart ranked as the 5th best goalkeeper (down to 36th with all positions included) in the world behind Manuel Neuer, Andriy Pyatov, Victor Valdes and Petr Cech. It would be tempting, then, to announce Guardiola’s apparent lack of conviction in Hart as seeming analytically foolish.

As much as I love Goalimpact, I am sceptical as to its value for assessing goalkeepers. Reason being, Goalimpact really only assesses players within teams. Whilst this is true of all positions, with, say, Thomas Muller’s or Mesut Ozil’s impact on goal differences essentially calculated against their replacements when they are off the field, there is far more variation in the frequency of replacement players and of replacements themselves among outfield players. A goalkeeper, meanwhile, is compared only against the relative standard of their likely single replacement, with the gulf in strength between starter and replacement varying hugely from club to club.

These concerns are borne out by Goalimpact’s incompatibility with some of the valuations provided by metrics which assess specific goalkeeping actions – models which gauge goalkeeping proficiency between clubs. Whilst Joe Hart and Bournemouth’s Artur Boruc rate as the second and fifth best goalkeepers in the Premier League on Goalimpact, they are a considerable distance from this when one looks at the various components of goalkeeping that can be broken down (at least, those that have been broken down at this point in time) – be it shotstopping, susceptibility to making errors, dealing with crosses, or pass accuracy.

In terms of shotstopping, for example, Hart comes out fairly well – though Cech, Kasper Schmeichel, David de Gea, Vito Mannone, Adrian, and arguably Jack Butland and Tom Heaton over admittedly small samples, are ahead of England’s number 1 in Premier League terms. Artur Boruc, meanwhile, can be considered one of the worst shotstoppers to have graced the Premier League in the past few seasons – with Rob Green, Maarten Stekelenburg, Rob Elliot, and, of concern for Bournemouth, Adam Federici amongst Boruc’s competitors for this anti-accolade.

yay

If we look at the Errors per 90 (where committing few is obviously good) of the 20 keepers to have played more than 5000 Premier League minutes since 2012/13, Hart and Boruc appear to be decidedly mid-table.

Errors

When it comes to dealing with crosses (watch this for more on my method of analysis), neither Hart nor Boruc can be considered ‘aggressive claimers’ – keepers who come for more crosses than average, and, having come, are more likely than average to successfully claim rather than opting to punch, or failing to claim.

HartBorucclaim.png

In terms of a pass accuracy metric I looked at introducing for the Premier League, Bundesliga, and La Liga in the first half of this season, Hart was average, with Boruc again amongst the worst.

I understand that Goalimpact deliberately overlooks such individual metrics, but these discrepancies have to be concerning. Whilst it is possible that Hart and Boruc are doing other, non-measured things to a world-class standard, such as quick distribution, excellent throwing, and stellar defensive organisation, do these possibilities really cancel out known relative deficiencies in shot-stopping, error-making, dealing with crosses, and pass accuracy?

I still think Goalimpact is a great tool for analysing players, with a mix of ‘micro’ individual stats, and ‘macro’ tools such as Goalimpact perhaps the ideal means of making an initial data-based judgment on a player’s quality. I find it, however, a real stretch to consider either Hart or Boruc’s Goalimpact ratings justified. And, on the flip side, there are a couple of keepers – Lukasz Fabianski and Ralf Fahrmann spring to mind – favoured by the individual metrics who go unrated by Goalimpact. For goalkeepers, then, maybe Goalimpact just doesn’t really work.

Advertisements

2 thoughts on “How good is Joe Hart? Why Goalimpact might not be a great way to assess keepers

  1. Sam, am wondering what you think of the credibility of GI in evaluating players in a team outside of the keeper position.

    While I quite like the idea of shapley values as applied to football, it seems like the default rotation/substitution scheme used by managers breaks it (eg play the starters often, then bring in the subs once the game is decided or to kill minutes). There seem to be other confounders too – players with long careers having GI values less sensitive and relevant to recent form, inability to normalize out team deployment & strategic mistakes by managers.

    Like

    1. I think GI definitely has value. It’s got flaws of course, as you list, but I’d look to use Shapley values for outfield players (and especially for defenders where p90 metrics don’t work as well) as one of many tools in a recruitment process.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s