There was a comment that was posted yesterday on the post "Did the Giants' Front Office Make the NL West Race Closer Than it Needed to Be?" that got me thinking. The comment was about how good of a stat wins above replacement is.
The comment came from Greg Wurz writer of the of the blog IN THE SHADOW OF BONDS, he said "I really don't know how much stock you can put in WAR."
So this afternoon I am trying to give Greg some piece of mind to how much stock can be put into WAR. I pulled the WAR Data for the Giants (I would have liked to do every team but I do have a day job and only an hour for lunch) and compared that to the Pythagorean Expected Wins and the actual wins for each team.
The Fangraphs data goes back to 2002 so this is the starting point and I looked at through last season (I didn't include this season because it isn't quite complete yet and I wouldn't want that to interfere with the results.) so a total of 8 seasons.
Here are the results:
WAR Predicted | Pythagorean Predicted | Actual Wins | |
2009 | 82.9 | 87 | 88 |
2008 | 76 | 67 | 72 |
2007 | 77.4 | 77 | 71 |
2006 | 78.2 | 76 | 76 |
2005 | 74.8 | 70 | 75 |
2004 | 88.9 | 89 | 91 |
2003 | 93.3 | 94 | 100 |
2002 | 101.6 | 99 | 95 |
Just looking at things with the naked eye suggests that they match up pretty closely. When you do a correlation you can see that both the Pythagorean and WAR predicted wins are highly correlated.
WAR Predicted | Pythagorean Predicted | |
Actual | 0.897051278 | 0.927742018 |
For a WAR even with the known issues a correlation of nearly 0.9 is very good especially when anything around 0.8 is considered to be strongly related. WAR can be improved especially in the measurement of defense and base running and even context specific results but even without these improvements it gives very robust results.
Fangraphs did a similar comparison looking at just last year's numbers and found very similar results of a correlation of 0.83. With that said I think that using WAR gives valuable information and I think that you can put a fair amount of stock into it.
Well thank you for the peace of mind! I do feel a little more peaceful!
ReplyDeleteThat being said I guess my point was when you are talking about a difference of 2.6 games it is tough to put stock in that and say the Giants would have a 3 game lead right now if Posey had started the season. Especially since we will never know if he would have come up and immediately produced the same way.
Also, in reading the fangraphs article they state that the average difference in prediction vs. reality is 6.4 games for a team WAR. How much of that difference is attributable to one player?
It's a debate that will probably rage between statheads for all eternity. But, for the record, I don't totally discount WAR but neither do I believe that it is an entirely accurate predictor of performance.
Especially when, in baseball, 6.4 games is the difference between first place and third place a lot of the time.