I finally broke down and got ESPN Insider, just for the month, because I wanted to see if I could use Chad Ford's tier system as something of a better approximation of scouting information than the actual draft order. I had hoped that the tier system would leverage the 'wisdom of crowds' a bit and take advantage of the ability to rate players in a longer term perspective (there's a number one pick every year no matter how strong or weak the class), would make it more informative.
Unfortunately, the tier system doesn't go back as far as I had hoped, and what rankings there were appeared to be no more or less informative than actual pick selection. Oh, well...
But, it did give me a chance to look at Kevin Pelton's draft rankings and compare them to mine. Pelton's system is based on his own WARP player rankings. This was particularly interesting because Pelton also rate international players. With a quick copy and paste I grabbed his rankings and compared them with P-AWS. They turned out to be pretty highly correlated, with an almost 90% correlation for an R2 of .79.
The two biggest outliers were PJ Hairston, who was 1.27 standard deviations higher in WARP than in my ranking and Jerami Grant, who was 1.07 standard deviations lower in WARP. As far as Hairston goes, Thanasis Antetokoumpo, another D-League player in the draft, is also ranked higher by WARP. So, there may be a systematic difference in how we are weighting D-League experience.
I also decided to look at my rankings compared to Layne Vashro's rankings based on win shares, Expected Win Peak (EWP). There are a slightly higher, though not in any meaningfully so, correlation between P-AWS and EWP, with a correlation of .90 for an R2 of .81. There the biggest divergence was on Embiid, with EWP much higher on him and Doug McDermott, who P-AWS ranked higher.
And, for good measure, I compared WARP and EWP. Again there was a fairly high correlation, though a little lower than either had with P-AWS, .83 for an R2 of .695. There the biggest outliers were Kyle Anderson, with EWP being much higher and, again, McDermott was an outlier with WARP being much higher on him than EWP.
One additional note is that none of the three models ranked exactly the same players, so I used whichever players were in any two result sets.
Because much of the value in the draft is at the top of the draft, I looked at the number of top ten prospects shared by each projection model combo.
- EWP and WARP shared 7 of the top 10
- P-AWS and WARP shared 8 of the top 10
- P-AWS and EWP also shared 8 of the top 10 prospects in common, both having Aaron Gordon, Jarari Parker, Joel Embiid, Jordan Adams, Kyle Anderson, Marcus Smart, Noah Vonleh and Tyler Ennis. Though, my total top ten includes a few international players, namely, Clint Capela and Nikola Jokic.
Then I ran one more comparison between the three metrics, that paints a slightly different picture of the amount of agreement between the metrics. I looked at the average absolute difference (ie regardless of plus or minus) in rank of the shared prospects with each metric.
- Once again WARP and P-AWS were the closest, the average difference in rank was 7.5 and the median was only 5 places. Because both Pelton's WARP and my P-AWS model rank international and D-League prospects they rated 79 of the same potential draftees. Meaning that the average difference was 9.6% of the list and the median difference was 6.3% of the list. The average can also be calculated as a coefficient of dispersion as 19.3%.
- Between P-AWS and EWP the average difference in rank was also 7.5 and the median was 6, but since they only ranked 61 of the same prospects, the average difference was 12.3% while the median was 9.8% of the list.
- Between EWP and WARP the average absolute difference in rank was 8 and the median difference was also 6 places. With 60 prospects ranked in common the average difference was 13.4% and the median difference was 10% of the list.