Over at Nylon Calculus, Will Schreefer shared some historical men's NCAA On/Off numbers covering 2009 to 2017. The numbers are available to copy via Google Docs, if you are so inclined.
I am, in fact, so inclined, so I copied Will's sheets and did some quick exploratory analysis. Schreefer provides a couple of interesting breakdowns including the offensive rating (ORtg) and defensive rating (DRtg) with each player on the court and then off, as well as the number of possessions with the player on and off the court. There are also columns that have the average Opponent ORtg and DRtg weighted by the possessions the player was in for each game, so if a player plays more against harder opponents or misses a scheduling cream puff, his opponent ORtg and DRtg numbers will look better than the team's overall opponent schedule might indicate.
The first thing I looked at is the year to year variation of different measures. To that I filtered out any players with fewer than 1,000 possessions on offense for each year (the defensive possession numbers are slightly different, but not enough to make any significant difference for the application of the filter). Then I identified the players individual years in the database by first year, first year plus 1, first year plus two and so on.
For this analysis I concentrated on the first year to second year relationships, which gave me a little over 4,000 players to analyze that played over 1,000 offensive possessions in both years. I looked at the coefficient of determination (R^2) for the Net On/Off numbers, the On numbers, Off numbers, as well as Opponent numbers.
There is only a very weak relationship between any of the Net numbers from year to year. Below is the graph of the Net Total plus/minus, which is plus-minus for the player on the court adjusted for the plus minus of the team with them off, showing a coefficient of determination of .017.
The numbers for Net Offense and Net Defense are pretty much the same. With so little relationship year to year, I would be very hesitant to read much into any potential draft prospect's Net On/Off numbers.
The raw ORtg and DRtg with the player on the court had stronger relationships. The R^2 for ORtg On year to year was .32 and for DRtg On it was .25 The On ORtg scatter plot is shown below:
Notably the R^2 for ORtg and DRtg for when the players in the study were Off the floor were lower, .16 and .12 respectively. This is in some part because of the smaller sample size with the players meeting my minimum possession count were off the court, but that just highlights the issue of getting a decent sample for On/Off comparison in a single year of college ball.(Average of ~ 1,350 for the On sample and ~ 800 for the Off sample).
Lastly, opponent ratings are very stable, reflecting that college teams play a pretty stable schedule from year to year. So I calculated Opponent Adjusted Offensive Ratings and Opponent Adjusted Defensive Ratings for every player. The calculation simply takes the On ORtg minus the Opponent DRtg (the lower the Opponent defensive rating the better the opponent's defense). For Defense the calculation is the Opponent's Offensive Rating minus the Player's On Defensive Rating, with the larger the number is the better. These numbers are more stable than the simple On ORtg and DRtg, and in a comparison of the entire NCAA population, at least, probably a better indicator than either the Net numbers or the raw On Ratings.
