Counting the Baskets

Some Play by Play Numbers by Age and Height

As part of some model prep I have been working on for a possible new draft model I have been exploring some play by play data. As a part of that, I wanted to do a quick hitter on those numbers. I was able to use the data via Will Schreefer to look a little more at how players score, how that changes as they age, how it changes by height and how guys who eventually become NBA players differ from those that top out at a lower level.

The two breakdowns I was interested in for this were the split between transition and half court scoring and unassisted scoring vs points coming from an assist. Transition ratio is simply the points a player scored in transition divided by their total point scored. Creation ratio is the unassisted points from the field divided by assisted points from the field, as the play by play data does not track free throw assists.

First thing is that Transition Ratio goes down slightly by age in college, while Creation Ratio tends to be even more stable:

The slight decline in Transition Ratio is due to a larger increase in half court scoring, not a decline in transition scoring. Meanwhile it's somewhat interesting to me that on average, at least, college players are able to increase their assisted scoring numbers at a comparable rate to their self created scoring.

Then I looked at the numbers by player height. As you can see below there is a clear downward trend in percent of points scored in transition by height, and a persistent higher percentage of points scored in transition by NBA bound college players, when height is accounted for.

There is a similar downward trend in Creation ratio by player height. Again the future NBA players tend to create a higher percentage of their own points when accounting for height. However here the gap pretty much disappears for taller players. For guards,showing the ability to sore with the ball in your hands at the college level is very important to be able to get into the NBA.

 

A couple of last nuggets, I was interested to see that there is a bigger gap on Transition ratio between NBA bound bigs and non-NBA bigs bound than there was in the Creation ratio, while for guards is was the opposite. The biggest growth in raw transition scoring per forty minutes by a good distance were the guards, and especially those who eventually make the NBA.

  

A New Benchmark System

 

Last year I explored a benchmark system for draft prospects by Ed Weiland. The benchmarks track whether a prospect reaches minimum statistical marks in a number of different statistical categories. In looking at Weiland’s benchmarks, I found that the benefits of a benchmarking  system is that it allows one to highlight potential red flags and rewards versatility in a way most draft models do not. The downside is the loss of information with a simple pass fail test on the benchmarks, outstanding efficiency is treated the same as adequate efficiency. Another important factor maybe the simplicity in conveying the information, meeting or not meeting specific benchmarks may be easier to convey to less analytically inclined readers than the output of a regression or ML model.

This year I am introducing my own benchmarking system designed to work without picking a position for every player, a complication that can make the system less reliable for players with positional versatility. I am using four benchmarks:

• Scoring- Does the player score 21 points per 80 possessions?
• Efficiency- Does the player’s two point percentage and free throw percentage add up to 1.25? (Allows positional diversity, players need to demonstrate ability to score inside monstrously efficiently or some evidence of ability to shoot).
• Offensive Activity- Do assists plus offensive rebounds equal at least 6? (Strong negative correlation between the two, indicates trade off of size and skill)
• Defensive Activity- Do steals plus blocks equal 2.5 or more?

Comparison to Draft Model

While the point of the benchmarks is not to displace or mimic the methodology of a trained model, I did look at both the individual benchmarks and the total percent of benchmarks met against the data used for my draft models. In both cases the benchmark data, combined with age, had significant positive correlations to measures of success in the NBA (though less than more detailed model). Offensive Activity showed the strongest relationship followed by Defensive Activity, Efficiency and Scoring in descending order. Also notable, when combined with the data for the draft models, percent of benchmarks reached had a modest positive effect, consistent with other measures of versatility I have tested in the past.

In a boosting regression using the two models as predictors the relative importance  in predicting power was 80% to the trained model and 20% to the benchmarks. (Measure used was the max  performance in years two through four as measured by a combination of box score stats and RAPM).

Examples from this Prospect Class

Looking at the bench marks applied to this year’s top prospects should give a little better idea of how they work and what information they can reveal.

There are six major prospects that make all four of the benchmarks, DeAndre Ayton, Luka Doncic, Wendell Carter, Shake Milton, Josh Okogie and Gary Clark.

Doncic, Ayton, and Wendell Carter are well known to anyone even casually following the draft. Milton and Clark are the older prospects on this short list, which takes a little of the shine off of making the benchmarks. All of the prospects that meet all of the benchmarks also rate reasonably well in the traditional model, with all in the top 30 by the traditional model rank.

On the other side are the three significant candidates that fail to meet a single benchmark, Lonnie Walker, Sviatoslav Mykhailiuk, and Hamidou Diallo, with the highest rated prospects to fail to meet all four benchmarks being Walker and Diallo, both rated in the first round by ESPN. Walker is the only one of these prospects to rate in the top 40 via my traditional model.

Age is the big factor not explicitly addressed by the benchmarks. A 23 year old hitting all four benchmarks is still not necessarily a strong prospect, and an 18 year old only hitting two is not necessarily a non-prospect. In the attached link, I added a column that factors the prospect score by age in order to get a more approximate evaluation,

 

Summer League Rookie Updates

I couldn’t quite wait for Summer League to be over officially, run the numbers to see what, if anything, we can take from the rookies’ performance. But with the Lakers sitting virtually everyone of note for the Vegas Championship, I figured it’s over enough. (Note to the NBA: Vegas Summer League is too long, this is why teams start sitting their lottery talent).

In the link here, I have the full run down of per 40 one number performances via Kevin Ferrigan’s Daily RAPM Estimate (DRE) and Alt WIn Score (AWS), a metric I use quite often in my draft models with data from RealGM. But, to me, the real focus of Summer League has to be the rookies.

Some of that is informed by a Kevin Pelton ESPN Insider article, indicating that Summer League adds a bit of predictive power to rookie performance but not to 2nd year players. It’s also plain that we just have less information on rookies, so new info relatively more valuable as is seeing them in a new team setting.

In order to get a quick and estimate on evaluating the stats out of SL, and maybe providing a bit of perspective, I re-calculated my Rookie performance draft model, first by substituting the SL stats for the college or European league stats and then used the SL stats to just update my original Rookie Model.

First the Summer League Only (SLO) run. The SLO model seems to match the buzz coming out of Vegas very well, with Lonzo Ball on top followed by Dennis Smith Jr, and Jayson Tatum, at two and three. Laker fans might want to frame this table with three Lakers prospects in the top 10.

    

Of course, for reference I should add that the scale of the projections line up so that a 5 is roughly equal to average production, or a 0 in plus/minus terms. Indicating that the SLO model projects every rookie but Ball to be below average next year.

For Celtics fans SLO is more mixed, Tatum comes out the third best, Zizic performed respectably, and uh, Semi Ojeleye was also there.

It is hugely unfair to project Markelle Fultz based on what amounted to about 64 minutes of playing time. Though he was actually helped by the the regression I applied, since he had performed below average in his brief court time.

So then, how much to actually adjust our expectations, if at all? Going again based on the Pelton article, I ran the rookie model with the SL numbers added, weighted at a 25% of the pre-NBA numbers. This gives a much more realistic evaluation to Summer League importance.

For example, in the tables below ordered by the SL Updated Rookie Model (appropriately, perhaps- SLURM) Fultz is still appropriately in the top 2. And the most any player has been adjusted up is three tenths, and down is five tenths (again the model results are scaled so that it’s roughly the same as being projected to be .3 better in a plus/minus model).

And below is the 2nd half of the summer league rookies by SLURM:

Maybe, we can hold off on the Kuzma Rookie Of the Year ceremony and wait to bury Fultz or Zach Collins. At least until the second game of Preseason.

New Celtics' Draftee Stats Compared to Peers

One way to get a sense of player's strengths and weakness is to chart them against their peers. The visual can give a more intuitive feel than numbers by themselves and it's much easier to take in information on a bunch of different players at a time.

I created one for the draft that groups the prospects stats in categories that are mostly independent and represent different aspects of my draft model's evaluation. All the stats are standardized in a z-score, with average players scoring at 0 and the spread between best and worst scaled by the standard deviation. That way the differences in each category  are set to the same visual scale.

The stats are then put in a box plot using position as the grouping variables, so a player's assists, for example is compared to guys at the same position, instead of comparing a power forward's assists to a point guard.

Below is an image of the rebounding category, with Celtics picks Semi Ojeleye and Jayson Tatum's mark in the chart highlighted with commentary.

The data  in the link is interactive so you can change the filters are hover over the screen to see more information on a particular data mark.

Among the highlighted stats for Jayson Tatum, he was:

• A very good scorer, among the best from the wing positions
• An above average rebounder at the three, but not great.
• Distribution was his biggest weakness, one the worst wings in that rating.
• Was above average for his position in blocks and steals rating.
• He was one of the youngest players in the draft, and played a very competitive college schedule.

For Semi Ojeleye highlights were:

• Very good rebounder as a three, combo forward potential
• One of the best wing scorers in the draft
• Weakness in blocks and steals
• Slightly above average for a wing in distribution
• Ojeleye is one of the older prospects in the draft.

 

Retrodictions Are Hard: Especially About Three Point Shooting

It's been almost three years since I posted my models to predict the three point accuracy of players coming into the NBA, a post called "Predictions Are Hard: Especially About Three Point Shooting".  Along with presenting the coefficients for the models, I also gave the results for some of the first round selections from that summers 2014 draft class. (The results for some of this year's class are up at a Nylon Calculus post here).

The models were trained on the shooting percentage first four years a player is in the league, so we're not quite to the time frame to give an exact test, or, retrodiction of those projections. But, it's too close for me not to give a little look in.

To test the models I compared the mean average error (MAE) of  model projections against the MAE of the player's pre-NBA (either college or overseas) three point percentage and the MAE compared to assuming the player is league average. The two models I presented performed virtually identically, both performing better than compared to he player's three point percentage and better than the simple average shooter assumption. The models had a MAE of 3% and 3.1%, while the MAE for the player's pre-NBA three point percentage was 5.7% and compared to the NBA average three point percentage the MAE was 4.2%.

The results for individual players featured in the original post are shown below, with their Pre-Draft three point shooting, NBA performance and the two models projections, then the error for each measure.

Overall this class has not been a good shooting class, with a 32.9% average player shooting percentage. Though slightly higher, 33.4%, if weighted for attempts. The models were slightly biased high compared to the actual performance projecting 34.9%, the player's pre-draft three point percentage was 35.7% and league average since 2002 at around 35.5% coming into this year. The biggest miss was on Adreian Payne, who never developed the three point shooting he needed to establish his game. James Young was a similar miss, under performing his projections in shooting (along with many other areas of the game).

Draft Benchmarks Compared to My Draft Model

Since I started doing draft analysis I have been interested in ways that I could supplement the data I had from the box scores and demographic data to help better predict NBA success. A good amount of this comes from scouts, including high school ranks, draft board rank, and specific trait ratings. That reflects my thoughts that analytic models should be complementary with scouting, not a substitute. 

 

Last year a feature at 538 had an interview with Ed Weiland, a draft analyst with his own site, Hoop Analyst. Weiland developed a benchmark system in various statistical categories with minimum achievement levels based on the player's position. For example, a shooting guard prospect is expected to:

• Hit at least 50% of his two pointers
• Hit 30% of his three pointers
• Score 18 points per 40 minutes
• Get 1.3 steals per 40 minutes,
• Get a sum of 7 in rebounds, steals and blocks per 40
• Have an assist to turn over ratio of at least .8

The rest of the criteria are at the 538 interview. 

 

The basic premise is that the more benchmarks a prospect misses, the more skeptical we should be about them performing in the NBA. The benchmarks are an interesting addition to look at prospects in addition to the models, I believe, as they look across multiple categories rather than adding them up as most draft models do. That lets us look at the the players versatility, or alternatively potential fatal flaws. In the past I have modeled on a few versatility measures, which had minor effects in the context of the model and in the end didn't adopt any in part because of positional biases. 

 

There are a couple of downsides to the benchmark approach compared to modeling. It limits information to yes/no on making or not the benchmark (and doesn't credit elite performance over adequate) and by drops demographic information like age (though Weiland is clear he considers informally)

 

In any case, contrasting the benchmark performance with my model estimate seemed like a good way to explore the current prospects as well as the flaws and benefits of each method. To do this I measured the percentage of benchmarks a player exceeds for his position. For example, a 1.0 indicating the player exceeds all benchmarks, or a .2 indicating he exceeds only 1 of 5 benchmarks. To be clear, this my own adaptation in order to be able to compare the benchmarks to my model results.

 

Generally, it's the cases where my model disagrees with the benchmarks that I think are the most informative. So below I have a chart of the players that are highly rated by model but miss on a couple of benchmarks.

 

 

The chart here visualizes the prospects performance using a standardized measures of statistical performance centered around a zero indicating average performance compared to other prospects.

 

 

 

Eight of the nine players under performing on benchmarks are big men.That likely says something about the difference between the two methods by position. Possibly that benchmarks miss style issues with the modern big man, or that the model is overly enamored of scoring or rebounding specialists. Seven of nine are younger than the average prospect, which is to be expected given that age isn't explicit in the benchmarks. 

 

But one other contrast that stands out is the treatment of specialists like Monte Morris with passing and distribution (the grey bar above), or Tony Bradly and Ivan Rabb with rebounds (the orange bar). The reliance on one skill should act as caution flags, I think for the model's result.

 

Next are the opposite cases, players not highly rated by the model, but hit most of their benchmarks.

 

 

In contrast all of these players are wings, mostly small forwards and all of them are older than average prospects.

 

As the stacked bar chart shows, here the opposite issue comes up for benchmarking, these prospects, for the most part, are competent across the board but don't excel in any skill. In order to be top out at more than a possible 10th man, a prospect probably needs to have at least one elite skill.

 

 

Lastly, there were three players that did well on both the benchmarks and in my model, but were not highly rated in the Draft Express top 100, Josh Hart. Mikal Bridges and Ethan Happ.

 

 

Happ is an interesting case, he puts up numbers across the board. But he lacks the size to play center in the NBA and lacks the shooting to play a modern power forward, much less a on the wing. That, of course, raises the question about whether these particular benchmarks are the best possible for today's game. Neither big man position has any criteria for shooting, arguably free throw percentage could be used as a proxy with a minimum that varies by position. 

 

In any case, the exercise has convinced me that benchmarks are worth looking at more and a useful way to examine prospects, as well as my model results.

2014 Draft Out Of Sample

A couple of years ago I put out my first draft projection in advance of the 2014 NBA draft. Now a few years gone by and I can run a preliminary test for the draft model on that 2014 draft class. The basic idea of the test is to compare the player's actual performance as measured by my minute adjusted AWS efficiency metric in their third and fourth year (just using the third year for this test of 2014 draft) to the rank placed by both my model and the performance of the actual NBA draft order. 

A couple of years ago I tested my draft model for the 2012 draft class, which was out of sample then from my analysis, against the performance of the actual draft. In that case the model out performed the NBA drafters, at least as measured by the metric that I use for my draft modelling.

In terms of average error, without regard to direction, the model predicted slightly better out of sample how well players rank by actual performance this year than the their spot selected in the draft. The average error for the model was 6.9 spaces from their "actual rank," while the draft order error was 8.1 spots. The full table is at the bottom of the page.

I want concentrate first on both the biggest misses and biggest hits for the model compared to the place they were selected in the draft. I think those cases give some of the most interesting lessons on what the model tells us relative to the draft decision makers.

 

Below are the five worst estimates for the model compared to where they were drafted:

 

• Jordan Adams, -21 places worse. my initial model loved Adams, placing him at the top of the class. Unfortunately Adams suffered a major leg injury that ended his NBA career, placing him among the least productive in the draft class. Injuries are a risk of the game and the draft. However, it depends on what question you are trying to answer as to whether you should include player's with career ending injuries in the model. A model on the average expected value of a pick should include them, because that is part of the draft risk. Translating college stats to NBA performance is a dicier question, however. There was nothing in in Jay WIlliams playing stats that would have let you know he would end his career early due to a motor cycle accident. In that case, there is a significant missing variable that is outside the question the model is trying to answer. In any case, Adams' rookie year was middling for a rookie, so it's not clear what stage he would be in career right now had he been healthy.
• Jarnell Stokes, -17 places worse: Stokes is an interesting case. He was a box score monster in the NCAA. But his fit with the modern NBA is problematic as a big man that doesn't excel at protecting the rim or have shooting range. He is under contract with Denver, but doesn't play
• Damien Inglis -18: Inglis was one of the youngest players in the draft and had decent numbers against grown men in France, big factor in his rating. Inglis also suffered a fairly major injury his rookie year. On the other hand he's yet to show significant development since returning to basketball that would lead one to think he warranted a high draft selection.
• Zach LaVine -13 places worse error. LaVine is actually performing better, at least as measured by my metric, than either his draft slot or the model projection as the 7th best of his draft class. LaVine is scoring at a high rate of efficiency and torching from three, both at a higher rate than we saw at UCLA.
• Rodney Hood -12 worse rank error. Hood is also better than his draft spot or the model would have predicted.

On the other side of the ledger, here are the five best model estimates compared to where the were drafted.

 

• Nikola Jokic, +31 in rank error. Jokic is rated as a top player in AWS terms this year. And as a passing young big man was very highly rated by my model. Often player's that dominate box score numbers and show flashes of everything but lack overt athleticism just don't stick around in the NBA. However, an interesting indication from a different study I did looking at "star" players and "busts" with both scout rankings and on court performance was that performance was relatively more important for becoming a "star", while scouting ratings were relatively more important for avoiding becoming a "bust." So, guys that put up numbers but aren't as good on the eye test might be the real high variance plays.
• Clint Capela, +19 in rank error. Capela has turned into a good young big man and was rated as 4th in AWS among this class. It is arguable that Capela is a guy that is overrated by a box score based rating, but he is definitely better than the 22nd selection where he was picked.
• Kyle Anderson, +15 in rank error, Anderson, with the nickname Slow-Mo is kind of the ultimate eye test vs analytics model player. In this case his efficiency rating is between the two measures, but closer to the rating of the model, whether the Spurs development machine plays into it or not.
• Nik Stauskas +14 in rank error, Stauskas  has resurrected his career to extent in Philly. However, he is still not one of the more productive players in his class, ranking 24th via AWS.  In this case, we shouldn't have necessarily trusted the crowd source process.
• Jerami Grant, +11 in rank error, Grant was rated as a late first round talent, which is about what he's been.
• Honorable mention to Adreian Payne, +10, as an older player that didn't dominate in college and looked like a reach in the middle of the first round.

I should note that in this case we're testing the model against the target variable for the training of the model, which gives the model an advantage even out of sample. So I quickly looked at the model vs win shares available from Basketball Reference. In that case the gap was much closer with the average error for the model being 7.2 places and the draft being 7.4.

Below is the full table:

 

Tableau:

 

end

Stacking Up the Celtics Picks

I went through a brief exercise to annotate the stats for the players picked by the Boston Celtics in this years draft using the same basic stats visualization I debuted prior to the draft. The added annotations bring those prospects into focus, and makes it easier to follow them across the different categories.

The main upshot is that Jaylen Brown's numbers at Cal were bad. We that already, but the categories bring it home, most notably on the distribution measure. None of this means Brown is doomed as a prospect, certainly, and as a Celtic fan I will be rooting for him and not for my models to be correct. But it indicates that him finding stardom would buck the historical trends, and reasons for optimism have to come from some where other than his performance at Cal

Other prospects:

• Guershon Yabusele, overall my projections like him much more than the consensus did, ranking the selection as less of a reach than the common reaction. Yabusele was rated close to average in nearly every category compared to other power forward prospects but is given a boost due to age and a higher competition level he faced against grown men in France.
• Ante Zicic was picked near his consensus status before the draft and close to where my model rated him. Unlike Yabusele, Zicic has a good deal more variance in his ratings, coming up higher in scoring and rebounding and lower in distributing the basketball.
• Demetrius Jackson fell quite a bit from his pre-draft ranking and was rated fairly well by my model. Again Jackson isn't a standout in any one area but displayed a solid overall game.
• Benjamin Bentil's one standout skill was scoring, but he below average witch everything else.

The scores are all standardized to average being zero with each category weighted to reflect the values in the model.

Official-ish Draft Board

With the NBA draft one week away, I wanted to get on the record with my semi-official big board draft list[1. I can't write I told you so if I don't tell you anything before hand.] Previously I gave my take on the near top of the draft after the big two of Ben Simmons and Brandon Ingram, but here I want to present the whole list.

The results are sorted by an ensemble of two models including my scouting informed PAWS model and a version using a mixed effects regression model that allows the performance and age variables to vary based on the tier they were drafted with three even tiers. Draft model is then applied using a Monte Carlo model to project the odds of the player being selected in each tier based on the the Draft Express mock draft. (I hope to do a later post with a bit more detail on the Tiered PAWS model).

The biggest surprise of the first round is Diamond Stone, projected on the bubble of the first and second round projected as the the third best prospect by the combined models, though the differences between Stone, Dragan Bender, Jakob Poeltl, and Denzel Valentine are virtually meaningless in terms of expected success.

This group goes a bit beyond the top sixty in order to include some players on the bubble of being drafted. The big mover, in the opposite direction compared to scouting expectations is Jaylen Brown. Brown does particularly poorly in the tiered model, which gives greater weight to the statistical performance on top tier prospects, an effect that usually increases their score, but Brown's poor statistics in college are too bad to lift his projection.

 

Draft Stats by Position

Draft stat visualization by position

In this visualization you can tab through the categories, filter by ranking from Draft Express, and scroll over to find or highlight names. Statistical visual highlights include how much Buddy Hield stands out as a scorer, Denzel Valentine as a distributor, Kris Dunn stands out in blocks and steals, Dragan Bender and Brandon Ingram in youth and Ben Simmons in the overall draft model rating.