Fishback: Does Psych Sheet Scoring Matter? Rehashing the Numbers from Women’s NCAA’s

What happens when you take economics professor, one wild swim meet, and 4,805 points? Swim Geek heaven. Arizona professor of economics Price V. Fishback brings us his analysis, commentary, and calculations on how much psych sheet scoring really matters. Fishback is a former collegiate swimmer at Butler, and coach, who has been serving as a meet announcer nationally and internationally for the better part of 30-years. He still announces Arizona meets, but this year has made a bit of a comeback by backing-up Sam (the man) Kendricks at NCAA’s.

The Table below shows the Final Meet Score and the Predicted final score from the psych sheet without diving points.  For comparisons with the psych sheet scores, I added the final score for just swimming events.  I also show the difference to see who moved up and moved down and show the diving scores.

The information shows why Georgia won the meet.  They not only came in with the most points in seeding.  They moved up another 55 points in the swimming and score in two diving events.   The other big gainers were Arizona, which added 67 points to the psych sheet.  North Carolina exploded at the meet even without Stephanie Peacock with an added 59 points.   Wisconsin and California also moved up 40 and 36 points, respectively.

Dave Parrington clearly deserved his Diving Coach of the Year Award, as Tennessee scored 72 points in diving with Purdue a distant second at 44 points.

One interesting thing [P.A. Announcer] Sam (Kendricks) and I saw in preparing for Friday night shows the timing of the taper and the many ways swimmers get to the meet.  In nearly every event on Friday, the swimmers who were in the final and had not scored in a prior NCAA in that event had often finished fifth or sixth at their respective conference meets.  A number of conference winners from the major conferences were in the consol final.

Rick Demont and I have a running debate about the usefulness of the psych sheet predictions.  He argues that they miss a great deal.   He certainly has the personal team experience to make the claim because Arizona has scored about 20 percent more points than the psych sheet predicted in most years over the past two decades.

The psych sheet times and rankings are clearly imperfect predictors of what will actually happen.  How swimmers perform at the NCAAs depends in part on a mixture of factors for which we do not have good measures:  illness, injury, their emotional state on the day of the swim, how much they had to rest or taper to make the meet, how they swim with someone going just as fast or faster right next to them, and a variety of other factors.  On numerous occasions I have talked to coaches and swimmers about their expectations, and they say even they are not sure.  On the other hand, the psych sheet times do distill a lot of information into one number.  That person has gone that time at least once, which makes it more likely they can do it again.  The fact that they swam the time also gives an indication that they have they have some mixture of a high level of natural ability and have put in the work necessary to succeed and get to the meet.

Take the information from the psych sheet times and consider that we are dealing with roughly 300 swimmers and aggregating to the team level.  Once you do that the predictions do pretty well.  In the graphs below you can see how closely they match.

However, I can defend the psych sheet predictions pretty easily. If you look below and run the correlations between the psych sheet score and the final swimming score, the correlation is strong.

[Arizona assistant coach] Rick Demont and I have a running debate about the usefulness of the psych sheet predictions.  He argues that they miss a great deal.   He certainly has the personal team experience to make the claim because Arizona has scored about 20 percent more points than the psych sheet predicted in most years over the past two decades.

The psych sheet times and rankings are clearly imperfect predictors of what will actually happen.  How swimmers perform at the NCAAs depends in part on a mixture of factors for which we do not have good measures:  illness, injury, their emotional state on the day of the swim, how much they had to rest or taper to make the meet, how they swim with someone going just as fast or faster right next to them, and a variety of other factors.  On numerous occasions I have talked to coaches and swimmers about their expectations, and they say even they are not sure.  On the other hand, the psych sheet times do distill a lot of information into one number.  That person has gone that time at least once, which makes it more likely they can do it again.  The fact that they swam the time also gives an indication that they have they have some mixture of a high level of natural ability and have put in the work necessary to succeed and get to the meet.

Take the information from the psych sheet times and consider that we are dealing with roughly 300 swimmers and aggregating to the team level.  Once you do that the predictions do pretty well.

In the graphs below you can see how closely they match.  The plots below show the relationships between the predictions based on the Psych Sheet and the Actual Results.   No Diving is included.   Figure 1 shows the rankings by place.  The diamonds show the line along which the rankings by the psych sheet and the actual scores are the same.   The correlation between the ranks for these 45 teams is 0.95.  The correlation for the actual ranks of the top 10 teams is 0.90.  The debate between Rick Demont and me about the accuracy of the predictions is really about the unmeasured and often unknowable factors that are driving the differences between the circles and the diamonds on the plots.

Figure 2 shows the score predictions from the psych sheet plotted against the final score.  The correlation between the final score and the psych sheet prediction for the top 45 teams is 0.98.  The correlation for the top 10 teams is 0.87.   The top ten team correlation is likely lower because there are fewer observations and because the average scores among the top 10 teams are much larger than among the next 45, thus leading to a larger variance.

Rank on Psych Sheet versus Final Placing. (Price V. Fishback)

Final Score plotted Against Psych Sheet Scoring, With No Diving in Either. (Price V. Fishback)