Fishback: Does Psych Sheet Scoring Matter? Rehashing the Numbers from Women’s NCAA’s

Braden Keith
by Braden Keith 16

March 25th, 2013 News

What happens when you take economics professor, one wild swim meet, and 4,805 points? Swim Geek heaven. Arizona professor of economics Price V. Fishback brings us his analysis, commentary, and calculations on how much psych sheet scoring really matters. Fishback is a former collegiate swimmer at Butler, and coach, who has been serving as a meet announcer nationally and internationally for the better part of 30-years. He still announces Arizona meets, but this year has made a bit of a comeback by backing-up Sam (the man) Kendricks at NCAA’s.

The Table below shows the Final Meet Score and the Predicted final score from the psych sheet without diving points.  For comparisons with the psych sheet scores, I added the final score for just swimming events.  I also show the difference to see who moved up and moved down and show the diving scores.

The information shows why Georgia won the meet.  They not only came in with the most points in seeding.  They moved up another 55 points in the swimming and score in two diving events.   The other big gainers were Arizona, which added 67 points to the psych sheet.  North Carolina exploded at the meet even without Stephanie Peacock with an added 59 points.   Wisconsin and California also moved up 40 and 36 points, respectively.

Dave Parrington clearly deserved his Diving Coach of the Year Award, as Tennessee scored 72 points in diving with Purdue a distant second at 44 points.

One interesting thing [P.A. Announcer] Sam (Kendricks) and I saw in preparing for Friday night shows the timing of the taper and the many ways swimmers get to the meet.  In nearly every event on Friday, the swimmers who were in the final and had not scored in a prior NCAA in that event had often finished fifth or sixth at their respective conference meets.  A number of conference winners from the major conferences were in the consol final.

Rick Demont and I have a running debate about the usefulness of the psych sheet predictions.  He argues that they miss a great deal.   He certainly has the personal team experience to make the claim because Arizona has scored about 20 percent more points than the psych sheet predicted in most years over the past two decades.

The psych sheet times and rankings are clearly imperfect predictors of what will actually happen.  How swimmers perform at the NCAAs depends in part on a mixture of factors for which we do not have good measures:  illness, injury, their emotional state on the day of the swim, how much they had to rest or taper to make the meet, how they swim with someone going just as fast or faster right next to them, and a variety of other factors.  On numerous occasions I have talked to coaches and swimmers about their expectations, and they say even they are not sure.  On the other hand, the psych sheet times do distill a lot of information into one number.  That person has gone that time at least once, which makes it more likely they can do it again.  The fact that they swam the time also gives an indication that they have they have some mixture of a high level of natural ability and have put in the work necessary to succeed and get to the meet.

Take the information from the psych sheet times and consider that we are dealing with roughly 300 swimmers and aggregating to the team level.  Once you do that the predictions do pretty well.  In the graphs below you can see how closely they match.

However, I can defend the psych sheet predictions pretty easily. If you look below and run the correlations between the psych sheet score and the final swimming score, the correlation is strong.

Obs

TEAM

Final Score

Psych Sheet Score without Diving

Swimming Final

Diving Points

Swimming Final Minus Psych Predictions

1

ga

477

404

459

18

55

2

cal

393

346

382

11

36

4

tn

325.5

268

253.5

72

-14.5

3

tam

323.5

371

312.5

11

-58.5

5

az

311

217

284

27

67

6

fl

305

346

305

0

-41

7

socal

291

282

271

20

-11

8

st

246

238

246

0

8

9

tx

186

136

158

28

22

10

mn

141

192

110

31

-82

11

in

115

123

98

17

-25

12

nc

102

43

102

0

59

13

au

87

99

87

0

-12

14

mo

71

43

57

14

14

15

wi

65

25

65

0

40

16

nd

55

39

50

5

11

17

ucla

54

16

33

21

17

18

va

45

62

45

0

-17

19

pu

44

0

0

44

0

20

mia

40

7

4

36

-3

21

col

37

29

37

0

8

22

lou

36

29

36

0

7

23

asu

31

19

14

17

-5

24

smu

26

24

26

0

2

25

vt

26

15

11

15

-4

26

ar

24

20

24

0

4

27

emu

23

0

0

23

0

28

osu

22

25

5

17

-20

29

psu

21

36

21

0

-15

30

mass

18

0

2

16

2

31

wv

13

16

13

0

-3

32

fsu

12

17

12

0

-5

33

wism

12

5

12

0

7

34

ya

11

31

11

0

-20

35

lsu

11

8

0

11

-8

36

mi

9

7

9

0

2

37

pr

4

9

4

0

-5

38

bu

4

0

4

0

4

39

towso

4

0

4

0

4

40

ncs

4

2

0

4

-2

41

fresno

2

0

0

2

0

42

un

0

8

0

0

-8

43

nw

0

5

0

0

-5

44

den

0

3

0

0

-3

[Arizona assistant coach] Rick Demont and I have a running debate about the usefulness of the psych sheet predictions.  He argues that they miss a great deal.   He certainly has the personal team experience to make the claim because Arizona has scored about 20 percent more points than the psych sheet predicted in most years over the past two decades.

The psych sheet times and rankings are clearly imperfect predictors of what will actually happen.  How swimmers perform at the NCAAs depends in part on a mixture of factors for which we do not have good measures:  illness, injury, their emotional state on the day of the swim, how much they had to rest or taper to make the meet, how they swim with someone going just as fast or faster right next to them, and a variety of other factors.  On numerous occasions I have talked to coaches and swimmers about their expectations, and they say even they are not sure.  On the other hand, the psych sheet times do distill a lot of information into one number.  That person has gone that time at least once, which makes it more likely they can do it again.  The fact that they swam the time also gives an indication that they have they have some mixture of a high level of natural ability and have put in the work necessary to succeed and get to the meet.

Take the information from the psych sheet times and consider that we are dealing with roughly 300 swimmers and aggregating to the team level.  Once you do that the predictions do pretty well.

In the graphs below you can see how closely they match.  The plots below show the relationships between the predictions based on the Psych Sheet and the Actual Results.   No Diving is included.   Figure 1 shows the rankings by place.  The diamonds show the line along which the rankings by the psych sheet and the actual scores are the same.   The correlation between the ranks for these 45 teams is 0.95.  The correlation for the actual ranks of the top 10 teams is 0.90.  The debate between Rick Demont and me about the accuracy of the predictions is really about the unmeasured and often unknowable factors that are driving the differences between the circles and the diamonds on the plots.

Figure 2 shows the score predictions from the psych sheet plotted against the final score.  The correlation between the final score and the psych sheet prediction for the top 45 teams is 0.98.  The correlation for the top 10 teams is 0.87.   The top ten team correlation is likely lower because there are fewer observations and because the average scores among the top 10 teams are much larger than among the next 45, thus leading to a larger variance.

Rank on Psych Sheet versus Final Placing. (Price V. Fishback(

Rank on Psych Sheet versus Final Placing. (Price V. Fishback)

 

Final Score plotted Against Psych Sheet Scoring, With No Diving in Either. (Price V. Fishback)

Final Score plotted Against Psych Sheet Scoring, With No Diving in Either. (Price V. Fishback)

16
Leave a Reply

Subscribe
Notify of

16 Comments
newest
oldest most voted
Inline Feedbacks
View all comments
CoachGB
11 years ago

I have to laugh at how the use of the term Psych sheet has become an official term. When we would show up and see the entry sheet (done in ditto) we would be chastised to ignore them because all they do is “Psych you out’. Why did this become the official title and not entry sheets. What a negative term. The results show why we give the medals out afterwards. Shout out congratulations to Kelsi Worrell who was picked to be in the Conso’s and that 23.9 first 50.

psychedupnotout
Reply to  CoachGB
11 years ago

The name could mean that they are used to get psyched up! 😉 Seeing where you match up against the competition has different effects on different swimmers. It will certainly mess with some people while it provides motivation for others.

CoachGB
Reply to  psychedupnotout
11 years ago

Do you remember when they did ” ditto sheets” or what that refers too?

Mac
11 years ago

Hey Braden – if you e-mail me the data I can actually calculate whether or not the psych sheet scores are significantly different from the real scores using statistics!

Reply to  Mac
11 years ago

Why bother? You are not dealing with a sample, it is the entire population, so statistical significance is… well, insignificant.

Mac
Reply to  Steve Schaffer
11 years ago

It’s a sample of all NCAA’s ever – one could in theory just run the regression and predict next year’s results.

StuartC
11 years ago

It’s interesting that if you look at the greatest variability it occurred in the top ten teams, while if you look at teams in bottom half the psyche sheet actually have a better fit.

It would be interesting to run a regressions on 1-10, 11-20, etc. I bet that the teams that have the fastest swims in psyche sheets also have the greatest variability – has a lot to do with swimmers that are at their peak performances that have just a slight “off” swim (like having a head cold with Leverenz) can really change the outcome. When top swimmers/athletes are slightly off, it brings them back to the field much quicker and can easily change the results of… Read more »

steve
11 years ago

SEC teams Florida and Texas A&M underachieved. Pretty common for Florida, but surprising for Texas A&M. Gator sprinters swam very fast but most swimmers did not perform as well as expected. Texas A&M had a very rough meet as well.

swimfan
11 years ago

The two figures are the same. Can you upload figure 2 again? I’d like to see seeded points vs actual. Interesting stuff!

Roger von Oech
11 years ago

Thanks for the analysis! With the exception of Wisconsin, it appears that most of the Big Ten teams swam BELOW what their seed times had predicted. It would be interesting to postulate the reasons for their dropoff. For example, the timing of their conference meet (too early?), or the lack of big meet experience? Any other guesses?

Anthony
11 years ago

Psych sheet scoring shows the Michigan men will win in a blowout.

SNARKY
11 years ago

Yes! Yes! Tables, spreadsheets, fancy graphs, and comments from former great swimmers.
All good stuff. But you never answered the question.

Did you forget the title of the article – “Does Psych Sheet Scoring Matter”?

Well? Does it?

SNARKY
Reply to  Braden Keith
11 years ago

Humm – maybe i was too snarky 🙂

Still, I think the editor posed a good question. Is it more than just organizing already available information?

You said its a “pretty useful tool”. To me that implies that the data is actionable. And I think that’s at the heart of the question of does it matter.

Would a coach make a change to the lineup, change any strategy, etc., based on the the graphs and data?

mcgillrocks
Reply to  SNARKY
11 years ago

i think the answer is yes and no

no, it does not determine the outcome of the meet so it does not matter
yes, it is a good way of determining how teams will place roughly

About Braden Keith

Braden Keith

Braden Keith is the Editor-in-Chief and a co-founder/co-owner of SwimSwam.com. He first got his feet wet by building The Swimmers' Circle beginning in January 2010, and now comes to SwimSwam to use that experience and help build a new leader in the sport of swimming. Aside from his life on the InterWet, …

Read More »