How Good are Mid Season Conference Projections?

We’ve done a lot of projections of conference meets based on in season times, (D1 big conferences, D1 small conferences, D2, D3 part 1, D3 part 2) but haven’t really quantified how reliable those projections are.

Swimulator has two methods for scoring out a conference meet, using swimmer’s best time so far this season or a swimmer’s average time in each event so far this season. How does each perform? Which is better? To answer this I looked at 10 D1 men’s conferences and compared the Swimulator scores last season through 1/24/16 for the average and top times to the actual scores at conference meets (which include diving, so there’s some unaccounted for variance).

The average times performed a bit better (though both methods performed pretty well). A regression of average time scores on actual scores gives an R^2 of .85; whereas, a regression of top time scores on actual scores give an R^2 of .79 (R^2 is the % of variance in the data accounted for in the regression). I also compared team’s places with each projection method to their actual place. Average time was again a bit better with and average place miss of 1.1 with a standard deviation of 1.1 vs an average place miss of 1.2 with a standard deviation of 1.1 for top times.

Top Times vs Actual Score

Average Times vs Actual Score

Here is the conference level data I used:

ACC

Atlantic 10

Big East

Big 10

Horizon

Ivy

Pac 12

SEC

Patriot

WAC