During the early part of last weekend’s U.S. National Championships, it seemed like many of the pre meet favorites were all hype. Justin Wright and Zach Harting went 1-2 in the men’s 200 fly ahead of Jack Conger. Blake Pieroni won the men’s 100 free, not all-everything star Caeleb Dressel. Andrew Seliskar won the men’s 200 free over favored Townley Haas.
On the other hand, there were plenty of events where the top seeds dominated. Katie Ledecky, Leah Smith, Ryan Murphy, Simone Manuel placed about where everyone one expected them to. So which happened? Was this a meet full of surprises or a predicable coronation of favorites?
To answer that we need a metric for expectations, so we can see how the results stack up against expectations and a baseline to compare to this year’s meet’s data. This year’s pick’em contest entries work as a measure of expectations. The pick’em contest asked readers to pick the top 4 places in every event. This can give us a good idea of what the general swim fan expected to happen at this meet.
As a baseline I looked at the pick’em entries from last year’s U.S. Summer Nationals.
There was a bit of clean up required before comparing the two data sets. First I removed any entries that didn’t submit a pick for every event. This left 422 entries from last year and 775 from this year. Next I adjusted the scoring rules of last year’s pick’em to match this year. This year’s scoring system was 7 points for correctly picking a swimmer 1st, 5 for 2nd, 4 for 3rd, and 3 for 4th. 1 point was awarded for picking a top 4 finisher in the top 4, but in the incorrect position.
The primary differences in last year’s scoring system were: 1. the single point for right swimmer/wrong spot in the top 4 wasn’t awarded and 2. the 100 and 200 free were scored to 6th. The points for each of the top 4 positions were the same. Given how similar the scoring systems were, people’s picking strategies should be pretty similar both years.
The median score in last year’s contest scored with this year’s rules was 269 (mean 267, standard deviation 26). The median score in this year’s contest was 273 (mean 270, sd 24). Those totals are remarkably similar. That would indicate that this year’s meet wasn’t abnormally predictable or unpredictable. The field as a whole performed almost exactly the same vs expectations both years.
The first and last day were the least predictable this year which is perhaps why it seemed that this meet was so unpredictable.
It’s possible that both this year and last year’s meets were outliers when compared to more years of data, but these two years were all the readily available and reasonably comparable pick’em data I had.
| Total | Day 1 | Day 2 | Day 3 | Day 4 | Day 5 | ||
| 2017 | Median | 269 | 51 | 57 | 58 | 52 | 49 |
| Average | 267.4 | 50.7 | 57.5 | 58.0 | 52.3 | 48.9 | |
| SD | 26.1 | 9.5 | 9.0 | 8.5 | 8.3 | 10.1 | |
| 2018 | Median | 273 | 45 | 65 | 63 | 59 | 40 |
| Average | 270.2 | 45.0 | 64.7 | 63.0 | 57.9 | 39.6 | |
| SD | 24.2 | 8.1 | 8.5 | 9.0 | 7.8 | 6.8 |
How Accurate are People’s Expectations
Given that we’re already digging around in the pick’em stats, let’s go a bit deeper. How good of a prediction are people’s picks?
Unsurprisingly, people pick favorites more often than the favorites come through. Over picking favorites is rational given the scoring rules. The rules don’t offer odds. There aren’t extra points for getting that one obscure pick correct. If, for example, everyone agrees Katie Ledecky has a 98% chance to win the 800 free, 100% of people should pick her to win.
Of the 33 swimmers picked by 90% or more of pick’em entries to finish in a particular spot, 82% actually did, less than picked as expected. Swimmers favored by 80%-90% of picks finished there 71% of the time. 70%-80% was 54% correct. Swimmers under performed their picked percentage all the way down to 10% picked. Swimmer’s picked less than 10% of the time over performed. Swimmer’s picked between 0% and 1% of the time finished in their picked positions 2% of the time. 22 swimmers placed in the top 4 after being picked by no one (This was out of a total of 19,147 swimmers not picked in the top 4 so the conversion rate was pretty low).
In general, a swimmer’s odds of actually finishing in a particular place track pretty well with how often they were picked to finish there. Things get a bit wonky in the 30%-60% range but that could be a sample size issue. This is only 2 years of data. For the most part, swimmers picked more often in a particular place actually finished there more often.
Combined 2017 and 2018 Data
| How Often a Swimmer was Picked | How Many Swimmers Were Picked That Often | How Often The Pick was Right |
| 90%-100% | 33 | 82% |
| 80%-90% | 17 | 71% |
| 70%-80% | 13 | 54% |
| 60%-70% | 16 | 38% |
| 50%-60% | 31 | 13% |
| 40%-50% | 58 | 38% |
| 30%-40% | 88 | 13% |
| 20%-30% | 215 | 16% |
| 10%-20% | 323 | 10% |
| 5%-10% | 253 | 8% |
| 1%-5% | 569 | 6% |
| 0%-1% | 1309 | 2% |
| 0% | 19147 | 0.11% |
Favorite Underdogs
This data playing with ‘expert’ opinions in order to find some trends in the state of US Swimming is more like old day alchemy in attempt of finding philosopher’s stone. You are comparing basically a white noise to a white noise. US Nationals are not an Olympic Games and are not even World Championships. That is actually phase 1 of two stage trials. It is not about showing best times. The correct question will be: how many favorites didn’t make PP and how many underdogs did make it. And under this angle the answer will be: business as usual.
Whoever played in this predictions game was consciously or unconsciously based her/his opinion on swimmer’s individual distribution functions operating with… Read more »
I don’t remember the rules in 2017, but I picked a lot of events with one swimmer to place 2nd – 4th (meaning I picked the same swimmer multiple times for the same event). My rationale: if I pick A,B,C,D and the results are B,A,D,C I’ve got zero correct picks. If I pick A,A,A,A I’ve got one (but only one) correct guess.
Expectedly, I placed above average (top 25%), but didn’t win anything.
This year, I picked 4 different swimmers for each event, based on intuition. I did worse (maybe top 50%?) , but I would have been more likely to get lucky and win.
Underdogs/comebacks