Revisiting The ISL Scoring System: An Analysis of Jackpot Points (Part 4)

Courtesy: Steve Gambino.

This is the fourth and final part of a series taking a look at the current ISL scoring system and the impact made by jackpot points.

After three 50s, your heart is pounding, your shoulders are screaming, your legs are burning. We made adjustments to the jackpot margin, spanning an entire spectrum from having no effect on any race to maximal effect on every race. The impact we saw was haphazard, yet underwhelming. We might feel like we’ve exhausted everything we had. But you’ve always got more in you than you think.

Last one, fast one.

The Predominant Factor

Throughout the entire spectrum of possible jackpot margins tested, only the results of 5 matches were affected: Match 2, Match 5, Match 6, Match 8, and Match 9. Altering the jackpot had no effect on the results of the other 8 matches. This is because the fundamental premise of the whole scoring system overshadows the nuances and complexity of the jackpot, nullifying its impact in the end. A giant subwoofer blaring sound so loudly that any shift in volume from individual phone speakers is essentially irrelevant. It’s the same conclusion that I came to when I analyzed the scoring system at the league’s inception – the fact that the scoring system of the ISL is designed so that the match results are predominantly determined by event winners.

As I wrote previously, a general strategy to excel in the ISL can be summarized as follows:

“Win every event that you can and recruit swimmers who have as near to guaranteed wins as possible. Lose as few events as possible. Then, secondarily, worry about the specifics of what happens in the middle.”

This still holds true, with or without the jackpot. If anything, it seems the jackpot may have amplified this concept. This becomes apparent if we score each match by simply counting the number of races each team wins – there’d be little difference from the official results.

Table 11: 2020 Match Results – Rank by Wins vs. Rank by Points
Match 1 Match 8
Team Wins Rank by Wins Rank by Points Points Team Wins Rank by Wins Rank by Points Points
CAC 17 1 1 567 CAC 21 1 1 507
ENS 13 2 2 463 LON 12 2 2 491.5
LAC 12 3 3 420 NYB 4 4 4 296.6
NYB 1 4 4 266 TOK 6 3 3 419
Match 2 Match 9
Team Wins Rank by Wins Rank by Points Points Team Wins Rank by Wins Rank by Points Points
AQC 5 4 4 344 ENS 19 1 1 573.5
DCT 6 3 3 350 IRO 7 3 3 415.5
IRO 9 2 2 392.5 TOK 10.5 2 2 428
LON 23 1 1 609.5 TOR 6.5 4 4 289
Match 3 Match 10
Team Wins Rank by Wins Rank by Points Points Team Wins Rank by Wins Rank by Points Points
AQC 6 4 4 260 AQC 4 4 4 255
LAC 16.5 1 1 535.5 CAC 19 1 1 558
TOK 12 2 2 506.5 LAC 14 2 2 495
TOR 8.5 3 3 401 LON 6 3 3 398
Match 4 Match 11
Team Wins Rank by Wins Rank by Points Points Team Wins Rank by Wins Rank by Points Points
CAC 24 1 1 610.5 ENS 17 1 1 580
DCT 3 4 4 287 LON 15 2 2 517.5
IRO 11 2 2 418.5 NYB 3 4 4 239
NYB 5 3 3 394 TOK 8 3 3 380.5
Match 5 Match 12
Team Wins Rank by Wins Rank by Points Points Team Wins Rank by Wins Rank by Points Points
DCT 3 4 4 287 CAC 22 1 1 605.5
LAC 16 1 2 478.5 IRO 8 3 3 340.5
LON 13 2 1 499 LAC 12 2 2 462
TOK 11 3 3 446 TOR 1 4 4 303
Match 6 Match 13
Team Wins Rank by Wins Rank by Points Points Team Wins Rank by Wins Rank by Points Points
AQC 4 3 4 290.5 CAC 17 1 1 561.5
ENS 24 1 1 609 ENS 12 2 2 464.5
NYB 4 3 3 354.5 LAC 5 4 4 298
TOR 11 2 2 448 LON 9 3 3 391
Match 7
Team Wins Rank by Wins Rank by Points Points
DCT 1 4 4 256
ENS 23 1 1 613
IRO 11 2 2 448
TOR 8 3 3 391

So, what’s the difference in outcomes if we scored the match officially vs. counted the number of wins for each team?

  • LA Current would swap with London Roar for 1st and 2nd place in Match 5.
  • Aqua Centurions would bump up into a tie for 3rd in Match 6.

And that’s it – nothing else. The other matches might as well have been scored based on wins alone.

So, the jackpot system had no impact on the 2020 season, but that is at least somewhat dependent on the specific margins. Yet even when adjusting the specific margins for maximal impact, the effect is pretty minimal. And also, the scoring system in general appears even less impactful relative to just counting who wins the most races!  As the jackpot system has some flaws, so it seems the scoring system, in its totality, does too.

Addressing this concern will require an analysis of its own. My hypothesis is that the problem is a result of some combination of the following factors

  • The specific values attributed to each place: [9,7,6,5,4,3,2,1]
  • The distribution of talent throughout the league, similar to how the effect of the jackpot margin for a particular event was a function of the distribution of talent in that event.
  • The parity of the league and draft selection process.
  • The number of teams and their permitted roster size.
  • The nature of quad meets where each team has two entries per race (as opposed to, for example, dual meets where each team has four entries)

Some of these things may have already been discussed, many of them have not as far as I’m aware, but they are worth investigating for another time. For now, let’s return to the questions we set out to answer from the beginning: Is the jackpot system good? If not, can it be salvaged or should it be left behind?

Jackpot System – Keep it, alter it, or forget it?

As much as we might like it to be, this is not a purely objective question, and certainly not an easy one. When the ISL tries to take the sport beyond the most simplified version of “whoever gets their hand to the wall first,” it requires introducing an additional layer of complexity and subjectivity. Maximizing the return for that additional complexity is important and thus should be deliberated carefully. The ISL has yet to achieve this with its current scoring system.

Further, evaluating a scoring system always comes down to how well it reflects what is valued. I’ve made some personal axioms clear in the past. The introduction of the jackpot system means the ISL no longer satisfies these, and thus, I no longer consider the league’s scoring system to be sufficient.

To me, this is a strong enough argument to remove the jackpot system going forward. Though I was initially intrigued by the concept, I do not believe its benefits outweigh its costs. However, if the ISL intends to keep it, then some adjustments should be made to make it more effective. At the very least, since the term “Jackpot” is a misnomer, it should either be rebranded or reformatted to better reflect its name; and considering how it rewards less-exciting races, I lean toward its reformatting. Additionally, in an effort to preserve equality between events and between genders, more care needs to be taken in determining the specific jackpot margins for each event; and they should be determined while considering the distribution of talent within each event. Finally, although it would require a closer look at the other elements of the scoring system and league structure, there hopefully exists a way to make the jackpot bonus have a more meaningful impact on match results and less exclusively determined by race wins. This should be pursued deliberately.

I do not think the jackpot system, nor the overall format of the scoring system more broadly is effective. Yet, as any of us who’ve dedicated a substantial amount of effort and energy to this sport well know, we rarely get things perfectly right away. No one breaks records on their first swim, wins golds at their first meet, nor even successfully or permanently masters stroke technique changes on their first attempt in practice. Progress is messy. We experiment, we try, we take two steps forward and one step back. We succeed, we fail, we have good swims and bad swims, and we learn from them all. We iterate, we grind, and repeat this over and over thousands of times in pursuit of excellence in our sport.

Similarly, through their experimentation and innovation, the ISL has provided much from which our sport can learn and grow. You, as I do, likely have ideas about how to improve this system. And I wouldn’t be surprised if the league has been discussing it behind the scenes as well. Thus, I will remain hopeful that we’ll eventually converge on the optimal format and system for our sport.

See Also


Steve grew up swimming in Middletown, CT.  Before recently moving to Worcester, MA in September, he spent the past five years in Rhode Island, teaching math at CCRI and coaching age group swimming with Crimson Aquatics.  Steve has an M.S. in Mathematics from University of Rhode Island, has previously served as a consultant for the ISL for the development of their rating system, and currently works as an Assistant Professor of Mathematics at Quinsigamond Community College in Worcester.

Leave a Reply

Notify of

1 Comment
oldest most voted
Inline Feedbacks
View all comments
2 years ago

Perhaps the author could devise an alternative scoring system that fixes the problems that they’ve outlined in this series and write an article about it.