Revisiting The ISL Scoring System: An Analysis of Jackpot Points (Part 1)

Courtesy: Steve Gambino.

This is the first of a four-part series taking a look at the current ISL scoring system and the impact made by jackpot points.

Introduction

The International Swimming League (ISL) has been changing the sport of swimming. Though in many ways this change has been promising, progress sometimes comes with growing pains. A prime example of this has been with the introduction of the Jackpot Bonus. First implemented in 2020, this system has attempted to add an extra level of excitement and unpredictability to races. While this seems good in theory and has worked on some level, it also has its faults, and thus, has attracted some criticism.

Given that I wrote an analysis of their scoring system back in 2019, shortly after the league began, I’ve wanted to revisit that discussion after the announcement of the jackpot system, particularly in light of the levied criticisms. I was further motivated to investigate after reading Barry Revzin’s analysis and article, published back in September, where he highlighted many of these critiques. To summarize, the jackpot system seems to accentuate the arguably too top-heavy nature of the league and isn’t transparent for viewers because of complexity, unclear graphics, and sometimes lagging or incorrect results. And, as Barry described, “for all of that confusion, all of the flashing lights, all of the extra math – they don’t wind up impacting the outcomes of meets.”

Scoring the meets with or without a jackpot bonus does not seem to change the outcomes of the matches. In other words, there is a lot of added complexity with no tangible impact. For a sport that can be oversimplified to “dive in on the buzzer and get your hand to the wall first,” the ISL’s tendency to seemingly and unnecessarily overcomplicate things in this way is a valid complaint. However, this is not even close to the whole story. So, what other problems arise when adding a jackpot system in this way? And do all of these complications mean we should leave it behind? Or can it be salvaged?

These are not easy questions to answer and will take some time to explore. So, I have broken this analysis into four parts, each addressing a unique aspect of the inner workings of the jackpot system. We’ll start small, beginning part one with the consequences that can be observed as a result of the mechanics of the rule, then dive deeper into the data to investigate the less apparent consequences for the remaining three parts.

It should be noted, for the purpose of this article, I’ll focus the majority of my analysis on the 2020 season, since that addresses the jackpot in its purest form. The jackpot didn’t exist in 2019 and although it can be hypothetically applied retroactively, it’s not a perfect comparison. 2021 also added an additional gimmick in scoring the 400’s, which could potentially muddle the data.

Quick Recap – How does the Jackpot System work?

This system was introduced in Season 2 of the ISL (2020) and, in summary, gives the winning athlete of each event an opportunity to earn more than the original 9 points assigned for first place by “stealing” the points of opponents who they beat by a predetermined margin. More specifically, each event is assigned a different jackpot margin. For any athlete who doesn’t win and finishes further behind the winner than the jackpot margin, the points they would have earned are instead awarded to the event winner. Consider the Men’s 100 FL from the first match of Season 2 – the first event where this new jackpot bonus came into play – where, the jackpot margin is 1.95 seconds. Table 1 below outlines how this affected the results.

The “Points Pre-Jackpot” Column indicates the points each swimmer would have earned under Season 1’s scoring system, without a jackpot bonus. The winning time was 49.58, so everyone who finishes in a time slower than 51.53, 49.58 + 1.95, would forfeit their points to the winner under the Season 2 scoring system. Because both Jan Switkowski and Kregor Zirk finished greater than 1.95 seconds behind the winner, Tom Shields, they forfeited their points to give Tom a 3-point bonus. This rule could affect everyone up to and including the second-place finisher. This means that an event winner has the potential to earn up to all of the 37 available points for a single race.

Let’s Start Shallow – What’s in a name?

Right off the bat, I haven’t heard a lot of commentary on the name itself, so I’m going to tackle it quickly here. The word “jackpot” implies there would be some kind of pot where points accumulate over time. Then there would be an opportunity for one athlete or team to “win the jackpot,” and collect a huge number of points at once. It’s a misnomer here. Just as an example, the name “jackpot” might more accurately describe a system where points accumulate in a “pot” whenever a swimmer is below an event’s cutoff time. A team or swimmer could collect those points later in the match for meeting some sort of achievement, for instance, winning the mixed relay. (Note: I’m not necessarily advocating for this system, just that it would be more suitable for the name “jackpot.”).

While this is most definitely a shallow criticism, for a system that already has issues with complexity and lack of clarity, a more accurately descriptive name couldn’t hurt.

Now, let’s begin to examine some of the more interesting consequences.

Does the Jackpot System value the highest level of competition?

As I wrote in my previous analysis from 2019, a scoring system is a reflection of a league’s values. However, the mechanics of this rule seem out of sync with what the league ought to prioritize here. I proposed 5 axioms for what makes a “good” scoring system. One being,

• “The system should value the most exciting/intense races the sport has to offer.”

Conversely, the jackpot system actually creates more value for less exciting races. Essentially, top athletes are rewarded more for winning against lesser swims, not better swims. For example, in Match 3 of the 2021 season, Sarah Sjostrom earned 24 points by winning the 50 Free and jackpotting Arina Surkova (5 pts), Marie Wattel (4 pts), Kim Busch (3 pts), Costanza Cocconcelli (2 pts), and Kornelia Fiedkiewicz (1 pts). Each swam slower than 24.48. This is .28 seconds slower than the average of all performances in the ISL’s history, 24.20. So, rather than racing the best of the best, and being rewarded for winning against the league’s top 50 freestylers, Sarah earns extra points for winning against below average performances. Is it just me or does this feel backwards?

To me, this point highlights the jackpot system’s flawed “ethic.”  Its process is flawed. Its mechanism does not function in line with the values that ought to be prioritized.

When the ISL first announced the jackpot system, my instinctual reaction was three-fold:

1. What makes this a “jackpot”?
2. Does this still value the best races the sport has to offer?
3. Will this just be a gimmick?

With only a superficial analysis, we’re left with an unsatisfactory answer to the first two questions. Barry Revzin’s article confirmed my suspicion for question three; it indeed appears to largely be a gimmick. This doesn’t bode well for jackpot system as a whole, but sometimes our initial instincts and intuition can lead us astray. Perhaps this is the case here. In part two, we’ll dive deeper into the data to find out.

ABOUT STEVE GAMBINO

Steve grew up swimming in Middletown, CT.  Before recently moving to Worcester, MA in September, he spent the past five years in Rhode Island, teaching math at CCRI and coaching age group swimming with Crimson Aquatics.  Steve has an M.S. in Mathematics from University of Rhode Island, has previously served as a consultant for the ISL for the development of their rating system, and currently works as an Assistant Professor of Mathematics at Quinsigamond Community College in Worcester.