Revisiting The ISL Scoring System: An Analysis of Jackpot Points (Part 3)

Courtesy: Steve Gambino.

This is the third of a four-part series taking a look at the current ISL scoring system and the impact made by jackpot points.

I think most people who’ve swum a 200 would agree, the third fifty is usually the toughest. In part one, we got off the blocks with some easy speed, commenting on the mechanistic issues with the jackpot system: complex, gimmicky, misnamed, and out of sync with what ought to be valued. In part two, we picked up our tempo a bit, getting into the data and exploring how the distribution of talent in a given event determines the jackpot margin’s impact in that event, and the potential discrepancies in value that this can lead to between events and between genders. As we flip turned to finish out part two, we pondered whether finding more optimal values for the jackpot margins could salvage the system. Now let’s dig deep, go to our legs, and build to a strong finish.

Exploring the Jackpot Margins – Is there an ideal margin?

As far as I know, the method for computing the jackpot margin has not been announced publicly, but I’ve observed that most jackpot margins are about 4% (rounded)* of the Short Course Meters (SCM) World Record for each event (or possibly the league records since they’re often the same or similar). And the specific margin chosen is surprisingly significant, particularly if we are to levy criticism toward the league about an unimpactful scoring rule. In fact, the criticism of its lack of impact may not be quite as justified as it previously seemed. I’ll explain in more detail, but first let’s get our bearings.

*Note: They seem to round the margin to a precision of 0.05 seconds. Considering 4%, sometimes this would need to be rounded up to match their official margins and sometimes it would be rounded down. For all of my computations going forward, I used the ceiling function for simplicity. This is to say 1.11 seconds rounds up to 1.15. Additionally, I’ll compute all margins as the given percentage of the SCM WR.  

First, we need to recognize that there exists a jackpot margin sufficiently large where no jackpot points are to be earned, effectively equivalent to not having a margin at all. For the 2020 season, the event with the largest margin of victory (relative to the SCM WR) is the Women’s 200 BR from Match 6, where Kelsey Wog beat Haley Black by 25.35 seconds. This is about 18.8% of the SCM WR of 2:14.57.

So, any jackpot margin greater than 18.8% wouldn’t yield any stolen jackpot points for the entire season (except for DQ’s or DNF’s, which are independent of time anyway). In other words, if you make the window large enough, every swim will make it through, and no jackpot bonus will be awarded.

Additionally, there exists a jackpot margin sufficiently small such that the winner steals all of the points from every other competitor (except in the instance of first place ties. Note that ties for 1st actually occur more than one might expect – a total of 18 times across the first three seasons). A margin of less than one hundredth of a second is guaranteed to suffice here, so let’s say 0.009 seconds.

You might expect that as you shift the jackpot margins from sufficiently large enough (for no jackpot bonus) to sufficiently small enough (for all points to be awarded as a jackpot bonus for every event), the amount of impact on the league standings would progressively increase. More specifically, you might guess that it would increase monotonically (never decreasing). This seems to make intuitive sense – the more jackpot points stolen, the more impact, right? However, this is not the case, and for me, this is where the story gets really curious.

Let’s explore the effects of changing the jackpot margin for the 2020 season. Starting at the sufficiently large end, 18.8%, and progressively adjusting the jackpot bonus by -0.1% at a time, we can see how and when the match results become affected. Table 6 summarizes all changes from 18.7% to 12.6% omitting all steps where nothing new occurred and gives a sense of when some of the events with the largest relative margins of victory began to be impacted by jackpot bonuses.

You can see that, not surprisingly, there isn’t much interesting happening when the margins are very large. While margins this large may impact some individual events, they do not impact the overall match results. Continuing to reduce the jackpot margin, it isn’t until we drop to 5% that we start to see some effect on the actual outcomes of a match – detailed in Table 7. At this point, the result of Match 9 would change – Iron would have overtaken Tokyo.

This remains true at a margin of 4.9%, but the results revert back to the original standings at 4.8%. We then see a small change to Match 2, where Aqua bumps up to tie for third with DC if the margin is set to 4.7%.

These are pretty small overall changes, but it is interesting to note that these changes occur when the hypothetical margin is greater than the official values (~4%), where we see no impact. So, the jackpot system can have an impact, it just isn’t at the specific margins that have been selected. Further, it isn’t just that the margins aren’t sensitive enough to be relevant either, since these listed above are all greater. It’s also not true that the margins are too sensitive, because if we continue to explore how decreasing them affects the results, we’ll see impact return and stay after 3.5%. Naturally, this leads to the question of what is actually occurring to cause the jackpot bonus to impact the match results. I will address this in the next section. For now, the full list of changes is outlined in the Table 8:

To summarize the main conclusions so far:

• The problem isn’t just that the jackpot margin isn’t sensitive enough – we see this because margins greater than the official, like 5%, cause an impact on the match results.
• The jackpot margin isn’t too sensitive either – we see this because margins less than the official, like 3%, cause an impact on the match results.
• The impact on match results does not increase monotonically as the jackpot margin decreases – if it did, we would see the number of results changed only increase or remain the same, never decrease. Instead, there are points where it jumps up and points where it drops down without a clear pattern.

Overall, the current jackpot margins have no impact on the results, but that’s not to say a jackpot system can’t or shouldn’t in general. Nudge those values a little higher or a little lower, and match outcomes may just change a bit accordingly.

So, what is so special about a 5% margin in Match 9, and Iron and Tokyo, that causes this to be the earliest (highest) place where jackpots are meaningful?

Match 9, Iron and Tokyo, and Understanding the Change at a 5% Margin

With a jackpot margin at 5.1%, the standings (and points earned) for Match 9 would be as follows:

1. Energy Standard (569.5)
2. Tokyo Frog Kings (423)
3. Iron (414.5)
4. Toronto Titans (299)

However, adjusting the jackpot margin to 5% – just by shortening the cutoff by a few hundredths of a second – the results change and the standings would be:

1. Energy Standard (559.5)
2. Iron (425.5)
3. Tokyo Frog Kings (423)
4. Toronto Titans (298)

The complete list of differences in the results of Match 9 are outlined in the Table 9.

Essentially, Nicholas Santos gains an additional 11 points for Iron by shortening the cutoff from 1.15 seconds (5.1% rounded) to 1.10 seconds (5.0% rounded). Specifically, with the margin set for 5.1%, the resulting cutoff time of 22.93 allows Chad and Andrey to sneak in without losing their points. When the margin is set to 5.0%, the cutoff time of 22.88 just barely cuts them off. This difference of a few hundredths yields an 11-point swing from Energy to Iron, allowing Iron to surpass Tokyo for second place.

Interestingly, this showcases our previous discussion around the Men’s 50 FL as a leading jackpot-heavy event. Hopefully this also begins to paint the picture of just how precarious the selection of a jackpot margin can be.

The difficulty with this selection is that the specific situations that yield an impact from a change in jackpot margin are incredibly circumstantial. This can be demonstrated even better if we compare the 5% margin to the 4% margin. Table 10 summarizes all of the changes between Tokyo and Iron:

The point swings are haphazard. A few point-boosts here, but a few point-drops there, end up yielding a negligible net effect in terms of the final match outcome. To understand the essence of this, I think the following analogy is appropriate:

Imagine you were standing in front of a crowd of a few hundred people. If each of them projected music from their phone speakers, you’ll hear the music play at a certain total loudness. Imagine then having a handful of people adjust the volume of their phone, some up by a few steps, some down by a few steps. While there may be a lot of individual changes occurring, you’d likely not notice this as it would have a negligible net effect on the total volume you’d perceive from the whole crowd. This is what is occurring with the jackpot points.

Looking at any one event in isolation, there appears to be a significant swing of points – just like if you only heard a single phone speaker, you’d notice the change in volume. As these aggregate over the entire set of events, the changes all meld together to the point where they’re no longer noticeable. While the internals of the system are dynamic, from an external perspective, it’s largely static.

Even then, this only tells the story of how a few of the matches could have been impacted. And those impacts may seem underwhelming. Yet, the story doesn’t end here; we’ve still got another fifty to swim. There is one big factor we’ve still got left in the tank, one more piece of the puzzle to consider. Let’s bring it home strong in part four.

See Also

• Part 1
• Part 2

ABOUT STEVE GAMBINO

Steve grew up swimming in Middletown, CT.  Before recently moving to Worcester, MA in September, he spent the past five years in Rhode Island, teaching math at CCRI and coaching age group swimming with Crimson Aquatics.  Steve has an M.S. in Mathematics from University of Rhode Island, has previously served as a consultant for the ISL for the development of their rating system, and currently works as an Assistant Professor of Mathematics at Quinsigamond Community College in Worcester.