Forecasting – Page 5 – Heavy Topspin

Unpredictable Bounces, Predictable Results

These days, the grass court season is the awkward stepchild of the tennis calendar. It takes place almost entirely within a single country’s borders, lasts barely a month, and often suffers from the absence of top players who prefer to rest after the French Open.

The small number of grass court events makes the surface problematic for analysts, as well. The surface behaves differently than hard or clay courts and rewards certain playing styles, so it’s reasonable to assume that many players will be particularly good or bad on grass. But with 90% of tour-level matches contested on other surfaces, many players don’t have much of a track record with which we can assess their grass-court prowess.

I was surprised, then, to find that grass court results are rather predictable. Elo-based forecasts of ATP grass court matches are almost as accurate as hard court predictions and considerably more effective than clay court forecasts. Even when we use “pure” surface forecasts–that is, predicting matches using ratings which draw only on results from that surface–grass court forecasts are a bit better than clay court predictions.

I started with a dataset of the roughly 50,000 ATP matches from 2000 through last week, excluding retirements and withdrawals. As a benchmark, I used official ATP rankings to make predictions for each of those matches. 66.6% of them were right, and the Brier score for ATP rankings over that span is .210. (Brier score measures the accuracy of a set of forecasts by averaging the squared error of each individual forecast, so a lower number is better. To put tennis-specific Brier scores in context, in 2016, ATP rankings had a .208 Brier score, and aggregate betting odds had a .189 Brier score.)

Let’s break that down by surface and compare the performance of ATP rankings, Elo, and surface-specific Elo. “F%” is the percentage of matches won by the favorite–as determined by that system, and “Br” is Brier score:

Surface  ATP F%  ATP Br  Elo F%  Elo Br  sElo F%  sElo Br  
Hard      67.3%   0.207   68.0%   0.205    68.5%    0.202  
Clay      66.1%   0.211   67.1%   0.211    67.0%    0.213  
Grass     66.0%   0.215   67.6%   0.207    68.5%    0.207

All three rating systems do best on hard courts, and for good reason: official rankings and overall Elo are more heavily weighted toward hard court results than they are clay or grass. Surface-specific Elo does best on hard courts for a similar reason: more data.

Already, though, we can see the unexpected divergence of clay and grass courts, especially with surface-specific Elo. It’s possible to explain overall Elo’s better performance on grass courts due to the presumed similarly between hard and grass–if a player excels on one, he’s probably good on the other, even if he’s horrible on clay. But that doesn’t explain sElo doing better on grass than on clay. There are 3.3 times as many tour-level matches on clay than on grass, so even allowing for the fact that players choose schedules to suit their surface preferences, almost everyone is going to have more results on dirt than on turf. More data should give us better results, but not here.

We can improve our forecasts even more by blending surface-specific ratings with overall ratings. After testing a wide range of possible mixes, it turns out that equally weighting Elo and sElo provides close to the best results. (The differences between, say, 60/40 and 50/50 are extremely small on all surfaces, so even where 60/40 is a bit better, I prefer to keep it simple with a half-and-half mix.) Here are the results for weighted surface Elos for all three surfaces:

Surface  ATP F%  ATP Br  
Hard      68.6%   0.202  
Clay      68.0%   0.207  
Grass     69.8%   0.196

Now grass courts are the most predictable of the major surfaces! Even when we use a weighted average of Elo and sElo, grass court forecasts rely on less data than those of the other surfaces–the surface-specific half of the grass court forecasts uses less than one-third the match results of clay court predictions and less than one-fifth the results of hard court forecasts. In fact, we can do at least as well–and perhaps a tiny bit better–with even less data: A 50/50 weighting of grass-specific Elo and hard-specific Elo is just as accurate as the half-and-half mix of grass-specific and overall Elo.

Regardless of the exact formula, it remains striking that we can predict ATP grass court results so accurately from such limited data. Even if one-third of ATP events were played on grass, I still wouldn’t have been surprised if grass court results turned out to be the least predictable. The more a surface favors the server–and it’s hardest to break on grass–the tighter the scoreline will tend to be, introducing more randomness into the end result. Despite that structural tendency, we’re able to pick winners as successfully on grass as on the more common surfaces.

Here’s my theory: Even though there aren’t many grass court events, the conditions at those few tournaments are quite consistent. Altitude is roughly sea level, groundskeepers follow the lead of the staff at Wimbledon, and rain clouds are almost always in sight. Compare that homogeneity to the variety of hard courts or clay courts. The high-altitude hard courts in Bogota are nothing like the slow ones in Indian Wells. The “clay” in Houston is only nominally equal to the crushed brick of Roland Garros. While grass courts are almost identical to each other, clay courts are nearly as different from each other as they are from other surfaces.

It makes sense that ratings based on a uniform surface would be more accurate than ratings based on a wide range of surfaces, and it’s reassuring to find that the limited available data doesn’t cancel out the advantage. This research also suggests a further path to better forecasts: grouping hard and clay matches by a more precise measure of surface speed. If 10% of tour matches are sufficient to make accurate grass court predictions, the same may be true of the slowest one-third of clay courts. More data is almost always better, but sometimes, precisely targeted data is best of all.

Is Jelena Ostapenko More Than the Next Iva Majoli?

Italian translation at settesei.it

Winning a Grand Slam as a teenager–or, in the case of this year’s French Open champion Jelena Ostapenko, a just-barely 20-year-old–is an impressive feat. But it isn’t always a guarantee of future greatness. Plenty of all-time greats launched their careers with Slam titles at age 20 or later, but three of the players who won their debut major at ages closest to Ostapenko’s serve as cautionary tales in the opposite direction: Iva Majoli, Mary Pierce, and Gabriela Sabatini. Each of these women was within three months of her 20th birthday when she won her first title, and of the three, only Pierce ever won another.

However, simply comparing her age to that of previous champions understates the Latvian’s achievement. Women’s tennis has gotten older over the last two decades: The average age of a women’s singles entrant in Paris this year was 25.6, a few days short of the record established at Roland Garros and Wimbledon last year. That’s two years older than the average player 15 years ago, and four years older than the typical entrant three decades ago. Headed into the French Open this year, there were only five teenagers ranked in the top 100; at the end of 2004, the year of Maria Sharapova’s and Svetlana Kuznetsova’s first major victories, there were nearly three times as many.

Thus, it doesn’t seem quite right to group Ostapenko with previous 19- and 20-year-old first-time winners. Instead, we might consider the Latvian’s “relative age”—the difference between her and the average player in the draw—of 5.68 years younger than the field. When I introduced the concept of relative age last week, it was in the context of Slam semifinalists, and in every era, there have been some very young players reaching the final four who burned out just as quickly. The same isn’t true of women who went on to win major titles.

In the last thirty years, only two players have won a major with a greater relative age than Ostapenko: Sharapova, who was 6.66 years younger than the 2004 US Open field, and Martina Hingis, who won three-quarters of the Grand Slam in 1997 at age 16, between 6.3 and 6.6 years younger than each tournament’s average entrant. The rest of the top five emphasizes Ostapenko’s elite company, including Monica Seles (5.29, at the 1990 French Open) and Serena Williams (5.26, at the 1999 US Open).

Each of those four women went on to reach the No. 1 ranking and win at least five majors–an outrageously optimistic forecast for Ostapenko, who, even after winning Roland Garros, is ranked outside the top ten. By relative age, Majoli, Pierce, and Sabatini are poor comparisons for Saturday’s champion–Majoli and Pierce were only three years younger than the fields they overcame, and Sabatini was only two years younger than the average entrant. By comparison, Garbine Muguruza, who won last year’s French Open at age 22, was two and a half years younger than the field.

Which is it, then? Unfortunately I don’t have the answer to that, and we probably won’t have a better idea for several more years. For most of the Open Era, until about ten years ago, the average age on the women’s tour fluctuated between 21 and 23. Thus, for the overall population of first-time major champions, actual age and relative age are very highly correlated. It’s only with the last decade’s worth of debut winners that the numbers meaningfully diverge. For Ostapenko and Muguruza–and perhaps Victoria Azarenka and Petra Kvitova–we have yet to see what their entire career trajectory will look like. To build a bigger sample to test the hypothesis, we’ll need a few more young first-time Slam winners, something we may finally see with Sharapova and Williams out of the way.

—

For more post-French Open analysis, here’s my Economist piece on Ostapenko and projecting major winners in the long term. Also at the Game Theory blog, I wrote about Rafael Nadal and his abssurd dominance on clay in a fast-court-friendly era.

Finally, check out Carl Bialik’s and my extra-long podcast, recorded Monday, with tons of thoughts and the winners and the fields in general.

Dominic Thiem and Reversible Blowouts

Italian translation at settesei.it

A few weeks ago in Rome, Dominic Thiem got destroyed by Novak Djokovic, 6-1 6-0. It was a letdown after Thiem’s previous-round upset of Rafael Nadal, and it seemed to provide a reminder of the old adage that tennis is about matchups. Even someone good enough to beat the King of Clay might struggle against a different sort of opponent.

Those struggles didn’t last. On Wednesday, Thiem faced Djokovic again, this time in the French Open quarterfinals, and won in straight sets. In less than three weeks, the Austrian bounced back from a brutal loss to defeat one of the greatest players of all time.

I’ve written before about the limited value of head-to-head records: When the head-to-head suggests that one player will win but the rankings disagree, the rankings prove to be the better forecaster. More sophisticated rating systems such as Elo would presumably do better still, though I haven’t done that exact test. There are certainly individual cases in which something specific about a matchup casts doubt on the predictiveness of the rankings, but if you have to pick one or the other, head-to-heads are the loser.

What about blowouts? Going into Wednesday’s quarterfinal, my surface-specific Elo ratings suggested that Thiem had a 26% chance of scoring the upset. The recent 6-1 6-0 loss was factored into those numbers, but only as a loss–there’s no consideration of severity. Should we have been even more skeptical of Thiem’s chances, given the most recent head-to-head result?

As it turns out, Thiem is far from the first player to turn things around after such a nasty scoreline. The most famous example is Robin Soderling, who lost 6-1 6-0 to Nadal in Rome in 2009, then bounced back to register one of the biggest upsets in tennis history, knocking out Rafa at Roland Garros. Few recoveries are so dramatic, but there are hundreds more.

Most players who lose lopsided scorelines–for today’s purposes, I’m considering any match in which the loser won two games or fewer–never get a chance to redeem themselves. I found roughly 2250 such matches in the ATP’s modern era, and the same two players met again less than half of those times. The fact that the head-to-head continues is a signal itself: Mediocre players–the ones you’d expect to lose badly–don’t get another chance. Even some top-20 players rarely meet each other on court, so the sort of player who earns the chance for redemption might have already proven that his lopsided loss was just an off day.

Of the 951 occasions that a player loses badly and faces the same opponent again, he gets revenge and wins the next match 277 times–about 29%. Crazy as it sounds, if the only thing we knew about Djokovic and Thiem entering Wednesday’s match was that Djokovic had won the last match 6-1 6-0, our base forecast would’ve been pretty close to the 26% that the much-more sophisticated Elo algorithm offered us.

29% is much higher than I expected, but it is lower than the typical rate for players in this situation. I found all head-to-heads of at least two meetings, and for every match after the first, counted whether it maintained or reversed the previous result. In addition to isolating lopsided scores, I also considered matches in which the loser won a set, on the assumption that those might be tighter matchups. Finally, for each of those categories, I tracked whether the follow-up matches were on the same surface as the previous one. Here are the results, with all win percentages shown from the perspective of the player who, like Thiem, lost the first encounter:

Score     Next Surface  Matches   Wins  Win %  
Any loss  All             68128  26586  39.0%  
Any loss  Same            31084  11855  38.1%  
Any loss  Diff            37044  14731  39.8%  
Bad loss  All               951    277  29.1%  
Bad loss  Same              457    128  28.0%  
Bad loss  Diff              494    149  30.2%  
Won set   All             26075  11286  43.3%  
Won set   Same            11766   4974  42.3%  
Won set   Diff            14309   6312  44.1%

The chances of recovering from a bad loss are better than I thought, but they are considerably worse than the odds that a player reverses the result after a less conspicuous scoreline–39%. The table also shows that the player seeking revenge is more likely to get it if the opportunity arises on a different surface, though not by a wide margin.

It’s clear that players are less likely to recover from a bad loss than from a more typical one, but how much of that is selection bias? After all, most of the players who lose 6-1 6-0 aren’t of the caliber of Thiem or Soderling, even if they are good enough to stick around in main draws and ultimately face the same opponent again.

To answer that question, I looked again at those 950 post-blowout matches, this time with pre-match Elo ratings. After eliminating everything before 1980 and a few other matchups with very little data, we were left with just under 600 data points. In this subset, Elo predicted that the players who lost badly had a 33.6% chance of winning the follow-up match. As we’ve seen, the actual success rate was 29%. Players who won lopsided matches outperformed their Elo forecast in the next meeting.

It’s not a huge difference, but enough to suggest that the matchup tells a little bit about how the next contest will go. One match can make a difference in the forecast–as long as it isn’t against Dominic Thiem.

—

Digging into the cases when a player lost badly and then recovered, I found a couple of entertaining examples:

Former No. 7 Harold Solomon beat Ivan Lendl in their first meeting, 6-1 6-1. Later that year, they met again at the US Open, and Lendl won, 6-1 6-0 6-0. Lendl also won their six matches after that.
Over the course of four years, Phil Dent and Mark Cox played three lopsided matches against each other. Cox won the first, Dent got revenge in the second, and Cox reversed things again in the third.

The Negative Impact of Time of Court

Italian translation at settesei.it

With 96 men’s matches in the books so far at Roland Garros this year, we’ve seen only one go to the absolute limit, past 6-6 in the fifth set. Still, we’ve had our share of lengthy, brutal five-set fights, including three matches in the first round that exceeded the four-hour mark. The three winners of those battles–Victor Estrella, David Ferrer, and Rogerio Dutra Silva–all fell to their second-round opponent.

A few years ago, I identified a “hangover effect” after Grand Slam marathons, defined as those matches that reach 6-6 in the fifth. Players who emerge victorious from such lengthy struggles would often already be considered underdogs in their next matches–after all, elite players rarely need to work so hard to advance–but marathon winners underperform even when we take their underdog status into account. (Earlier this week, I showed that women suffer little or no hangover effect after marathon third sets.)

A number of readers suggested I take a broader look at the effect of match length. After all, there are plenty of slugfests that fall just short of the marathon threshold, and some of those, like Ferrer’s loss yesterday to Feliciano Lopez, 6-4 in the final set, are more physically testing than some of those that reach 6-6. Match time still isn’t a perfect metric for potential fatigue–a four-hour match against Ferrer is qualitatively different from four hours on court with Ivo Karlovic–but it’s the best proxy we have for a very large sample of matches.

What happens next?

I took over 7,200 completed men’s singles matches from Grand Slams back to 2001 and separated them into groups by match time: one hour to 1:29, 1:30 to 2:00, and so on, up to a final category of 4:30 and above. Then I looked at how the winners of all those matches fared against their next opponents:

Prev Length   Matches  Wins  Win %  
1:00 to 1:29      448   275  61.4%  
1:30 to 1:59     1918  1107  57.7%  
2:00 to 2:29     1734   875  50.5%  
2:30 to 2:59     1384   632  45.7%  
3:00 to 3:29      976   430  44.1%  
3:30 to 3:59      539   232  43.0%  
4:00 to 4:29      188    64  34.0%  
4:30 and up        72    23  31.9%

The trend couldn’t be any clearer. If the only thing you know about a Slam matchup is how long the players spent on court in their previous match, you’d bet on the guy who recorded his last win in the shortest amount of time.

Of course, we know a lot more about the players than that. Andy Murray spent 3:34 on court yesterday, but even with his clay-court struggles this year, we would favor him in the third round against most of the men in the draw. As I’ve done in previous studies, let’s account for overall player skill by estimating the probability of each player winning each of these 7,200+ matches. Here are the same match-length categories, with “expected wins” (based on surface-specific Elo, or sElo) shown as well:

Prev Length   Wins  Exp Wins  Exp Win %  Ratio  
1:00 to 1:29   275       258      57.5%   1.07  
1:30 to 1:59  1107      1058      55.2%   1.05  
2:00 to 2:29   875       881      50.8%   0.99  
2:30 to 2:59   632       657      47.5%   0.96  
3:00 to 3:29   430       445      45.6%   0.97  
3:30 to 3:59   232       244      45.3%   0.95  
4:00 to 4:29    64        77      41.2%   0.83  
4:30 and up     23        30      42.1%   0.76

Again, there’s not much ambiguity in the trend here. Better players spend less time on court, so if you know someone beat their previous opponent in 1:14, you can infer that he’s a very good player. Often that assumption is wrong, but in the aggregate, it holds up.

The “Ratio” column shows the relationship between actual winning percentage (from the first table) and expected winning percentage. If previous match time had no effect, we’d expect to see ratios randomly hovering around 1. Instead, we see a steady decline from 1.07 at the top–meaning that players coming off of short matches win 7% more often than their skill level would otherwise lead us to forecast–to 0.76 at the bottom, indicating that competitors tend to underperform following a battle of 4:30 or longer.

It’s difficult to know whether we’re seeing a direct effect of time of court or a proxy for form. As good as surface-specific Elo ratings are, they don’t capture everything that could possibly predict the outcome of a match, especially micro-level considerations like a player’s comfort on a specific type of surface or at a certain tournament. sElo also needs a little time to catch up with players making fast improvements, particularly when they are very young. All this is to say that our correction for overall skill level will never be perfect.

Thus, a 75-minute win may improve a player’s chances by keeping him fresh for the next round … or it might tell us that–for whatever reason–he’s a stronger competitor right now than our model gives him credit for. One point in favor of the latter is that, at the most extreme, less time on court doesn’t help: Players don’t appear to benefit from advancing via walkover. That isn’t a slam-dunk argument–some commentators believe that walkovers could be detrimental due to the long resulting layoff at a Slam–but it does show us that less time on court isn’t always a positive.

Whatever the underlying cause, we can tweak our projections accordingly. Murray could be a little weaker than usual tomorrow after his length battle yesterday with Martin Klizan. Albert Ramos, the only man to complete a second-rounder in less than 90 minutes, might be playing a bit better than his rating suggest. It’s certainly evident that match time has something to tell us even when players aren’t stretched to the breaking point of a marathon fifth set.

Men’s Doubles On the Dirt

Angelique Kerber wasn’t the only top seed to crash out early at this year’s French Open. In the men’s doubles draw, the top section opened up when Henri Kontinen and John Peers, the world’s top-ranked team, lost to the Spanish pair of David Marrero and Tommy Robredo. It’s plausible to attribute the upset to the clay, as Kontinen-Peers have tallied a pedestrian five wins against four losses on the dirt this season and one could guess that the Spaniards are at their strongest on clay.

Fortunately we don’t have to guess. Using a doubles variant of sElo–surface-specific Elo, which I began writing about a few days ago in the context of women’s singles–we can make rough estimates of how Kontinen/Peers would fare against Marrero/Robredo on each surface. The top seeds are solid on all surfaces–less than a year ago, they won a clay title in Hamburg–but stronger on hard courts. sElo ranks them 4th and 8th on hard, but 10th and 13th on clay among tour regulars. Marrero is the surface-specialist of the bunch, ranking 37th on clay and 78th on hard. Robredo throws a wrench into the exercise, as he has played very little doubles recently, only eight events since the beginning of 2016.

Using these numbers–including those derived from Robredo’s limited sample–we find that sElo would have given Kontinen/Peers a 73.6% chance of winning yesterday, compared to a 78.3% advantage on a hard court. Even if we adjust Robredo’s clay-court sElo to something closer to his all-surface rating, the top seeds still look like 69% favorites.

A more striking example comes from yesterday’s other big upset, in which Julio Peralta and Horacio Zeballos took out Feliciano Lopez and Marc Lopez. On any surface, the Lopezes are the superior team, but Peralta and Zeballos have a much larger surface differential:

Player    Hard sElo  Clay sElo  
M Lopez        1720       1804  
F Lopez        1713       1772  
Zeballos       1651       1756  
Peralta        1517       1770

On a hard court, sElo gives the Lopezes a 68.1% chance of winning this matchup. But on clay, the gap narrows all the way to 53.6%. It’s still a bit of an upset for the South Americans, but not one that should come as much of a surprise.

Mismatches

I’ve speculated in the past that surface preferences aren’t as pronounced in doubles as they are in singles. Regardless of surface, points are shorter, and many teams position one player at the net even on the dirt. While some hard-courters are probably uncomfortable on clay (and vice versa), I wouldn’t expect the effects to be as substantial as they are in singles.

The numbers tell a different story. Here are the top ten, ranked by hard court sElo:

Rank  Player          Hard sElo  
1     Jack Sock            1947  
2     Nicolas Mahut        1893  
3     Marcelo Melo         1883  
4     Henri Kontinen       1879  
5     P-H Herbert          1862  
6     Bob Bryan            1851  
7     Mike Bryan           1846  
8     John Peers           1842  
9     Bruno Soares         1829  
10    Jamie Murray         1828

By clay court sElo:

Rank  Player                Clay sElo  
1     Mike Bryan                 1950  
2     Bob Bryan                  1950  
3     P-H Herbert                1894  
4     Nicolas Mahut              1889  
5     Jack Sock                  1887  
6     Robert Farah               1850  
7     Juan Sebastian Cabal       1849  
8     Pablo Cuevas               1824  
9     Rohan Bopanna              1812  
10    John Peers                 1810

Jamie Murray and Bruno Soares, who appear in the hard court top ten, sit outside the top 25 in clay court sElo. Robert Farah and Juan Sebastian Cabal are 41st and 42nd in hard court sElo, despite ranking in the clay court top seven. Pablo Cuevas, another clay court top-tenner, is 87th on the hard court list.

To go beyond these anecdotes–noteworthy as they are–we need to compare the level of surface preference in men’s doubles to other tours. To do that, I calculated the correlation coefficent between hard court and clay court sElo for the top 50 players (ranked by overall Elo) in men’s doubles, men’s singles, and women’s singles. (I don’t yet have an adequate database to generate ratings for women’s doubles.)

In other words, we’re testing how much a player’s results on one surface predict his or her results on the other major surface. The higher the correlation coefficient, the more likely it is that a player will have similar results on hard and clay. Here’s how the tours compare:

Tour             Correl  
Men's Singles     0.708  
Women's Singles   0.417  
Men's Doubles     0.323

In contrast to my hypothesis above, surface preferences in men’s doubles appear to be much stronger than in either men’s or women’s singles. (And there’s a huge difference between men’s and women’s singles, but that’s a subject for another day.)

Randomness

I suspect that the low correlation of surface-specific Elos in men’s doubles is partly due to the more random nature of doubles results. Because the event is more serve-dominated, there are more close sets ending in tiebreaks, and because of the no-ad, super-tiebreak format used outside of Slams, tight matches are decided by a smaller number of points. Thus, every doubles player’s results–and their various Elo ratings–reflect the influence of chance more than the singles results are.

Another consideration–one that I haven’t yet made sense of–is that surface-specific ratings don’t improve doubles forecasts they way that they do men’s and women’s singles predictions. As I wrote on Sunday, sElo represents a big improvement over surface-neutral Elo for women’s forecasts, and in an upcoming post, I’ll be able to make some similar observations for the men’s game. Using Brier score, a measure of the calibration of predictions, we can see the effect of using surface-specific Elo ratings in 2016 tour-level matches:

Tour             Elo Brier  sElo Brier  
Men's Singles        0.202       0.169  
Women's Singles      0.220       0.179  
Men's Doubles        0.171       0.181

The lower the Brier score, the more accurate the forecasts. This isn’t a fluke of 2016: The differences in men’s doubles Brier scores are around 0.01 for each of the last 15 seasons. By this measure, Elo does a very good job predicting the outcome of men’s doubles matches, but the surface-specific sElo represents a small step back. It could be that the smaller sample–using only one surface’s worth of results–is more damaging to forecasts in doubles than it is in singles.

Doubles analytics is particularly uncharted territory, and there’s plenty of work remaining for researchers even in this narrow subtopic. There’s lots of work to do for the world’s top doubles players as well, now that we can point to a noticeably weaker surface for so many of them.

The Steadily Less Predictable WTA

Italian translation at settesei.it

Update: The numbers in this post summarizing the effectiveness of sElo are much too high–a bug in my code led to calculating effectiveness with post-match ratings instead of pre-match ratings. The parts of the post that don’t have to do with sElo are unaffected and–I hope–remain of interest.

One of the talking points throughout the 2017 WTA season has been the unpredictability of the field. With the absence of Serena Williams, Victoria Azarenka, and until recently, Petra Kvitova and Maria Sharapova, there is a dearth of consistently dominant players. Many of the top remaining players have been unsteady as well, due to some combination of injury (Simona Halep), extreme surface preferences (Johanna Konta), and good old-fashioned regression to the mean (Angelique Kerber).

No top seed has yet won a title at the Premier level or above so far this year. Last week, Stephanie Kovalchik went into more detail, quantifying how seeds have failed to meet expectations and suggesting that the official WTA ranking system–the algorithm that determines which players get those seeds–has failed.

There are plenty of problems with the WTA ranking system, especially if you expect it to have predictive value–that is, if you want it to properly reflect the performance level of players right now. Kovalchik is correct that the rankings have done a particularly poor job this year identifying the best players. However, there’s something else going on: According to much more accurate algorithms, the WTA is more chaotic than it has been for decades.

Picking winners

Let’s start with a really basic measurement: picking winners. Through Rome, there had been more than 1100 completed WTA matches. The higher-ranked player won 62.4% of those. Since 1990, the ranking system has picked the winner of 67.9% of matches, and topped 70% during several years in the 1990s. It never fell below 66% until 2014, and this year’s 62.4% is the worst in the 28-year time frame under consideration.

Elo does a little better. It rates players by the quality of their opponents, meaning that draw luck is taken out of the equation, and does a better job of estimating the ability level of players like Serena and Sharapova, who for various reasons have missed long stretches of time. Since 1990, Elo has picked the winner of 68.6% of matches, falling to an all-time low of 63.1% so far in 2017.

For a big improvement, we need surface-specific Elo (sElo). An effective surface-based system isn’t as complicated as I expected it to be. By generating separate rankings for each surface (using only matches on that surface), sElo has correctly predicted the winner of 76.2% of matches since 1990, almost cracking 80% back in 1992. Even sElo is baffled by 2017, falling to it’s lowest point of 71.0% in 2017.

(sElo for all three major surfaces is now shown on the Tennis Abstract Elo ratings report.)

This graph shows how effectively the three algorithms picked winners. It’s clear that sElo is far better, and the graph also shows that some external factor is driving the predictability of results, affecting the accuracy of all three systems to a similar degree:

Brier scores

We see a similar effect if we use a more sophisticated method to rate the WTA ranking system against Elo and sElo. The Brier score of a collection of predictions measures not only how accurate they are, but also how well calibrated they are–that is, a player forecast to win a matchup 90% of the time really does win nine out of ten, not six out of ten, and vice versa. Brier scores average the square of the difference between each prediction and its corresponding result. Because it uses the square, very bad predictions (for instance, that a player has a 95% chance of winning a match she ended up losing) far outweigh more pedestrian ones (like a player with a 95% chance going on to win).

In 2017 so far, the official WTA ranking system has a Brier score of .237, compared to Elo of .226 and sElo of .187. Lower is better, since we want a system that minimizes the difference between predictions and actual outcomes. All three numbers are the highest of any season since 1990. The corresponding averages over that time span are .207 (WTA), .202 (Elo), and .164 (sElo).

As with the simpler method of counting correct predictions, we see that Elo is a bit better than the official ranking, and both of the surface-agnostic methods are crushed by sElo, even though the surface-specific method uses considerably less data. (For instance, the clay-specific Elo ignores hard and grass court results entirely.) And just like the results of picking winners, we see that the differences in Brier scores of the three methods are fairly consistent, meaning that some other factor is causing the year-to-year differences:

The takeaway

The WTA ranking system has plenty of issues, but its unusually bad performance this year isn’t due to any quirk in the algorithm. Elo and sElo are structured completely differently–the only thing they have in common with the official system is that they use WTA match results–and they show the same trends in both of the above metrics.

One factor affecting the last two years of forecasting accuracy is the absence of players like Serena, Sharapova, and Azarenka. If those three played full schedules and won at their usual clip, there would be quite a few more correct predictions for all three systems, and perhaps there would be fewer big upsets from the players who have tried to replace them at the top of the game.

But that isn’t the whole story. A bunch of no-brainer predictions don’t affect Brier score very much, and the presence of heavily-favored players also make it more likely that massively surprising results occur, such as Serena’s loss to Madison Brengle, or Sharapova’s ouster at the hands of Eugenie Bouchard. Many unexpected results are completely independent of the top ten, like Marketa Vondrousova’s recent title in Biel.

While some of the year-to-year differences in the graphs above are simply noise, the last several years looks much more like a meaningful trend. It could be that we are seeing a large-scale changing of a guard, with young players (and their low rankings) regularly upsetting established stars, while the biggest names in the sport are spending more time on the sidelines. Upsets may also be somewhat contagious: When one 19-year-old aspirant sees a peer beating top-tenners, she may be more confident that she can do the same.

Whatever influences have given us the WTA’s current state of unpredictability, we can see that it’s not just a mirage created by a flawed ranking system. Upsets are more common now than at any other point in recent memory, whichever algorithm you use to pick your favorites.

Del Potro’s Draws and the Possible Persistence of Bad Luck

Italian translation at settesei.it

Tennis’s draw gods have not been kind to Juan Martin del Potro this year.

In Acapulco and Indian Wells, he drew Novak Djokovic as his second-match opponent. In Miami, Delpo got a third-rounder with Roger Federer. In each of the March Masters events, with 1,000 ranking points at stake, del Potro was handed the most difficult opponents for his first round against a fellow seed. Thanks in part to the resulting early exits, one of the most dangerous players on tour is still languishing outside of the top 30 in the ATP rankings.

When I wrote about the Indian Wells quarter of death–the section of the draw containing del Potro, Djokovic, Federer, Rafael Nadal, and Nick Kyrgios–I attempted to quantify the effect of the draw on each player’s expected ranking points. Before each player’s name was placed in the bracket, my model predicted that Delpo would earn about 150 ranking points–the weighted average of his likelihood of reaching the third round, the fourth round, and so on–and after the draw was conducted, his higher probability of a clash with Djokovic knocked that number down to just over 100. That negative effect was one of the worst of any player in the tournament.

The story in Miami is similar, if less extreme. Pre-draw, Delpo’s expected points were 183. Post draw: 155. In the four tournaments he has entered this year, he has been uniformly unlucky:

Tournament    Pre-Draw  Post-Draw  Effect  
Delray Beach      89.3       74.0  -17.1%  
Acapulco         121.5       97.1  -20.1%  
Indian Wells     154.6      102.5  -33.7%  
Miami            182.9      155.4  -15.0%  
TOTAL            548.2      429.0  -21.7%

*The numbers above for Indian Wells are slightly different than what I published in the Indian Wells article, since the simulations I ran for this post consider the entire 96-player field, not just the 64-player second round.

The good news, as we’ll see, is that it’s virtually impossible for this degree of misfortune to continue. The bad news is that those 119 points are gone forever, and at Delpo’s current position in the ranking table, that disadvantage will affect his tournament seeds, which in turn will result in worse draws (earlier meetings with higher-ranked players, independent of luck) for at least another few weeks.

Before we go any further, let me review the methodology I’m using here. (If you’re not interested, skip this paragraph.) For “post-draw” expected points, I’m taking jrank-based forecasts–like the ones on the front page of Tennis Abstract–and using each player’s probability of each round to calculate a weighted average of expected points. “Pre-draw” forecasts are much more computationally demanding. In Miami, for instance, Delpo could’ve faced any of the 64 unseeded players in the second round and been slated to meet any of the top eight seeds in the third round. For each tournament, I ran a Monte Carlo simulation with the tournament seeds, generating a new draw and simulating the tournament–100,000 times, then summing all those outcomes. So in the pre-draw forecast, Delpo had a one-eighth chance of getting Fed in the third round, a one-eighth chance of getting Kei Nishikori there, and so on.

It seems clear that a 22%, 119-point rankings hit over the course of four tournaments is some seriously bad luck. Last year, there were about 750 instances of a player being seeded at an ATP tournament, and in fewer than 60 of those, the draw resulted in an effect of -22% or worse on the player’s expected ranking points. And that’s just one tournament! The odds that Delpo would get such a rough deal in all four of his 2017 tournaments are 1 in more than 20,000.

Over the course of a full season, draw luck mostly evens out. It’s rare to see an effect of more than 10% in either direction. Last year, Thiemo de Bakker saw a painful difference of 18% between his pre-draw and post-draw expected points in 12 ATP events, but everyone else with at least that many tournaments fell between -11% and +11%, with three-quarters of players between -5% and +5%. Even when draw luck doesn’t balance itself out, the effect isn’t as bad as what Delpo has seen in 2017.

Del Potro’s own experience in 2016 is a case in point. His most memorable event of the season was the Olympics, where he drew Djokovic in the first round, so it’s easy to recall his year as being equally riddled with bad luck. But in his 12 other ATP events, the draw aided him in six–including a +34% boost at the US Open–and hurt him at the other six. Altogether, his 2016 ATP draws gave him a 5.9% advantage over his “pre-draw” expected points–a bonus of 17 ranking points. (I didn’t include the Olympics, since no ranking points were awarded there.)

Taken together, Delpo’s 2016-17 draws have deprived him of about 100 ranking points, which would move him three spots up the ranking table. So even with a short stretch of extreme misfortune, draw luck hasn’t affected him that much. Last year’s most extreme case among elite players, Richard Gasquet, suffered a similar effect: His draws knocked down his expected take by 9%, or 237 points, a difference that would bump him up from #22 to #19 in this week’s ranking list.

There are many reasons to believe that del Potro is a much better player than his current ranking suggests, such as his Elo rating, which stands at No. 7. But his ATP ranking reflects his limited schedule and modest start last year much more than it does the vagaries of each week’s brackets. The chances are near zero that he will continue to draw the toughest player in each tournament’s field in the earliest possible round, so we’ll soon have a better idea of what exactly he is capable of, and where exactly he should stand in the rankings.

The Indian Wells Quarter of Death

Italian translation at settesei.it

The Indian Wells men’s draw looks a bit lopsided this year. The bottom quarter, anchored by No. 2 seed Novak Djokovic, also features Roger Federer, Rafael Nadal, Juan Martin del Potro, and Nick Kyrgios. It doesn’t take much analysis to see that the bracket makes life more difficult for Djokovic, and by extension, it cleared the way for Andy Murray. Alas, Murray lost his opening match against Vasek Pospisil on Saturday, making No. 3 seed Stan Wawrinka the luckiest man in the desert.

The draw sets up some very noteworthy potential matches: Federer and Nadal haven’t played before the quarterfinal since their first encounter back in 2004, and Fed hasn’t played Djokovic before the semis in more than 40 meetings, since 2007. Kyrgios, who has now beaten all three of the elites in his quarter, is likely to get another chance to prove his mettle against the best.

I haven’t done a piece on draw luck for awhile, and this seemed like a great time to revisit the subject. The principle is straightforward: By taking the tournament field and generating random draws, we can do a sort of “retro-forecast” of what each player’s chances looked like before the draw was conducted–back when Djokovic’s road wouldn’t necessarily be so rocky. By comparing the retro-forecast to a projection based on the actual draw, we can see how much the luck of the draw impacted each player’s odds of piling up ranking points or winning the title.

Here are the eight players most heavily favored by the pre-draw forecast, along with the their chances of winning the title, both before and after the draw was conducted:

Player                 Pre-Draw  Post-Draw  
Novak Djokovic           26.08%     19.05%  
Andy Murray              19.30%     26.03%  
Roger Federer            10.24%      8.71%  
Rafael Nadal              5.46%      4.80%  
Stan Wawrinka             5.08%      7.14%  
Kei Nishikori             5.01%      5.67%  
Nick Kyrgios              4.05%      2.62%  
Juan Martin del Potro     4.00%      2.34%

These odds are based on my jrank rating system, which correlates closely with Elo. I use jrank here instead of Elo because it’s surface-specific. I’m also ignoring the first round of the main draw, which–since all 32 seeds get a first-round bye–is just a glorified qualifying round and has very little effect on the title chances of seeded players.

As you can see, the bottom quarter–the “group of death”–is in fact where title hopes go to die. Djokovic, who is still considered to be the best player in the game by both jrank and Elo, had a 26% pre-draw chance of defending his title, but it dropped to 19% once the names were placed in the bracket. Not coincidentally, Murray’s odds went in the opposite direction. Federer’s and Nadal’s title chances weren’t hit quite as hard, largely because they weren’t expected to get past Djokovic, no matter when they faced him.

The issue here isn’t just luck, it’s the limitation of the ATP ranking system. No one really thinks that del Potro entered the tournament as the 31st favorite, or that Kyrgios came in as the 15th. No set of rankings is perfect, but at the moment, the official rankings do a particularly poor job of reflecting the players with the best chances of winning hard court matches. The less reliable the rankings, the better chance of a lopsided draw like the one in Indian Wells.

For a more in-depth look at the effect of the draw on players with lesser chances of winning the title, we need to look at “expected ranking points.” Using the odds that a player reaches each round, we can calculate his expected points for the entire event. For someone like Kyle Edmund, who would have almost no chance of winning the title regardless of the draw, expected points tells a more detailed story of the power of draw luck. Here are the ten players who were punished most severely by the bracket:

Player                 Pre-Draw Pts Post-Draw Pts  Effect  
Kyle Edmund                    28.8          14.3  -50.2%  
Steve Johnson                  65.7          36.5  -44.3%  
Vasek Pospisil                 29.1          19.4  -33.2%  
Juan Martin del Potro         154.0         104.2  -32.3%  
Stephane Robert                20.3          14.2  -30.1%  
Federico Delbonis              20.0          14.5  -27.9%  
Novak Djokovic                429.3         325.4  -24.2%  
Nick Kyrgios                  163.5         124.6  -23.8%  
Horacio Zeballos               17.6          14.1  -20.0%  
Alexander Zverev              113.6          91.5  -19.4%

At most tournaments, this list is dominated by players like Edmund and Pospisil: unseeded men with the misfortune of drawing an elite opponent in the first round. Much less common is to see so many seeds–particularly a top-two player–rating as the most unlucky. While Federer and Nadal don’t quite make the cut here, the numbers bear out our intuition: Fed’s draw knocked his expected points from 257 down to 227, and Nadal’s reduced his projected tally from 195 to 178.

The opposite list–those who enjoyed the best draw luck–features a lot of names from the top half, including both Murray and Wawrinka. Murray squandered his good fortune, putting Wawrinka in an even better position to take advantage of his own:

Player              Pre-Draw Pts  Post-Draw Pts  Effect  
Malek Jaziri                21.9           31.6   44.4%  
Damir Dzumhur               29.1           39.0   33.9%  
Martin Klizan               27.6           36.4   32.1%  
Joao Sousa                  24.7           31.1   25.9%  
Peter Gojowczyk             20.4           25.5   24.9%  
Tomas Berdych               93.6          116.6   24.6%  
Mischa Zverev               58.5           72.5   23.8%  
Yoshihito Nishioka          26.9           32.6   21.1%  
John Isner                  80.2           97.0   21.0%  
Andy Murray                369.1          444.2   20.3%  
Stan Wawrinka              197.8          237.7   20.1%

Over the course of the season, quirks like these tend to even out. Djokovic, on the other hand, must be wondering how he angered the draw gods: Just to earn a quarter-final place against Roger or Rafa, he’ll need to face Kyrgios and Delpo for the second consecutive tournament.

If Federer, Kyrgios, and del Potro can bring their ATP rankings closer in line with their true talent, they are less likely to find themselves in such dangerous draw sections. For Djokovic, that would be excellent news.

Measuring the Performance of Tennis Prediction Models

With the recent buzz about Elo rankings in tennis, both at FiveThirtyEight and here at Tennis Abstract, comes the ability to forecast the results of tennis matches. It’s not far fetched to ask yourself, which of these different models perform better and, even more interesting, how they fare compared to other ‘models’, such as the ATP ranking system or betting markets.

For this, admittedly limited, investigation, we collected the (implied) forecasts of five models, that is, FiveThirtyEight, Tennis Abstract, Riles, the official ATP rankings, and the Pinnacle betting market for the US Open 2016. The first three models are based on Elo. For inferring forecasts from the ATP ranking, we use a specific formula¹ and for Pinnacle, which is one of the biggest tennis bookmakers, we calculate the implied probabilities based on the provided odds (minus the overround)².

Next, we simply compare forecasts with reality for each model asking If player A was predicted to be the winner ($latex P(a) > 0.5$), did he really win the match? When we do that for each match and each model (ignoring retirements or walkovers) we come up with the following results.

Model		% correct
Pinnacle	76.92%
538		75.21%
TA		74.36%
ATP		72.65%
Riles		70.09%

What we see here is how many percent of the predictions were actually right. The betting model (based on the odds of Pinnacle) comes out on top followed by the Elo models of FiveThirtyEight and Tennis Abstract. Interestingly, the Elo model of Riles is outperformed by the predictions inferred from the ATP ranking. Since there are several parameters that can be used to tweak an Elo model, Riles may still have some room left for improvement.

However, just looking at the percentage of correctly called matches does not tell the whole story. In fact, there are more granular metrics to investigate the performance of a prediction model: Calibration, for instance, captures the ability of a model to provide forecast probabilities that are close to the true probabilities. In other words, in an ideal model, we want 70% forecasts to be true exactly in 70% of the cases. Resolution measures how much the forecasts differ from the overall average. The rationale here is, that just using the expected average values for forecasting will lead to a reasonably well-calibrated set of predictions, however, it will not be as useful as a method that manages the same calibration while taking current circumstances into account. In other words, the more extreme (and still correct) forecasts are, the better.

In the following table we categorize the set of predictions into bins of different probabilities and show how many percent of the predictions were correct per bin. This also enables us to calculate Calibration and Resolution measures for each model.

Model    50-59%  60-69%  70-79%  80-89%  90-100% Cal  Res   Brier
538      53%     61%     85%     80%     91%     .003 .082  .171
TA       56%     75%     78%     74%     90%     .003 .072  .182
Riles    56%     86%     81%     63%     67%     .017 .056  .211
ATP      50%     73%     77%     84%     100%    .003 .068  .185
Pinnacle 52%     91%     71%     77%     95%     .015 .093  .172

As we can see, the predictions are not always perfectly in line with what the corresponding bin would suggest. Some of these deviations, for instance the fact that for the Riles model only 67% of the 90-100% forecasts were correct, can be explained by small sample size (only three in that case). However, there are still two interesting cases (marked in bold) where sample size is better and which raised my interest. Both the Riles and Pinnacle models seem to be strongly underconfident (statistically significant) with their 60-69% predictions. In other words, these probabilities should have been higher, because, in reality, these forecasts were actually true 86% and 91% percent of the times.³ For the betting aficionados, the fact that Pinnacle underestimates the favorites here may be really interesting, because it could reveal some value as punters would say. For the Riles model, this would maybe be a starting point to tweak the model.

In the last three columns Calibration (the lower the better), Resolution (the higher the better), and the Brier score (the lower the better) are shown. The Brier score combines Calibration and Resolution (and the uncertainty of the outcomes) into a single score for measuring the accuracy of predictions. The models of FiveThirtyEight and Pinnacle (for the used subset of data) essentially perform equally good. Then there is a slight gap until the model of Tennis Abstract and the ATP ranking model come in third and fourth, respectively. The Riles model performs worst in terms of both Calibration and Resolution, hence, ranking fifth in this analysis.

To conclude, I would like to show a common visual representation that is used to graphically display a set of predictions. The reliability diagram compares the observed rate of forecasts with the forecast probability (similar to the above table).

The closer one of the colored lines is to the black line, the more reliable the forecasts are. If the forecast lines are above the black line, it means that forecasts are underconfident, in the opposite case, forecasts are overconfident. Given that we only investigated one tournament and therefore had to work with a low sample size (117 predictions), the big swings in the graph are somewhat expected. Still, we can see that the model based on ATP rankings does a really good job in preventing overestimations even though it is known to be outperformed by Elo in terms of prediction accuracy.

To sum up, this analysis shows how different predictive models for tennis can be compared among each other in a meaningful way. Moreover, I hope I could exhibit some of the areas where a model is good and where it’s bad. Obviously, this investigation could go into much more detail by, for example, comparing the models in how well they do for different kinds of players (e.g., based on ranking), different surfaces, etc. This is something I will spare for later. For now, I’ll try to get my sleeping patterns accustomed to the schedule of play for the Australian Open, and I hope, you can do the same.

—

Peter Wetz is a computer scientist interested in racket sports and data analytics based in Vienna, Austria.

Footnotes

1. $latex P(a) = a^e / (a^e + b^e) $ where $latex a $ are player A’s ranking points, $latex b $ are player B’s ranking points, and $latex e $ is a constant. We use $latex e = 0.85 $ for ATP men’s singles.

2. The betting market in itself is not really a model, that is, the goal of the bookmakers is simply to balance their book. This means that the odds, more or less, reflect the wisdom of the crowd, making it a very good predictor.

3. As an example, one instance, where Pinnacle was underconfident and all other models were more confident is the R32 encounter between Ivo Karlovic and Jared Donaldson. Pinnacle’s implied probability for Karlovic to win was 64%. The other models (except the also underconfident Riles model) gave 72% (ATP ranking), 75% (FiveThirtyEight), and 82% (Tennis Abstract). Turns out, Karlovic won in straight sets. One factor at play here might be that these were the US Open where more US citizens are likely to be confident about the US player Jared Donaldson and hence place a bet on him. As a consequence, to balance the book, Pinnacle will lower the odds on Donaldson, which results in higher odds (and a lower implied probability) for Karlovic.

Forecasting Davis Cup Doubles

One of the most enjoyable aspects of Davis Cup is the spotlight it shines on doubles. At ATP events, doubles matches are typically relegated to poorly-attended side courts. In Davis Cup, doubles gets a day of its own, and crowds turn out in force. Even better, the importance of Davis Cup inspires many players who normally skip doubles to participate.

Because singles specialists are more likely to play doubles, and because most Davis Cup doubles teams are not regular pairings, forecasting these matches is particularly difficult. In the past, I haven’t even tried. But now that we have D-Lo–Elo ratings for doubles–it’s a more manageable task.

To my surprise, D-Lo is even more effective with Davis Cup than it is with regular-season tour-level matches. D-Lo correctly predicts the outcome of about 65% of tour-level doubles matches since 2003. For Davis Cup World Group and World Group Play-Offs in that time frame, D-Lo is right 70% of the time. To put it another way, this is more evidence that Davis Cup is about the chalk.

What’s particularly odd about that result is that D-Lo itself isn’t that confident in its Davis Cup forecasts. For ATP events, D-Lo forecasts are well-calibrated, meaning that if you look at 100 matches where the favorite is given a 60% chance of winning, the favorite will win about 60 times. For the Davis Cup forecasts, D-Lo thinks the favorite should win about 60% of the time, but the higher-rated team ends up winning 70 matches out of 100.

Davis Cup’s best-of-five format is responsible for part of that discrepancy. In a typical ATP doubles match, the no-ad scoring and third-set tiebreak introduce more luck into the mix, making upsets more likely. A matchup that would result in a 60% forecast in the no-ad, super-tiebreak format translates to a 64.5% forecast in the best-of-five format. That accounts for about half the difference: Davis Cup results are less likely to be influenced by luck.

The other half may be due to the importance of the event. For many players, regular-season doubles matches are a distant second priority to singles, so they may not play at a consistent level from one match to the next. In Davis Cup, however, it’s a rare competitor who doesn’t give the doubles rubber 100% of their effort. Thus, we appear to have quite a few matches in which D-Lo picks the winner, but since it uses primarily tour-level results, it doesn’t realize how heavily the winner should have been favored.

Incidentally, home-court advantage doesn’t seem to play a big role in doubles outcomes. The hosting side has won 52.6% of doubles matches, an edge which could have as much to do with hosts’ ability to choose the surface as it is does with screaming crowds and home cooking. This isn’t a factor that affects D-Lo forecasts, as the system’s predictions are as accurate when it picks the away side as when it picks the home side.

Forecasting Argentina-Croatia doubles

Here are the D-Lo ratings for the eight nominated players this weekend. The asterisks indicate those players who are currently slated to contest tomorrow’s doubles rubber:

Player                 Side  D-Lo     
Juan Martin del Potro  ARG   1759     
Leonardo Mayer         ARG   1593  *  
Federico Delbonis      ARG   1540     
Guido Pella            ARG   1454  *  
                                      
Ivan Dodig             CRO   1856  *  
Marin Cilic            CRO   1677     
Ivo Karlovic           CRO   1580     
Franco Skugor          CRO   1569  *

As it stands now, Croatia has a sizable advantage. Based on the D-Lo ratings of the currently scheduled doubles teams, the home side has a 189-point edge, which converts to a 74.8% probability of winning. But remember, that’s the chance of winning a no-ad, super-tiebreak match, with all the luck that entails. In best-of-five, that translates to a whopping 83.7% chance of winning.

Making matters worse for Argentina, it’s likely that Croatia could improve their side. Argentina could increase their odds of winning the doubles rubber by playing Juan Martin del Potro, but given Delpo’s shaky physical health, it’s unlikely he’ll play all three days. Marin Cilic, on the other hand, could very well play as much as possible. A Cilic-Ivan Dodig pairing would have a 243-point advantage over Leonardo Mayer and Guido Pella, which translates to an 89% chance of winning a best-of-five match. Even Mayer’s Davis Cup heroics are unlikely to overcome a challenge of that magnitude.

Given the likelihood that Pella will sit on the bench for every meaningful singles match, it’s easy to wonder if there is a better option. Sure enough, in Horacio Zeballos, Argentina has a quality doubles player sitting at home. The two-time Grand Slam doubles semifinalist has a current D-Lo rating of 1758, almost identical to del Potro’s. Paired with Mayer, Zeballos would bring Argentina’s chances of upsetting a Dodig-Franco Skugor team to 43%. Zeballos-Mayer would also have a 32% chance of defeating Dodig-Cilic.

A full Argentina-Croatia forecast

With the doubles rubber sorted, let’s see who is likely to win the 2016 Davis Cup. Here are the Elo– and D-Lo-based forecasts for each currently-scheduled match, shown from the perspective of Croatia:

Rubber                      Forecast (CRO)  
Cilic v Delbonis                     90.8%  
Karlovic v del Potro                 15.8%  
Dodig/Skugor v Mayer/Pella           83.7%  
Cilic v del Potro                    36.3%  
Karlovic v Delbonis                  75.8%

Elo still believes Delpo is an elite-level player, which is why it makes him the favorite in the pivotal fourth rubber against Cilic. The system is less positive about Federico Delbonis, who it ranks 68th in the world, against his #41 spot on the ATP computer.

These match-by-match forecasts imply a 74.2% probability that Croatia will win the tie. That’s more optimistic than the betting market which, a few hours before play begins, gives Croatia about a 65% edge.

However, most of the tweaks we could make would move the needle further toward a Croatia victory. Delpo’s body may not allow him to play two singles matches at full strength, and the gap in singles skill between him and Mayer is huge. Croatia could improve their doubles chances if Cilic plays. And if there is a home-court or surface advantage, it would probably work against the South Americans.

Even more likely than a Croatian victory is a 1-1 split of the first two matches. If that happens, everything will hang in the balance tomorrow, when the world tunes in to watch a doubles match.