Elo Ratings – Page 4 – Heavy Topspin

Picking Favorites With Better Davis Cup Rankings

Yesterday, the ITF announced the seedings for the first new-look Davis Cup Finals, to be held in Madrid this November. The 18-country field was completed by the 12 home-and-way ties contested last weekend. Those 12 winners will join France, Croatia, Spain, and USA (last year’s semi-finalists) along with the two wild cards, recent champions Argentina and Great Britain.

The six nations who skipped the qualifying round will make up five of the top six seeds. (Spain is 7th, while Belgium, who had to qualify, is 4th.) The preliminary round of the November event will feature six round-robin groups of three, each consisting of one top-six seed, a second country ranked 7-12, and a third ranked 13-18. Seeding really matters, as a top position (deserved or not!) guarantees that a side will avoid dangerous opponents like last year’s finalists France and Croatia. Even the difference between 12 and 13 could prove decisive, as a 7-through-12 spot ensures that a nation will steer clear of the always-strong Spaniards, who are seeded 7th.

The seeds are based on the Davis Cup’s ranking system, which relies entirely on previous Davis Cup results. While the formula is long-winded, the concept is simple: A country gets more points for advancing further each season, and recent years are worth the most. The last four years of competition are taken into consideration. It’s not how I would do it, but the results aren’t bad. Four or five of the top six seeds will field strong sides, and one of the exceptions–Great Britain–would have done so had Andy Murray’s hip cooperated. Spain is obviously misranked, but given the limitations of the Davis Cup ranking system, it’s understandable, as the 2011 champions spent 2015 and 2016 languishing outside the World Group.

We can do better

The Davis Cup rankings have several flaws. First, they rely heavily on a lot of old results. If we’re interested in how teams will compete in November, it doesn’t matter how well a side fared three or four years ago, especially if some of their best players are no longer in the mix. Second, they don’t reflect the change in format. Until last year, doubles represented one rubber in a best-of-five-match tie. A good doubles pair helped, but it wasn’t particularly necessary. Now, there are only two singles matches alongside the doubles rubber. The quality of a nation’s doubles team is more important than it used to be.

Let’s see what happens to the rankings when we generate a more forward-looking rating system. Using singles and doubles Elo, I’m going to make a few assumptions:

Each country’s top two singles players have a 75% chance of participating (due to the possibility of injury, fatigue, or indifference), and if either one doesn’t take part, the country’s third-best player will replace him.
Same idea for doubles, but the top two doubles players have an 85% chance of showing up, to be replaced by the third-best doubles player if necessary.
The three matches are equally important. (This isn’t technically true–the third match is likely to be necessary less than half the time, though when it does decide the tie, it is twice as important as the other two matches.)
Andy Murray won’t play.

Those assumptions allow us to combine the singles and doubles Elo ratings of the best players of each nation. The result is a weighted rating for each side, one that has a lot of bones to pick with the official Davis Cup rankings.

Forward-looking rankings

The following table shows the 18 countries at the Davis Cup finals along with the 12 losing qualifiers. For each team, I’ve listed their Davis Cup ranking, and their finals seed (if applicable). To demonstrate my results, I’ve shown each nation’s weighted Elo rank and rating and their hard-court Elo rank and rating. The table is sorted by hard-court Elo:

Country  DC Rank  Seed  Elo Rank   Elo  sElo Rank  sElo  
ESP            7     7         1  1936          1  1891  
CRO            2     2         2  1898          2  1849  
FRA            1     1         3  1880          3  1845  
USA            6     6         4  1876          4  1835  
RUS           21    17         7  1855          5  1827  
AUS            9     9         5  1857          6  1820  
SRB            8     8         8  1849          7  1808  
GER           11    11         6  1855          8  1799  
AUT           16              10  1800          9  1766  
ARG            3     3         9  1803         10  1755  
                                                         
Country  DC Rank  Seed  Elo Rank   Elo  sElo Rank  sElo  
GBR            5     5        11  1796         11  1750  
SUI           24              14  1763         12  1749  
ITA           10    10        12  1780         13  1745  
CAN           14    13        13  1777         14  1744  
JPN           17    14        15  1735         15  1719  
BEL            4     4        17  1688         16  1673  
CZE           13              16  1712         17  1661  
NED           19    16        18  1685         18  1643  
BRA           28              20  1659         19  1638  
IND           20              21  1652         20  1621  
                                                         
Country  DC Rank  Seed  Elo Rank   Elo  sElo Rank  sElo  
SVK           29              22  1645         21  1617  
CHI           22    18        19  1682         22  1609  
KAZ           12    12        26  1582         23  1574  
COL           18    15        24  1597         24  1551  
SWE           15              27  1570         25  1542  
BIH           27              28  1552         26  1540  
POR           26              23  1610         27  1535  
HUN           23              25  1583         28  1533  
UZB           25              29  1491         29  1489  
CHN           30              30  1468         30  1465

Spain is the comfortable favorite, regardless of whether we look at overall Elo or hard-court Elo. When the draw is conducted, we’ll see which top-six seed is unlucky enough to end up with the Spaniards in their group, and whether the hosts will remain the favorite.

The biggest mismatch between the Davis Cup rankings and my Elo-based approach is in our assessment of the Russian squad. Daniil Medvedev is up to sixth in my singles Elo ratings, with Karen Khachanov at 10th. Those ratings might be a little aggressive, but as it stands, Russia is the only player with two top-ten Elo singles players. Spain is close, with Rafael Nadal ranked 2nd and Roberto Bautista Agut 11th, and the hosts have the additional advantage of a deep reservoir of doubles talent from which to choose.

In the opposite direction, my rankings do not forecast good things for the Belgians. David Goffin has fallen out of the Elo top 20, and there are no superstar doubles players to pick up the slack. In a just world, Spain and Belgium will land in the same round-robin group–preferably one without the Russians as well.

Madrid or Maldives

The results I’ve shown assume that every top singles player has the same chance of participating. That’s certainly not the case, with high-profile stars like Alexander Zverev telling the press that they’ll be spending the week on holiday in the Maldives. Some teams are heavily dependent on one singles player who could make or break their chances with a decision or an injury.

As it stands, Germany is 8th in the surface-weighted Elo. If we take Zverev entirely out of the mix, they drop to a tie for 14th with Japan. It’s something the German side would prefer to avoid, but it’s not catastrophic, partly because the Germans were never among the favorites, and partly because Zverev could play only one singles rubber per tie and the doubles replacements are competent.

Even more reliant on a single player is the Serbian side, which qualified last weekend without the help of their most dangerous threat, Novak Djokovic. With Djokovic, the Serbs rank 7th–a case where my surface Elo ratings almost agree with the official rankings. But without the 15-time major winner, the Serbs fall down to a tie with Belgium in 16th place. While the Serbs are unlikely to take home the trophy regardless, Novak would make a huge difference.

The draw will take place next Thursday. We’ll check back then to see which sides have the best forecasts, nine months out from the showdown in Madrid.

Novak Djokovic and the Narrowing Slam Race

Italian translation at settesei.it

It doesn’t take a statistician, or even a spreadsheet, to recognize that the 2019 Australian Open wasn’t Novak Djokovic’s most difficult path to a major title. We can debate whether the straight-set win over Rafael Nadal in the final was due to Djokovic’s utter dominance or a subpar performance from (a possibly still recovering) Rafa. But there’s more to a grand slam title than the final, and the only top-18 opponent Novak faced in the first six rounds was Kei Nishikori, who retired after 52 minutes.

On the traditional grand slam leaderboard, quality of competition doesn’t matter. Roger Federer has 20, Nadal has 17, and now Djokovic has 15. As I’ve written before, the race is closer than that, since Nadal’s and Djokovic’s opponents have, on average, been stronger than Federer’s. My metric for “adjusted slams” estimates the likelihood that a typical major titlist would defeat the specific seven opponents that a player faced, based on their surface-weighted Elo at the time of the match. (I’ve also used this approach for Masters titles.) The explanation is a mouthful, but the underlying idea is simple: Some majors represent greater achievements than others, both because some eras offer stiffer competition and because some draws are particularly daunting.

A slam title against an average level of competition is worth exactly 1. Tougher paths are worth more than 1, and easier draws are worth less. Here is the current leaderboard, with each player’s raw tally, average difficulty rating of their titles, and adjusted total:

Player          Slams  Avg Diff  Adj Slams  
Roger Federer      20      0.88       17.7  
Rafael Nadal       17      1.01       17.1  
Novak Djokovic     15      1.11       16.6

(The numbers in this post do not all precisely agree with those I’ve published in the past, because I’ve improved the accuracy of my Elo-based rating system. All three of the players have seen their adjusted slam totals decrease, because the improved Elo algorithm eliminates some of the Elo “inflation” that overvalued recent achievements.)

These three guys have often had to go through each other, but Djokovic has had the toughest paths of all. The average difficulty of his first 12 majors was 1.2, higher than all but three of Rafa’s titles, one of Roger’s, and two of those won by Pete Sampras. Only recently has he been able to boost his total without quite so much of a challenge. His Australian Open title was worth 0.84 majors, only the fourth of his titles against a below-average set of opponents. It was, however, tougher than Wimbledon or the US Open last year, which were worth 0.77 and 0.65, respectively.

It’s unlikely, of course, that the current leaderboard–adjusted or otherwise–will be the final reckoning among these three men. But on the adjusted list, they will probably remain tightly packed. Because the rest of the pack has weakened, with Andy Murray and Stan Wawrinka no longer regular features of the second week, major titles aren’t what they used to be. Early in the decade, it wasn’t uncommon for a player to beat multiple members of the big four en route to a title and add at least 1.2 to his adjusted tally.

In 2018, slam difficulty was barely half of that recent peak level:

Year    Avg Diff  
2002        0.73  
2003        0.65  
2004        0.82  
2005        0.95  
2006        0.77  
2007        0.93  
2008        1.05  
2009        1.00  
2010        0.95  
2011        1.19  
2012        1.23  
2013        1.22  
2014        1.28  
2015        1.12  
2016        1.27  
2017        0.91  
2018        0.69

This could all change, especially if Djokovic wins a Roland Garros title by upsetting Nadal. (Nothing generates high competition-adjusted numbers like beating Nadal on clay.) But it’s more likely that these three men will have to keep incrementing their totals by 0.6s and 0.7s. While that could be enough to put Rafa or Novak on top by the end of the 2019, it won’t give anyone a commanding lead. It’s a good thing that there’s a lot more to the GOAT debate than slam totals, because slam totals–when properly adjusted for the difficulty of achieving them–make it awfully hard to pick a winner.

Identifying Underrated Players With Minor League Elo

I’ve just revised my published Elo ratings (men, women) to better reflect the performance of players who mostly compete at the (men’s) ATP Challenger and (women’s) ITF levels. Previously, my Elo ratings used only tour-level main-draw matches. For top players, it makes very little difference–not only do Novak Djokovic and Simona Halep play no matches at the lower levels, they rarely encounter opponents who spend much time there. But for the second tier of players, the effect can be substantial.

The Elo system rates players according to the quality of their opponents. Beat a good player with a high rating, and your own rating will jump by a healthy margin. Beat a weakling, and your rating will inch up a tiny bit. Essentially, Elo looks at each result and asks, “Based on this new result, how much do we need to adjust our earlier rating?” When Bianca Andreescu upset Caroline Wozniacki in Auckland last week, the system responded by upping Andreescu’s rating by quite a bit, and by penalizing Wozniacki more than for the typical loss. After a more predictable result, like Djokovic’s defeat of Damir Dzumhur, ratings barely move.

It’s important to understand the basic mechanics of the system, but the main takeaway for most fans is that Elo just works. The algorithm generates more accurate player ratings (and resulting match forecasts) than the official ATP and WTA rankings, among other attempts to rank players. Now, you can see Elo rankings for a much wider range of players.

Of of my main uses of Elo ratings is identifying players whose official rankings haven’t caught up to reality. For instance, a few months ago I noted when Daniil Medvedev moved into the Elo top ten, even though he has yet to crack that threshold on the official list. Most players who reach the top ten on the Elo table eventually do the same in the ATP or WTA rankings. Another two current examples are Aryna Sabalenka and Ashleigh Barty, considered by Elo to be two of the top three women on tour right now, even though neither is in the top ten of the WTA rankings. That may be too aggressive, and the margins at the top of the women’s list are tiny right now, but it is a clear signal that these women’s results bear watching. (We talked about this on the most recent Tennis Abstract podcast.)

Now that we have unified Elo lists that cover more players, let’s dig deeper. For each tour, let’s find the players current outside the official top 100s who are rated the highest by the more sophisticated formula. First, the ATP:

Player                  ATP Rank  Elo Rank  
David Ferrer                 124        36  
Thanasi Kokkinakis           145        62  
Miomir Kecmanovic            126        66  
Jack Sock                    105        77  
Reilly Opelka                102        84  
Ricardas Berankis            107        86  
Marcos Baghdatis             122        87  
Gilles Muller                137        88  
Daniel Evans                 190        89  
Viktor Troicki               201        90  
Horacio Zeballos             182        92  
Jared Donaldson              115        94  
Mikael Ymer                  196        95  
Egor Gerasimov               157       100  
Lloyd Harris                 119       102  
Tommy Paul                   195       104  
Guillermo Garcia Lopez       101       106  
Felix Auger Aliassime        106       108  
Alexei Popyrin               149       109  
Dudi Sela                    240       114

One thing that pops out from the list is the number of veterans. Elo ratings are “stickier” than ATP rankings, since the official system works with only 52 weeks worth of results. Elo ratings make constant adjustments, but quality performances–even when they are more than 52 weeks old–continue to affect current ratings for some time. David Ferrer has had a hard time staying healthy enough to compete at his former level, but according to Elo, he remains fairly dangerous when he is able to take the court.

Fortunately the list isn’t all veterans. Elo suggests that younger players such as Thanasi Kokkinakis, Miomir Kecmanovic, Mikael Ymer, and Tommy Paul are better than their current rankings indicate.

The WTA list is even more laden with veterans, players who are still competing at a high level, if not as frequently as they used to:

Player                WTA Rank  Elo Rank  
Lucie Safarova             105        39  
Coco Vandeweghe            100        40  
Shuai Peng                 129        43  
Svetlana Kuznetsova        106        50  
Sara Errani                114        52  
Varvara Lepchenko          134        80  
Laura Siegemund            110        84  
Kristyna Pliskova          101        96  
Anna Kalinskaya            167        97  
Viktorija Golubic          104        98  
Ivana Jorovic              117        99  
Marie Bouzkova             120       103  
Kateryna Bondarenko        140       104  
Sachia Vickery             123       105  
Veronika Kudermetova       111       107  
Sabine Lisicki             198       109  
Vitalia Diatchenko         131       112  
Yanina Wickmayer           126       113  
Nao Hibino                 115       114  
Danielle Lao               169       115

Part of the reason why so few prospects appear on this list is because of my decision to exclude ITF $25Ks. For example, up-and-coming 18-year-old Kaja Juvan, who knocked out Yanina Wickmayer in Australian Open qualifying today, hasn’t played nearly enough matches at higher levels to appear on my Elo list. But last year, she was 29-7 at ITF $25Ks, and won her last ten matches at that level.

Another issue is that the most promising women tend to climb into the top 100 more quickly. Another 18-year-old, Dayana Yastremska, rocketed up the rankings with a tour-level title in Hong Kong last fall. She sits at No. 59 on the WTA table, but after 13 top-100 wins in 2018, Elo is even more optimistic, placing her at No. 27, just ahead of Maria Sharapova and Venus Williams.

I’ll continue to update these expanded Elo ratings weekly and use them to generate forecasts for every tour-level and Challenger event. Enjoy!

Daniil Medvedev’s Leading Elo Indicator

Italian translation at settesei.it

It is shaping up to be a breakthrough season for 22-year-old Russian Daniil Medvedev. His Tokyo title two weeks ago was his first at the ATP 500 level and his third on the season, after earlier triumphs in Sydney and Winston-Salem. The run in Japan was a particularly notable step, since he knocked out three top-20 players along the way. He had only four top-20 victories in the entire season leading up to Tokyo, and two of those were against the slumping Jack Sock.

His ATP ranking is rising alongside his results. The Winston-Salem title moved him into the top 40, and the Tokyo trophy resulted in a leap to No. 22. After a first-round win in Shanghai last week, Medvedev crept to his current career-high of No. 21. With a couple of wins in Moscow this week, he could overtake Milos Raonic and reach the top 20.

The improvement on the ATP ranking table is nothing next to the Russian’s race to the top of the Elo list. Last Monday, with the Japanese title in the books, Medevdev rose to No. 8 on my men’s Elo ranking. Since then, he has dropped two places but remains in the top ten, ahead of Marin Cilic, Kevin Anderson, and a host of others who outrank him on the official ATP list.

Given the discrepancy, what do we believe? Is Medvedev inside the top 10 or outside the top 20? Is Elo a leading indicator–that is to say, an early-warning signal for future ATP ranking milestones–or a misleading one? Elo is designed to be forward-looking, tuned to forecast upcoming match outcomes and weighting wins and losses based on the quality of the opponent. The official rankings explicitly consider a year’s worth of results, with no adjustments for quality of competition. In theory, Elo should be the better of the two measures for predicting longer-term results, but that assumes the algorithm works well, and that it doesn’t overreact to short-term successes. Let’s take a look at past differences between the two systems and see what the future might hold for the 22-year-old.

Precedents

Since 1988, 102 men have debuted in the ATP top ten. A slightly larger number, 113, have shown up in the top ten of my Elo ratings. There’s a very substantial overlap between the two, with 94 names appearing in both categories. Thus, 8 players have reached the ATP top ten without clearing the Elo threshold, while 19 have rated a spot in the Elo top ten without convincing the ATP computer to agree.

Here are the eight ATP top-tenners whose Elos have never merited the same status:

Player               ATP Top Ten Debut  ATP Top Ten Weeks  
Jonas Svensson                19910325                  5  
Nicolas Massu                 20040913                  2  
Radek Stepanek                20060710                 12  
Jurgen Melzer                 20110131                 14  
Juan Monaco                   20120723                  8  
Kevin Anderson                20151012                 31  
Pablo Carreno Busta           20170911                 17  
Lucas Pouille                 20180319                  1

A few of these players could still make progress on the Elo list, especially Kevin Anderson, who is currently 11th, a miniscule five points behind Medvedev.

Here is the longer list of Elo top-ten players without any weeks in the official top ten:

Player                 Elo Top Ten Debut  Elo Top Ten Weeks  
Carl Uwe Steeb                1989/05/22                  3  
Andrei Cherkasov              1990/12/11                  1  
Goran Prpic                   1991/05/20                  1  
David Wheaton                 1991/07/08                  9  
Jerome Golmard                1999/05/03                  2  
Dominik Hrbaty                2001/01/15                  2  
Jan Michael Gambill           2001/04/06                  6  
Nicolas Escude                2002/02/25                  4  
Younes El Aynaoui             2002/05/20                  2  
Paul Henri Mathieu            2002/10/14                  8  

Player                 Elo Top Ten Debut  Elo Top Ten Weeks
Agustin Calleri               2003/05/19                  2  
Taylor Dent                   2003/10/06                 10  
Andrei Pavel                  2004/05/10                  2  
Robby Ginepri                 2005/10/24                  1  
Ivo Karlovic                  2007/11/12                  3  
Roberto Bautista Agut         2016/02/22                  1  
Nick Kyrgios                  2016/03/04                 62  
Stefanos Tsitsipas            2018/08/13                  3  
Daniil Medvedev               2018/10/08                  2

* I define ‘weeks’ a little differently for Elo ratings, as ratings are generated only for those weeks with an ATP-level tournament or Davis Cup tie.

Most of these guys came very close to cracking the ATP top ten. For example, David Wheaton’s peak ranking was No. 12. With the exception of Nick Kyrgios, no one spent more than ten weeks in the Elo top ten without eventually reaching the same standard according to the ATP formula. This list shows that it’s possible to have a brief peak that cracks the Elo top ten but doesn’t last long enough to reflect the kind of success that the official ranking system was designed to reward. About one in six players with a top-ten Elo rating never reached the ATP top ten, though as we can see, the odds of remaining an Elo-only star fall quickly with each additional week in the top ten.

Kyrgios is a perfect example of the differences between the two approaches to player ranking. The Australian has recorded a number of high-profile upsets, which are the fastest way to climb the Elo list. But knocking out the second-ranked player in the world, as Kyrgios did to Novak Djokovic at Indian Wells last year, doesn’t have much impact on the ATP ranking when it happens in the fourth round. Usually, a player who can oust the elites will start piling up wins in a form that the official computer will appreciate. But Kyrgios, unlike just about every player in history with his talent, hasn’t done that.

In short, Elo will always elevate a few players to top-ten status even if they’ll never deserve the same treatment from the ATP formula. It’s too early to say whether Medvedev fits that mold. But where Elo really excels is identifying top players before the ATP computer does. Of the 94 cases since 1988 in which a man debuted in both top tens, Elo was first to anoint the player a top-tenner in 76 of them–better than 80%. The official rankings were first 10 times, and the two systems tied in the other eight instances. On average, players reached the Elo top ten about 32 weeks before the ATP top ten.

Here are the 11 most extreme gaps in which Elo got there first, along with the top-ten debuts of the Big Four:

Player               ATP Debut   Elo Debut  Week Diff  
Mariano Puerta      2005/07/25  2000/06/12        267  
Marc Rosset         1995/07/10  1990/11/05        244  
Fernando Gonzalez   2006/04/24  2002/10/07        185  
Guillermo Canas     2005/05/09  2002/08/05        144  
Mikhail Youzhny     2007/08/13  2004/11/15        143  
Gaston Gaudio       2004/06/07  2002/04/29        110  
Richard Gasquet     2007/07/09  2005/06/20        107  
Tomas Berdych       2006/10/23  2004/10/11        106  
Robin Soderling     2009/10/19  2007/10/08        106  
Mark Philippoussis  1999/03/29  1997/03/24        105  
Jack Sock           2017/11/06  2016/01/18         94  
                                                       
Player               ATP Debut   Elo Debut  Week Diff  
Roger Federer       2002/05/20  2001/02/19         65  
Andy Murray         2007/04/16  2006/08/21         34  
Novak Djokovic      2007/03/19  2006/07/31         33  
Rafael Nadal        2005/04/25  2005/02/21          9

And in case you’re curious, the ten cases in which the ATP computer beat Elo to the punch:

Player              ATP Debut   Elo Debut  Week Diff  
Stan Wawrinka      2008/05/12  2010/10/25        128  
David Ferrer       2006/01/30  2007/05/28         69  
Janko Tipsarevic   2011/11/14  2012/05/13         26  
Rainer Schuettler  2003/06/09  2003/08/25         11  
Tommy Robredo      2006/05/08  2006/07/24         11  
Fernando Verdasco  2009/02/02  2009/04/06          9  
Albert Costa       1997/04/21  1997/05/26          5  
Nicolas Almagro    2011/04/25  2011/05/22          4  
John Isner         2012/03/19  2012/04/15          4  
Jiri Novak         2002/10/14  2002/10/21          1

The 32-week average difference is suggestive. As I’ve noted, Elo ratings are optimized to forecast the near future, so at least in theory, they reflect each player’s level right now. The ATP algorithm tallies each man’s performance over 52 weeks, with equal weight given to the first and last weeks in that timeframe. Setting aside improvement and decline due to age, that means the ATP computer is telling us how each player was performing, on average, 26 weeks ago. If Medvedev continues to oust top-20 players on a regular basis and claims another 500-level title or two, he could well be 26 or 32 weeks away from a top-ten debut.

Elo isn’t designed to make long-term forecasts–the tools needed to do so, for the most part, have yet to be invented. And the system occasionally gives high ratings to players who don’t sustain them for very long. But in general, a superlative Elo rating is a sign that a similar mark on the ATP ranking list isn’t far behind. So far, Kyrgios has managed to defy the odds, but the smart money still points to an eventual ATP top-ten debut for Medvedev.

The Rosy Forecast of Arnya Sabalenka’s Elo Rating

Italian translation at settesei.it

It’s been almost two weeks since Aryna Sabalenka’s last title, and the next one is starting to feel overdue. With respect to Naomi Osaka’s ascent, the Belarussian is the hottest rising star on the women’s tour right now, with two titles in the last two months, plus two more finals earlier in the season. The 20-year-old is 8-4 against the top ten this year, with wins over Caroline Wozniacki, Petra Kvitova, Elina Svitolina, and Karolina Pliskova.

It takes time for all of these wins to show up in the WTA rankings. Sabalenka nudged into the top 20 after winning New Haven in August, and rose as high as 11th last Monday, though she is set to fall back to 14th after failing to defend her title in Tianjin this week. While the official ranking is a lagging indicator, Elo ratings react more quickly, especially to high-profile upsets like the ones Sabalenka has been recording almost every week.

Sabalenka’s Elo rating has rocketed to the top of the list. Through last week’s matches, she sits at second overall, behind Simona Halep, but closer to Halep than to third-place Wozniacki. After knocking out Caroline Garcia in Beijing last week, she briefly took over the Elo top spot before handing it back after her quarter-final loss to Qiang Wang. Still, an overall ranking of #2 is a lot more suggestive of future stardom than the WTA computer’s report of #11.

When Elo looks at hard court matches alone, it is even more optimistic, putting Sabalenka at the very top of the list. Elo would narrowly favor the Belarussian in a hard-court match against Halep and, assuming the draw treated both players equally, would make Sabalenka the early favorite for the 2019 Australian Open title.

What should we make of this? Is it time to appoint Sabalenka the next superstar, or ought we treat Elo ratings with more circumspection? Let’s take a look at players who have topped the Elo list in the past to get a better idea.

Since 1984, only 29 women (including Sabalenka) have reached the #1 or #2 spot on the overall Elo list. 19 of them got to #1 in the official rankings. Here are the other ten:

Player               Peak  
Petra Kvitova           2  
Conchita Martinez       2  
Jana Novotna            2  
Agnieszka Radwanska     2  
Elina Svitolina         3  
Gabriela Sabatini       3  
Elena Dementieva        3  
Samantha Stosur         4  
Johanna Konta           4  
Aryna Sabalenka        11

This is pretty good company. Svitolina could still reach #1, and several of the others were expected to attain even greater heights than they did. The only warning sign here is Johanna Konta, who isn’t the best comp for a young star, as she didn’t crack the top two until close to her 26th birthday.

The group of women who have ranked #1 on the hard-court specific Elo ranking table is even more select. Sabalenka is only the 17th player since 1984 to head the list, and 14 of the 17 have topped the official rankings as well. The only other exceptions are Svitolina and Konta.

If there’s ever a good time to anoint a 14th-ranked player the future of the sport, I’d say this is it. Elo isn’t perfect, and it’s possible that the algorithm has overreacted to a series of upsets in a season packed full of them. But if the system has made a mistake, it’s one that it doesn’t make very often. Sabalenka has only won four main-draw matches at majors, so maybe that 2019 Australian Open title is too much to ask. But in the long term, one grand slam title might be a mere harbinger of even greater things to come.

The All-Time ATP Masters Race Is Even Closer Than You Think

Italian translation at settesei.it

As I write this, Novak Djokovic has earned himself a place in the semi-finals at the Shanghai Masters, putting him two wins away from his 32nd career ATP Masters title. A championship this week would leave him one behind Rafael Nadal, the all-time leader with 33 Masters victories. Roger Federer, also in action today in Shanghai, is not far back, with a career total of 27.

Masters tallies aren’t as important as grand slam counts, but they make up an important part of an elite player’s resume. For one thing, there are more of them, and the mix of surfaces–more clay, no grass, and an indoor event–adds to our knowledge of a player’s range of skills. It’s no surprise that Nadal, Djokovic, and Federer are miles ahead of the pack on this list, as on so many others.

But all Masters aren’t created equal. At last year’s Madrid Masters, Nadal had to defeat Djokovic, as well as clay-court giant Dominic Thiem and the ever-threatening Nick Kyrgios. Six months later in Paris, Jack Sock earned the same number of Masters titles by coasting down a path that included only one player ranked in the top 35. Like major titles, Masters championships are heavily influenced by luck, and when we focus on raw totals, we trust that fortune tends to even out.

It doesn’t even out, even for the top players who have played Masters events for more than a decade and racked up dozens of titles. To account for opponent quality and the difficulty of each title, I applied the same algorithm I’ve used in the past to rate slam titles. [1] The formula spits out a number for each Masters title, where 1 is average, less than 1 is easier than the norm, and greater than 1 is more difficult. Sock’s Paris trophy was the luckiest in recent years, clocking in at 0.39, while David Nalbandian’s 2007 Madrid title was the most difficulty, rating 1.92. These extremes nothwithstanding, almost every championship comes in between 0.5 and 1.5.

The all-time tally

Let’s start by looking at the top ten in “adjusted Masters.” The table below shows the results of my formula, alongside each player’s actual Masters count, and the average rating of the tournaments he has won:

Player          Adj Masters  Masters  Average  
Rafael Nadal           35.4       33     1.07  
Novak Djokovic         35.0       31     1.13  
Roger Federer          28.0       27     1.04  
Andre Agassi           15.0       17     0.88  
Andy Murray            15.0       14     1.07  
Pete Sampras           11.2       11     1.02  
Thomas Muster           7.5        8     0.94  
Michael Chang           6.4        7     0.91  
Boris Becker            5.4        5     1.08  
Jim Courier             5.0        5     1.00

Boris Becker and Jim Courier aren’t the only men to have recorded five Masters titles, but they are only ones who have done so against average or better competition. Andy Roddick won five, but the algorithm gives him credit for just under four, and it is even more harsh on Marat Safin, whose five victories translate into only 3.2 adjusted Masters.

The real story is at the top of the list, where adjusting for competition almost eliminates the gap between Nadal and Djokovic. Both men have won their titles against more difficult than average competition (often, by beating each other), but Djokovic has faced the tougher paths. If he wins on Sunday in Shanghai, he’ll overtake Nadal’s adjusted tally.

Also of note is the near-tie between Andre Agassi and Andy Murray. Agassi holds three more trophies, but won them against the weakest competition of anyone in the top ten. Murray has dealt with much of the same field that Nadal and Djokovic have, so it’s no surprise to see his difficulty rating well above 1.0.

The Paris swoon

Sock’s title last year was unquestionably weak, but not entirely out of character for Bercy. With the exception of the short-lived Essen Masters, Paris titles have come the cheapest of any other tour stop at this level:

Tournament            Years  Average Rating  
Madrid (clay)            10            1.18  
Rome Masters             29            1.09  
Indian Wells Masters     29            1.07  
Stuttgart Masters         6            1.05  
Stockholm Masters         5            1.04  
Hamburg Masters          19            1.02  
Miami Masters            29            1.01  
Monte Carlo Masters      29            0.98  
Canada Masters           29            0.97  
Cincinnati Masters       29            0.97  
Madrid (hard)             7            0.97  
Shanghai Masters          9            0.95  
Paris Masters            28            0.84  
Essen Masters             1            0.80

Paris was played on carpet until 2006, and that may be a factor. When I first ran the numbers, I used carpet-specific Elo ratings, which are limited by a relatively small sample. I tried again using hard-court ratings for carpet events, and while individual numbers shifted up and down, the overall results were about the same. Bercy was particularly weak during the carpet era, and it has gotten stronger, but I’m confident this is a feature of the results in the 1990s and early 2000s, not merely an artifact of Elo rating quirks.

Still, broadly speaking, fast courts seem to result in lower ratings. I suspect that’s because at predominantly best-of-three events, early-round upsets are more likely to occur on the quickest surfaces. Fast courts, then, effectively gut the field for the eventual champion. It certainly worked for Sock last year. But it’s no guarantee–the five most difficult Masters titles all came on hard courts, and one of them took place in Paris.

Peak Nalbandian

At the end of 2007, Nalbandian enjoyed two of the most glorious weeks of tennis ever played. In Madrid, he defeated Nadal in the quarters, Djokovic in the semis, and Federer in the final, not to mention Tomas Berdych and Juan Martin del Potro in the early rounds. Two weeks later, he beat Federer and Nadal again in Paris, along with wins over David Ferrer, Richard Gasquet, and Carlos Moya. Those two titles rate 1.92 and 1.70, respectively, and are two of the three most difficult since the Masters series began.

(Oddly enough, the only man who could stop Nalbandian that fall was Stan Wawrinka, who beat him both in Vienna and Basel. Wawrinka’s slam titles rate as the most difficult in that category.)

Here are the 20 most difficult Masters titles, along with their ratings:

Year  Event         Surface  Champion            Rating  
2007  Madrid        Hard     David Nalbandian      1.92  
2014  Canada        Hard     Jo Wilfried Tsonga    1.78  
2007  Paris         Hard     David Nalbandian      1.70  
2007  Canada        Hard     Novak Djokovic        1.68  
2009  Indian Wells  Hard     Rafael Nadal          1.61  
2009  Madrid        Clay     Roger Federer         1.52  
2017  Madrid        Clay     Rafael Nadal          1.52  
2016  Madrid        Clay     Novak Djokovic        1.51  
2011  Indian Wells  Hard     Novak Djokovic        1.50  
2013  Indian Wells  Hard     Rafael Nadal          1.50  
                                                         
Year  Event         Surface  Champion            Rating  
2010  Canada        Hard     Andy Murray           1.48  
2011  Rome          Clay     Novak Djokovic        1.48  
2012  Rome          Clay     Rafael Nadal          1.47  
2010  Indian Wells  Hard     Ivan Ljubicic         1.45  
2004  Hamburg       Clay     Roger Federer         1.44  
2015  Cincinnati    Hard     Roger Federer         1.44  
2013  Rome          Clay     Rafael Nadal          1.43  
2015  Canada        Hard     Andy Murray           1.43  
2008  Monte Carlo   Clay     Rafael Nadal          1.42  
2015  Madrid        Clay     Andy Murray           1.42

Nalbandian and Jo Wilfried Tsonga stand out at the top, but after that, it’s a whole lot of Big Four. Even in the next ten toughest paths, Djokovic, Nadal, and Federer account for seven.

Like just about any adjustment to high-profile counting stats, tweaking Masters titles for difficulty doesn’t exactly clear up the debate over the greatest players of all time. This is only one small part of that conversation. However, seeing the wide range of challenges faced by Masters champions is a necessary reminder that not all titles are the same, even if they all count for one thousand ATP ranking points.

—

[1] Here’s how I first described the algorithm:

To evaluate the overall difficulty of grand-slam titlists’ draws, I used Elo—a rating system that assesses a player’s strength based on his won-lost record and the quality of his opponents—to measure the skill of an average major champion. I then estimated the probability that such a player would have won all seven matches against the opponents that each tournament’s victor had to face. For each win, I credit the champion with the difference between one and the Elo forecast: If an average slam champion on the tournament’s surface had a 90% chance of winning the match, the player gets 0.1 points (1 – 0.9); if a typical major winner would have gone in with a 20% shot, he’s assigned 0.8. Summing all the matches for each winner and applying the algorithm to the last several decades of grand slams results in an average credit of 1.23 per titlist, so I then divided each sum by 1.23 to normalise the results.

For Masters events, it’s five or six matches instead of seven, and the final step involves dividing by 1.34 instead of 1.23.

Forecasting the 2018 Laver Cup

Embed from Getty Images

Italian translation at settesei.it

It’s that time of year again: group selfies in suits, dodgy Davis Cup excuses, and a reminder that it takes more than six continents just to equal Europe. That’s right, it’s Laver Cup.

Last year, I worked out a forecast of the event, walking through a variety of ways in which captains Bjorn Borg and John McEnroe could use their rosters and ultimately predicting a 16-8 win for Team Europe. As it happened, both captains intelligently deployed their stars, and the result was 15-9. This year, the competitors are a little different and the home court has moved from Prague to Chicago, but the format remains the same.

Let’s start with a look at the rosters. I’ve included two additional players for reference: Juan Martin del Potro, scheduled to play for Team World, but withdrew; and Pierre Hugues Herbert, the doubles specialist Borg hasn’t realized he needs. Each player is shown alongside his surface-weighted singles Elo rating and surface-weighted doubles “D-Lo” rating:

EUROPE                       Singles Elo  Doubles D-Lo  
Novak Djokovic                      2137          1667  
Roger Federer*                      2097          1700  
Alexander Zverev                    1971          1690  
David Goffin                        1960          1582  
Grigor Dimitrov                     1928          1719  
Kyle Edmund                         1780          1542  
                                                        
WORLD                        Singles Elo  Doubles D-Lo  
Kevin Anderson                      1914          1692  
Nick Kyrgios                        1910          1668  
John Isner                          1887          1800  
Diego Sebastian Schwartzman         1814          1540  
Frances Tiafoe                      1772          1544  
Jack Sock                           1724          1925  
                                                        
ALSO                                                    
Juan Martin Del Potro               2062          1678  
Pierre Hugues Herbert               1691          1890

* Federer has played very little tour-level doubles for a long time. Last year I estimated his D-Lo at 1650; he played rather well last year, so I’ll bump him up to 1700 this time around.

Especially with Delpo on the sidelines, Europe looks to dominate the singles. The doubles leans in World’s favor, largely because Jack Sock is so good, especially in comparison with guys who have focused on singles.

Format review

Let’s do a quick refresher on the format. Laver Cup takes place over three days, each of which has three singles matches and one doubles match. Each player must play singles at least once, and no doubles pairing can repeat itself. Day 1 matches are worth one point each, day 2 matches are worth 2 points each, and day 3 matches are worth 3 points each. If there’s a 12-12 tie at the end of day 3, a single doubles set–in which a previously-used team may compete–will decide it all.

Given that format, the best way for the captains to use their rosters is to stick their three worst singles players on day 1 duty, then use their best three on both day 2 and day 3. For doubles, they should use their best doubles player every day, with the best partner on day 3, next best on day 2, and third best on day 1. As I’ve mentioned, Borg and McEnroe came close to this last year, although Borg didn’t use Rafael Nadal (his best doubles player) in day 3 doubles, and he generally overused Tomas Berdych. Both decisions are understandable, as Nadal may not have been physically able to play every possible match, and Berdych was in front of a Czech crowd.

Now that we know the captains will act in a reasonably savvy way, we can forecast the second edition with a little more confidence than the inaugural one.

The forecast

Nadal’s absence this year will hurt the Europe squad on both singles and doubles. Combined with a small step backward for Federer’s singles game, this year’s Laver Cup figures to be closer than last year. Recall that my forecast a year ago called for a 16-8 Europe victory, and the result was 15-9.

Assuming optimal usage, the 2018 forecast gives Europe a 67.6% chance of winning, with a most likely final score of 14-10. There’s a nearly one-in-ten shot that we’ll see a 12-12 tie, in which the superior doubles capabilities of Team World give them the edge, with a 70.7% probability of winning the tie-breaking set.

Were del Potro not so fragile, this could get even more interesting. Swap out Frances Tiafoe for the Tower of Tandil, and Europe’s chances fall to 56.8%, with a most likely final score of 13-11.

Nothing McEnroe could have done, short of going to medical school a few decades ago, could have put the Argentine on his team this week. But Borg has less of an excuse for failing to maximize the potential of his team. Unlike World, with its world-beating doubles specialist, Europe has a stunning singles roster that rarely takes to the doubles court. As we’ve seen, one doubles player can take the court three times, plus the potential 12-12 tie-breaking set. The specialist would need to play singles only once, on the low-leverage first day.

The obvious choice is Pierre Hugues Herbert, a top-five doubles player with the ability to play respectable singles as well. The Frenchman would be considerably more valuable than Kyle Edmund, who is a better singles player, but not good enough to be of much help to an already loaded side. (I made a similar point last year and illustrated it with Herbert’s partner, Nicolas Mahut. Since then, Herbert has taken the lead over his Mahut in both singles and doubles Elo ratings.)

When we sub in Herbert for Edmund, the simulation spits out the best result yet for Europe. Against the actual World team (that is, no Delpo), the hypothetical Europe squad would have a 74.6% chance of winning, with the likely final score between 14-10 and 15-9. Herbert and a mediocre partner would still be the underdogs in a tie-breaking final set against Sock and John Isner, but the presence of a legitimate doubles threat would narrow the odds to about 58/42.

We won’t get to see either Delpo or Herbert in Chicago this year, but we can expect a slightly more competitive Laver Cup than last year. Add in home court advantage, and the result is even less of a foregone conclusion. It’s no match for last week’s Davis Cup World Group play-offs, but I suspect it’ll make for more compelling viewing this weekend than the final rounds in Metz and St. Petersburg.

Jack Sock, Doubles King Once Again

Embed from Getty Images

Italian translation at settesei.it

A couple of years ago, I wrote an article introducing D-Lo, an Elo-like rating system for doubles, which crowned Jack Sock as the best doubles player on the men’s tour. Sock grabbed the top spot in October 2016 and hung on for about nine months, largely by not playing very much. A couple of first-round losses in Washington and Montreal last summer sent him tumbling, landing at 8th after the US Open and as low as 14th going into this year’s Australian Open.

Despite his preference for singles, Sock has rocketed back into the lead, first pairing with John Isner for the Indian Wells title, and then partnering Mike Bryan (replacing injured brother Bob) to win both Wimbledon and the US Open. With the exception of one week immediately after Indian Wells, Sock sits at the head of the D-Lo table for the first time in more than a year. Here are the current top ten, along with their ratings:

Rank  Player                 D-Lo  
1     Jack Sock              1949  
2     Bob Bryan              1930  
3     Mike Bryan             1917  
4     Pierre Hugues Herbert  1906  
5     Nicolas Mahut          1893  
6     Jamie Murray           1886  
7     Bruno Soares           1883  
8     Oliver Marach          1867  
9     Robert Farah           1863  
10    Nikola Mektic          1863

Yes, that’s the injured Bob Bryan in the second spot. More on that in a moment.

A quick refresher on the D-Lo system: It mostly works like a standard Elo algorithm, in that players gain points for winning matches and lose points for losing them, based on the quality of the opponent and the amount of prior information already baked into their ratings. A big upset earns more points than a victory over an equal, and for players with fewer prior matches, the effect of each match is greater. Thus, Sock got a few more points than Mike did for winning the 12 matches at the last two slams, because we knew relatively less about him before those tournaments.

D-Lo assumes that the quality of each team is equal to the average of the two players. If a team wins, each member of the partnership gains points, with one tweak: If the two players have different ratings, their ratings slightly move toward the average of the two. This is because it’s impossible to know how much each player contributes to a win. The system is designed so that, after a year or so of playing together, the two mens’ ratings will meet in the middle. It’s an imperfect system, but it does a reasonably good job of forecasting results, which means it usually provides a solid representation of each player’s skill level.

Back to the matter at hand: Doubles ratings have been particularly volatile this year, with five different men (Sock, Bob, Pierre Hugues Herbert, Mate Pavic, and Henri Kontinen) holding the #1 spot, and two more (Nicolas Mahut and John Peers) peaking at #2. This parity means that no player has a particularly high rating. Two years ago, Sock’s mark of 1949 would have been good for only fourth (behind himself, Herbert, and Mahut), and several players (the Bryans, Herbert, and Daniel Nestor among them) have peaked with ratings above 2000.

Take a look at how much the rank order has fluctuated since the beginning of 2018:

For clarity’s sake, I’ve left off Oliver Marach (whose rating tracks closely with that of his partner, Pavic, and whose season hasn’t lived up to its early promise) and Peers (ditto, with Kontinen). Herbert has reached the highest level of anyone this season, but a rough second half so far has left him behind the American trio of Jack, Bob, and Mike.

Back to the curious case of Bob Bryan. The Bryan brothers’ title at the Madrid Masters this year gave the twins their highest D-Lo ratings in nearly two years. “Standard” Elo doesn’t penalize players for absence, so Bob’s mark has remained at 1930 ever since. (I’ve added an injury/absence penalty in my singles Elo ratings, but haven’t done so for D-Lo. I suspect there is less of an effect, but still a measureable one, in doubles.) Mike’s rating has slipped because of some bad results apart from the pair of majors, and only Sock has caught up.

If Bob is healthy enough to play this fall, the twins are expected to pair up for the World Tour Finals, once again leaving the best doubles player in the world out of the field. In that case, Sock, down to 157th in the ATP singles race, could end up spending that week playing the new ATP Challenger event in Houston. Without their young compatriot in the way, the Bryans will be back in familiar territory, headed to London as the favorites for another year-end title.

Handling Injuries and Absences With Tennis Elo

Italian translation at settesei.it

For the last year or so, every mention of my ATP and WTA Elo ratings has required some sort of caveat. Ratings don’t change while players are absent from the tour, so Serena Williams, Novak Djokovic, Andy Murray, Maria Sharapova, and Victoria Azarenka were all stuck at the top of their tour’s Elo rankings. When their layoffs started, they were among the best, and even a smattering of poor results (or a near season’s worth, in the case of Sharapova) isn’t enough to knock them too far down the list.

This is contrary to common sense, and it’s very different from how the official ATP and WTA rankings treat these players. Common sense says that returning players probably aren’t as good as they were before a long break. The official rankings are harsher, removing players entirely after a full year away from the tour. Serena probably isn’t the best player on tour right now (as Elo insisted during her time off), but she’s also much more of a threat than her WTA ranking of No. 454 implies. We must be able to do better.

Before we fix the Elo algorithm, let’s take a moment to consider what “better” means. Fans tend to get worked up about rankings and seedings, as if a number confers value on the player. The official rankings are, by design, backward-looking: They measure players based on their performance over the last 52 weeks, weighted by how the tour prioritizes events. (They are used in a forward-looking way, for tournament seedings, but the system is not designed to be predictive of future results.) In this way, the official rankings say, “this is how good she has played for the last year.” Whatever her ability or potential, Serena (along with Vika, Murray, and Djokovic) hasn’t posted many positive results this year, and her ranking reflects that.

Elo, on the other hand, is designed to be predictive. Out of necessity, it can only use past results, but it uses those results in a way to best estimate how well a player is competing right now–our best proxy for how someone will play tomorrow, or next week. Elo ratings–even the naive ones that said Serena and Novak are your current No. 1s–are considerably better at predicting match outcomes than are the official rankings. For my purposes, that’s the definition of “better”–ratings that offer more accurate forecasts and, by extension, the best approximation of each player’s level right now.

The time-off penalty

When players leave the tour for very long, they return–at least on average, and at least temporarily–at a lower level. I identified every layoff of eight weeks or longer in ATP history, taken by a player with an Elo rating of 1900 or above*. In their first matches back on tour, their pre-break Elo overestimated their chances of winning by about 25%. It varies a bit by the amount of time off: eight- to ten-week breaks resulted in an overestimation around 17%, while 30- to 52-week breaks meant Elo overestimated a player’s chances by nearly 50% upon return. There are exceptions to every rule, like Roger Federer at the 2017 Australian Open, and Rafael Nadal, who won 14 matches in a row after his two-month break this season, but in general, players are worse when they come back.

* I used the cutoff of 1900 because, below that level, some players are alternating between the ATP and Challenger tours. My Elo algorithm doesn’t include challenger results, so for lower-rated players, it’s not clear which timespans are breaks, and which are series of challenger events. Also, the eight-week threshold doesn’t count the offseason, so an eight-week layoff might really mean ~16 weeks between events, with the break including the offseason.

Translated into Elo terms, an eight-week break results in a drop of 100 Elo points, and a not-quite-one-year break, like Andy Murray’s current injury layoff, means a drop of 150 points. Making that adjustment results in an immediate improvement in Elo’s predictiveness for the first match after a layoff, and a small improvement in predictiveness for the first 20 matches after a break.

Incorporating uncertainty

Elo is designed to always provide a “best estimate”–when a player is new on tour, we give him a provisional rating of 1500, and then adjust the rating after each match, depending on the result, the quality of the opponent, and how many matches our player has contested. That provisional 1500 is a completely ignorant guess, so the first adjustment is a big one. Over time, the size of a player’s Elo adjustments goes down, because we learn more about him. If a player loses his first-ever match to Joao Sousa, the only information we have is that he’s probably not as good as Sousa, so we subtract a lot of points. If Alexander Zverev loses to Sousa after more than 150 career matches, including dozens of wins over superior players, we’ll still dock Zverev a few points, but not as many, because we know so much more about him.

But after a layoff, we are a bit less certain that what we knew about a player is still relevant. Djokovic a great example right now. If he lost six out of nine matches (as he did between the Australian Open fourth round and Madrid) without missing any time beforehand, we’d know it was a slump, but most of us would expect him to snap out of it. Elo would reduce his rating, but he’d remain near the top. Since he missed the second half of last season, however, we’re more skeptical–perhaps he’ll never return to his former level. Other cases are even more clear-cut, as when a player returns from injury without being fully healed.

Thus, after a layoff, it makes sense to alter how much we adjust a player’s Elo ratings. This isn’t a new idea–it’s the core concept behind Glicko, another chess rating system that expands on Elo. Over the years, I’ve tinkered with Glicko quite a bit, looking for improvements that apply to tennis, without much success. Changing the multiplier that determines rating adjustments (known as the k factor) doesn’t improve the predictiveness of tennis Elo on its own, but combined with the post-layoff penalties I described above, it helps a bit.

The nitty-gritty: After a layoff, I increase the multiplier by a factor of 1.5, and then gradually reduce it back to 1x over the next 20 matches. The flexible multiplier slightly improves the accuracy of Elo ratings for those 20 matches, though the difference is minor compared to the effect of the initial penalty.

No more caveats*

* I thought it would be funny to put an asterisk after “no more caveats.”

Post-layoff penalties and flexible multipliers end up bringing down the current Elo ratings of the players who are in the middle of long breaks or have recently come back from them, giving us ranking tables that come closer to what we expect–and should do a better job of predicting the outcome of upcoming matches. These changes to the algorithm also have minor effects on the ratings of other players, because everyone’s rating depends on the rating of all of his or her opponents. So Taro Daniel’s Elo bounce from defeating Djokovic in Indian Wells doesn’t look quite as good as it did before I implemented the penalty.

On the ATP side, the new algorithm knocks Djokovic down to 3rd in overall Elo, Murray to 6th, Jo-Wilfried Tsonga to 21st, and Stan Wawrinka to 24th. That’s still quite high for Novak considering what we’ve seen this year, but remember that the Elo algorithm only knows about his on-court performances: A six-month break followed by a half-dozen disappointing losses. The overall effect is about a 200-point drop from his pre-layoff level; the “problem” is that his Elo a year ago reflected how jaw-droppingly good he had recently been.

The WTA results match my intuition even better than I hoped. Serena falls to 7th, Sharapova to 18th, and Azarenka to 23rd. Because of the flexible multiplier, a few early wins for Williams will send her quickly back up the rankings. Like Djokovic, she rates so high in part because of her stratospheric Elo rating before her time off. For her part, Sharapova still rates higher by Elo than she does in the official rankings. Despite the penalty for her one-year drug suspension, the algorithm still treats her prior success as relevant, even if that relevance fades a bit more every week.

Elo is always an approximation, and given the wide range of causes that will sideline a player, not to mention the spectrum of strategies for returning to the tour, any rating/forecasting system is going to have a harder time with players in that situation. That said, these improvements give us Elo ratings that do a better job of representing the current level of players who have missed time, and they will allow us to make superior predictions about matches and tournaments involving those players.

Under the hood

If you’re interested in some technical details, keep reading.

Before making these adjustments, the Brier score for Elo-based predictions of all ATP matches since 1972 was about 0.20. For all matches that involved at least one player with an Elo of 1900 or better, it was 0.17. (Not only are 1900+ players better, their ratings tend to be based on more data, which at least partly explains why the predictions are better. The lower the Brier score, the better.)

For the population of about 500 “first matches” after layoffs for qualifying players, the Brier score before these changes was 0.192. After implementing the penalty, it improved to 0.173.

For the 2nd through 20th post-comeback matches, the Brier score for the original algorithm was 0.195. After adding the penalty, it was 0.191, and after making the multiplier flexible, it fell a bit more to 0.190. (Additional increases to the post-layoff multiplier had negative results, pushing the Brier score back to about 0.195 when the 2nd-match multiplier was 2x.) I realize that’s a tiny change, and it very possibly won’t hold up in the future. But in looking at various notable players over the course of their comebacks, that’s the option that generated results that looked the most intuitively accurate. Since my intuition matched the best Brier score (however miniscule the difference), it seems like the best option.

Finally, a note on players with multiple layoffs. If someone misses six months, plays a few matches, then misses another two months, it doesn’t seem right to apply the penalty twice. There aren’t a lot of instances to use for testing, but the limited sample confirms this. My solution: If the second layoff is within two years of the previous comeback, combine the length of the two layoffs (here: eight months), find the penalty for a break of that length, and then apply the difference between that penalty and the previous one. Usually, that results in second-layoff penalties of between 10 and 50 points.

Rafael Nadal and the Greatest Single-Tournament Performances

Italian translation at settesei.it

In the last two weeks, Rafael Nadal recorded his 11th titles in both Monte Carlo and Barcelona. His career records at those two events, along with his ten Roland Garros championships, reflect a level of dominance never before seen on a single surface. They have to be considered among the greatest achievements in tennis history, and perhaps in all of sport.

The tennis fan in me is content to speculate about whether anyone will ever stop him. The analyst wants to dig deeper: Has Nadal’s performance at one of the tournaments been even better than the rest? How do these single-event records compare to other exploits, such as Roger Federer’s trophy haul at Wimbledon, or Bjorn Borg’s nearly-undefeated career at the French Open?

Barcelona by the numbers

Let’s start with Barcelona. Since 2005–we’ll ignore his 2003 appearance as a 16-year-old wild card–he has played the event 13 times, winning 11 of them. That’s a won-loss record of 57-2.

Usually, I would calculate the probability of a player winning so many tournaments in that many chances, then come up with a tiny percentage that would represent his odds of achieving such a feat. That would miss the mark here. Instead, I want to look at the problem from the opposite perspective: In order to win so many titles, how good must Nadal be?

We already know that Rafa is the best of all time on clay, in general. Using the Elo rating system, his peak surface-specific rating–that is, Elo calculated using only results on clay courts–is over 2,500, better than anyone else on clay … or anyone else on any surface. (Nadal’s current clay-specific Elo is around 2,400, and the closest things he has to rivals on the surface right now, Dominic Thiem and Kei Nishikori, sit at about 2,190 and 2,150. Stefanos Tsitsipas’s rating is 1865.) Since Rafa has posted his best results at these three events, it stands to reason that his tournament-specific levels are even higher.

Here, then, is the method we can use to figure that out. First, for each year he entered Barcelona, determine his path to the title. (For the 11 titles, that’s easy; for the other two, we use the players he would have faced had he kept winning.) Using each opponent’s clay court Elo rating at the time of the match, we can determine the odds that various hypothetical (and dominant) players would have progressed through the draw and won the title.

Here is Nadal’s path to the 2018 title, showing each player’s pre-match clay court Elo*, along with the odds that Rafa (given his own current rating) would beat him:

Round  Opponent                 Opp Elo  p(Rafa W)  
R32    Roberto Carballes Baena     1767      97.3%  
R16    Guillermo Garcia Lopez      1769      97.2%  
QF     Martin Klizan               1894      94.5%  
SF     David Goffin                2079      84.5%  
F      Stefanos Tsitsipas          1900      94.3%

* from this point on, the clay court Elos I use are 50/50 blends of clay-specific Elo–that is, a rating calculating only with clay court results–and overall Elo. The blended rating is the one that has proven best at predicting match outcomes. Nadal is the all-time leader in this category as well, with a 50/50 clay Elo that peaked around 2,510.

Given those five single-match probabilities, the odds that Nadal would win the tournament were just over 70%. That’s dominant, but it’s not 11-out-of-13 dominant.

What if Rafa were underrated by Elo, at least in Barcelona? Here is the probability that a player at various Elo ratings would have beaten the five opponents that he faced last week:

Clay Elo  p(2018 Title)  
2200              41.2%  
2250              50.4%  
2300              59.1%  
2350              66.9%  
2400              73.6%  
2450              79.3%  
2500              83.9%  
2550              87.6%  
2600              90.5%

It turns out that this year’s title path was one of the weakest since 2005. It is roughly equivalent to the players Nadal needed to defeat in 2006 (with Nicolas Almagro in the semis and Tommy Robredo in the final), and a bit tougher than last year’s route, which didn’t feature a top-50 player until Thiem in the final. The toughest was his hypothetical path in 2015, when he lost to Fabio Fognini in the second round. Had he progressed, he would have faced David Ferrer in the semis and Nishikori in the final.

Once we figure out the quality of Rafa’s opponents (and would-have-been opponents, for the two years he lost early), we can work out the odds that any player–given those paths–would have won the tournament each year.

If we assume that Rafa’s average level since 2005 is the same as his current level–a clay Elo of around 2,400–the odds that he would have won 11 Barcelona titles in 13 tries is 13.0%. We don’t have the luxury of replaying those 13 tournaments in a few thousand alternate universes, so it’s not entirely clear what to make of that number–was Rafa lucky? would he do it again, given the chance? is he actually way better than an Elo level of 2,400 in Barcelona?

I don’t know the answer to those questions; all we know is what happened. To compare (un)decimas (and related accomplishments by other players), we’re going to look at the Elo level that would have resulted in the achievement at least 50% of the time. In other words, how good would Nadal have to have been to give himself a 50/50 chance at winning 11 Barcelona titles in 13 tries?

At various clay Elo levels, here are the odds that Rafa would have completed the Barcelona undécima:

Clay Elo  p(11 of 13)  
2300             1.0%  
2350             4.6%  
2400            13.0%  
2450            28.0%  
2500            47.2%  
2550            64.2%  
2600            77.7%  
2650            87.3%  
2700            93.1%

Thus, a player with a clay Elo of about 2,505 would have had a 50% chance of matching Nadal’s feat at his home tournament. To put it another way: At this event, over a span of 14 years, he has played at a level roughly equal to his career peak which, incidentally, is the all-time best clay Elo rating ever achieved by an ATP player.

Comparing las (un)decimas

I hope that my method makes sense and seems like a reasonable way of quantifying a rare feat. Algorithm in hand, we can compare Nadal’s Barcelona record with his efforts in Monte Carlo and Paris.

Monte Carlo

Rafa has entered 14 times since 2005 (again, excluding his 2003 appearance) and won 11. That’s a bit less impressive than 11-of-13, but the competition level is much higher. Only last year’s tournament, in which the opposing finalist was Albert Ramos, is in the same league as most of the Barcelona draws.

Sure enough, the Monte Carlo undécima is lot more impressive. To have a 50% chance of winning 11 titles in 14 attempts, a player would need a clay Elo of about 2,595, almost 100 points higher than the comparable number for Barcelona, and well above the level any player has ever achieved, even at their peak.

Roland Garros

At the French Open, Nadal has entered 13 times, winning 10. The field is even more challenging than in Monte Carlo, but on the other hand, the five-set format gives a greater edge to favorites, lessening the chance of an underdog scoring an upset with two magical sets.

The Roland Garros 10-of-13 is not quite as eye-popping as the record at Monte Carlo. The clay Elo required to give a player a 50% chance of matching Nadal’s French Open feat is “only” around 2,570–still better than any player has ever attained, but a bit short of the comparable mark for Monte Carlo.

But wait … what about 2016? Rafa won two rounds and then withdrew from his third-rounder against Marcel Granollers. I don’t know whether that should count, but at least for argument’s sake, we should run the numbers without it, treating Nadal’s French Open record as 10 titles in 12 appearances, not 13. In that case, the clay Elo that would give a player a 50/50 shot at matching the record is 2,595–the same as the Monte Carlo number.

At the moment, Monte Carlo appears to be the tournament where Nadal has played his very best. With another French Open a few weeks away, though, that answer may be temporary.

Rafa vs other record holders

A few other players have racked up impressive totals at single events. Wikipedia has a convenient list, and a few accomplishments stand out: Federer’s tallies at Wimbledon, Basel, and Halle, Guillermo Vilas’s eight titles in Buenos Aires, and Borg’s six French Open titles in only eight appearances.

Let’s have a look at how they compare, ranked by the surface-specific Elo rating that would give a player a 50% chance of equaling the feat:

Player   Tourney          Wins  Apps  50% Elo  
Nadal    Monte Carlo        11    14     2595  
Nadal    French Open*       10    12     2595  
Nadal    French Open        10    13     2570  
Borg     French Open**       6     7     2550  
Nadal    Barcelona          11    13     2505  
Borg     French Open         6     8     2475  
Vilas    Buenos Aires***     8    10     2285  
Federer  Wimbledon           7    18     2285  
Federer  Halle               8    15     2205  
Federer  Basel               8    15     2180

* excluding 2016

** excluding 1973, when Borg was 16 years old, and lost in the fourth round

*** excluding 1969-71, both because Vilas was very young, and due to sketchy data

The only single-event achievement that ranks with Nadal’s is Borg’s record at Roland Garros–and even then, only when we don’t consider Borg’s loss there as a 16-year-old. Federer’s records in Wimbledon, Halle, and Basel are impressive, but fail to rate as highly because he has entered those tournaments so many times. Federer didn’t appear on tour ready to win everything on his chosen surface, the way Rafa did, and those early losses are part of the reason that his records at these tournaments are so low.

We never needed any numbers to know that Nadal’s accomplishments at his three favorite tournaments are among the best of all time. With these results, though, we can see just how dominant he has been, and how few achievements in tennis history can even compare. The scary thing: A month from now, I may need to come back and update this post with even more eye-popping numbers. The greatest show on clay courts isn’t over yet.