Category: Research

Italian translation at settesei.it

This week’s pair of ATP warmups for the Australian Open provide quite the contrast.

In Sydney, only one seeded player (the hardly automatic Andreas Seppi) reached the semifinals, and only one other even made the quarters. Across the ditch in Auckland, three of the final four are among the top four seeds, and the fourth, Gael Monfils, would typically sport a ranking in the same range.

Sydney fits a conventional narrative, while Auckland confounds it. The week before a Grand Slam, many of the top players are out of action, while those who are in action … well, let’s just say warmups don’t always appear to be their top priority.

Winning in 250s

The ATP schedule gives us a convenient natural experiment in order to determine whether slam warmups really are different.

(For convenience, I’m using the term “warmups.” However, we’re only looking at tournaments the week before a slam starts. Sydney is included, but not Brisbane, even though events two weeks before Austrlian and Wimbledon are generally called “warmups.”)

Since 2009, all of the lowest rung of tour-level events have been worth 250 points to the winner. Conveniently, all tourneys the week before slams have fallen into this category.

To see if players seem to treat slam warmups differently from other events, we can simply compare results from warmups to those from other 250s. It isn’t perfect, since a few 250s have draws of more than 32 players and the field quality isn’t identical in all tourneys at this level, but by looking at a few different metrics, we can limit the impact of those quibbles.

Who cares?

Let’s start by simply counting wins and losses of seeded players. In slam warmups from 2009 through 2012, seeds won about 61% of matches against unseeded opponents (224 of 365), while in other 250s, seeds win over 70% of those matches (1499 of 2129). That’s a substantial difference.

To eliminate the quirks of the bigger 250 draw at Queen’s Club, and perhaps toss out some first-round retirements as well, let’s consider the records that seeds have posted in specific rounds.

In the round of 16 at slam warmups, seeds have gone 71-50, for a winning percentage of 58.7%. At other 250s, seeds have won 591 against 223 losses, a percentage of 72.6%.

In the quarterfinals of slam warmups, seeds have beaten unseeded players in 33 of 46 matches–71.7% of encounters. In other 250s, similar matchups have gone to the seeded player 200 of 275 times, or 72.7% of the time.

It seems that many top-ranked players show up at slam warmups with the intent of getting one or two matches under their belt. (Or perhaps fulfilling an obligation to a sponsor.) Those players don’t perform up to their usual standard. But as shown by the comparable records in quarterfinals, those who come to compete play at their usual level.

A few other looks

One issue that seems to have a particular impact in slam warmups is last-minute withdrawals, like that of second-seed Gilles Simon in Sydney this week. Those don’t show up in the won-loss records.

To consider the overall picture, including withdrawals, we can count the number of seeds who reach the semifinals in our different categories of ATP 250s.

In slam warmups, the semifinal fields in the last four years have consisted of 53 seeds and 43 nonseeds–about 55% top-ranked players. In other 250 semifinals, we’ve seen 365 seeds against 191 nonseeds–66% seeds.

Yet another angle is the performance of the top four seeds. In 250s, the 5 through 8 seeds are often barely distinguishable from the rest of the pack. For example, in Sydney this week, those last four seeds are Florian Mayer, Radek Stepanek, Jeremy Chardy, and Marcel Granollers. Not much difference between those guys and unseeded semifinalists Julien Benneteau, Kevin Anderson, and Bernard Tomic.

There’s no clear line between first-rank guys and the rest of the pack, but taking the top half of the seeds seems as good as any other option.

The results are similar to what we saw with the larger pool of seeds. Overall, when a top-four seed played a non-top-four opponent in a slam warmup, he won 65% of matches (129 of 199). In other 250s, he won 74% (978 of 1321).

In the round of 16, top-fours went 51-24 in slam warmups, for a record of 68%, compared to 76% (366-114) in other 250s.

Where the top four seeds differ from other seeds is in the quarterfinal round. In slam warmup QFs, top-fours went 31-20, winning 61% of matches. In other tourneys, they won 71% (261-105). Perhaps the first-round bye in many slam warmups means that top seeds want two warmup matches, but no more.

As mentioned, these experiments give us imprecise results, as they don’t take into account the exact field quality of the various 250s. While they may not be the final word on this question, these numbers do strongly indicate that higher-ranked players don’t view slam warmups as particularly important. Against a similar pool of opponents, they win far more matches in 250s at other times throughout the year.

Perhaps that’s one reason why winning an Aussie Open warmup doesn’t forecast any particular level of success in Melbourne–these are tournaments where some of your most highly-ranked opponents just aren’t trying as hard as usual.

In his remarkable comeback this year, Brian Baker has already recorded two top-20 scalps, along with seven other victories against players in the top 100.   In the same span of six months, he’s also lost to a player barely inside the top 400, and suffered another six defeats against guys outside the top 100.

This is inconsistency of historic magnitude.  The list of players he’s beaten may actually be more impressive than the list of those who have beaten him!  Adding to the confusion, we don’t have any other recent results from him.  We can’t just wave our hands and point to his 2011 performance level as an accurate indicator of his current level.

One measurement of player ability, the ATP ranking system, places him at #78, a number that seems just as ridiculous when he’s beating Philipp Kohlschreiber at a Masters event as when he’s losing to Maxime Authom at a challenger.  But overall, the ATP estimate doesn’t seem too far-fetched.  It’s certainly better than what jrank (my rating system) spits out.  That algorithm doesn’t know what to do with such a limited track record, so it places him far outside the top 100.

We can do better.  As we’ll see, Baker’s results suggest he belongs on the cusp of the top 50.

Uniquely limited results

Imagine a completely unknown player is given a wild card into a major event.  We don’t know where he came from or who he might have beaten in the past.  He’s a completely blank slate.  If we wanted to estimate his ability level, we would have to wait until we got some results.

If that player won an opening-round match against the 17th-best player in the world, our best guess would be that he is better than #17, but we wouldn’t know how much.  If he lost that opening round match, we would assume he is worse than #17.  We might use statistics from that match to estimate how much better or worse than #17.

As our unknown kept playing more matches, we would update our estimate, using additional data as it came in.

(You might protest that in the early going, we should regress our estimate to the mean, since if some random guy came out of nowhere, he probably isn’t one of the 16 best tennis players in the world–there was a reason he was nowhere.  And, in such a real-world scenario, you would be right.  But such a case, what is the mean?  If a baseball player is called up from Triple-A, an intelligent observer, such as a scout or team executive, considers him at least marginally MLB-level, so we would regress our estimate to the level of marginal MLB players.  But if a player receives a wild card into a tennis tournament, what do we know?)

Few tennis players in history have come closer to this unknown than Brian Baker.  Sure, everyone has to start somewhere, but usually “somewhere” is a long string of futures tournaments, followed by an even longer string of challengers.  By the time a player bags his first top-20 scalp, we have lots and lots of data to work with.

When other players were racking up several dozen matches every year, Brian Baker was rehabbing injuries and coaching college tennis.  We can only judge him based on a small number of recent results.  And those results are particularly contradictory.

Working backward

Intuitively, it’s tough to accept that a single player has beaten a bunch of good players and lost to several weaker ones.  No matter how good that guy is, such a set of outcomes is unlikely.

But how unlikely?  That question is the key to estimating Baker’s current level.

Rather than assuming Baker is playing at a certain level (like that of #78) and scratching our heads at his inconsistency, we can work backwards–take his results and determine the likelihood that he is playing at various levels.

For instance, we could assume that Baker is #5 in the world.  If so, some of his results would be very predictable (like the two wins against Blake Strode) and others would be particularly jarring.  We could go further and calculate the probability that the #5 player in the world would amass Baker’s specific match record.  Those odds, of course, are vanishingly small.

If you repeat the process for every possible ranking, you get a probability that #5, or #12, or #77 would win the matches Baker has won and lose the matches he has lost.  One of those probabilities will be higher than the others, and that’s our best guess of how highly we should regard the American.

(If you’re interested in methodology, click “Continue Reading” below.)

Using this method, we discover that Baker has played at the level of someone with about 820 ATP ranking points, putting him around #54, in a tight pack with Grigor Dimitrov, Gilles Muller, Alejandro Falla, and Lukas Lacko.  With every match he plays, we can continue to fine-tune our estimate.

There are many factors we need to ignore to do an analysis like this, largely because of the limited data that led us to the topic in the first place.  Many of Baker’s worst results have come on hard courts; perhaps he will prove over a longer period to be stronger on clay and grass.  If his ability level has changed over the last six months, as seems very likely, this approach fails to take it into consideration.

But because of the unique nature of Baker’s comeback, which makes it difficult to assume anything about his ability level–this approach allows us to a make a reasonably good guess.  And with such a strange mix of great wins and rough losses, a good guess is all we can hope for.

Continue reading How Good is Brian Baker?

For all the talk of 30-somethings at the top of the modern men’s game, tennis players decline quickly.  30 may be the new 20, but 35 is still the same old 35, and 35-year-old tennis players are usually found on the champions tour, the doubles court, or national television.

Yet Tommy Haas, aged 34 years and 5 months, is enjoying a resurgence, having reached three finals in the last two months–on three different surfaces.  He’s one of the hottest players on tour of any age.

34-year-olds don’t do things like that.  In the last ten years, players 34 and older have accounted for fewer than 1% of wins on the ATP tour.  From 2008 to 2011, all 34-year-olds–combined–won a total of 17 tour-level matches.  In the five months since his birthday, Haas has won 22.

To find a point of comparison, we need to go back five years, to the 2007 campaign of Fabrice Santoro, and slightly earlier, to Andre Agassi‘s 2004 season.  Agassi at 34 was better than Haas at 34, winning 37 tour-level matches and reaching two grand slam quarterfinals.  Agassi was the best “old” player since Jimmy Connors and the only man in the discussion since the 1970s.

Yet already, Haas is among the best 34-and-overs in ATP history.  His 22 wins since his 34th birthday are good for 28th on the all-time list, ahead of Fred Stolle and just behind Roy Emerson.  But that understates Haas’s accomplishment.  With the exceptions of Santoro, Agassi, and Connors (whose 178 wins-past-34 are good for 2nd on the all time list, behind Ken Rosewall), everyone on the list retired more than 20 years ago.

Comparisons to Haas’s contemporaries do a better job of illustrating how unusual he is.  The only two older men to have won a match on tour this year are Arnaud Clement and Ruben Ramirez Hidalgo, neither of whom are a factor anywhere but the challenger tour.  The other 34-year-old to win some matches this season is hyper-fit warrior Michael Russell, who took advantage of the weak draws in Atlanta and Los Angeles.

As long as he stays healthy, Haas is far from finished.  According to Jrank, he’s the 11th-best hard court player in the game right now. He may not have another grand slam final ahead of him, as Agassi did at the same age, but he has more wins in his future than most players a decade his junior.

Italian translation at settesei.it

Marathon sets are again the talk of tennis.  We won’t soon forget Roger Federer’s 19-17 third-set win over Juan Martin Del Potro … or Roger’s weak performance in the match that followed.

The unusual Olympic format–best-of-three, no final-set tiebreak–brought several issues to the fore.  Should best of three be enough for slams?  It certainly gave us plenty of dramatics last week.  And is it finally time to end the no-tiebreak madness?  For all of the occasional drama, do we really need to see even more service holds in John Isner matches?

Peter Bodo makes the case for a marathon-free world:

[M]y main reason for embracing the final-set tiebreaker is not the obvious one that would be cited by most time-sensitive television producers. The real problem with deuce sets is that when a match goes as long as Federer v. Delpo or even Jo-Wilfried Tsonga v. Milos Raonic (that one went 25-23, for Tsonga) the reward for the winner’s heroic feat is almost always a quick subsequent loss.

As Bodo goes on to illustrate, this seems anecdotally true.  But who cares about anecdotes?  This is a testable hypothesis.

As we’ll see, there is a noticeable hangover effect when a player has fought through a marathon fifth set.  But the alternative–a fifth-set tiebreak–produces nearly the same hangover.

There have been 146 marathon fifth sets–matches in which the final set reached 6-6–in Grand Slam tennis since the beginning of 2001.  The record of those 146 winners in their next round is dreadful: 43-103, or 29.5%.  It’s even worse than that, actually.  Four times, two marathon men went on to play each other, so four of those wins were inevitable.

However, that isn’t the end of the story.  To prove that fifth-set marathons significantly weaken their winners, we need to establish two things: (1) They had a decent shot at beating their next opponents anyway, and (2) if a fifth-set tiebreak were played, their chances would have been better.

Post-marathon underdogs

The first issue is a bit sneaky.  If a player has to go deep into the fifth set to win in the early rounds, he’s hardly a dominating presence in the draw.  Consider the extreme case of Yen Hsun Lu, who in 2010, beat Andy Roddick in a 9-7 fifth set, advancing to play Novak Djokovic in the Wimbledon quarters.  Sure, Lu was tired, but what were the odds of an upset even if Roddick lost in three?  Top players rarely need five hours to push through an early-round opponent.

To quantify this, we can turn to jrank-driven predictions.  Using these measures of each player’s ability level at the time of the match, we can estimate the actual chances of our 146 marathon men.

The marathon men would have been underdogs in their next match no matter what.  On average, each one had a 43.4% chance of winning, meaning that of the 146 matches, they should have won 63 of them.  Even adjusting for their underdog status, they seem to have suffered from their marathons–they won 43 of those matches, barely two-third the number that they “should” have won.

Almost-but-not-quite marathons

We’ve established that once a player enters the uncharted territory beyond 6-6, his chances of winning the next match are substantially weakened.  But surely the fatigue didn’t set in right at the moment the chair umpire called “6-6.”  Even if the fifth set is a bagel, simply playing five sets of professional-level tennis is exhausting, and might impact one’s performance a day or two later.

The most relevant set of matches for comparison are US Open five-setters that went to a final-set tiebreak.  Since 2001, we have 40 of those.  In their next matches, the winners of the almost-marathons went a dismal 11-29 (27.5%)–worse than the marathon men!

Compared to their expectations, though, they did a bit better.  Those forty men, on average, had a 38% chance of winning their next matches, meaning we would expect them to win about 15 of the 40.  Relative to the predictions we would have made at the time, this small sample of fifth-set-tiebreak winners outperformed the marathon men, but just barely.

For a bigger sample, we can turn to the slightly shorter–but still epic–matches that end 7-5 in the fifth.  Of the 95 such matches since 2001, the 7-5 winners went on win 49, or 51.5% of their next matches!  This despite the fact they were collective underdogs, expected to win only 48%, or 46 of those matches.

What now?

Since the 7-5 group performed so differently in their next matches, it’s tempting to speculate why they did so.  My best guess: If a player manages a break before the set goes 6-6, he’s relatively fresh, physically and mentally.  The sort of player who can break at 5-5 or 6-5 is one who can come back a day or two later and plow through another three or four hard-fought sets.

By contrast, matches that get to 6-6–whether they end in a tiebreak or not–are usually battles of attrition.  Think Isner-Mahut: The longer it lasted, the less likely either player could challenge the other’s serve.  That brand of tennis had set in before 6-6 in the fifth: If one of the players pulled out a 7-4 tiebreak, it wouldn’t say much about his fitness or mental stamina, simply that someone is bound to get lucky for a point or two.

Based on the limited data we have, there just isn’t much difference between the after-effects of fifth-set marathons and fifth-set tiebreaks.  In both cases, the marathon men weren’t going to be favored anyway, and their fatigue hurts them even more.  Changing format to fifth-set tiebreaks would have little effect on future outcomes–it would just make those matches a bit more dependent on a lucky bounce.

Italian translation at settesei.it

Last night, when Jo Wilfried Tsonga finally defeated Milos Raonic, it was on a match-ending break of serve.  Conventional wisdom suggests that’s often how it goes.  Whoever serves first in a long set seems to have the advantage.  There’s less pressure to hold serve at 7-7 (or 47-47) than there is at 7-8.

Tsonga won his contest with a match-ending break point; Isner finished off his 70-68 set on Mahut’s serve; and when Federer and Roddick went to 14-14 in the 2009 Wimbledon final, Roger held for 15-14 before breaking the American.  Is it a trend?

As it turns out, those three high-profile matches have misled us.  Based on the limited data available, the first server in fifth-set epics has little or no advantage.

(Third-set epics are so rare that we might as well ignore them–the Olympics is the only tournament where men play best-of-three with no tiebreak in the final set.)

We don’t know who served first for every marathon fifth set in tennis history, but we can figure it out for some.  The ATP has limited stats for most matches back to 1991, and those stats include numbers of service games.  When the number of service games is equal for both players, we’re stuck at square one.  When one player has more than the other, that guy must have served the first game of the match–and the last.  Since marathon sets must contain an even number of games, we know who served first in the final set.

The result is a pool of 138 matches in which the fifth set ended at 8-6 or higher and we know who served first.  Of those, the guy who served first–at 0-0, 1-1, 6-6, and so on–won the match 67 times (48.6%).  It’s a coin toss.

If we take pressure out of the equation, this makes perfect sense.  If two guys have gotten to 6-6 in the fifth set, they’re playing as equally as two tennis players can play.  It’s only when we consider the stress of serving to stay in the match that we start to suspect that one player–but not the other–won’t be able to hold up his end.

For a bigger dataset, we can look to similar situations.  Consider 5-setters that end 7-5 in the fifth.  Those don’t have the cachet of matches that go farther, but they are quite epic in their own right.  We know who served first in 86 such matches, and of those, the man who served first won only 38 (44.2%).  It’s not exactly proof that the first server has a disadvantage, but it does cast more doubt on the conventional wisdom.

If want more than 200 or so matches, we need to weaken our definition of “epic.”  Tiebreaks aren’t relevant here, since we’re looking for instances where one player was broken under pressure.  But we can use best-of-three contests that ended 7-5.

With so many more best-of-three matches on the schedule, our dataset is now much bigger.  We know who served first for 753 tour-level matches that ended 7-5 in the third.  Of these, the player who served first went 412-341, winning nearly 55% of matches.

If you want evidence that the conventional wisdom is correct, there you go.  If a match reaches 5-5 in the deciding set and ends with a break, there is, altogether, a 53% chance that the first server wins.

But with our more limited data, it’s impossible to draw the same conclusion about five-setters once they head into the barely-charted territory beyond 6-6.

Italian translation at settesei.it

Roughly two-thirds of ATP tour-level tournaments have byes in the draw.  31 events–including the two this week, in Kitzbuhel and Los Angeles–have 28-man fields, with first-round byes for the top four seeds.

The obvious beneficiaries are the top four seeds.  They get free passes into the second round, eliminating the chance they’ll be handed a first-round exit.  It’s also a guarantee of greater prize money and more ranking points.  First-round byes are such a feature of the ATP tour, at least in part, because they help smaller tournaments convince big-name players to sign up.

Of course, you can’t simply hand an advantage to the top four seeds without affecting others.  In this most common format, a 28-man field with eight seeds and four byes, there are three important groups: The top four seeds, the bottom four seeds, and the rest of the field.

The top four seeds: The main effect of byes on the top four seeds is that, as noted, they don’t have play first-round matches.  The extent of that effect depends on how much of a threat the first-rounder would’ve been.

To quantify these effects, I ran simulations for the 2012 Estoril tournament.  First, I simulated the draw as the tournament was played, with 28 players and top seeds of Juan Martin Del Potro, Richard Gasquet, Stanislas Wawrinka, and Albert Ramos.  Second, I added the next four players on the alternate list to the draw in place of the byes.  To eliminate any bias stemming from the specific arrangement of the draw, I re-generated the brackets for each simulation.

In the 32-man field, Delpo won his first round match about 90% of the time, Gasquet and Wawrinka about 80%, and Ramos just under 60%.  Accordingly, Delpo didn’t benefit too much from the bye, but Ramos gained enormously.

However, when measured by expected ranking points, none of these four men gained as much as skipping the first round would suggest.  For instance, if Delpo would win only 90% of his first-round matches, removing that impediment would be expected to raise his other outcomes by (1/0.9 – 1), or 11%.  In fact, in the 28-man simulation, he gained only 9.5% over his 32-man expectation.

The slight difference is due to the other top seeds.  If Delpo is more likely to reach, say, the semifinals, then the same effect applies to Gasquet and Wawrinka, the two men who would be most likely to knock him out of the draw.  So while the bye itself increases Delpo’s expected ranking points by 11%, the increased probability of facing the other top seeds reduces it a bit.

Still, the net effect on the top four seeds is overwhelmingly positive.  For Gasquet and Wawrinka, the bye itself increases their expectations by 27% each, for a net effect of 24%, while for Ramos, the bye is a 74% increase, resulting in a net effect of 70%.

The next four seeds: The men seeded five through eight are the losers.  They must play a first-round match–which, in the Estoril example, they each have about a 60% chance of winning–but they are more likely to face one of the top four seeds later on.

The average effect of adding byes to the draw is a 5% decrease in expected ranking points for these lower four seeds.  They aren’t guaranteed to reach the quarterfinal, but in the 28-man version, if they do reach the quarters, they are at least 10% more likely to face a higher-ranked opponent.

The rest of the pack: Nearly everyone else benefits.  The effect of byes touches unseeded players in two ways, which work in opposite directions.  First, and most significantly, no one has to play a top-four seed in the first round.  In Estoril, the toughest first round opponent was 5th-seed Denis Istomin, not exactly a fearsome name in the locker room.  Because of the byes, nearly every player has a 40% chance of reaching the second round.

The countervailing force is a minor one–not enough to neutralize the advantage of missing top seeds in the first round.  When the field shrinks from 32 to 28, the average opponent is a bit better.  If four additional players were added to the Estoril field, they wouldn’t be automatically placed in the positions of the byes.  They would be randomly placed in the draw like everyone else.  Having those four lower-ranked players would give some players even easier first-round matches.

But on balance, for unseeded players, the goal is simply to win a match or two.  The best way to increase their chances of doing so is to keep the best players out of their path for as long as possible.  Byes take care of that.  The net benefit to unseeded players is an addition of 1% to 3% of their expected ranking points.  Generally speaking, the worse the player, the bigger the benefit.

The one exception to this rule is if an unseeded player is actually better than some of the seeds.  According to jrank, Igor Andreev was a better player than 8th-seed Flavio Cipolla going into Estoril.  Thus, the logic that applies to the bottom seeds applies to him.  He was likely to advance to the quarterfinals, so the effect of the byes was mainly to give him a tougher quarterfinal opponent.  In each tournament, this might affect one or two players–in Estoril, Andreev was the only one.

One more consideration: As we’ve seen, 23 of the 28 players benefited from the byes.  And the five players who were negatively affected didn’t lose too much.  How is that possible?

There’s one more group we haven’t talked about: The four players who aren’t included in a 32-man draw.  They don’t have much of a chance of reaching the final rounds, but they wouldn’t be much worse than the rest of the unseeded pack.

One of the players I used for this example, Igor Sijsling, just missed the cut, but in a 32-man draw, he would have been expected to take home 23 ranking points and about $9,000.  By adding four byes, the tournament is essentially taking what it would have given to Sijsling and three other players and divvying it up among the remaining 28.  The pie is the same size, but fewer players can claim a slice.

In the end, those four “missing” players are the only real losers, and they always have the option to head to a challenger for the chance of picking just as many points, even if they probably don’t come with as many dollars.

The winners, beyond the top seeds and the tournament organizers, are ultimately the fans.  When top players have more reason to play small tournaments, we get to watch more high-profile matchups, and ATP 250s look a bit less like Kitzbuhel and a bit more like Doha.