Category: Match charting

The Match Charting Project Reaches 5,000 Matches

Italian translation at settesei.it

Now this is a milestone. Last night, The Match Charting Project–my volunteer-driven effort to collect shot-by-shot logs of professional tennis–posted it’s 5,000th match! The magic-numbered chart was of one of last weekend’s Davis Cup Qualifiers matches, between Robin Haase and Lukas Rosol, charted by Zindaras, who just began contributing to the project. Number 5,001 is already up–a log of Sunday’s Hua Hin final between Dayana Yastremska and Ajla Tomljanovic.

MCP charts reveal data that simply isn’t available anywhere else. We track every shot–its type and direction–as well as the direction of every serve and and the depth of every return. All told, we’ve amassed these records for over 770,000 points, and almost 3 million shots. (At time of writing, we’re just over 2,992,000.) The dataset has made possible all kinds of research projects, like my recent Economist post about anti-Novak Djokovic tactics, an attempt to quantify the value of smashes, an evaluation of Kei Nishikori’s unusual return stance, and a look at the evolution of Juan Martin del Potro’s backhand.

When I launched the project in 2013, I never imagined we would amass so much information. My goal then was depth, not breadth. Now we have both. The 100 or so charters who have contributed to the project have combined to log nearly every grand slam final back to 1980, most ATP Masters finals back to 1990, and an increasing number of grand slam semi-finals and WTA Premier title matches. More recently, we’ve covered every tour-level final in 2018 and 2019, every head-to-head meeting between members of the big four, and nearly every final contested by any of the big four.

5,000 is a lot

The breadth of the available data goes beyond those high-profile matches. We have at least one charted match for nearly 1,100 different players, at least 10 matches for 268 players, 20 or more for 117 players, 50-plus for 33 players, and over 100 matches for 11 different players. It’s increasingly possible to use MCP data to track the evolution of individual players, something I assumed would always fall outside the scope of the project. And unlike many sports analytics initiatives, this one is gender balanced. Women’s matches make up 47% of the total, despite the fact that vintage women’s matches are considerably harder to track down. (To say nothing of more recent difficulties with WTA streaming.)

It’s fitting that the 5,000th match was logged by a new contributor, because the first several weeks of 2019 have been one of the best periods in the project’s history both for the number of charters and the volume of matches logged. We’ve already charted more than 150 matches from the 2019 season alone, including 79 from the Australian Open. Spearheading that effort has been another new charter, tsitsi, who has contributed more than 100 matches since joining up about a month ago.

Thanks are in order for everyone who has contributed to the project. About 100 people have charted matches, and some of them have been truly prolific. Edo has logged 661 matches, including many of the grand slam finals and semi-finals. In addition to Edo and tsitsi, eight more charters have been responsible for at least 50 matches apiece: Isaac, Lowell, ChapelHeel66, Edged, Palaver, Salvo, 1HandBH, and DebLDecker.

The next 5,000

I hope you’ll join us. Here’s my “quick start” guide to charting, along with 11 reasons to give it a go. Tennis is a complicated sport, so there’s a bit of a learning curve, but I think it’s worth the investment.

Even if you’re still on the fence about charting yourself, I encourage all fans to take greater advantage of the data on offer. A single chart, like this one of the Australian Open men’s final, contains thousands of data points describing various aspects of the match. What I find most illuminating is to compare those single-match numbers with tour, surface, and player averages. For most of the stats on each page, you can move your cursor over the number and see all of those averages. You can also find the player-specific averages on pages like this one, for Petra Kvitova. Researchers can dig into a significant chunk of the raw data, here.

My goal with Tennis Abstract, the blog, and the Match Charting Project has always been to get smarter about tennis–to better understand what’s really happening on court, and never to take the conventional wisdom at face value. I’d say we’re making progress.

Petra Kvitova’s Current Status: Low Risk, High Reward

Italian translation at settesei.it

For more a decade, Petra Kvitova has been one of the most aggressive women in tennis. She aims for the corners, hits hard, and lets the chips fall where they may. Sometimes the results are ugly, like a 6-4 6-0 loss to Monica Niculescu in the 2016 Luxembourg final, but when it works, the rewards–two Wimbledon titles, for starters–more than make up for it.

She’s currently riding another wave of winners. Her 11-match win streak–which has involved the loss of only a single set–puts her one more victory away from a third major championship. The 28-year-old Czech has gotten this far by persisting with her big-hitting style, but with a twist: In Melbourne, she’s not missing very often. While she’s ending as many points as ever on her own racket, she’s missing less often than many of her more conservative peers.

In her last five matches at the Australian Open, from the second round through the semi-finals, 7.9% of her shots (including serves) have resulted in unforced errors. In the 88 Petra matches logged by the Match Charting Project, that’s the stingiest five-match stretch of her career. In charted matches since 2010, the average WTA player hits unforced errors on 8.0% of their shots. So Kvitova, the third-most aggressive player on tour, is somehow making errors at a below-average rate. It’s high-risk, high-reward tennis … without the risk.

And it isn’t because her go-for-broke tactics have changed. In Thursday’s semi-final against Danielle Collins, her aggression score–an aggregate measure of point-ending shots including winners, induced forced errors, and unforced errors–was 30.5%, the third-highest of all of her charted matches since her 2017 return to the tour. Her overall aggression score in Melbourne, 28.2%, is also higher than her career average of 27.1%.

In other words, she’s making fewer errors, and the missing errors are turning into point-ending shots in her favor. The following graph shows five-match rolling averages of winners (and induced forced errors) per shot and unforced errors per shot for all charted matches in Kvitova’s career:

Even with the winner and error rates smoothed out by five-match rolling averages, these are still some noisy trend lines. Still, some stories are quite clear. This month, Kvitova is hitting winners at close to her best-ever rate. Her average since the second round in Melbourne has been 20.3%, as high as anything she’s posted before with the exception of her 2014 Wimbledon title. (I’ve never tried to adjust winner totals for surface; it’s possible that the difference can be explained entirely by the grass.)

And most strikingly, this is as big a gap between winner rate and error rate as she’s achieved since her 2014 Wimbledon title run. In fact, between the second round and semi-finals at that tournament, she averaged 8.1% errors and 20.0% winners. Both of her numbers in Australia this year have been a tiny bit better.

Best of all, the error rate has–for the most part–seen a steady downward trend since 2016. The recent error spike is largely due to her three losses in Singapore last October and a bumpy start to this season in Brisbane. We can’t write those off entirely–perhaps Kvitova will always suffer through weeks when her aim goes awry–but she appears to have put them solidly behind her.

None of this is a guarantee that Petra will continue to avoid errors in Saturday’s final against Naomi Osaka. I could’ve written something about her encouraging error rates before the tour finals in Singapore last fall, and she failed to win a round-robin match there. And Osaka is likely to offer a stiffer challenge than any of Kvitova’s previous six opponents in Melbourne, even if her second serve doesn’t. That said, a stingy Kvitova is a terrifying prospect, one with the potential to end the brief WTA depth era and dominate women’s tennis.

Dayana Yastremska Hits Harder Than You

Italian translation at settesei.it

At the 2019 Australian Open, tennis balls have more to fear than ever before. Serena Williams is back and appears to be in top form, Maria Sharapova is playing well enough to oust defending champion Caroline Wozniacki, and Petra Kvitova has followed up her Sydney title with a stress-free jaunt through the first three rounds.

And then there are the youngsters. Hyper-aggressive 20-year-old Aryna Sabalenka crashed out in the third round against an even younger threat, Amanda Anisimova. But still in the draw, facing Serena on Saturday, is the hardest hitter of all, 18-year-old Ukrainian Dayana Yastremska. Watch a couple of Sabalenka matches, and you might wonder if we’ve reached the apex of aggression on the tennis court. Nope: Yastremska turns it up to 11.

When Lowell first introduced his aggression score metric a few years ago, Kvitova was the clear leader of the pack, the player who ended points–for good or ill–most frequently with the ball on her racket. Madison Keys wasn’t far behind, with Serena coming in third among the small group of players for which we had sufficient data. Since then, two things have changed: The Match Charting Project now has a lot more data on many more players, and a new generation of ball-bashers has threatened to make the rest of the tour look like weaklings in comparison.

The aggression score metric packs a lot of explanatory power in a simple calculation: It’s the number of point-ending shots (winners, unforced errors, or shots that induce a forced error from the opponent) divided by the number of shot opportunities. The resulting statistic ranges from about 10% at the lower extreme–Sara Errani’s career average is 11.6%–to 30%* at the top end. Individual matches can be even higher or lower, but no player with at least five charted matches sits outside of that range.

* Readers with a keen memory or a penchant for following links will notice that in Lowell’s orignial post, Kvitova’s aggregate score was 33% and Keys was also a tick above 30%. I’m not sure whether those were flukes that have since come back down with larger samples, or whether I’m using a slightly different formula. Either way, the ordering of players has remained consistent, and that’s the important thing.

Here are the top ten most aggressive WTA tour regulars of the 2010s before Sabalenka and Yastremska came along:

Rank  Player                      Agg 
1     Petra Kvitova             27.1%  
2     Julia Goerges             26.8%  
3     Serena Williams           26.8%  
4     Jelena Ostapenko          26.5%  
5     Camila Giorgi             26.0%  
6     Madison Keys              25.9%  
7     Coco Vandeweghe           25.9%  
8     Sabine Lisicki            25.6%  
9     Anastasia Pavlyuchenkova  24.0%  
10    Maria Sharapova           23.2%

All of these women rank among the top 15% of most aggressive players. They end points more frequently on their own racket than plenty of competitors we also consider aggressive, like Venus Williams (21.9%), Karolina Pliskova (21.6%), and Johanna Konta (22.3%). Ostapenko bridges the gap between the two generations; she wasn’t part of the discussion when aggression score was first introduced, though once she started winning matches, it was immediately clear that she’d challenge Kvitova at the top of this list.

Here’s the current top ten:

Rank  Player               Agg  
1     Dayana Yastremska  28.6%  
2     Aryna Sabalenka    27.6%  
3     Petra Kvitova      27.1%  
4     Julia Goerges      26.8%  
5     Serena Williams    26.8%  
6     Jelena Ostapenko   26.5%  
7     Viktoria Kuzmova   26.0%  
8     Camila Giorgi      26.0%  
9     Madison Keys       25.9%  
10    Coco Vandeweghe    25.9%

Yastremska, Sabalenka, and even Viktoria Kuzmova have elbowed their way into the top ten. Yastremska’s and Kuzmova’s places on this list might be a little premature, since their scores are based on only seven and nine matches, respectively. But Sabalenka’s pugnaciousness is well-documented: her Petra-topping score of 27.6% is an average across almost 30 matches.

Tennis tends to swing between extremes, with one generation developing skills to counteract the abilities of the previous one. It’s not yet clear whether the aggression of these young women will catapult them to the top–after all, Sabalenka won only five games today against Anisimova, whose aggression score is a more modestly high 23.0%. Perhaps as they gain experience, they’ll develop more well-rounded games and return Kvitova to her place at the top.

In the meantime, we have the privilege of watching some of the hardest hitters in WTA history battle it out. Tomorrow, Yastremska will contest her first third round at a major in a must-watch match against Serena. There will be fireworks, but it’s safe to say there won’t be much in the way of rallies.

What I Should’ve Known About Playing Styles and Upsets

In the podcast Carl Bialik and I recorded yesterday, I mentioned a pet theory I’ve had for awhile, that upsets are more likely in matches between players with contrasting styles. The logic is fairly simple. If you have two counterpunchers going at it, the better counterpuncher will probably win. If two big servers face off, the better big server should have no problem. But if a big server plays a counterpuncher … then, all bets are off.

We’ve seen Rafael Nadal struggle against the likes of John Isner and Dustin Brown, and and we’ve seen big servers neutralized by their opposites, as in Marin Cilic’s 1-6 record against Gilles Simon. There are upsets when similar styles clash, as well, but as untested theories go, this one is appealing and not obviously flawed.

Then, to kick off the 2019 Australian Open, Reilly Opelka knocked out Isner. Playing styles don’t come much more evenly matched, and the veteran was the heavy favorite. It was a perfect example of the kind of match I would expect to follow the script, yet the underdog came out on top. They played four tiebreaks and there were only two breaks of serve, but Opelka didn’t even need the Australian Open’s new fifth-set 10-point tiebreak. While it’s just one match, of course, it suggested that I ought to look more closely at my assumptions.

After a couple of hours playing with data this afternoon, my theory is no longer untested … and it turned out to be flawed. Fortunately, it isn’t just another negative result. Playing style is related to upset likelihood, but not in the way I predicted.

Measuring predictability

Let me explain how I tested the idea, and we’ll work our way to the results. First, I used used Match Charting Project data to calculate aggression score for every ATP player with at least 10 charted matches since 2010. Aggression score is, essentially, the percentage of shots that end the point (by winner, unforced error, or inducing a forced error), as will serve as our proxy for playing style. That gives us a group of 106 players, from the conservative Simon and Yoshihito Nishioka with aggression scores around 13%, to the freewheeling Brown and Ivo Karlovic, with scores nearing 30%. I divided those 106 players into quartiles (by number of matches, not number of players, so each quartile contains between 21 and 31 players) so we could see how each general playing style fares against the others. Here are the groups:

(Aggression score conflates two things: big serving/big hitting and tactical aggression. Isner is sometimes not particularly aggressive, but because of his size and serve skill, he is able to end points so frequently that, statistically, he appears to be extremely aggressive. Accordingly, I’ll refer to “big servers” and “aggressive players” interchangeably, even though in reality, there are plenty of differences between the two groups.)

Limiting our view to these 106 men, I found just over 11,000 matches to evaluate and divided them into groups based on which quartiles the two players fell into. Each of the ten possible subsets of matches, like Q1 vs Q2, or Q4 vs Q4, contains at least 400 examples.

For every match, I used surface-adjusted Elo ratings to determine the likelihood that the favorite would win. That gives us pre-match odds that aren’t quite as accurate as what sportsbooks might offer, though they’re close.

Those pre-match odds are key to determining whether certain groups are more predictable than others. If there are 100 matches in which the favorite is given a 60% chance of winning, and the favorites win 70 of them, we’d say that the results were more predictable than expected. If the favorites win only 50, the results were less predictable.

Goodbye, pet theory

For the matches in each of the ten quartile-vs-quartile subsets, I calculated the average favorite’s chance of winning (“Fave Odds”), then compared that to the frequency with which the favorites went on to win (“Fave Win%”). The table below shows the results, along with the relationship between those two numbers (“Ratio”). A ratio of 1.0 means that matches within the subset are exactly as predictable as expected; higher ratios mean that the favorites were even better bets than the odds gave them credit for, and lower ratios indicate more upsets than expected.

[table id=1 /]

There’s a striking finding here: The largest ratio, marking the most predictable bucket of matches, is for the most conservative pairs of players, while the smallest ratio, pointing to the most frequent upsets, is for the most aggressive players.

Before analyzing the relationship, let’s check one more thing. The very best players aren’t evenly divided throughout the quartiles, since Q1 has two of the big four. Elo-based match predictions–one of the building blocks of these results–are tougher to get right for the best players and the most uneven matchups, so we need to be careful whenever the elites might be influencing our findings. Therefore, let’s look at the same numbers, but this time for only those matches in which the favorite has a 50% to 70% chance of winning. This way, we exclude many of the best players’ matchups and all of their more lopsided contests:

[table id=2 /]

We discard about 40% of our sample, but the predictability trend remains the generally the same. In both the overall sample and the narrower 50%- to 70%-favorite subset, the strongest relationship I could find was between the predictability ratio and the quartile of the less aggressive player. In other words, a counterpuncher is likely to have more predictable results–regardless of whether he faces a big server, a fellow counterpuncher, or anyone in between–than a more aggressive player.

Back to basics

My initial theory is clearly wrong. I expected to find that Q1 vs Q1 matches were more predictable than average, and I was right. But by my logic, I also guessed that Q4 vs Q4 matches went according to script, and that other pairings, like Q1 vs Q4, would be more upset-prone. I would have done better had I let the neighbor’s cat make my predictions for me.

Instead, we find that that matches with more aggressive players are more likely to result in surprises. That doesn’t sound so groundbreaking, and it’s something I should’ve seen coming. Big servers tend to hold serve more often and break serve less frequently, meaning that their matches end with narrower margins, opening the door for luck to play a larger role, especially when sets and matches are determined by tiebreaks.

After all this, you might be thinking that I’ve squandered my afternoon, plus another few minutes of your attention, arriving at something obvious and unremarkable. I agree that it’s not that exciting to proclaim that big servers are more influenced by luck. But there’s still a useful–even surprising–discovery buried here.

Exponential upset potential

We know that the most one-dimensional players are more subject than others to the ups and downs of luck, thanks to the narrow margins of tiebreaks. For a man who rarely breaks serve, no match is a guaranteed win; for a man who rarely gets broken, no opponent is impossible to beat. However, I would have expected that the unpredictability of big servers was already incorporated into our match predictions, via the Elo ratings of the big servers. If a player has unusually random results, we’d expect his rating to drift toward tour average. That’s one reason that it’s very difficult for poor returners to reach the very top of the rankings.

But apparently, that isn’t quite right. The randomness-driven Elo ratings of our big servers do a nearly perfect job of predicting match outcomes against counterpunchers, and they’re only a little bit too confident against the more middle-of-the-road players in Q2 and Q3. Against each other, though, upsets run rampant. That extremely volatile fraction of results–the tiebreak-packed outcomes when the biggest servers face off–only accounts for part of these players’ ratings.

We’re accustomed to getting unpredictable results from the most aggressive players, with their big serves, inconsistent returns, and short rallies. Today’s findings give us a better idea of when these do and do not occur. Against counterpunchers, things aren’t so unpredictable after all. But when big servers play each other, we expect the unexpected–and the results are even more unpredictable than that.

Match Charting Project Update and New Template

The Match Charting Project is a crowd-sourced, volunteer effort to gather exhaustive shot-by-shot data on professional tennis matches. We’re closing in on the 5,000-match mark, and are building a wide range of meaningful datasets for subsets of players and matches. We have shot-by-shot records of nearly all grand slam finals, most Masters finals, many major semi-finals and Premier finals, all head-to-head matches between members of the Big Four, and nearly every match ever played by Simona Halep.

Here’s the complete list of charted matches, and here’s an example of the data assembled for a single player.

I hope you’re inspired by what we’ve already achieved to contribute to the project. We have several dedicated charters and over 100 people have charted matches over the lifetime of the project, but the more data we have, the more valuable the entire effort becomes. Click here to find out more about getting started.

The immediate impetus for today’s post is the updated Excel template I’m releasing today, version 0.3.0. Revising the template was a necessity ahead of the 2019 Australian Open, because of the unique new rules under which AO matches will be played. The template now handles several rules variations, including 2019 AO rules (a super-tiebreak at 6-6 in the deciding set), 2019 Wimbledon rules (a standard tiebreak at 12-12 in the deciding set), and ATP NextGen Finals rules (no-ad, first to four games, standard tiebreak at 3-3). We’ve already posted the first charted match from the NextGen Finals, last year’s title match between Tsitsipas and De Minaur.

If you’re already familiar with charting and the MCP Excel template, all you need to know is that you can enter “A” in cell B14 to indicate that the match is played under 2019 AO rules. (For 2019 Wimbledon, use “T”, and for NextGen Finals, use “N”.)

I’ve also made an addition to the shot-by-shot syntax to handle situations in which a player stops a rally to challenge (or have a mark checked) but is proven wrong. If you’re charting, check the Instructions tab in the new Excel template for more details.

Finally, the MatchStats tab now includes a running tally of total shots and average rally length.

For those of you who are already contributors to the MCP: Thank you very much for your efforts. For everyone else, I hope 2019 is the tennis season when you decide to give it a try.

Click here to download the new template.

The Right Amount of Serve-and-Volley

Italian translation at settesei.it

In modern tennis, players approach the net at their own peril, especially behind their serve. Technological advances in both strings and rackets have made passing shots faster and more accurate, giving an added edge to the returner. It’s hard to imagine the game changing so that serve-and-volleying would once again become a dominant tactic.

Yet pundits and commentators often suggest that players should approach the net more often, sometimes advocating for more frequent serve-and-volleying. In a recent article at FiveThirtyEight, Amy Lundy brought some numbers to the discussion, pointing out that at the US Open this year, women have won 76% of their serve-and-volley points and men have won 66%. She also provides year-by-year numbers from the women’s Wimbledon draw showing that for more than a decade, the serve-and-volley success rate has hovered around the mid-sixties.

Sounds good, right? Well… not so fast. Through the quarter-finals in New York, men had won roughly 72% of their first-serve points. Most serve-and-volley attempts come on first serves, so a 66% success rate when charging the net doesn’t make for much of a recommendation. The women’s number of 76% is more encouraging, as the overall first-serve win rate in the women’s draw is about 64%. But as we’ll see, WTA players are usually much less successful.

Net game theory

When evaluating a tactic, we have to start by recognizing that players and coaches generally know what they’re doing. Sure, they make mistakes, and they can fall into suboptimal patterns. But it would be a big surprise to find that they’ve left hundreds of points on the table by ignoring a well-known option. If more frequent serve-and-volleying was such a slam dunk, wouldn’t players be doing so?

I dug into Match Charting Project data to get a better idea of how often players are using the serve-and-volley, how successful it has been. and, just as important, how successful they’ve been when they aren’t using it. The results are considerably more mixed than the serve-and-volley cheerleaders would have it.

Let’s start with the women. In close to 2,000 charted matches from 2010 to the present, I found 429 player-matches with at least one serve-and-volley attempt. After excluding aces, regardless of whether the server was intending to approach, those 429 players combined for 1,191 serve-and-volley attempts–95% of them on first serves–of which they won 747. Had those players not serve-and-volleyed on those 1,191 points and won at the same rate as their first- and second-serve baseline points in the same matches, they would have won 725 points. In other words, serve-and-volleying resulted in a winning percentage of 62.7%, and staying back was good for 60.9%. Just to be clear, this is a direct comparison of success rates for the same players against the same opponents, controlling for the differences between first and second serves.

A difference of nearly two percentage points is nothing to sneeze at, but it’s a far cry from the more than ten percent gap we’ve seen on the women’s side at the US Open this year. And it might not be enough of a benefit for many players to overcome their own discomfort or lack of familiarity with the tactic.

When we apply the same analysis to the men, the results are downright baffling. We have more data to work with here: In nearly 1,500 charted matches from 2010 to the present, more than half of the possible player-matches (1,631) tried at least one serve-and-volley. About four in five–once again excluding aces–were first serves. The tour-wide success rate was similar to what we’ve seen at the Open this year, at 66.8%.

Controlling for first and second serves, the same servers, at the same tournaments, facing the same opponents, won points at a 72.2% rate when they weren’t serve-and-volleying. That’s a five percentage point gap* that says men, on average, and serve-and-volleying too much.

* Technical note: These overall rates simply tally all the serve-and-volley attempts and successes for all players. Thus, they may give too much weight to frequent netrushers. I ran the same calculation in two other ways: giving equal weight to each player-match, and weighting each player-match by ln(a+1), where a is the number of serve-and-volley attempts. In both cases the gap shrunk a bit, to four percentage points, which doesn’t change the conclusion.

I was shocked to see this result, and I’m not sure what to make of it. It’s roughly the same for men who serve-and-volley frequently as for those who don’t, so it isn’t just an artifact of, say, the odd points that an Ivo Karlovic or Dustin Brown plays from baseline, or the low-leverage status of the occasional point when a baseliner decides to serve-and-volley. Since I don’t have a good explanation for this, I’m going to settle for a much weaker claim that I can make with more confidence: The evidence doesn’t suggest that men, in general, should serve-and-volley more.

Data from the women’s game is more encouraging for those who would like to see more serve-and-volleying, but it is still rather modest. Certainly, the 76% success rate in Flushing this year is a misleading indicator of what WTA players can expect to reap from the tactic on a regular basis. It’s possible that some women should come in behind their serves more often. But the overall evidence from a couple thousand matches suggests sticking to the baseline is just as good of a bet–if not better.

New Match Charting Project Excel Template

For the first time in more than two years, I’m ready to release a new, substantially improved version of the MatchChart excel template, the platform on which volunteers log matches for the Match Charting Project.

New in version 0.2.0:

  • Color-coding of the players, as well as game-ending and set-ending rows, to make it easier to keep track of where you are in a match;
  • A new shot code for drop volleys, to differentiate them from other volleys;
  • Total points won and total points shown in the MatchStats tab;
  • Options to handle certain now-rare match formats, such as tiebreaks at 8-all (as in some 1970s Wimbledons) and matches with no tiebreaks at all.

If you’re already an experienced contributor, just click here to download the new version. Take a quick look through the Instructions tab, as I’ve highlighted the relevant changes.

If you’re new to the MCP, please take a look at my Quick Start Guide, after which you can give the new template a spin.

The Last 156 Men’s Grand Slam Finals

I’m proud to report a big new milestone for the Match Charting Project! We’ve completed the set of men’s Grand Slam finals back to 1980, something that I’ve aspired to since the early days of the project, and a project that has drawn on a lot of effort from many contributors to the project, especially Edo, who is responsible for a huge part of this accomplishment.

Here’s the complete list of charted slam finals, with links to the shot-by-shot data for each match.

From 1980 to this year’s Australian Open, that’s 152 consecutive men’s finals. I went on a bit of a spree last week, which extended the set back to 1979 and upped the total to 156. We’ve got a few earlier slam finals in the database as well, though there’s a limit to how much more we’ll be able to achieve: Before the late 1970s, video quality and availability decreases sharply.

For researchers, as well as those interested in tennis history, this is valuable stuff, made even more useful by its completeness. With the exception of a handful of missing points here and there, the Match Charting Project now includes a wealth of data for the entirety of all of these matches: serve direction, shot types, strategic choices (like serve-and-volleying), and much more, all in a standard format.

It’s particularly satisfying the check off the last few items on a list. (In this case, the final missing pieces were 1987 Roland Garros and the 1981 Australian Open.) Even though 156 matches is a small fraction of the nearly 4,000 contests tracked as part of the MCP, the subset’s completeness means that we can study it without worrying about the non-random nature of video availability and fan interest. If you want to look into, for instance, how net play has changed at Wimbledon over the last four decades, we’ve got the entire run.

In that vein, we are working on several other noteworthy subsets: Masters 1000 finals, 2018 tour-level finals, meetings between members of the big four, and finals played by members of the big four, among others.

We’re getting close to the complete run of women’s slam finals, as well. We’re up to 137 of the 152 since 1980, and have them all from 1999-present. We haven’t been able to find video for the rest, most notably the 1998 US Open (Davenport-Hingis), 1994 Roland Garros (Graf-Pierce), 1994 US Open (Sanchez-Graf), and 1991 Australian Open (Seles-Novotna). The complete list is here, and the remainder date from 1980-86. If you can help us find any of these, please let me know!

As always, if you find this project interesting, please contribute. Our 2.3 million shots worth of detailed data didn’t appear by magic–we rely on volunteers to chart matches, and I hope you’ll join our ranks. Here’s why I think you should, and here’s how you can get started.

 

 

 

The Most Aggressive ATP Returners

In yesterday’s post, I outlined a new method to measure return aggression. Using Aggression Score (AS) as a starting point, I made some adjustments in order to treat return winners (and induced forced errors) and return errors separately. The resulting metric–Return Aggression Score (RAS)–gives equal weight to return winners and return errors. A positive RAS represents an aggressive return game, while a negative number indicates a more conservative one. The most aggressive single-match performances were nearly four standard deviations above the mean, while player averages varied between about one standard deviation above and below the mean.

We can now point the algorithm at the ATP, and calculate RAS for each player in the 1,500 or so 2010-present men’s matches logged by the Match Charting Project.

The difference between the frequency of return errors and return winners is even greater for men than it is for women. The WTA tour averages, as we saw yesterday, are 17.8% and 5.5%, respectively, and the men’s averages are 20.9% and 4.1%. Thus, treating the two categories separately is even more important when analyzing ATP matches.

The overall range in single-match RAS figures is about the same as it is for women. The most aggressive one-match returners are nearly four standard deviations above the mean (a RAS mark near 4.0), while the lowest are almost two standard deviations below (RAS marks near -2.0). What differs between genders is that the most aggressive men’s single-match performances are not clustered around one player, as Serena Williams dominates the women’s list. Of the top ten one-match men’s RAS marks, only one player appears twice, and that is partly an accident:

Year  Event         Returner      Opponent   RAS  
2015  Halle         Berdych       Karlovic  3.96  
2014  Halle         D Brown       Nadal     3.72  
2016  Stuttgart     Marchenko     Groth     3.49  
2014  Aus Open      Dolgopolov    Berankis  2.99  
2016  Dallas CH     Tiafoe        Groth     2.91  
2014  Bogota        J Wang        Karlovic  2.79  
2015  Fairfield CH  Tiafoe        D Brown   2.72  
2017  Montpellier   De Schepper   M Zverev  2.64  
2015  Madrid        Isner         Kyrgios   2.60  
2014  Halle         An Kuznetsov  D Brown   2.58

Two factors make it more likely a returner appears on this list: His opponent, and the surface. Facing a serve-and-volleyer means adopting a higher-risk return strategy, and playing on a faster surface has a similar effect. Four of the top ten matches here were played on grass, and seven of the ten returners faced opponents who often come in behind their serves. Frances Tiafoe is partly responsible for his double-appearance here, but I suspect it has more to do with his opponents.

Grass is, by far, the most extreme surface in its effect on return tactics. Here are the numbers for each court type, along with the RAS of the average match on that surface:

Surface  RetE%  RetW%    RAS  
Hard     21.4%   4.1%   0.04  
Grass    25.3%   5.6%   0.54  
Clay     18.5%   3.5%  -0.24  
Average  20.9%   4.1%   0.00

Even though the average clay court match isn’t as extreme as a grass court match in this regard, the ten least aggressive single-match return performances all took place on clay, five of them recorded by Rafael Nadal.

Player averages

The Match Charting Project has at least 10 matches (2010-present) for about 75 players. Here is the top quintile, the 15 most aggressive players of that group:

Player                 Matches  RetPts   RAS  
Dustin Brown                11     676  1.90  
Ivo Karlovic                16    1116  0.85  
John Isner                  30    2202  0.77  
Alexandr Dolgopolov         20    1417  0.76  
Philipp Kohlschreiber       18    1334  0.69  
Lukas Rosol                 11     841  0.67  
Vasek Pospisil              14     812  0.62  
Andrey Kuznetsov            11     585  0.54  
Benoit Paire                17    1198  0.54  
Jeremy Chardy               14     923  0.39  
Kevin Anderson              23    1681  0.39  
Kei Nishikori               47    3128  0.38  
Milos Raonic                42    3211  0.34  
Sam Querrey                 17    1219  0.31  
Fernando Verdasco           17    1109  0.30

There’s aggression, and then there’s Dustin Brown. No other player is one full standard deviation above average, and he is nearly two, more than twice as aggressive as the next-most tactically extreme ATPer.

We don’t see quite the same extremes in the other direction, just a bunch of clay-courters:

Player                  Matches  RetPts    RAS  
Jiri Vesely                  11     716  -0.76  
Marcel Granollers            12     746  -0.64  
Paolo Lorenzi                13     912  -0.58  
Inigo Cervantes Huegun       10     705  -0.58  
Tommy Robredo                10     622  -0.57  
Damir Dzumhur                11     688  -0.56  
Guido Pella                  11     749  -0.51  
Guillermo Garcia Lopez       10     734  -0.49  
Casper Ruud                  16    1000  -0.48  
Hyeon Chung                  10     621  -0.48  
Rafael Nadal                157   11773  -0.42  
Richard Gasquet              36    2180  -0.42  
Roberto Bautista Agut        25    1633  -0.42  
Diego Schwartzman            44    3289  -0.42  
Juan Martin Del Potro        42    2900  -0.40

These least-aggressive numbers are partly a reflection of playing styles, and partly the surface, as we’ve already seen.

Next, let’s look at how much players alter their style to the circumstances. Here are 16 players–top guys along with some others I found interesting–along with their average RAS numbers on the three major surfaces:

Player                   RAS   Hard   Clay  Grass  
John Isner              0.77   0.71   1.03   0.72  
Marin Cilic             0.28   0.09   0.02   1.38  
Jo Wilfried Tsonga      0.24   0.31  -0.22   0.38  
Gilles Muller           0.10   0.07  -0.74   1.13  
Roger Federer           0.08   0.04  -0.07   0.40  
Grigor Dimitrov         0.07   0.12  -0.30   0.28  
Novak Djokovic          0.02   0.03  -0.12   0.25  
Nick Kyrgios            0.02  -0.06   0.07   1.20  
Jack Sock              -0.08  -0.09   0.08         
Stanislas Wawrinka     -0.09  -0.11  -0.23   0.95  
Alexander Zverev       -0.13  -0.06  -0.33   0.18  
Andy Murray            -0.20  -0.25  -0.32   0.15  
Dominic Thiem          -0.24  -0.13  -0.40   0.25  
Juan Martin Del Potro  -0.40  -0.43  -0.58  -0.07  
Diego Schwartzman      -0.42  -0.34  -0.45         
Rafael Nadal           -0.42  -0.25  -0.76   0.57

The big servers have some surprises in store: John Isner is more aggressive on the return on clay than on other surfaces, and Jack Sock and Nick Kyrgios show the same, at least compared to hard courts. Marin Cilic is extremely aggressive on the grass court return, but his clay court tactics are similar to those on hard courts. In stark contrast is Gilles Muller, second only to Nadal as a conservative returner on clay, but quite aggressive on other surfaces.

One of the many underexplored topics in tennis analytics is the different ways players change  (or choose not to change) their tactics on different surfaces. While comparing Return Aggression Score by surface is a tiny step in that direction, it does suggest just how much those strategies vary.

As always, a reminder that analyses like these are only possible with the volunteer-generated shot-by-shot logs of the Match Charting Project. I hope you’ll contribute.

 

Measuring Return Aggression

In the last couple of years, I’ve gotten a lot of mileage out of a metric called Aggression Score (AS), first outlined here by Lowell West. The stat is so useful due to its simplicity. The more aggressive a player is, the more she’ll rack up both winners and unforced errors. AS, then, is essentially the rate at which a player hits winners and unforced errors.

Yet one limitation lies in Aggression Score’s simplicity. It works best when winners and unforced errors move together, and when they are roughly similar. If someone is having a really bad day, her unforced errors might skyrocket, resulting in a higher AS, even if the root cause of the errors is poor play, not aggression. On the flip side, a locked-in player will see her AS increase by hitting more winners, even if those winners are more a reflection of good form than a high-risk tactic.

I’ve long wanted to extend the idea behind Aggression Score to return tactics, but when we narrow our view to the second shot of the rally, the simplicity of the metric becomes a handicap. On the return, the vast majority of “aggressive” shots are errors, so the results will be swamped by error rate, minimizing the role of return winners, which are a more reliable indicator. Using Match Charting Project data from 2010-present women’s tennis, returns result in errors 18% of the time, while they turn into winners (or they induce forced errors) less than one-third as often, 5.5% of the time. The appealingly simple Aggression Score formula, narrowed to consider only returns of serve, won’t do the job here.

Return aggression score

Let’s walk through a formula to measure return aggression, using last month’s Miami final between Sloane Stephens and Jelena Ostapenko as an example. Tallying up return points (excluding aces and service winners), along with return errors* and return winners** for both players from the match chart, we get the following:

Returner          RetPts  RetErr  RetWin  RetE%  RetW%  
Sloane Stephens       64       9       1  14.1%   1.6%  
Jelena Ostapenko      63      11       6  17.5%   9.5%

* “errors” are a combination of forced and unforced, because most return errors are scored as forced errors, and because the distinction between the two is so unreliable as to be meaningless. Some forced error returns are nearly impossible to make, so they don’t really belong in this analysis, but with the state of available data, it’ll have to do.

** throughout this post, I’ll use “winners” as short-hand for “winners plus induced forced errors” — that is, shots that were good enough to end the point.

These numbers make clear which of the two players is the aggressive one, and they confirm the obvious: Ostapenko plays much higher-risk tennis than Stephens does. In this case, Ostapenko’s rates are nearly equal to or above the tour averages of 17.8% and 5.5%, while both of Stephens’s are well below them.

The next step is to normalize the error and winner rates so that we can more easily see how they relate to each other. To do that, I simply divide each number by the tour average:

Returner          RetE%  RetW%  RetE+  RetW+  
Sloane Stephens   14.1%   1.6%   0.79   0.28  
Jelena Ostapenko  17.5%   9.5%   0.98   1.73

The last two columns show the normalized figures, which reflect how each rate compares to tour average, where 1.0 is average, greater than 1 means more aggressive, and less than 1 means less aggressive.

We’re not quite done yet, because, as Ostapenko and Stephens illustrate, return winner rates are much noisier than return error rates. That’s largely a function of how few there are. The gap between the two players’ normalized rates, 0.28 and 1.73, looks huge, but represents a difference of only five winners. If we leave return winner rates untouched, we’ll end up with a metric that varies largely due to movement in winner rates–the opposite problem from where we started.

To put winners and errors on a more equal footing, we can express both in terms of standard deviations. The standard deviation of the adjusted error ratio is 0.404, while the standard deviation of the adjusted winner ratio is 0.768, so when we divide the ratios by the standard deviations, we’re essentially reducing the variance in the winner number by half. The resulting numbers tell us how many standard deviations a certain statistic is above or below the mean, and these final results give us winner and error rates that are finally comparable to each other:

Returner          RetE+  RetW+  RetE-SD  RetW-SD  
Sloane Stephens    0.79   0.28    -0.52    -0.93  
Jelena Ostapenko   0.98   1.73    -0.05     0.95

(Math-oriented readers might notice that the last two steps don’t need to be separate; we could just as easily think of these last two numbers as standard deviations above or below the mean of the original winner and error rates. I included the intermediate step to–I hope–make the process a bit more intuitive.)

Our final stat, Return Aggression Score (RAS) is simply the average of those two rates measured in standard deviations:

Returner          RetE-SD  RetW-SD    RAS  
Sloane Stephens     -0.52    -0.93  -0.73  
Jelena Ostapenko    -0.05     0.95   0.45

Positive numbers represent more aggression than tour average; negative numbers less aggression. Ostapenko’s +0.45 figure is higher than about 75% of player-matches among the nearly 4,000 in the Match Charting Project dataset, though as we’ll see, it is far more conservative than her typical strategy. Stephens’s -0.73 mark is at the opposite position on the spectrum, higher than only one-quarter of player-matches. It is also lower than her own average, though it is higher than the -0.97 RAS she posted in the US Open final last fall.

The extremes

The first test of any new metric is whether the results actually make sense, and we need look no further than the top ten most aggressive player-matches for confirmation. Five of the top ten most aggressive single-match return performances belong to Serena Williams, and the overall most aggressive match is Serena’s 2013 Roland Garros semifinal against Sara Errani, which rates at 3.63–well over three standard deviations above the mean. The other players represented in the top ten are Ostapenko, Oceane Dodin, Petra Kvitova, Madison Keys, and Julia Goerges–a who’s who of high-risk returning in women’s tennis.

The opposite end of the spectrum includes another group of predictable names, such as Simona Halep, Agnieszka Radwanska, Caroline Wozniacki, Annika Beck, and Errani. Two of Halep’s early matches are lowest and third-lowest, including the 2012 Brussels final against Radwanska, in which her return aggression was 1.6 standard deviations below the mean. It’s not as extreme a mark as Serena’s performances, but that’s the nature of the metric: Halep returned 46 of 48 non-ace serves, and none of the 46 returns went for winners. It’s tough to be less aggressive than that.

The leaderboard

The Match Charting Project has shot-by-shot data on at least ten matches each for over 100 WTA players. Of those, here are the top ten, as ranked by RAS:

Player                    Matches  RetPts   RAS  
Oceane Dodin                   11     665  1.18  
Aryna Sabalenka                11     816  1.12  
Camila Giorgi                  19    1155  1.07  
Mirjana Lucic                  11     707  1.05  
Julia Goerges                  27    1715  0.94  
Petra Kvitova                  65    4142  0.90  
Serena Williams                91    5593  0.90  
Jelena Ostapenko               35    2522  0.88  
Anastasia Pavlyuchenkova       21    1180  0.78  
Lucie Safarova                 34    2294  0.77

We’ve already seen some of these names, in our discussion of the highest single-match marks. When we average across contests, a few more players turn up with RAS marks over one full standard deviation above the mean: Aryna Sabalenka, Camila Giorgi, and Mirjana Lucic-Baroni.

Again, the more conservative players don’t look as extreme: Only Madison Brengle has a RAS more than one standard deviation below the mean. I’ve included the top 20 on this list because so many notable names (Wozniacki, Radwanska, Kerber) are between 11 and 20:

Player                Matches  RetPts     RAS  
Madison Brengle            11     702   -1.06  
Monica Niculescu           32    2099   -0.93  
Stefanie Voegele           12     855   -0.85  
Annika Beck                16    1181   -0.78  
Lara Arruabarrena          10     627   -0.72  
Johanna Larsson            14     873   -0.65  
Barbora Strycova           20    1275   -0.63  
Sara Errani                25    1546   -0.60  
Carla Suarez Navarro       36    2585   -0.55  
Svetlana Kuznetsova        27    2271   -0.55 

Player                Matches  RetPts     RAS  
Viktorija Golubic          16    1272   -0.53  
Agnieszka Radwanska        96    6239   -0.51  
Yulia Putintseva           22    1552   -0.51  
Caroline Wozniacki         80    5165   -0.50  
Christina McHale           11     763   -0.48  
Angelique Kerber           93    6611   -0.46  
Louisa Chirico             13     806   -0.44  
Darya Kasatkina            26    1586   -0.43  
Magdalena Rybarikova       12     725   -0.41  
Anastasija Sevastova       30    1952   -0.40

A few more notable names: Halep, Stephens and Elina Svitolina all count among the next ten lowest, with RAS figures between -0.30 and -0.36. The most “average” player among game’s best is Victoria Azarenka, who rates at -0.08. Venus Williams, Johanna Konta, and Garbine Muguruza make up a notable group of aggressive-but-not-really-aggressive women between +0.15 and +0.20, just outside of the game’s top third, while Maria Sharapova, at +0.63, misses our first list by only a few places.

Unsurprisingly, these results track quite closely to overall Aggression Score figures, as players who adopt a high-risk strategy overall are probably doing the same when facing the serve. This metric, however, allows to identify players–or even single matches–for which the two strategies don’t move in concert. Further, the approach I’ve taken here, to separate and normalize winners and errors, rather than treat them as an undifferentiated mass, could be applied to Aggression Score itself, or to other more targeted versions of the metric, such as a third-shot AS, or a backhand-specific AS.

As always, the more data we have, the more we can learn from it. Analyses like these are only possible with the work of the volunteers who have contributed to the Match Charting Project. Please help us continue to expand our coverage and give analysts the opportunity to look at shot-by-shot data, instead of just the basics published by tennis’s official federations.