The pandemic offers a wealth of natural experiments. Ever wondered how the presence of fans affects players? Before last March, we were mostly limited to speculation, because fans were almost always there. The tours have made various compromises to keep the action going, so we have a wealth of data from all sorts of different scenarios–with or without linespeople, with or without towelkids, and of course, with or without fans.
The closest thing we have to a “pure” natural experiment concerning the effect on fans on tennis players is the 2020 US Open. Flushing Meadows is usually packed with spectators on most courts, while in 2020, it was empty save a handful of support staff. There are confounding variables aplenty, such as the aforementioned lack of linespeople (on most courts) and towelkids, and we also must keep in mind that players entered the 2020 US Open with less recent match play than usual. It isn’t a perfect natural experiment–such things are exceedingly rare–but it is better than tennis usually offers.
What should we expect from spectator-free tennis? One suggestion comes from Ben Cohen and Joshua Robinson, who found in August that both basketball and soccer players were shooting more accurately in empty stadiums:
NBA players are making a higher percentage of their free throws and hitting corner 3-pointers at rates the league has never seen. Soccer players are striking dead balls more precisely than they did before the pandemic. Without the distraction of screaming fans, one part of their games seems to have improved: shooting.
We can already speculate that tennis won’t be so clear cut. For one thing, there weren’t screaming fans before the pandemic. For another, everything in tennis is a tradeoff: If you’re serving more accurately, you might be tempted to try for a bit more power or aim closer to the corners. The “accuracy” effect, then, wouldn’t show up as accuracy, but as increased speed, or some mix of several measures. But let’s not rush to throw in the towel (as it were)–let’s look at the numbers.
US Open, now and then
We’ll check four different stats for an empty-stadium effect: first serve in, double faults per second serve (the inverse of second serves made), first serve points won, and average first serve speed.
For each stat, we’ll calculate averages for men and women from 2019 and 2020 US Open single main draw matches, adjusted for player. (I’m using the data available in my slam_pointbypoint GitHub repository.) That is, we’ll limit our focus to those players who appeared in both tournaments and weight each player’s effect by the year they played the least. A player who served 300 points in 2019 and 100 points in 2020 will have a weight of 100 points in both calculations; a player who served 250 points in both years will have a weight of 250 points in both. This corrects for the different mix of players (and the amount that each player competed) in the two adjacent years, which might otherwise affect the numbers in a misleading way.
Here are the results:
WOMEN 2020 2019 Change
First in 61.8% 61.5% 0.5%
DF/second 13.7% 13.4% 2.0%
First won 66.4% 62.2% 6.6%
First KM/H 158.6 155.2 2.1%
MEN 2020 2019 Change
First in 61.7% 59.5% 3.7%
DF/second 10.7% 11.3% -5.4%
First won 72.9% 71.2% 2.3%
First KM/H 186.2 184.8 0.8%
The women didn’t really improve their accuracy: a slight uptick in first serves in, and a bigger decrease in second serves made. On the other hand, they won way more of their first-serve points in 2020 than in 2019, and they served 3.4 KM/h faster. That puts the accuracy figure in perspective–no, they didn’t make more first serves, but it appears that they traded speed for accuracy. They did quite well in the bargain.
The men, on average, took a different approach. They made more first serves and committed fewer double faults (Alexander Zverev notwithstanding), but they didn’t increase their first serve speed as much. Men also won more first serve points, though their gain was not the enormous boost seen by the women.
Improvements in context
Based on these year-to-year comparisons, it looks like both men and women served better without spectators. The women’s giant boost in first serve points won suggests that there are other factors beyond those we can easily measure–perhaps players were missing first serves at a typical rate not only because they were hitting harder, but also because they were aiming for the corners. It’s also possible that post-restart rustiness affected returns more than serves–in lockdown, it’s easier to drill your own serves than to keep in practice against elite-level first serves.
Another consideration is the usual year-to-year fluctuations. For women, the small changes in first serves in and second serves missed are less than half the magnitude of the year-to-year changes at the US Open between 2015 and 2019. These numbers will always drift up and down for a variety of reasons, sheer randomness not least among them.
The 6.6% jump in the women’s rate of first serve points won, on the other hand, is quite unusual. The average fluctuation in the previous four pairs of years is 1.7%. The serve speed increase is also unusually large. It’s a 2.1% jump, compared to a typical movement of about 0.7%.
The 2019-to-2020 changes in men’s rates are less noteworthy in context, even if they do tell a suggestive story. The rate of first serves made is surprisingly noisy, fluctuating an average of 2.7% in each pair of years between 2015 and 2019, so the fans-to-no-fans shift of 3.7% doesn’t prove much of anything. The double fault and serve speed changes are no greater than previous fluctuations.
The only slightly convincing “pandemic effect” on the men’s side is the percentage of first serve points won. As we’ve seen, men won 2.3% more such points in 2020 than in 2019, adjusted for the mix of players–an increase half-again as large as the typical fluctuation of 1.5%. That’s hardly a slam-dunk case for better post-restart serving. It could be pure luck, or it could be attributed to a mix of the many confounding variables I’ve already mentioned.
Serving isn’t shooting
This stuff is complicated. Penalty kickers in soccer have an objective that is clear to all–to score a goal. While tennis servers have a similarly simple aim–to win the point–the only part of the point they can completely control is the serve, and given the tradeoffs between speed, precision, and keeping the ball in the box, there’s no single variable that tells us whether a player is serving better.
Things get even hairier when we look for data beyond the US Open. We could do the same exercise for the last few years of French Opens, but remember that the post-restart Roland Garros had a sprinkling of paying fans. Is that halfway between an empty stadium and normalcy? Is it worse than a full stadium, because individual voices are easier to discern? Is it a mix, because the French fans filled up the stands for their native players and left other courts empty? I have no idea.
It is clear that women served harder than usual at the 2020 US Open, and they won way more first serve points than in recent years. Men served more accurately, even if their success rate didn’t translate into the same whopping success than the women’s adjustments did. What we can’t say for sure is how much of those shifts can be attributed to the empty stands in Flushing last year. Even the purest natural experiments don’t always return bulletproof findings.