Italian translation at settesei.it
At the end of Turing’s Cathedral, George Dyson suggests that while computers aren’t always able to usefully respond to our questions, they are able to generate a stunning, unprecedented array of answers–even if the corresponding questions have never been asked.
Think of a search engine: It has indexed every possible word and phrase, in many cases still waiting for the first user to search for it.
Tennis Abstract is no different. Using the menus on the left-hand side of Roger Federer’s page–even ignoring the filters for head-to-heads, tournaments, countries, matchstats, and custom settings like those for date and rank–you can run five trillion different queries. That’s twelve zeroes–and that’s just Federer. Judging by my traffic numbers, it will be a bit longer before all of those have been tried.
Every filter is there for a reason–an attempt to answer some meaningful question about a player. But the vast majority of those five trillion queries settle debates that no one in their right mind would ever have, like Roger’s 2010 hard-court Masters record when winning a set 6-1 against a player outside the top 10. (He was 2-0.)
The danger in having all these answers is that it can be tempting to pretend we were asking the questions–or worse, that we were asking the questions and suspected all along that the answers would turn out this way.
The Hawkeye data on tennis broadcasts is a great example. When a graphic shows us the trajectory of several serves, or the path of the ball over every shot of a rally, we’re looking at an enormous amount of raw data, more than most of us could comprehend if it weren’t presented against the familiar backdrop of a tennis court. Given all those answers, our first instinct is too often to seek evidence for something we were already pretty sure about–that Jack Sock’s topspin is doing the damage, or Rafael Nadal’s second serve is attackable.
It’s tough to argue with those kind of claims, especially when a high-tech graphic appears to serve as confirmation. But while those graphics (or those results of long-tail Tennis Abstract queries) are “answers,” they address only narrow questions, rarely proving the points we pretend they do.
These narrow answers are merely jumping-off points for meaningful questions. Instead of looking at a breakdown of Novak Djokovic’s backhands over the course of a match and declaring, “I knew it, his down-the-line backhand is the best in the game,” we should realize we’re looking at a small sample, devoid of context, and take the opportunity to ask, “Is his down-the-line backhand always this good?” or “How does his down-the-line backhand compare to others?” Or even, “How much does a down-the-line backhand increase a player’s odds of winning a rally?”
Unfortunately, the discussion usually stops before a meaningful question is ever asked. Even without publicly released Hawkeye data, we’re beginning to have the necessary data to research many of these questions.
As much as we love to complain about the dearth of tennis analytics, too many people draw conclusions from the pseudo-answers of fancy graphics. With more data available to us than ever before, it is a shame to mistake narrow, facile answers for broad, meaningful ones.
Excellent point and one that can’t be brought up enough.
The more context, the better. And we need to beware adding context (a storyline) that the data can’t address.
As a naïve fan who doesn’t work with data in this context at all, I wonder where the following query is too story-driven, or might be answerable – or whether the available samples wouldn’t be large enough to provide an answer:
I was re-watching a video of the Tennis Channel broadcast of Federer’s latest London match against Djokovic & listening, with little real interest, to Paul Annacone’s tepid commentary. My attention perked up after Federer won a long rally and Annacone was spoon-fed a question by the play-by-plan man about whether Federer should be engaging in such long rallies as a tactic. Of course Annacone said “no.” Federer might have won this rally, he added, but that was mere luck; really he should be trying to keep the rallies short, esp. against Djokovic.
I didn’t agree with this. Normally it would make seem to sense, but there were mitigating factors and previous history that Annocone seemed to overlook. The major factor was the surface – indoor hard. I’ve probably heard too many pundits say so, but my own belief is that this surface cuts down on errors from Federer, letting him stay in rallies more w/fewer shanks etc. And the history that came to my mind were two matches, both against Nadal – 2009 in Madrid, and 2011 in London, when he not only shut out Nadal but bagled him in the second set. Both matches saw long rallies in which Federer did not seek to set up a quick point but instead patiently hammered away at Nadal’s weaker backhand side. The 2011 match was notable for how well Federer was able to defend & how extraordinarily patient he was – Nadal was defending quite well & but Federer would simply reset and continue rather than seek a quick end. The surface allowed him to do it. He did the same thing in Madrid, perhaps because that surface too was more to his liking than Nadal’s.
Against Djokovic, Federer displayed similar rally tactics – sharply angling his strokes, changing up the height, etc., on both wings. And again, just as against Nadal in 2011, he began to draw errors from his opponent. So to me Annacone’s prescription was empty coach-speak.
It would seem easy enough to pull out the longer rallies & beyond that which player won more of them. However I’m not aware of any data set that can reveal the key moments where a player starts using a tactic because he senses confusion or hesitation developing in his opponent. It might well be that early on in the match Djokovic won the bigger share of the long rallies. How could we isolate those points and games later on in the match when it seemed Federer saw a faltering on Djokovic’s part & began rallying more, without cherry-picking? We may tell a story that “Federer denied Djokovic rhythm, and so Djokovic began making uncharacteristic errors,” but this doesn’t seem a data-driven story.
So perhaps this sort of query falls smack into the dark hole between data & anecdote?
Yeah, that’d be a tough one. Turning points are particularly difficult for any kind of analysis — they often seem obvious in retrospect, but how on earth do you spot one as it’s happening? And if you pick them out after the match, are you looking at a real “turning point,” or just the moment when results happened to change? (Any non-blowout match is going to give you some of the latter, even if it’s just random.)
Even if we picked arbitrary end points (e.g. compared first-set stats to second-set stats), it’s difficult to measure intention. You might find that rallies were longer in the second set than the first; maybe Fed hit fewer winners and fewer UFEs, suggesting less aggression … you could tentatively conclude something about what happened, but without knowing what the players were thinking, it’d be a big leap to say it was because Fed spotted a weakness.
I think that, in time, we’ll develop some proxies for these things (especially if we get hawkeye data or something similar) — perhaps we could look at how often Fed hits to Djok’s FH over the course of the match and say that in some particular timeframe, he was trying to break down that shot. Or use Lowell’s aggression index for different subsets of the match to take a crack at the tactic you’re talking about. I can also imagine a sort of ‘unpredictability’ index to indicate when a player is making more of an effort to mix things up.
But all of those will always be proxies — probably useful and insightful in the aggregate, probably drowned out by noise for single matches. Player often try things and don’t execute, or try and are foiled by their opponents, or try and then abandon shortly after … or even try and execute properly and get results, but not enough differently from their normal patterns that we’d notice in a one-match/150-point sample.
Tough stuff, and much easier to ruminate over the difficulties than to solve them.
Very good points! One issue that is claimed is that coaches want some sense out of the large quantity of analytics. Often they choose 2-4 things to carry home for the player. So while Djokovic’s backhand might have a ton of data, perhaps his team is only concerned with the backhand pass against Federer at the net. I did a significant body of analytics over the past 11-13 years. It included collecting thousands of shot sequences and analysis can generate general patterns. Such analytics might tell us what the most likely patterns are over several matchesz I actually collected data to randomize or generalize, rather than specifics. But both can tell you plenty. For example, we might be interested in only Nadal-Djokovics rallies over 10 shots long but you might miss the iceberg by looking only at the tip (of analytics). I’d often rather chart all the point sequences of several matches to really see if a statistical pattern emerges. There are benefits for both. It is hard to predict an opponents tactical strengths that day but understanding their frequent patterns and tendencies help form tactical plans and predictability in tactical situations.