Quantifying the Bias of an ATP Draw
ATP tennis draws are biased in favor of top-ranked players. If you’re ranked in the top four, you won’t face another top-16 player until the round of 16, you won’t face a top-8 player until the quarters, and you won’t face a fellow top-4 player until the semis. If you’re unseeded (out of the top 32 in slams, 16 or 8 in smaller tourneys), you’ll probably have to face a top-16 player just to get into the round of 16 … and you might draw a top-4 player in the first round.
This is the way it is, and it’s not going to change anytime soon. Since it’s the nature of the beast, we should better understand the effects of this system.
In short: The more highly ranked you are, the easier it probably will be to win the first few matches of a tournament. The further you go in the draw, the more points you earn, and the higher your ranking stays. A higher pre-tournament ranking–regardless of actual skill!–increases your odds of a better performance.
Thus, the rankings lag behind changes in skill level, creating a bias against rapidly-improving youngsters and players returning from long absences.
Let’s play around with this a bit. Before the Australian Open started, I published the odds that each player in the main draw would reach any given round. From that, we can calculate “expected points,” which gives us a way of directly comparing each player’s chances, given his skill level and his draw.
For instance, Nadal’s expected points were 1056, Federer’s were 857, and 11th-seed Jurgen Melzer’s were 227.
What happens if swap two players’ positions in the draw? Let’s try #4 and the top-ranked non-seed. Going into the tournament, #4 was Robin Soderling, and Phillip Kohlschreiber was #34, unseeded. In the draw as it actually happened, Soderling’s expected points were 515 and Kohlschreiber’s were 56, thanks in part to a 2nd round matchup against Tomas Berdych.
If we exchange Soderling’s and Kohlschreiber’s draw positions and run the simulation, we get very different results. Soderling’s expected points are 353 (down 31 percent) and Kohlschreiber’s expected points improve to 103 (up 84 percent).
The Soderling/Kohlschreiber swap may be an outlier. We can do better, and besides, I don’t want to type “Kohlschreiber” anymore.
Let’s try a new simulation. For each run, we’ll randomize the draw positions, so Nadal has an equal chance of drawing Marcos Daniel, Roger Federer, or anybody else in the first round.
The differences in the results are substantial. Nearly 75 percent of players have their expected points change by more than 10 percent. 39 of the 128 players see their expected points decrease with randomization, and those players are disproportionately seeds. The seeds are disproportionately high seeds.
Two types of players seem to benefit from the status quo:
- High seeds. They are guaranteed non-seeded opponents for at least two rounds, and lower-seeded opponents for a round or two after that.
- Marginal players who get lucky draws. The player who the Aussie Open draw benefited the most was wild card Benoit Paire. He was one of the weakest players in the field, but in the first round, he drew Flavio Cipolla, one of the few competitors who was even weaker.
Of the 39 players who do better under the original draw, 19 are seeds and 9 are WCs or qualifiers, mostly in situations like Paire’s. That leaves only 11 middle-of-the-pack, unseeded players who weren’t disadvantaged by the draw.
If the draw had been randomized, half of the field (mostly unseeded players) would have seen their expected points increase by more than 10 percent. 52 players would have jumped by 20 percent or more, 37 by more than 30 percent, and 22 by more than 50 percent.
To some extent, the bias is mitigated over the course of a season. Players like Kohlschreiber are disadvantaged by the draw so long as their ranking stays in the unseeded-but-good 33-50 range for slams, but in smaller tournaments, such a player is often seeded.
And, of course, by playing 20-30 tournaments, the draws are randomized for some players. Paire got lucky by drawing Cipolla in Melbourne, but he could just as easily have found himself pitted against a top seed.
As is intuitively obvious, draws are biased in favor of the top players, and that is one thing that isn’t mitigated by a year’s worth of tournaments. The top 12 seeds all did better in the actual draw simulation than in the randomized simulation, and I expect that would be true for the vast majority of tournaments.
If some players are consistently “winning” through draw bias, there must be losers. As we’ve seen, lower-ranked players can win big or lose big in a draw, but it stands to reason that, over the course of the season, they lose a little bit. At least until they overcome and disadvantage and become top-ranked players themselves.