We love competitive Pokémon here at Nugget Bridge. We largely think that TPCi has done a great job of improving the tournament system this season by making thoughtful changes such as adding a top cut to North American Regionals, changing the format of the European National tournaments to mirror North American Nationals, and applying a Championship Point system to the Video Game Championships (VGC). Pokémon is never going to get the type of support for competitive play that games like League of Legends, Starcraft, or most fighting games get because competitive players aren’t the primary audience in this case, but I think the changes made to the Play! Pokémon system this year make it pretty clear that the people running Pokémon’s tournament system want our circuit to be as competitive as possible.
One thing that differentiates Pokémon from those other competitive games is that no one is going to make any real money off of Pokémon’s prizes. We play this game in spite of it not having the type of monetary appeal many other games do because we care about it. We like the people we compete with and against, a fairly large group of committed players — in spite of Pokémon’s relatively small VGC playerbase — that has become very chummy. We like the strategy, prediction, and creativity involved in competitive Pokémon, in spite of the game involving more luck than any of us would prefer. Perhaps most of all, we enjoy competing, and a big part of what makes that competition fun is the format of the events. It is part of why Regionals has always felt like a chore on the way to Nationals and, for the best players, on the way to Worlds. Regardless of how much we enjoy the game, the competition part of the game can only be as good of an experience as the format allows it to be. With Pokémon mostly heading in the right direction from a competitive standpoint, one of the most important things to most of the players is ironing out the format of the game’s most important events. With prizes not being a major factor in Pokémon, getting the players to the place in the standings they deserve to be at while letting everyone play a reasonable number of games is the most important part of our tournaments.
In spite of the significant progress TPCi has made this year, it seems that every year there are a handful of needed improvements made to the system that get overshadowed by one or two strange decisions. Once again, all of the positive changes TPCi has made made this year are being overshadowed as we approach the penultimate events of the season by one huge mistake: the decision to force small top cut caps on United States Regionals (8) and, most importantly, on European and North American Nationals (16). Frustratingly, Pokémon VGC endures these smaller top cuts even as Pokémon TCG events, who have had many more years to iron out their qualification system, use the same Swiss system but transition into top cuts that are double our size or larger for the same number of players.
The top cut problem is fundamentally a Swiss problem. Swiss qualification rounds are a colossal improvement over the dark days of full single elimination, but Swiss has two major flaws when applied to Pokémon that require a sizable top cut to mitigate: the vast differences in the difficulty of opponents each player faces and the inherent randomness of a game with as much probability as Pokémon. Small top cuts accent these weaknesses of the Swiss system, greatly reducing the odds of the best players at each event getting to the much more reliable best-of-three rounds. I expect that the reasons for these small top cuts are a combination of TPCi trying to do the most it can with limited time at events while trying to make sure only the best players in tournaments are getting to the end. However, the small top cut sizes right now are often having the opposite impact and are preventing some of the best players from getting the chance to perform in the best-of-three stage that vastly improves the validity of results in a luck-heavy game like Pokémon.
The Role of Swiss in Pokémon
Once the format of only Worlds and then US Nationals, Swiss has expanded to include every live VGC tournament for good reason. Swiss creates a dramatically better experience for players than single elimination did: everyone who attends the event gets to play at least six to eight games and a single strange loss doesn’t eliminate anyone from the event on its own. Pokémon should absolutely continue running the Swiss stage very similarly to how they have in the past, as it is working beautifully and events in 2012 and 2013 are much more enjoyable than they were in 2011 and 2010. Hopefully, the improved format leads to more players coming back to tournaments and buying games. VGC tournaments need to help the game grow first, and the Swiss format helps make growth happen. The need for that growth causes some potential tournament formats other competitive video games use not to be viable, such as truly invitation only events (see US Nationals becoming open to everyone and Worlds inexplicably having a Last Chance Qualifier), but I think as long as events maintain the 6-8 rounds of Swiss that goal is being adequately achieved.
However, Swiss has a job to do for the event beyond just letting everyone play more games. Like the group or pools stage in most other competitive games, the Swiss stage in Pokémon is the qualification system that needs to find a way to send the best players to the bracket stage (top cut). The Swiss system already had a pretty extensive history of failing at this goal, and the top cuts this year being so much smaller than the system is designed for is causing problems to be much more frequent than in the past. The current format for National tournaments is likely going to create even stranger situations than the format for US Regionals did in that regard. Simply put, for a variety of reasons the best players aren’t winding up in the limited top cut slots with enough regularity. US Nationals was already an event that didn’t have enough top cut slots, and mirroring the new, downsized format in Europe with reduced top cut slots rather than increased slots is going to exacerbate that problem. Pokémon tournaments are more fun and competitive for players when more players get a chance to prove their skill in the main event rather than hoping they get through Swiss on what often comes down to an arbitrary tiebreaker like opponent win % or a couple of unfortunate early pairings with other skilled players.
Unlike most other video games, and even Pokémon’s own TCG, we only get to play in four to five official tournaments per year. With so few tournaments per year to draw points from, it is extremely important that each event is organized in a way that allows the best players to come out on top if the results of our circuit are to be reflective of our game’s best players in a each year. Currently, the impact of the random number generator and tournament-induced randomness of match-up difficulty is causing the system itself to have far more impact on the results of the tournament than the players themselves.
The Failings of Swiss: Un Golpe Crítico!
Pokémon is a video game that involves a lot of probability. I want to emphasize all losses to “luck” are not equal, and that a large part of the skill element of Pokémon is executing thoughtful probability management to avoid unnecessary risks. It is fair to say that one of the biggest parts of both battling and team building is doing everything you can to put the odds in your favor. However, Pokémon is also a game where the best you can do most of the time is just that: putting the odds in your favor.No matter how carefully you manage your odds in a tournament, if your opponents are close to your skill level, situations will occur where your opponent can get that timely Rock Slide flinch, Heat Wave burn, Sand Veil activation, Draco Meteor dodge, dreaded fully paralyzed/confused off of Prankster Thunder Wave/Swagger, or any number of other fantastic low probability rolls and change the path of a match — sometimes by too much for the otherwise superior player to recover from. We spend all year trying to dodge bullets, but it is a reality in the game we play that sometimes the bad numbers will come up and the RNG will take a weird game from us; something anyone who has played in a Wi-Fi tournament knows all too well. The luck in Pokémon is something every honest trainer knows is going to be a major factor in each of their tournament runs.
As competitive players, what we want from tournaments is a format that mitigates the impact of this sort of ridiculousness as much as possible. No one wants to be winning or losing all tournaments on this sort of (im)probability, so it is important that the format of tournaments has some cushion room to make it less likely players get “Pokémoned”, as we often say. Mitigating the impact of randomness and giving players more of a chance to fight against it is the single most important reason we want a higher percentage of players top cutting our events and getting into the best of three stage where they have more wiggle room to outplay their opponents. It is also the same reason single elimination best-of-1 in the past was a terrible format players hated.
While the current format of events is much improved, more progress still needs to be made for the final results of events to include the most deserving players. Players are much more likely to pick up two losses from these sort of RNG-influenced shenanigans during 8 rounds of Swiss than they are twice in three games of a top cut match. As such, the heads-up games usually produce the more deserving winner. Best-of-one Swiss tries to adjust for luck a little bit by not immediately eliminating players for losing one game to strangeness, but it fails to do an adequate job, especially with only one X-2 cutting at most events. With only one undefeated player going into the last round of Swiss at most events, for instance, strange luck in the last round of Swiss will almost always eliminate players from contention a round away from a best-of-three scenario. It is important for the validity of a tournament’s results to get as many players as possible into the top cut stage while still offering a tangible reward for doing well in the Swiss stage of the tournament.
With all of the probability in Pokémon, some of us tend to compare Pokémon more to other games of combined chance and skill (such as poker) rather than to most other video games, and I think the parallel works pretty well here. You have more control over your fate in Pokémon than in poker, but winning only 6/8 hands of heads-up in poker instead of 7/8 wouldn’t be something anyone would expect to end their days in a Poker tournament — yet in Pokémon it will be season ending if TPCi doesn’t expand the top cut for the remaining National tournaments. I don’t want to beat the idea of luck in the game over the head too much as we all know it is an obnoxious but inseparable part of our game. Luck is something the players have to do their best to mitigate, but for tournaments to be everything they can be, luck is something the tournament organizers have to help mitigate, too.
The Failings of Swiss: Strength of Schedule
I think the pairing system is by leaps and bounds the biggest problem with using Swiss and a small top cut. While the more conventional type of luck is easy to notice — it’s pretty hard to miss getting ravaged by a critical hit or not being able to move because you’re frozen — the impact of random pairings creating vastly different schedules for each player is harder to notice. Pokémon’s Swiss pairings system is based on a faulty assumption: that every player with the same record has the same skill. This assumption is used both to determine pairings, since you are (usually) randomly paired with someone who has the same record as you, as well as to determine the first tiebreaker of opponent win %, which includes only the win percentage of players at the event in question, rather than a more accurate sample size like the season. I don’t think I need to rip too hard at why a player’s record in a single tournament isn’t very reflective of their overall ability, but I’ll showcase some tournaments where the Swiss placement system did a poor job of placing players based on the skill they’ve shown in other events, which is usually caused by a wonky schedule of opponents.
Somewhat famously, VG Tournament Organizer AlphaZealot has said he doesn’t think players who go X-2 in Swiss events should be top cutting. I think in other types of games’ pool stages, such as in a MOBA or fighting game, not losing more than twice throughout the tournament is often a reasonable expectation for championship teams. However, the Swiss format has some qualities that pool stages in most other games do not. One big one is that Swiss ensures that there is no more than one undefeated player going into the second tier of play, unlike most other games where players or teams are separated into many more groups that can each have an undefeated player or team. Another is that those groups are usually broken up with some sort of seeding system so that the most successful players or teams don’t have to worry about knocking each other out early. Since our format forces most players to take one loss by pairing players with equal records, a tough match-up late in a tournament ensures one of two deserving players is eliminated. While some X-2s cut the European Nationals, in North America with a 16 player top cut, only 8 players cut per flight can cut. This means that if we have a tournament the same size as last year’s US Nationals tournament no more than one 6-2 per flight will cut. Even events that are cutting a decent chunk of 6-2 players, such as the Italian National tournament, have been failing to cut players who have proven that they are among the best in the world in the more reliable best-of-three format in the past because of the small top cut caps.
Milan
While Milan cut enough big name players that no one seems to care, I think it’s actually a really fantastic example of why the capped top cut is bad for the game. The 18th and 19th place finishers wound up being Fransesco Pardini and Guillermo Castilla (Kasty), who you may remember from their top 8 finishes in the 2011 and 2012 World Championships respectively. Given how well Matty (4-2 Swiss/2nd Place Worlds 2011, 3-3 Worlds 2012, 1st Place Milan) and Sekiam (3-3 Worlds 2012, lost top 8 to Matty in Milan) did in the event, I’d say that it is pretty reasonable to extrapolate that Fransesco (4-2 Worlds 2011, top 8) and Guillermo (4-2 Worlds 2012, top 8) would have at least been contenders in the top cut given their similar performances in the tougher Worlds fields in the past. If the top cut had included them, like it would have if the TCG top cut numbers were used instead of the arbitrary VGC cut-off, we might be looking at a somewhat different set of tournament results. All four of those players proved their skill in the vastly superior best-of-3 Swiss stage at Worlds, which tends to be a much better metric of ability than the highly volatile best-of-1 Swiss format National tournaments use. The best-of-1 Swiss format puts players roughly where they should be, but not even close to definitively where they belong, and being a couple spots off here cost these players the chance to continue on. A more precise mechanism like the head-to-head top cut is needed to clean up Swiss’ mistakes, which wasn’t allowed to happen here. While many of the big names did manage to make the top 16 in Milan, it shouldn’t be overlooked that the 17-32 range also included players like Ravi Vazirani (who also went 3-3 at Worlds last year), Lati (who wouldn’t have had to deal with that spirit crushing removal from the top cut due to the TO’s error if it was 32 like it should have been), Billa, and Osirus.
2012 US Nationals
It is strange to see US Nationals’ top cut being reduced given that it is a tournament that has perhaps been the flagship example of Swiss not seeding top cut very well. While 2011 Nationals is perhaps an even better example, with a field of about 60 players leading to a 14 seed (Wolfey) defeating a 16 seed (Dan) in the finals, both of whom were 5-2 in the Swiss rounds, last year has some pretty fantastic ridiculousness to add to it.
The shallower 2012 flight B, which sent some of the best players like Crow, Wolfey, sandman, and Zach forward, also managed to send some players that should not have been in the top cut like CaptKirby, who essentially had never played VGC before, but cut because he had a particularly easy Swiss schedule. That flight left some strong players on the outside who would have been included if Nationals had cut to 64, which again, it should have if we were following the rules TPCi made for TCG because we had more than 256 players. Cut players in that flight include 2013 #3 CP seed Stephen (6-2) and a former National Champion in OmegaDonut (6-2). The highest seeded 5-3 was 2013 Regional Champion cmoeller22, who had an almost 70% opponent win percentage in spite of not having bye inflation helping him out and is one of the tournament’s best examples of a slanted strength of schedule knocking out a player who deserved to continue.
The deeper, less top-heavy, and bye-filled flight A had a massive amount of cannibalism among the more established players. Without a lot of good match-up data to pull from for other players, on my own I played four people who won Regionals in 2012 or 2013, four people who top cut, and one who got top 4 in Nationals the previous year, which accounted for seven of my eight opponents with some overlap. While I think my experience was probably the most slanted, every round there were a few people who are in the top 50 CP or so this year or who won Regionals last year taking each other out. The result was a top 16 cut from that flight where, looking back at the standings, I don’t even know who a couple of those players are. I think it’s somewhat telling that very few players from those 16 are doing well in 2013 (even the known players, actually): only OneEyedWonderWeasel (14th) and Shiloh (23rd) are currently in this year’s top 32, and if you continue to the top 100, only MangoSol (86th) and Branflakes (91st) are added to the list. We were left with some bigger names getting snubbed at 6-2, with myself at 17th in the flight, Tyler, BadIntent, and HagridTwin all missing, as well as a host of 5-3 players who probably could have beaten most of the players in the top cut if they had more reasonable Swiss schedules including Regional winners JiveTime (who I played round 7 in a game that had to eliminate one of us) and Alaka, TheGr8, 2013 Regionals runner-up Calm Lava, and 2010-11 Worlds participant Metabou. It’s also worth noting neither Huy nor I managed to cut Nationals but ended up qualifying for Worlds through the best-of-three LCQ and finishing a couple more opponent wins from cutting Worlds (9th and 10th, respectively), so we must have been doing something right that year in spite of how Nationals’ Swiss stage played out.
It’s probably worth noting that if it was only 16 players cut last year, we’d have lost the following players to the cut:
- sandman, who finished 2nd at Nationals and proved that finish was no fluke by following it up with a top 4 finish at Worlds.
- TTS, who finished top 4, qualifying for Worlds from US Nationals for the third year in a row.
- theamericandream38, who ended up being some shenanigans in top 8 away from making it to Worlds himself, and who finished 19th at Worlds the year before.
Nationals 2012′s top 32 cut started with the top seed from one flight being paired up with the other flight’s 16 seed, the 2nd with the 15th, etc. Flight 1 ended up getting completely crushed in spite of starting with a much deeper roster of high-end players by a final total of 4 wins to 12 wins in the round of 32. A big part of why I think this happened (beyond people like Wolfe, sandman, Crow, and Zach being likely to have beaten anyone they played against) was the Swiss system being the Swiss system. The pairings are completely random and players don’t face schedules of opponents with even remotely the same level of difficulty, the results of which are painfully evident in flight A in 2012 Nationals. There is a reason most other competitive games use systems like round robin group stages to determine who qualifies for the second stage of events when they aren’t going to cut many players: it is to avoid situations like last year’s Nationals, where scheduling helped contribute to good players having bad records and helped some players who probably finished a bit above their ability making it into top cut. The situation with which 6-2s made it in is also a good example of the Swiss problem with Opponent Win % being based on a single event rather than the entire season, as a bunch of players playing difficult schedules whiffed even when they made it to X-2 while weaker players floated to the top, avoiding the established players as they took each other out.
2013 Virginia Regionals
The worst example from Regionals by far was the Virgina Regional, which is pretty much the most ridiculous example that I can imagine of why we need to expand top cuts. While the top 8 was still better than most Regionals’ top cuts were, as even the lesser known guys here are probably better than their reputations, it left some awfully strong players out who would have outclassed some of the top cut. I’d wager most people who clicked the link already noticed, but three of the four Americans who cut Worlds in 2012 finished in that 9th-16th range which would have been top cut spots in TCG but aren’t in VGC because of our arbitrary top cut cap. Also featured in that range is ryuzaki, who finished 9th at Worlds in 2011 and ended up making it to the finals of the next Regional she attended. While it’s almost too easy to use him as an example, Ray missing a cut anywhere is pretty crazy, but his schedule is among the toughest I can recall at a Regional: he hit three players who are currently in Worlds positions in Cybertron (5-3), Nightblade7000 (6-2), and ryuzaki (6-2), as well as some tough rounds against jio (5-3) and AP Frank (6-2). Once again, the players he faced had much lower than normal overall records because of the battling against each other, and Ray only got a 70% Opponent Win % from the five opponents I just mentioned even though three of them are currently ranked in the top 8 of the country rather than a Regional.
I think it’s pretty obvious that all four of those players deserved to be in the top cut if top cut is supposed to reflect the event’s most skilled players. To be blunt, I think it’s also pretty obvious that if we played a round of 16, the lower seeds would have taken at least half of the games, even with some great players like TDS, Nightblade7000, JiveTime, and dtrain in the cut, but the cut-off cost them the chance to prove it. For Trista and Matt, it might even have cost them Worlds invites, as both of them are sitting pretty close to the edge right now at 8th and 14th, respectively. 5th place overall in CP Cybertron was also sitting at 5-3 in that event after a tough schedule that included an early visit with reigning champion Ray Rizzo. All five of those players (with Wolfe being the unmentioned Worlds player) have proven in a variety of events that they might be the five best players in North America, not just this Regional, but the format was working against them some here.
2012 Missouri Regionals
While not nearly as bad as Virginia, St. Louis was another tournament where I noticed some of the same things happening during as what happened at US Nationals, with a lot of good players facing off in Swiss leading to some weird results. I know I personally hit both eventual winner FonicFrog and Wisconsin winner cmoeller22 in the last two rounds of Swiss. I ended up going 1-1 in these games, but only two of us could cut because of how the pairings worked out. This was a frustrating experience given that FonicFrog’s powerful round 1 top cut opponent was using the always sound TerraCott strategy with a Scarf Blastoise in the back, which collapsed in top cut about as easily as it sounds like it would. Meanwhile, Cory’s day ended without a chance to prove he deserved to finish higher because he had some bad breaks with his opponents, even though I think it’s fairly obvious he’s a better player than some of the players who finished above him… which he proved by winning his next Regional. It is also notable that this was one of the many tournaments where the 8-0 from Swiss lost in the first round of top cut to a 6-2 in two quick and decisive games, showing again that Swiss isn’t doing a very precise job of placing players. There were several other strong players like Shiloh, Smith, and Captain Falcon watching top cut instead of playing in it for a variety of factors in spite of the wonky top cut.
Pokémon isn’t a game where we have a lot of live events, so when we do have tournaments it is important that they get results that reflect the skill level of the players who competed accurately. That didn’t happen well enough at any of these events.
Why The Best Players Need To Get To The Best of Three Stage
The best-of-three stage is an essential part of getting the right results in Pokémon tournaments. It vastly reduces the impact of one game’s worth of luck or gimmickry on the results of a tournament and helps ensure the best player is winning matches. A big part of why Worlds is considered the best tournament of the year by the players isn’t just the prizes it awards or the spectacle of being an international event, it is the best-of-three Swiss format ensuring that the right players are winning most of the matches. While we understand it isn’t always viable to use best-of-three Swiss due to time, getting the top fourth of players into the best-of-three stage so they can battle it out like the TCG players do is a necessary improvement for our tournaments. Players are much less likely to pick up two losses from bad luck during two games of a three game set than during eight rounds of Swiss, and those rounds with the reduced impact of luck are what should be deciding whether or not players are eliminated, not the heavily luck-influenced Swiss stage. While players don’t face opponents of equal difficulty in the top cut stage either, at that point if you’re going to win a tournament you have to be able to beat anyone in a best-of-three anyway, and it is much easier to avoid losing a series than a single game.
As the top cut of Nationals 2012 showed, once we get to the best-of-3 stage the players who overachieved a little during the Swiss stages of the tournament get knocked out pretty quickly. However, the standings still wind up being pretty messed up if that cut is too small. After last year’s Nationals, we’d expected this year we would probably be facing an increase from a 32-player top cut to a 64-player top cut to match the TCG system for a tournament the size of US Nationals. Instead, TPCi is planning on going the other direction and dropping the top cut to 16 for some reason. Unless they plan on seeding Swiss rounds and/or using best-of-three Swiss to mitigate Swiss’ two big problems — which isn’t very feasible, presumably — a reduction in top cut participants is a pretty big mistake because the Swiss qualification stage needs much more error correction than it’s currently being given as a result.
Swiss performs a function as well as it can. It gets lots of players lots of games and gets players closer to where they belong in the standings than single elimination on its own ever did. However, the TCG is right to have a more inclusive top cut stage in a game that has as much probability as Pokémon does, especially when using a qualification system that creates as many anomalies as Swiss does. We want the best players getting to top cut so they can show what they can do, not hoping they get there through arbitrary tiebreakers, in-game luck, the Russian roulette that is Swiss pairings, or to have them miss the cut in spite of having pretty strong tournaments because the organizers have a view on what makes a good top cut that doesn’t coincide with the players’ view. The players who deserve to win will prove it in the top cut even if they have to play an extra round or two of best-of-three, but they can’t win if they aren’t allowed to play.
If you have strong opinion on this, please, post in the thread. Like this post, like the Facebook post about the article, share the Facebook post, retweet the tweet about the article, whatever. This is the sort of thing we need to get as many eyes on as we can because it is important to our community. I’d love to hear more about other people’s experiences with the top cut, too — I tried to keep my personal experience to only a couple mentions here, but I’ve had quite a few tournaments where I felt like people were getting the short end of the stick because of strange pairings or strange incidents of Pokémon occurring during Swiss and I’m sure the rest of you have, too. I hope this doesn’t come off as too hostile to TPCi, but this is a big deal to the players as anything with the game could be, as I imagine will be obvious once this has been around the web for a while. The changes this year have been pretty great for the most part but this, this is a big red mark on an otherwise improved system.
Photo Credits: Robbie M. (Biff). Check out more of his event coverage on his YouTube channel.
The post The Problem With Small Top Cuts in VGC appeared first on Nugget Bridge.