My article about testing Stoneforge Mystic generated a lot of conversation, both here on Modern Nexus and elsewhere. Judging by the reaction, people really do want some actual data and scientific method brought into discussions of the Modern banned list. Which means that I've got a lot of work to do.
But I also know that if I don't establish some guidelines, I'll have to rehash the same arguments every time. So what I'm going to do today is establish my permanent methodology going forward, and set the ground rules. I don't expect my methods will make everyone happy. I am only one man, and I have limited time and resources with which to test. Thus I have to make compromises.
It is impossible to adequately test against every deck in Modern. It is extremely impractical to test against all of Tier 1. I will be strategic about the deck I use to test a card, and the decks I play against—to yield not only the most representative results, but also the most useful results. I will never use decks where the impact of a banned card is expected to be indeterminate or mediocre. It's a waste of my time and yours to produce medium results. All decks will be built based on archetypes being played in the metagame at the time I start testing.
The Test Deck
Whenever possible, the test deck will be one that got a card banned in the first place, updated for current Modern. When that isn't possible I will use a deck that the banned card is likely to improve by a considerable amount. There will be a known list (hopefully played to reasonable finishes) for establishing my baseline win percentage, and an "improved" version of that list for the actual test. This is how I approached the testing of Stoneforge Mystic.
The reason is simple: the purpose of this exercise is to determine if known or suspected broken cards are still broken. For example, testing Ponder in Jeskai would not produce very interesting or helpful results. The deck currently plays Serum Visions, and Ponder is better than Visions. Therefore we would expect Jeskai's overall win percentage to improve. However, Jeskai is a control deck that will win slowly. Even with the consistency boost from the second-best cantrip in Magic, the change will not be that dramatic and our conclusion would be weak. Storm or Ad Nauseam, on the other hand, will benefit far more from an additional cantrip than a fair deck would. Those archetypes are a far better indicator of whether or not Ponder is safe to unban (I firmly believe it isn't, just for the record).
Think about the issue like this: Wizards doesn't ban cards just because of power level. It bans because of degeneracy or oppression. It is very hard for a truly fair deck to be either. Unless they're Jund with Bloodbraid Elf and Deathrite Shaman. The fair uses of a card are not on trial—what is, is whether they're still as unfair as before. If we want to evaluate that aspect, we must try to put the card in an unfair shell. This may not always be possible, but I will preference less fair shells over more fair ones.
My testing gauntlet will be a representative selection of decks from all ends of the spectrum. I will prefer Tier 1 decks whenever possible, though for unfair combo decks I will most likely need to dip into Tier 2. These are decks with proven power and resilience, and they will do the best job of teasing out a banned card's power. A lower-tier deck may have interesting gameplay with a banned card, but it will not indicate the actual strength of the card in the overall metagame. If we want to learn what is the best, we must play the best.
I want to see how the full spectrum from the truly fair, to the "fair" or pseudo-fair, to the unfair, will be affected. A broadly improved win percentage may indicate that the card is oppressive. A more localized impact might signal a metagame shift which may or may not be welcome. This is also the only way I can try to model the total reaction that a card will have on Modern.
When I build gauntlet decks they will all be "average" builds of the currently played decks as of the moment I start the testing. I will not try to adapt the gauntlet decks to the banned card. The answer why is twofold. First, I don't want to take the time to try and figure out how a deck would react. I'm not a normal player of any of these decks, and my perceptions may not be in line with the reality of the decks' strategic considerations. If I guess and get it wrong it will ruin the results.
The second reason follows from the first: the need to limit variables. For any experiment to be scientifically valid, you need to focus your testing on as few variables as possible. In economics, we use graphs with several curves on them to describe supply and demand. If you shift a single curve, say supply, the results are easy to quantify and predict. If you shift both curves the results become indeterminate, dependent on the magnitude of the shifts. If I start adapting the gauntlet decks, I add in uncertainty that might mask the real impact behind an improperly built test deck. Thus I will stick to known knowns.
To obtain valid results, I will use the exact same procedure for each card. Once I begin testing with one configuration I will never change that configuration, lest it invalidate my previous data. I am not so committed to this series that I'm willing to redo hundreds of datapoints because of a new methodological revelation.
To try and prevent suboptimal builds, I will goldfish and playtest the decks before testing begins. This will also help me learn the decks to reduce play mistakes affecting the results. Some will happen regardless, but it should be small enough not to impact the end result.
Every test session will begin with at least 10 practice matches so that whoever I've roped into this madness and I can get a feel for the matchup. We will then play 50 matches with the baseline deck and 50 with the test deck. For Magic Online testing misclicks will be taken into account. If they're correctable through a change in play, we will do so—otherwise the match will be thrown out. All testing sessions for a given deck will take place as close together as possible.
The Test Subjects
Now to the really critical part: what am I actually willing to test? Not every card on the banlist needs to be tested. It should be obvious why Hypergenesis, Rite of Flame, Dread Return, or Cloudpost will not and should not ever be freed. Proven broken cards are not up for consideration; realistic options only please. Of the less obvious ones, some are still not up for consideration.
No Jund Cards!
I will not be testing Bloodbraid Elf, Deathrite Shaman, or Punishing Fire. Hard no. Absolutely not. I don't care how much you whine or how good your arguments are, up to and including masters theses on the impact of Bloodbraid Elf's unbanning on global happiness. I will not test them.
Jund is a very good deck. It has been since Modern's inception. The only times it hasn't been Tier 1 were when something very odd was going on in the metagame. It has reached Tier 0 levels in the past when it had access to some of the banned cards. It doesn't need any more toys, so these overpowered cards can stay where they are.
Stop. Be honest with yourself. Will the boost to whatever deck you care to name actually be greater than the boost Jund receives? No. It won't. Jund doesn't need a boost, and no deck will benefit enough to justify that. No Jund cards.
The frustration caused by the Counter-Top lock is only one part of my concerns. It isn't the experienced Miracles players I have an issue with—it's the derps. Those fools who waste time looking at the same three cards over and over again, dragging out the game ad infinatum. Legacy tournaments always run late thanks to Top players, and it just gets worse the more inexperienced the players are. For the sake of tournament run times, Top should stay banned. This is also why I will not look at Second Sunrise. I have better things to do than watch you play with yourself.
No Twin or Pod
Splinter Twin was a good deck. I think banning it was a mistake. We know what would happen if it were unbanned. There's nothing interesting or new to learn here. Birthing Pod falls under the same category.
Cards I Will Consider
I wouldn't choose any of these cards myself, but if there's enough community interest and strong arguments are made, I'm willing to try them.
- The Artifact Lands - Many Affinity players I know say that Affinity doesn't want these anymore. The only decks I can imagine wanting them are Eggs and maybe Thopter combo. I have no interest in trying them out, but I will relent if the outcry is sufficient.
- Green Sun's Zenith - I suspect that repeated tutoring and mana acceleration is just as powerful and warping now as it was when this was axed. Still, if a really good argument is backed by overwhelming support, I'll give it a try.
- Preordain - Preordain and Ponder were banned for making combo too consistent. I suspect this will still be the case, but I am willing to consider experimenting with the weaker of the two. If it's too strong, Ponder definitely is, so consider improved Serum Visions the gatekeeper to that test.
Cards I'm Actively Interested In
This is not a declaration that I will test every single card listed here. Merely that I would do so without needing convincing. Which ones I ultimately test will be determined by time and other constraints.
- Chrome Mox - Fast mana is generally a bad idea, but many have complained about Affinity getting the only mox in Modern. I can see an argument that if everyone had access to the more balanced mox it would reduce Affinity's speed advantage. I think that the unfair uses will still be greater and it will empower combo far more than other decks, but that is pure speculation that should be evaluated.
- Dark Depths - Ghost Quarter is very much a card these days, as is Path to Exile. Blood Moon also sees extensive play, unlike back in Extended. Marit Lage may not be the terror that it once was.
- Dig Through Time - Dig was always overshadowed by Treasure Cruise and never really got an opportunity to prove itself in Modern. Splinter Twin was the best argument for banning the card, but with that combo gone it might be okay to free Dig. At the very least it deserves a trial.
- Jace, the Mind Sculptor - This is a ridiculous card, but I'm willing to believe Modern can handle it—if only to shut up everyone who argues that Nahiri, the Harbinger and Emrakul are more powerful.
- Seething Song - I freely admit that this one is speculative on my part and likely a bad idea, but it is possible that the addition of more targeted discard and better hate cards would keep Storm down sufficiently even with the Song. It's probably still really broken, but it costs enough that I am willing to give it another shot. I also like the card and enjoy Storm, so this is a convenient excuse to play them.
So there you have it: the six cards I'm actively eyeing and the three I'm willing to be convinced to try. What happens next? You!
Your Voice, Your Choice
For one week, until my next article goes up, I will be watching this article's comment section. Post which card you want me to test next and why. I will test the card that receives the most votes.
Every reader who posts a preference will have their first, and only their first, card request counted. You can double-post or make multiple requests to your heart's content—only the first card from the earliest post will be counted. Any request not on the approved list or which I specifically said I wouldn't test will be discounted. Only posts made here on Modern Nexus in the comments section of this article will be counted. I encourage you to discuss it as much as you want elsewhere, but trying to scour the internet for every forum post on the subject won't be feasible for me.
If this goes like last time, I should have results from the first test around April, 2017. Join me next week when I announce the winner and continue the Beginner's Guide.