menu

[bizblog] Email Newsletters and Split Testing

Are you a Quiet Speculation member?

If not, now is a perfect time to join up! Our powerful tools, breaking-news analysis, and exclusive Discord channel will make sure you stay up to date and ahead of the curve.

I know more than a few of our readers like to get a peek at the business behind QS.com, which is why I'm writing about split-testing our newsletter. Here's the Wikipedia definition of it:

A/B testing or split testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates.

Essentially, you select one thing you want to change about your message and leave everything else the same. Then you see which one people like more.

We use Mailchimp, an incredibly powerful and incredibly free newsletter delivery program. It meters just about everything about our newsletter mailings, be they to Insiders only or our general, free mailing list (sign up!).I can tell when people give us a few minutes of their time when they read our content, and I can find out whether Insiders are looking at our article recap links in our periodical Insider-only emails.

Among other things, Mailchimp lets me split-test single variables. This time, it was whether the headline "Make money on Grove of the Burnwillows" would be read by more people than "The biggest engine in Modern..." I used hype-y headlines on both - I like an email to actually tell the reader in the subject line what it's about, instead of being something like "QS Free Newsletter, [Date]." Keep reading to find out which one was better! I think next time I'll try testing a boring headline with a hyped headline and seeing which one our readers prefer. That is, clearly, another thing you can test with A/B tests.

You may have received the email yesterday, or you may have gotten it this morning. I wasn't leaving you out! What I did was schedule Mailchimp to send the "winning" email to the full list. It selected about a quarter of the list and sent it to them. I then set it to send the email that was opened more to the rest of the list. Why did it take a day? With usual newsletters, timeliness isn't that important, so the default is a day. I think I'll explore setting it to half a day or a matter of hours. It's clear from Twitter that people were interested in getting it and discussing it. It should have sent to the full list yesterday afternoon.

I personally like writing our newsletter because I love talking with fans, but it's also an opportunity to show people what our Insider service is about. I detest getting on newsletter lists that just send out ugly pitches and don't entertain at all, so I don't want to inflict that on anyone. With split testing, I can send the most efficient messages to our subscribers and hopefully, encourage some of them to sign up.

It turns out that "Make money on Grove of the Burnwillows" beat the other headline by 3%, which isn't that significant. The other headline, though, had at least 5% more clicks on the internal links (my Survival article and the Insider signup page), so there may be something to that.

I am still finding my legs on the subject of what people like to read about in newsletters. Expect more unlocked Insider articles and format commentary, along with occasional non-sequitors (I just discovered this history documentary I love is online!) or things like decklists that are heating up (wait till you see my Modern Gifts deck).

I'll say one last thing on split testing: we will never split test our up-to-the-minute Insider email alerts. The overwhelming value of those is their timeliness, and I'd never do something to delay them.

Douglas Linn

Doug Linn has been playing Magic since 1996 and has had a keen interest in Legacy and Modern. By keeping up closely with emerging trends in the field, Doug is able to predict what cards to buy and when to sell them for a substantial profit. Since the Eternal market follows a routine boom-bust cycle, the time to buy and sell short-term speculative investments is often a narrow window. Because Eternal cards often spike in value once people know why they are good, it is essential for a trader to be connected to the format to get great buys before anyone else. Outside of Magic, Doug is an attorney in the state of Ohio.  Doug is a founding member of Quiet Speculation, and brings with him a tremendous amount of business savvy.

View More By Douglas Linn

Posted in Free

Have you joined the Quiet Speculation Discord?

If you haven't, you're leaving value on the table! Join our community of experts, enthusiasts, entertainers, and educators and enjoy exclusive podcasts, questions asked and answered, trades, sales, and everything else Discord has to offer.

Want to create content with Quiet Speculation?

All you need to succeed is a passion for Magic: The Gathering, and the ability to write coherently. Share your knowledge of MTG and how you leverage it to win games, get value from your cards – or even turn a profit.

Join the conversation

Want Prices?

Browse thousands of prices with the first and most comprehensive MTG Finance tool around.


Trader Tools lists both buylist and retail prices for every MTG card, going back a decade.

Quiet Speculation