Inside test results from IWT “Labs”
September 28th, 2012 - 8 Comments
What do you guys do for fun? Go out to bars? See a movie?
What about run carefully designed, iterative multi-variate tests to study human behavior?
What? Is that weird?
Check out this test data from I Will Teach You To Be Rich Labs, where we’re running dozens of tactical and theoretical tests at any given time.
I wanted to try a really simple test — two completely different designs to see which would produce more email subscriptions. So we designed two totally different pages for a recent contest I ran:
With over 20,000 data points and statistical confidence, which do you think worked better?
Yes, this is a table of data. Yes, this will remind you of college when you didn’t understand the data there, either.
ANSWER: No change!
Despite being totally different designs, the results were basically identical.
One of the surprising secrets of testing is that MOST tests produce no change. Online marketers don’t tell you this because it’s not sexy and kind of depressing. It’s like guys going out to bars, predator style, then the next morning, telling their friends last night was “CRAZY”…but neglecting to mention the fact that they didn’t really meet any girls.
In my experience running tests, once in awhile, you’ll find a small change (3%), and once in a blue moon, you’ll get a huge huge win — a total twist that changes everything. To put things in perspective, I find about 2 HUGE WINS every year….out of hundreds of tests. But they are big enough to pay for me running IWT Labs and giving you dirty jokes every couple days on my email list.
Btw, people who follow me on twitter.com/ramit will know of one that I recently wrote about that transformed my business. If you’re not following me there, and you want to see me make fun of some of the really dumb comments I get, check it out.
Anyway, back to you. I could spend all day talking about testing theory, but we both know you don’t care. Most people are like ZOMG RAMIT GIMME THE GOOD TESTS PLS~!~##
Tests That Produced Big Results
I know you’d rather hear about the Big Win tests, so I figured I’d share a few good ones with you.
- My friend Noah Kagan, founder of AppSumo, ran one simple price test that skyrocketed his conversions and revenue while simultaneously lowering his returns. (Can you guess what the test was?)
- Dating expert Nicholas Kho used testing to discover the one text message that will get 90% of VIPs (and women) to respond to you.
- Primal nutrition expert Mark Sisson used testing to change his body composition with just one 20-minute workout a week.
If you’re curious about the exact test results, I created a new ebook that includes 15 little life experiments you can use today.
It’s totally free. No upsells, no nothing. Get your copy today, my treat: http://littlelifehacks.com
P.S. If you click through to that page, you’ll also see that I’m giving away 3 iPhone 5s to kick off the launch of this ebook. Right now there aren’t a lot of participants, so if you participate you’ll have pretty good odds of winning. Here’s the link again: http://littlelifehacks.com
I decided to take a last-minute ski trip this weekend. Here’s a quick email I sent to my friends: ...Read More
Let’s say you meet a new person at a party this weekend. Isn’t it interesting how you might ...Read More