Being that I work in the eCommerce space, we talk a lot about A/B testing and it has also become a huge topic of conversation on the internet/twitter/facebook. Over the past two years, there have been more tools emerging that seem to blow away the previous competition. One tool that seems to be really pushing the boundaries is called Optimizely, which I have had the pleasure of trying out this weekend.
To give a little background on this company, it was started by Dan Siroker and Pete Koomen, both were product managers at Google. Dan Siroker eventually left Google to work on the Obama 2008 campaign as the Director of Analytics, where he worked on A/B and multivariate testing to increase signups, donations for the Obama website. The optimization was hard because of the amount of IT involvement that was required each time a campaign needed to be setup. I’ve experienced this too, you have a great idea to test – but the implementation can take 4 hours, it makes you rethink what to actually test, which shouldn’t be the case.
One more very interesting fact to mention is that these guys just received $23 million in Series A funding and they are growing at 400% year over year. This company is definitely headed somewhere interesting, excited to see what becomes of this platform. This is a company to keep a close eye on over the next year!
Creating an account on Optimizely was dead easy, I signed up for the cheapest plan ($19/month) to try some testing out on one of my websites (stamfordctguide.com). Within about 15 minutes, I had a live testing running, which to me is incredible after having experience with other platforms such as Performable and SiteSpect.
Optimizely Implementation and Peformance
|Project running 4 tests with jQuery included||Project running 4 tests without jQuery|
If you are curious about uptime and response time of the snippet, Optimizely publishes all of this information using Pingdom for the current day and also a past history. I love how open this company is about their software and performance!
This is the part of the software that I LOVE, LOVE, LOVE. There are two ways to setup experiments:
- A firebug style inspector tool that highlights each element, then thru a contextual menu on the screen you can edit, move, resize, swap, etc. This will be easy for anyone to setup experiments.
- A code editor that expects/accepts jQuery to write all of your experiments. Anytime you use the firebug-style inspector tool, it is actually writing jQuery behind-the-scenes. That’s MINT! It really puts the power of creating A/B experiments in the hands of the front-end developers and away from IT.
The product support is excellent, they have created a great knowledgebase where you can view questions on setup, configuration of testing, reports, customization and you can also ask your own public questions. It’s pretty easy to find the answer you are looking for either via searching the existing questions or by posting a public question.
Optimizely has been publishing a lot of great video content on setup, etc. through their Youtube channel. If you are looking for a good intro, please check out the video below.
I will be writing more tutorials around setting up different types of tests using jQuery, so check back often.