Wierd Results A/B Test in Google Website Optimizer

Posted by Yisroel on Pro Webmasters See other posts from Pro Webmasters or by Yisroel
Published on 2011-03-17T14:57:44Z Indexed on 2012/04/12 23:45 UTC
Read the original article Hit count: 271

Filed under:

testing

|

a-b-testing

|

google-website-optimizer

I set up a test in Google Website Optimizer that has a 3 variations - original (A), B, and C. In order to further validate the results of the test, I added a variation C that is exactly the same as the original. And thats where the results get weird.

6 days in to the test, the best performing variation is C. It outperforms the original by 18.4%! How is that possible?

Do I now discount the results of this test entirely?

© Pro Webmasters or respective owner

Related posts about testing

Automated unit testing, integration testing or acceptance testing

as seen on Programmers - Search for 'Programmers'
TDD and unit testing seems to be the big rave at the moment. But it is really that useful compared to other forms of automated testing? Intuitively I would guess that automated integration testing is way more useful than unit testing. In my experience the most bugs seems to be in the interaction… >>> More
Modifying a HTML page to fix several "bugs" add a function to next/previous on a option dropdown

as seen on Stack Overflow - Search for 'Stack Overflow'
SOF, I've got a few problems plaguing me at the moment and am wondering if anyone could assist me with them. I'm trying to get Next Class | Previous Class to act as buttons so that when Next Class is clicked it will go to the next item in the dropdown list and for previous it would go to back one… >>> More
Automation testing tool for Regression testing of desktop application

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi I am working on a desktop application which uses Infragistic grids. We need to automate the regression tests for same. QTP alone does not support this, we need to buy new plug in for same which my company is not very much interested in. Do we have any open source tool for automating regression… >>> More
Oracle Functional Testing Suite Advanced Pack for Oracle EBS Now Available

as seen on Oracle Blogs - Search for 'Oracle Blogs'
There’s new news about automated testing of E-Business Suite using the Oracle Application Testing Suite, a.k.a, “OATS”. E-Business Suite Development is pleased to announce the availability of the new Oracle Functional Testing Suite Advanced Pack for Oracle E-Business Suite. The new pack, available… >>> More
Do You Know How OUM defines the four, basic types of business system testing performed on a project? Why not test your knowledge?

as seen on Oracle Blogs - Search for 'Oracle Blogs'
Testing is perhaps the most important process in the Oracle® Unified Method (OUM). That makes it all the more important for practitioners to have a common understanding of the various types of functional testing referenced in the method, and to use the proper terminology when communicating with each… >>> More

Related posts about a-b-testing

Automated unit testing, integration testing or acceptance testing

as seen on Programmers - Search for 'Programmers'
TDD and unit testing seems to be the big rave at the moment. But it is really that useful compared to other forms of automated testing? Intuitively I would guess that automated integration testing is way more useful than unit testing. In my experience the most bugs seems to be in the interaction… >>> More
Oracle Functional Testing Suite Advanced Pack for Oracle EBS Now Available

as seen on Oracle Blogs - Search for 'Oracle Blogs'
There’s new news about automated testing of E-Business Suite using the Oracle Application Testing Suite, a.k.a, “OATS”. E-Business Suite Development is pleased to announce the availability of the new Oracle Functional Testing Suite Advanced Pack for Oracle E-Business Suite. The new pack, available… >>> More
Automation testing tool for Regression testing of desktop application

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi I am working on a desktop application which uses Infragistic grids. We need to automate the regression tests for same. QTP alone does not support this, we need to buy new plug in for same which my company is not very much interested in. Do we have any open source tool for automating regression… >>> More
Modifying a HTML page to fix several "bugs" add a function to next/previous on a option dropdown

as seen on Stack Overflow - Search for 'Stack Overflow'
SOF, I've got a few problems plaguing me at the moment and am wondering if anyone could assist me with them. I'm trying to get Next Class | Previous Class to act as buttons so that when Next Class is clicked it will go to the next item in the dropdown list and for previous it would go to back one… >>> More
.htaccess with public folder

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a directory structure with the following on localhost: http://localhost/testing/ A directory structure exists inside of testing as follows: /testing/public /testing/public/index.php /testing/public/img /testing/public/css ..etc for the js and swf directories A .htaccess file is inside… >>> More