How to write a winning hypothesis for your CRO tests

What is a hypothesis?

A hypothesis is a statement that introduces a research question and proposes an expected result based on limited evidence. It is usually used as a starting point for further investigations. In any scientific field, the hypothesis should form the basis of the experiment.

How is a hypothesis used in CRO?

Writing your hypothesis is one of the most important steps in the CRO process. If done well, a good hypothesis will form the perfect foundation to build a test plan on. We would recommend producing a separate test plan for each test. Your test plan should detail what changes are part of the test and how you intend to measure the success of this test. This plan should contain as much detail as possible – including screenshots and instructions for any changes, as well as a clear hypothesis for the outcome of the test. It can also be used to keep track of any other important information relating to the test at hand, such as any background information, the type of test being conducted (e.g. AB test).

Why should I write a hypothesis?

A good hypothesis is based on more than just guesswork. Your hypothesis should highlight the impact that you expect to have. As well as measuring the outcome of your test, by writing a hypothesis you can qualify whether the outcome happened the way you expected it to. If the hypothesis logic doesn’t seem right, the chances are that your test will not work as expected. The hypothesis is the foundation of your experiment. A poorly constructed one is likely to create confusion and not get you where you want to go!

How do I structure a hypothesis?

The easiest way to start off your hypothesis is to start at the end by focusing on the business outcome that you are hoping to achieve. From there, think about the human behaviour that would be most likely to trigger that outcome. Lastly, consider the visual or functional changes that could be made to trigger that behavioural reaction.

We would recommend following the following formula when it comes to writing your hypothesis:

By [making this change] visitors will [have this/these behavioural reaction(s)] thereby [triggering this impact on conversion/revenue.

If we break this formula down, this is what it represents:

• By [making this change] - the change, representing the cause of the effect
  ‍
• Visitors will [have this/these behavioural reaction(s)] - the immediate effect of the change
  ‍
• Thereby [triggering this impact on conversion/revenue] - the business effect of the change

Try to include as much detail in your hypothesis as possible - be specific about the results that you are expecting to see!

For example:

By highlighting the free returns policy next to the Complete Order button, visitors will feel reassured that they can easily return any unwanted items with no hidden costs, thereby increasing conversion.

What if my hypothesis is proven wrong?

If your results do not match what your hypothesis predicted, don’t try to change or manipulate your hypothesis to fit. Keep an open mind. If the results are not what you expected, you can learn from them. This will enable you to conduct more effective tests in the future!

Reforming a hypothesis

Adapting and rerunning a test might be easier and take less resources than starting a brand new one. If your test does fail to produce a winner, return to your hypothesis and ask yourself why that might have been the case. By using the data you have collected, reforming your hypothesis should be a fairly simple process. Changing up your hypothesis to react to what you have learned from the previous version of the test is a lot easier than coming up with the original hypothesis as you have results and numbers to work from. You should be able to get something new out of the door much faster!