A/B Testing & Personalization
What is "Personalization"?
Personalization is a strategy in which advertisements, messages and experiences are displayed on a website to visitors with identical or similar characteristics (segments).
Client vs. Server Side Testing?
The first thing that differentiates your A/B testing requirements is client-side or server-side A/B testing. This aspect is often overlooked. Nevertheless, it should be chosen based on your needs.
- Client-side: commonly used to optimize conversion rates in marketing or funnel, for example by creating page variations directly on the users’ browser.
- Server-side: when you need to test more in depth in relation to the visual changes, such as products (features) or experience for engagement, retention and more.
What is a null hypothesis?
A/B testing is actually used to examine hypotheses. There are two typical concepts in hypothesis testing: the null hypothesis and the alternative hypothesis. Usually, the null hypothesis indicates that the performance of the two variants A and B are identical, while the alternative hypothesis states that they are not.
How many users do I need for trust-worthy testing?
The wrong interpretation of statistical significance is one of the most frequent and serious mistakes committed in A/B testing. Usually the minimum required traffic is calculated using the following key figures:
- The conversion rate of our control variation (variation A)
- Minimum difference between the conversion values of variations
- Confidence level
- Statistical “Power”
For a sample calculation please use our runtime calculator on this page.
How often should I run A/B tests?
There are different opinions regarding this matter, however we recommend continuous testing. You should have a clear goal and enough page visitors, in order to achieve statistical relevance within an acceptable period of time.
Do A/B tests have negative effects on SEO?
Many mistakenly think that A/B tests could have a negative impact on SEO. The truth is, that websites rather improve through A/B tests which results in better ranking.
1. Collect Ideas & Hypotheses
2. Prioritize Ideas & Hypotheses
3. Implement and Execute Campaigns
4. Evaluation & Analysis