A/B Testing & Personalization

%

%

%

A/B TESTING

What is "Personalization"?

Personalization is a strategy in which advertisements, messages and experiences are displayed on a website to visitors with identical or similar characteristics (segments).

Client vs. Server Side Testing?

The first thing that differentiates your A/B testing requirements is client-side or server-side A/B testing. This aspect is often overlooked. Nevertheless, it should be chosen based on your needs.

  • Client-side: commonly used to optimize conversion rates in marketing or funnel, for example by creating page variations directly on the users’ browser.
  • Server-side: when you need to test more in depth in relation to the visual changes, such as products (features) or experience for engagement, retention and more.
What is a null hypothesis?

A/B testing is actually used to examine hypotheses. There are two typical concepts in hypothesis testing: the null hypothesis and the alternative hypothesis. Usually, the null hypothesis indicates that the performance of the two variants A and B are identical, while the alternative hypothesis states that they are not.

How many users do I need for trust-worthy testing?
  • The wrong interpretation of statistical significance is one of the most frequent and serious mistakes committed in A/B testing. Usually the minimum required traffic is calculated using the following key figures:

    • The conversion rate of our control variation (variation A)
    • Minimum difference between the conversion values of variations
    • Confidence level
    • Statistical “Power”

    For a sample calculation please use our runtime calculator on this page.

How often should I run A/B tests?

There are different opinions regarding this matter, however we recommend continuous testing. You should have a clear goal and enough page visitors, in order to achieve statistical relevance within an acceptable period of time.

Do A/B tests have negative effects on SEO?

Many mistakenly think that A/B tests could have a negative impact on SEO. The truth is, that websites rather improve through A/B tests which results in better ranking.

Loop-Workflow

1. Collect Ideas & Hypotheses

2. Prioritize Ideas & Hypotheses

3. Implement and Execute Campaigns

4. Evaluation & Analysis