[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"cms-page:\u002Fblog\u002Funderstanding-the-workings-of-bayesian-ab-testing":3},{"date":4,"date_gmt":4,"content":5,"sticky":7,"format":8,"categories":9,"collections":12,"coauthors":13,"class_list":15,"acf":25,"head_links":454,"breadcrumbs":455,"reading_minutes":182,"category":463,"head":466},"2024-12-19T17:20:33",{"rendered":6},"\n\u003Cp>In digital marketing,\u003Ca rel=\"noreferrer noopener\" href=\"\u002Fdigital-experience\u002Fcro\u002Fab-testing\u002F\"> A\u002FB testing\u003C\u002Fa> is a technique for comparing two versions of a variable, such as a landing page layout or ad creative, to determine which version performs best with the target audience. The audience is randomly divided into two groups: one group is exposed to version A (the control), and the other to version B (the variant). Key performance indicators (KPI’s), such as conversions, are then tracked to evaluate each version’s performance.\u003C\u002Fp>\n\n\n\n\u003Cp>However, simply observing which version has a higher number of conversions doesn’t guarantee that the observed difference is due to the change itself. Random fluctuations or sampling variability can cause apparent differences that may not be meaningful. So, how can we be confident that any observed difference reflects a true effect rather than just chance? This is where statistics and \u003Ca rel=\"noreferrer noopener\" href=\"\u002Fdigital-experience\u002Fcro\u002F\">data science\u003C\u002Fa> come into play to help quantify the uncertainty around our results and provide a rigorous basis for decision-making.\u003C\u002Fp>\n\n\n\n\u003Cdiv class=\"table-of-contents\">\u003Cul>\u003Cli class=\"heading-h2\">\u003Ca href=\"#the-bayesian-approach-to-a-b-testing\">The Bayesian approach to A\u002FB testing\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#frequentist-approach\">Frequentist approach\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#bayesian-approach\">Bayesian approach\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h2\">\u003Ca href=\"#bayesian-a-b-testing-implementation-in-cro\">Bayesian A\u002FB Testing Implementation in CRO\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#define-your-hypothesis\">Define your hypothesis\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#collect-and-interpret-your-data\">Collect and Interpret your data\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#choose-your-prior-conversion-rates\">Choose your prior conversion rates\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#calculate-the-posterior-conversion-rates\">Calculate the posterior conversion rates\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#calculate-the-probability-of-superiority\">Calculate the probability of superiority\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#calculate-the-conversion-rate-uplift\">Calculate the conversion rate uplift\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#interpret-your-results\">Interpret your results\u003C\u002Fa>\u003C\u002Fli>\u003Cli class=\"heading-h3\">\u003Ca href=\"#take-action\">Take action\u003C\u002Fa>\u003C\u002Fli>\u003C\u002Ful>\u003C\u002Fdiv>\n\n\n\n\u003Ch2 class=\"wp-block-heading\" id=\"the-bayesian-approach-to-a-b-testing\">The Bayesian approach to A\u002FB testing\u003C\u002Fh2>\n\n\n\n\u003Cp>Statistics provides a framework for quantifying uncertainty, allowing us to make informed decisions even in the presence of incomplete information. This uncertainty is quantified through probability, giving us a way to understand the \u003Cstrong>likelihood\u003C\u002Fstrong> of different outcomes based on available data.\u003C\u002Fp>\n\n\n\n\u003Cp>In \u003Ca rel=\"noreferrer noopener\" href=\"\u002Fblog\u002Fbayesian-statistics-in-media-mix-modelling-a-real-world-analogy\u002F\">Bayesian statistics\u003C\u002Fa>, probability is viewed as a measure of certainty in a particular outcome and \u003Cstrong>prior\u003C\u002Fstrong> knowledge or beliefs are incorporated into the analysis. In Bayesian A\u002FB testing, the process begins with a prior belief (or distribution) about the effectiveness of each version (A and B), which might be informed by historical data, expert opinion, or previous experiments. As new data from the A\u002FB test is collected, the Bayesian approach updates this prior belief using Bayes’ theorem, resulting in a \u003Cstrong>posterior\u003C\u002Fstrong> probability distribution that reflects both the prior knowledge and the new evidence. This approach is particularly useful because it allows you to continuously refine your estimates as more data becomes available, leading to probability-based confidence in which version is likely to perform better.\u003C\u002Fp>\n\n\n\n\u003Cp>In A\u002FB testing, both Bayesian and Frequentist approaches serve distinct purposes:\u003C\u002Fp>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"frequentist-approach\">Frequentist approach\u003C\u002Fh3>\n\n\n\n\u003Cp>This approach is often valued for its objectivity and clear decision criteria, such as achieving \u003Ca rel=\"noreferrer noopener\" href=\"\u002Fblog\u002Fstatistical-significance-ppc\u002F\">statistical significance\u003C\u002Fa> at a specific \u003Ca rel=\"noreferrer noopener\" href=\"\u002Fblog\u002Fdigital-marketing-glossary\u002F#digital-marketing-terms-p-t\">p-value\u003C\u002Fa> threshold. However, it requires a fixed sample size and doesn’t directly tell us the probability that one version is better than the other. It only indicates whether the observed difference is likely due to chance. \u003C\u002Fp>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"bayesian-approach\">Bayesian approach\u003C\u002Fh3>\n\n\n\n\u003Cp>This approach, by contrast, provides a direct probability that one version is better than the other, allowing for a more flexible interpretation of the results and enabling dynamic decision-making as data accumulates. This flexibility is particularly advantageous in digital marketing, where strategies often need to be adjusted in real-time based on rapidly changing user behaviours and market conditions.\u003C\u002Fp>\n\n\n\n\u003Ch2 class=\"wp-block-heading\" id=\"bayesian-a-b-testing-implementation-in-cro\">Bayesian A\u002FB Testing Implementation in CRO\u003C\u002Fh2>\n\n\n\n\u003Cp>The remainder of this blog will walk you through how to implement your own Bayesian A\u002FB test. While there are many \u003Ca rel=\"noreferrer noopener\" href=\"\u002Fblog\u002Fthe-best-a-b-testing-tools\u002F\">A\u002FB testing tools\u003C\u002Fa> available that can quickly provide probabilities, uplift estimates, and other key metrics, understanding the internal workings behind these tools is crucial. Knowing how concepts like posterior probabilities, credible intervals, and Bayesian inference work enables you to interpret results more effectively and make smarter decisions. It also helps you recognize the limitations of automated tools, ensuring that your strategies are based on sound analysis rather than blind trust in numbers.\u003C\u002Fp>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"define-your-hypothesis\">Define your hypothesis\u003C\u002Fh3>\n\n\n\n\u003Cp>The first step in any A\u002FB test is to define your \u003Ca rel=\"noreferrer noopener\" href=\"\u002Fblog\u002Fwhere-to-start-with-a-testing-hypothesis\u002F\">hypothesis\u003C\u002Fa>, this is an experiment after all.\u003C\u002Fp>\n\n\n\n\u003Cp>In the context of \u003Ca rel=\"noreferrer noopener\" href=\"\u002Fdigital-experience\u002Fcro\u002F\">conversion rate optimisation\u003C\u002Fa> (CRO), the \u003Cstrong>null hypothesis (H\u003C\u002Fstrong>\u003Cstrong>\u003Csub>0\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong>)\u003C\u002Fstrong> and the \u003Cstrong>alternative hypothesis (H₁)\u003C\u002Fstrong> are defined as follows:\u003C\u002Fp>\n\n\n\n\u003Cul class=\"wp-block-list\">\n\u003Cli>\u003Cstrong>Null Hypothesis (H₀):\u003C\u002Fstrong> There is no difference in the performance between version A (the control) and version B (the variant). Any observed difference in conversion rates is purely due to random chance. Mathematically, this is expressed as:\u003C\u002Fli>\n\u003C\u002Ful>\n\n\n\n\u003Cp class=\"has-text-align-center\">\u003Cstrong>H\u003C\u002Fstrong>\u003Cstrong>\u003Csub>0\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong>:\u003C\u002Fstrong> \u003Csub> \u003C\u002Fsub>CR\u003Csub>A\u003C\u002Fsub> = CR\u003Csub>B\u003C\u002Fsub>\u003C\u002Fp>\n\n\n\n\u003Cul class=\"wp-block-list\">\n\u003Cli>\u003Cstrong>Alternative Hypothesis (H₁):\u003C\u002Fstrong> There is a statistically significant difference in performance between version A and version B. This means that any observed difference is likely due to the actual effect of the changes made in version B, rather than random chance. This can be one-sided (suggesting that one version performs better than the other) or two-sided (suggesting that the two versions perform differently, without specifying which is better). A two-sided alternative hypothesis can be mathematically expressed as:\u003C\u002Fli>\n\u003C\u002Ful>\n\n\n\n\u003Cp class=\"has-text-align-center\">\u003Cstrong>H\u003Csub>1\u003C\u002Fsub>:\u003C\u002Fstrong> CR\u003Csub>A\u003C\u002Fsub> ≠ CR\u003Csub>B\u003C\u002Fsub>\u003C\u002Fp>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"collect-and-interpret-your-data\">Collect and Interpret your data\u003C\u002Fh3>\n\n\n\n\u003Cp>Randomly assign users from the target audience to one of groups A or B and collect the relevant conversion data. Your dataset should look something like the following:\u003C\u002Fp>\n\n\n\n\u003Cfigure class=\"wp-block-table\">\u003Ctable>\u003Ctbody>\u003Ctr>\u003Ctd>\u003C\u002Ftd>\u003Ctd>\u003Cstrong>Sample Size\u003C\u002Fstrong>\u003C\u002Ftd>\u003Ctd>\u003Cstrong>Number of Conversions\u003C\u002Fstrong>\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>\u003Cstrong>Group A\u003C\u002Fstrong>\u003C\u002Ftd>\u003Ctd>2100\u003C\u002Ftd>\u003Ctd>58\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>\u003Cstrong>Group B\u003C\u002Fstrong>\u003C\u002Ftd>\u003Ctd>2155\u003C\u002Ftd>\u003Ctd>79\u003C\u002Ftd>\u003C\u002Ftr>\u003C\u002Ftbody>\u003C\u002Ftable>\u003C\u002Ffigure>\n\n\n\n\u003Cp>To interpret our data, we need to think about the underlying likelihood of our data.\u003C\u002Fp>\n\n\n\n\u003Cp>We know that a random user from group A will convert with an unknown conversion rate CR\u003Csub>A\u003C\u002Fsub>. As this outcome is binary (convert = 1 or not convert  = 0), the event that a random user (\u003Cstrong>X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>1A\u003C\u002Fsub>\u003C\u002Fstrong>) in group A will convert, given the conversion rate CR\u003Csub>A\u003C\u002Fsub>, can be described using a \u003Ca rel=\"noreferrer noopener\" href=\"https:\u002F\u002Fmathworld.wolfram.com\u002FBernoulliDistribution.html\">Bernoulli distribution\u003C\u002Fa>:\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">\u003Cstrong>X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>1A\u003C\u002Fsub>\u003C\u002Fstrong> \u003Cstrong>| \u003C\u002Fstrong>CR\u003Csub>A\u003C\u002Fsub> ~ \u003Cstrong>Bernoulli\u003C\u002Fstrong>(CR\u003Csub>A\u003C\u002Fsub>)\u003C\u002Fp>\n\n\n\n\u003Cp>Given we have N\u003Csub>A\u003C\u002Fsub> users in group A, the \u003Cstrong>likelihood\u003C\u002Fstrong> of our data for group A is given by the sum of N\u003Csub>A\u003C\u002Fsub> Bernoulli trials.\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">(\u003Cstrong>X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>1A\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> \u003C\u002Fstrong>+\u003Cstrong> X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>2A\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> \u003C\u002Fstrong>+ … +\u003Cstrong> X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>NA\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> | \u003C\u002Fstrong>CR\u003Csub>A\u003C\u002Fsub>)\u003C\u002Fp>\n\n\n\n\u003Cp>Which follows the \u003Ca rel=\"noreferrer noopener\" href=\"https:\u002F\u002Fmathworld.wolfram.com\u002FBinomialDistribution.html\">Binomial distribution\u003C\u002Fa>:\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">(\u003Cstrong>X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>1A\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> \u003C\u002Fstrong>+\u003Cstrong> X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>2A\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> \u003C\u002Fstrong>+ … +\u003Cstrong> X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>NA\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> | \u003C\u002Fstrong>CR\u003Csub>A\u003C\u002Fsub>) ~ \u003Cstrong>Binomial\u003C\u002Fstrong>(N\u003Csub>A\u003C\u002Fsub>,CR\u003Csub>A\u003C\u002Fsub>)\u003C\u002Fp>\n\n\n\n\u003Cp>Through similar analysis, the \u003Cstrong>likelihood\u003C\u002Fstrong> of our data for group B is given by:\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">(\u003Cstrong>X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>1B\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> \u003C\u002Fstrong>+\u003Cstrong> X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>2B\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> \u003C\u002Fstrong>+ … +\u003Cstrong> X\u003C\u002Fstrong>\u003Cstrong>\u003Csub>NB\u003C\u002Fsub>\u003C\u002Fstrong>\u003Cstrong> | \u003C\u002Fstrong>CR\u003Csub>B\u003C\u002Fsub>) ~ \u003Cstrong>Binomial\u003C\u002Fstrong>(N\u003Csub>B\u003C\u002Fsub>,CR\u003Csub>B\u003C\u002Fsub>)\u003C\u002Fp>\n\n\n\n\u003Cp>Now we’ve collected our data and determined the likelihood distributions of both groups follow a binomial distribution, the next step is to choose your prior.\u003C\u002Fp>\n\n\n\n\u003Cfigure class=\"wp-block-image size-full\">\u003Cimg loading=\"lazy\" decoding=\"async\" width=\"1211\" height=\"611\" src=\"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fbinomial-likelihood-distribution.png?auto=compress%2Cformat&q=90\" alt=\"\" class=\"wp-image-31404\"\u002F>\u003C\u002Ffigure>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"choose-your-prior-conversion-rates\">Choose your prior conversion rates\u003C\u002Fh3>\n\n\n\n\u003Cp>A \u003Cstrong>non-informative prior\u003C\u002Fstrong> is used when we have little to no prior knowledge about the likely outcomes. It ensures the data will primarily drive the results rather than any preconceived assumptions. An \u003Cstrong>informative prior\u003C\u002Fstrong>, on the other hand, incorporates existing knowledge or beliefs about the expected outcome based on past experiments, domain expertise, or historical data. Informative priors can risk introducing unwanted bias if the prior is inaccurate. \u003C\u002Fp>\n\n\n\n\u003Cp>In the case where we have very little (if any) knowledge of expected outcomes, a commonly used non-informative prior is a \u003Cstrong>Beta\u003C\u002Fstrong>(1,1) distribution, a special case of the \u003Ca rel=\"noreferrer noopener\" href=\"https:\u002F\u002Fmathworld.wolfram.com\u002FBetaDistribution.html\">Beta distribution\u003C\u002Fa>.\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">CR\u003Csub>A\u003C\u002Fsub> ~ \u003Cstrong>Beta\u003C\u002Fstrong>(1,1)\u003C\u002Fp>\n\n\n\n\u003Cp>This prior assumes the true conversion rate exists anywhere between 0 and 1, with equal probability.\u003C\u002Fp>\n\n\n\n\u003Cfigure class=\"wp-block-image size-full\">\u003Cimg loading=\"lazy\" decoding=\"async\" width=\"1211\" height=\"611\" src=\"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fbeta-distribution.png?auto=compress%2Cformat&q=90\" alt=\"\" class=\"wp-image-31407\"\u002F>\u003C\u002Ffigure>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"calculate-the-posterior-conversion-rates\">Calculate the posterior conversion rates\u003C\u002Fh3>\n\n\n\n\u003Cp>In Bayesian statistics, calculating the posterior distribution is essential to updating beliefs about a hypothesis given observed data, and this is achieved using Bayes’ theorem. Here, Bayes’ theorem combines prior knowledge (the prior distribution) with new data (through the likelihood) to compute the posterior distribution, which represents an updated probability of the hypothesis.\u003C\u002Fp>\n\n\n\n\u003Cp>For the full computation of the posterior, we need the prior, the likelihood, and the marginal probability of the observed data. The marginal probability (also called the evidence) is typically challenging to calculate directly since it involves integrating all possible values of the parameter, which can be computationally expensive or infeasible in complex models such as \u003Ca rel=\"noreferrer noopener\" href=\"\u002Fblog\u002Fmedia-mix-modelling\u002F\">Marketing Mix Models (MMM)\u003C\u002Fa>. Therefore, in many cases, we rely on advanced sampling techniques, such as Markov Chain Monte Carlo (MCMC), to approximate the posterior.\u003C\u002Fp>\n\n\n\n\u003Cfigure class=\"wp-block-image size-full\">\u003Cimg loading=\"lazy\" decoding=\"async\" width=\"834\" height=\"524\" src=\"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fmarkov-chain-monte-carlo.png?auto=compress%2Cformat&q=90\" alt=\"\" class=\"wp-image-31406\"\u002F>\u003C\u002Ffigure>\n\n\n\n\u003Cp>However, in Bayesian A\u002FB testing, if we use a non-informative prior, such as a \u003Cstrong>Beta\u003C\u002Fstrong>(1,1) distribution, our computation simplifies significantly. This prior (and any Beta prior) is \u003Cstrong>conjugate\u003C\u002Fstrong> to the binomial likelihood, meaning that when combined with binomial data, it produces a posterior that is also in the \u003Cstrong>Beta distribution family\u003C\u002Fstrong>. Conjugate priors are particularly useful in Bayesian analysis because they allow for an exact analytical solution of the posterior without the need for sampling methods. \u003C\u002Fp>\n\n\n\n\u003Cp>By choosing a Beta prior with parameters that are either non-informative or reflective of prior knowledge, we can obtain an updated Beta posterior that directly incorporates our A\u002FB testing results, making the Bayesian analysis more straightforward and computationally efficient.\u003C\u002Fp>\n\n\n\n\u003Cp>After working through the Mathematics, we obtain the following posterior distribution for CR\u003Csub>A\u003C\u002Fsub>:\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">(CR\u003Csub>A\u003C\u002Fsub> | \u003Cstrong>X\u003Csub>1A\u003C\u002Fsub> \u003C\u002Fstrong>+\u003Cstrong> X\u003Csub>2A\u003C\u002Fsub> \u003C\u002Fstrong>+ … +\u003Cstrong> X\u003Csub>NA\u003C\u002Fsub>\u003C\u002Fstrong>) ~\u003Cstrong> Beta\u003C\u002Fstrong>(\u003Cstrong>Σx\u003Csub>A\u003C\u002Fsub>\u003C\u002Fstrong>+1, N\u003Csub>A\u003C\u002Fsub>–\u003Cstrong>Σ\u003C\u002Fstrong>x\u003Cstrong>\u003Csub>A\u003C\u002Fsub>\u003C\u002Fstrong>+1)\u003C\u002Fp>\n\n\n\n\u003Cp>Where\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">\u003Cstrong>Σx\u003Csub>A\u003C\u002Fsub> \u003C\u002Fstrong>is the total number of observed conversions in group A\u003C\u002Fp>\n\n\n\n\u003Cp>Through similar analysis, the \u003Cstrong>posterior\u003C\u002Fstrong> distribution for CR\u003Csub>B\u003C\u002Fsub> is given by:\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">(CR\u003Csub>B\u003C\u002Fsub> | \u003Cstrong>X\u003Csub>1B\u003C\u002Fsub> \u003C\u002Fstrong>+\u003Cstrong> X\u003Csub>2B\u003C\u002Fsub> \u003C\u002Fstrong>+ … +\u003Cstrong> X\u003Csub>NB\u003C\u002Fsub>\u003C\u002Fstrong>) ~\u003Cstrong> Beta\u003C\u002Fstrong>(\u003Cstrong>Σx\u003Csub>B\u003C\u002Fsub>\u003C\u002Fstrong>+1, N\u003Csub>B\u003C\u002Fsub>–\u003Cstrong>Σx\u003Csub>B\u003C\u002Fsub>\u003C\u002Fstrong>+1)\u003C\u002Fp>\n\n\n\n\u003Cp>Where\u003C\u002Fp>\n\n\n\n\u003Cp class=\"has-text-align-center\">\u003Cstrong>Σx\u003Csub>B\u003C\u002Fsub> \u003C\u002Fstrong>is the total number of observed conversions in group B\u003C\u002Fp>\n\n\n\n\u003Cfigure class=\"wp-block-image size-full\">\u003Cimg loading=\"lazy\" decoding=\"async\" width=\"1010\" height=\"554\" src=\"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fposterior-conversion-rate-for-versions-a-b.png?auto=compress%2Cformat&q=90\" alt=\"\" class=\"wp-image-31409\"\u002F>\u003C\u002Ffigure>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"calculate-the-probability-of-superiority\">Calculate the probability of superiority\u003C\u002Fh3>\n\n\n\n\u003Cp>The \u003Cstrong>probability of superiority\u003C\u002Fstrong> quantifies the likelihood that one group (B) performs better than another (A). It is calculated by taking the difference between the posterior distributions of both versions and then determining the proportion of the resulting distribution that is greater than zero. This area under the density curve where the difference is positive represents the confidence that variant B is superior to control A.\u003C\u002Fp>\n\n\n\n\u003Cp>For the visual below, the probability superiority is 95.2%.\u003C\u002Fp>\n\n\n\n\u003Cfigure class=\"wp-block-image size-full\">\u003Cimg loading=\"lazy\" decoding=\"async\" width=\"1001\" height=\"553\" src=\"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fprobability-of-superiority.png?auto=compress%2Cformat&q=90\" alt=\"\" class=\"wp-image-31408\"\u002F>\u003C\u002Ffigure>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"calculate-the-conversion-rate-uplift\">Calculate the conversion rate uplift\u003C\u002Fh3>\n\n\n\n\u003Cp>The \u003Cstrong>relative conversion rate uplift\u003C\u002Fstrong> is given by the following:\u003C\u002Fp>\n\n\n\n\u003Cp>CR\u003Csub>B\u003C\u002Fsub> – CR\u003Csub>A\u003C\u002Fsub> \u002F CR\u003Csub>A\u003C\u002Fsub>\u003C\u002Fp>\n\n\n\n\u003Cp>As we have posterior distributions for both CR\u003Csub>A\u003C\u002Fsub> and CR\u003Csub>B\u003C\u002Fsub>, we can now estimate the posterior  relative conversion rate uplift. This is highly beneficial because the posterior provides \u003Cstrong>Highest Density Intervals (HDIs)\u003C\u002Fstrong> for the relative conversion rate uplift, offering a clear range within which the true uplift is most likely to fall. Unlike traditional methods that give a binary “significant or not” result, HDIs enable you to quantify uncertainty and understand the potential variability in uplift.\u003C\u002Fp>\n\n\n\n\u003Cfigure class=\"wp-block-image size-full\">\u003Cimg loading=\"lazy\" decoding=\"async\" width=\"1015\" height=\"553\" src=\"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fposterior-relative-conversion-rate-uplift.png?auto=compress%2Cformat&q=90\" alt=\"\" class=\"wp-image-31405\"\u002F>\u003C\u002Ffigure>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"interpret-your-results\">Interpret your results\u003C\u002Fh3>\n\n\n\n\u003Cp>The \u003Cstrong>probability of superiority\u003C\u002Fstrong> gives you the probability that one version is better than the other. In this case, variant B has a 95.2% probability of being superior to control A, meaning we have evidence to reject the null hypothesis and can be confident that version B is likely the better choice.\u003C\u002Fp>\n\n\n\n\u003Cp>The \u003Cstrong>posterior conversion rates\u003C\u002Fstrong> can provide valuable insights too. You can use them to see the range of possible true conversion rates for each version. This range reflects uncertainty, so a wider interval means your data might not provide a clear answer, while a narrow interval boosts confidence. In this case, both distributions appear to have similar dispersion, but variant B is slightly wider. Although both are very similar in terms of uncertainty, we’re slightly less confident in the true conversion rate of variant B than control A.\u003C\u002Fp>\n\n\n\n\u003Cp>Finally, the \u003Cstrong>posterior conversion rate uplift\u003C\u002Fstrong> shows how much better one version might perform, along with credible intervals that capture its uncertainty. In this case, the mean conversion rate uplift is 34.3%, meaning we estimate variant B will generate 34.3% more conversions on average than variant A. By analysing the credible interval, we can infer that variant B will generate between -4.56% and 83.68% more conversions with variant A with 95% probability. As this interval is very wide, we must take extreme caution when implementing variant B as standard.\u003C\u002Fp>\n\n\n\n\u003Cp>When interpreting the results of your Bayesian A\u002FB test, focus on the story your data is telling. Instead of just focusing on the average values, pay attention to credible intervals – they help you weigh the potential benefits of a change against the risks of uncertainty. By understanding both the insights and limitations of your results, you’ll make smarter, more confident decisions for your business.\u003C\u002Fp>\n\n\n\n\u003Ch3 class=\"wp-block-heading\" id=\"take-action\">Take action\u003C\u002Fh3>\n\n\n\n\u003Cp>With the results of your Bayesian A\u002FB test in hand, it’s time to take action. The analysis shows that variant B has a 95.2% probability of being superior to control A, with an estimated average uplift of 34.3% in conversion rate. This strong evidence suggests implementing variant B as the new standard is a smart choice. Use the insights from the credible intervals (-4.56% to 83.68%) to set realistic expectations for performance improvements, account for the potential risks and plan your strategy accordingly. \u003C\u002Fp>\n\n\n\n\u003Cp>Beyond simply adopting variant B, leverage what you’ve learned to iterate further by identifying the elements of variant B that drove the uplift and test additional refinements. Bayesian A\u002FB testing isn’t just about finding “winners”, it’s about continuously learning and improving to drive sustainable growth for your business.\u003C\u002Fp>\n\n\n\n\u003Cfigure class=\"wp-block-image size-full\">\u003Ca rel=\"noreferrer noopener\" href=\"\u002Fmedia-solutions\u002F\">\u003Cimg loading=\"lazy\" decoding=\"async\" width=\"3840\" height=\"540\" src=\"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F04\u002Ff0sKYLsk-Measurement_Banner_01.jpg?auto=compress%2Cformat&q=90\" alt=\"\" class=\"wp-image-27925\"\u002F>\u003C\u002Fa>\u003C\u002Ffigure>\n",false,"standard",[10,11],280,39,[],[14],120,[16,17,18,19,20,21,22,23,24],"post-31394","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-engineering","category-digital-marketing",{"page_content_modules":26,"hero":178},[27,135],{"acf_fc_layout":28,"reduced":29,"top_overlap":-1,"bottom_underlap":-1,"disable_observer":29,"title":30,"content":6,"authors":31},"blog_post_content",true,"Understanding the workings of Bayesian A\u002FB testing",[32],{"id":33,"path":34,"display_name":35,"job_title":36,"description":37,"avatar_url":38,"acf":39},29,"\u002Fabout\u002Four-team\u002Fharry-brace\u002F","Harry Brace","Director, Head of Solutions","I'm the Director, Head of Solutions at Impression. I spend my time debugging and implementing tracking using Google Tag Manager, managing data warehousing platforms and building Looker Studio reports!","https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress&fit=crop&fm=jpg&h=150&q=80&w=150&bg=CFE6F9",{"full_bio":40,"job_title":36,"specialist_service_links":41,"photo":79},"\u003Cp>Director, Head of Solutions. Joined Impression in November 2017.\u003C\u002Fp>\n\u003Cp>I gained 4 years of experience as a web developer before moving to Impression as a digital account manager in 2017. I moved across to the Analytics department in 2019 and I’ve been strengthening and growing our Analytics offering ever since.\u003C\u002Fp>\n\u003Cp>Day to day, I spend my time debugging and implementing tracking using Google Tag Manager, managing data warehousing platforms and building Looker Studio reports!\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Something that I’m proud of:\u003C\u002Fstrong>\u003Cbr \u002F>\nBuilding the analytics department over the last couple of years with the help of Aaron and now Alex H.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Outside of work:\u003C\u002Fstrong>\u003Cbr \u002F>\nFind me socialising, skiing or gaming.\u003C\u002Fp>\n",[42,60,70],{"page":43,"link_text":59},{"ID":44,"post_author":45,"post_date":46,"post_date_gmt":46,"post_content":47,"post_title":48,"post_excerpt":47,"post_status":49,"comment_status":50,"ping_status":50,"post_password":47,"post_name":51,"to_ping":47,"pinged":47,"post_modified":52,"post_modified_gmt":52,"post_content_filtered":47,"post_parent":53,"guid":54,"menu_order":53,"post_type":55,"post_mime_type":47,"comment_count":56,"filter":57,"url_path":58},374,"98","2021-01-11 16:25:32","","Analytics","publish","closed","analytics","2026-09-28 11:36:16",0,"\u002F?page_id=374","page","0","raw","\u002Fanalytics\u002F","Web Analytics",{"page":61,"link_text":64},{"ID":62,"post_author":45,"post_date":63,"post_date_gmt":63,"post_content":47,"post_title":64,"post_excerpt":47,"post_status":49,"comment_status":50,"ping_status":50,"post_password":47,"post_name":65,"to_ping":47,"pinged":47,"post_modified":66,"post_modified_gmt":66,"post_content_filtered":47,"post_parent":67,"guid":68,"menu_order":53,"post_type":55,"post_mime_type":47,"comment_count":56,"filter":57,"url_path":69},17916,"2022-06-20 08:56:55","Tag Management","tag-management","2026-09-28 11:05:27",17552,"\u002F?page_id=17916","\u002Fmedia-solutions\u002Ftag-management\u002F",{"page":71,"link_text":74},{"ID":72,"post_author":45,"post_date":73,"post_date_gmt":73,"post_content":47,"post_title":74,"post_excerpt":47,"post_status":49,"comment_status":50,"ping_status":50,"post_password":47,"post_name":75,"to_ping":47,"pinged":47,"post_modified":76,"post_modified_gmt":76,"post_content_filtered":47,"post_parent":67,"guid":77,"menu_order":53,"post_type":55,"post_mime_type":47,"comment_count":56,"filter":57,"url_path":78},17140,"2022-05-27 14:22:19","Google Marketing Platform","google-marketing-platform","2026-09-25 10:43:13","\u002F?page_id=17140","\u002Fmedia-solutions\u002Fgoogle-marketing-platform\u002F",{"ID":80,"id":80,"title":81,"filename":82,"filesize":83,"url":84,"link":85,"alt":47,"author":86,"description":47,"caption":47,"name":87,"status":88,"uploaded_to":53,"date":89,"modified":89,"menu_order":53,"mime_type":90,"type":91,"subtype":92,"icon":93,"width":94,"height":95,"sizes":96},25135,"Harry-Brace2","Harry-Brace2.png",306919,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&q=90","\u002Fharry-brace2\u002F","12","harry-brace2","inherit","2023-10-03 16:03:03","image\u002Fpng","image","png","\u002Fwp\u002Fwp-includes\u002Fimages\u002Fmedia\u002Fdefault.png",696,600,{"thumbnail":97,"thumbnail-width":98,"thumbnail-height":98,"medium":99,"medium-width":100,"medium-height":101,"medium_large":102,"medium_large-width":103,"medium_large-height":104,"large":105,"large-width":106,"large-height":107,"1536x1536":108,"1536x1536-width":109,"1536x1536-height":110,"2048x2048":111,"2048x2048-width":112,"2048x2048-height":113,"post-thumbnail":114,"post-thumbnail-width":115,"post-thumbnail-height":116,"12_col":117,"12_col-width":118,"12_col-height":119,"default_landscape":120,"default_landscape-width":121,"default_landscape-height":95,"default_portrait":122,"default_portrait-width":95,"default_portrait-height":121,"6_col_and_margin_short":123,"6_col_and_margin_short-width":115,"6_col_and_margin_short-height":119,"11_col_and_margin":124,"11_col_and_margin-width":125,"11_col_and_margin-height":119,"9_col_rectangle":126,"9_col_rectangle-width":127,"9_col_rectangle-height":128,"11_col":129,"11_col-width":130,"11_col-height":119,"staff_image_696_600":131,"staff_image_696_600-width":94,"staff_image_696_600-height":95,"rectangular_thumbnail":132,"rectangular_thumbnail-width":133,"rectangular_thumbnail-height":134},"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=640&q=90&w=640","640","https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=scale&h=172&q=90&w=200",200,172,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=scale&h=662&q=90&w=768",768,662,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=1000&q=90&w=1892","1892","1000","https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=scale&h=1324&q=90&w=1536",1536,1324,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=scale&h=1766&q=90&w=2048",2048,1766,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=1180&q=90&w=1860",1860,1180,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=870&q=90&w=2880",2880,870,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=600&q=90&w=800",800,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=800&q=90&w=600","https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=870&q=90&w=1860","https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=870&q=90&w=3080",3080,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=1400&q=90&w=2150",2150,1400,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=870&q=90&w=2640",2640,"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=600&q=90&w=696","https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress%2Cformat&fit=crop&h=300&q=90&w=750",750,300,{"acf_fc_layout":136,"reduced":29,"top_overlap":137,"bottom_underlap":137,"disable_observer":-1,"heading":138,"cta_text":139,"cta_link":140,"items":141},"related_module","small","See more posts","See all articles","\u002Fblog\u002F",[142,156,166],{"type_key":17,"title":143,"excerpt":144,"to":145,"postedAt":146,"readingTime":147,"image":148,"authors":149,"category":153},"A guide to WebMCP for marketing leaders","\u003Cp>The structure of web traffic is undergoing a fundamental structural transition. We’re certainly seeing the impacts of these changes en masse. Whilst SEO visibility is driving, in part, Generative Engine Optimisation (“GEO”) visibility, clicks in traditional SEO rankings are harder to come by, and customer behaviour is fragmented as users are using new places to […]\u003C\u002Fp>\n","\u002Fblog\u002Fa-guide-to-webmcp-for-marketing-leaders\u002F","2026-08-27T13:51:35",13,"https:\u002F\u002Fimages.impression.co.uk\u002F2026\u002F08\u002Fyasa-design-studio-Ot2wQ8jF43s-unsplash.jpg?auto=compress%2Cformat&fit=crop&h=640&q=90&w=640",[150],{"name":151,"image":152},"Aaron Dicks","https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FAaron-Dicks2.png?auto=compress&fit=crop&fm=jpg&h=150&q=80&w=150&bg=CFE6F9",{"to":154,"name":155},"\u002Fblog\u002Fcategory\u002Fdigital-marketing\u002F","Digital Marketing",{"type_key":17,"title":157,"excerpt":158,"to":159,"postedAt":160,"readingTime":161,"image":162,"authors":163,"category":165},"Navigating Agentic Commerce with Universal Commerce Protocol","\u003Cp>As artificial intelligence transitions from information retrieval to autonomous task execution, the underlying infrastructure supporting ecommerce is undergoing a fundamental architectural shift.  The Universal Commerce Protocol (“UCP”) provides the unified communication layer required for AI agents to browse, negotiate, and execute transactions directly across merchant back-ends such as payment gateways. It’s an open-source standard co-developed […]\u003C\u002Fp>\n","\u002Fblog\u002Fnavigating-agentic-commerce-with-universal-commerce-protocol\u002F","2026-08-27T13:51:17",9,"https:\u002F\u002Fimages.impression.co.uk\u002F2026\u002F08\u002Fa-c-dqg4QeDNUi8-unsplash.jpg?auto=compress%2Cformat&fit=crop&h=640&q=90&w=640",[164],{"name":151,"image":152},{"to":154,"name":155},{"type_key":17,"title":167,"excerpt":168,"to":169,"postedAt":170,"readingTime":171,"image":172,"authors":173,"category":177},"Bing vs Google: Search Engine Comparison 2026","\u003Cp>In the Bing vs Google debate, many would point to the widespread popularity of Google – the world’s largest search engine – as evidence that users prefer it to its closest rival. Figures suggest that Google has 1 billion daily active users compared to Bing’s 100 million. Yet whilst Google continues to dominate the global […]\u003C\u002Fp>\n","\u002Fblog\u002Fbing-differ-google\u002F","2026-06-11T16:15:35",31,"https:\u002F\u002Fimages.impression.co.uk\u002F2022\u002F10\u002FBing-logo.jpg?auto=compress%2Cformat&fit=crop&h=640&q=90&w=640",[174],{"name":175,"image":176},"Jonathan Theuring","https:\u002F\u002Fimages.impression.co.uk\u002F2021\u002F05\u002F6bhGbuWP-Jonathan-Theuring.png?auto=compress&fit=crop&fm=jpg&h=150&q=80&w=150&bg=CFE6F9",{"to":154,"name":155},{"type":179,"style":180,"heading":30,"date":4,"updatedAt":181,"reading_minutes":182,"image":183,"imageAlt":30,"categories":184,"collections":453},"blog-post","light","2024-12-19T17:20:34",10,"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fhans-isaacson-O6uM94iMyVk-unsplash.jpg?auto=compress%2Cformat&fit=crop&h=1180&q=90&w=1860",[185,368],{"id":10,"count":182,"description":186,"link":187,"name":188,"slug":189,"taxonomy":190,"parent":53,"meta":191,"acf":192,"head_links":193,"resource_count":53,"yoast_title":194,"yoast_meta":195,"yoast_json_ld":238,"_links":336},"Behind the scenes of any good advertising campaign, customer experience or \u003Ca href=\"\u002Fmedia-solutions\u002Fmeasurement\u002F\">data analysis\u003C\u002Fa> is the lifting, shifting and transformation of millions of rows of raw data. At Impression our \u003Ca href=\"\u002Fmedia-solutions\u002F\">Solutions\u003C\u002Fa> team comprises data engineers familiar with advertising platforms, APIs, various industry tooling and web scraping -- and of course databases like \u003Ca href=\"\u002Fmedia-solutions\u002Fmodern-data-stack\u002F\">customer data platforms\u003C\u002Fa>. Check out the posts below.","\u002Fblog\u002Fcategory\u002Fdata-engineering\u002F","Data Engineering","data-engineering","category",[],{"hidden":7,"sub_title_override":188,"title_override":47},[],"Data Engineering - Impression",[196,198,201,204,207,210,212,215,217,220,223,226,229,232,235],{"property":197,"content":194},"title",{"name":199,"content":200},"robots","index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1",{"property":202,"content":203},"canonical","https:\u002F\u002Fadmin.impressiondigital.com\u002Fblog\u002Fcategory\u002Fdata-engineering\u002F",{"name":205,"content":206},"og:locale","en-GB",{"property":208,"content":209},"og:type","article",{"property":211,"content":194},"og:title",{"property":213,"content":214},"og:description","Behind the scenes of any good advertising campaign, customer experience or data analysis is the lifting, shifting and transformation of millions of rows of raw data. At Impression our Solutions team comprises data engineers familiar with advertising platforms, APIs, various industry tooling and web scraping — and of course databases like customer data platforms. Check out the posts below.",{"property":216,"content":203},"og:url",{"property":218,"content":219},"og:site_name","Impression",{"property":221,"content":222},"og:image","https:\u002F\u002Fimages.impression.co.uk\u002F2021\u002F06\u002FABFPXguH-Opengraph-BLUE.jpg?auto=compress%2Cformat&q=90",{"property":224,"content":225},"og:image:width",1200,{"property":227,"content":228},"og:image:height",630,{"property":230,"content":231},"og:image:type","image\u002Fjpeg",{"property":233,"content":234},"twitter:card","summary_large_image",{"property":236,"content":237},"twitter:site","@impressiontalk",{"@context":239,"@graph":240},"https:\u002F\u002Fschema.org",[241,249,259,269],{"@type":242,"@id":243,"url":243,"name":194,"isPartOf":244,"breadcrumb":246,"inLanguage":248},"CollectionPage","https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002Fcategory\u002Fdata-engineering\u002F",{"@id":245},"https:\u002F\u002Fwww.impressiondigital.com\u002F#website",{"@id":247},"https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002Fcategory\u002Fdata-engineering\u002F#breadcrumb","en",{"@type":250,"@id":247,"itemListElement":251},"BreadcrumbList",[252,257],{"@type":253,"position":254,"name":255,"item":256},"ListItem",1,"Home","https:\u002F\u002Fwww.impressiondigital.com\u002F",{"@type":253,"position":258,"name":188},2,{"@type":260,"@id":245,"url":256,"name":219,"description":47,"publisher":261,"inLanguage":206,"potentialAction":263},"WebSite",{"@id":262},"https:\u002F\u002Fwww.impressiondigital.com\u002F#organization",{"@type":264,"target":265,"query-input":268},"SearchAction",{"@type":266,"urlTemplate":267},"EntryPoint","https:\u002F\u002Fwww.impressiondigital.com\u002Fsearch?q={search_term_string}","required name=search_term_string",{"@type":270,"@id":262,"name":219,"url":256,"logo":271,"image":277,"sameAs":278,"@context":282,"email":283,"foundingDate":284,"legalName":285,"vatID":286,"description":287,"telephone":288,"address":289,"location":296,"areaServed":329},"Organization",{"@type":272,"inLanguage":206,"@id":273,"url":274,"contentUrl":274,"width":275,"height":276,"caption":219},"ImageObject","https:\u002F\u002Fwww.impressiondigital.com\u002F#\u002Fschema\u002Flogo\u002Fimage\u002F","https:\u002F\u002Fcdn.impression.co.uk\u002F2021\u002F06\u002FA5LuTDrq-Impression_Wordmark.png",920,176,"https:\u002F\u002Fcdn.impression.co.uk\u002F2021\u002F04\u002FFAVxDeOS-Nottingham.jpg",[279,280,281],"https:\u002F\u002Fwww.linkedin.com\u002Fcompany\u002Fimpression-digital-limited","https:\u002F\u002Fwww.facebook.com\u002Fimpression\u002F","https:\u002F\u002Fwww.instagram.com\u002Fimpressiontalk\u002F","https:\u002F\u002Fwww.schema.org\u002F","hello@impressiondigital.com","2012-11-07","Impression Digital Limited","GB186363774","Impression is an award-winning digital marketing agency based in the UK and USA. We offer SEO agency, PPC agency, Digital PR agency and Analytics services to companies of all sizes across the UK, North America, Europe and beyond. Get in touch with us if you think we can work together.","01158 242 212",{"@type":290,"streetAddress":291,"addressLocality":292,"addressRegion":293,"postalCode":294,"addressCountry":295},"PostalAddress","Fothergill House, 16 King Street","Nottingham","Nottinghamshire","NG1 2AS","United Kingdom",[297,306,318],{"@type":298,"name":299,"geo":300,"address":304,"photo":277,"telephone":305},"Place","Impression Digital - Nottingham",{"@type":301,"latitude":302,"longitude":303},"GeoCoordinates","52.95501401025914","-1.1494887460285232",{"@type":290,"streetAddress":291,"addressLocality":292,"addressRegion":293,"postalCode":294,"addressCountry":295},"+44 01158 242 212",{"@type":298,"name":307,"geo":308,"address":311,"photo":316,"telephone":317},"Impression Digital - London",{"@type":301,"latitude":309,"longitude":310},"51.50237208377361","-0.09229841534284115",{"@type":290,"streetAddress":312,"addressLocality":313,"addressRegion":314,"postalCode":315,"addressCountry":295},"201 Borough High St","Southwark","London","SE1 1JA","https:\u002F\u002Fcdn.impression.co.uk\u002F2021\u002F04\u002FwD0pPVWz-London.jpg","+44 020 7112 8148",{"@type":298,"name":319,"geo":320,"address":323},"Impression Digital - New York",{"@type":301,"latitude":321,"longitude":322},"40.726722746949","-74.00527778220652",{"@type":290,"streetAddress":324,"addressLocality":325,"addressRegion":326,"postalCode":327,"addressCountry":328},"160 Varick St","New York","NY","10013","USA",[330,333,335],{"@type":331,"name":332},"Country","Worldwide",{"@type":331,"name":334},"GBR",{"@type":331,"name":328},{"self":337,"collection":347,"about":350,"wp:post_type":353,"curies":364},[338],{"href":339,"targetHints":340},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fcategories\u002F280",{"allow":341},[342,343,344,345,346],"GET","POST","PUT","PATCH","DELETE",[348],{"href":349},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fcategories",[351],{"href":352},"\u002Fwp-json\u002Fwp\u002Fv2\u002Ftaxonomies\u002Fcategory",[354,356,358,360,362],{"href":355},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fposts?categories=280",{"href":357},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fwhite_papers?categories=280",{"href":359},"\u002Fwp-json\u002Fwp\u002Fv2\u002Ftools?categories=280",{"href":361},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fwebinars?categories=280",{"href":363},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fevents?categories=280",[365],{"name":366,"href":367,"templated":29},"wp","https:\u002F\u002Fapi.w.org\u002F{rel}",{"id":11,"count":369,"description":370,"link":154,"name":155,"slug":371,"taxonomy":190,"parent":53,"meta":372,"acf":373,"head_links":374,"resource_count":375,"yoast_title":376,"yoast_meta":377,"yoast_json_ld":397,"_links":430,"path":154},282,"\u003Cp class=\"p1\">Keeping up to date with the latest digital trends, strategies, and announcements can be difficult. Our experts regularly blog about the digital marketing news that’s stirring up the industry, so check back regularly or \u003Ca href=\"\u002Fnewsletter\u002F\">sign up to our newsletter\u003C\u002Fa> to stay informed!\u003C\u002Fp>\r\n\u003Cp class=\"p1\">\u003Cspan style=\"font-weight: 400\">Impression is a multi-award-winning \u003C\u002Fspan>\u003Ca href=\"\u002Fdigital-marketing\u002F\">\u003Cspan style=\"font-weight: 400\">digital marketing agency\u003C\u002Fspan>\u003C\u002Fa>\u003Cspan style=\"font-weight: 400\"> that provides services\u003C\u002Fspan> that provides services including \u003Ca href=\"\u002Fdigital-marketing\u002Fstrategy\u002F\">Strategy\u003C\u002Fa>, \u003Ca href=\"\u002Fseo\u002F\">SEO\u003C\u002Fa>, \u003Ca href=\"\u002Fpaid-media\u002Fppc\u002F\">PPC\u003C\u002Fa>, \u003Ca href=\"\u002Fdigital-experience\u002Fcro\u002F\">CRO\u003C\u002Fa>, \u003Ca href=\"\u002Fpr-social\u002Fdigital-pr\u002F\">PR\u003C\u002Fa>, \u003Ca href=\"\u002Fanalytics\u002F\">Analytics\u003C\u002Fa> and \u003Ca href=\"\u002Fmedia-solutions\u002Fmedia-effectiveness\u002F\">Media Effectiveness\u003C\u002Fa> to its wide range of clients. Browse our latest thinking around digital marketing below, or \u003Ca href=\"\u002Fget-in-touch\u002F\">get in touch\u003C\u002Fa> to discuss how we can help you.\u003C\u002Fp>","digital-marketing",[],{"sub_title_override":47,"title_override":47,"hidden":7},[],96,"Digital Marketing Blog | Tips, Advice & Guides | Impression",[378,379,382,383,385,386,387,388,389,390,391,392,393,394,395,396],{"property":197,"content":376},{"property":380,"content":381},"description","Keeping up to date with the latest digital marketing trends, strategies, and announcements can be difficult. Browse our tips and advice in our latest blogs!",{"name":199,"content":200},{"property":202,"content":384},"https:\u002F\u002Fadmin.impressiondigital.com\u002Fblog\u002Fcategory\u002Fdigital-marketing\u002F",{"name":205,"content":206},{"property":208,"content":209},{"property":211,"content":376},{"property":213,"content":381},{"property":216,"content":384},{"property":218,"content":219},{"property":221,"content":222},{"property":224,"content":225},{"property":227,"content":228},{"property":230,"content":231},{"property":233,"content":234},{"property":236,"content":237},{"@context":239,"@graph":398},[399,404,408,412],{"@type":242,"@id":400,"url":400,"name":376,"isPartOf":401,"description":381,"breadcrumb":402,"inLanguage":248},"https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002Fcategory\u002Fdigital-marketing\u002F",{"@id":245},{"@id":403},"https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002Fcategory\u002Fdigital-marketing\u002F#breadcrumb",{"@type":250,"@id":403,"itemListElement":405},[406,407],{"@type":253,"position":254,"name":255,"item":256},{"@type":253,"position":258,"name":155},{"@type":260,"@id":245,"url":256,"name":219,"description":47,"publisher":409,"inLanguage":206,"potentialAction":410},{"@id":262},{"@type":264,"target":411,"query-input":268},{"@type":266,"urlTemplate":267},{"@type":270,"@id":262,"name":219,"url":256,"logo":413,"image":277,"sameAs":414,"@context":282,"email":283,"foundingDate":284,"legalName":285,"vatID":286,"description":287,"telephone":288,"address":415,"location":416,"areaServed":426},{"@type":272,"inLanguage":206,"@id":273,"url":274,"contentUrl":274,"width":275,"height":276,"caption":219},[279,280,281],{"@type":290,"streetAddress":291,"addressLocality":292,"addressRegion":293,"postalCode":294,"addressCountry":295},[417,420,423],{"@type":298,"name":299,"geo":418,"address":419,"photo":277,"telephone":305},{"@type":301,"latitude":302,"longitude":303},{"@type":290,"streetAddress":291,"addressLocality":292,"addressRegion":293,"postalCode":294,"addressCountry":295},{"@type":298,"name":307,"geo":421,"address":422,"photo":316,"telephone":317},{"@type":301,"latitude":309,"longitude":310},{"@type":290,"streetAddress":312,"addressLocality":313,"addressRegion":314,"postalCode":315,"addressCountry":295},{"@type":298,"name":319,"geo":424,"address":425},{"@type":301,"latitude":321,"longitude":322},{"@type":290,"streetAddress":324,"addressLocality":325,"addressRegion":326,"postalCode":327,"addressCountry":328},[427,428,429],{"@type":331,"name":332},{"@type":331,"name":334},{"@type":331,"name":328},{"self":431,"collection":436,"about":438,"wp:post_type":440,"curies":451},[432],{"href":433,"targetHints":434},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fcategories\u002F39",{"allow":435},[342,343,344,345,346],[437],{"href":349},[439],{"href":352},[441,443,445,447,449],{"href":442},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fposts?categories=39",{"href":444},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fwhite_papers?categories=39",{"href":446},"\u002Fwp-json\u002Fwp\u002Fv2\u002Ftools?categories=39",{"href":448},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fwebinars?categories=39",{"href":450},"\u002Fwp-json\u002Fwp\u002Fv2\u002Fevents?categories=39",[452],{"name":366,"href":367,"templated":29},[],[],[456,458,460,461],{"path":457,"name":255},"\u002F",{"path":140,"name":459},"Blog",{"path":187,"name":188},{"path":462,"name":30},null,[464,465],{"id":10,"name":188,"path":187},{"id":11,"name":155,"path":154},{"title":467,"script":468,"meta":540,"link":570},"Understanding the workings of Bayesian A\u002FB testing | Impression",[469],{"type":470,"key":471,"innerHTML":472},"application\u002Fld+json","yoast-json-ld",{"@context":239,"@graph":473},[474,491,498,500,509,513,531],{"@type":475,"@id":476,"isPartOf":477,"author":479,"headline":30,"datePublished":481,"dateModified":482,"mainEntityOfPage":483,"wordCount":484,"publisher":485,"image":487,"thumbnailUrl":489,"articleSection":490,"inLanguage":206},"Article","https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002Funderstanding-the-workings-of-bayesian-ab-testing\u002F#article",{"@id":478},"https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002Funderstanding-the-workings-of-bayesian-ab-testing\u002F",{"name":35,"@id":480},"https:\u002F\u002Fadmin.impressiondigital.com\u002F#\u002Fschema\u002Fperson\u002Fa18b79c97abbcb2444ace02aa88250ae","2024-12-19T17:20:33+00:00","2024-12-19T17:20:34+00:00",{"@id":478},2003,{"@id":486},"https:\u002F\u002Fadmin.impressiondigital.com\u002F#organization",{"@id":488},"https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002Funderstanding-the-workings-of-bayesian-ab-testing\u002F#primaryimage","https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fhans-isaacson-O6uM94iMyVk-unsplash.jpg?auto=compress%2Cformat&q=90",[188,155],{"@type":492,"@id":478,"url":478,"name":467,"isPartOf":493,"primaryImageOfPage":494,"image":495,"thumbnailUrl":489,"datePublished":481,"dateModified":482,"breadcrumb":496,"inLanguage":248},"WebPage",{"@id":245},{"@id":488},{"@id":488},{"@id":497},"https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002Funderstanding-the-workings-of-bayesian-ab-testing\u002F#breadcrumb",{"@type":272,"inLanguage":206,"@id":488,"url":489,"contentUrl":489,"width":499,"height":118},1920,{"@type":250,"@id":497,"itemListElement":501},[502,503,505,507],{"@type":253,"position":254,"name":255,"item":256},{"@type":253,"position":258,"name":459,"item":504},"https:\u002F\u002Fwww.impressiondigital.com\u002Fblog\u002F",{"@type":253,"position":506,"name":188,"item":243},3,{"@type":253,"position":508,"name":30},4,{"@type":260,"@id":245,"url":256,"name":219,"description":47,"publisher":510,"inLanguage":206,"potentialAction":511},{"@id":262},{"@type":264,"target":512,"query-input":268},{"@type":266,"urlTemplate":267},{"@type":270,"@id":262,"name":219,"url":256,"logo":514,"image":277,"sameAs":515,"@context":282,"email":283,"foundingDate":284,"legalName":285,"vatID":286,"description":287,"telephone":288,"address":516,"location":517,"areaServed":527},{"@type":272,"inLanguage":206,"@id":273,"url":274,"contentUrl":274,"width":275,"height":276,"caption":219},[279,280,281],{"@type":290,"streetAddress":291,"addressLocality":292,"addressRegion":293,"postalCode":294,"addressCountry":295},[518,521,524],{"@type":298,"name":299,"geo":519,"address":520,"photo":277,"telephone":305},{"@type":301,"latitude":302,"longitude":303},{"@type":290,"streetAddress":291,"addressLocality":292,"addressRegion":293,"postalCode":294,"addressCountry":295},{"@type":298,"name":307,"geo":522,"address":523,"photo":316,"telephone":317},{"@type":301,"latitude":309,"longitude":310},{"@type":290,"streetAddress":312,"addressLocality":313,"addressRegion":314,"postalCode":315,"addressCountry":295},{"@type":298,"name":319,"geo":525,"address":526},{"@type":301,"latitude":321,"longitude":322},{"@type":290,"streetAddress":324,"addressLocality":325,"addressRegion":326,"postalCode":327,"addressCountry":328},[528,529,530],{"@type":331,"name":332},{"@type":331,"name":334},{"@type":331,"name":328},{"@type":532,"@id":533,"name":35,"image":534,"description":37,"sameAs":537,"url":539},"Person","https:\u002F\u002Fwww.impressiondigital.com\u002F#\u002Fschema\u002Fperson\u002Fa18b79c97abbcb2444ace02aa88250ae",{"@type":272,"inLanguage":206,"@id":535,"url":536,"contentUrl":536,"caption":35},"https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress&fit=fit&fm=jpg&h=96&q=90&w=9615847ead018a2de28b7216e39fc92f32","https:\u002F\u002Fimages.impression.co.uk\u002F2023\u002F10\u002FHarry-Brace2.png?auto=compress&fit=fit&fm=jpg&h=96&q=90&w=96",[538],"https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fharrison-brace-137b8482\u002F","https:\u002F\u002Fwww.impressiondigital.com\u002Fabout\u002Four-team\u002Fharry-brace\u002F",[541,542,544,545,546,548,549,550,553,555,557,559,560,561,562,564,565,567,569],{"name":199,"content":200},{"property":205,"content":543},"en_GB",{"property":208,"content":209},{"property":211,"content":467},{"property":213,"content":547},"In digital marketing, A\u002FB testing is a technique for comparing two versions of a variable, such as a landing page layout or ad creative, to determine which version performs best with the target audience. The audience is randomly divided into two groups: one group is exposed to version A (the control), and the other to […]",{"property":216,"content":478},{"property":218,"content":219},{"property":551,"content":552},"article:publisher","https:\u002F\u002Fwww.facebook.com\u002Fimpression",{"property":554,"content":481},"article:published_time",{"property":556,"content":482},"article:modified_time",{"property":221,"content":558},"https:\u002F\u002Fimages.impression.co.uk\u002F2024\u002F12\u002Fopen-graph-inner-workings-ab-testing.png?auto=compress%2Cformat&q=90",{"property":224,"content":225},{"property":227,"content":228},{"property":230,"content":90},{"name":563,"content":35},"author",{"name":233,"content":234},{"name":566,"content":558},"twitter:image",{"name":568,"content":237},"twitter:creator",{"name":236,"content":237},[]]