Internet Marketing - Planning & Management | Conversion Analyst

Mohsin Khawaja

Subscribe to Mohsin Khawaja: eMailAlertsEmail Alerts
Get Mohsin Khawaja via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Related Topics: Objective-C Developer, Construction News

Blog Feed Post

Landing Page Optimization

The primary objective of landing page testing is to predict the behavior of your audience

Landing page testing: The primary objective of landing page testing is to predict the behavior of your audience given the specific content on the landing page that they see. You will collect a limited sample of data during your test, summarize and describe it, and predict how people from the same traffic sources will act when interacting with the page. The ultimate goal is to find the best possible version of the landing page among all of the variations that you are testing.


The word variable (when used by it) means a page element that you have selected. For example, a variable might be the headline of your landing page, or a whole-page redesign. In multivariate testing, a variable is also commonly referred to as a factor.

Branching Factor

The total number of possible values for a discrete variable is called its branching factor. the branching factor must be at least 2 (the original version and one alternative).


A recipe is a unique combination of variable values in your test.

Multivariate Testing

The purpose of multivariate testing is to simultaneously gather information about multiple variables, and then conduct an analysis of the data to determine which recipe results in the best performance.

Multivariate testing approaches differ on three important dimensions:

  • How the test is constructed (deciding what to test)
  • How the data is collected

The data can be collected in a full factorial or fractional factorial fashion

  • How the data is analyzed

The subsequent analysis can be either parametric or nonparametric

If you choose an arbitrary test construction, you may not be able to use fractional factorial approaches. Likewise, if you choose fractional factorial data collection and test construction, you automatically lock yourself into a very restricted subset of parametric models for your subsequent data analysis (i.e., non-parametric analysis is impossible if you conduct fractional factorial data collection).

Search Space Size

The number of unique recipes in your test is your search space size.

Test Construction

When you are deciding how to construct your test, there is an important distinction that must be made: Is your test structured or unstructured?

Unstructured designs are by far the simplest to implement. You simply choose exactly how many variables you want to test, and the branching factor for each one. As long as you are reasonable, you can choose very different numbers of values for each variable. For example, you can pick 7 different headline, 2 button colors, and 9 calls-to-action. This allows you to pay particular attention to the test elements that you think will result in the greatest performance improvements, and devote more of your test sampling to finding the best values for them. The branching factor is simply a function of how many good creative alternatives you want to test for a particular variable. If you choose full factorial data collection, you will enjoy the benefits of unstructured designs.

Structured designs are an artifact of fractional factorial data collection. By assuming a specific underlying model, they force you to have a specific shape to your test. In other words, the number of variables and their branching factors are predefined, and cannot be violated. For example, you might be forced to have one variable with a branching factor of two and seven variables with a branching factor of three. There are a number of standard test constructions to choose from, but you must use one of them.

Data Analysis

Parametric data analysis in landing page optimization builds a model of how the variables tested (the "independent variables") impact the conversion rate (the "dependent variable"). For each recipe in your search space, the model will produce a prediction of the expected conversion rate (or other optimization criterion of interest). Unless you happened to have sampled data on the exact recipe predicted by the model as being the best, you do not really know if the prediction will hold up. That is why it is critical to run follow-up A-B split tests between the predicted best challenger recipe and the original baseline recipe for all parametric data analysis methods.

By contrast, non-parametric data analysis does not try to build a model based on the input variables. Non-parametric methods try to identify the best challenger recipe, but without being able to tell anything about why it is the best, or exactly how much better it is than your baseline.

The two approaches are unrelated and are answering different questions. They are both recognition of the fundamental reality that only so much useful information can be extracted from your data collection sample. The only question is what you want to do with the data. You can try to create a general model of the output variable and try to describe it in terms of the input variables, or you can find the best individual recipe and not know why it is the best.

Although there are some difference among these common fractional factorial methods, their basic predictive power, required data sample size, and underlying assumptions are pretty similar. The main difference lies in the test construction and shape of the search spaces that each can be used for. So if you are going to use any of these methods, you should base your decision on your familiarity with each and the number and branching factor of the variables in your test. There is no reason to prefer the Taguchi method over Plackett-Burman, or Latin squares, but since it is gaining currency in landing page testing I will focus on it here.


There is much more to add in this article and there are certain drawbacks and advantages of each data collection and analysis method. But to conclude my opinion is that Full Factorial Non-parametric Testing methodology is optimum for most of the testing needs. However, there can be other views, and if you have a certain opinion, I will like to hear it on the comments below, thanks.

Read the original blog entry...

More Stories By Mohsin Khawaja

Mohsin Khawaja is a Internet Marketing Manager at Intellectual Works.He occasionally writes on various IT topics along with various SEO, SEM and Internet Marketing articles.