Multivariate landing page optimization

Multivariate landing page optimization (MVLPO) is a specific form of landing page optimization where multiple variations of visual elements (e.g., graphics, text) on a page are evaluated. For example, a given page may have k choices for the title, m choices for the featured image or graphic, and n choices for the company logo. This example yields k×m×n landing page configurations.

The first application of an experimental design for MVLPO was performed by Moskowitz Jacobs Inc. in 1998 as a simulation/demonstration project for LEGO. MVLPO did not become a mainstream approach until 2003 or 2004.

Multivariate landing page optimization can be executed in a live (production) environment, or through market research surveys and simulations. Examples of live environment applications include Omniture.com, Google website optimizer, Memetrics.com, Widemile.com, and Optimost.com. An example of survey/simulation applications is StyleMap.Net.

Overview

Multivariate landing page optimization is based on experimental design (e.g., discrete choice, conjoint analysis, Taguchi methods, etc.), which tests a structured combination of webpage elements. Some vendors use full factorial approach (e.g., Memetrics xOs that tests all possible combinations of elements). This approach requires less sample size (typically, many thousands) to achieve statistical importance than traditional fractional Taguchi designs and is one reason that Choice Modeling won the Nobel Prize in year 2000. Fractional designs typically used in simulation environments require the testing of small subsets of possible combinations and have a higher margin for error. Some critics of the approach raise the question of possible interactions between the elements of the web pages and the inability of most fractional designs to address the issue.

To resolve these limitations, an advanced simulation method based on the Rule Developing Experimentation paradigm (RDE) has been introduced. RDE creates individual models for each respondent, discovers any and all synergies and suppressions between the elements, uncovers attitudinal segmentation, and allows for databasing across tests and over time).

Live environment execution

In live environment MVLPO execution, a special tool makes dynamic changes to the web site, so the visitors are directed to different executions of landing pages created according to an [experimental design]. The system keeps track of the visitors and their behavior (including their conversion rate, time spent on the page, etc.) and with sufficient data accumulated, estimates the impact of individual components on the target measurement (e.g., conversion rate).

Advantages

• This approach is very reliable because it tests the effect of variations as a real life experience, generally transparent to the visitors.
• It has evolved to a relatively simple and inexpensive to execute approach (e.g., Google Optimizer).

Disadvantages

(applicable mostly to the tools prior to Google Optimizer):

• High cost
• Complexity involved in modifying a production-level website
• Long time it may take to achieve statistically reliable data caused by variations in the amount of traffic, which generates the data necessary for the decision
• This approach may not be appropriate for low traffic / high importance websites when the site administrators do not want to lose any potential customers.

Many of these drawbacks are reduced or eliminated with the introduction of the Google Website Optimizer – a free DIY MVLPO tool that made the process more democratic and available to the website administrators directly.

Simulation (survey) execution

A simulation (survey) based MVLPO is built on advanced market research techniques. In the research phase, the respondents are directed to a survey, which presents them with a set of experimentally designed combinations of the landing page executions. The respondents rate each execution (screen) on a rating question (e.g., purchase intent). At the end of the study, regression model(s) are created (either individual or for the total panel). The outcome relates the presence/absence of the elements in the different landing page executions to the respondents’ ratings and can be used to synthesize new pages as combinations of the top-scored elements optimized for subgroups, segments, with or without interactions.

Advantages

• Much faster and easier to prepare and execute (in many cases) compared to the live environment optimization
• It works for low traffic websites
• Usually produces more robust and rich data because of a higher control of the design.

Disadvantages

• Possible bias of a simulated environment as opposed to a live one
• A necessity to recruit and optionally incentivise the respondents.