Test: Will it work and sell?

Once there is a design that is believed will meet customer requirements and, therefore, will sell in the market, it may seem the work is done. The new product must now just be produced (a manufacturing, not a new product development, issue) and then sold (a sales issue). But this is not the end. The design must still be tested before being manufactured and launched because it could still fail in the market. As I noted in Chapter 1, 80% of new products fail soon after launch. So product failure is more the norm and not the exception. Pre-launch testing will not guarantee success but it will help reduce or minimize the probability of failure. Pre-launch testing, of course, is costly and these costs will increase in direct proportion to the amount of testing which could kill a product before its launch. The testing costs consist of prototype or description production, test subject recruiting (someone has to use the product in a test), test design, test implementation, test surveillance (for security and confidentiality), and results analytics, interpretation, and reporting. An excessive amount of testing could also delay launch since testing also requires time so the product is then late to market. The competition is then given a window of opportunity that your product may never be able to close. This is another cost of testing.

Despite these costs, pre-launch testing actually serves several purposes. Since the product has not been launched, there is still time to hone and refine the product concept. Have the settings for all the product attributes, those identified as the most critical for customers, been properly set? What should the price point be now that this has to be set for launch? In many instances, what name should be attached to the product? And how will the product fare against competitive products? What market share can be expected? What volume can be expected? These are all pre-launch questions.

There are two forms of pre-launch testing that, in one instance, could be used separately but in another could be used jointly. The first form is discrete choice modeling and the second is market testing. The former is an econometric analysis procedure with a well-established methodology. The second is a more traditional way of testing: put the product in the market and see what happens. The problem with this latter approach is that many intervening market forces will impact customers’ purchase decisions about the product, just as in real market settings, as well as for other products; both the new product and competitive products are affected. This may produce unwanted sales impacts. For instance, a new consumer product will have to be sold in a retail store. This new product could adversely impact all sales of the store which means the merchant has to be compensated for any losses.

A better approach is to combine the two in a controlled setting. At the AT&T Consumer Lab1 mentioned in the Preface, consumers were brought into a lab setting and given a new product or, in most instances, a prototype of the new product to touch, look at, and use (if possible with a prototype). The potentially expensive market test was avoided. Once the consumers had examined the new product, they were surveyed about their opinions and in many instances were asked to complete discrete choice exercises regarding the product. A variation of this approach is sometimes called a clinic which I describe below.

This chapter is divided into six sections. In the first, I discuss discrete choice analysis for determining take rates (i.e., expected market share), expected volume sold, and willingness-to-pay. This constitutes an experimental approach to market testing. The second section is focused on actual market testing, primarily in hands-on clinics. I discuss issues associated with clinics and some ways to analyze data. The third section is devoted to market segmentation for more focused testing. The fourth section deals with multiple versions of a product, not just testing them but determining which combination should be marketed. This is TURF analysis applied to product testing. The fifth section contains the usual software discussion while the sixth section is a summary.

Discrete choice analysis

In this section, I will describe the fundamental principles for a discrete choice study. A study is called “discrete choice” because a customer is observed to choose one product over one or more other products in some type of setting, hence the choice is discrete. The setting could be part of a market research survey so the choice is a stated preference choice: the customer states his or her preference for one product over another. This framework was developed to mimic actual market choices in a consumer market in which a consumer goes to a store, sees several products on a store shelf (e.g., cereals) and selects just one to buy on that shopping occasion.

Another setting could be in an actual market (e.g., a grocery store) so the consumer reveals his/her preference for one product over another. This is a revealed preference choice. If the store is, again, a grocery store and the product is cereal, then a consumer is observed to select one cereal over another from the store shelf. This might seem comparable to the market studies I just mentioned. There is a difference, though. In a market test study, sales are tracked in the same manner as a post-launch tracking study which I discuss in Chapter 7. In a revealed preference study, however, only the selection is analyzed; not the unit sales. The end result is the same: product performance. The approach is different.

A stated preference choice study is common in market research. Paczkowski [2018J provides a good, complete discussion of stated preference market research studies. For a theoretical discussion and application to revealed preference choice studies, see Train |2009], In some instances, revealed and stated preference studies are combined. See Paczkowski [2018] for a brief discussion. Regardless of the setting, the underlying principles are the same and it is these principles that I will discuss in the next subsection.

Product configuration vs. competitive offerings

A central concept in a discrete choice study is the choice set containing discrete product options presented to a customer. The product options are typically some composition or configuration of the new product as well as competitor products. Each product, new and competitor, consists of a collection of product attributes or characteristics much like the conjoint attributes I described in Chapter 3. Examples of attributes are size, weight, form, portability, warranty, brand name, and, of course, price. Each attribute has specified levels. If weight is an attribute, its levels could be 1 ounce, 3 ounces, 5 ounces. In a stated preference choice study, the attribute levels are prespecified both for the new product and the competitive products. They are then arranged using statistical design of experiment principles to create choice sets, each set consisting of one variant of each product. It may be difficult to vary attribute levels for a competitive product simply because that product is not under the control of the market researcher. They could, however, artificially change some features they believe the competitor might change in the future based on competitive analysis and assessments. The price point is an easy example. Size is another, especially if there is a history of size changes or competitive market forces are pushing for, say, smaller sizes.

A revealed preference choice study is more difficult to design because product attributes cannot be easily changed. It may be possible to manipulate price points, but other attributes may be impossible. Some creative methods have to be devised to artificially change them after data are collected by calculating new variables that will change. In transportation mode studies, for example, price per commute time can be calculated where the price of a ticket does not change. The commute time from different stations and by time-of-day and day-of-week, however, do change so the price per commute time will change.

Several choice sets are typically created that meet desirable criteria. One criterion is that no product clearly dominates others in the same choice set. Suppose the attributes for all products are weight, size, and price point. If all products in a single set have the same weight and size but one has the lowest price, then that lowest priced product will always be selected by consumers; it would dominate all others and always “win.” A second criterion is that the same configuration cannot repeat in a single choice set because a consumer could not distinguish between them (they are, after all, the same product) so no insight into consumer choice is gained. See Paczkowski [2018] for a discussion of choice set construction. Also see Paczkowski |2016| for an example application using the JMP software.

Once the choice sets are developed, they are presented to a customer, one set at a time. Following each presentation, the customer is asked to select his/her preferred product in the set. This is repeated for each choice set. Sometimes a likelihood to purchase question is asked after each customer choice. The idea for this follow-up question is to develop a calibration weight for choice to adjust the choices to reflect the fact that what customers say they would buy is not what they would buy if they really went to a store. In a revealed preference study, they are observed to actually make a choice and pay for it; in a stated preference study they do not pay for anything. The act of paying makes a big difference. The calibration weight is supposed to adjust for this.

< Prev   CONTENTS   Source   Next >