Questionnaire Design

The questionnaire is a structured technique for collecting primary data in a marketing survey. It is a series of written or verbal questions for which the respondent provides answers. A well-designed questionnaire motivates the respondent to provide complete and accurate information.

The survey questionnaire should not be viewed as a stand-alone tool. Along with the questionnaire there is field work, rewards for the respondents, and communication aids, all of which are important components of the questionnaire process.

Steps to Developing a Questionnaire

The following are steps to developing a questionnaire - the exact order may vary somewhat.

• Determine which information is being sought.
• Choose a question type (structure and amount of disguise) and method of administration (for example, written form, email or web form, telephone interview, verbal interview).
• Determine the general question content needed to obtain the desired information.
• Determine the form of response.
• Choose the exact question wording.
• Arrange the questions into an effective sequence.
• Specify the physical characteristics of the questionnaire (paper type, number of questions per page, etc.)
• Test the questionnaire and revise it as needed.

Required Information

To determine exactly which information is needed, it is useful to construct tables into which the data will be placed once it is collected. The tables will help to define what data is needed and what is not needed.


Question Type and Administration Method

Some question types include fixed alternative, open ended, and projective:

• Fixed-alternative questions provide multiple-choice answers. These types of questions are good when the possible replies are few and clear-cut, such as age, car ownership, etc.
• Open-ended questions allow the respondent to better express his/her answer, but are more difficult to administer and analyze. Often, open-ended questions are administered in a depth interview. This technique is most appropriate for exploratory research.
• Projective methods use a vague question or stimulus and attempt to project a person's attitudes from the response. The questionnaire could use techniques such as word associations and fill-in-the-blank sentences. Projective methods are difficult to analyze and are better suited for exploratory research than for descriptive or causal research.

There are three commonly used rating scales: graphic, itemized, and comparative.

• Graphic - simply a line on which one marks an X anywhere between the extremes with an infinite number of places where the X can be placed.
• Itemized - similar to graphic except there are a limited number of categories that can be marked.
• Comparative - the respondent compares one attribute to others. Examples include the Q-sort technique and the constant sum method, which requires one to divide a fixed number of points among the alternatives.

Questionnaires typically are administered via a personal or telephone interview or via a mail questionnaire. Newer methods include e-mail and the Web.


Question Content

Each question should have a specific purpose or should not be included in the questionnaire. The goal of the questions is to obtain the required information. This is not to say that all questions directly must ask for the desired data. In some cases questions can be used to establish rapport with the respondent, especially when sensitive information is being sought.

Sensitive questions can be posed in ways to increase response likelihood and to facilitate more honest responses. Some techniques are:

• Place the question in a series of less personal questions.
• State that the behavior or attitude is not so unusual.
• Phrase the question in terms of other people, not the respondent.
• Provide response choices that specify ranges, not exact numbers.
• Use a randomized response model giving the respondent pairs of questions with a randomly assigned one to answer. The interviewer does not know which question the person is answering, but the overall percentage of people assigned to the sensitive question is known and statistics can be calculated.

Form of Question Response

Questions can be designed for open-ended, dichotomous, or multichotomous responses.

• Open-ended responses are difficult to evaluate, but are useful early in the research process for determining the possible range of responses.
• Dichotomous questions have two possible opposing responses, for example, "Yes" and "No".
• Multichotomous questions have a range of responses as in a multiple choice test.

The questionnaire designer should consider that respondents may not be able to answer some questions accurately. Two types of error are telescoping error and recall loss.

• Telescoping error is an error resulting from the tendency of people to remember events as occurring more recently than they actually did.
• Recall loss occurs when people forget that an event even occurred. For recent events, telescoping error dominates; for events that happened in the distant past, recall loss dominates.

Question Wording

The questions should be worded so that they are unambiguous and easily understood. The wording should consider the full context of the respondent's situation. In particular, consider the who, what, when, where, why, and how dimensions of the question.

For example, the question,

"Which brand of toothpaste do you use?"

might seem clear at first. However, the respondent may consider "you" to be the family as a whole rather than he or she personally. If the respondent recently changed brands, the "when" dimension of the question may be relevant. If the respondent uses a different, more compact tube of toothpaste when traveling, the "where" aspect of the question will matter.

A better wording of the question might be,

"Which brand of toothpaste have you used personally at home during the past 6 months? If you have used more than one brand, please list each of them."

When asking about the frequency of use, the questions should avoid ambiguous words such as "sometimes", "occasionally", or "regularly". Rather, more specific terms such as "once per day" and "2-3 times per week" should be used.


Sequence the Questions

Some neutral questions should be placed at the beginning of the questionnaire in order to establish rapport and put the respondent at ease. Effective opening questions are simple and non-threatening.

When sequencing the questions, keep in mind that their order can affect the response. One way to correct for this effect is to distribute half of the questionnaires with one order, and the other half with another order.


Physical Characteristics of the Questionnaire

Physical aspects such as the page layout, font type and size, question spacing, and type of paper should be considered. In order to eliminate the need to flip back and forth between pages, the layout should be designed so that a question at the bottom of the page does not need to be continued onto the next page. The font should be readable by respondents who have less-than-perfect visual acuity. The paper stock should be good quality to project the image that the questionnaire is important enough to warrant the respondents' time. Each questionnaire should have a unique number in order to better account for it and to know if any have been lost.


Test and Revise the Questionnaire

The questionnaire should be pre-tested in two stages before distributing. In the first stage, it should be administered using personal interviews in order to get better feedback on problems such as ambiguous questions. Then, it should be tested in the same way it will be administered. The data from the test should be analyzed the same way the administered data is to be analyzed in order to uncover any unanticipated shortcomings.

Different respondents will answer the same questionnaire differently. One hopes that the differences are due to real differences in the measured characteristics, but that often is not the case. Some sources of the differences between scores of different respondents are:

• True differences in the characteristic being measured.
• Differences in other characteristics such as response styles.
• Differences in transient personal factors such as fatigue, etc.
• Differences in situation, such as whether spouse is present.
• Differences in the administration, such as interviewer tone of voice.
• Differences resulting from sampling of items relevant toward the characteristic being measured.
• Differences resulting from lack of clarity of the question - may mean different things to different people.
• Differences caused by mechanical factors such as space to answer, inadvertent check marks, etc.

Conjoint Analysis

When asked to do so outright, many consumers are unable to accurately determine the relative importance that they place on product attributes. For example, when asked which attributes are the more important ones, the response may be that they all are important. Furthermore, individual attributes in isolation are perceived differently than in the combinations found in a product. It is difficult for a survey respondent to take a list of attributes and mentally construct the preferred combinations of them. The task is easier if the respondent is presented with combinations of attributes that can be visualized as different product offerings. However, such a survey becomes impractical when there are several attributes that result in a very large number of possible combinations.

Fortunately, conjoint analysis can facilitate the process. Conjoint analysis is a tool that allows a subset of the possible combinations of product features to be used to determine the relative importance of each feature in the purchasing decision. Conjoint analysis is based on the fact that the relative values of attributes considered jointly can better be measured than when considered in isolation.

In a conjoint analysis, the respondent may be asked to arrange a list of combinations of product attributes in decreasing order of preference. Once this ranking is obtained, a computer is used to find the utilities of different values of each attribute that would result in the respondent's order of preference. This method is efficient in the sense that the survey does not need to be conducted using every possible combination of attributes. The utilities can be determined using a subset of possible attribute combinations. From these results one can predict the desirability of the combinations that were not tested.

Steps in Developing a Conjoint Analysis

Developing a conjoint analysis involves the following steps:

1. Choose product attributes, for example, appearance, size, or price.
2. Choose the values or options for each attribute. For example, for the attribute of size, one may choose the levels of 5", 10", or 20". The higher the number of options used for each attribute, the more burden that is placed on the respondents.
3. Define products as a combination of attribute options. The set of combinations of attributes that will be used will be a subset of the possible universe of products.
4. Choose the form in which the combinations of attributes are to be presented to the respondents. Options include verbal presentation, paragraph description, and pictorial presentation.
5. Decide how responses will be aggregated. There are three choices - use individual responses, pool all responses into a single utility function, or define segments of respondents who have similar preferences.
6. Select the technique to be used to analyze the collected data. The part-worth model is one of the simpler models used to express the utilities of the various attributes. There also are vector (linear) models and ideal-point (quadratic) models.

The data is processed by statistical software written specifically for conjoint analysis.

Conjoint analysis was first used in the early 1970's and has become an important marketing research tool. It is well-suited for defining a new product or improving an existing one.

Marketing Research

Managers need information in order to introduce products and services that create value in the mind of the customer. But the perception of value is a subjective one, and what customers value this year may be quite different from what they value next year. As such, the attributes that create value cannot simply be deduced from common knowledge. Rather, data must be collected and analyzed. The goal of marketing research is to provide the facts and direction that managers need to make their more important marketing decisions.

To maximize the benefit of marketing research, those who use it need to understand the research process and its limitations.

Marketing Research vs. Market Research

These terms often are used interchangeably, but technically there is a difference.

Market research deals specifically with the gathering of information about a market's size and trends. Marketing research covers a wider range of activities. While it may involve market research, marketing research is a more general systematic process that can be applied to a variety of marketing problems.

The Value of Information

Information can be useful, but what determines its real value to the organization? In general, the value of information is determined by:

• The ability and willingness to act on the information.
• The accuracy of the information.
• The level of indecisiveness that would exist without the information.
• The amount of variation in the possible results.
• The level of risk aversion.
• The reaction of competitors to any decision improved by the information.
• The cost of the information in terms of time and money.

The Marketing Research Process

Once the need for marketing research has been established, most marketing research projects involve these steps:

1. Define the problem
2. Determine research design
3. Identify data types and sources
4. Design data collection forms and questionnaires
5. Determine sample plan and size
6. Collect the data
7. Analyze and interpret the data
8. Prepare the research report

Problem Definition

The decision problem faced by management must be translated into a market research problem in the form of questions that define the information that is required to make the decision and how this information can be obtained. Thus, the decision problem is translated into a research problem. For example, a decision problem may be whether to launch a new product. The corresponding research problem might be to assess whether the market would accept the new product.

The objective of the research should be defined clearly. To ensure that the true decision problem is addressed, it is useful for the researcher to outline possible scenarios of the research results and then for the decision maker to formulate plans of action under each scenario. The use of such scenarios can ensure that the purpose of the research is agreed upon before it commences.


Research Design

Marketing research can classified in one of three categories:

• Exploratory research
• Descriptive research
• Causal research

These classifications are made according to the objective of the research. In some cases the research will fall into one of these categories, but in other cases different phases of the same research project will fall into different categories.

• Exploratory research has the goal of formulating problems more precisely, clarifying concepts, gathering explanations, gaining insight, eliminating impractical ideas, and forming hypotheses. Exploratory research can be performed using a literature search, surveying certain people about their experiences, focus groups, and case studies. When surveying people, exploratory research studies would not try to acquire a representative sample, but rather, seek to interview those who are knowledgeable and who might be able to provide insight concerning the relationship among variables. Case studies can include contrasting situations or benchmarking against an organization known for its excellence. Exploratory research may develop hypotheses, but it does not seek to test them. Exploratory research is characterized by its flexibility.
• Descriptive research is more rigid than exploratory research and seeks to describe users of a product, determine the proportion of the population that uses a product, or predict future demand for a product. As opposed to exploratory research, descriptive research should define questions, people surveyed, and the method of analysis prior to beginning data collection. In other words, the who, what, where, when, why, and how aspects of the research should be defined. Such preparation allows one the opportunity to make any required changes before the costly process of data collection has begun.

There are two basic types of descriptive research: longitudinal studies and cross-sectional studies. Longitudinal studies are time series analyses that make repeated measurements of the same individuals, thus allowing one to monitor behavior such as brand-switching. However, longitudinal studies are not necessarily representative since many people may refuse to participate because of the commitment required. Cross-sectional studies sample the population to make measurements at a specific point in time. A special type of cross-sectional analysis is a cohort analysis, which tracks an aggregate of individuals who experience the same event within the same time interval over time. Cohort analyses are useful for long-term forecasting of product demand.

• Causal research seeks to find cause and effect relationships between variables. It accomplishes this goal through laboratory and field experiments.

Data Types and Sources

Secondary Data
Before going through the time and expense of collecting primary data, one should check for secondary data that previously may have been collected for other purposes but that can be used in the immediate study. Secondary data may be internal to the firm, such as sales invoices and warranty cards, or may be external to the firm such as published data or commercially available data. The government census is a valuable source of secondary data.

Secondary data has the advantage of saving time and reducing data gathering costs. The disadvantages are that the data may not fit the problem perfectly and that the accuracy may be more difficult to verify for secondary data than for primary data.

Some secondary data is republished by organizations other than the original source. Because errors can occur and important explanations may be missing in republished data, one should obtain secondary data directly from its source. One also should consider who the source is and whether the results may be biased.

There are several criteria that one should use to evaluate secondary data.

• Whether the data is useful in the research study.
• How current the data is and whether it applies to time period of interest.
• Errors and accuracy - whether the data is dependable and can be verified.
• Presence of bias in the data.
• Specifications and methodologies used, including data collection method, response rate, quality and analysis of the data, sample size and sampling technique, and questionnaire design.
• Objective of the original data collection.
• Nature of the data, including definition of variables, units of measure, categories used, and relationships examined.

Primary Data
Often, secondary data must be supplemented by primary data originated specifically for the study at hand. Some common types of primary data are:

• demographic and socioeconomic characteristics
• psychological and lifestyle characteristics
• attitudes and opinions
• awareness and knowledge - for example, brand awareness
• intentions - for example, purchase intentions. While useful, intentions are not a reliable indication of actual future behavior.
• motivation - a person's motives are more stable than his/her behavior, so motive is a better predictor of future behavior than is past behavior.
• behavior

Primary data can be obtained by communication or by observation. Communication involves questioning respondents either verbally or in writing. This method is versatile, since one needs only to ask for the information; however, the response may not be accurate. Communication usually is quicker and cheaper than observation. Observation involves the recording of actions and is performed by either a person or some mechanical or electronic device. Observation is less versatile than communication since some attributes of a person may not be readily observable, such as attitudes, awareness, knowledge, intentions, and motivation. Observation also might take longer since observers may have to wait for appropriate events to occur, though observation using scanner data might be quicker and more cost effective. Observation typically is more accurate than communication.

Personal interviews have an interviewer bias that mail-in questionnaires do not have. For example, in a personal interview the respondent's perception of the interviewer may affect the responses.

Questionnaire Design

The questionnaire is an important tool for gathering primary data. Poorly constructed questions can result in large errors and invalidate the research data, so significant effort should be put into the questionnaire design. The questionnaire should be tested thoroughly prior to conducting the survey.

Measurement Scales

Attributes can be measured on nominal, ordinal, interval, and ratio scales:

• Nominal numbers are simply identifiers, with the only permissible mathematical use being for counting. Example: social security numbers.
• Ordinal scales are used for ranking. The interval between the numbers conveys no meaning. Median and mode calculations can be performed on ordinal numbers. Example: class ranking
• Interval scales maintain an equal interval between numbers. These scales can be used for ranking and for measuring the interval between two numbers. Since the zero point is arbitrary, ratios cannot be taken between numbers on an interval scale; however, mean, median, and mode are all valid. Example: temperature scale
• Ratio scales are referenced to an absolute zero values, so ratios between numbers on the scale are meaningful. In addition to mean, median, and mode, geometric averages also are valid. Example: weight

Validity and Reliability

The validity of a test is the extent to which differences in scores reflect differences in the measured characteristic. Predictive validity is a measure of the usefulness of a measuring instrument as a predictor. Proof of predictive validity is determined by the correlation between results and actual behavior. Construct validity is the extent to which a measuring instrument measures what it intends to measure.

Reliability is the extent to which a measurement is repeatable with the same results. A measurement may be reliable and not valid. However, if a measurement is valid, then it also is reliable and if it is not reliable, then it cannot be valid. One way to show reliability is to show stability by repeating the test with the same results.

Attitude Measurement

Many of the questions in a marketing research survey are designed to measure attitudes. Attitudes are a person's general evaluation of something. Customer attitude is an important factor for the following reasons:

• Attitude helps to explain how ready one is to do something.
• Attitudes do not change much over time.
• Attitudes produce consistency in behavior.
• Attitudes can be related to preferences.

Attitudes can be measured using the following procedures:

• Self-reporting - subjects are asked directly about their attitudes. Self-reporting is the most common technique used to measure attitude.
• Observation of behavior - assuming that one's behavior is a result of one's attitudes, attitudes can be inferred by observing behavior. For example, one's attitude about an issue can be inferred by whether he/she signs a petition related to it.
• Indirect techniques - use unstructured stimuli such as word association tests.
• Performance of objective tasks - assumes that one's performance depends on attitude. For example, the subject can be asked to memorize the arguments of both sides of an issue. He/she is more likely to do a better job on the arguments that favor his/her stance.
• Physiological reactions - subject's response to a stimuli is measured using electronic or mechanical means. While the intensity can be measured, it is difficult to know if the attitude is positive or negative.
• Multiple measures - a mixture of techniques can be used to validate the findings, especially worthwhile when self-reporting is used.

There are several types of attitude rating scales:

• Equal-appearing interval scaling - a set of statements are assembled. These statements are selected according to their position on an interval scale of favorableness. Statements are chosen that has a small degree of dispersion. Respondents then are asked to indicate with which statements they agree.
• Likert method of summated ratings - a statement is made and the respondents indicate their degree of agreement or disagreement on a five point scale (Strongly Disagree, Disagree, Neither Agree Nor Disagree, Agree, Strongly Agree).
• Semantic differential scale - a scale is constructed using phrases describing attributes of the product to anchor each end. For example, the left end may state, "Hours are inconvenient" and the right end may state, "Hours are convenient". The respondent then marks one of the seven blanks between the statements to indicate his/her opinion about the attribute.
• Stapel Scale - similar to the semantic differential scale except that 1) points on the scale are identified by numbers, 2) only one statement is used and if the respondent disagrees a negative number should marked, and 3) there are 10 positions instead of seven. This scale does not require that bipolar adjectives be developed and it can be administered by telephone.
• Q-sort technique - the respondent if forced to construct a normal distribution by placing a specified number of cards in one of 11 stacks according to how desirable he/she finds the characteristics written on the cards.

Sampling Plan

The sampling frame is the pool from which the interviewees are chosen. The telephone book often is used as a sampling frame, but have some shortcomings. Telephone books exclude those households that do not have telephones and those households with unlisted numbers. Since a certain percentage of the numbers listed in a phone book are out of service, there are many people who have just moved who are not sampled. Such sampling biases can be overcome by using random digit dialing. Mall intercepts represent another sampling frame, though there are many people who do not shop at malls and those who shop more often will be over-represented unless their answers are weighted in inverse proportion to their frequency of mall shopping.

In designing the research study, one should consider the potential errors. Two sources of errors are random sampling error and non-sampling error. Sampling errors are those due to the fact that there is a non-zero confidence interval of the results because of the sample size being less than the population being studied. Non-sampling errors are those caused by faulty coding, untruthful responses, respondent fatigue, etc.

There is a tradeoff between sample size and cost. The larger the sample size, the smaller the sampling error but the higher the cost. After a certain point the smaller sampling error cannot be justified by the additional cost.

While a larger sample size may reduce sampling error, it actually may increase the total error. There are two reasons for this effect. First, a larger sample size may reduce the ability to follow up on non-responses. Second, even if there is a sufficient number of interviewers for follow-ups, a larger number of interviewers may result in a less uniform interview process.

Data Collection

In addition to the intrinsic sampling error, the actual data collection process will introduce additional errors. These errors are called non-sampling errors. Some non-sampling errors may be intentional on the part of the interviewer, who may introduce a bias by leading the respondent to provide a certain response. The interviewer also may introduce unintentional errors, for example, due to not having a clear understanding of the interview process or due to fatigue.

Respondents also may introduce errors. A respondent may introduce intentional errors by lying or simply by not responding to a question. A respondent may introduce unintentional errors by not understanding the question, guessing, not paying close attention, and being fatigued or distracted.

Such non-sampling errors can be reduced through quality control techniques.


Data Analysis - Preliminary Steps

Before analysis can be performed, raw data must be transformed into the right format. First, it must be edited so that errors can be corrected or omitted. The data must then be coded; this procedure converts the edited raw data into numbers or symbols. A codebook is created to document how the data was coded. Finally, the data is tabulated to count the number of samples falling into various categories. Simple tabulations count the occurrences of each variable independently of the other variables. Cross tabulations, also known as contingency tables or cross tabs, treats two or more variables simultaneously. However, since the variables are in a two-dimensional table, cross tabbing more than two variables is difficult to visualize since more than two dimensions would be required. Cross tabulation can be performed for nominal and ordinal variables.

Cross tabulation is the most commonly utilized data analysis method in marketing research. Many studies take the analysis no further than cross tabulation. This technique divides the sample into sub-groups to show how the dependent variable varies from one subgroup to another. A third variable can be introduced to uncover a relationship that initially was not evident.


Conjoint Analysis

The conjoint analysis is a powerful technique for determining consumer preferences for product attributes.


Hypothesis Testing

A basic fact about testing hypotheses is that a hypothesis may be rejected but that the hypothesis never can be unconditionally accepted until all possible evidence is evaluated. In the case of sampled data, the information set cannot be complete. So if a test using such data does not reject a hypothesis, the conclusion is not necessarily that the hypothesis should be accepted.

The null hypothesis in an experiment is the hypothesis that the independent variable has no effect on the dependent variable. The null hypothesis is expressed as H0. This hypothesis is assumed to be true unless proven otherwise. The alternative to the null hypothesis is the hypothesis that the independent variable does have an effect on the dependent variable. This hypothesis is known as the alternative, research, or experimental hypothesis and is expressed as H1. This alternative hypothesis states that the relationship observed between the variables cannot be explained by chance alone.

There are two types of errors in evaluating a hypotheses:

• Type I error: occurs when one rejects the null hypothesis and accepts the alternative, when in fact the null hypothesis is true.
• Type II error: occurs when one accepts the null hypothesis when in fact the null hypothesis is false.

Because their names are not very descriptive, these types of errors sometimes are confused. Some people jokingly define a Type III error to occur when one confuses Type I and Type II. To illustrate the difference, it is useful to consider a trial by jury in which the null hypothesis is that the defendant is innocent. If the jury convicts a truly innocent defendant, a Type I error has occurred. If, on the other hand, the jury declares a truly guilty defendant to be innocent, a Type II error has occurred.

Hypothesis testing involves the following steps:

• Formulate the null and alternative hypotheses.
• Choose the appropriate test.
• Choose a level of significance (alpha) - determine the rejection region.
• Gather the data and calculate the test statistic.
• Determine the probability of the observed value of the test statistic under the null hypothesis given the sampling distribution that applies to the chosen test.
• Compare the value of the test statistic to the rejection threshold.
• Based on the comparison, reject or do not reject the null hypothesis.
• Make the marketing research conclusion.

In order to analyze whether research results are statistically significant or simply by chance, a test of statistical significance can be run.


Tests of Statistical Significance

The chi-square ( c² ) goodness-of-fit test is used to determine whether a set of proportions have specified numerical values. It often is used to analyze bivariate cross-tabulated data. Some examples of situations that are well-suited for this test are:

• A manufacturer of packaged products test markets a new product and wants to know if sales of the new product will be in the same relative proportion of package sizes as sales of existing products.
• A company's sales revenue comes from Product A (50%), Product B (30%), and Product C (20%). The firm wants to know whether recent fluctuations in these proportions are random or whether they represent a real shift in sales.

The chi-square test is performed by defining k categories and observing the number of cases falling into each category. Knowing the expected number of cases falling in each category, one can define chi-squared as:

c² = å ( O_i - E_i )² / E_i

where

        O_i = the number of observed cases in category i,
        E_i = the number of expected cases in category i,
        k = the number of categories,
        the summation runs from i = 1 to i = k.

Before calculating the chi-square value, one needs to determine the expected frequency for each cell. This is done by dividing the number of samples by the number of cells in the table.

To use the output of the chi-square function, one uses a chi-square table. To do so, one needs to know the number of degrees of freedom (df). For chi-square applied to cross-tabulated data, the number of degrees of freedom is equal to

( number of columns - 1 ) ( number of rows - 1 )

This is equal to the number of categories minus one. The conventional critical level of 0.05 normally is used. If the calculated output value from the function is greater than the chi-square look-up table value, the null hypothesis is rejected.

ANOVA
Another test of significance is the Analysis of Variance (ANOVA) test. The primary purpose of ANOVA is to test for differences between multiple means. Whereas the t-test can be used to compare two means, ANOVA is needed to compare three or more means. If multiple t-tests were applied, the probability of a TYPE I error (rejecting a true null hypothesis) increases as the number of comparisons increases.

One-way ANOVA examines whether multiple means differ. The test is called an F-test. ANOVA calculates the ratio of the variation between groups to the variation within groups (the F ratio). While ANOVA was designed for comparing several means, it also can be used to compare two means. Two-way ANOVA allows for a second independent variable and addresses interaction.

To run a one-way ANOVA, use the following steps:

1. Identify the independent and dependent variables.
2. Describe the variation by breaking it into three parts  - the total variation, the portion that is within groups,  and the portion that is between groups (or among groups for more than two groups). The total variation (SS_total) is the sum of the squares of the differences between each value and the grand mean of all the values in all the groups.  The in-group variation (SS_within) is the sum of the squares of the differences in each element's value and the group mean. The variation between group means (SS_between) is the total variation minus the in-group variation (SS_total - SS_within).
3. Measure the difference between each group's mean and the grand mean.
4. Perform a significance test on the differences.
5. Interpret the results.

This F-test assumes that the group variances are approximately equal and that the observations are independent. It also assumes normally distributed data; however, since this is a test on means the Central Limit Theorem holds as long as the sample size is not too small.

ANOVA is efficient for analyzing data using relatively few observations and can be used with categorical variables. Note that regression can perform a similar analysis to that of ANOVA.


Discriminant Analysis

Analysis of the difference in means between groups provides information about individual variables, it is not useful for determine their individual impacts when the variables are used in combination. Since some variables will not be independent from one another, one needs a test that can consider them simultaneously in order to take into account their interrelationship. One such test is to construct a linear combination, essentially a weighted sum of the variables. To determine which variables discriminate between two or more naturally occurring groups, discriminant analysis is used. Discriminant analysis can determine which variables are the best predictors of group membership. It determines which groups differ with respect to the mean of a variable, and then uses that variable to predict new cases of group membership. Essentially, the discriminant function problem is a one-way ANOVA problem in that one can determine whether multiple groups are significantly different from one another with respect to the mean of a particular variable.

A discriminant analysis consists of the following steps:

1. Formulate the problem.
2. Determine the discriminant function coefficients that result in the highest ratio of between-group variation to within-group variation.
3. Test the significance of the discriminant function.
4. Interpret the results.
5. Determine the validity of the analysis.

Discriminant analysis analyzes the dependency relationship, whereas factor analysis and cluster analysis address the interdependency among variables.


Factor Analysis

Factor analysis is a very popular technique to analyze interdependence. Factor analysis studies the entire set of interrelationships without defining variables to be dependent or independent. Factor analysis combines variables to create a smaller set of factors. Mathematically, a factor is a linear combination of variables. A factor is not directly observable; it is inferred from the variables. The technique identifies underlying structure among the variables, reducing the number of variables to a more manageable set. Factor analysis groups variables according to their correlation.

The factor loading can be defined as the correlations between the factors and their underlying variables. A factor loading matrix is a key output of the factor analysis. An example matrix is shown below.

	Factor 1	Factor 2	Factor 3
Variable 1
Variable 2
Variable 3
Column's Sum of Squares:

Each cell in the matrix represents correlation between the variable and the factor associated with that cell. The square of this correlation represents the proportion of the variation in the variable explained by the factor. The sum of the squares of the factor loadings in each column is called an eigenvalue. An eigenvalue represents the amount of variance in the original variables that is associated with that factor. The communality is the amount of the variable variance explained by common factors.

A rule of thumb for deciding on the number of factors is that each included factor must explain at least as much variance as does an average variable. In other words, only factors for which the eigenvalue is greater than one are used. Other criteria for determining the number of factors include the Scree plot criteria and the percentage of variance criteria.

To facilitate interpretation, the axis can be rotated. Rotation of the axis is equivalent to forming linear combinations of the factors. A commonly used rotation strategy is the varimax rotation. Varimax attempts to force the column entries to be either close to zero or one.


Cluster Analysis

Market segmentation usually is based not on one factor but on multiple factors. Initially, each variable represents its own cluster. The challenge is to find a way to combine variables so that relatively homogenous clusters can be formed. Such clusters should be internally homogenous and externally heterogeneous. Cluster analysis is one way to accomplish this goal. Rather than being a statistical test, it is more of a collection of algorithms for grouping objects, or in the case of marketing research, grouping people. Cluster analysis is useful in the exploratory phase of research when there are no a-priori hypotheses.

Cluster analysis steps:

1. Formulate the problem, collecting data and choosing the variables to analyze.
2. Choose a distance measure. The most common is the Euclidean distance. Other possibilities include the squared Euclidean distance, city-block (Manhattan) distance, Chebychev distance, power distance, and percent disagreement.
3. Choose a clustering procedure (linkage, nodal, or factor procedures).
4. Determine the number of clusters. They should be well separated and ideally they should be distinct enough to give them descriptive names such as professionals, buffs, etc.
5. Profile the clusters.
6. Assess the validity of the clustering.

Marketing Research Report

The format of the marketing research report varies with the needs of the organization. The report often contains the following sections:

• Authorization letter for the research
• Table of Contents
• List of illustrations
• Executive summary
• Research objectives
• Methodology
• Results
• Limitations
• Conclusions and recommendations
• Appendices containing copies of the questionnaires, etc.

Concluding Thoughts

Marketing research by itself does not arrive at marketing decisions, nor does it guarantee that the organization will be successful in marketing its products. However, when conducted in a systematic, analytical, and objective manner, marketing research can reduce the uncertainty in the decision-making process and increase the probability and magnitude of success.