MaxDiff Questions

MaxDiff asks participants to choose the most and least preferred item from several small sets. Use MaxDiff to compare features, claims, benefits, or messages that participants might otherwise rate similarly. Optional conversational probes explore the reasoning behind those choices.

​

Key concepts

• Item: a concept being compared, with a text label and optional image.
• Set: the items shown together for one best-and-worst choice.
• Preference share: a model estimate of relative preference within the tested item pool. Shares sum to 100%; they are not purchase probabilities or market shares.
• Utility: a relative preference score. Higher values mean stronger estimated preference within the same question.

​

Setting up a MaxDiff question

1. Add a MaxDiff question to the topic guide.
2. Write the question and add the item labels and any images. Check translations for each interview language.
3. Set the positive and negative choice labels. The defaults are Most appealing and Least appealing; use labels that match the comparison you want participants to make.
4. Set Items shown per set to an integer from 3 to 7; the default is 4. The item library must contain at least that many items.
5. Review Sets per participant, which is calculated automatically as ceil(3 × number of items ÷ items per set). Twelve items shown four at a time produce nine sets.
6. Configure probing and test the interview, including image readability and the time needed to complete all sets.

The task generator uses previous assignments to improve item, pair, and position balance. Balancing is an optimization, not a promise that every item or pair has identical exposure. Increasing the item pool increases participant work even when each set stays small.

​

Probing settings

Use the question’s follow-up control to choose no follow-up questions or a probing depth. Depth applies per probed item, so deeper probing can add substantial interview time. The probing plan targets up to three distinct items: a strongest item, a weakest item, and a middle item. The middle selection also considers which items have received less probing in prior interviews. The five-prior-participant threshold applies to comparisons with earlier participants; it is not a requirement for selecting a middle probe. Probe targets come from the participant’s observed choices, not from the later Bayesian model. Ties and limited choice information can affect which items are selected. Treat explanations as qualitative evidence for those items, not as a reason supplied for every item in the library.

​

Participant experience

Participants work through the assigned sets, selecting a positive and negative choice for each. Configured conversational probes follow the choice task. Review the participant experience for device requirements and interruption behavior. Keep the item pool and wording stable during collection. Changes can affect comparability and invalidate an existing analysis; use editing and history before changing a launched study.

​

Analysis

Open the MaxDiff question in Question Analysis. The chart offers Preference share, Utility scores, and Best vs worst picks. Bayesian preference analysis requires at least 10 eligible completed interviews with usable MaxDiff choices. This is a processing threshold, not a guarantee that the sample is sufficient for a research decision. Test, hidden, deleted, and incomplete interviews are excluded from the model dataset. A skipped question or task without both choices does not supply a completed choice task. Analysis runs in the background. While the model is pending, running, or unavailable, use the observed pick counts. Follow the analysis control to run or retry the calculation when offered; do not interpret missing estimates as zero preference.

​

Per-item metrics

Best vs worst picks shows observed selections. Consider how often each item was shown when interpreting those counts. Preference shares and utilities are model estimates rather than raw vote percentages. For the full sample, the chart can show a 95% posterior interval around an estimate. This describes uncertainty under the fitted model. It does not establish that the sample represents the target population or guarantee that a small ranking difference is meaningful.

​

Per-respondent utilities

The model stores interview-level utility estimates. Applying an interview filter does not fit a new model to that segment: the filtered view converts stored posterior-mean utilities to shares and averages over the matching interviews. The chart labels this a Filtered approximation. Full-sample intervals are omitted because they do not describe the filtered segment; extreme shares can be overstated by this approximation. Organization admins can choose Download raw data for a CSV containing separate task-choice and respondent-utility sections. This export covers the question’s eligible dataset, not just the current chart filter. Utilities are included only when the stored analysis has completed and matches the current dataset; otherwise the choices remain available without those utility rows.

​

Statistical methodology

Conveo fits a hierarchical Bayesian choice model: individual preference estimates share information through a population distribution. Each task models a best choice followed by a worst choice among the remaining items. Full-sample shares aggregate across respondents and posterior draws. The fitting process checks convergence and can retry with different fitting settings or a simplified model. A simplified or degraded result can carry a notice; review that notice and uncertainty before relying on precise differences. Hierarchical pooling helps estimate individual preferences from limited observations, but does not remove sampling bias, poor item wording, or uncertainty. Shares are relative to the tested alternatives. Adding or removing alternatives changes the comparison, so shares from different item pools should not be treated as directly interchangeable.

​

Practical guidance

​

Sample size

Plan for the decisions and segment comparisons you need. Ten interviews unlock calculation; they do not validate a ranking. Small segments, sparse exposures, and closely matched items need particular caution. There is no universal sample size that guarantees reliable results.

​

Item count

Use a focused pool of distinct alternatives. The set count grows with the item count, and probing adds time afterward. Preview the full experience rather than judging burden from a single set.

​

Writing items

Use clear, comparable descriptions of similar length. Avoid combining several benefits in one item or mixing broad concepts with narrowly specified features. If images matter, keep their presentation consistent and test them on participant devices.

​

When MaxDiff isn’t the right fit

MaxDiff measures relative preference among supplied alternatives. It does not directly measure willingness to pay, absolute demand, or preference for an omitted alternative. Use open questions to discover alternatives and pricing questions for price research.

​

Integrations

Link the ranking to qualitative evidence in Question Analysis, use interview filters to explore relevant groups, and validate interpretations against participants’ actual explanations before including them in reports.