Nr. of choice sets per respondent

[amela_95]

Hi everyone,

I'm attempting to set up two discrete choice experiments for the first time and have questions regarding the number of choice sets per respondent, for which I would appreciate another opinion. Both DCEs are part of the same survey involving approximately 3,000 respondents. Since the survey encompasses other experiments as well, I'm constrained in the number of choice sets I can present to participants.

My first experiment is relatively straightforward, featuring 2 unlabeled choices with 8 attributes, each with two levels. I'm utilizing an orthogonal optimal design, yielding a D-optimality of 100%. The number of choice sets is 24.

Q1) Number of choice sets per respondent: Is it feasible to randomly assign only one choice set out of the 24 to each respondent? What are the drawbacks of this approach?

The second experiment closely resembles the first one, except there are now 9 attributes with two levels.

Q2) Number of choice sets per block: If I employ blocking for this experiment, would it suffice to have only 3 choice sets per block? What about two choice sets per block?

Thanks so much for your help and advise.
All the best,
Amela

[Michiel Bliemer]

There are several drawbacks of using a small number of choice sets per respondent, the most important one being learning effects. In the first choice task, the respondent is often still trying to understand the choice task and only after a few choice tasks they better understand their own behaviour and what matters to them.

Giving only a single choice task to a respondent is not recommended if you would like to estimate a choice model with preference heterogeneity (e.g. latent class or mixed logit) as this creates a cross-sectional data set instead of a panel data set. Having at least 2 observations per respondent yields more reliable parameter estimates, especially with respect to standard deviations in the mixed logit model.

Given that it takes quite a bit of time to explain a choice experiment to respondents, including all alternatives and attributes, where showing a choice task is actually a relatively short amount of time, giving only 1 or 2 choice tasks does not seem a good approach to me. The additional time to show an extra choice task is small compared to the time it takes to explain all the alternatives, attributes, and the choice context (which typically remain the same across choice tasks).

If the choice is between 1, 2, or 3, then I would recommend 3, but 2 choice tasks is possible. I would not recommend 1.

Michiel

[amela_95]

Thanks so much for your answers.

This is very interesting and helpful. The first DCE is actually a replication of an existing study that randomly allocated 1 choice task per respondent. She used a logit model for the estimation, and a heterogeneous analysis by gender was conducted. But I see how this is suboptimal and better results would be achieved if respondents evaluated more than one choice tasks. I did not think about it in that way.

Thanks so much