choice-metrics.com

Dear all,

I'm currently designing my first DCE, and I have a question about blocking.

The plan for our DCE (so far) is as follows:

- 7 attributes, each with 5 levels
- From preliminary discussions with survey companies, we think we have a budget for around 1,000 online participants
- From the literature that I've read, it seems that 12 choice sets per participant is appropriate in terms of maximising data collection whilst not incurring responder bias
- We have data from a previous pilot study (n=171), so are hoping to use a Bayesian efficient design to take into account these priors
- We plan to use multinomial logit and mixed logit regression models to analyse the data
- We will be interested in main effects only, and not interaction terms

A recently published study very similar to ours (in terms of both the number of attributes and levels and type of analysis they conducted) divided their survey into 20 blocks of 12 choice sets. If we were to use this level of blocking, we would have approximately 50 respondents per choice pair, which from the literature I've read seems like a sufficient number.

However, I was wondering if anyone could point me in the direction of any studies that could guide me in terms of generating the optimum number of blocks given the proposed design of our DCE? I've been struggling to find studies in this area so far....

Any help regarding this would be most appreciated.

Apologies in advance if I haven't provided enough information, have used some incorrect terminology or have been unclear about anything - I'm still trying to get my head around this area!

Best wishes,

Tom

The question is, how large does your design need to be to have sufficient variation. With 20 blocks of 12 choice tasks you need a design with 240 rows, which is quite large. There is nothing wrong with such a large design, but most people generally only have a handful of blocks. If you have a very large number of coefficients to estimate, a large design is necessary. If not, then a smaller design would have sufficient variation.

The question of variation in your data set was discussed by Sandor and Wedel (2005), where they give different choice tasks to different respondents (called a heterogeneous design, essentially using multiple blocks) and compare this to a design where all respondents face the same choice tasks (called a homogeneous design, essentially a single block).

https://journals.sagepub.com/doi/10.1509/jmkr.42.2.210.62285

Kessels (2016) compared homogeneous designs and heterogeneous designs, and found that designs do not need to be very large to contain sufficient variation and information.

https://www.sciencedirect.com/science/article/abs/pii/S1755534515300889

A large number of blocks is not an problem, so it never hurts to have more blocks, but you may not need a large number of blocks. Note that the number of respondents per block is not relevant, many well-known academics give random choice tasks to respondents, essentially a very large design with only 1 respondent per block. It is the size of the design (number of choice tasks) that matters, not really the number of blocks, unless you want to compare behaviour across blocks where each block for example considers a different scenario.

Michiel

Hi Michiel,

Thank you very much for your detailed response.

I'll have look at the articles you have attached and think about it again.

If I have any more questions regarding this I will post them on this thread.

Best wishes,

Tom