appropriate sample size, blocks

[hblee]

Dear all,

I have questions about appropriate sample sizes and blocks. Let me describe my survey:

Choice tasks and possible sample size

I have 60 choice tasks.
I will possibly have around 500 respondents.

Three alternatives

Alt1: product (unlabeled)
Alt2: product (unlabeled)
Alt3: neither (I would not choose option A and B)

Three attributes

Att1: a generic attribute measured by percentage, two levels, [20%, 30%]
Att2: a generic discrete attribute, five levels, [0, 1, 2, 3, 4]
Att3: price, three levels, [$20, $30, $40]

Case 1. If I use 4 blocks, each respondent will answer 15 choice tasks (=60/4). Each block has 125 respondents (=500/4).

Case 2. If I use 5 blocks, each respondent will answer 12 choice tasks (=60/5). Each block has 100 respondents (=500/5).

Model

I plan to use a mixed multinomial logit model (MMNL). I have an interest in main effects. But I also plan to consider the interaction between an attribute and a respondent characteristic (respondent characteristic is a continuous variable).

Questions

Question 1. Are Cases 1 and 2 good regarding the sample size and the number of blocks?

Question 2. If both Cases 1 and 2 are reasonable, which one is preferred?


Any feedback would be appreciated. Thank you.

(For information, about the same survey, I posted another topic in the forum: Choice experiments - Ngene. But I am posting this topic separately here because this question is about choice experiments in general).

[Michiel Bliemer]

The required number of respondents (= sample size) depends on (i) available budget, and (ii) importance of each attribute. If attributes are not very important to respondents, then you would only be able to obtain statistically significant parameter estimates with a very large sample size, whereas for important attributes (such a price/cost) with a sufficient range only a few responsible would be required. The larger the sample size, the better, and typically I try to get at least 1000 respondents, but 500 is a decent amount.

Regarding the number of blocks, you need to consider fatigue. Can respondents handle 15 choice tasks? Or is 12 more reasonable? If you ask 15 in a row, respondents may get fatigued and start answering more randomly towards the end, so you could consider splitting the choice tasks in two portions, 8+7 with other types of questions in the survey in between. Your choice tasks are relatively simple so I think that 15 would be okay, although some pre-testing could tell you if it is too much or not, and maybe switch to 12. For estimating a panel mixed logit model, it does not really matter whether respondents are given 12 or 15 choice tasks, but 12 would definitely be enough to pick up enough information from each respondent. Note that Case 1 would give you more data since 125*4*15 = 7500 choice observations, while 100*5*12 = 6000 choice observations. I think that a fairer comparison would be if Case 2 also had 125 respondents per block such that the total number of choice observations is the same. In that case, I would select Case 2 as there is less risk of fatigue.

Michiel

[hblee]

I really appreciate your detailed explanation. Your explanation is very helpful.

Thank you for your kind response.