Choice tasks and sample size for mixed logit

[sab]

Hello, I'm new to DCEs and have been using this forum to improve my understanding. Could I check a few things about my design:

6 attributes, each with 4 levels (2 continuous, 4 categorical - effects coded)
Single profile choice tasks - binary response mechanism (Yes/No)
Parameters to estimate for mnl model = 15 (2x1 + 4x3 + 1 constant)

I have no prior estimates of the coefficients so will run a pilot study first to get these. I'm planning to create a fractional factorial design using a mnl model. I expect there to be preference heterogeneity, so for the main study I would use a mixed logit or latent class model, depending on model performance.

1) Does this mean for generating the fractional factorial design at the pilot stage and deciding number of choice tasks, I should work on the assumption that there will be double the amount of parameters to estimate - because there will be a mean and SD for each coefficient for a mixed logit model? In which case, I would need a minimum of 30 choice tasks (s>=k/(j-1))? Presumably I would then need to multiply this by 3 to ensure enough variation?

2) I understand that sample size estimates aren't very meaningful without known priors, but I want to understand the calculation as I have a limited budget. I have used Orme's approximate formulae at this stage (N>500c/(t*a), where c = largest number of levels, t=number of choice tasks and a = number of alternatives). I know from pre-testing that participants can feasbily complete about 12 choice tasks. My understanding is that the minimum sample size (e.g. N> 500*4 / 12 * 2, so N>83.3) applies to each block of the DCE? So, N=> 84 * number of blocks?

3) My other research question involves looking at income group differences: 40% of the sample will be in income group Level 1 and then 20% will be in each of the 3 other income group levels. I am confused about whether exploring this impacts the necessary sample size. To explore preference differences between the income groups, do I need to treat each income group as a different version of the DCE (i.e. 84 ppts for each income group?), and then would at least 84 need to be in each income group within each block of the DCE?

Thanks very much in advance.

[Michiel Bliemer]

1) Yes I usually consider double the number of parameters for a mixed logit model, although the minimum design size is based on a single respondent filling out all choice tasks in the experiment and therefore if multiple respondents fill out the survey you could estimate heterogeneity without increasing the number of rows in the design. For more complex models like latent class and mixed logit I usually want more variation in the data and therefore doubling the number of rows is not a bad idea.

2) I have never used Orme's formulae and given that it is merely a rule of thumb it is unclear how accurate or useful it is, so I will have to refer to Orme. The only accurate calculation is based on t-ratios, priors, and estimated standard errors, which is the ones that Ngene reports, but that indeed relies on reliable priors.

3) No you can use the same design for all income groups, you can test for income differences in your econometric model during the model estimation phase.

Michiel

[sab]

Hi Michiel,

Thanks very much for this. I want to check I've understood point 1 re minimum design size and rows. To analyse the data using a mixed logit model, I would need 30 choice tasks as a minimum. Then I would have to multiply this by 3 to give enough variation in the data, so in this case I would need 90 choice tasks in total, blocked to give a feasible number for each participant to complete? I just want to check this because I don't think I have the budget to create a DCE of this size, in which case I will rethink the number of attributes/levels to be included.

[Michiel Bliemer]

You theoretically only need 15, not 30, because multiple respondents give answers to the 15 choice tasks and hence heterogeneity can be picked up without increasing the number of rows in the design. So 15*3 = 45 rows would have sufficient variation.

I do not understand you do not have "have the budget to create a DCE of this size". The cost of a DCE is the same, no matter how many rows you have in your design. The design size merely refers to the number of unique choice tasks (rows) you have in your design, but you still only give a fixed subset to each respondent and it is the SAMPLE SIZE (not design size) that matters for cost, i.e. the number of respondents.

Michiel

[sab]

Thanks very much Michiel, that's good to know re 45 rows. My concern was that including 90 rows would require several blocks of the DCE and therefore require more participants per block. However I understand that I won't be able to calculate this until I have known priors from the pilot. Thanks again!

[Michiel Bliemer]

You do not necessarily need more participants per block, having only 1 participant suffices. In RP data all attribute levels may be individual-specific, so each 'block' would be unique to a respondent and you are still able to estimate any model

Michiel

[sab]

Ah I see, thanks Michiel!