modelling minimal design DCEs

[jveld]

My colleagues and I conducted a DCE with a minimal D-efficient design. In other words we used as less choice sets as possible, because our topic was very complicated and we thought that answering our choice sets was highly cognitively demanding.

Currently we are modelling our data and I have a few questions. Because we included 9 choice sets, from a statistical point of view we can only include 9 parameters in our analysis. In our case, given the number of attribute levels and their coding, this would mean we can only conduct a MNL model. However we do have 550 respondents so we tried modelling an RP model. Nlogit could also provide an RP model with proper model fit (better fit compared to the MNL model based on the AIC and Log likelihood) and the retrieved parameter estimates make sense and do not differ very much from the MNL results.

- Would you advise to present these RP model results even though we technically do not have enough degrees of freedom (and then in the discussion report this limitation)? Or would you advise to stick with the MNL results (this means that preference heterogeneity and panel structure of our data could not be taken into account)

Although Nlogit fitted a proper RP model with all attributes included as random parameters, the constant term (i.e., modelling either of the two program options against opt-out) connot be modelled as random. If this constant is included as a random parameter the model 'explodes'. The constant gets a estimate of about 12 with a SE of about 2 (SD = 10 & SE = 1.5). This obviously is an indication that this model cannot be fitted.

- What could be explanations for this? Does this imply multicolinearity of the constant with other attribute estimates? Can we 'just' conclude that the constant should be held fixed and model an RP with only the attributes set as random? Is this due to our minimal design? Or are there other explanations?

Hope you can answer these questions

Kind regards
Jorien

[johnr]

Hi Jorien

I would suggest using the RP model. Firstly, the degrees of freedom necessary for a design is S*(J-1) => K where S are the number of choice sets, J the number of alternatives and K the number of attributes. It is not clear from your post whether the design is binary (J = 2) in which case the d.f. will equal K as you suggest. If it is ternary, then the d.f. would be 9*(3-1) = 18. Note that this is simply required to invert the Hessian - any less rows, and the inversion will result in a singularity issue.

Hence, if I have two attributes, time and cost and linear parameter estimates, I could in theory have only one choice task assuming J = 3 and no constants. That is

S J TT C
1 1 4 5
1 2 10 3
1 3 18 1

Can be inverted if I have only a parameter for TT and C.

This assumes a single respondent also, however assuming more respondents will not necessarily solve the problem. I assume I want to estimate a constant, so I repeat the choice task.

S J Con TT C
1 1 1 4 5
1 2 0 10 3
1 3 0 18 1
2 1 1 4 5
2 2 0 10 3
2 3 0 18 1

The Hessian for this will not invert either.

In terms of the exploding constant, this might or might not be a design issue. The random constant is the same as assuming heteroskedastic error terms which may not hold in this data. It is an empirical issue. If you believe it is a d.f. issue caused by the design, try treating the parameters associated with the design attributes as fixed parameters and estimating a random constant term. This will not be a perfect test as the result may be picking up heterogeneity linked to the attributes indirectly, but if it fails, it might indicate that there is a problem in assuming a model with a heteroskedastic error term.

John