Error in ood design

[miq]

Hi Guys,

I encountered the 'Something went unexpectedly wrong' error message for the following design:

design
; alts = alt1, alt2, alt3
; rows = 4
; orth = ood
; con
; model:
U(alt1) = bsq*SQ[-1] /
U(alt2) = bA1*A1[1,0]
+ bA2*A2[1,0]
+ bA3*A3[1,0]
+ bC*C[-0.25,-0.75,-1.25,-1.75] /
U(alt3) = bA1*A1
+ bA2*A2
+ bA3*A3
+ bC*C $

Replacing
U(alt1) = bsq*SQ[-1] /
with
U(alt1) = bsq /
works well.

Best,

Mik

[johnr]

Hi Mik

OODs are sequentially generated orthogonal designs. They work by constructing an orthogonal design for the first alternative, and then using what are known as design generators, constructing subsequent alternatives based on the design for the first alternative. The design algorithm does not account for status quo or no choice alternatives and all alternatives must have the same attributes and levels. So in your syntax, Alt1 is redundant. Further, the algorithm converts the levels internally to be orthonormal coded (a form of non-linear coding assumed by Street and Burgess). This means that the degrees of freedom for the design is sumK Lk-1 where Lk is the number of levels associated with attribute k. In your case, A1, A2 and A3 have L = 2 or 2 -1 = 1 d.f. each and C has 4 - 1 = 3 d.f.

Overall, the design therefore has 1 +1 +1 +3 = 5 d.f. however the d.f. available to the design is S*(J - 1) => K where S is the number of tasks and J the number of alternatives, if you now assume only 2 alts (ignore the SQ alternative) then you have 4 *(2 -1) = 4 < = 5. Hence, the design is not identified in this case. You will need to increase the number of rows. This works (but not very good - if you increase it to 40, you get a better design)

design
; alts = alt2, alt3
; rows = 8
; orth = ood
; con
; model:
U(alt2) = bA1*A1[1,0]
+ bA2*A2[1,0]
+ bA3*A3[1,0]
+ bC*C[-0.25,-0.75,-1.25,-1.75] /
U(alt3) = bA1*A1
+ bA2*A2
+ bA3*A3
+ bC*C $

John

[miq]

Hi John,

Thank you for very comprehensive and thorough reply.

Does this mean I can never have OOD design which would take ASCs into account or this only holds for ASCs for opt-out/sq alternatives?

Regarding the degrees of freedom - I know the formula and in fact NGENE warns about this and switches to more rows. I have 2 follow up questions though, as I noted NGENE does not warn about not enough d.f. for MXL models (i.e. seems to ignore d.f. required to identify s.d. parameters):
- So the SQ alternative should never be counted in calculating the degrees of freedom even if I generate e.g., d-efficient design for rppanel?
- On a similar note, should I somehow account for blocking to make a rppanel model identified or is the only thing that matters the total number of rows?

Thank you very much in advance for help!

Best,

Mik

[Michiel Bliemer]

As John mentioned, OODs (Street and Burgess designs) are quite restrictive.
They can only be used when:
1. The alternatives are completely generic (with the same alternatives and the same attribute levels across alternatives),
and also implicitly assumes that:
2. Your coefficients are orthonormally coded, and
3. Your priors are all equal to zero.

Note that a status quo alternative is a labelled alternative and as such an OOD design does not exist. Removing your SQ alternative may help, but if you are not assuming orthonormal coding in your model estimations and if the coefficients are not zero, you loose efficiency.

You can use status quo alternatives in D-efficient designs, you can also set the priors to zero or very small numbers (i.e. 0.00001 or -0.00001) to indicate the sign of the coefficients (if known), such that Ngene can automatically remove dominant alternatives (using alts = SQ, alt2*, alt3*).

The degrees of freedom should be automatically calculated taking the standard deviations of the random coefficients in a mixed logit model into account. Blocking has no influence on the degrees of freedom. If you include a status quo alternative, you need less rows, as the rule is:
number of parameters to estimate (including constants and standard deviations) <= (number of alternatives - 1) * number of rows

Including a status quo increases the number of alternatives, and therefore increases the amount of information retrieved from each choice observations.

Michiel

[miq]

Hi Michiel,

Thank you - this helps!

Mik