Pilot experimental design - Orthogonal design useful?

[JasonOng]

hi,

If there is no information to inform a prior for a d-efficient design, is there any advantage of creating an orthogonal design for the pilot and then using those priors to inform the d-efficient design for the main study?
From reading this forum, it seems most are using d-efficient design with uninformative priors (+/- use of signs for certain attributes when appropriate) for the pilot.
Or does it make no practical difference in the end?
thanks for your insight into this.

Kind regards,
Jason Ong.

[Michiel Bliemer]

In specific cases, a D-efficient design with zero priors will give you an orthogonal design, so there is a correlation between orthogonal designs and efficient designs with uninformative priors.

The main disadvantage of orthogonal designs is that they may include strictly dominant alternatives. In Ngene we can automatically remove such problematic choice tasks in a D-efficient design with signed priors (- or +) by adding an asterisk (*) after alternative names in the ;alts command. For this reason, you may want to use a D-efficient design with small positive or negative priors. If the signs of the priors are unknown, then dominant alternatives are not a concern and using an orthogonal design will be a fine choice since it covers the attribute level space nicely, but using a D-efficient design with zero priors will work equally well.

Michiel

[JasonOng]

Dear Michiel

Thanks for your prompt and very helpful answer.
A couple of follow up questions

1) I tried to run my code below adding the *, but it comes up with this error.

A valid initial random design could not be generated after approximately 10 seconds. In this time, of the 206199 attempts made, there were 0 row repetitions, 363 alternative repetitions, and 205836 cases of dominance. There are a number of possible causes for this, including the specification of too many constraints, not having enough attributes or attribute levels for the number of rows required, and the use of too many scenario attributes. A design may yet be found, and the search will continue for 10 minutes. Alternatively, you can stop the run and alter the syntax.

The only variable that has an order is cost... all my other attributes are categorical. So, is adding the * not wise as technically there aren't any dominant alternatives (i.e. my non-cost attributes are not ordinal in nature... but I also want to get rid of repetitions which (please correct me if I am wrong), adding the * helps with identifying repetitions?

Code: Select all

Design
; alts = A*, B*, C
; rows = 24
; eff = (mnl,d) 
; block =2
; con
; model:
U(A) = b1[-0.0001]*Cost[500,1000,2000,0] +
       b2.effects[0|0|0|0|0|0|0]*Loc[1,2,3,4,5,6,7,0] +
       b3.effects[0|0]*Test[1,2,0] +
       b4.effects[0|0|0]*Person[1,2,3,0] +
       b5.effects[0]*Access[1,0]/
U(B) = b1*Cost +
       b2.effects*Loc +
       b3.effects*Test +
       b4.effects*Person +
       b5.effects*Access/
U(C) = b6[0]
$

2) If I want to analyze my data later using Latent Class Analysis as well as RPL, should I use the rppanel function for my experimental design instead of mnl? Is there an equivalent function for LCA or is MNL okay for this?

Thanks again for your help.

Best regards,
Jason Ong

[Michiel Bliemer]

1) Put the prior of cost to zero, that will resolve the issue and Ngene will remove repetitions of the same choice task.

2) Ngene cannot optimise for latent class models. Optimising for rppanel is possible but takes very long in the optimisation. In general, it is fine to optimise for MNL and use the same data for estimating an rppanel model later. Instead of optimising for rppanel, you can evaluate designs (optimised for mnl) for rppanel in Ngene. Note that this is generally only useful if you have some informative priors.

Michiel