Pilot experimental design - Orthogonal design useful?

This forum is for posts that specifically focus on Ngene.

Moderators: Andrew Collins, Michiel Bliemer, johnr

Pilot experimental design - Orthogonal design useful?

Postby JasonOng » Thu Nov 01, 2018 1:20 am

hi,

If there is no information to inform a prior for a d-efficient design, is there any advantage of creating an orthogonal design for the pilot and then using those priors to inform the d-efficient design for the main study?
From reading this forum, it seems most are using d-efficient design with uninformative priors (+/- use of signs for certain attributes when appropriate) for the pilot.
Or does it make no practical difference in the end?
thanks for your insight into this.

Kind regards,
Jason Ong.
JasonOng
 
Posts: 4
Joined: Mon Oct 29, 2018 12:50 am

Re: Pilot experimental design - Orthogonal design useful?

Postby Michiel Bliemer » Thu Nov 01, 2018 7:36 am

In specific cases, a D-efficient design with zero priors will give you an orthogonal design, so there is a correlation between orthogonal designs and efficient designs with uninformative priors.

The main disadvantage of orthogonal designs is that they may include strictly dominant alternatives. In Ngene we can automatically remove such problematic choice tasks in a D-efficient design with signed priors (- or +) by adding an asterisk (*) after alternative names in the ;alts command. For this reason, you may want to use a D-efficient design with small positive or negative priors. If the signs of the priors are unknown, then dominant alternatives are not a concern and using an orthogonal design will be a fine choice since it covers the attribute level space nicely, but using a D-efficient design with zero priors will work equally well.

Michiel
Michiel Bliemer
 
Posts: 598
Joined: Tue Mar 31, 2009 4:13 pm

Re: Pilot experimental design - Orthogonal design useful?

Postby JasonOng » Thu Nov 01, 2018 5:28 pm

Dear Michiel

Thanks for your prompt and very helpful answer.
A couple of follow up questions

1) I tried to run my code below adding the *, but it comes up with this error.

A valid initial random design could not be generated after approximately 10 seconds. In this time, of the 206199 attempts made, there were 0 row repetitions, 363 alternative repetitions, and 205836 cases of dominance. There are a number of possible causes for this, including the specification of too many constraints, not having enough attributes or attribute levels for the number of rows required, and the use of too many scenario attributes. A design may yet be found, and the search will continue for 10 minutes. Alternatively, you can stop the run and alter the syntax.

The only variable that has an order is cost... all my other attributes are categorical. So, is adding the * not wise as technically there aren't any dominant alternatives (i.e. my non-cost attributes are not ordinal in nature... but I also want to get rid of repetitions which (please correct me if I am wrong), adding the * helps with identifying repetitions?

Code: Select all
Design
; alts = A*, B*, C
; rows = 24
; eff = (mnl,d) 
; block =2
; con
; model:
U(A) = b1[-0.0001]*Cost[500,1000,2000,0] +
       b2.effects[0|0|0|0|0|0|0]*Loc[1,2,3,4,5,6,7,0] +
       b3.effects[0|0]*Test[1,2,0] +
       b4.effects[0|0|0]*Person[1,2,3,0] +
       b5.effects[0]*Access[1,0]/
U(B) = b1*Cost +
       b2.effects*Loc +
       b3.effects*Test +
       b4.effects*Person +
       b5.effects*Access/
U(C) = b6[0]
$


2) If I want to analyze my data later using Latent Class Analysis as well as RPL, should I use the rppanel function for my experimental design instead of mnl? Is there an equivalent function for LCA or is MNL okay for this?

Thanks again for your help.

Best regards,
Jason Ong
JasonOng
 
Posts: 4
Joined: Mon Oct 29, 2018 12:50 am

Re: Pilot experimental design - Orthogonal design useful?

Postby Michiel Bliemer » Mon Nov 05, 2018 8:20 am

1) Put the prior of cost to zero, that will resolve the issue and Ngene will remove repetitions of the same choice task.

2) Ngene cannot optimise for latent class models. Optimising for rppanel is possible but takes very long in the optimisation. In general, it is fine to optimise for MNL and use the same data for estimating an rppanel model later. Instead of optimising for rppanel, you can evaluate designs (optimised for mnl) for rppanel in Ngene. Note that this is generally only useful if you have some informative priors.

Michiel
Michiel Bliemer
 
Posts: 598
Joined: Tue Mar 31, 2009 4:13 pm


Return to Ngene

Who is online

Users browsing this forum: No registered users and 2 guests

cron