Eff design with 0 priors really better than a FF design?

[rich_imr]

Greetings,

The Ngene manual explains that "the wider the attribute level range, the higher the efficiency of the design can be (p 105)." I assume this means, in efficient designs, Ngene chooses attribute levels for alternatives as to maximize level difference across alternatives and choice sets in the design. I've also read in a few board posts that a fractional factorial design (FFD) assumes 0 for the parameter priors and exactly equates to an efficient design for an MNL model assuming 0 for the parameter priors and D-error as the optimality criterion.

Questions:
1. Will an ;eff = (mnl,d) design with 0 priors that is orthogonal still be more efficient than a orthogonal FFD, since the attribute levels were chosen in a way to minimize the standard errors (even though there is no directional information for the betas) and FFD levels were merely chosen to be orthogonal? Section 7.1.6 of the manual was unclear to me.

2. Is there any level of correlation across attributes in an efficient design at which an orthogonal FFD would be better, based on the criterion that a MNL model with an orthogonal FFD can be estimated with certainty (albeit in another forum post John explains that Hensher Rose and Greene (2005) provide data with correlations up to 0.9 and models were successfully estimated)?

3. Lets say an analyst has zero priors and develops a orthogonal FFD in Ngene. Would Ngene be able to find a orthogonal efficient design with the same criteria used to develop the orthogonal FFD?

4. Using the design syntax on pg. 75 of the Ngene manual, I created both a FFD and an efficient design (i.e. by switching ;orth = sim to ;eff = (mnl,d)). The correlation structure of the efficient design generated from the 338 evaluation (I let Ngene run for an hour and it did not produce a new design after 338) is as follows:
Attribute alt1.a alt1.b alt2.a alt2.c Block
alt1.a 1 0 -1 0 0
alt1.b 0 1 0 0 0
alt2.a -1 0 1 0 0
alt2.c 0 0 0 1 0
Block 0 0 0 0 1

Could a model from this design be estimable?

Thank you for any help.

Best regards,

Richard

[johnr]

Hi Richard

Ngene attempts to locate the attribute levels that maximises (mininises) whatever efficiency measure is chosen, subject to whatever constraints the analyst imposes. Unless the analyst allows for continuous levels, or lets go completely attribute level balance, Ngene will not select only the levels at the end of the attribute range. It will likely select each attribute level (end points plus those in the middle) with equal number - the likely in the above statement depends on a number of other factors such as whether attribute level balance is possible in the number of tasks requested by the user, etc.

1. Given enough time, I would expect Ngene to to find the orthogonal design which is most likely to be optimal for this set of assumptions. It would likely depend on the algorithm as to how long it would take.
2. This is a loaded question. The data set which you can download with book is a labelled experiment. Logit models are difference in utility models, only part of which is the X matrix, and one only need look at the equation to derive Z (given in previous posts) to see that the betas come into play via the probabilities in a very non-linear way, when calculating the Hessian. Hence, one should take into account not just the X matrix, but also the non-linear role the betas play in the AVC matrix. One cannot really say much about how correlation of the X matrix affects a logit model without considering the betas.
3. See 1 above.
4. Yes. In an unlabelled experiments with zero priors, only the correlation structure within alternatives matter. In this case, the correlation structure is such that you have perfect (-ve) correlation between alternatives. This is a typical outcome (though by no means does it have to occur - it will depend on the design generator used, etc.) of Street and Burgess type designs. Given zero priors, having minimum overlap (the levels are never the same across alternatives for the same attribute) will maximise the differences in utility. What this result shows is that you have minimum overlap.

John

[rich_imr]

Thank you very much johnr for your response.