Interaction terms with orthogonality and optimal design

[zshroff]

I am conducting an unlabeled DCE to study issues related to job preferences of workers. I would like my design to have 6-8 attributes with 2-4 levels. Attributes and levels are common across both alternative choice sets to be provided. In addition to orthogonality ( or near orthogonality) and attribute level balance, I am looking to have minimal attribute overlap ( occasional overlap is fine) and allow for two way interactions ( for which I have used the foldover command). I would like some assistance on how to create the following choice set in Ngene, the manual is not very clear on whether interactions can be accommodated in an OOD design, or is there another way to accommodate these ?
The code that I felt appropriate is
Design
; alts=alt 1, alt 2
; rows=12
;orth=ood
;foldover
;model:
U (alt1)= b1+b2*A[0,1,2,3] +b3*B[0,1] +b4*C [0,1] + b5*D[ 0,1] + b6*E[0,1,2,3] + b7*F[0,1] /
U (alt2)= b2*A +b3*B +b4*C + b5*D + b6*E + b7*F $

Is this appropriate ? If not what can I do to get all these properties ?

Also, with foldover, the manual seems to indicate that the foldover block in the output shows , so foldover block 1 should be given to one respondent, block 2 to another. I wanted to confirm this.

Thank you,
Zubin

[Michiel Bliemer]

That syntax would work (although you have to remove the spaces in the names of the alternatives), and would give you a sequential orthogonal design with attribute level balance and due to the foldover will have all two-way interaction effects orthogonal with all main effects (although across two-way interactions effects may be correlated). You can also add interaction effects in the utility function, like +b8*E*F. Ngene can also report the D-error of the OOD design, so if you want the D-error to include the interaction effect, then also include the interaction effect in the utility function. An OOD design is generated using the algorithm by Street and Burgess, which assumes only main effects (and a whole bunch of other assumptions), so the D-optimality that is reported does not include any interactions. Note that the constant is ignored in OOD designs (and other orthogonal designs). I assume you can remove the constant in your syntax, since it is an unlabelled experiment (although in estimation you can include this constant for looking at left-to-right reading bias for example).

Alternatively, you can create a sequential orthogonal D-efficient design in Ngene with the following syntax:

Code: Select all

Design
;alts=alt1, alt2
;rows=12
;orth=seq
;eff = (mnl,d)
;model:
U (alt1)= b2*A[0,1,2,3] +b3*B[0,1] +b4*C [0,1] + b5*D[ 0,1] + b6*E[0,1,2,3] + b7*F[0,1] + b8*E*F /
U (alt2)= b2*A +b3*B +b4*C + b5*D + b6*E + b7*F + b8*E*F $

The fold-over is not necessary in this case, as the interaction effect is included in the utility function, and the ;eff command takes this interaction into account when computing and optimizing the D-error. Minimum overlap, however, will not be guaranteed (OOD does).

Is there a certain reason why you would like to use an orthogonal design? You do not have any information on the prior parameters? Moving away from an orthogonal design using non-zero priors could significantly improve your design efficiency.

It is correct that block 1 is assigned to one respondent, and block two to another respondent.

[zshroff]

Thanks. I don't have any information on my priors to put into the utility function. I am sure that I will have a few more questions to ask as I go ahead with designing my experiment.
Thanks once again for your help,
Zubin

[zshroff]

Thanks for your help, just to follow up on this. As you mentioned, if I want to get an accurate measure of the D optimality with an OOD design, it would be better to create interaction terms for each of my two way interactions. The following question may sound really silly, but I would like to confirm it.

The choice sets as presented to the respondents in the following design will still have two alternatives each with seven attributes of varying levels. (Basically interaction term output alt1e*alt1f for instance which will be shown in the evaluation output is not to be shown to the respondent). Sorry for this seemingly obvious question. Also, due to these individual interaction terms I have removed the foldover command

The final design that I plan to use is
Design
; alts=alt1, alt2
; rows=24
;orth=ood
;model:
U (alt1)= b1+b2*A[0,1,2,3] +b3*B[0,1] +b4*C [0,1] + b5*D[ 0,1] + b6*E[0,1,2,3] + b7*F[0,1] +b8*F*E +b9*F*D /
U (alt2)= b2*A +b3*B +b4*C + b5*D + b6*E + b7*F +b8*F*E+ b9*F*D $

Second, when I run the evaluation software, it gives me a warning - Defaulting to prior values for zero for the following priors
b1,b2......b9. Is this because it is the evaluation version or is it due to my not giving values in a utility function.

Third, when I run the software, I get two different versions Evaluation 1 and Evaluation 7, both have an identical measure of D optimality, in that case is their any guide on which one I should use. And more broadly ,what level of D optimality is considered good at the minimum ?

Also, what is the importance of putting an asterix on the alternatives ( you have suggested that for unlabelled experiments), given that I have not specified any priors for my utility function . Does utility balancing then come in at all into the equation ?

Finally, I want to use clogit and mixed logit to analyze the data. I just wanted to make sure that that fits in with the choice sets generated as a result of the algorithm above.

Thank you once again for your assistance,
Ngene has really enabled me to go ahead with designing this experiment.
Zubin

[Michiel Bliemer]

1. Yes the respondent only sees 6 attributes, the interactions are considered by the analyst only.
2. The priors default to zero if you do not give them any values.
3. Equal values of D optimality indicate no different in efficiency, you can perhaps choose based on the actual choice tasks that come out of the design, which design makes more sense to you. I cannot really comment on the minimum D-optimality that would be good. Myself, I would not use orthogonal designs and rather go for D-efficient designs with nonzero priors obtained from a pilot study, but this is not the route you are choosing.
4. The asterix does not do anything if you do not specify priors. The asterix is for ruling out dominant choice alternatives, not utility balancing.
5. OOD is designed for MNL (conditional logit), do the D-optimality does not say anything about the efficiency when you estimate a mixed logit model. Then you have to use the ;eff = (rppanel,d) command for example, with nonzero priors. You can evaluate an OOD design for such a mixed logit model by specifying the mixed logit model specifically in the syntax.

I hope this helps.

Michiel

[zshroff]

Thank you for your reply. I had a couple of more questions about using OOD algorithms related to the design I have mentioned in the post above.

1) When running the OOD algorithm, the Pearson Product Moment shows a perfectly negative correlation (-1) for the two alternatives of the same attribute ( if it has two levels, for example alt1a and alt2a show a PPM of -1, for attributes with more levels the PPM is -0.2, for instance). I understand that this is probably by construction.

Does this have a bearing on the estimation of main effects ? I am concerned about multicollinearity and should not have a model that does not run. I plan to analyse my data using a conditional logit model.


2) Does using an OOD algorithm ( and more broadly ensuring minimal overlap with no common levels across the choice sets) in any way hinder the estimation of interaction terms ? I had read in an article that a minimal overlap design could preclude the estimation of interaction terms.

3) Is it fine to use the OOD algorithm to generate two way interactions where one of the terms has more than two levels ?

4)In your last reply you had mentioned that ' the interactions are considered by the analyst only' . How are these to be incorporated ? Are the levels entered on the attributes used to create the appropriate interaction terms for the analysis?

Thank you,
Zubin

[Michiel Bliemer]

To answer your questions:

1) OOD designs (which are referred to as Street and Burgess designs, see their papers) minimize overlap, and therefore may have perfect (negative) correlations across alternatives. This is not a problem, it is even an optimal way of doing it, if the attributes are generic across alternatives. Note that you are estimating only a single parameter for both alternatives, so essentially there is no multicollinearity (only if you would estimate 2 parameters separately).

2) I an not an expert in Street and Burgess designs (you may want to consult their papers), but as far as I know they can be used to estimate interaction effects. Just as before enter the interactions into the utility function, and Ngene will come up with a design that can also estimate the interaction terms (if not, the D-error would be infinite).

3) You can have more than 2 levels for interactions, for example one attribute with 2 levels and another attribute with 3 levels. I do not see why this would not be possible.

4) The analyst writes down the utility functions including the interaction terms, hence they are explicitly considered in the utility function. The respondent is not aware of these utility functions. So you, as an analyst, will estimate additional parameters for the interactions.

Michiel

[zshroff]

Thanks for your help with this, will be working on the final design soon with the non-evaluation version
Zubin

[zshroff]

Hi,
I recently started working with the non-evaluation version of Ngene and getting output, leading to this round of questions.
This was the design I used
Design
; alts=alt1, alt2
; rows=24
;orth=ood
;model:
U(alt1)= b2*A[0,1] +b3*B[0,1] +b4*C [0,1] + b5*D[ 0,1] + b6*E[0,1,2,3] + b7*F[0,1] /
U(alt2)= b2*A +b3*B +b4*C + b5*D + b6*E + b7*F $

This gave me an orthogonal , balanced design with no overlap and no dominant alternative over any choice set


With any syntax other than this ( and I mean any other number of rows or changing number of attributes or the number of levels of any attribute), one of the choice sets in the design always has one option that is dominant over the other( all attributes have higher values in one alternative). This severely restricts my ability to add attributes and levels.
I have three questions about this
a) Is there any way of preventing this , or getting choice sets that are not dominant, orthogonal designs don’t let me use conditions
b) Is there any harm to estimation ( in terms of bias or the model not running) in including the dominant option in the analysis, or is it just a question of having a choice set that does not provide much information ?
c) Would it be preferable to use the dominant choiceset in the analysis and see how the model varies for individuals who answer rationally or not or 1) drop it during analysis 2) change the value of one variable in what is shown to the respondent to alter the choice set, or is this last option dangerous in terms of playing with the orthogonality and attribute level balance.

Thanks once again for your help,
Zubin

[Michiel Bliemer]

The issues you are experiencing are because
(1) you are using an orthogonal design, which puts a significant constraint on your designs, and
(2) you are not providing any priors, such that Ngene is not able to determine whether dominant alternatives exist.

Therefore, as said before, I would always recommend an efficient design with priors (at least the sign of the parameter, such that Ngene can rule out dominant alternatives), letting go of orthogonality.
Given the fact that you want an orthogonal design (which is not efficient and can contain 'silly' choice tasks), all you can do is changing the levels in the choice tasks that contain strictly dominant alternatives, as you do not want them in your design (people may not take your survey seriously) and you definitely do not want them in your analyses (they will bias your results). Changing choice tasks will lead to non-orthogonality and likely also attribute level balance. So again my question, why do you need an orthogonal design, as orthogonality is not required for estimating choice models?