combination of scenario-variables, covariates and model-avg.

[svenga]

Dear Choice-Metric team,

first of all thank you for providing such an excellent tool and manual! It is just extremely valuable for a researcher like myself, with no previous practical experience with the DCE method!

Some background:
We would like to conduct a stated preference DCE study in the container shipping industry, asking different types of shippers (e.g. freight forwarders vs. industrial shippers) for their choice of container liner services. Later in the model estimation we would like to compare preferences for those two groups. Furthermore we would like to confront each shipper with different scenarios along two dimensions (contract vs. spot bookings and high-value vs. low-value commodities). Again we would like to compare model results for the different scenarios. Alternatives will be UNLABELLED, so ALL parameters are generic (no no-choice option planned so far). Currently we are looking for a pre-study design to obtain better priors, as we only know/expect signs and a ranking of the main-effect parameters right now.

Here my questions:
1) From my understanding we would incorporate the different scenarios in Ngene by using two scenario variables with two levels each, right? For this purpose, do I 'just' include the two scenario variables in J-1 of my alternatives in the model section (as given in the code below)? Or do I need to interact the variables with one of the main effects, because of my alternatives being unlabelled and all parameters being generic?
2) Additionally I would like to incorporate "type of shipper" as a covariate. Again my question is, if I implement this correctly in the Ngene code by using a .covar variable in J-1 alternatives? Or do I need to do it differently given the unlabelled experiment?
3) (How) Could I later combine this with a model averaging approach to optimize the design for MNL and MMNL estimation? I haven't yet found a way to combine ;fisher and ;eff properties correctly for such a case and I am generally not sure if Ngene allows such a case...

Thank you very much for any answers to the above!
Best,
Sven

My current code:
Design

;alts(FF) = a1, a2, a3, a4
;alts(BCO) = a1, a2, a3, a4

;rows = 6
;eff=F1(mnl,d)
;fisher(F1)=des1(FF[0.5], BCO[0.5])

;model(FF):
U(a1) = bmTT[-0.2]*TT[0,1,2,3,4,5] + bmRE[0.1]*RE[0,1,2] + bmSF[-0.1]*SF[0,1,2] + bmEA[0.1]*EA[0,1,2]
+ bmCO[0.1]*CO[0,1,2] + bmQD[0.1]*QD[0,1,2] + bmFR[-0.2]*FR[0,1,2,3,4,5]
+ biFR_TT*FR*TT + biFR_RE*FR*RE + biFR_SF*FR*SF + biFR_EA*FR*EA + biFR_CO*FR*CO + biFR_QD*FR*QD + biTT_RE*TT*RE + biTT_SF*TT*SF
+ bsCT * CT[1,2] + bsCV * CV[1,2]
+ shippertype * shippertype.covar[1] /
U(a2) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT*FR*TT + biFR_RE*FR*RE + biFR_SF*FR*SF + biFR_EA*FR*EA + biFR_CO*FR*CO + biFR_QD*FR*QD + biTT_RE*TT*RE + biTT_SF*TT*SF
+ bsCT * CT[CT] + bsCV * CV[CV]
+ shippertype * shippertype.covar[1] /
U(a3) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT*FR*TT + biFR_RE*FR*RE + biFR_SF*FR*SF + biFR_EA*FR*EA + biFR_CO*FR*CO + biFR_QD*FR*QD + biTT_RE*TT*RE + biTT_SF*TT*SF
+ bsCT * CT[CT] + bsCV * CV[CV]
+ shippertype * shippertype.covar[1] /
U(a4) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT*FR*TT + biFR_RE*FR*RE + biFR_SF*FR*SF + biFR_EA*FR*EA + biFR_CO*FR*CO + biFR_QD*FR*QD + biTT_RE*TT*RE + biTT_SF*TT*SF

;model(BCO):
U(a1) = bmTT[-0.2]*TT[0,1,2,3,4,5] + bmRE[0.1]*RE[0,1,2] + bmSF[-0.1]*SF[0,1,2] + bmEA[0.1]*EA[0,1,2]
+ bmCO[0.1]*CO[0,1,2] + bmQD[0.1]*QD[0,1,2] + bmFR[-0.2]*FR[0,1,2,3,4,5]
+ biFR_TT*FR*TT + biFR_RE*FR*RE + biFR_SF*FR*SF + biFR_EA*FR*EA + biFR_CO*FR*CO + biFR_QD*FR*QD + biTT_RE*TT*RE + biTT_SF*TT*SF
+ bsCT * CT[1,2] + bsCV * CV[1,2]
+ shippertype * shippertype.covar[2] /
U(a2) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT*FR*TT + biFR_RE*FR*RE + biFR_SF*FR*SF + biFR_EA*FR*EA + biFR_CO*FR*CO + biFR_QD*FR*QD + biTT_RE*TT*RE + biTT_SF*TT*SF
+ bsCT * CT[CT] + bsCV * CV[CV]
+ shippertype * shippertype.covar[2] /
U(a3) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT*FR*TT + biFR_RE*FR*RE + biFR_SF*FR*SF + biFR_EA*FR*EA + biFR_CO*FR*CO + biFR_QD*FR*QD + biTT_RE*TT*RE + biTT_SF*TT*SF
+ bsCT * CT[CT] + bsCV * CV[CV]
+ shippertype * shippertype.covar[2] /
U(a4) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT*FR*TT + biFR_RE*FR*RE + biFR_SF*FR*SF + biFR_EA*FR*EA + biFR_CO*FR*CO + biFR_QD*FR*QD + biTT_RE*TT*RE + biTT_SF*TT*SF
$

[Michiel Bliemer]

Dear Sven,

I am afraid I do not have the time to completely understand your experimental design, but I will try to answer your questions. The first two are actually regular experimental design questions, not Ngene questions.
1. In an unlabelled experiment, if you include a scenario you have to interact it with another variable, otherwise it is identical across all alternatives and it has no impact (logit models only work on differences between utility functions). Or you have to add a labelled alternative without the scenario (like the no-choice).
2. The same answer holds for covariates, you need to interact them in an unlabelled experiment, or add a labelled alternative.
3. I think you should be able to do model averaging in this experiment with MNL and MMNL (although panel MMNL will take very very long). If you post the syntax I will try to see if I can get it to run.

Michiel

[svenga]

Dear Michiel,

thank you very much for your help a couple of weeks ago!! We have conducted a small pre-study now to obtain some priors and I am now trying to find a design for the main study. I have dropped the idea of model averaging in the first place, but now I am having two other questions:

1. am struggling with the combination of 2 covariate (+1 scenario) variables. No matter what I try, it seems that Ngene cannot find a design that suits, even after 200K+ runs no valid design was found... Do I maybe still have any syntax issues in my code, especially in the covariate/scenario variables? Maybe a problem with too large priors, as stated in the alert message? (The problem only occured when I 'switched' the former scenario variable bcCT from being a scenario variable to a covariate variable (incl. all the addistional model specifications needed...) even with the rather large model set up I was able to find some good designs, when I only had one covariate variable...)
2. i have 7 main effects in the model (pre-study tells me still okay for target-participants), so a lot of 2-way-interactions could be thought of. Would you recommend to include them all in the design and opt-out the seemingly irrelevant ones during the modelling process, or better try to select only those ones expected to (maybe) be relevant and only include the selected when searching for a design...?

Happy about any hints regarding my code (for your orientation: the first 1-2 rows in the alternative specifications are main effects, next row 2-way interactions, then find the scenario and covariates in the last 3 rows):
Thanks in advance, Sven

Design

;alts(FF_C) = a1, a2, a3, a4
;alts(FF_S) = a1, a2, a3, a4
;alts(BCO_C) = a1, a2, a3, a4
;alts(BCO_S) = a1, a2, a3, a4


;rows = 12
;eff=F1(mnl,d)
;fisher(F1)=des1(FF_C[0.35], FF_S[0.25], BCO_C[0.2], BCO_S[0.2])
;bdraws=halton(200)

;model(FF_C):
U(a1) = bmTT[(n,-0.1,0.1)]*TT[0,1,2,3,4,5] + bmRE[(n,0.55,0.15)]*RE[0,1,2] + bmSF[(n,0.1,0.15)]*SF[0,1,2] + bmEA[(n,0.55,0.15)]*EA[0,1,2]
+ bmCO[(n,0.3,0.15)]*CO[0,1,2] + bmQD[(n,0.15,0.15)]*QD[0,1,2] + bmFR[(n,-0.3,0.1)]*FR[0,1,2,3,4,5]
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[1,2]
+ bcCT * CT.covar[1]
+ bcST * ST.covar[1] /
U(a2) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[CV]
+ bcCT * CT.covar[1]
+ bcST * ST.covar[1] /
U(a3) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[CV]
+ bcCT * CT.covar[1]
+ bcST * ST.covar[1]

;model(FF_S):
U(a1) = bmTT[(n,-0.1,0.1)]*TT[0,1,2,3,4,5] + bmRE[(n,0.55,0.15)]*RE[0,1,2] + bmSF[(n,0.1,0.15)]*SF[0,1,2] + bmEA[(n,0.55,0.15)]*EA[0,1,2]
+ bmCO[(n,0.3,0.15)]*CO[0,1,2] + bmQD[(n,0.15,0.15)]*QD[0,1,2] + bmFR[(n,-0.3,0.1)]*FR[0,1,2,3,4,5]
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[1,2]
+ bcCT * CT.covar[2]
+ bcST * ST.covar[1] /
U(a2) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[CV]
+ bcCT * CT.covar[2]
+ bcST * ST.covar[1] /
U(a3) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[CV]
+ bcCT * CT.covar[2]
+ bcST * ST.covar[1]

;model(BCO_C):
U(a1) = bmTT[(n,-0.1,0.1)]*TT[0,1,2,3,4,5] + bmRE[(n,0.55,0.15)]*RE[0,1,2] + bmSF[(n,0.1,0.15)]*SF[0,1,2] + bmEA[(n,0.55,0.15)]*EA[0,1,2]
+ bmCO[(n,0.3,0.15)]*CO[0,1,2] + bmQD[(n,0.15,0.15)]*QD[0,1,2] + bmFR[(n,-0.3,0.1)]*FR[0,1,2,3,4,5]
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[1,2]
+ bcCT * CT.covar[1]
+ bcST * ST.covar[2] /
U(a2) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[CV]
+ bcCT * CT.covar[1]
+ bcST * ST.covar[2] /
U(a3) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[CV]
+ bcCT * CT.covar[1]
+ bcST * ST.covar[2]

;model(BCO_S):
U(a1) = bmTT[(n,-0.1,0.1)]*TT[0,1,2,3,4,5] + bmRE[(n,0.55,0.15)]*RE[0,1,2] + bmSF[(n,0.1,0.15)]*SF[0,1,2] + bmEA[(n,0.55,0.15)]*EA[0,1,2]
+ bmCO[(n,0.3,0.15)]*CO[0,1,2] + bmQD[(n,0.15,0.15)]*QD[0,1,2] + bmFR[(n,-0.3,0.1)]*FR[0,1,2,3,4,5]
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[1,2]
+ bcCT * CT.covar[2]
+ bcST * ST.covar[2] /
U(a2) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[CV]
+ bcCT * CT.covar[2]
+ bcST * ST.covar[2] /
U(a3) = bmTT*TT + bmRE*RE + bmSF*SF + bmEA*EA + bmCO*CO + bmQD*QD + bmFR*FR
+ biFR_TT[0]*FR*TT + biFR_RE[0]*FR*RE + biFR_SF[0]*FR*SF + biFR_EA[0]*FR*EA + biFR_CO[0]*FR*CO + biFR_QD[0]*FR*QD + biTT_RE[0]*TT*RE
+ bsCV * CV[CV]
+ bcCT * CT.covar[2]
+ bcST * ST.covar[2]
$

[Michiel Bliemer]

Apologies for the late reply...

I think I answered your first question in another post.
Regarding your second question, there is no harm in putting more interactions in your design, but then you may also require more rows in your design. I would suggest to include the interactions that you think are most relevant and that you think you will use in your final model estimation, and leave out the ones that are not logical or you think you would not include in your final estimation. Even if you leave them out, you will likely still be able to estimate that interaction effect. By including them, you optimise their standard errors in estimation, but by including more interactions (= more coefficients) your other coefficients may see larger standard errors in your main effects as a trade-off for lower standard errors in your interaction effects. So that is why I would suggest only to include the ones that you think are most important, as you probably want to focus most on the main effects.

Michiel