To create realistic choice sets with trade-offs between the alternatives, I need to introduce a few design constraints, which I am unclear how to do in this particular case.
For context:
I would like to understand which information types transit users most need to plan a journey. There are 8 attributes that represent information types with attribute levels representing the quality/certainty of the information types: 4 have 3 levels (exact, estimate, none) and 4 have 2 levels (exact, none). So for example, 'frequencies' is an attribute, and a user could choose to say they need either 'exact', 'estimated' or 'no' frequency information to plan a journey.
In a given choice set, there are 3 alternatives.
On running the design without conditions, I see that the proposed designs would include choice sets that would have 'no-duh' choices (e.g. 1 alternative has all 'exact' attribute levels, 1 alternative has a mix of 'estimate' and 'none' levels, and 1 alternative has all 'none' levels save for 1 'estimate' level.)
I would like to include the following rules:
1) Each alternative must contain at least 1 'exact' attribute level
2) Each alternative must contain exactly 4 'none' attribute levels
How would I go about incorporating in such rules? Is there a source besides the NGene User Manual with syntax terms that can be used and examples?
In the design below, [0] represents 'none' and [1] represents 'exact':
Design
;alts = alt1*, alt2*, alt3*
;rows = 24
;block = 6
;eff = (mnl, d)
; model:
U(alt1) =
+ b2.dummy[-.1|.1] * fare[0,1,2]
+ b3.dummy[-.1|.1] * departure[0,1,2]
+ b4.dummy[-.1|.1] * arrival[0,1,2]
+ b5.dummy[-.1|.1] * frequency[0,1,2]
+ b6.dummy[-.1] * safeon[0,1]
+ b7.dummy[-.1] * safewait[0,1]
+ b8.dummy[-.1] * safewalk[0,1]
+ b9.dummy[-.1] * identifier[0,1]
/
U(alt2) = b2.dummy*fare + b3.dummy*departure + b4.dummy*arrival + b5.dummy*frequency + b6.dummy*safeon + b7.dummy*safewait + b8.dummy*safewalk + b9.dummy*identifier /
U(alt3) = b2.dummy*fare + b3.dummy*departure + b4.dummy*arrival + b5.dummy*frequency + b6.dummy*safeon + b7.dummy*safewait + b8.dummy*safewalk + b9.dummy*identifier
$
Setting Design Constraints and Criteria
Moderators: Andrew Collins, Michiel Bliemer, johnr
4 posts
• Page 1 of 1
Setting Design Constraints and Criteria
by bbryseck » Wed Oct 14, 2020 6:30 pm
- bbryseck
- Posts: 6
- Joined: Mon Sep 21, 2020 11:44 pm
Re: Setting Design Constraints and Criteria
by Michiel Bliemer » Wed Oct 14, 2020 9:41 pm
On this forum you can find thousands of syntax examples, including many with constraints.
Some constraints require a bit of creativity, for example in the syntax below I was able to include your two constraints by choosing convenient values for the levels of the dummy coded attributes. The values of these levels are meaningless but can be used to our advantage. You will see that running this syntax generates designs that satisfy your constraints. However, adding these constraints somehow results in an Undefined D-error, which typically means that the model cannot be estimated. I am not sure why, but the constraints seem too strict.
If I relax the second constraint to "each alternative should contain 3 or 4 'none' attribute levels", then it can find a suitable design.
Michiel
Some constraints require a bit of creativity, for example in the syntax below I was able to include your two constraints by choosing convenient values for the levels of the dummy coded attributes. The values of these levels are meaningless but can be used to our advantage. You will see that running this syntax generates designs that satisfy your constraints. However, adding these constraints somehow results in an Undefined D-error, which typically means that the model cannot be estimated. I am not sure why, but the constraints seem too strict.
- Code: Select all
Design
;alts = alt1*, alt2*, alt3*
;rows = 24
;block = 6
;eff = (mnl, d)
;alg = mfederov
;require:
? each alternative must contain at least 1 'exact' attribute level
alt1.fare = 0 or alt1.departure = 0 or alt1.arrival = 0 or alt1.frequency = 0 or alt1.safeon = 0 or alt1.safewait = 0 or alt1.safewalk = 0 or alt1.identifier = 0,
alt2.fare = 0 or alt2.departure = 0 or alt2.arrival = 0 or alt2.frequency = 0 or alt2.safeon = 0 or alt2.safewait = 0 or alt2.safewalk = 0 or alt2.identifier = 0,
alt3.fare = 0 or alt3.departure = 0 or alt3.arrival = 0 or alt3.frequency = 0 or alt3.safeon = 0 or alt3.safewait = 0 or alt3.safewalk = 0 or alt3.identifier = 0,
? each alternative must contain exactly 4 'none' attribute levels
alt1.fare + alt1.departure + alt1.arrival + alt1.frequency + alt1.safeon + alt1.safewait + alt1.safewalk + alt1.identifier >= 400,
alt2.fare + alt2.departure + alt2.arrival + alt2.frequency + alt2.safeon + alt2.safewait + alt2.safewalk + alt2.identifier >= 400,
alt3.fare + alt3.departure + alt3.arrival + alt3.frequency + alt3.safeon + alt3.safewait + alt3.safewalk + alt3.identifier >= 400,
alt1.fare + alt1.departure + alt1.arrival + alt1.frequency + alt1.safeon + alt1.safewait + alt1.safewalk + alt1.identifier < 500,
alt2.fare + alt2.departure + alt2.arrival + alt2.frequency + alt2.safeon + alt2.safewait + alt2.safewalk + alt2.identifier < 500,
alt3.fare + alt3.departure + alt3.arrival + alt3.frequency + alt3.safeon + alt3.safewait + alt3.safewalk + alt3.identifier < 500
;model:
U(alt1) = b2.dummy[-0.1|0.1] * fare[100,0,1] ? 100 = none, 0 = exact, 1 = estimate
+ b3.dummy[-0.1|0.1] * departure[100,0,1]
+ b4.dummy[-0.1|0.1] * arrival[100,0,1]
+ b5.dummy[-0.1|0.1] * frequency[100,0,1]
+ b6.dummy[-0.1] * safeon[100,0] ? 100 = none, 0 = exact
+ b7.dummy[-0.1] * safewait[100,0]
+ b8.dummy[-0.1] * safewalk[100,0]
+ b9.dummy[-0.1] * identifier[100,0]
/
U(alt2) = b2 * fare
+ b3 * departure
+ b4 * arrival
+ b5 * frequency
+ b6 * safeon
+ b7 * safewait
+ b8 * safewalk
+ b9 * identifier
/
U(alt3) = b2 * fare
+ b3 * departure
+ b4 * arrival
+ b5 * frequency
+ b6 * safeon
+ b7 * safewait
+ b8 * safewalk
+ b9 * identifier
$
If I relax the second constraint to "each alternative should contain 3 or 4 'none' attribute levels", then it can find a suitable design.
- Code: Select all
Design
;alts = alt1*, alt2*, alt3*
;rows = 24
;block = 6
;eff = (mnl, d)
;alg = mfederov
;require:
? each alternative must contain at least 1 'exact' attribute level
alt1.fare = 0 or alt1.departure = 0 or alt1.arrival = 0 or alt1.frequency = 0 or alt1.safeon = 0 or alt1.safewait = 0 or alt1.safewalk = 0 or alt1.identifier = 0,
alt2.fare = 0 or alt2.departure = 0 or alt2.arrival = 0 or alt2.frequency = 0 or alt2.safeon = 0 or alt2.safewait = 0 or alt2.safewalk = 0 or alt2.identifier = 0,
alt3.fare = 0 or alt3.departure = 0 or alt3.arrival = 0 or alt3.frequency = 0 or alt3.safeon = 0 or alt3.safewait = 0 or alt3.safewalk = 0 or alt3.identifier = 0,
? each alternative must contain 3 or 4 'none' attribute levels
alt1.fare + alt1.departure + alt1.arrival + alt1.frequency + alt1.safeon + alt1.safewait + alt1.safewalk + alt1.identifier >= 300,
alt2.fare + alt2.departure + alt2.arrival + alt2.frequency + alt2.safeon + alt2.safewait + alt2.safewalk + alt2.identifier >= 300,
alt3.fare + alt3.departure + alt3.arrival + alt3.frequency + alt3.safeon + alt3.safewait + alt3.safewalk + alt3.identifier >= 300,
alt1.fare + alt1.departure + alt1.arrival + alt1.frequency + alt1.safeon + alt1.safewait + alt1.safewalk + alt1.identifier < 500,
alt2.fare + alt2.departure + alt2.arrival + alt2.frequency + alt2.safeon + alt2.safewait + alt2.safewalk + alt2.identifier < 500,
alt3.fare + alt3.departure + alt3.arrival + alt3.frequency + alt3.safeon + alt3.safewait + alt3.safewalk + alt3.identifier < 500
;model:
U(alt1) = b2.dummy[-0.1|0.1] * fare[100,0,1] ? 100 = none, 0 = exact, 1 = estimate
+ b3.dummy[-0.1|0.1] * departure[100,0,1]
+ b4.dummy[-0.1|0.1] * arrival[100,0,1]
+ b5.dummy[-0.1|0.1] * frequency[100,0,1]
+ b6.dummy[-0.1] * safeon[100,0] ? 100 = none, 0 = exact
+ b7.dummy[-0.1] * safewait[100,0]
+ b8.dummy[-0.1] * safewalk[100,0]
+ b9.dummy[-0.1] * identifier[100,0]
/
U(alt2) = b2 * fare
+ b3 * departure
+ b4 * arrival
+ b5 * frequency
+ b6 * safeon
+ b7 * safewait
+ b8 * safewalk
+ b9 * identifier
/
U(alt3) = b2 * fare
+ b3 * departure
+ b4 * arrival
+ b5 * frequency
+ b6 * safeon
+ b7 * safewait
+ b8 * safewalk
+ b9 * identifier
$
Michiel
- Michiel Bliemer
- Posts: 1885
- Joined: Tue Mar 31, 2009 4:13 pm
Re: Setting Design Constraints and Criteria
by bbryseck » Wed Oct 21, 2020 1:53 am
Thanks so much! This helped solve a lot of headaches.
Just another quick question - for simplicity sake I have been using a multinomial logit model for this design, but after I do a pilot study (~100 participants) I want to update the design with a better understanding of the parameter priors, and possibly switch to a MMNL version for my full survey. Is it okay to switch from one model to another?
Just another quick question - for simplicity sake I have been using a multinomial logit model for this design, but after I do a pilot study (~100 participants) I want to update the design with a better understanding of the parameter priors, and possibly switch to a MMNL version for my full survey. Is it okay to switch from one model to another?
- bbryseck
- Posts: 6
- Joined: Mon Sep 21, 2020 11:44 pm
Re: Setting Design Constraints and Criteria
by Michiel Bliemer » Wed Oct 21, 2020 8:21 am
Optimising for panel MMNL models is almost impossible, only for the smallest of models (e.g. estimating 2 or 3 coefficient0s and 12 choice tasks) it may be possible. With your design dimensions, it would take years of computation time to optimise the design, so I would suggest just optimise for the MNL model using Bayesian priors. You can evaluate that design for panel MMNL instead of optimising for it.
In Bliemer and Rose (2010) we found that optimising for MNL models typically leads to designs that are also good for panel MMNL models, so we always recommend simply optimising for an MNL model and possibly evaluating the found design for the panel MMNL model.
Bliemer, M.C.J., and J.M. Rose (2010) Construction of experimental designs for mixed logit models allowing for correlation across choice observations. Transportation Research Part B, Vol. 44, No. 6, pp. 720-734.
Michiel
In Bliemer and Rose (2010) we found that optimising for MNL models typically leads to designs that are also good for panel MMNL models, so we always recommend simply optimising for an MNL model and possibly evaluating the found design for the panel MMNL model.
Bliemer, M.C.J., and J.M. Rose (2010) Construction of experimental designs for mixed logit models allowing for correlation across choice observations. Transportation Research Part B, Vol. 44, No. 6, pp. 720-734.
Michiel
- Michiel Bliemer
- Posts: 1885
- Joined: Tue Mar 31, 2009 4:13 pm
4 posts
• Page 1 of 1
Return to Choice experiments - Ngene
Who is online
Users browsing this forum: No registered users and 38 guests