Cond with require / Dummy or not / working with SurveyEngine

[Joseph Wu]

To the Ngene support team,

I am a PhD student at the University of Queensland, currently conducting a choice study on mode selection concerning micro-mobility. I also apply SurveyEngine as part of my survey design.

Here is my current script of Ngene design:

Code: Select all

design
;alts = pmm, ptsmm, car
;rows = 48
;block = 8
;eff = (mnl, d)
;alg = mfederov
;bdraws = sobol(200)
;cond:
if(ptsmm.SMMTT=2, ptsmm.IVTT_2=0 and ptsmm.RC_2=0 and ptsmm.SMMTC=0),
if(ptsmm.SMMTT=4, ptsmm.IVTT_2=1 and ptsmm.RC_2=1 and ptsmm.SMMTC=1),
if(ptsmm.SMMTT=6, ptsmm.IVTT_2=2 and ptsmm.RC_2=2 and ptsmm.SMMTC=2),
if(ptsmm.SMMTT=8, ptsmm.IVTT_2=3 and ptsmm.RC_2=3 and ptsmm.SMMTC=3),
if(ptsmm.CL_2=0, ptsmm.IVTT_2=[0,1]),
if(ptsmm.CL_2=1, ptsmm.IVTT_2=[1,2]),
if(ptsmm.CL_2=2, ptsmm.IVTT_2=[2,3]),
if(car.CL_3=0, car.IVTT_3=[0,1]),
if(car.CL_3=2, car.IVTT_3=[1,2])

;require:
ptsmm.CL_2<= car.CL_3,
pmm.BW <= ptsmm.BW

;model:
U(pmm) = b1.dummy[(n,0,0.2)|(n,0,0.2)|(n,0,0.2)|(n,0,0.2)] * W[0,1,2,3,4]
        + b2.dummy[-0.02|-0.01]                            * IVTT[0,1,2]
        + b3.dummy[-0.02|-0.01]                            * RC[0,1,2]
        + b4.dummy[-0.04|-0.03|-0.02|-0.01]                * BW[0,1,2,3,4]
        + asc1[0]
/
U(ptsmm) = b1                          * W
        + b5.dummy[0.03|0.02|0.01]     * IVTT_2[0,1,2,3]
        + b6[-0.01]                    * SMMTT[2,4,6,8]
        + b7[-0.01]                    * WT[2,5,8]
        + b8[-0.01]                    * AT_2[1,3,5]
        + b9.dummy[0.03|0.02|0.01]     * RC_2[0,1,2,3]
        + b10.dummy[0.03|0.02|0.01]    * SMMTC[0,1,2,3]
        + b11.dummy[0.02|0.01]         * CL_2[0,1,2]
        + b4                           * BW
        + asc2[0]
/
U(car) = b1                      * W
        + b12.dummy[-0.02|-0.01] * IVTT_3[0,1,2]
        + b13.dummy[-0.02|-0.01] * AT_3[0,1,2]
        + b14.dummy[-0.02|-0.01] * RC_3[0,1,2]
        + b15.dummy[-0.02|-0.01] * PC_3[0,1,2]
        + b16.dummy[0.02|0.01]   * CL_3[0,1,2]
$

I have several inquiries regarding my experimental design:
1.I aim to implement constraints in my design, represented as cond: and require: in Ngene. However, cond: cannot be used alongside mfederov, and require: necessitates specifying the algorithm employed. Is there a method to incorporate both cond: and require: simultaneously? I reviewed the manual but couldn't find an example.
2. Certain attribute values in my study are computed based on participants' responses or predefined values. For instance, IVTT (Travel time), RC (Running cost), and SMMTC (Travel cost) vary around a base value, with a variance of ±25%, based on reported travel distance. Additionally, AT (Access time) and PC (Parking cost) are determined using predefined logic on the basis of their responses. Despite being numerical, should these attributes still be treated as dummy coding, similar to my current script?
3. I'm contemplating including two sum attributes for total travel time and cost in my survey to provide participants with a clear understanding. These values will be computed based on related attribute values in the design. Should I proceed with designing the experiment as usual and then insert specific columns with all 0 for these two attributes?
4. Do you have any further suggestions or recommendations regarding my script?

I greatly appreciate your insights and feedback and eagerly await your response.

Best regards,
Yikang

[Michiel Bliemer]

1. Currently Ngene only supports ;require/reject for mfederov and ;cond for the default swapping algorithm, this has to do with the fact that each type of algorithm constructs designs differently and needs constraints to be specified in a certain way. But it is almost always possible to reformulate a require/reject constraint into a conditional constraint and the other way around. For example, you can reformulate your require constraints as:

;cond:
if(ptsmm.CL_2=1, car.CL_3=[1,2]),
if(ptsmm.CL_2=2, car.CL_3=2),
if(pmm.BW=1, ptsmm.BW=[1,2,3,4]),
if(pmm.BW=2, ptsmm.BW=[2,3,4]),
if(pmm.BW=3, ptsmm.BW=[3,4]),
if(pmm.BW=4, ptsmm.BW=4)

2. It seems that you are pivoting attribute levels around self-reported reference levels. In that case, you could consider generating different designs with absolute levels based on different combinations of reported attribute levels. This would require generating a large number of designs a priori but it would avoid having to implement a good amount of survey logit. Alternatively, you can compute attribute levels on the fly and generate a single design based around average levels for IVTT, RC and SMMTC and then convert these to relative pivots. Since these attributes are numerical, you do not need to dummy code them. However, since you are using near-zero priors, I would recommend to use dummy coding at the design stage if possible, but this may conflict with pivoting.

3. This has to do with presentation format of the design, so this is a question to SurveyEngine on how to do this in their platform. But yes, you would generate the design as usual. In your analysis, you may need to account for the fact that you also show the sum, as some people may only look at the total and at the components, so it may require special econometric modelling.

4. You need to add alternative-specific constants. If you are interested in market shares or demand forecasting, I would also add ;con to the script. Most importantly, you have created perfectly correlated attributes in your first four conditional constraints such that you will not be able to estimate your model and it will result in an infinite (undefined) D-error. You need variation in your design to be able to estimate the coefficients, so please relax these constraints.

Michiel

[Joseph Wu]

Hi Michiel,

Thank you for your thorough feedback and suggestions.

Regarding the issue of too many constraints, I'd like to explain the logic behind my survey design.

For the alternative combining public transport (pt) and shared micro-mobility (smm) - ptsmm - I've set four levels of smm travel time (SMMTT) (2, 4, 6, 8 mins), which then determine the attribute levels of ptsmm.IVTT, ptsmm.RC, and ptsmm.SMMTC. To ensure consistency, I've imposed four constraints to always align them with the corresponding SMMTT levels.

Since the values of travel time and travel cost are also influenced by reported travel distance, and creating a bunch of designs for all configurations is overly complex, I've handled the calculation of these related attribute levels within the SurveyEngine platform using variables. For example, I've created variables like IVTT_base, IVTT_plus_%25, and IVTT_minus_%25 to represent three levels of IVTT for pmm and car, and these variables are then inserted into the SurveyEngine design as attribute levels. In other words, while these values are ultimately real numbers, from an experimental design standpoint, their levels lack specific values. Instead, they're represented by variables that only indicate the order of these levels in terms of magnitude. This is why I believe employing dummy coding for these variables in the experimental design is appropriate.

Regarding the weather attribute (W), which remains constant across all alternatives as a scenario attribute, I plan to use the expression b1*W[W] for ptsmm and car.

To address the challenge of too many constraints, I've considered three approaches:

1. Implementing a method similar to that used for the weather attribute. Since these attributes have the same levels after dummy coding, I could apply expressions like b9.dummy[0.03|0.02|0.01]* RC_2[IVTT_2] and b10.dummy[0.03|0.02|0.01]*SMMTC[IVTT_2]. Would this approach differ significantly from the current method of setting constraints?

2. Focusing solely on SMMTT in the experimental design and creating corresponding columns for other attributes in .Excel before importing the design into SurveyEngine.

3. Adjusting the survey design to mitigate the perfect correlation between these attributes.

I would greatly appreciate your input and suggestions on these options.

Best regards,
Yikang

[Michiel Bliemer]

Option 1 will never work because you cannot estimate both b9 and b10 if your attributes are fully correlated, so it leads to the same issue.

Option 3 may work if you do not create 1-on-1 relationships between variables, and instead provide some freedom for variables to move. A typical example is that short distance trips may have travel times 5,10,15, while medium distance trips have 15,20,25, and long distance trips have 25,30,40 or something. This avoids perfect correlation.

If all variables move exactly together, then you need to capture this in a single combined variable with only a single (dummy coded) coefficient.

Michiel

[Joseph Wu]

Hi Michiel,

I have further questions about the scenario attribute.

In my DCE, the first attribute is weather, represented as W in the script. Upon reviewing the manual, I observed that the scenario attribute can be designated as W[W] for alternatives following the first one to maintain consistency in values. However, this applies to unlabelled DCEs where one attribute holds a certain prior across alternatives. In contrast, with labelled DCEs, the scenario attribute may exert different effects on different alternatives. For instance, rainy weather might negatively impact the preference for a micro-mobility device but positively influence the preference for a car. Should I specify the different priors of W for all alternatives?

Additionally, since I'm utilizing Ngene with SurveyEngine, scenario attributes only require specification once for the initial alternative. Consequently, I only specify W for my U(pmm), excluding it for subsequent alternatives, and assigning them all zeros in SurveyEngine. Would this omission have any significant implications?

Best,
Yikang

[Michiel Bliemer]

The use of W[W] is useful in both unlabelled and labelled experiments, but in unlabelled alternatives you multiply with a generic parameter whereas in labelled alternatives you multiply with an alternative-specific parameter, just like any other attribute. So yes, you specify different parameters for W, which also means different priors.

Scenario variables are not specified only for the initial alternative, but rather are typically stated above each choice task in the description. E.g.
"Consider making a trip to work and suppose that it is rainy. Which mode of transport would you choose?"
So while it is part of your experimental design, it is treated differently when you format the choice tasks in SurveyEngine. I suggest you put a question on the SurveyEngine forum on how best to do that in their survey platform as I do not know.

Michiel

[Joseph Wu]

Hi Michiel,

Thank you for your reply. And I believe it's necessary to specify priors of scenario attribute Weather for every alternatives. And I add b16 and b17 respectively in the code below. However, this script can work before I add these two parameters, and cannot find any design after adding them. May I ask the problems here?

Code: Select all

design
;alts = pmm, ptsmm, car
;rows = 60
;block = 10
;eff = (mnl, d, mean)
;bdraws = sobol(200)
;cond:
if(ptsmm.IVTT_2=0, ptsmm.MMTT_2=[6,8]),
if(ptsmm.IVTT_2=1, ptsmm.MMTT_2=[4,6]),
if(ptsmm.IVTT_2=2, ptsmm.MMTT_2=[2,4]),
if(ptsmm.RC_2=0, ptsmm.MMTT_2=[2,4]),
if(ptsmm.RC_2=1, ptsmm.MMTT_2=[4,6]),
if(ptsmm.RC_2=2, ptsmm.MMTT_2=[6,8]),
if(ptsmm.CL_2=0, ptsmm.IVTT_2=[0,1]),
if(ptsmm.CL_2=2, ptsmm.IVTT_2=[1,2]),
if(car.CL_3=0, car.IVTT_3=[1,2]),
if(car.CL_3=2, car.IVTT_3=[0,1]),
if(ptsmm.CL_2=1, car.CL_3=[1,2]),
if(ptsmm.CL_2=2, car.CL_3=2),
if(pmm.BW=1, ptsmm.BW=[1,2,3,4]),
if(pmm.BW=2, ptsmm.BW=[2,3,4]),
if(pmm.BW=3, ptsmm.BW=[3,4]),
if(pmm.BW=4, ptsmm.BW=4)
;con

;model:
U(pmm) = b1.dummy[(n,0,0.2)|(n,0,0.2)|(n,0,0.2)|(n,0,0.2)] * W[0,1,2,3,4]
        + b2.dummy[-0.02|-0.01]                            * MMTT[0,1,2]
        + b3.dummy[-0.02|-0.01]                            * RC[0,1,2]
        + b4.dummy[-0.04|-0.03|-0.02|-0.01]                * BW[0,1,2,3,4]
        + asc1[0]
/
U(ptsmm) = b5.dummy[0.02|0.01]          * IVTT_2[0,1,2]
        + b6[-0.01]                    * MMTT_2[2,4,6,8]
        + b7[-0.01]                    * WT_2[2,5,8]
        + b8[-0.01]                    * ST_2[1,3,5]
        + b9.dummy[0.02|0.01]          * RC_2[0,1,2]
        + b10.dummy[0.02|0.01]         * CL_2[0,1,2]
        + b4                           * BW
        + asc2[0]
        + b16.dummy[(n,0,0.2)|(n,0,0.2)|(n,0,0.2)|(n,0,0.2)] * W[W]
/
U(car) = b11.dummy[-0.02|-0.01] * IVTT_3[0,1,2]
        + b12.dummy[-0.02|-0.01] * WLKST_3[0,1,2]
        + b13.dummy[-0.02|-0.01] * RC_3[0,1,2]
        + b14.dummy[-0.02|-0.01] * PC_3[0,1,2]
        + b15.dummy[0.02|0.01]   * CL_3[0,1,2]
        + b17.dummy[(n,0,0.2)|(n,0,0.2)|(n,0,0.2)|(n,0,0.2)] * W[W]
$


[Michiel Bliemer]

You need to distinguish how you put a scenario variable in the model and how you present it to respondents.

As I explained in my previous response, in SurveyEngine you show scenarios typically above the choice task, to make clear that this is the choice context across all alternatives. But in the model, you cannot add a scenario variable as a main effect in all alternatives since it is constant across all alternatives. In the same way, we cannot add constants in all utility functions as this overspecified the model, we need to normalise one of the constants to zero. So you need to normalise one of the scenario variables to zero by not including it in one of the alternatives, and then interpreting the coefficients of the other scenario variables as RELATIVE TO this alternative. It does not matter from which alternative you remove the scenario variable.

Michiel

[Joseph Wu]

Hi Michiel,

I am grateful for all your support as we prepare to start our pilot study.

We have incorporated "Opt-out" and "Status-quo" options in our SP experiments. To ensure we gather meaningful trade-offs, even when participants choose "Opt-out" or "Status-quo" initially, we have implemented a second choice to obtain additional information. After reviewing the forum, I noticed your suggestion to use "Best and second-best" choices instead of "Best-worst" in such cases. Given that we only have three alternatives per experiment, this approach allows us to obtain a complete ranking of participants' preferences.

My question is whether this "Best and second-best" setting will influence the experimental design when using Ngene, and if we need to adjust the script compared to my current one.

Code: Select all

design
;alts = pmm, ptsmm, car
;rows = 60
;block = 10
;eff = (mnl, d, mean)
;bdraws = sobol(200)
;cond:
if(ptsmm.IVTT_2=0, ptsmm.MMTT_2=[6,8]),
if(ptsmm.IVTT_2=1, ptsmm.MMTT_2=[4,6]),
if(ptsmm.IVTT_2=2, ptsmm.MMTT_2=[2,4]),
if(ptsmm.CL_2=0, ptsmm.IVTT_2=[0,1]),
if(ptsmm.CL_2=2, ptsmm.IVTT_2=[1,2]),
if(car.CL_3=0, car.IVTT_3=[1,2]),
if(car.CL_3=2, car.IVTT_3=[0,1]),
if(ptsmm.CL_2=1, car.CL_3=[1,2]),
if(ptsmm.CL_2=2, car.CL_3=2),
if(pmm.BW=1, ptsmm.BW=[1,2,3,4]),
if(pmm.BW=2, ptsmm.BW=[2,3,4]),
if(pmm.BW=3, ptsmm.BW=[3,4]),
if(pmm.BW=4, ptsmm.BW=4)
;con

;model:
U(pmm) = b1.dummy[(n,0.01,0.2)|(n,0.01,0.2)|(n,0.01,0.2)|(n,0.01,0.2)] * W[0,1,2,3,4]
        + b2.dummy[-0.02|-0.01]                            * MMTT[0,1,2]
        + b3.dummy[-0.02|-0.01]                            * RC[0,1,2]
        + b4.dummy[-0.04|-0.03|-0.02|-0.01]                * BW[0,1,2,3,4]
        + asc1[0]
/
U(ptsmm) = b5.dummy[0.02|0.01]         * IVTT_2[0,1,2]
        + b6[-0.01]                    * MMTT_2[2,4,6,8]
        + b7[-0.01]                    * WT_2[1,3,5]
        + b8.dummy[0.02|0.01]          * RC_2[0,1,2]
        + b9.dummy[0.02|0.01]         * CL_2[0,1,2]
        + b4                           * BW
        + asc2[0]
        + b15.dummy[(n,0.01,0.2)|(n,0.01,0.2)|(n,0.01,0.2)|(n,0.01,0.2)]* W[W]
/
U(car) = b10.dummy[-0.02|-0.01] * IVTT_3[0,1,2]
        + b11.dummy[-0.02|-0.01] * WLKST_3[0,1,2]
        + b12.dummy[-0.02|-0.01] * RC_3[0,1,2]
        + b13.dummy[-0.02|-0.01] * PC_3[0,1,2]
        + b14.dummy[0.02|0.01]   * CL_3[0,1,2]
$


Additionally, for the pilot study, should we continue using the MNL model or switch to the Ordered Logit model to obtain informative priors? What would be your recommendation? Also, what is your suggestion for model selection in the final analysis?

Thank you so much for your time and help.

Best regards,
Yikang

[Michiel Bliemer]

It is very challenging to optimise designs for best-best responses because it is not known in advance what best choices respondents will have and what the remaining alternatives are in the second best choice. For best and second best choices, I generally just use a design that is optimised for the first best choice and not for the second-best choice. This has never led to any issue and I suspect it will also be highly efficient for the second best choice.

I do not understand why you would want to use an ordered logit model because the set of alternatives {pmm, ptsmm, car, statuquo, optout} are not ordered/ordinal. For generating an efficient design, assuming an mnl model would be approriate. You can also include the status quo and optout alternatives into your utility functions during design generation.

Michiel