Cond with require / Dummy or not / working with SurveyEngine

[Joseph Wu]

Hi Michiel,

The alternative "Car" is the Status-quo in this context. I have designed several experiments for different segments of participants based on their current mode of choice. Asking "First and Second-best" allows me to understand their ranking of the three options, which is why I believe the Ordered Logit model would be appropriate for this scenario.

If we apply the MNL model for an efficient design, should the model be built solely on their first choice, excluding the second choice?

Best regards,
Yikang

[Michiel Bliemer]

Car as status quo alternative has (possibly self-reported) attribute levels, therefore for an efficient design you would generally include this alternative and its levels to the model specification. The same for an optout alternative, you would typically add this to the model specification, where the optout alternative has 0 utility or a constant.

You should only use an ordered logit model if the dependent variable has a natural ordering (e.g. number of cars, departure time, etc). Mode of transport does not have a natural ordering and therefore an ordered logit model is not appropriate. Obtaining a full ranking per choice task has nothing to do with ordered logit. If you have best-best or best-worst data, you typically explode the data and estimate an MNL model (or mixed, latent class etc).

I would design the experiment based on the first choice, excluding the second choice. There does not exist software that can optimise designs for both the first and second choice, although it is theoretically possible.

Michiel

[Joseph Wu]

Dear Michiel,


Thank you for all your support. Our study has now been distributed for the pilot phase.

Since we are using pivoted attribute levels, we aim to apply the 'mean' value for the experimental design. Additionally, we use best and second-best choices to obtain a complete ranking of alternatives.

I want to confirm the process for experimental design after the pilot study:

Building an MNL model based on the first choice from the pilot data to obtain priors.
1. Calculating the mean values of each pivoted attribute for all levels.
2. Applying the priors and mean values of each level in Ngene to create an informative experimental design.
3. My question is whether we should consider Bayesian priors in this process.

Another question pertains to modeling: In your study (Rose, J. M., Bliemer, M. C., Hensher, D. A., & Collins, A. T. (2008). Designing efficient stated choice experiments in the presence of reference alternatives. Transportation Research Part B: Methodological, 42(4), 395-406), how do you handle best and second-best choices? You mentioned in the CMA course that introducing second-best choices can help mitigate status-quo bias to some extent and ensure meaningful trade-offs from participants. Given that a majority of participants may choose the status-quo, how would you build a model based solely on the best choice? Alternatively, would you consider building exploded models using more than one choice?

Thank you for your guidance.

Best regards,
Yikang

[Michiel Bliemer]

Assuming that you are talking about numerical attributes, you simply compute the mean attribute level for each attribute to define your status quo alternative. Alternatively, if your attribute levels vary widely across the population, you generate multiple designs, e.g. one for short distance, one for medium distance, and one for long distance.

Then you estimate the model, possible segmented across the population as above, and obtain parameter estimates and standard errors. These inform Bayesian priors.

If you only want to use the first-best choice, then you simply ignore the second-best choice observation in your data analysis.
If you want to use both first-best choice and second-best choice, then you use all the data whereby you indeed create two choice sets, one for the first-best choice whereby all alternatives are available, and another choice set whereby the chosen alternative is removed.

Michiel

[Joseph Wu]

Dear Michiel,

I am building MNL models on my pilot data to update the priors for efficient design.

At this stage, I have constructed simple MNL models that only include the experiment attributes appearing in the DCEs and do not consider interactions. As a result, the model performance is not optimal, but the coefficients are reasonable. Is this acceptable for now, or should I enhance the model to obtain more accurate coefficients?

Additionally, instead of using the coefficients as fixed priors, I am considering using the standard errors to inform Bayesian priors for experimental design in Ngene. Would this be a better approach?

Best regards,
Yikang

[Michiel Bliemer]

It is fine to not estimate interaction effects, you can choose to also omit then in the experimental design phase or include them with a zero prior.

Bayesian priors are preferred in most cases. If the standard errors are very large and you are worried about extreme draws from the distribution, you may want to use eff = (mnl,d,median) instead of mean, together with bdraws = sobol(...).

Michiel