Adding opt-out alternative

[kkasia]

Dear Ngene Experts,

What do you think about adding an opt-out ("none") alternative to the questionnaire AFTER a design without opt-out option was created?

Utility for opt-out alternative is set to zero, but it has an impact on design efficiency, so it seems to be essential to add an opt-out when creating efficient design. What about orthogonal designs? Is it fine to add "none" option after the design without this option was created?

The thing is that I created an orthogonal design with two alternatives for a pilot study without adding a "none" option. It was later decided to add it in the survey. Luckily it was only the pilot study, however I need to run the analysis to obtain estimates (and use them as priors for the main study efficient design) and now I am worried if lack of opt-out in the design can have an impact on the analysis.

I would be thankful for your thoughts.

Kasia

[Michiel Bliemer]

Adding an opt-out alternative after generating the design affects the efficiency of any design in practice.

An orthogonal design is essentially an efficient design assuming that coefficients are zero and some other assumptions. Under the assumption that coefficients are zero, adding an opt-out alternative does not change its efficiency and the design remains orthogonal. However, because the coefficients are not zero in practice, adding an opt-out alternative does change its efficiency. Since you are using an orthogonal design, I assume that efficiency is not that important to you in the first place. It is generally not a problem to add an opt-out alternative to an orthogonal or efficient design, you will still be able to estimate your model.

There is only one case in which adding an opt-out may create a problem, namely if the opt-out is a quite dominant alternative and most people would select it. For example, we have done an experiment in the past where we had 3 alternatives: (i) travel via toll road A, (ii) travel via toll road B, and (iii) stay at home (the opt-out). Given that we did this survey in a country that had no history with toll roads, about 90% of the respondents selected "stay at home" as some sort of protest vote against toll roads. Given that we were interested in determining willingness to pay for toll roads, this would mess up our analysis. However, it is easy to overcome this problem. When adding an opt-out, we always include an additional forced (conditional) choice. After first asking the (unconditional) choice including the opt-out, if they select the opt-out, we ask a second question: "Which road would you choose if you HAVE to travel?" in which you omit the opt-out alternative. This way, you capture both choices, including and excluding the opt-out. Given that they have already looked at the attribute levels for their first choice, their second choice can be made fairly quickly. You can use both responses in estimating your model.

Michiel

[kkasia]

Thank you for your answer and explanations, Michiel!
Now I see it more clearly.

Nice example with toll roads. To be honest I am a bit reluctant when it comes to adding an opt-out, but you showed me that forced choice after opt-out is a perfect solution to overcome the issue and my prejudices.

In that case, when creating a design for the survey with both opt-out and forced question, we do not have to include opt-out, right? Still a bit confused here.

Kind regards,
Kasia

[Michiel Bliemer]

Yes you can leave out the opt-out in the design of the experiment.

Michiel

[kkasia]

Thank you!

Kasia