Design-choice without certain prior knowledge

[AnyM]

Hello,

first of all thanks for providing this forum for help, I already learned a lot scrolling through other threads! But now, before I start my experiment, I am unsure about which final design to choose and would be very thankful for some guidance.
I want to design a choice experiment where I assess the willingness to participate of people. I’ve got 6 attributes, two with two levels (dummys) and four with three levels. I want to check for two interaction effects and I aim to calculate the willingness to get for participation at the end.
However, I am unsure on which design I should use. In the literature, there have been other studies conducting choice experiments on my topic, some using orthogonal designs, some using d-efficient designs with pilot studies.
I won’t be able to do a pilot study and the results from the other studies are very different to each other (ranging from 0.05 to 0.14 for example). Additionally, in all these studies, it has been assumed that the contract for participation would be binding. But I’d like to add a dummy that makes the contract flexible, which will likely affect the influence of certain attributes, so the results from other studies would be quite misleading. I’m now considering whether I should work with bayesian efficient designs, with efficient designs and only giving the parameter signs, or if I should just use orthogonal designs as well?
So I’ve got three design ideas. Could you help me in deciding which one would be better for my case, or how I should change them?

First, orthogonal (here I’d need to delete one attribute so that I dont get too many choise sets. This is a bit problematic but doable. But it is my least preferred way):

Code: Select all

Design
;alts=alt1, alt2, alt3
;rows=12
;orth=seq
;eff = (mnl,d)
;wtp = wtp1(b2,b3/b5)
;model:
U (alt1)= b2*A[0,1,2] +b3*B[0,1,2]
+b4.dummy*FLEX[0,1] +b5*E [0,1,2] + b6*F[0,1,2]
+b7*FLEX*A + b8*FLEX*B /
U (alt2)= b2*A+b3*B+b4.dummy*FLEX+b5*E+ b6*F+b7*FLEX*A + b8*FLEX*B
$

Second, D-efficient only considering signs:

Code: Select all

? final survey design
design
;alts = alt1, alt2, alt3
;rows = 12
;eff = (mnl,d);
wtp = wtp1(b2,b3/b6);
model:
U(alt1) = b2[-0.001]*A[0,1,2] + b3[0.001]*B[0,1,2]
+b4.dummy[0.001]*FLEX[0,1] +b5.dummy[0.001]*ZYKLUS[0,1]
+ b6[0.001]*E[0,1,2]+ b7[0.001]*F[0,1,2]
+b8[0.001]*FLEX*A + b9[-0.001]*FLEX*B /
U(alt2) = b2*A + b3*B
+b4.dummy*FLEX +b5.dummy*ZYKLUS
+ b6*E+b7*F
+b8*FLEX*A + b9*FLEX*B
$

Third, bayesian:

Code: Select all

? final survey design
design
;alts = alt1, alt2, alt3
;rows = 12
;eff = (mnl,d,mean)
;wtp = wtp1(b2,b3/b6)
;model:
U(alt1) = b2[(u,-0.1,-0.01)]*A[0,1,2] + b3[(u,0.1,0.9)]*B[0,1,2]
+b4.dummy[(u,0,0.5)]*FLEX[0,1] +b5.dummy[(u,0,0.1)]*ZYKLUS[0,1]
+ b6[(u,0.01,0.8)]*E[0,1,2]+ b7[(u,0.01,0.8)]*F[0,1,2]
+b8[(u,0,0.1)]*FLEX*A + b9[(u,0,0.1)]*FLEX*B /
U(alt2) = b2*A + b3*B
+b4.dummy*FLEX +b5.dummy*ZYKLUS
+ b6*E+b7*F
+b8*FLEX*A + b9*FLEX*B
$

Any comment is appreciated. Thank you!

Kind regards

AnyM

[Michiel Bliemer]

A few things to note:

1. You cannot use ;wtp if you have near-zero priors because you would be dividing by zero, you should only use ;wtp with reliable non-zero priors.
2. Uninformative Bayesian priors are possible, but it is difficult setting appropriate ranges for the uniform distribution.
3. You need to add a constant to alt1 and alt2, relative to the opt-out alternative.
4. Your number of rows is very small, I would suggest using 24 rows (with 2 blocks) or preferably 36 rows (with 3 blocks of 12). This does not increase the number of choice tasks you give to a respondent, but you increase variety in your data, which benefits model estimation. You can only randomly select X choice tasks out of 36 to give to a respondent. If you are using an online survey instrument such as SurveyEngine then this can be done automatically.
5. Dummy coded attributes need to be specified such that the last level is the base level, so b4.dummy[0.01] * FLEX[0,1] means that level 0 is preferred over level 1. If this is incorrect, you need to set b4.dummy[0.01] * FLEX[1,0], where the last level 0 is the base level with zero utility, and the utility for level 1 is 0.01.
6. If all your priors are zero, then it is often best to dummy code all attributes (just for the sake of generating the design, you do not need to dummy code all attributes in model estimation) when you are generating an efficient design as I have suggested several times on the forum (I will not explain here again why).

So my preferred script would be this:

Code: Select all

? final survey design
design
;alts = alt1, alt2, optout
;rows = 36
;block = 3
;eff = (mnl,d)
;model:
U(alt1) = b1[0]
        + b2.dummy[-0.01|-0.02]  * A[1,2,0]
        + b3.dummy[0.01|0.02]    * B[1,2,0]
        + b4.dummy[0.01]         * FLEX[0,1]   ? perhaps should be FLEX[1,0]
        + b5.dummy[0.01]         * ZYKLUS[0,1] ? perhaps should be ZYKLUS[1,0]
        + b6.dummy[0.01|0.02]    * E[1,2,0]
        + b7.dummy[0.01|0.02]    * F[1,2,0]
        + b8[0.001]              * A * FLEX.dummy[1]
        + b9[-0.001]             * B * FLEX.dummy[1]
        /
U(alt2) = b1+b2*A+b3*B+b4*FLEX+b5*ZYKLUS+b6*E+b7*F+b8*FLEX*A+b9*FLEX*B
$

Michiel

[AnyM]

Hello Michiel,

thank you for your answer, this helped a lot!
The point regarding the wtp makes sense to me. As I understand, I can still calculate the willingness to pay at the end of the survey from my results? Then what is the statistical disadvantage of doing it after, without specifying it in the design beforehand? I see that the C-error might not be optimised in that case, but what does that mean for the final results?

The rest is very clear to me, thanks again.

[Michiel Bliemer]

Yes you can still calculate the WTP in your study. The D-error optimises for estimating the beta's (i.e., it minimises the standard errors of beta's), while the C-error optimises for estimating ratios of beta's (i.e., it minimises the standard error of beta_k/beta_cost, which is a subtle difference). Both will result in an optimised design and in 99% of the studies the D-error is used instead of the C-error, even when WTP estimates are the objectives, so you will be fine.

Michiel