C-error in WTP efficient designs

[claudiab]

Dear all,
I am working on a CE study where we will present for each choice task two product alternatives and a no-buy alternative.
I have generated a D-efficient design, with constrains and priors = 0, that has been used in a pilot survey with 100 subjects. I have estimated a MNL model to obtain the priors for the generation of a Bayesian design. Specifically, following suggestions on the forum, I have divided by 2 the priors (both mean and st errors), and specified the mean values = 0 of the parameters which were statistically insignificant in the MNL model. I have assumed a normal distribution for all priors.
I would like to generate a Bayesian design on the basis of WTP efficiency criterion. I am not very familiar with this kind of design and I have some questions . Here is the syntax I have used:

Code: Select all

Design
;alts = alt1, alt2, nobuy
;rows = 48
;block =12;
;eff = (mnl,wtp(wtp1))
;wtp = wtp1(*/b1)
;con
;cond:
if (alt1.INV=[0], alt1.COM=[0]),
if (alt2.INV=[0], alt2.COM=[0]),
if (alt1.COM=[0], alt1.INV=[0]),
if (alt2.COM=[0], alt2.INV=[0])
;model:
U(alt1) = b1[-0.23]*PPPRICE[1,2,3,4]         
        + b2.dummy[(n,0.70,0.13)|(n,0.36,0.12)|(n,0.33,0.12)]*STANDARDS[3,2,1,0]
        + b3[(n,3.18,0.44)]*FPRICE[0.05,0.1,0.2,0.3]
        + b4[(n,0,1.60)]*INV[0,0.05,0.10]
        + b5.dummy[(n,0.60,0.18)|(n,0.58,0.18)|(n,0,0.18)]*COM[3,2,1,0]
/U(alt2) = b1*PPPRICE
        + b2*STANDARDS
        + b3*FPRICE
        + b4*INV
        + b5*COM/
U(nobuy)= b0[(n,0,0.24)]$


First of all, I do hope I have used the appropriate syntax and please see below the efficiency estimates I have obtained:

Code: Select all

             Bayesian                        
       Fixed   Mean   Std dev.   Median   Minimum   Maximum            
D error   0.482119   0.491894   0.006665   0.490296   0.48222   0.521154            
A error   6.511585   6.676097   0.29205   6.647395   6.237344   7.796395            
B estimate   84.551825   81.475317   8.39142   83.43224   55.881892   95.484354            
S estimate   11.894459   115231.9194   840837.3707   1088.419697   34.102464   8485557.686            
WTP(wtp1) estimate   1134.533193   1231.279807   109.305549   1206.002631   1067.009648   1840.774782            
WTP(wtp1) n   Infinity   115228.9605   840822.1028   1092.704408   39.391768   8484252.056            
                              
Prior   b1   b2(d0)   b2(d1)   b2(d2)   b3   b4   b5(d0)   b5(d1)   b5(d2)   b0
Fixed prior value   -0.23   0.7   0.36   0.33   3.18   0   0.6   0.58   0   0
Sp estimates   4.514377   4.114633   10.582062   11.894459   3.79845   Undefined   9.556447   10.049788   Undefined   Undefined
Sp t-ratios   0.92248   0.966252   0.602519   0.568308   1.005664   0   0.634027   0.618269   0   0
Sb mean estimates   4.611486   4.741807   21.746327   39.386371   4.155939   17954.6242   2164.039473   18.180973   92730.11347   3073.650829
Sb mean t-ratios   0.912947   0.954193   0.594327   0.558814   0.990834   0.17711   0.619799   0.603889   0.157729   0.235089


I am not sure whether in WTP efficient designs using priors=0 would be ok.
Second, as far as I understood, “WTP(wtp1) estimate” would indicate the C-error values (please correct me if I am wrong) and should suggest the minimum number or design replications necessary to obtain statistically significant WTP values. Basically, my output tells me that, assuming my priors are correct, I need a number of 1134 * #blocks (12) respondents to obtain statistically significant WTP estimates. Did I understand correctly?
Also, what is the difference between “WTP(wtp1) estimate” and “WTP(wtp1) n”?
I apologize in advance if this has already been source of discussion on the forum or if I have missed the explanation on the manual.
Thanks a lot,
Claudia

[johnr]

Hi Claudia

With the exception of the denominator in a defined WTP estimate using C-error, you can apply any prior to the design, including zero. Unfortunately, zeros can cause issues in calculating efficiency measures such as sample size requirements. The sample size estimate is basically stating the smallest sample size you need to detect whether a parameter is statistically significantly different from zero (assuming a t of 1.96) from zero. The smaller the parameter (prior), the larger the sample size required to detect if it is different from zero.

If you look through the first block of output, the first column of numbers starting with 0.482119 represents the efficiency measures using any fixed priors and the means of the Bayesian priors – for example, -0.23 for b1 and 3.18 for b3. The 0.482119 is the D-error assuming the fixed priors and treating the means of Bayesian priors as fixed priors. The 1134.533193 you refer to is the C-error estimate (i.e., (wtp1) estimate) using these priors. The second column in this block of output are the Bayesian errors. For the C-error, the Bayesian C-error is actually 12.31.279807. Note that this is the C-error, not the WTP sample size which is the last row within this block of output.
Note that for b4, the mean of the Bayesian distribution is 0, and hence the sample size required to detect whether this is statistically different from 0 is infinite. This is what is being reported – in the row labelled WTP(wtp1) n – if you have a zero prior, you need an infinite sample size. The next value, 115,228.9605 is the actually number of design replications needed. Again, this is very large number simply because you have zero priors (same reasoning only it is taking draws and didn’t hit 0, but in the draws, did take values close to zero – mathematically, this number should be 0 also, but the number reported is simply simulation error). But this is the number you are after.

To demonstrate, if you use only parameters with non-zero priors in your calculation (e.g., ;wtp = wtp1(b2,b3/b1), you will see the number of design replications drop to under 60 in a few seconds. Note I didn’t include b5 in this, as one of the dummies has a zero mean, which I don’t think can be isolated in the syntax.

John

[claudiab]

John,
Very appreciated. Everything is clear. Thank you very much.
Finally, I followed your suggestion and ran the following syntax:

Code: Select all

Design
;alts = alt1, alt2, nobuy
;rows = 48
;block =12;
;eff = (mnl,wtp(wtp1))
;wtp = wtp1(b2,b3/b1)
;con
;cond:
if (alt1.INV=[0], alt1.COM=[0]),
if (alt2.INV=[0], alt2.COM=[0]),
if (alt1.COM=[0], alt1.INV=[0]),
if (alt2.COM=[0], alt2.INV=[0])
;model:
U(alt1) = b1[-0.23]*PPPRICE[1,2,3,4]         
        + b2.dummy[(n,0.70,0.13)|(n,0.36,0.12)|(n,0.33,0.12)]*STANDARDS[3,2,1,0]
        + b3[(n,3.18,0.44)]*FPRICE[0.05,0.1,0.2,0.3]
        + b4[(n,0,1.60)]*INV[0,0.05,0.10]
        + b5.dummy[(n,0.60,0.18)|(n,0.58,0.18)|(n,0,0.18)]*COM[3,2,1,0]
/U(alt2) = b1*PPPRICE
        + b2*STANDARDS
        + b3*FPRICE
        + b4*INV
        + b5*COM/
U(nobuy)= b0[(n,0,0.24)]$


And yes, everything drops down! As far as I understood, the C-error relates to the efficiency of the design at the generation stage, but C-error estimates = 97.01 is still big, right?
Also, I have noticed that the four price levels are allocated in a way that respondents are always presented with the trade-off among the two product alternatives between only two out of all the possible combinations. I always have 1 - 4 and 2 - 3 combinations and never 2 - 4 or 2 - 1, for example. Am I doing something wrong?
Thanks again for everything,
Claudia

[johnr]

Hi Claudia

Yes, the C-error is quite large which is an indication that there may be something not quite right with the design. However, before I get to that, I just noticed that you are not optimising for your Bayesian priors. The command

;eff = (mnl,wtp(wtp1))

will optimise on the fixed priors, even though you have specified Bayesian priors. The command as you have it will show the Bayesian priors but not optimise on them, You need to tell it what to do with the Bayesian priors as in:

;eff = (mnl,wtp(wtp1),mean)
;bdraws = halton(200)

Now back to your design. To look for issues, start simple. Remove the Bayesian priors and start with a fixed prior design as in

Code: Select all

Design
;alts = alt1, alt2, nobuy
;rows = 48
;block =12;
;eff = (mnl,wtp(wtp1))
;wtp = wtp1(b2,b3/b1)
;con
;cond:
if (alt1.INV=[0], alt1.COM=[0]),
if (alt2.INV=[0], alt2.COM=[0]),
if (alt1.COM=[0], alt1.INV=[0]),
if (alt2.COM=[0], alt2.INV=[0])
;model:
U(alt1) = b1[-0.23]*PPPRICE[1,2,3,4]         
        + b2.dummy[0.70|0.36|0.33]*STANDARDS[3,2,1,0]
        + b3[3.18]*FPRICE[0.05,0.1,0.2,0.3]
        + b4[0]*INV[0,0.05,0.10]
        + b5.dummy[0.60|0.58|0.18]*COM[3,2,1,0]
/U(alt2) = b1*PPPRICE
        + b2*STANDARDS
        + b3*FPRICE
        + b4*INV
        + b5*COM/
U(nobuy)= b0[0.24]$

Running this syntax, and computing the choice probabilities (you can get these out of Ngene), you will note that the average choice probability for the three alternatives is around 0.375, 0.375, 0.25. Now increase the range of the price levels to [1,3,5,7] and you will see the average probabilities for the alternatives go to 0.33,0.33,0.33 (as well as huge improvements in the efficiency obtained). Efficient designs typically have an optimal set of choice probabilities (sometimes referred to as magic Ps). Given that the choice probabilities of all alternatives are a function of the combination of all levels, the algorithm will try and find the combination of levels that produce these probabilities. In this case, it wants to find choice probabilities of 0.33,0.33,0.33 but given the ranges imposed on the levels, it can't. To get the best choice probabilities it can, it is choosing the price level combinations you are seeing.

As you will see, increasing the range of the levels will generally improve efficiency. But be careful, the study requirements should in the most part rule the design, not the math underlying the design rule the study requirements. Unless finding a design proves intractable for the study, I personally don't like changing the study inputs.

John

[claudiab]

John,
Thanks a lot for this. I really appreciate it. Ok.. if I thought everything was clear, now it is even more clear!
Re the magic Ps, I ran the syntax without the Bayesian Priors, but I did not obtain any improvements in terms of C-error though. However, I “solved” the problem of price levels allocation by eliminating the “;con” command (that, I think, would not be actually correct). Anyway, as you suggested, I got the best output by increasing the price levels range, but I do not think this is behaviorally feasible since they would not reflect the real market prices of the product.
Thank you! I learned a lot!
Ciao ciao,
Claudia