D-error of efficient designs in pilot surveys

[kimsauha]

Hello,

I am a student and doing choice experiment for the first time. This forum has been very helpful. I have a few questions about Bayesian design. Please excuse me if my questions have been asked before.

My experimental design is unlabeled, characterized by two product alternatives and a no-buy alternative. I am planning to use a Bayesian design. I am about to start the pilot study to define the priors and for this phase I am using a D-efficient design. Below is the script of my D-efficient design:

design
;alts = Alt1*, Alt2*, Alt3
;rows = 16
;eff = (mnl,d)
;block = 2
;model:
U(Alt1) = b1.dummy[0.00001|0.00001]*lc[2,1,0]
+ b2.dummy[0.00001|0.00001|0.00001]*hs[3,2,1,0]
+ b3.dummy[0.00001]*or[1,0]
+ b4[-0.00001]*price[7.99,9.99,10.99,12.99,16.99]/
U(Alt2) = b1.dummy*lc
+ b2.dummy*hs
+ b3.dummy*or
+ b4*price
$


As far as I understand, eight is the minimum number of choice tasks I can use to have my parameters significant at 95% of confidence level (I am planning to use a mixed logit model in my final analysis).
I am planning to use a D-efficient design with 16 choice tasks divided in two blocks of eight choice tasks since this is the D-efficient design I can obtain with the smallest D-error (0.24) while controlling for alternatives dominance and repetitions.

In my study, I will have a relatively small sample (I have different treatments and I can not afford more than 150 observations per each treatment) and I am afraid that blocking the design may request to have a bigger sample. If this is the case, my question is how important is the "size" of the D-error in pilot study phase. Can I use a design with a lower number of choice tasks (eight or twelve) but with a higher D-error (0.54 or 0.34) or is it better to use the 16 choice tasks in two blocks design with a smaller D-error (0.24)?

If the latter option is what you suggest, may I then use a design with 8 choice tasks (one block) in the final Bayesian design?

Thank you very much!

[Michiel Bliemer]

You have 7 parameters to estimate, and in mixed logit you will have 14 (K) parameters, and with 3 (J) alternatives you will need S*(J-1) >= K, that means the number of rows S should be S*(3-1) >= 14, hence S >=7. So indeed 8 rows would work. It is always wise to use more to add a bit more variation in your dataset, so 16 (blocked in 2) would be a good number I suppose. You can use 8 or 12 choice tasks as well, the efficiency per choice task is quite similar with the numbers you provide, but 16 is better than 8 (since 0.24 < 0.54/2). I would also use 16 choice tasks in your final Bayesian design, as 8 is really a minimum number and more variation in the model is likely needed for estimating mixed logit models.

Note that the above degrees of freedom calculation has nothing to do with the fact that all parameters are significant at the 95% confidence level. Without any reliable parameter priors, nothing can be said about the required sample size or significance. With dummy variables you will typically need a much larger sample than with continuous attributes.