Comparing the results of similar experiments

[Brig]

Hi,

I am conducting a choice experiment for which I would like to compare the results obtained by two different groups (say, A and B). My question relates to: How identical do the choice experiments have to be between the two groups in order to allow the comparison?

For example, I generate two designs using the same model (same Bayesian efficient design with the same priors, means and standard variations) but I block it into two different numbers (n for group A and 2n for group B).

I end up with two designs that have similar D-efficiencies (1% difference, although I'm not sure they can be compared even in the case where the number of blocks varies?) with identical specifications except for the number of blocks. However, the choice cards for the two designs are different.

My question is: Can the results of the two choice experiments be compared if the cards/number of blocks are not identical?

Thanks a lot for you advice on that,

Baptiste

[Michiel Bliemer]

Yes, you can compare results of the two groups even if the choice tasks are different. Of course you cannot rule out that differences across group A and B are caused by differences in choice tasks, but this is generally not an issue since parameter estimates are unbiased no matter what choice tasks you use and should converge to the same parameters with large sample sizes no matter what choice tasks you use.

What you can do is generate an efficient design with (near) zero priors and use that design for both groups. If the levels are different for the groups, you can simply relabel the attribute levels of the design. Using zero priors, such relabelling retains the efficiency of the design.

Michiel