Comparing the results of similar experiments

This forum is for posts covering broader stated choice experimental design issues.

Moderators: Andrew Collins, Michiel Bliemer, johnr

Comparing the results of similar experiments

Postby Brig » Wed Oct 12, 2022 12:34 am

Hi,

I am conducting a choice experiment for which I would like to compare the results obtained by two different groups (say, A and B). My question relates to: How identical do the choice experiments have to be between the two groups in order to allow the comparison?

For example, I generate two designs using the same model (same Bayesian efficient design with the same priors, means and standard variations) but I block it into two different numbers (n for group A and 2n for group B).

I end up with two designs that have similar D-efficiencies (1% difference, although I'm not sure they can be compared even in the case where the number of blocks varies?) with identical specifications except for the number of blocks. However, the choice cards for the two designs are different.

My question is: Can the results of the two choice experiments be compared if the cards/number of blocks are not identical?

Thanks a lot for you advice on that,

Baptiste
Brig
 
Posts: 3
Joined: Wed Aug 17, 2022 9:02 pm

Re: Comparing the results of similar experiments

Postby Michiel Bliemer » Wed Oct 12, 2022 8:55 am

Yes, you can compare results of the two groups even if the choice tasks are different. Of course you cannot rule out that differences across group A and B are caused by differences in choice tasks, but this is generally not an issue since parameter estimates are unbiased no matter what choice tasks you use and should converge to the same parameters with large sample sizes no matter what choice tasks you use.

What you can do is generate an efficient design with (near) zero priors and use that design for both groups. If the levels are different for the groups, you can simply relabel the attribute levels of the design. Using zero priors, such relabelling retains the efficiency of the design.

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm


Return to Choice experiments - general

Who is online

Users browsing this forum: No registered users and 3 guests