choice-metrics.com

I am new to choice experiments (and even more so) to design of those experiments.
I am trying to design a first trial run to get better priors for a larger experiment; People would need to choose between 3 alternatives (2 unlabelled, and 1 SQ ; The attributes are YIELD (3 levels), DROUGHT (3 levels); ENV (effect coded; 3 levels); and CULT (effect coded; 2 levels). I had to add some constraints for combinations not making sense and arrive at the code attached.

However I have questions regarding the use of * to eliminated dominated strategies:

Given the fact that alt3 is the status quo, can I also attach a * ; my understanding is that SQ is "labeled" and therefore we cannot use the *; but when I not include the * then I get several combinations where the SQ is the obvious winner such as that one:

alt1 alt2 alt3
pay 800 800 0
yield 400 400 400
drought 100 100 100
env 0 1 2
cult 1 0 1

Thank you in advance to experienced users for their insights.



Design
?Bayesian efficient design
;alts = alt1*, alt2*, alt3
;rows=24
;block=4
;eff = (mnl,wtp(ref1),mean)
;wtp = ref1(y,d,e,c/pay)
;cond:
if(alt1.DROUGHT=100, alt1.YIELD=400),
if(alt2.DROUGHT=100, alt2.YIELD=400),
if(alt1.DROUGHT=50, alt1.YIELD=[600,1200]),
if(alt2.DROUGHT=50, alt2.YIELD=[600,1200]),
if(alt1.DROUGHT=33, alt1.YIELD=[600,1200]),
if(alt2.DROUGHT=33, alt2.YIELD=[600,1200])
;

model:
U(alt1) = pay[-.001] * PAY[800,1600,3200]
+ y[(u,0.003, 0.006)] * YIELD[400, 600, 1200]
+ d[(u,-.1, -0.07)] * DROUGHT[100, 50, 33]
+ e.effect[(u,0.15,0.2)|(u,0.25,0.3)] * ENV[1,2,0]
+ c.effect[(u,0.15,0.25)] * CULT[1,0] /
U(alt2) = pay * PAY + y * YIELD + d * DROUGHT + e.effect * ENV + c.effect* CULT /
U(alt3) = ASC[.5] + y * YIELDsq[400]+ d * DROUGHTsq[100] + e.effect* ENVsq[1,2,0](0,24,0) + c.effect* CULTsq[1,0](24,0)
$

You can (and probably should) include a * for the status quo alternative to avoid dominant alternatives.

What you may find (not sure) is that you have too many constraints in your design, as you have already put in strong conditions (such that only 5 combinations exist between DROUGHT and YIELD), and you are trying to find 24 choice tasks of which none contain any dominant alternatives, while there is not much room in your dimensions to find them (you only have a few levels per attribute). It may be easier to find designs with less than 24 choice tasks, and it may be easier finding them with less levels.

Further, I think your priors are perhaps not very good. I used your syntax to generate a design, and alt3 is chosen with very very low probabilities, they are almost zero. It has a DROUGHT of 100, with a quite negative prior (all the way to minus 1), such that DROUGHT now completely dominates your design efficiency.

Michiel

Thank you indeed for your very useful tips. I have reduced the number of scenarios to 18 into 3 blocks; but how do I know this is still ok to capture enough information for the analysis later on? I understand the different efficiencies measures are there to help, but it is not obvious to me to decide between a 18 scenario set and a 24 scenarios sets. In other words, I do not have clear cutout points!

On a matter related to dominated/ing strategies, it is not clear to me whether we are looking for a balanced probability between the 2 unlabelled alternatives (alt1 and alt2 in my case) and not too many SQ when we observe probabilities; or if we are looking for a balanced design between the 3 alternatives (ideally by looking at utilities, 6 should fall into alt2 6 into alt2 and 6 into alt3)

I am not sure I was clear... let me know if my question made sense?

Best

Damien

Since you only have a 5 coefficients (K) and you have 3 alternatives (J). The number of choice tasks should be at minimum S, where S*(J-1)>=K, in other words, S = 2 (only two questions) would in theory already be enough. You have 18 which is more than enough. You can also choose for 6 or 12, and you can inspect and compare the efficiencies. Clearly, with a smaller design, you will need more respondents. You can look at the sample size (s-) estimates, noting that these estimates in Ngene are the number of design replications, so if you have a design with 18 rows and you need 10 replications, and another design with 6 rows that require 25 replications, then the 6-row design is more efficient because 6*25 < 18*10. Sometimes large designs become more inefficient as Ngene will try to include the most efficient questions first, and then fill the rest of the design with less efficient questions.

Regarding utilities/probabilities, you want to see for two alternatives probabilities like 0.8-0.2 or 0.6-0.4, but not 0.99-0.01 (dominant altenative) and not 0.5-0.5 (indifferent). For three alternatives, you probably should want probabilities like 0.2-0.3-0.5 and not 0.1-0.9-0.0.

About all of the above is written in literature, and is taught in courses we and others give in experimental design. I hope that helps.

Michiel

Thank you for sharing this with me. It should clearly simplify my task.

Damien