Orth + eff OR orth = ood OR eff design

[Benjamin]

Hi!

I am totally new, have read through all relevant posts and most parts of the manual, but still struggles, and have really tight time limit.

We are running a DCE on 1200 respondents in Tanzania on preferences about president candidates and gas revenue distribution (3 regions 400, each), as part of a bigger survey.
We are piloting on thursday, and have then 1 day to make changes => if we want more efficient design.

For Pilot we used these attributes:

gender (man, woman)
party (ruling, oposition)
corruption (never, accused)
education (free, not free)
reg_nat (equal share, gas region benefit)
expectations (exaggerate, moderate)
earnings increase (as result of president coming to power)

Design
;alts = President_A, President_B
;rows = 20
;orth = seq
;eff = (mnl, d)
;block = 2, minsum
;model:
U(President_A) =
b1 * gender [0,1] +
b2 * party [0,1] +
b3 * corruption [0,1] +
b4 * education [0,1] +
b5 * reg_nat [0,1] +
b6 * expectations [0,1] +
b7 * earnings_increase [1000,5000,10000,15000] /
U(President_B) =
b1 * gender +
b2 * party +
b3 * corruption +
b4 * education +
b5 * reg_nat +
b6 * expectations +
b7 * earnings_increase $


QUESTIONS

1. Based on sample size (1200), nr of choice sets (10*2), attributes (7). and no priors, should I use:
orth = seq, eff = (mnl,d) => d-error = 0.02
OR
orth = ood => gives d-Optimality = 95%
OR
something else?

2a. If we receive priors (OR make some guesses). How much better do you think our choice sets will be? What improves? Smaller st.dev?
2b. Should we then remove ;orth=seq, but keep ;eff=(mnl, d) ? Or should we add some more "stuff"?

Really sorry about basic questions, but we have used Orthogonality because we dont understand the others, would love to get these questions answered .

Thanks!

[Michiel Bliemer]

1. I would use eff with zero priors and remove orth. A D-efficient design with zero priors is near-orthogonal, so imposing another constraint is not really necessary. If you insist on imposing strict orthogonality, I would go with your current syntax, I would no go for ood.

2a. It is difficult to tell how much better the choice sets will be since this is case specific. It would be a lot, it would be little. But what changes are the standard errors, and hence you will have higher t-ratios and more statistically significant parameter estimates. When using priors, it is best to use Bayesian priors in order to avoid prior misspecification. Using zeros (as you have for the pilot study) is always safe.

2b. Yes, I would remove ;orth = seq.

In addition, if you know some of the signs of the parameters (I think you will for several of them), then please use b[-0.0000001] and b[0.0000001] for negative and positive signs, respectively. This will not affect the D-error (much), but it will allow Ngene to check for dominant alternatives in which the attributes of one alternative are always better than the attributes of the other alternative. In order for Ngene to do this check, you will need to use ;alts = President_A*, President_B* (i.e., simply add an asterisk to each alternative).

Further your syntax is fine I think.

[Benjamin]

Thanks!!
This is really helpful!

Bayesian sounds good, I'm going for that. But when I read through the forum there are so many options one can add when doing Bayesian..

Is this all I need?: (run 1 time and than done?)

Code: Select all

;alts = President_A, President_B
;rows = 20
;eff = (mnl, d, mean)
;block = 2, minsum
;model:

U(President_A) =
b1[(n,0.0002,0.00001)] * gender [0,1] +
b2[(n,0.0002,0.00001)] * party [0,1] +
b3[(n,0.0002,0.00001)] * corruption [0,1] +
b4[(n,0.0002,0.00001)] * education [0,1] +
b5[(n,0.0002,0.00001)] * reg_nat [0,1] +
b6[(n,0.0002,0.00001)] * expectations [0,1] +
b7[(n,0.0002,0.00001)] * earnings_increase [1000,5000,10000,15000] /

U(President_B) =
b1 * gender +
b2 * party +
b3 * corruption +
b4 * education +
b5 * reg_nat +
b6 * expectations +
b7 * earnings_increase $


QUESTIONS

1. I must state a nr of draws?
2. rdraws or bdraws?
3. Halton, gauss or someone/something else?
and how many?
4. wtp+ref..?
5. rep?
6. is MNL correct or MMNL?

7. From this post (viewtopic.php?f=2&t=219) you mention to make different models, should I do that when making a Bayesian design?

8. When deciding parameters, and only stating one parameter, but in "gender" I have man=1 and woman=0, if I chose parameter [-0,1], does that mean:
A: gender has negative effect on the y (or utility)
B: woman has stronger eff than man?
C: something else..? sorry but i really struggle here...
-How is the relation between man and woman in the parameter? is the man's parameter 0, but woman -0,1 ?

Thanks again for all help!

[Michiel Bliemer]

I was actually suggesting a Bayesian efficient design after you have done the pilot study, since the Bayesian priors come out of model estimation using the pilot study data set (i.e., you use the parameter estimates as the mean of the normal distribution, and the standard errors as the standard deviation). You can also use Bayesian priors for the pilot study, but I am not sure what you should choose for your standard deviations, so that is why I only advised to use Bayesian priors for the main study, it is not that important for the pilot study.

You need to add ;bdraws (Bayesian draws) to your syntax, e.g.
;bdraws = gauss(3)

This will do 3^7 = 2187 draws per design evaluation. You can also opt for gauss(2) or sobol(500) or something like that.

[Benjamin]

Yes, I was talking about the actual survey with Bayesian

The pilot is done, but we are changing attributes, so the only thing we know is sign!

QUESTION: Do we then use eff design or Bayesian if we only have sign?

Is this good for that?:

Code: Select all

Design
;alts = President_A*, President_B*
;rows = 20
;eff = (mnl, d)
;block = 2, minsum
;model:
U(President_A) =
b1[0.000001] * gender [0,1]       +
b2[0.000001] * religion[0,1]       +
b3[0.000001] * party [0,1]          +
b4[0.000001] * corruption [0,1]   +
b5[0.000001] * reg_nat [0,1]       +
b6[0.000001] * expectations [0,1] /

U(President_B) =
b1 * gender       +
b2 * religion      +
b3 * party          +
b4 * corruption  +
b5 * reg_nat       +
b6 * expectations $

[Michiel Bliemer]

See my post earlier, the sign is used for removing dominant alternatives. You put all positive values in the design now, I am not sure whether that is correct (I would expect a negative sign for corruption, but maybe I misinterpret). You can always use an efficient design. The more you know about the parameters, the better. If you only know the sign, this is more information than just putting in zeros. An orthogonal design assumes all zero priors and does not optimise, so you will always be better off than an orthogonal design.

[Benjamin]

Thanks!

Yes all are positive, (never done corruption)..

I know all signs, should I use 0.000001 OR 0.01
When I use 0.01 the d-error is 0,2 but S = 900..

[Michiel Bliemer]

The S-error has no meaning when the priors are not provided. Sample sizes only make sense if you use priors that describe actual behaviour. Clearly, if the priors are zero, the S-estimate is undefined.