Several questions on design for pilot

[feppink]

Beste Michiel,

After several years, I have found my way back to Ngene and I have a few questions that I hope you can respond to. For context: my colleagues and I will be conducting a WTP study on recreational values in a park. Every respondent will have a status quo with trails/viewpoints/biodiversity all set to current levels and price 0. An initial design will be piloted to get priors for a final Bayesian design.

I started with the following linear, not-very-informative Bayesian setup (which I evaluate as an rppanel model, but I left out the code):

Code: Select all

Design
;alts(_mnl) = alt1, alt2, sQuo
;rows = 30
;block = 3
;eff = 25000000000 * _mnl(mnl, d, mean) + _mnl(mnl, wtp(wtp1), mean)
;wtp(_mnl) = wtp1(b1, b2, b3/b4)
;bdraws = gauss(3)
;rdraws = gauss(3)
;rep = 100
;model(_mnl):
U(sQuo) = b0[(n, -0.1, 0.01)] +
          b1[(n, 0.001, 0.0001)] * trails_ref[0] + 
          b2[(n, 0.001, 0.0001)] * views_ref[0] +
          b3[(n, 0.001, 0.0001)] * biodiv_ref[0] +
          b4[(n, -0.00003, 0.000001)] * price_ref[0] /
U(alt1) = b1 * trails_piv[0, 1, 2] + ? 0 = current, 1 = improved, 2 = much improved
          b2 * views_piv[0, 1, 2] +
          b3 * biodiv_piv[0, 1, 2] +
          b4 * price_piv[25, 50, 75, 100] /
U(alt2) = b1 * trails_piv[0, 1, 2] +
          b2 * views_piv[0, 1, 2] +
          b3 * biodiv_piv[0, 1, 2] +
          b4 * price_piv[25, 50, 75, 100]
$


I think this code is alright but working with it did give me a few general questions:
Q1: Is there a way to decide whether the much longer search time when using alt1*, alt2* is worth it over simply using alt1, alt2?
Q2: Is model averaging with a massive weight to compensate for very high C-error and S-estimates (due to, I assume, having almost-0 priors) still useful?
Q3: Is gauss(3) a universally acceptable setting? If I try to set gauss(x,x,..) manually, I tend to get errors that I did not set the right number of draws regardless of how many/few values I have between the brackets.
Q4: How do I set Ngene to account for estimating b4 as a random parameter with negative lognormal distribution? The model will simply estimate b4 but this (untransformed) value can be high and so lead to price dominating the utility functions for the final design.

Now, trails/views/biodiv will be modelled as dummy variables and I found this post http://choice-metrics.com/forum/viewtopic.php?f=2&t=386 and that code seems to work:

Code: Select all

Design
;alts(_mnl_dummy) = alt1, alt2, sQuo
;rows = 30
;block = 3;

eff = _mnl_dummy(mnl, d) ? + 25000000000 * _mnl(mnl, d, mean) + _mnl(mnl, wtp(wtp1), mean)
? ;wtp(_mnl) = wtp1(b1, b2, b3/b4)
?;bdraws = mlhs(150)
?;rdraws = mlhs(150)
;alg = mfederov(candidates = 1000)

;require:
alt1.price > 0,
alt2.price > 0,
sQuo.trails = 0 and sQuo.views = 0 and sQuo.biodiv = 0 and sQuo.price = 0

;model(_mnl_dummy):
U(alt1) = b1.dummy[(n, 0.001, 0.0005) | (n, 0.002, 0.001)] * trails[1, 2, 0] +
          b2.dummy[(n, 0.001, 0.0005) | (n, 0.002, 0.001)] * views[1, 2, 0] +
          b3.dummy[(n, 0.001, 0.0005) | (n, 0.002, 0.001)] * biodiv[1, 2, 0] +
          b4[(n, -0.00003, 0.000001)] * price[0, 25, 50, 75, 100] /
U(alt2) = b1 * trails +
          b2 * views +
          b3 * biodiv +
          b4 * price /
U(sQuo) = b0[(n, -0.1, 0.01)] +
          b1 * trails +
          b2 * views +
          b3 * biodiv +
          b4 * price
$


Q5: If I uncomment (and experiment with) any of the currently commented settings, Ngene immediately stops the search with the "Something went unexpectedly wrong." message. Are the default settings universally sufficient?
Q6: Can I leave out the reference level for linear and continuous variables in the non-status-quo alternatives? (I understand all alternatives must include all levels of dummy variables.)
Q7: To develop a design with price as a continuous variable, do I simply remove the price attribute, let this code run, and then follow the procedure in the manual? It seems to me that is not worth it in this case, as I think the qualitative descriptions of the improvements do not lend themselves to extrapolation and forecasting.

Many thanks in advance for your time and insight,
met vriendelijke groet,
Florian

[Michiel Bliemer]

Hi Florian,

There are quite a few issues, my recommended syntax is below.

Code: Select all

Design
;alts = alt1*, alt2*, sQuo*
;rows = 30
;block = 3;
eff = (mnl, d)
;alg = mfederov(candidates = 1000)
;require:
sQuo.trails = 0,
sQuo.views = 0,
sQuo.biodiv = 0

;model:
U(alt1) = b1.dummy[0.001 | 0.002] * trails[1, 2, 0]
        + b2.dummy[0.001 | 0.002] * views[1, 2, 0]
        + b3.dummy[0.001 | 0.002] * biodiv[1, 2, 0]
        + b4vvvvvvvvvvv[-0.00003] * price[25, 50, 75, 100](6-10,6-10,6-10,6-10)
        /
U(alt2) = b1 * trails
        + b2 * views
        + b3 * biodiv
        + b4 * price
        /
U(sQuo) = b0[0]
        + b1 * trails
        + b2 * views
        + b3 * biodiv
        + b4 * price_sq[0]
$

To answer your questions:

Q1: If you have unlabelled alternatives you should generally always use alt1* and alt2* to avoid dominant alternatives and repeated choice tasks as it tells Ngene that the alternatives are unlabelled.
Q2: When you use near-zero priors, you should ignore S-estimates and C-error is not properly defined so you cannot use. S-error and C-error should only be used with informative priors, preferably parameter estimates from a pilot study.
Q3: gauss(3) is fine if your distribution is sufficiently narrow. In your case, your Bayesian priors were uninformative with extremely small standard deviations, so they are essentially fixed. I have removed all Bayesian priors since are not useful as currently specified. Once you have obtained parameter estimates from a pilot study you can use them as informative Bayesian priors where the standard error identifies the unreliability of the prior and hence can be used as standard deviation in the normally distributed prior.
Q4: Only normal and uniform distributions are currently recognised in Ngene. Currently your priors are noninformative, set to near-zero (only indicating the sign), you should either set all priors as noninformative or informative, but not a mix. With the near-zero fixed prior price cannot become dominant. If you do want situations with dominant price, you can use a uniform distribution, (u,-0.05,0) or something.
Q5: This likely has to do with the fact that C-error is not defined for priors at or near zero, you are dividing by near-zero.
Q6: You can leave out all attributes that result in zero utility for the status quo level. In this case, probably all of them since the dummy variables are set to their base levels. I left them in the model as it looks nicer, but the outcome should be the same.
Q7: I specified price as continuous, and added attribute level constraints 6-10 to ensure some degree of attribute level balance. You should use a continuous price if you are interested in WTP since making price dummy coded will make WTP computations more complex and difficult to interpret.

Michiel

[Michiel Bliemer]

Hi Florian,

There are quite a few issues, my recommended syntax is below.

Code: Select all

Design
;alts = alt1*, alt2*, sQuo*
;rows = 30
;block = 3
;eff = (mnl, d)
;alg = mfederov(candidates = 1000)
;require:
sQuo.trails = 0,
sQuo.views = 0,
sQuo.biodiv = 0

;model:
U(alt1) = b1.dummy[0.001 | 0.002] * trails[1, 2, 0]
        + b2.dummy[0.001 | 0.002] * views[1, 2, 0]
        + b3.dummy[0.001 | 0.002] * biodiv[1, 2, 0]
        + b4[-0.00003] * price[25, 50, 75, 100](6-10,6-10,6-10,6-10)
        /
U(alt2) = b1 * trails
        + b2 * views
        + b3 * biodiv
        + b4 * price
        /
U(sQuo) = b0[0]
        + b1 * trails
        + b2 * views
        + b3 * biodiv
        + b4 * price_sq[0]
$

To answer your questions:

Q1: If you have unlabelled alternatives you should generally always use alt1* and alt2* to avoid dominant alternatives and repeated choice tasks as it tells Ngene that the alternatives are unlabelled.
Q2: When you use near-zero priors, you should ignore S-estimates and C-error is not properly defined so you cannot use. S-error and C-error should only be used with informative priors, preferably parameter estimates from a pilot study.
Q3: gauss(3) is fine if your distribution is sufficiently narrow. In your case, your Bayesian priors were uninformative with extremely small standard deviations, so they are essentially fixed. I have removed all Bayesian priors since are not useful as currently specified. Once you have obtained parameter estimates from a pilot study you can use them as informative Bayesian priors where the standard error identifies the unreliability of the prior and hence can be used as standard deviation in the normally distributed prior.
Q4: Only normal and uniform distributions are currently recognised in Ngene. Currently your priors are noninformative, set to near-zero (only indicating the sign), you should either set all priors as noninformative or informative, but not a mix. With the near-zero fixed prior price cannot become dominant. If you do want situations with dominant price, you can use a uniform distribution, (u,-0.05,0) or something.
Q5: This likely has to do with the fact that C-error is not defined for priors at or near zero, you are dividing by near-zero.
Q6: You can leave out all attributes that result in zero utility for the status quo level. In this case, probably all of them since the dummy variables are set to their base levels. I left them in the model as it looks nicer, but the outcome should be the same.
Q7: I specified price as continuous, and added attribute level constraints 6-10 to ensure some degree of attribute level balance. You should use a continuous price if you are interested in WTP since making price dummy coded will make WTP computations more complex and difficult to interpret.

Michiel

[feppink]

Thank you, Michiel, for cleaning up my code. Your responses to my questions are quite clear but I do have two follow-ups, if you do not mind.

Q6: You can leave out all attributes that result in zero utility for the status quo level. In this case, probably all of them since the dummy variables are set to their base levels. I left them in the model as it looks nicer, but the outcome should be the same.

From this thread http://choice-metrics.com/forum/viewtopic.php?f=4&t=814, I got the impression that status quo levels must occur in other alternatives too. You are saying this only applies if the status-quo level is non-zero, regardless of type (dummy or continuous)?

Q7: I specified price as continuous, and added attribute level constraints 6-10 to ensure some degree of attribute level balance. You should use a continuous price if you are interested in WTP since making price dummy coded will make WTP computations more complex and difficult to interpret

I think the title of chapter 9 of the manual put me on the wrong foot in asking my question. Is the procedure in chapter 9.2 necessary when including continuous variables or rather an option that one could implement to see if meaningful efficiency improvements can be gained?

Thank you,
Florian

[Michiel Bliemer]

To answer your first question, yes they generally need to be there, but if they happen to be zero (as in your case), you can leave them out as adding zero to utility does not change. But in order not to confuse people, the general rule is that you include them in your utility function, which I have done in the rewritten syntax.

I now better understand your other question. You are referring to continous LEVELS, not continuous ATTRIBUTES. That chapter in the Ngene manual specifically talks about continuous levels, i.e. any value for price between 2 and 10, instead of discrete levels 2,4,6,8,10. The latter is most common, continuous levels is not common but possible. I would recommend using fixed levels, so you can ignore that specific chapter in the Ngene manual.

Michiel