Bayesian design - advice on standard errors (and more!)

[P_King]

Dear Michiel,

I hope you are well.

My project is assessing public preferences for the design of nature-based flood risk management schemes. Attributes and levels below:
1. Tree type: no trees (0); mostly conifer trees (1), mostly deciduous trees (2), mixed trees (3)
2. Water storage type: no water storage (0); mostly pond storage (1); mostly floodplain storage (2); mixed storage
3. Size of scheme: 8km2 (1); 16km2 (2); 32 km2 (3)
4. Cost: £25 (1); £75 (2); £125 (3); £175 (4)

Pilot syntax:

Code: Select all

Design
;alts = alt1, alt2, sq
;rows = 12
;block = 2
;eff = (mnl,d)
;model:
U(alt1) = b1 * Tree[1,2,3,4]
        + b2 * Water[1,2,3,4]
        + b3 * Land[1,2,3]
        + b4[-1] * Cost[1,2,3,4]
        /
U(alt2) = b1*Tree + b2*Water +b3*Land + b4*Cost

$

This pilot revealed statistcally significant results for all attributes/levels except 'Size of scheme'. Here is the new syntax with priors addedd (and dummy coding)

Code: Select all

Design
;alts = alt1, alt2, sq
;rows = 12
;block = 2
;eff = (mnl,d)
;model:
U(alt1) = b1.dummy[1.15|2.64|1.12] * tree[1,2,3,0]
        + b2.dummy[0.68|0.93|0.71] * water[1,2,3,0]
        + b3[0.01] * size[1,2,3]
        + b4[-0.01] * cost[1,2,3,4]
        /
U(alt2) = b1*tree + b2*water +b3*effect + b4*cost

Based off feedback/focus groups conducted post-pilot, I have now adjusted the 'size' attribute. Due to this, I have been advised to use a Bayesian design to account for the uncertainty of changing this attribute. My draft syntax for the bayesian design is below (using '2' for standard error on all attributes). I have also included a parameter prior for my new attribute 'effect' (1.0) - an assumed prior based off conversations with participants and background reading.

Code: Select all

Design
;alts = alt1, alt2, sq
;rows = 12
;block = 2
;eff = (mnl,d)
;model:
U(alt1) = b1.dummy[(n,1.15,2)|(n,2.64,2)|(n,1.12,2)] * tree[1,2,3,0]
        + b2.dummy[(n,0.68,2)|(n,0.93,2)|(n,0.71,2)] * water[1,2,3,0]
        + b3[(n,1.0,2)] * effect[1,2,3]
        + b4[(n,-0.01,2)] * cost[1,2,3,4]
        /
U(alt2) = b1*tree + b2*water +b3*effect + b4*cost

Following from this I have questions to gain clarification on my work so far.

Questions:
1. Do I base the standard errors in the Bayesian design off the standard errors from the pilot? Do I apply standard errors to all attributes?
2. Is it possible to ensure two levels (0 = no trees, 0 = no water storage) do not appear together in the scenarios?
3. I have tried to format the scenarios/levels so I can check the designs more easily rather than change each level every time I generate a new design. I have seen your syntax to do this, but for some reason I can't get it to work on mine. This is what I put:

Code: Select all

Design
;alts = alt1, alt2, sq
;rows = 12
;block = 2
;eff = (mnl,d)
;model:
U(alt1) = b1.dummy[(n,1.15,2)|(n,2.64,2)|(n,1.12,2)] * tree[1,2,3,0] ? 0 = none (base), 1 = Conifer, 2 = Deciduous, 3 = Mix
        + b2.dummy[(n,0.68,2)|(n,0.93,2)|(n,0.71,2)] * water[1,2,3,0] ? 0 = none (base), 1 = Ponds, 2 = Floodplain, 3 = Mix
        + b3[(n,1,2)] * effect[2,6,8] ? Peak flood reduction (%)
        + b4[(n,-0.01,2)] * cost[25,75,125,175] ? annual council tax (£)
        /
U(alt2) = b1*tree + b2*water +b3*effect + b4*cost

$

Many thanks for your time. I really appreciate your guidance.

Best wishes,

Phoebe

[Michiel Bliemer]

First, a few important comments.

* It would be have been better if you also used dummy coding when generating the design for the pilot study (and set all priors to zero).
* For numerical attributes you need to use the actual attribute levels, so instead of effect[1,2,3] and cost[1,2,3,4] you should use effect[2,6,8] and cost[25,75,125,175] as you do in your final script. Of course you cannot use the priors that you used to estimate parameters with levels 1,2,3. you will need to re-estimate the mode using attribute levels 2,6,8 and 25,75,125,175. But perhaps you have done that and your other script are wrong.

Answers to your questions:
1. You should NOT set Bayesian priors manually, so using 2 for all is very problematic, especially (n,-0.01,2) will generate a very INefficient design. For Bayesian priors (n,mu,stdev), you set mu to the parameter estimate and you set stdev to the standard error for that parameter estimate.
2. You can use the ;cond property, see page 156 of the Ngene v1.3 manual.
3. In your script you merely indicated the formatted levels in a comment. You need to use the scenario formatting editor, see Chapter 10 of the Ngene v1.3 manual.

Michiel

[P_King]

Dear Michiel,

Many thanks for your response. That was very helpful.

I have now adjusted my syntax based off your recommendations (eg., inserting my standard errors from my pilot model output, as well as inserting a contraint to avoid 'no trees' and 'no water storage'). I have left the 'effect' attribute with a wider standard deviation as i do not have a parameter prior from my pilot for this attribute. Therefore, I assume there will be more uncertainty with this one. Would this be the right approach? Many thanks

Phoebe

Code: Select all

Design
;alts = alt1*, alt2*, sq
;rows = 12
;block = 2
;eff = (mnl,d,mean)
;cond:
if(alt1.tree=0, alt1.water=[1,2,3]),
if(alt2.tree=0, alt2.water=[1,2,3])
;model:
U(alt1) = b1.dummy[(n,1.15,0.48)|(n,2.64,0.47)|(n,1.12,0.34)] * tree[1,2,3,0]
        + b2.dummy[(n,0.68,0.38)|(n,0.93,0.34)|(n,0.71,0.73)] * water[1,2,3,0]
        + b3[(n,1,2)] * effect[2,4,8]
        + b4[(n,-0.01,0.01)] * cost[25,75,125,175]
        /
U(alt2) = b1.dummy*tree + b2.dummy*water + b3*effect + b4*cost
$

[Michiel Bliemer]

Some of your priors are still problematic and result in a very inefficient design:

b3[(n,1,2)] * effect[2,4,8]

Please use an appropriate prior from a pilot study.

Michiel