choice-metrics.com

Hi Ngene users,

I have an empirical question about specifying the levels of the status quo in a health economics application. We are designing an experiment with two unlabelled options compared to a status quo (type) alternative (called 'elsewhere'). I am a bit unclear about both the design and modelling implications of defining levels for the SQ option. In this case, the levels of some SQ attributes overlap with levels that appear in the generic alternatives A & B, other levels are unique. The simplest option would be not to specify levels for the SQ and use a constant when modelling, using a design such as:

Design
;alts = A*, B*, elsewhere
;rows = 36
;block = 6
;orth = sim
;foldover
;model:
U(A) = b1.effects[0]*invite[0,1] + b2.effects[0]*number[0,1] + b3.effects[0]*family[0,1]
+ b4.effects[0|0]*involve[0,1,2] + b5.effects[0]*content[0,1] + 67.effects[0|0]*confid[0,1,2] /
U(B) = b1*invite + b2*number + b3*family + b4*involve + b5*content + b6*confid $

[We use a foldover design here to give us some flexibility when specifying interactions later on].

It has been suggested that we don't specify the attributes and levels of the SQ in the survey, although the description of the SQ is such that it implies some of the levels of some attributes. How these will be interpreted is as yet unknown (we haven't piloted this yet).

Another option is to specify the attributes and levels and use constraints to set a constant (non-zero) utility for the SQ. If I only include levels which overlap with the generic alternatives, such a design might be:

Design
;alts = A*, B*, elsewhere
;rows = 36
;block = 6
;eff = (mnl,d)
;alg = mfederov(candidates=100)
;require:
elsewhere.invite=1,
elsewhere.family=1,
elsewhere.confid=2
;model:
U(A) = b1.effects[0]*invite[0,1] + b2.effects[0]*number[0,1] + b3.effects[0]*family[0,1]
+ b4.effects[0|0]*involve[0,1,2] + b5.effects[0]*content[0,1] + b6.effects[0|0]*confid[0,1,2] /
U(B) = b1*invite + b2*number + b3*family + b4*involve + b5*content + b6*confid /
U(elsewhere) = b0[0] + b1*invite + b3*family + b6*confid $

[We are using zero priors at this stage as we have no clear indication/hunch of the direction of most of the non-linear attributes (no cost attribute included)].

If I include additional levels in the SQ for attribute levels such as 'this is unknown' (which in this case may be reasonably expected to have a utility value other than zero), the design is bigger and might be:

Design
;alts = A*, B*, elsewhere
;rows = 36
;block = 6
;eff = (mnl,d)
;alg = mfederov(candidates=100)
;require:
elsewhere.invite=1,
elsewhere.family=1,
elsewhere.confid=2,
elsewhere.number=2,
elsewhere.invole=4,
elsewhere.content=2
;reject:
A.number=2,
B.number=2,
A.involve=4,
B.involve=4,
A.content=2,
B.content=2
;model:
U(A) = b1.effects[0]*invite[0,1] + b2.effects[0|0]*number[0,1,2] + b3.effects[0]*family[0,1]
+ b4.effects[0|0|0]*involve[0,1,2,3] + b5.effects[0|0]*content[0,1,2] + b6.effects[0|0]*confid[0,1,2] /
U(B) = b1*invite + b2*number + b3*family + b4*involve + b5*content + b6*confid /
U(elsewhere) = b0[0] + b1*invite + b2*number + b3*family + b4*involve + b5*content + b6*confid $

Unsurprisingly, the increasing complexity of constraints means I can't get designs to run for the second two options presented here.

My question then is where to from here? My instinct is that explicitly specifying the levels of SQ will have behavioural consequences, possibly influencing choice probabilities, and the small amount of literature I could find suggests that this is probably the case. I am not sure of the implications of leaving these levels out of the design, but specifying them in the survey (and subsequently, how to code these attributes for analysis), but I am sure that it will be difficult to find a design if I do specify them in the design.

If this was the only study I was working on, I might think about re-thinking our attributes and levels. However, I have another three designs in progress and nearly all of them suffer from a similar problem. I am sure people in other disciplines, especially environmental economics, are also dealing with this. Are there many people testing these issues about how best to specify the SQ /opt out empirically and if so, what has been your experience to date?

If the moderators have suggestions for the designs presented that would also be most helpful - I suspect adding priors, even from a handful of people, might be a good plcae to start. However, the biggest problem will still be the number of constraints I expect. Is this correct?

Best reagrds,

Harald.

You can get rid of a couple of effects coded variables, as with 2 levels they are the same as using linear coded levels. This allows you to directly set the attribute levels for these attributes in the status quo. Further, I think you can put all constraints into a single require statement, such that it would simplify to:

Code: Select all

Design
;alts = A*, B*, elsewhere
;rows = 36
;block = 6
;eff = (mnl,d)
;alg = mfederov(candidates=100)
;require:
elsewhere.confid=2,
elsewhere.number=2,
elsewhere.involve=3,
elsewhere.content=2,
A.number<2,
B.number<2,
A.involve<3,
B.involve<3,
A.content<2,
B.content<2
;model:
U(A) = b1[0]*invite[0,1] + b2.effects[0|0]*number[0,1,2] + b3s[0]*family[0,1]
+ b4.effects[0|0|0]*involve[0,1,2,3] + b5.effects[0|0]*content[0,1,2] + b6.effects[0|0]*confid[0,1,2] /
U(B) = b1*invite + b2*number + b3*family + b4*involve + b5*content + b6*confid /
U(elsewhere) = b0[0] + b1*invitesq[1] + b2*number + b3*familysq[1] + b4*involve + b5*content + b6*confid $


Please note that there are several minor mistakes in your syntax. You have spelled 'involve' as 'invole' and you refer to value 4 while it only has levels up to 3. Please carefully check the syntax, and see if this helps.

It seems that your model specification is problematic. The following syntax will run:

Design
;alts = A*, B*, elsewhere
;rows = 36
;block = 6
;eff = (mnl,d)
;alg = mfederov(candidates=100)
;require:
elsewhere.confid=2,
elsewhere.number=2,
elsewhere.involve=3,
elsewhere.content=2
;model:
U(A) = b1[0]*invite[0,1] + b2.effects[0|0]*number[0,1,2] + b3[0]*family[0,1]
+ b4.effects[0|0|0]*involve[0,1,2,3] + b5.effects[0|0]*content[0,1,2] + b6.effects[0|0]*confid[0,1,2] /
U(B) = b1*invite + b2*number + b3*family + b4*involve + b5*content + b6*confid /
U(elsewhere) = b0[0] + b1*invitesq[1] + b2*number + b3*familysq[1] + b4*involve + b5*content + b6*confid $

However, adding A.number<2 or any other constraint that limits alternatives A and B to attain the reference levels will yield a model that is not estimable. I will ask my colleagues John and Andrew to confirm, but I believe that you must have the reference level appear in the A and B attributes, it cannot just appear in the status quo alternative. This seems to me the underlying problem. If you are not using dummy or effects coding, then clearly there is no problem as with linear coding there is no reference level.

Hi

The problem I believe is the exact same problem as that posted by by laspen » Mon May 19, 2014 9:38 pm. I refer you to my reply to that post.

John

Hi John and Michiel,

thanks so much for your feedback, really helpful.

Just to clarify (I read Cooper et al 2012 but couldn't find a definitive answer), for a non-linear attribute where the level of the SQ only appears in the SQ and remains fixed (and so you apply a hybrid coding scheme during model estimation as in the paper), do you leave this attribute out of the SQ utility function when specifying the design in Ngene?

Thanks again