How do design levels impact estimates of marginal trade offs

[sj_tan_]

Hi all,

I am working with two sets of data, both samples of the same population. For each sample, I conducted a choice experiment with a cost attribute roughly centered around $5. In one (sample 1), the spread of the attribute was slightly lower, and the design was such that people made comparisons between $3 and $4, $1 and $2, but never the lowest and highest values. In the other (sample 2), the spread of the attribute was slightly larger, and the design was such that people made comparisons between the lowest and highest values frequently, e.g. $1 and $10.

Marginal WTP for the two samples is much larger in the sample where respondents were asked to make comparisons between very low and very high values.

Is there a mathematical explanation for this? I'm assuming what is going on is that people in sample 2 picked options with high costs and a higher rate than we would have extrapolated using the results from sample 1. Both designs were d-efficient designs created in NGene for MNL models.

Is there a term or explanation for this in the experimental design literature -- framing effect, attribute non-attendance, something else?

Thanks so much!

[Michiel Bliemer]

If the range for cost is narrow, then people may actually not be trading off between cost and other attributes. In transport, researchers that are interested in trade-offs between travel time and travel cost often look at so-called boundary values, which reflect whether respondents are actually trading off on time and cost, or whether they always choose the cheapest or fastest route. Those researchers argue that you need a sufficient range in order to get all respondents making trade offs. If the range in cost is small and many respondents are not trading off (i.e., they simply select the alternative where other attributes are more favourable, e.g. lower travel time), then this will affect willingness-to-pay measures.

Unrelated to boundary values, you may want to read Hensher (2004).

Hensher, D.A. (2004) Identifying the influence of stated choice design dimensionality on willingness to pay for travel time savings. Journal of Transport Economics and Policy, Vol. 38, No. 3, pp. 425-446.

Michiel

[sj_tan_]

Thanks so much! That is really helpful. One thing I didn't clarify in my earlier post is that although the WTP in the sample with more comparisons between close $ values is much lower than in the other sample, they are still statistically and economically significant. That's why my first thought was that there could be something along the lines of an anchoring or framing effect in the other sample, which desensitized respondents to seeing larger values. I will look into boundary values as you suggested.

[Michiel Bliemer]

If half of the population does not make trade-offs and the other half of the population does (because everyone has different boundary values / WTP values), then parameters may still be statistically significant even though they are biased.

[sj_tan_]

That makes sense, thank you for your help