Pivoted design without status quo alternative

[Arnaud Blaser]

Dear Ngene forum users,

I am new to both DCE design and Ngene, hence I’ll do my best to avoid silly questions (I DID read the manual, forum, literature and took a course before writting on this forum.)

I am basically trying to design an experiment with three unlabelled alternatives with 4 attributes having all 4 levels (number of levels is open to debate, but 4 seems to work fine). Because of time constraint we want to show only 4 choices to each respondents, hence I went for a 12 rows design divided in 3 blocks.

Design
;alts = alt1*, alt2*, alt3*
;rows = 12
;block = 3
;alg=mfederov
;reject: alt1.eff=alt2.eff or alt1.eff=alt3.eff or alt2.eff=alt3.eff
;eff=(mnl,d)
;model:
U(alt1) = btech.dummy[0|0|0] * tech[0,1,2,3] + beff[0.01] * eff[200,600,1000,1400] + binv[-0.001] * inv[15000,25000,35000,45000] + bsub[0.001] * sub[3000,6000,9000,12000] /
U(alt2) = btech * tech + beff * eff + binv * inv + bsub * sub /
U(alt3) = btech * tech + beff * eff + binv * inv + bsub * sub $

My main question is with respect to the “eff” attribute. Basically this attribute, standing for “efficiency”, is going to be a pivoted attribute showing respondents how much money per year they are saving with this option (this is a home retrofit DCE). For different reasons (hypothetical mandatory retrofit context) we do not want to include a status quo option. What respondents are going to see is how much money they are going to save, but in the background this amount will be computed as a % reduction of their current heating costs (10%, 30%, 50%, 70%). We want to stick with a single design, hence what I did is I assumed the average yearly heating costs (based on literature) of 2000 and derived levels (savings) of 200, 600, 1000, 1400 to create the design. During the experiment however these assumed numbers will be tailored (pivoted) based on respondents’ specific current heating costs. Is this the correct way to do it or am I missing an important Ngene feature here?

I have another question regarding the priors; the only things we can reasonably assume are; A) a positive utility from “eff” and “sub” (yearly savings and government subsidy) and B) a negative utility from “inv” (investment costs). We can also assume a higher impact on utility from 1 USD in yearly savings than 1 USD in subsidy or investment (the reason being that 1 USD in yearly savings is going to occur each year, while the other are a one-time cost or gain). For these reasons I specified somewhat arbitrary fixed priors of 0.01, -0.001 and 0.001 (we have no assumptions regarding the dummy coded attribute). Is this a reasonable way to do it or is Ngene going to create a more flexible design with random (Bayesian) priors? Based on these priors the respective probabilities look reasonably balanced (see below).

alt1 alt2 alt3
0.244728 0.665241 0.090031
0.211942 0.576117 0.211942
0.211942 0.211942 0.576117
0.422319 0.155362 0.422319
0.576117 0.211942 0.211942
0.576117 0.211942 0.211942
0.259496 0.035119 0.705385
0.843795 0.04201 0.114195
0.211942 0.211942 0.576117
0.843795 0.114195 0.04201
0.211942 0.576117 0.211942
0.422319 0.155362 0.422319

Finally note that we are lucky enough to be collecting data within a big project, with an estimated final sample size (fully completed questionnaire) of 3,500. I suppose this large sample makes it much easier to find a "good enough" design.

Many thanks in advance for your help
Arnaud

[Michiel Bliemer]

Hi,

I would suggest that you read Section 8.3 in the manual about pivot designs. You can provide a reference level and Ngene will find you the percentages.

Starting with almost zero priors is fine for a pilot study, but you will loose a lot of efficiency in a large data collection. Can I suggest that you use the design that you created using zero priors for a pilot study in which you ask 10 colleagues to answer all 12 questions, and estimate the coefficients based on this data? This should give you normally distributed Bayesian priors (i.e. the beta's that come out of estimation, and the standard errors that you can use as standard deviations in a normally distributed prior). Then use these Bayesian priors to find an efficient design that you use for your large survey.

[Arnaud Blaser]

Many thanks for your suggestion about the priors. I think we will make use of the planned survey pre-test to estimate them more precisely.

About the pivot; I did read section 8.3 but I am still unsure of how well it applies to our situation. This thing is we do not want to show respondents a reference of status quo alternative because we want to force them to choose one of the options (this does make sense in our hypothetical setting). The only think we want is to show them a tailored "savings" attributes based on their current heating costs and all exemles I can find in the manuals imply specifing/showing such a reference alternative. Do you suggest it is safe to use Ngene to generate a design with a reference alternative and then simply delete it or not allowing respondents to select it?

[Michiel Bliemer]

Apologies, I misunderstood. You are right that at this moment it is not possible to use pivot designs without a reference alternative. It is a good idea to be able to separately define reference levels of attributes without having a reference alternative. I will add it to our list of 'most wanted features'

I would NOT suggest generating a pivot design and then simply delete the reference alternative, as an efficient design is generated under the assumption of the reference alternative. In case you want to generate a single design, I think that your proposal of using absolute attribute levels for an average respondent will work best, and then convert the levels manually to percentages. In case you want to generate multiple designs for different reference levels, you can generate them separately, or use a model averaging approach (Section 7.4 of the manual).

[Arnaud Blaser]

Excellent. Many thanks for your feedback on this particular case of pivot.