choice-metrics.com

Hi Experts:
I have some questions about pivot designs, and hope you can help me. I have five quantitaive attributes and one price attribute. Taking advantage of online survey, I am thinking of conducting a separate design for each respondent. But before having the reference for each respondent, how should I make design? I was suggested to use '0' as reference or 'sample average' as reference (code shown below), and import the design into Qualtrics, and let it generate the actual reference for each respondent when respondents answer the survey. I am not quite sure whether it is the right way, any suggestions? Thank you!

If the above does not work, I am thinking about the segment design. There two groups, one employed (n=50) and one retired (n=100). My other question is: if I use heterogenous pivot design with sample average as reference in each segment, may I analysis the two groups together? As I have sample size issue, I hope to analyse the employed and the retired together. Thank you!

Connie

Code: Select all

Design
;alts=alt1,alt2,alt3
;rows=12
;eff=(mnl,d)
;alg=mfederov
;model:
U(alt1)=b1*A.ref[0]+
        b2*B.ref[0]+
        b3*C.ref[0]+
        b4*D.ref[0]+
        b5*E.ref[0]+
        b6*Price.ref[15]
/
U(alt2)=b1*A.piv[-5,0,5]+
        b2*B.piv[-5,0,5]+
        b3*C.piv[-2,0,2]+
        b4*D.piv[-5,0,5]+
        b5*E.piv[-1,0,1]+
        b6*Price.piv[-10%,0%,10%]
/
 
U(alt3)=b1*A.piv[-5,0,5]+
        b2*B.piv[-5,0,5]+
        b3*C.piv[-2,0,2]+
        b4*D.piv[-5,0,5]+
        b5*E.piv[-1,0,1]+
        b6*Price.piv[-10%,0%,10%]
 
$
 

1. I suggest reading Rose et al. (2008). What we found is that optimising the design for "average" attribute levels across the population may work quite well. In that case, you use actual levels to optimise your design and you convert them into pivots (absolute or relative) for the survey (in which you also use rounding-off techniques). As far as I am aware, this is not possible or difficult to do in Qualtrics. Another possibility is to create a library of designs. In this case, you create several designs, one for each class of respondents. You implement these all as separate versions of the choice experiment and once you know the reference level of the respondent you dynamically point the respondent to the appropriate choice tasks in Qualtrics.

Rose, J.M., M.C.J. Bliemer, D.A. Hensher, and A. Collins (2008) Designing Efficient Stated Choice Experiments in the Presence of Reference Alternatives. Transportation Research Part B, Vol. 42, pp. 395-406.

2. Yes you can analyse both groups at the same time. If your alternatives are unlabelled, then you introduce a dummy variable with 0 = retired, 1 = employed, and you create interactions between this dummy variable and the other attributes. If your alternatives are labelled, then you can still do this, but in addition you can also add the dummy variable as a main effect into the utility function.

Michiel

Thank you, Professor Michiel, very helpful.

Dear Michiel,

let me take advantage of this post to ask you a related question.

We are designing a choice experiment to test the preference for meat deriving from an old breed of pig.
We want to use a reference alternative with fixed charateristics (regular fresh ham from modern pig breed).
The old breed has many characteristics that are very different from the modern breed, so we decided to use a labelled design. We plan to ask the respondents about the type of fresh ham they usually buy, so the reference alternative can have different levels for each respondent.

I understand that I should use a pivot design in this case, but my problem is that the reference alternative have very different levels from the other alternatives, so I wonder whether this is still the right solution.

For instance, for price, the reference level would be about 50 HRK/kg, while the old breed alternatives should vary between 90 and 180 HRK/kg. For the rearing system modern pigs are raised conventionally, while the old breed's farming styles (we have selected two options) cannot be used for the modern pigs.

I have tried to write the following code

Code: Select all

? Pivot design
Design   
;alts = ref, BSP1, BSP2   
;rows = 12   
;eff = (mnl,d)
;model:   
U(ref)  = b0       
        + b1[-.01]            *PRICEref[35]
/
U(BSP1) = b1                  *PRICE[70, 115, 160]
        + b2.effects[1.5|2.0] *GEO[1,2,0]
        + b3.effects[0.5|1.0] *COL[1,2,0]
        + b4.effects[0.5|0.75]*REAR[1,2,0]
/   
U(BSP2) = b1                  *PRICE                   
        + b2.effects          *GEO       
        + b3.effects          *COL     
        + b4.effects          *REAR     
$



It works, but I have the feeling that something is wrong.

I assume that the different characteristics in terms of GEO, COL, and REAR should be captured in b0, but I am not sure.

What do you think?

Maurizio

Hi Maurizio,

You mention that that the reference alternative will be different for different people, but you only specify a constant and a price attribute. If the refernce alternative can be different types of ham, should you not include an attribute (or multiple attributes) that describes the type of ham?

It is perfectly fine to use different price levels across alternatives. And yes, b0 would capture the combined utility of GEO, COL and REAR for the reference alternative. If the levels of the reference alternative can also appear in the other two alternatives, then you need to specify the utility functions differently, see below.

Code: Select all

? Pivot design
Design   
;alts = ref, BSP1*, BSP2*   ? checking for dominance and repetitions, assuming BSP1 and BSP2 are generic alternatives   
;rows = 24
;block = 2   
;eff = (mnl,d)
;alg = mfederov   ? the modified Federov algorithm is needed for including require constraints that fixes the levels for the attributes in the reference alternative
;require:
ref.GEO = 0, ref.COL = 0, ref.REAR = 0
;model:   
U(ref)  = b0       
        + b1[-0.01]            * PRICEref[35]
        + b2.effects[1.5|2.0]  * GEO[1,2,0]
        + b3.effects[0.5|1.0]  * COL[1,2,0]
        + b4.effects[0.5|0.75] * REAR[1,2,0]
        /
U(BSP1) = b1 * PRICE[70,115,160](6-10,6-10,6-10)  ? the modified Federov algorithm cannot satisfy attribute level balance, therefore I added constraints on the minimum and maximum number of times each attribute level appears within the design; note that with 24 rows, (8,8,8) would mean perfect attribute level balance
        + b2 * GEO
        + b3 * COL
        + b4 * REAR
        /   
U(BSP2) = b1 * PRICE                   
        + b2 * GEO       
        + b3 * COL     
        + b4 * REAR     
$

While 12 rows in the design is sufficient for estimating the model, I would recommend increasing the number of rows to 24 and blocking it in two. Your MNL model has 8 parameters, but if you later on estimate a mixed logit or latent class model model, you end up estimating 16+ parameters, which requires more variation in your data.

Michiel

Hello Michiel,

thanks a lot, very clear and helpful.

My idea is to use a reference alternative that varies according to the respondent's replies, without any pre-defined value. I am not sure whether, in this case, I should put pre-defined levels in the design. I assume that in your code the 0 level is "undefined", is it right?

Another issue, in the REAR attribute I have the level 0 that is fixed and defined for ref, but it can be only 1 or 2 for BSP.

I tried to introduce this aspect in the code, but it does not work properly:

Code: Select all

? Pivot design 1
Design   
;alts = ref, BSP1*, BSP2*   ? checking for dominance and repetitions, BSP1 and BSP2 are generic alternatives
;rows = 24
;block = 2   
;eff = (mnl,d)
;alg = mfederov   ? the modified Federov algorithm is needed for including require constraints that fixes the levels for the attributes in the reference alternative
;require:
ref.GEO = 0, ref.COL = 0, ref.REAR = 0
;reject:
BSP1.GEO = 0, BSP2.GEO = 0, BSP1.COL = 0, BSP2.COL = 0, BSP1.REAR = 0, BSP2.REAR = 0
;model:   
U(ref)  = b0[-2.0]       
        + b1[-0.01]            * PRICEref[35]
        + b2.effects[1.5|2.0]  * GEO[1,2,0]
        + b3.effects[0.5|1.0]  * COL[1,2,0]
        + b4.effects[0.5|0.75] * REAR[1,2,0]
        /
U(BSP1) = b1 * PRICE[70,120,170](6-10,6-10,6-10)  ? the modified Federov algorithm cannot satisfy attribute level balance, therefore I added constraints on the minimum and maximum number of times each attribute level appears within the design; note that with 24 rows, (8,8,8) would mean perfect attribute level balance
        + b2 * GEO
        + b3 * COL
        + b4 * REAR
        /   
U(BSP2) = b1 * PRICE                   
        + b2 * GEO       
        + b3 * COL     
        + b4 * REAR     
$



This code gives the error

Error: There were problems generating a fractional factorial of choice tasks. For the modified federov algorithm, increasing the number of candidates might assist.

Should I constrain the options in this design using the ;reject property, or maybe it is better to use a fully labelled experiment and have alt = ref, BSPout, BSPsem (that is, pork from conventional, outdoor and semi-outdoor rearing)?

I tried this code using ood:

Code: Select all

? Optimal orthogonal in the difference choice (labelled) design
Design   
;alts = BSPout, BSPsem, ref
;rows = 12   
;orth = ood
;model:   
U(BSPout) = b1[0]             *PRICE[70, 120, 170]
        + b2.effects[0]       *GEO[1,2]
        + b3.effects[0]       *COL[1,2]
        + b4.effects[0]       *REAR[1,2]
/   
U(BSPsem) = b1                *PRICE               
        + b2.effects          *GEO     
        + b3.effects          *COL   
        + b4.effects          *REAR     
/
U(ref) = b0[0]
$



I also wrote the code for the efficient design.

Code: Select all

? Efficient (labelled) design
Design   
;alts = BSPout*, BSPsem*, ref
;rows = 12   
;eff = (mnl,d)
;alg = mfederov
;model:   
U(BSPout) = b1[0]             *PRICE[70, 120, 170]
        + b2.effects[0]       *GEO[1,2]
        + b3.effects[0]       *COL[1,2]
        + b4.effects[0]       *REAR[1,2]
/   
U(BSPsem) = b1                *PRICE               
        + b2.effects          *GEO     
        + b3.effects          *COL   
        + b4.effects          *REAR     
/
U(ref)  = b0[0]
        + b1                  *PRICEref[35]
$



I cannot insert the GEOref COLref and REARref options.


Thanks in advance for your suggestions.
Maurizio

You need to carefully consider parameter identifiability. When Ngene reports an Undefined D-error, it tells you that the model that you have formulated cannot be estimated and likely has an identification issue. This has to do with dummy/effects coding and fixing one alternative to specific levels that do not appear in other alternatives, there are several posts on this forum about such identification issues in model estimation.

The model below in the syntax is identifiable, I removed the constant and I allow GEO and COL to use level 0 in BSP. I assume that level 0 is the level used in the reference alternative, whatever it is. All parameters can now be estimated. But if GEO and COL cannot be level 0 for BSP, then your model can no longer be estimated and you can only include a constant (see my previous message). Model identification is tricky business and needs to be analysed for each study separately.

Given that BSP alternatives are labelled, I removed the dominance checks. You may need to add a constant in one of these alternatives.

To simplify the experiment design generation process, you could consider only designing BSP1 and BSP2 and completely leave out the reference alternative when generating an efficient design because you actually do not know what this alternative will be anyway, this is respondent specific. Then later on when you estimate the model, you include the reference alternative from the data and you need to work out what model you can estimate given identification constraints.

Code: Select all

Design   
;alts = ref, BSP1, BSP2
;rows = 24
;block = 2   
;eff = (mnl,d)
;alg = mfederov   ? the modified Federov algorithm is needed for including require constraints that fixes the levels for the attributes in the reference alternative
;require:
ref.GEO = 0, ref.COL = 0, ref.REAR = 0,
BSP1.REAR > 0, BSP2.REAR > 0
;model:   
U(ref)  = b1[-0.01]            * PRICEref[35]
        + b2.effects[1.5|2.0]  * GEO[1,2,0]
        + b3.effects[0.5|1.0]  * COL[1,2,0]
        + b4.effects[0.5|0.75] * REAR[1,2,0]
        /
U(BSP1) = b1 * PRICE[70,120,170](6-10,6-10,6-10)  ? the modified Federov algorithm cannot satisfy attribute level balance, therefore I added constraints on the minimum and maximum number of times each attribute level appears within the design; note that with 24 rows, (8,8,8) would mean perfect attribute level balance
        + b2 * GEO
        + b3 * COL
        + b4 * REAR
        /   
U(BSP2) = b1 * PRICE                   
        + b2 * GEO       
        + b3 * COL     
        + b4 * REAR     
$

Good luck!

Michiel

Hello Michiel,

yes, reflecting on the design I coded, I think we cannot include the ref alternative with a constrained level that is excluded from the other profiles, there would be a perfect match between the ASC and the ref attribute level.

I think we'll go for the labelled experiment with only the ASC for ref alternative, the other characteristics will be either fixed or dependent on the respondents' habits.

This is the code I am thinking about, the REAR attribute is dropped and moved to the alternative labels:

Code: Select all

? Efficient (labelled) design with 2 ASCs
Design   
;alts = BSPout*, BSPsem*, ref
;rows = 24
;block = 2   
;eff = (mnl,d)
;alg = mfederov
;model:   
U(BSPout) = bBPSout[0]
        + b1[0]               *PRICE[70, 120, 170]
        + b2.effects[0|0]     *GEO[1,2,3]
        + b3.effects[0|0]     *COL[1,2,3]
/   
U(BSPsem) =  bBPSsem[0]
        + b1                  *PRICE               
        + b2.effects          *GEO     
        + b3.effects          *COL   
/
U(ref)  = b1                *PRICEref[35]
$


Thanks a lot for the useful discussion!

Maurizio

Yes this syntax looks good, but I would remove ;alg = mfederov because you no longer need constraints. The default swapping algorithm guarantees attribute level balance.

Michiel

Thank you, Michel.

We shall use the default algorithm.

Maurizio