Pilot design with attribute level constraints

[gabrielcalderonUR]

Dear all,
I'm working on an experimental design for my thesis, which aims to understand the substitution of tobacco-related products. For this, I consider a labeled design with an opt-out option. The atributtes and levels are the following:

For the pilot study I am producing a D-efficient design with ngene and the following is the code:

Code: Select all

; alts= ecigdisp*, ecigrech*, cig*, alt4
; rows= 18
; block= 2
; eff= (mnl,d)
; alg= mfederov
; require:
cig.externality=2, cig.hide=1 , cig.harm=1

;model:

U(ecigdisp)  =    price[-0.000001]*  price_disp[0.025,0.050,0.100,0.160]   ? $ Price of disposable cigarette
 + flavour.dummy[0.0001|0.00001]* flavour[0,1,2]                      ? Flavour: traditional=0, mint=1, fruits=2
 + harm.dummy[-0.0001]* harm[0,1]                                          ? Self-harm: less harmful 0(base), equally harmful=1
 + ext.dummy[-0.001|-0.00001|-0.00001]* externality[0,1,2,3]   ? Harm to others: no=0(base), moderate=1, high=2, unkown=3     
 + hide.dummy[-0.00001]*  hide[0,1]                                           ? Hide smoke: easy to hide=0(base), hard to hide=1
 + int1*  price_disp * flavour                                                        ? Interaction term between price_disp & flavour
                                 
                   
             /
U(ecigrech)=   price[-0.000001*  price_rech[0.035,0.045,0.055,0.080]  ? $ Price of rechargable cigarette
      + flavour.dummy[0.0001|0.00001]*  flavour    ? Flavour: traditional=0, mint=1, fruits=2
                + harm.dummy[-0.0001]* harm       ? Self-harm: less harmful than traditional cig=0(base), equally harmful=1
      + ext.dummy[-0.001|-0.00001|-0.00001]*  externality   ? Harm to others: no harm=0(base), moderate=1, high=2, unkown=3     
      + hide.dummy[-0.00001]* hide       ? Hide smoke: easy to hide=0(base), hard to hide=1
                + int1*  price_rech * flavour             ? Interaction term between price_rech & flavour
                                 
                       
            /

U(cig)=       price[-0.000001]* price_cig [0.025,0.05,0.200,0.350]  ? $ Price of traditional tocabbo package
      + flavour.dummy[0.0001|0.00001]* flavour                  ? Flavour: traditional=0, mint=1, fruits=2
                + harm.dummy[-0.0001]* harm                        ? Self-harm - always harmful (harmful=1)
      + ext[-0.001|-0.00001|-0.00001]* externality       ? Harm to others - always harmful
      + hide.dummy[-0.00001]*  hide          ? Hide smoke - always hard to hide

     
        /

U(alt4)= b0
$


I left the code running for up to two hours, the D-error is 0.941978 and the A-error is 2.78576
Question 1: Since for the alternative cig (i.e. traditional cigarette) there is only one level for the attributes "ext", "hide" and "harm", is the way I have written it in the code correct?
Question 2: This is a design for an experiment. This would be the design I follow for the pilot study. Should I now consider a Bayesian D-efficient design, even though I have no priors? or should I continue with the D-efficient design as presented here?
Question 3: The program returns the following warning: "One or more attributes will not have level balance with the number of rows specified: ecigdisp.price_disp, ecigdisp.externality, ecigrech.price_rech, ecigrech.externality, cig.price_cig, cig.externality". In fact, with two of these attributes we obtain only two levels. What should be the rule for determining the number of rows? I am currently using Rose and Bliemer (2022) equation 3

I am also a bit concerned about the size of the D-error.
Any additional recomendations to improve the design would be really appreciated
Thank you so much

[Michiel Bliemer]

There were several issues with your script, I tried to fix them in the script below, please have a look.
I added constants, I added interaction effects for cig, I changed your interaction effects to be appropriate for dummy coded flavour. Note that the LAST level in Ngene is the base level.
When using (near) zero priors, an efficient design will typically only use the outer levels as this provides maximum trade-off, so in that case I typically dummy code also the numerical attributes, in this case price. Your price levels were a bit odd, very small values and very unevenly spaced, but maybe that was on purpose.

Code: Select all

design
;alts= ecigdisp*, ecigrech*, cig*, optout
;rows= 24
;block= 3
;eff= (mnl,d)
;alg= mfederov
;require:
cig.externality=2, cig.hide=1 , cig.harm=1

;model:
U(ecigdisp)  = asc_ecigdisp[0]
             + price.dummy[-0.01|-0.02|-0.03]  * price_disp[1,2,3,0]                  ? $ Price of disposable cigarette
             + flavour.dummy[0|0]              * flavour[1,2,0]                       ? Flavour: traditional=0 (base), mint=1, fruits=2
             + harm.dummy[-0.001]              * harm[1,0]                            ? Self-harm: less harmful 0(base), equally harmful=1
             + ext.dummy[-0.001|-0.002|-0.002] * externality[1,2,3,0]                 ? Harm to others: no=0(base), moderate=1, high=2, unkown=3     
             + hide.dummy[-0.001]              * hide[1,0]                            ? Hide smoke: easy to hide=0(base), hard to hide=1
             + int1                            * price_disp * flavour.dummy[1]        ? Interaction term between price_disp & mint flavour
             + int2                            * price_disp * flavour.dummy[2]        ? Interaction term between price_disp & fruits flavour
             /
U(ecigrech)  = asc_ecigrech[0]
             + price.dummy                     * price_rech[1,2,3,0]                  ? $ Price of rechargable cigarette
             + flavour.dummy                   * flavour
             + harm.dummy                      * harm
             + ext.dummy                       * externality
             + hide.dummy                      * hide
             + int1                            * price_rech * flavour.dummy[1]
             + int2                            * price_rech * flavour.dummy[2]
             /
U(cig)       = asc_cig[0]
             + price.dummy                     * price_cig[1,2,3,0]                   ? $ Price of traditional tocabbo package
             + flavour.dummy                   * flavour               
             + harm.dummy                      * harm                   
             + ext.dummy                       * externality       
             + hide.dummy                      * hide         
             + int1                            * price_cig * flavour.dummy[1]
             + int2                            * price_cig * flavour.dummy[2]
$


Answers:
Q1: Yes
Q2: Using uninformative local priors as you have done is fine for the pilot study
Q3: 18 rows is not divisible by 4 levels, so it cannot obtain attribute level balance. Your number of rows will need to be divisible by 2, 3, and 4. I changed it therefore to 24 rows.

Your large D-error was the result of your very small attribute levels for price, which was not really problematic but you may want to change the unit of price (make it cents instead of dollars or something).

Michiel

[gabrielcalderonUR]

Hi Michiel,

I hope you're doing well. I wanted to apologize for the delay in getting back to you.

Thank you so much for your help in correcting the Ngene syntax for my pilot study. Your advice was incredibly helpful, and I truly appreciate the time you took to guide me.

Best regards,
Gabriel

[gabrielcalderonUR]

Hi Michiel,

I hope this message finds you well.

I recently ran the pilot study using the fixed design and obtained priors for my parameters. I am now considering generating a Bayesian D-efficient design for my experiment.

I am using Ngene and the code is the following:

Code: Select all

design
;alts= ecigdisp*, ecigrech*, cig*, optout
;rows= 24
;block= 3
;eff= (mnl,d,mean)
;alg= mfederov
;bdraws= sobol(200)
;require:
cig.externality=2, cig.hide=1 , cig.harm=1

;model:
U(ecigdisp)  = asc_ecigdisp[(n,0.924,0.236)]
             + price.dummy[(n,-0.375,0.195)|(n,-0.304,0.201)|(n,-0.003,0.113)]  * price_disp[1,2,3,0](1-21,1-21,1-21,1-21)  ? $ Price of disposable cigarette
             + flavour.dummy[(n,0.186,0.084)|(n,0.212,0.063)]                     * flavour[1,2,0]                     
             + harm.dummy[(n,-0.338,0.074)]                                       * harm[1,0]                           
             + ext.dummy[(n,-0.111,0.09)|(n,-0.537,0.12)|(n,-0.209,0.101)]        * externality[1,2,3,0]               
             + hide.dummy[(n,0.006,0.076)]                                        * hide[1,0]                           
             + int1[(n,0.304,0.142)]                                              * price_disp * flavour.dummy[1]       
             + int2[(n,-0.013,0.098)]                                             * price_disp * flavour.dummy[2]       
             /
U(ecigrech)  = asc_ecigrech[(n,1.029,0.239)]
             + price.dummy                     * price_rech[1,2,3,0](1-21,1-21,1-21,1-21) ? $ Price of rechargable cigarette
             + flavour.dummy                   * flavour                                     
             + harm.dummy                      * harm
             + ext.dummy                       * externality
             + hide.dummy                      * hide
             + int1                            * price_rech * flavour.dummy[1]
             + int2                            * price_rech * flavour.dummy[2]
             /
U(cig)       = asc_cig[(n,2.33,0.289)]
             + price.dummy                     * price_cig[1,2,3,0](1-21,1-21,1-21,1-21) ? $ Price of traditional tocabbo package
             + flavour.dummy                   * flavour                    ? Flavour: traditional=0 (base), mint=1, fruits=2
             + harm.dummy                      * harm                     ? Self-harm: less harmful 0(base), equally harmful=1
             + ext.dummy                       * externality              ? Harm to others: no=0(base), moderate=1, high=2, unkown=3     
             + hide.dummy                      * hide                        ? Hide smoke: easy to hide=0(base), hard to hide=1
             + int1                            * price_cig * flavour.dummy[1]    ? Interaction term between price_disp & mint flavour
             + int2                            * price_cig * flavour.dummy[2]     ? Interaction term between price_disp & fruits flavour
$


I left the code running for up to one hour, the D-error is 0.43 and the A-error is 0.74

I have a few questions regarding the design:
1. Im again a bit concerned about the size of the D-error and the A-error. Are these values reasonable for the design?
2. Does the atributte balance hold with the Bayesian design?
3. Should I change the price coding from a dummy to a continous level, or should I keep it as a dummy?

As with the pilot design, any additional recomendations to improve the design would be really appreciated

Thank you so much
Gabriel

[Michiel Bliemer]

1. D-error looks fine, it is expected to be high with so many dummy variables
2. With the modified Federov algorithm, attribute level balance is not guaranteed. But with dummy coding it is expected to have reasonable balance. For numerical attributes you may want to impose constraints (as you do with price).
3. I usually do not use dummy coding for numerical attributes when I have informative priors, but I may impose attribute level constraints.

Are your actual price levels 1, 2, and 3? If not, you will need to use the levels that you will be using in model estimation as in the interaction effect with flavour it is assuming that the levels are coded as 1, 2, and 3 (dollars/euros). But this does not look to be the case, since level 1 has a larger disutility than level 3. So please have a look at the coding of your price attributes.

Michiel

[gabrielcalderonUR]

Hi Michiel,
Thank you so much for your response.

As per your suggestion, I ran the design using price as a continuous variable and imposed constraints. Additionally, I corrected the priors obtained from my pilot study. The updated code is as follows:

Code: Select all

design
;alts= ecigdisp*, ecigrech*, cig*, optout
;rows= 24
;block= 3
;eff= (mnl,d,mean)
;alg= mfederov
;bdraws= sobol(200)
;require:
cig.externality=2, cig.hide=1 , cig.harm=1

;model:
U(ecigdisp)  = asc_ecigdisp[(n,0.337,0.115)]
             + price[(n,-0.514,0.465)]  * price_disp[0.025,0.050,0.100,0.160](1-21,1-21,1-21,1-21)   ? $ Price of disposable cigarette
             + flavour.dummy[(n,0.117,0.089)|(n,0.064,0.07)]                   * flavour[1,2,0]                 
             + harm.dummy[(n,-0.186,0.053)]                                    * harm[1,0]                           
             + ext.dummy[(n,-0.054,0.069)|(n,-0.309,0.085)|(n,-0.071,0.068)]   * externality[1,2,3,0]                 
             + hide.dummy[(n,0.008,0.047)]                                     * hide[1,0]                           
             + int1[(n,0.072,1.013)]                                           * price_disp * flavour.dummy[1]       
             + int2[(n,0.354,0.773)]                                           * price_disp * flavour.dummy[2]       
             /
U(ecigrech)  = asc_ecigrech[(n,0.465,0.105)]
             + price                           * price_rech[0.035,0.045,0.055,0.080](1-21,1-21,1-21,1-21) ? $ Price of rechargable cigarette
             + flavour.dummy                   * flavour
             + harm.dummy                      * harm
             + ext.dummy                       * externality
             + hide.dummy                      * hide
             + int1                            * price_rech * flavour.dummy[1]
             + int2                            * price_rech * flavour.dummy[2]
             /
U(cig)       = asc_cig[(n,1.472,0.255)]
             + price                           * price_cig[0.025,0.05,0.200,0.350](1-21,1-21,1-21,1-21) ? $ Price of traditional tocabbo package
             + flavour.dummy                   * flavour       ? Flavour: traditional=0 (base), mint=1, fruits=2         
             + harm.dummy                      * harm           ? Self-harm: less harmful 0(base), equally harmful=1       
             + ext.dummy                       * externality      ? Harm to others: no=0(base), moderate=1, high=2, unkown=3       
             + hide.dummy                      * hide                ? Hide smoke: easy to hide=0(base), hard to hide=1
             + int1                            * price_cig * flavour.dummy[1]  ? Interaction term between price_disp & mint flavour
             + int2                            * price_cig * flavour.dummy[2]    ? Interaction term between price_disp & fruits flavour
$


The prices are in dollars for all three alternatives. I kept them relatively small to ensure reasonable cost comparisons between all alternatives. The specific price levels for each alternative are detailed in the table I included in my first message. Please feel free to review it for context, or let me know if you need any further clarification!

My concern is that when prices are treated as a continuous variable, the D-error increases from 0.43 to 1.41, and the A-error also increases to 6.59

I have a few questions regarding the design:

1. Should I stick with the design that has the lower D-error, even though prices are dummy-coded?
2. How can I improve the D-error in the current design with price as a continuous variable?
3. Why does the D-error increases when price is treated as a continuous variable?

Thank you so much for your help. This forum has been incredibly helpful!

Best regards,
Gabriel

[Michiel Bliemer]

It is indeed surprising to see the D-error increase so much, that is not what I would expect.

Looking at your prior for price and your price levels, something is not right. When you use dummy coding, the utility contribution of price levels range from 0 to -0.375. However, when you consider it as a numerical variable, the utility function of the price levels range from -0.0514*0.025 = -0.00129 to -0.0514*0.160 = -0.00822. In other words, the price has negligible effect on choice and the price parameter cannot easily be estimated and the D-error goes up. So there is clearly inconsistency between the levels you used in model estimation and the levels you put in Ngene. Please check the exact price levels that you have in the dataset that you used to estimate the model, they are unlikely to be 0.025 to 0.160.

Michiel

[gabrielcalderonUR]

Thanks Michiel,

I have reviewed my model estimation, and I can confirm that the levels are consistent with those used in the Ngene design.
However, in the pilot design, I treated prices as dummy variables, following the code you provided. When estimating the model and treating prices as a continuous variable, I replaced the price levels in ascending order with the corresponding levels in cents.

For example:
Being price1-> Equivalent price in cents of 10 puffs of a disposable e-cigarette (3500 puffs)

replace price1= 0.025 if price1==0
replace price1= 0.050 if price1==1
replace price1= 0.100 if price1==2
replace price1= 0.160 if price1==3

If I estimate the model in this way, I am concerned that I might be introducing an inconsistency. Specifically, Ngene treated prices as dummy variables in the pilot design, and transforming them into a continuous variable during estimation may not align with the original design structure and could generate inconsistencies in the utility function.

Could you please confirm if this approach introduces any issues? I would appreciate your advice on whether I should modify the estimation process to ensure consistency with the pilot design and minimize the D-error

Should I stick with prices as dummy variables and remove interactions? Or is there a better way to manage them?

Best regards,
Gabriel

[Michiel Bliemer]

You can use dummy coding for price without too much issue I think, even thought the price levels are different and hence dummy coding in this case is not entirely consistent.

Your parameter estimate for the numerical variable simply does not make sense to me. When you use dummy coding or numerical coding they should show similar contributions to utility, but in your case they are entirely different. The dummy coded coefficients make sense, but the coefficient using numerical coding seems far too small (it looks like you used price levels of 2.5 in model estimation while using 0.025 in Ngene, with a factor 100 difference), so something is not right. But I cannot really help you further with that.

Michiel