Unusual experimental design

[rem_max]

Dear Prof. Bliemer, Dr. Collins and forum participants,
firstly I want to thank you for providing this great software .

Secondly I have a few questions about a survey design we are currently trying to implement and get going:

The survey is esentially about wheter participants would choose a parcel locker/ delivery point or home delivery for a product they've ordered online. So we have 2 labeled Alternatives: Parcel Locker/deliver point and Home Delivery.
There is also the possibility to include multiple parcel lockers simultaneously as alternatives, but the scope and specifications of the ultimate goal of our survey/ model specifies only 1 parcel locker. Furthermore we
are somewhat restricted in attribute-creation and selection because of lacking data for the model implementation. This leads to the situation, that the alternative "Home Delivery" has no (varying) attributes at all (also no atttributes inside the utility function) and somewhat acts like an opt-out I guess. The alternative "parcel locker" has 6 attributes with 2-4 attribute levels.

There is little information about the priors of the attributes, albeit it should be possible to make an educated guess if they're negative or positive.

After some research on this forum I have concluded that using an efficient design with very small priors or no priors at all to start with would be the best option. We plan to conduct a small pilot study with about 10 people and use the results to optimise the designs with the gained priors for the large study. One option would have also been to use an orthogonal design to start with, but because of our attribute levels, this would result in a minimum of 70+ choice situation, which we consider too much, even if blocked.
Before this Sated-Choice Section we included a few sociodemographic questions, which we plan to include in the model at the end (as far as I've read on this forum, including these during the design phase in Ngene is not recommended/ necessary).

Our current design looks like this:

Design
;alts = HomeDelivery, ParcelLocker
;rows = 24
;eff = (mnl,d)
;block = 2

;model:

U(HomeDelivery) = ASC[0]
/
U(ParcelLocker) = dist.dummy[0|0|0] * dist[1,2,3,0] ? Distance from home to parcel locker in meters: 0=100-#Ref; 1=300; 2=800; 3=1600
+ time.dummy[0|0] * time[1,2,0] ? opening hours: 0=24/7-#Ref; 1=Mo-Sun 8-18; 2=Mo-Fr 8-18
+ lag.dummy[0|0] * lag[1,2,0] ? Location parcel locker/ del point: 0=near shopping-#Ref; 1=near gas station/ post office; 2=near public transit hub
+ verk.dummy[0] * verk[1,0] ? possibility of combining parcel collection with existing trip: 0=Yes-#Ref; 1=Separate trip from home necessary
+ park.dummy[0] * park[1,0] ? parking spot near parcel locker: 0=Yes-#Ref; 1=no
+ inc[0] * inc[0,3,6] ? monetary Incentive (delivery fee reduction) in €:
$

The D-Error of this Design is around 0.63, which seems quite large in my opinion.
If I use very small priors (-0.00001 etc.) the D-Error is about the same.

Our question now if this design approach seems senseable at all? I've never seen an example of a design that has 2 alternatives while one of the utility functions is basically empty.
Some of our researchers also worry about the missing level balance inside the designs and blocks, caused by the efficient design. Especially for attributes like distance which has 4 levels there seems to be some danger
if a design has a low number of appearances of 1 particular level. Is this a legitimate worry? Maybe there is a way to balance only the most important attributes? I also thought about just going through the list of generated designs and picking the
one with the best balance (while also not deviating too far from the lowest D-Error).


Thank you so much for yout help!

Best regards,
Max

[Michiel Bliemer]

The HomeDelivery option can be considered a status quo alternative, which may have some attributes like cost for example. See below an example syntax where I also include cost for home delivery. Without knowing the cost of home delivery it is difficult to make a choice, for example comparing $103 with $100 is not the same as comparing $13 with $10. You could impose constraints on the design, such as ;cond: if(homedelivery.cost _home = 15, parcellocker-cost_locker=[6,9]).

Code: Select all

Design
;alts = HomeDelivery, ParcelLocker
;rows = 24
;eff = (mnl,d)
;block = 2

;model:

U(HomeDelivery) = ASC[0]
                + cost[0]           * cost_home[9,12,15]
                /
U(ParcelLocker) = dist.dummy[0|0|0] * dist[1,2,3,0]      ? Distance from home to parcel locker in meters: 0=100-#Ref; 1=300; 2=800; 3=1600
                + time.dummy[0|0]   * time[1,2,0]        ? opening hours: 0=24/7-#Ref; 1=Mo-Sun 8-18; 2=Mo-Fr 8-18
                + lag.dummy[0|0]    * lag[1,2,0]         ? Location parcel locker/ del point: 0=near shopping-#Ref; 1=near gas station/ post office; 2=near public transit hub
                + verk.dummy[0]     * verk[1,0]          ? possibility of combining parcel collection with existing trip: 0=Yes-#Ref; 1=Separate trip from home necessary
                + park.dummy[0]     * park[1,0]          ? parking spot near parcel locker: 0=Yes-#Ref; 1=no
                + cost              * cost_locker[3,6,9] ? monetary Incentive (delivery fee reduction) in €:
$

Your model is not unusual, I have seen quite some choice experiments with the status quo and a new option. If you wish, you could add another ParcelLocker alternative. This would capture more information specifically about trade-offs between the attributes of the parcel locker. This may be important if you are interested in determining willingness-to-pay. It would also bring the D-error down. If you are only interested in forecasting market shares then the second parcel locker alternatives is not that important.

Code: Select all

Design
;alts = HomeDelivery, ParcelLocker1, ParcelLocker2
;rows = 24
;eff = (mnl,d)
;block = 2

;model:

U(HomeDelivery) = ASC[0]
                + cost[0]           * cost_home[9,12,15]
                /
U(ParcelLocker1) = dist.dummy[0|0|0] * dist[1,2,3,0]      ? Distance from home to parcel locker in meters: 0=100-#Ref; 1=300; 2=800; 3=1600
                + time.dummy[0|0]   * time[1,2,0]        ? opening hours: 0=24/7-#Ref; 1=Mo-Sun 8-18; 2=Mo-Fr 8-18
                + lag.dummy[0|0]    * lag[1,2,0]         ? Location parcel locker/ del point: 0=near shopping-#Ref; 1=near gas station/ post office; 2=near public transit hub
                + verk.dummy[0]     * verk[1,0]          ? possibility of combining parcel collection with existing trip: 0=Yes-#Ref; 1=Separate trip from home necessary
                + park.dummy[0]     * park[1,0]          ? parking spot near parcel locker: 0=Yes-#Ref; 1=no
                + cost              * cost_locker[3,6,9] ? monetary Incentive (delivery fee reduction) in €:
                /
U(ParcelLocker2) = dist * dist
                + time * time
                + lag * lag
                + verk * verk
                + park * park
                + cost * cost_locker
$

Using zero priors is fine, only for detecting dominant alternatives it is useful to use small positive/negative priors, but you have a labelled experiment and hence no strict dominance exists.

Attribute level balance is not necessary within a block, but if you believe that a block is not sufficiently balanced then you could make manual changes by swapping choice tasks between blocks, this does not affect the efficiency of the design.

Michiel

[Michiel Bliemer]

Btw, this article may be of interest to you:

https://www.emerald.com/insight/content/doi/10.1108/IJPDLM-01-2021-0013/full/html

[rem_max]

Dear Prof. Bliemer,
thanks for the useful information you have given me so far. We have conducted our pilot survey and got somewhat plausible priors.
Before starting our main survey I have one more general question:
We plan to have 500 participants which are representatively distributed by age group, gender and state of origin. If we now use a blocked design with 2 blocks, is it necesarry that this represenative distribution by age, gender, location is also kept within each block? Or is it enough to randomly assign one of the 2 blocks to a person? I sadly haven't found any further information on this problem.

Thanks for your help!

[Michiel Bliemer]

You can randomly assign a block to a person. Information about what block was shown to a respondent will not be used in the choice model. Blocking is only done to get a reasonable degree of attribute level balance of choice tasks shown to a respondent so that everyone sees a range of attribute levels.

Michiel