"Pivot" design without fixed alternative?

[peyman_07]

Hey Michiel,

Based on your advice, I have defined a library of designs for my survey. As described before, a taxi driver drops off a passenger and needs to select one of these three options: waiting at the drop-off location to receive next ride request, cruising based his experience to find a new request, and following the application recommendations. The attributes are the time of day (TD) and location (L) which are context variables. The design is pivoted around Time of Day which is a dummy attribute as shown below:

td_morning.dummy [0|0]*TD_startshift_morning [8,12,16]
td_afternoon.dummy [0|0]*TD_startshift_afternoon [12,16,20]
td_evening.dummy[0|0]*TD_startshift_evening [16,20,24]

So, I have three different designs based on this categorization. My questions are: a) Given that this attribute is a dummy one, I was wondering if I need to estimate three different models for estimating the "td" parameter for each level? If not, how many parameters need to be defined and estimated to enable us to interpret each level? b) The sample size would be around 500. Does each design need to be completed by one-third of the respondents? If not, may it result in insignificant "td" parameter(s) of the design having a small number of respondents?

Thanks,
Peyman.

[Michiel Bliemer]

I am not sure how best to include this in utility functions. I would likely create an additional dummy variable called timeofday with levels morning, afternoon, and evening, which are essentially covariates of the drivers that you use to pivot around, and then create interaction effects with this timeofday dummy variable and the three startshift dummy variables such that the utility function includes the relevant timeshift variables for each respondent while the other dummy variables drop out in the interaction, e.g. if a respondent works in the morning then only the morningstartshift variables remain in the model while the other ones get value zero. This would allow you to estimate a single model with all three startshift dummy variables while you would also add the timeofday dummy variable as a main effect to account for the best level. Note that this is not a straightforward model and you need to carefully think about your utility functions.

Your sample size does not need to be split equally in 3 parts, but of course if one part receives only a few observations than those dummy variable parameters may not be significant.

If you have further Ngene specific questions please let me know.

Michiel

[peyman_07]

Hi Michiel,

In the Ngene manual, all types of pivot design have been listed. However, I could not find a detailed discussion of their (dis)advantages. Previously, you advised me to use segmentation (a library of designs) for my design and referred me to one of your papers with more than 1000 offline designs. Would you please give me more insights into the benefits and drawbacks of this method compared to the other forms of pivot design? I would also appreciate it if you introduce me any paper that elaborates on it.

Thanks,
Peyman.

[Michiel Bliemer]

A library of designs is not specifically discussed in the literature I think. It is something that I often do and recommend to others in courses I teach. I call it a "library of designs" but it essentially refers to multiple version of the survey where you link to specific questions dynamically based on responses from respondents. Therefore, it is a feature of the survey instrument, not so much of the design. You can often find multiple versions of surveys and dynamic links in online survey instruments like Qualtrics. In Ngene, you simply generate multiple designs as you normally would. While I thought this was straightforward, your comment tells me that maybe we should include a discussion on this in the Ngene manual.

Respondent-specific designs done with pen and paper ask respondents some questions in the beginning of the survey, and then depending on their answer, they state "Go to question X of this survey", referring to the appropriate set of questions for the respondent. This is the same way a library of designs works, it is simply a set of questions in the survey instrument that respondents are referred to dynamically within the survey instrument.

Advantages of a library of designs:
You create all designs a-priori and can check that all questions make sense. In a pivot design the levels are not known in advance and you do not know what attribute levels are provided by the respondent. For example, if the respondent puts in a price of 0 dollars, then relative pivots of -5% and +5% do not work. If a respondent puts in a price of 1 dollar, then an absolute pivot of -5 dollars does not work. If a respondent puts in a price of 85 dollar, then a relative pivot of +15% would mean 97.75 dollars, which would need rounding off. In other words, a pivot design requires careful implemation in the survey instrument, wwhereas a library of designs simply refers to the appropriate set of questions in the survey. In Ngene, pivot designs have several limitations, e.g. we require the inclusion of a reference alternative, while with a library of designs you have complete flexibility.

Disadvantage of a library of designs:
You need to create several sets of survey questions. For the study with more than 1000 different designs, it means implementing 1000 sets of questions in the survey instrument. Most people only have a limited library, for example 5 of 10, so it is often not a big issue.

A study where we generated 315 experimental designs is reported in Batley et al. (2019), although not many details are provided.

Batley, Bates, Bliemer, et al. (2019) New appraisal values for travel time saving and reliability in Great Britain. Transportation, Vol. 46, pp. 583-621.

Michiel