D-Efficient Design

[Yashin Ali]

Dear Professor,

I am working on EV adoption for the city of Innsbruck, Austria. While I planned to make a D-efficient design choice experiment, my a-priori estimates will be based on previous works of literature. My question is, How reliable are those a priori estimates given that the geographical location, sampling size are way too different from my work?

Should I rely on, simulation BOT for my a priori estimates while building the survey in SurveyEngine?

[Michiel Bliemer]

There are a few strategies you could follow.

1. Priors from the literature are indeed often unreliable because of scaling issues and different cultural and geographical backgrounds as you indicate. You could choose conservative priors by shrinking all priors from the literature towards zero (e.g. dividing by 2 or 3).

2. You can use zero priors in a pilot study, give that to 10% of your sample, estimate parameters that you use as priors for a design for your main study that you give to 90% of the population. This is the strategy that I typically follow.

The simulation bot in SurveyEngine RANDOMLY selects choices, there is no behaviour behind it, so these are USELESS as priors and you should NOT use.

Michiel

[Yashin Ali]

Dear Prof,

So What I have understood is setting up the zero as my priors and give it to my 10% of the targeted sample size. Once I receive the response from the 10% of the population, I will use them as my priory for the 90% of the population. So it does include 2 step Study process?

1. Pilot Study
2. Main Study

[Michiel Bliemer]

Yes I always follow that 2-step process where I do a pilot study to check whether the survey makes sense to respondents (I add some additional questions about difficulty of the survey and a comment field). I also use pilot study data to estimate parameters that I can use as priors for the main study.

Michiel

[Yashin Ali]

Thank You Professor.

I would like to add here something.
Do i have to provide the whole survey to 10% of my population for the pilot study or just the part of the Choice Experiment ?

[Michiel Bliemer]

That depends on what you like to use your pilot study for. I use the pilot study also to completely test my survey and see if the responses make sense, so I give the respondents my complete survey, not just the experiment.

Note that you will likely also do some pre-testing with a small set of people to get some written feedback or you ask them to fill out the survey in your presence and give you verbal feedback to make sure that they understand the questions.

Michiel

[Yashin Ali]

Thank You Professor for your reply.

If I am not wrong than, D-efficient and S efficient designs are the worst when you don't have any information about the priors.Hence in that case, Orthogonal design will be most efficient design?

In some software such as (Sawtooth), the Complete Enumeration designs gives an efficient design without setting up the priors while I understand from the literature and the books says that for D-efficient and S efficient design it is necessary to set up the priors first before the parameter estimation.

I am confused about the actual answer because of the different opinions for the priors. Could you please explain it?

2. If the Choice experiment is symmetric (attributes having the same number of levels), does it conclude that orthogonal and efficient designs are same ?

[Michiel Bliemer]

If you have no information, you simply set all priors to zero.

Orthogonal designs are efficient under specific conditions as shown in Street, Burgess and Louviere (2005), https://www.sciencedirect.com/science/article/pii/S0167811605000510. These conditions include zero priors, unlabelled alternatives, and orthogonal polynomial coding.

In Ngene you can create such optimal (efficient) orthogonal designs using ;orth = ood.

Michiel

[Yashin Ali]

Dear Sir,

Is it possible to understand the efficient design by looking into to csv file or the spreadsheet file containing the levels and their attributes?
Are there certain characteristics for D-efficient design?