choice-metrics.com

Dear Michiel,

I am trying to generate a partial choice set design following the method in the section 8.11 of the Ngene Manual. However, I have a couple of doubts which I could not resolve by myself.


1) Is there any limit for the total number of alternatives? I mean, would it be ok to apply such method with, for example, 15 alternatives?


2) How does the candidate set design is specifically created? The Manual says that it can be created in Excel, R, or Matlab. Thus, I imagine that a complete candidate set is created outside Ngene. Also, the Manual says that in the case of complete candidate set be very large, one can use an incomplete design with a random selection of 1% of all possible choice tasks. Thus, I imagine that I should first create the complete candidate set and then sample a subset from that (e.g., 1%). Because it is not clear to me this process, I would really appreciate if you could check the steps below (or even share a reproducible example from R):

GENERATING THE CANDIDATE SET DESIGN IN R
a. Generate a complete design
b. Random select 1% of all possible choice tasks
c. Evenly randomly alocate 999 across the attributes

GENERATING THE PARTIAL CHOICE SET DESIGN IN NGENE
d. Finally, generate the partial choice set in NGENE using a synthax such as

Design
;alts = car, train, bus, tram, bike
;rows = 15
;eff = (mnl,d)
;alg = mfederov(candidates = partial_choice_sets.csv)
;model:
U(car) = b1[0.3] +
b2[-0.05] * ctime[15,20,25,999] +
b3[-0.3] * fuel[1,2,999] +
b4[-0.4] * toll[0,1,999] /
U(train) = b5[0.2] +
b6[-0.04] * ttime[10,15,20,999] +
b7[-0.08] * accegg[5,10,15,999] +
b8[-0.08] * transfer[0,5,10,999] +
b9[-0.3] * fare[2,3,999] /
U(bus) = b10[-0.2] +
b11[-0.06] * btime[15,20,25,999] +
b7 * accegg +
b8 * transfer +
b9 * fare2[1,2,999] /
U(tram) = b12[0.1] +
b6 * ttime +
b7 * accegg +
b8 * transfer +
b9 * fare2 /
U(bike) = b13[-0.08] * biketime[20,30,40,999]
$

Does it sound correctly?


3) Does this method work for experiments in which the quantity of alternatives vary across choice sets? For example, I would like to show choice sets varying between four, five, and six alternatives rather than to show a fixed number (e.g., four alternatives per choice set). If yes, is there any special procedure in the candidate set design generation?


Thank you very much for your time!

Best, Rafael Lionello.

Hi Rafael,

1. No there is no limit

2. When the total number of possible choice tasks is very large (which will likely be the case if you have 15 alternatives) then first generating a complete design will not be feasible. In that case, you need to randomly generate choice tasks in a smarter way as shown with pseudo code below.

For rownr = 1:Totalrows (e.g., 10000)
-- Randomly select 4 to 6 alternatives
-- Randomly select attribute levels of those 4 to 6 alternatives
-- Set attribute levels of other attributes to 999 or -999 (depending on sign of prior) or for dummy coding set to base level
Next row

3. Yes, your candidate set can contain choice sets of various size, e.g. 4 to 6. But note that choice sets with 6 alternatives generally result in more Fisher information, so when generating an effcicient design it is likely that most selected choice tasks would have 6 alternatives. So what you could do is create 3 candidate sets, with 4, 5, and 6 alternatives respectively. And then generate 3 efficient designs based on each of these sets and then simply pool the three designs into a single design.

Michiel

I have created a series of spreadsheets that create candidate sets for a full or partial choice set design with up to 10 alternatives (each with up to 10 attributes). The spreadsheets can be downloaded via the link below.

https://cloudstor.aarnet.edu.au/plus/s/yTPQWAZpi2LFOCj

For models with more than 10 alternatives you can create a similar spreadsheet.
(I am happy to assist with this, but then we will need to discuss my involvement in the research).

Michiel

Thank you very much, Professor!

Dear Professor,

How should I treat a opt-out alternative (which is always present in the choice tasks) in the spreadsheets you have shared? Since it has no attribute, should I use the spreadsheet with the number of alternatives minus the opt-out?

For example, if my choice set has 5 alternatives + opt-out, Should I use the "candidate set generator 5 alts" spreadsheet?

Thank you very much.

Rafael.

With an opt-out alternative you define all 6 alternatives in the Ngene script but you only specify utility functions for 5 alternatives and in the spreadsheet you indeed also only consider 5 alternatives. In this case, Ngene sets the utility of the opt-out to 0, so it still gets a choice probability.

;alts = alt1, alt2, alt3, alt4, alt5, optout
;model:
U(alt1) = ... /
...
U(alt5) = ...
$

Hi Prof. Bliemer,

Greetings.

Thank you for creating the 'partial choice task generator'. Do you also have a similar excel generator for 'partial profile'?

Regards,
Kuldeep

Michiel Bliemer wrote:I have created a series of spreadsheets that create candidate sets for a full or partial choice set design with up to 10 alternatives (each with up to 10 attributes). The spreadsheets can be downloaded via the link below.

https://cloudstor.aarnet.edu.au/plus/s/yTPQWAZpi2LFOCj

For models with more than 10 alternatives you can create a similar spreadsheet.
(I am happy to assist with this, but then we will need to discuss my involvement in the research).

Michiel

No I do not have partial profile generators at the moment. I will see if I can do this sometime soon.

An alternative way of doing it is to let Ngene generate a large random design (using ;fact and ;rows = 50000 or something), then copy & paste these to Excel, and then in Excel determine how much level overlap there is in a choice task (using if-statements) and only keep the rows that have the desired overlap.

Michiel

I have now created a spreadsheet that creates a candidate set with overlapping attribute levels, which is what is needed for a partial profile design.

In the spreadsheet you put in the levels for each attribute and the number of attributes you want to overlap. Note that this number is treated as a minimum, as other attributes may also have overlap. If you prefer a candidate set where an exact number of attributes have overlap, then you can consider removing the undesirable rows from the candidate set.

https://www.dropbox.com/scl/fi/ly03e0kzvwg2i2flcl867/Partial-profile-candidate-set-generator.xlsx?rlkey=1mrp3q1yjgi4tewwh8rp4qwyn&dl=0

If you spot any mistakes in the spreadsheet, please let me know.

Michiel

Thank you Prof. Bliemer for prompt action and assistance. I will try using this excel and keep you informed if there is any issue.

Regards,
Kuldeep