Determining the Number of Choice Sets for a Frac. Fac. Des.

[acanakci]

Hello,

As we understand from the NGene manual, it does not directly provide a fractional factorial model built upon a specified resolution to determine the number of unique choice sets needed for a desired resolution level. Instead, it offers a fractional factorial design where we provide the number of choice sets, and it returns a D-efficient/A-efficient design based on this constraint.

We have 8 attributes with an average of 3 levels each. We aim to handle both first-level and second-level interactions. To create a discrete choice experiment in NGene, we need to determine the number of choice sets required to run the algorithm. Could you please recommend a method to determine the appropriate number of choice sets to provide to NGene in order to achieve a D-efficient or A-efficient design?

Thank you very much for your time and assistance!

Beyza

[Michiel Bliemer]

My recommended method is as follow.

Minimum number of rows = Number of parameters / (Number of alternatives - 1)

I recommend multiplying this minimum number of rows with at least 3 to have enough variation in your data.

When you count the number of parameters, you include any constants, coefficients of dummy variables, and coefficients of interaction effects. If you have interactions between dummy variables with K and L levels, then you will have (K-1)*(L-1) interaction terms and interaction parameters. If you are unsure, please post your Ngene script and I can tell you how many parameters you have in the model.

Michiel

[acanakci]

Hello,

Thank you very much for your clear response.

To conduct an explicit partial profile design, I externally created the dataset. We have 9 attributes with an average of 4 levels each. Since generating all possible combinations is impractical due to certain limitations, I followed a heuristic approach which resulted in approximately 53,000 rows. Based on our forum discussions, I will randomly select 5,000 rows out of these 53,000 to run the Federov Algorithm.

Here’s my process: I heuristically created 53,000 rows by varying only 2 attributes between 2 alternatives and keeping the other 7 attributes constant. I found all pairs (9 choose 2 = 36) and their combinations by considering the attribute levels. Then, I combined all combinations with 30 randomly sampled constant attributes among the alternatives. I did not consider all combinations of the constant attributes. Among these 53,000 rows, I will randomly select 5,000 and provide them to NGene, specifying that I want to have 138 questions/final choice sets (calculated using the formula you suggested: (number of parameters / number of alternatives - 1) * 3).

Therefore, the pattern is: 53,000 heuristically created rows -> 5,000 randomly sampled rows fed to NGene -> 138 questions (46 sets of 3) are created at the end.

Based on your experience, will this approach result in a significantly low D-Efficiency value? Is there a way to anticipate the D-Efficiency or can we have a trial run to finalize our decision to purchase the licence?

Thank you very much for your reply!

Beyza

[Michiel Bliemer]

A few things to consider:

(1) If you only vary 2 attributes among 9, there may be a very high likelihood that this creates dominant alternatives. If your attributes are on a nominal measurement scale then there should be no issue, but if they are ordinal or numerical then you will need to remove choice tasks in your candidate set that contain a dominant alternative. Ngene could automatically do this for you, but you can also do this yourself when you construct the candidate set.

(2) If you only vary 2 attributes among 9, you will capture much less information per choice task than when you vary all 9 attributes per choice task. This means that you may need a larger design than usual.

A random sample of 5000 rows should be sufficient to get a good design. You could always choose to do multiple runs with a different selection of 5000 rows, or you can see what the impact would be of using 2000, 5000, or 10000 rows in the candidate set on design efficiency.

When generating the design you may want to play around with the number of rows in the design to see how it impacts efficiency per choice task.

I believe that you can run in the demo version of Ngene and it will show you the efficiency, but it will only show the design when you purchase a licence.

Michiel

[acanakci]

Hello,

Just to be sure:
I have 9 attributes. I want to generate a design in which I handle both first and second level interactions. Then, would I have 36 (9 choose 2) + 9 = 45, 45/2 (=3-1) = 22.5, 22.5* 3 = (approximately) 68 choice tasks?

Thank you very much in advance!

Beyza

[Michiel Bliemer]

I do not understand your calculation of 45, but if that is the number of parameters then yes. It depends on the type of attributes (numerical or categorical).

I would probably multiply with more than 3 as I suggested earlier because you are using partial profiles and therefore you are capturing much less information than in designs with full profiles. It never hurts to use a larger design.

Michiel