Implicit Partial Profiles - Error Undefined

[acanakci]

Hello,

May I kindly ask you to elaborate on the potential reasons I get the "Error Undefined" result with the following syntax, even though my utility functions are not complicated and the prior parameters are non-zero?

Design
;alts = Alt1, Alt2
;rows = 180
;eff = (mnl, d)
;alg = mfederov(candidates = selection_1000_modification_2_comma_delimited.csv)
;block = 20
;model:
U(Alt1) = b1.dummy[0.2] * ToolSupply[0,1] +
b2.dummy[0.15] * Weather[0,1] +
b3.dummy[0.25|0.25] * Traffic[0,1,2] +
b4.dummy[0.2|0.2] * Insurance[0,1,2] +
b5.dummy[0.15] * Locker[0,1] +
b6.dummy[0.3|0.3|0.3|0.3] * WorkingHours[0,1,2,3,4] +
b7[0.1] * BikeLines[0:1] +
b8[0.25] * CompPerPackage[40:160] +
b9[0.2] * CompPerKM[20:60] /

U(Alt2) = b1 * ToolSupply +
b2 * Weather +
b3 * Traffic +
b4 * Insurance +
b5 * Locker +
b6 * WorkingHours +
b7 * BikeLines +
b8 * CompPerPackage +
b9 * CompPerKM $

Thank you.

[Michiel Bliemer]

1. Please write out your levels for the numerical attributes. Instead of CompPerPackage[40:160], which is not defined in Ngene, simply write something like CompPerPackage[40,60,80,100,120,140,160]. Ngene does know a shortcut, namely 40:160:20, where the last value is the step size. The syntax 40:160 can only be used in combination with another algorithm, which I would not recommend.

2. You may want to use ;alts = alt1*, alt2* to automatically avoid dominant alternatives, although this will not be a major issue in your script.

3. Your priors are somewhat problematic; a prior of 0.25 for CompPerPackage, where levels range from 40 to 160, is very large. Instead of choosing priors manually, they preferably come from a pilot study as it is very difficult to guess priors and choosing wrong priors can make the design very inefficient.

If you still receive an error message after fixing 1, then the issue is likely in your candidate set. Make sure that all levels in the candidate set also appear in the script.

Michiel

[acanakci]

Thank you very much! I will fix the issues you mentioned. Although it may not be relevant to my previous question, may I ask how we add the "opt-out" to our final design? How should I arrange my script accordingly?

Beyza

[Michiel Bliemer]

For an opt-out you add a third alternative with only a constant:

U(optout) = b0[..]

Michiel

[acanakci]

Hello,

According to your suggestion, I fixed the levels for compensation per package, per km, and proportion of bike lanes on the route as follows:

" Design
;alts = alt1*, alt2*
;rows = 180
;eff = (mnl, d)
;block = 30
;model:
Design
;alts = alt1*, alt2*
;rows = 180
;eff = (mnl, d)
;block = 30
;model:
U(Alt1) = b1.dummy[0.00001] * ToolSupply[0,1] +
b2.dummy[0.00001] * Weather[0,1] +
b3.dummy[0.00001|0.00001] * Traffic[0,1,2] +
b4.dummy[0.00001|0.00001] * Insurance[0,1,2] +
b5.dummy[0.00001] * Locker[0,1] +
b6.dummy[0.00001|0.00001|0.00001|0.00001] * WorkingHours[0,1,2,3,4] +
b7[0.00001] * BikeLines[0,0.25,0.5,0.75,1] +
b8[0.00001] * CompPerPackage[40,80,120,160] +
b9[0.00001] * CompPerKM[20,30,40,50,60] +
i1[0.00001] * ToolSupply.dummy[1] * Weather.dummy[1] +
i2[0.00001] * ToolSupply.dummy[1] * BikeLines +
i3[0.00001] * ToolSupply.dummy[1] * Traffic.dummy[1] +
i4[0.00001] * ToolSupply.dummy[1] * Traffic.dummy[2] +
i5[0.00001] * ToolSupply.dummy[1] * Insurance.dummy[1] +
i6[0.00001] * ToolSupply.dummy[1] * Insurance.dummy[2] +
i7[0.00001] * ToolSupply.dummy[1] * WorkingHours.dummy[1] +
i8[0.00001] * ToolSupply.dummy[1] * WorkingHours.dummy[2] +
i9[0.00001] * ToolSupply.dummy[1] * WorkingHours.dummy[3] +
i10[0.00001] * ToolSupply.dummy[1] * WorkingHours.dummy[4] +
i11[0.00001] * BikeLines * Weather.dummy[1] +
i12[0.00001] * Traffic.dummy[1] * Weather.dummy[1] +
i13[0.00001] * Traffic.dummy[2] * Weather.dummy[1] +
i14[0.00001] * Insurance.dummy[1] * Weather.dummy[1] +
i15[0.00001] * Insurance.dummy[2] * Weather.dummy[1] +
i16[0.00001] * Locker.dummy[1] * Weather.dummy[1] +
i17[0.00001] * CompPerPackage * Weather.dummy[1] +
i18[0.00001] * CompPerKM * Weather.dummy[1] +
i19[0.00001] * BikeLines * Insurance.dummy[1] +
i20[0.00001] * BikeLines * Insurance.dummy[2] +
i21[0.00001] * BikeLines * WorkingHours.dummy[1] +
i22[0.00001] * BikeLines * WorkingHours.dummy[2] +
i23[0.00001] * BikeLines * WorkingHours.dummy[3] +
i24[0.00001] * BikeLines * WorkingHours.dummy[4] +
i25[0.00001] * Traffic.dummy[1] * Insurance.dummy[1] +
i26[0.00001] * Traffic.dummy[1] * Insurance.dummy[2] +
i27[0.00001] * Traffic.dummy[2] * Insurance.dummy[1] +
i28[0.00001] * Traffic.dummy[2] * Insurance.dummy[2] +
i29[0.00001] * Traffic.dummy[1] * WorkingHours.dummy[1] +
i30[0.00001] * Traffic.dummy[2] * WorkingHours.dummy[1] +
i31[0.00001] * Traffic.dummy[1] * WorkingHours.dummy[2] +
i32[0.00001] * Traffic.dummy[2] * WorkingHours.dummy[2] +
i33[0.00001] * Traffic.dummy[1] * WorkingHours.dummy[3] +
i34[0.00001] * Traffic.dummy[2] * WorkingHours.dummy[3] +
i35[0.00001] * Traffic.dummy[1] * WorkingHours.dummy[4] +
i36[0.00001] * Traffic.dummy[2] * WorkingHours.dummy[4] +
i37[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[1] +
i38[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[2] +
i39[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[3] +
i40[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[4] +
i41[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[1] +
i42[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[2] +
i43[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[3] +
i44[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[4] +
i45[0.00001] * CompPerKM * WorkingHours.dummy[1] +
i46[0.00001] * CompPerKM * WorkingHours.dummy[2] +
i47[0.00001] * CompPerKM * WorkingHours.dummy[3] +
i48[0.00001] * CompPerKM * WorkingHours.dummy[4] / ..."

I have a couple of questions related to this change:
Could I preserve their continuous nature? Is this way correct to handle them? Should I add an interaction term for each level, just as the discrete ones?

One more question related to one of the previous pieces of advice (writing alts = alt1*, alt2*), when I add it to my script as above I encounter the following error:
"A valid initial random design could not be generated after approximately 10 seconds. In this time, of the 13899 attempts made, there were 0 row repetitions, 0 alternative repetitions, and 13899 cases of dominance. There are a number of possible causes for this, including the specification of too many constraints, not having enough attributes or attribute levels for the number of rows required, and the use of too many scenario attributes. A design may yet be found, and the search will continue for 10 minutes. Alternatively, you can stop the run and alter the syntax."

When I write alts = alt1, alt2; then I get the error value: 0.108278. Is this a suitable/well-accepted value?

Thank you very much!

Beyza

[Michiel Bliemer]

The error message explains exactly what is the problem: "In this time, of the 13899 attempts made, there were ... 13899 cases of dominance." In other words, using the default swapping algorithm, it is not possible to find a feasible design without any dominant alternatives because your number of rows is very large. The solution is to switch to the modified Federov algorithm by adding:

;alg = mfederov

Note that attributes like Weather are typically scenario variables that are constant across both alternatives, so you may need to add:

;require:
alt1.weather = alt2.weather

and create interactions with the weather attribute instead of adding it as a main effect because that would otherwise lead to an unidentifiable model, or add an opt-out alternative.

Michiel

[acanakci]

Thank you for your response. I realize I had multiple questions in my previous post, which might have led to some being overlooked. May I kindly re-ask the unanswered ones?

Additionally, could you please explain the issue with the "Weather" attribute? Why should it be kept constant across alternatives?

Lastly, the syntax line you suggested, is it used to ensure the levels of specific attributes remain the same across alternatives, similar to explicit partial profiles?

Beyza

[Michiel Bliemer]

If weather is a scenario variable, it should be fixed across alternatives. For example: Suppose it is raining, would you choose the car or bicycle? It is raining no matter one chooses the car or bicycle.

But perhaps in your case weather is not a scenario variable, so if weather is an attribute of your alternatives then you can vary it across alternatives.

The require command that I provided indeed creates attribute level overlap,similar to explicit partial profiles, but differently to explicit partial profiles this attribute will always be overlapping (in partial profiles it is only overlapping sometimes).

Michiel

[acanakci]

Thank you for your assistance so far.


I have the following design:

Design
;alts = alt1*, alt2*
;rows = 80
;eff = (mnl, d)
;block = 20
;alg = mfederov
;model:
U(Alt1) = b1.dummy[0.00001] * ToolSupply[0,1] +
b2.dummy[0.00001] * Weather[0,1] +
b3.dummy[0.00001|0.00001] * Traffic[0,1,2] +
b4.dummy[0.00001|0.00001] * Insurance[0,1,2] +
b5.dummy[0.00001] * Locker[0,1] +
b6.dummy[0.00001|0.00001|0.00001] * WorkingHours[0,1,2,3] +
b7[0.00001] * BikeLines[0,0.25,0.5,0.75,1] +
b8[0.00001] * CompPerPackage[40,80,120,160] +
b9[0.00001] * CompPerKM[20,30,40,50,60] +
i1[0.00001] * ToolSupply.dummy[1] * Weather.dummy[1] +
i2[0.00001] * ToolSupply.dummy[1] * BikeLines +
i3[0.00001] * ToolSupply.dummy[1] * Traffic.dummy[1] +
i4[0.00001] * ToolSupply.dummy[1] * Traffic.dummy[2] +
i5[0.00001] * ToolSupply.dummy[1] * Insurance.dummy[1] +
i6[0.00001] * ToolSupply.dummy[1] * Insurance.dummy[2] +
i10[0.00001] * BikeLines * Weather.dummy[1] +
i11[0.00001] * Traffic.dummy[1] * Weather.dummy[1] +
i12[0.00001] * Traffic.dummy[2] * Weather.dummy[1] +
i13[0.00001] * Insurance.dummy[1] * Weather.dummy[1] +
i14[0.00001] * Insurance.dummy[2] * Weather.dummy[1] +
i16[0.00001] * CompPerPackage * Weather.dummy[1] +
i17[0.00001] * CompPerKM * Weather.dummy[1] +
i18[0.00001] * BikeLines * Insurance.dummy[1] +
i19[0.00001] * BikeLines * Insurance.dummy[2] +
i20[0.00001] * BikeLines * WorkingHours.dummy[1] +
i21[0.00001] * BikeLines * WorkingHours.dummy[2] +
i22[0.00001] * BikeLines * WorkingHours.dummy[3] +
i23[0.00001] * Traffic.dummy[1] * Insurance.dummy[1] +
i24[0.00001] * Traffic.dummy[1] * Insurance.dummy[2] +
i25[0.00001] * Traffic.dummy[2] * Insurance.dummy[1] +
i26[0.00001] * Traffic.dummy[2] * Insurance.dummy[2] +
i33[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[1] +
i34[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[2] +
i35[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[3] +
i36[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[1] +
i37[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[2] +
i38[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[3] +
i39[0.00001] * CompPerKM * WorkingHours.dummy[1] +
i40[0.00001] * CompPerKM * WorkingHours.dummy[2] +
i41[0.00001] * CompPerKM * WorkingHours.dummy[3] +
i42[0.00001] * BikeLines * Traffic.dummy[1] /

U(Alt2) = b1 * ToolSupply +
b2 * Weather +
b3 * Traffic +
b4 * Insurance +
b5 * Locker +
b6 * WorkingHours +
b7 * BikeLines +
b8 * CompPerPackage +
b9 * CompPerKM +
i1 * ToolSupply.dummy[1] * Weather.dummy[1] +
i2 * ToolSupply.dummy[1] * BikeLines +
i3 * ToolSupply.dummy[1] * Traffic.dummy[1] +
i4 * ToolSupply.dummy[1] * Traffic.dummy[2] +
i5 * ToolSupply.dummy[1] * Insurance.dummy[1] +
i6 * ToolSupply.dummy[1] * Insurance.dummy[2] +
i10 * BikeLines * Weather.dummy[1] +
i11 * Traffic.dummy[1] * Weather.dummy[1] +
i12 * Traffic.dummy[2] * Weather.dummy[1] +
i13 * Insurance.dummy[1] * Weather.dummy[1] +
i14 * Insurance.dummy[2] * Weather.dummy[1] +
i16 * CompPerPackage * Weather.dummy[1] +
i17 * CompPerKM * Weather.dummy[1] +
i18 * BikeLines * Insurance.dummy[1] +
i19 * BikeLines * Insurance.dummy[2] +
i20 * BikeLines * WorkingHours.dummy[1] +
i21 * BikeLines * WorkingHours.dummy[2] +
i22 * BikeLines * WorkingHours.dummy[3] +
i23 * Traffic.dummy[1] * Insurance.dummy[1] +
i24 * Traffic.dummy[1] * Insurance.dummy[2] +
i25 * Traffic.dummy[2] * Insurance.dummy[1] +
i26 * Traffic.dummy[2] * Insurance.dummy[2] +
i33 * Insurance.dummy[1] * WorkingHours.dummy[1] +
i34 * Insurance.dummy[1] * WorkingHours.dummy[2] +
i35 * Insurance.dummy[1] * WorkingHours.dummy[3] +
i36 * Insurance.dummy[2] * WorkingHours.dummy[1] +
i37 * Insurance.dummy[2] * WorkingHours.dummy[2] +
i38 * Insurance.dummy[2] * WorkingHours.dummy[3] +
i39 * CompPerKM * WorkingHours.dummy[1] +
i40 * CompPerKM * WorkingHours.dummy[2] +
i41 * CompPerKM * WorkingHours.dummy[3] +
i42 * BikeLines * Traffic.dummy[1] $

When I edit it as follows:

Design
;alts = alt1*, alt2*
;rows = 80
;eff = (mnl, d)
;block = 20
;alg = mfederov
;model:
U(Alt1) = b1.dummy[0.00001] * ToolSupply[0,1] +
b2.dummy[0.00001] * Weather[0,1] +
b3.dummy[0.00001|0.00001] * Traffic[0,1,2] +
b4.dummy[0.00001|0.00001] * Insurance[0,1,2] +
b5.dummy[0.00001] * Locker[0,1] +
b6.dummy[0.00001|0.00001|0.00001] * WorkingHours[0,1,2,3] +
b7[0.00001] * BikeLines[0:1] +
b8[0.00001] * CompPerPackage[40:160] +
b9[0.00001] * CompPerKM[20:60] +
i1[0.00001] * ToolSupply.dummy[1] * Weather.dummy[1] +
i2[0.00001] * ToolSupply.dummy[1] * BikeLines +
i3[0.00001] * ToolSupply.dummy[1] * Traffic.dummy[1] +
i4[0.00001] * ToolSupply.dummy[1] * Traffic.dummy[2] +
i5[0.00001] * ToolSupply.dummy[1] * Insurance.dummy[1] +
i6[0.00001] * ToolSupply.dummy[1] * Insurance.dummy[2] +
i10[0.00001] * BikeLines * Weather.dummy[1] +
i11[0.00001] * Traffic.dummy[1] * Weather.dummy[1] +
i12[0.00001] * Traffic.dummy[2] * Weather.dummy[1] +
i13[0.00001] * Insurance.dummy[1] * Weather.dummy[1] +
i14[0.00001] * Insurance.dummy[2] * Weather.dummy[1] +
i16[0.00001] * CompPerPackage * Weather.dummy[1] +
i17[0.00001] * CompPerKM * Weather.dummy[1] +
i18[0.00001] * BikeLines * Insurance.dummy[1] +
i19[0.00001] * BikeLines * Insurance.dummy[2] +
i20[0.00001] * BikeLines * WorkingHours.dummy[1] +
i21[0.00001] * BikeLines * WorkingHours.dummy[2] +
i22[0.00001] * BikeLines * WorkingHours.dummy[3] +
i23[0.00001] * Traffic.dummy[1] * Insurance.dummy[1] +
i24[0.00001] * Traffic.dummy[1] * Insurance.dummy[2] +
i25[0.00001] * Traffic.dummy[2] * Insurance.dummy[1] +
i26[0.00001] * Traffic.dummy[2] * Insurance.dummy[2] +
i33[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[1] +
i34[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[2] +
i35[0.00001] * Insurance.dummy[1] * WorkingHours.dummy[3] +
i36[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[1] +
i37[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[2] +
i38[0.00001] * Insurance.dummy[2] * WorkingHours.dummy[3] +
i39[0.00001] * CompPerKM * WorkingHours.dummy[1] +
i40[0.00001] * CompPerKM * WorkingHours.dummy[2] +
i41[0.00001] * CompPerKM * WorkingHours.dummy[3] +
i42[0.00001] * BikeLines * Traffic.dummy[1] /

U(Alt2) = b1 * ToolSupply +
b2 * Weather +
b3 * Traffic +
b4 * Insurance +
b5 * Locker +
b6 * WorkingHours +
b7 * BikeLines +
b8 * CompPerPackage +
b9 * CompPerKM +
i1 * ToolSupply.dummy[1] * Weather.dummy[1] +
i2 * ToolSupply.dummy[1] * BikeLines +
i3 * ToolSupply.dummy[1] * Traffic.dummy[1] +
i4 * ToolSupply.dummy[1] * Traffic.dummy[2] +
i5 * ToolSupply.dummy[1] * Insurance.dummy[1] +
i6 * ToolSupply.dummy[1] * Insurance.dummy[2] +
i10 * BikeLines * Weather.dummy[1] +
i11 * Traffic.dummy[1] * Weather.dummy[1] +
i12 * Traffic.dummy[2] * Weather.dummy[1] +
i13 * Insurance.dummy[1] * Weather.dummy[1] +
i14 * Insurance.dummy[2] * Weather.dummy[1] +
i16 * CompPerPackage * Weather.dummy[1] +
i17 * CompPerKM * Weather.dummy[1] +
i18 * BikeLines * Insurance.dummy[1] +
i19 * BikeLines * Insurance.dummy[2] +
i20 * BikeLines * WorkingHours.dummy[1] +
i21 * BikeLines * WorkingHours.dummy[2] +
i22 * BikeLines * WorkingHours.dummy[3] +
i23 * Traffic.dummy[1] * Insurance.dummy[1] +
i24 * Traffic.dummy[1] * Insurance.dummy[2] +
i25 * Traffic.dummy[2] * Insurance.dummy[1] +
i26 * Traffic.dummy[2] * Insurance.dummy[2] +
i33 * Insurance.dummy[1] * WorkingHours.dummy[1] +
i34 * Insurance.dummy[1] * WorkingHours.dummy[2] +
i35 * Insurance.dummy[1] * WorkingHours.dummy[3] +
i36 * Insurance.dummy[2] * WorkingHours.dummy[1] +
i37 * Insurance.dummy[2] * WorkingHours.dummy[2] +
i38 * Insurance.dummy[2] * WorkingHours.dummy[3] +
i39 * CompPerKM * WorkingHours.dummy[1] +
i40 * CompPerKM * WorkingHours.dummy[2] +
i41 * CompPerKM * WorkingHours.dummy[3] +
i42 * BikeLines * Traffic.dummy[1] $

I get the following Error: The number of rows specified in the ';rows' property is greater than the size of the full factorial, which has 0 rows.

1- I would like to inquire how I can specify that the following variables—compensation per package, compensation per km, and proportion of bike lanes—are continuous in the syntax provided.
2- Additionally, the error value (related to the above first syntax I provided) is approximately 0.20. Could you kindly advise if this is a valid value or if further adjustments are necessary?

Thank you!

Beyza

[Michiel Bliemer]

For continuous attribute levels, which are not commonly used in experimental design, you need to switch the algorithm:

;alg = neldermead

However, note that optimising over a continuous space using zero priors will likely result in only the extreme levels of your range being used. So specifying range [40:160] will likely result in mostly values 40 and 160 being used because this is optimal from an efficiency point of view as the trade-offs are larger. So for this reason, most people prefer to use a discrete number of attribute levels in the design, to ensure that there are trade-offs not only at the extreme values, but also with middle levels.

Even when using discrete levels, say [40,80,120,160], when using (near) zero priors, the most efficient design would be comparing 40 vs 160 and 80 vs 120 across the alternatives, so this would again limit the variation in the data. So when I have zero priors, I typically use either an orthogonal design or I dummy code all attributes, including numerical attributes.

To answer your second question, D-errors have no meaning as they are case specific. It is not possible to say whether 0.2 is good or bad, all you can do is compare D-errors for the same model and priors across different designs and pick the design with the lowest D-error.

Michiel