# Pinewood Derby Race Method Evaluation A CASE STUDY

## Problem Summary

A group of 28 Cub Scouts will be ready to race their Pinewood Derby Cars in hopes of winning one of the 4 Trophies their pack offers and in hopes of being one of the 4 Scouts who will represent our pack at the big District Races. The Pack Committee has asked me to recommend a racing plan that will do a accurate job of awarding the trophies and selecting our pack's representatives.

In addition to accuracy, the committee would like for all of the Cubs to be involved for most of the racing. They asked for something other than a classic elimination style competition because of the relatively early "fall-out" of so many participants. They think that since the boys put so much effort into building their cars, they ought to get to race them more than 3 or 4 times.

Finally, they would like for the racing to last about 60 to 90 minutes!

We have a 4-lane track with an electronic finish line that registers finish places 1 through 4 for the heat. If the contestants for each heat were known in advance, about 90 4-car heats could be completed in the available time.

The following are some cases I evaluated:

1. Stearns: 13 rounds, 7 heats per round (91 heats).

2. Stearns: 10 rounds, 7 heats per round (70 heats).
Select 13 top scorers as participants in Perfect 13-4 (13 heats). Total 83 heats.

3. Stearns: 10 rounds, 7 heats per round (70 heats).
Select 7 top scorers as participants in Perfect 7-3 (7 heats). Total 77 heats.

4. Stearns: 10 rounds, 7 heats per round (70 heats).
Select 7 top scorers as participants in Complementary-Perfect 7-3 (14 heats). Total 84 heats.

5. Young and Pope: 2 rounds, 28 heats per round (56 heats).
Select 7 top scorers as participants in Complementary-Perfect 7-3 (14 heats). Total 70 heats.

The 10-Round Stearns could be the first 10 rounds of a 13-round Chart. But how well would it work? It all depends upon how well the Stearns chart fragment could select the fastest 4 cars as a group of 7 or 13 cars. As it turns out, the 13-round Stearns chart and the 10-round Stearns fragment perform much better for screening than for trophy assignment or representative selection.

Each of these cases has the flexibility to adapt to other initial numbers of participants.

## Summary of Results

Method4-Trophy ErrorTop-4 ErrorTotal Time
(minutes)
Finals Time
(minutes)
% of Time with
Full Participation
13-Round Stearns28.10%7.30%910100%
10-Round Stearns
plus P13-4 (1)
23.12%5.32%831384%
10-Round Stearns
plus P7-3 (1)
19.30%3.77%77791%
10-Round Stearns
plus CP7-3 (2)
12.21% 2.72%841483%
2-Round Young and Pope
plus CP7-3 (2)
12.19% 2.70%701480%

## Case 1: 13-Round (91 heat) Stearns

From the 13-Round Stearns simulation, we immediately see the following information:

Accuracy measures: 13-Round 28-Car Stearns
TrophyRepresentation
4-Trophy:71.90%Top-4:92.70%

Error measures: 13-Round 28-Car Stearns
TrophyRepresentation
4-Trophy:28.10%Top-4:7.30%

Error Measures are "the other side" of accuracy measures. If I got 72% right, then I got 100% - 72% = 28% wrong! (I like to use error measures because the impact of decisions is easier to visualize. For instance if the method error is 28%, then about one in three of the results is wrong! If I use a method with 28% error to award the 4 top trophies, then one or more of the boys (on the average) did not get a trophy that he deserved. For me, thinking this way changes the analysis from a mathematical exercise to a study of real effects on real, live boys.)

On the plus side, all of the scouts were involved in the racing from beginning to end.

## Case 2: 10-Round (70 heat) Stearns plus(13 heat) Perfect 13-4 (1)

Use a 10 round Stearns Chart to select 13 finalists; Race the finalists using a Perfect 13-4 (1) Chart.

From the 10-Round Stearns simulation, we immediately see the following information:

Accuracy measures: 10-Round 28-Car Stearns
TrophyRepresentationScreening
13-Trophy:(ignored)Top-13:(ignored)4n13:100.00%

From the Perfect 13-4 (1) simulation, we immediately see the following information:

Accuracy measures: P13-4 (1)
TrophyRepresentation
4-Trophy:76.88%Top-4:94.68%

To form the composite accuracies, the two relevant measures are multiplied... 100% of the deserving advanced and 76.88% of those present were awarded the correct trophy.

Accuracy measures: Composite
TrophyRepresentation
4-Trophy:76.88%Top-4:94.68%

Again, error measures tell the real tale...

Error measures: Composite
TrophyRepresentation
4-Trophy:23.12%Top-4:5.32%

The accuracy is better. About a quarter of the trophies are, on the average, awarded to the wrong boy.

This method will keep all of the Scouts racing for the first 80+% of the time (10 races each, 70 minutes). The last few minutes will be spent running the finals: 13 heats with 4 races for each of 13 scouts.

## Case 3: 10-Round (70 heat) Stearnsplus (7 heat) Perfect 7-3 (1)

Use a 10-round Stearns Chart to select 7 finalists; Race the finalists using a Perfect 7-3 (1) Chart.

From the 10-Round Stearns simulation, we immediately see the following information:

Accuracy measures: 10-Round 28-Car Stearns
TrophyRepresentationScreening
7-Trophy:(ignored)Top-7:(ignored)4n7:99.88%

From the Perfect 7-3 (1) simulation, we immediately see the following information:

Accuracy measures: P7-3 (1)
TrophyRepresentation
4-Trophy:80.80% Top-4:96.35%

To form the composite accuracies, the two relevant measures are multiplied... 99.88% of the deserving advanced and 80.80% of those present were awarded the correct trophy. So 99.88% of 80.80% yields a composite average of 80.70%.

Accuracy measures: Composite
TrophyRepresentation
4-Trophy:80.70% Top-4:96.23%

The error measures say ...

Error measures: Composite
TrophyRepresentation
4-Trophy:19.30%Top-4:3.77%

The accuracy is better. About a fifth of the trophies are, on the average, awarded to the wrong boy.

This method will keep all of the Scouts racing for the first 90+% of the time (10 races each, 70 minutes). The last few minutes will be spent running the finals: 7 heats with 3 races for each of 7 scouts.

## Case 4: 10-Round (70 heat) Stearnsplus (14 heat) Complementary-Perfect 7-3 (2)

Use a 10-round Stearns Chart to select 7 finalists; Race the finalists using a Complementary-Perfect 7-3 (2) Chart.

From the 10-Round Stearns simulation, we again see the following information:

Accuracy measures: 10-Round 28-Car Stearns
TrophyRepresentationScreening
7-Trophy:(ignored)Top-7:(ignored)4n7:99.88%

From the Complementary-Perfect 7-3 (2) simulation, we immediately see the following information:

Accuracy measures: CP7-3 (2)
TrophyRepresentation
4-Trophy:87.90%Top-4:97.40%

To form the composite accuracies, the two relevant measures are multiplied... 99.88% of the deserving advanced and 87.90% of those present were awarded the correct trophy. So 99.88% of 87.90% yields a composite average of 87.79%

Accuracy measures: Composite
TrophyRepresentation
4-Trophy:87.79%Top-4:97.28%

The error measures say ...

Error measures: Composite
TrophyRepresentation
4-Trophy:12.21% Top-4:2.72%

The accuracy is better. About one eighth of the trophies are, on the average, awarded to the wrong boy. The representation error has been reduced to less than one in thirty!

This method will keep all of the Scouts racing for the first 80+% of the time (10 races each, 70 minutes). The last few minutes will be spent running the finals: 14 heats with 6 races for each of 7 scouts.

## Case 5: 2-Round (56 heat) 2-Round Young and Popeplus (14 heat) Complementary-Perfect 7-3 (2)

Use a 2-round Young and Pope (Partial Perfect N) Chart to select 7 finalists; Race the finalists using a Complementary-Perfect 7-3 (2) Chart.

From the 2-round Young and Pope simulation, we see the following information:

Accuracy measures: 2-Round 28-Car Young and Pope
TrophyRepresentationScreening
7-Trophy:(ignored)Top-7:(ignored)4n7:99.90%

From the Complementary-Perfect 7-3 (2) simulation, we immediately see the following information:

Accuracy measures: CP7-3 (2)
TrophyRepresentation
4-Trophy:87.90%Top-4:97.40%

To form the composite accuracies, the two relevant measures are multiplied... 99.90% of the deserving advanced and 87.90% of those present were awarded the correct trophy. So 99.90% of 87.90% yields a composite average of 87.81%

Accuracy measures: Composite
TrophyRepresentation
4-Trophy:87.81%Top-4:97.30%

The error measures say ...

Error measures: Composite
TrophyRepresentation
4-Trophy:12.19% Top-4:2.70%

The accuracy is about the same as Case 4. About one eighth of the trophies are, on the average, awarded to the wrong boy. The representation error has been reduced to less than one in thirty!

This method will keep all of the Scouts racing for the first 80% of the time (8 races each, 56 minutes). The last few minutes will be spent running the finals: 14 heats with 6 races for each of 7 scouts.

## Reference: Analysis Tool, Parameters and Explanation

constructed a very nice piece of software for chart simulation, "DerbySim". It is available in the "Software" section of his Cub Scout Pack's Website, The results are more or less consistent with what I produced, but his program is much more sophistocated than what I built for my C-64. The results are not directly comparable because, with his mathematics and programming prowess, his more powerful computer and his better programming languages, he was able to bridge many of my simplifying assumptions.

The following table shows the parameters that I used with "DerbySim" program for this study. These are not necessarily the best numbers to model the track and competitors. They seem reasonable, but runs using other models may be useful in validating the overall strategy.

ParametersCountMeanStd. Deviation
Car Speed  10.0"
Lane Speed  1.0"
Lub Breakin .5".2"
Threshold2 1
Random element  .5"
Reduction4
Trials1000
ScoringLinear: 4-3-2-1

Here is a brief description of the numbers:

Car Speed
Finish location in inches relative to an imaginary "reference car" attributable to the car's speed. Randomized at the beginning of each trial.
Lane Speed
Finish location in inches relative to an imaginary "reference car" attributable to the each lane's characteristics. Randomized at the beginning of each trial.
Lub Breakin
Finish location in inches relative to an imaginary "reference car" attributable to the performance improvement from "breakin" of the lubricant. Randomized at the beginning of each trial.
Finish location in inches relative to an imaginary "reference car" attributable to the performance degradation from loss of the lubricant. Randomized at the beginning of each trial.
Threshold
The number of runs at which the lubricant transitions from breakin to degradation.
Random element
From run to run a car will experience a variation from its average performance within +/- 0.5 inches 67% of the time. and within +/- 1.0 inch 96% of the time. This factor is randomized for each run of each car.
Reduction
Number of top cars required from the screening.
Trials
How many times the chart was simulated to produce the averages quoted.
Tie-Break
Scoring
Linear: 1st place, 4 points; 2nd place, 3 points; etc.

## Reference: Analysis of 13-Round Stearns

This is a 13-Round Stearns Chart, as published at fine website. The chart was created using Peronto's implementation of Stearns Method.

Chart:
Within each round, rows are heats, columns are lanes. For example in Round 1 Car 7 races in lane 2 of heat 4.

Round 1 Round 2 Round 3 Round 4
1714242202542429271827112610
112015221432811211161518132514
25216132681358241441915817
47387122111326103232207
518231622271861217112535928
128122615162928197202442221
910192710231719622523166112

Round 5 Round 6 Round 7 Round 8
2191181821320212326224162619
517277512415107222441068
181028152728231427168202312018
13192571316255141219211525
1323241292217261841172231227
206261412281028625917132821
1642221164191513113714915

Round 9 Round 10 Round 11 Round 12
6243228248991220231523311
102052117320132214182724188
23825224523122127426134920
1716117161410119132282114726
13271926152527176151810162822
42812152119222241116312195
141918266718115251076171225

Round 13
26132215
3162721
202495
2523828
111172
141964
1218710

Although the chart has some imperfections, it does seem to conform to the Stearns Specifications. It also has the Stearn's strengths, in that every car races 13 times!

The chart is scored with linear point assignment... 1st place in heat = 4 points, 2nd place = 3 points, etc. Other simulations indicate that this scoring method provides more accurate results than the "1st place in heat = 1 point, rest of cars 0 points" scoring recommended in the Stearns program documentation.

The accuracy measures are explained in good detail in the documentation for DerbySim. In summary, they are ...

N-Trophy
Percentage of time that a car among the N objectively fastest receives correct trophy.
Top-N
Percentage of time that a car among the N objectively fastest placed in top N.
NnX
Percentage of time that a car among the N objectively fastest placed in top X. This measures the chart's utility as a "screening" tool.

Accuracy measures: 13-Round 28-Car Stearns
TrophyRepresentationScreening
1-Trophy:90.40%Top-1:90.40%
2-Trophy:83.70%Top-2:91.50%
3-Trophy:77.03%Top-3:91.90%
4-Trophy:71.90%Top-4:92.70%4n4:92.70%
5-Trophy:67.40%Top-5:93.04%4n5:98.38%
6-Trophy:62.90%Top-6:92.65%4n6:99.50%
7-Trophy:59.49%Top-7:93.79%4n7:99.88%
8-Trophy:57.06%Top-8:94.65%4n8:99.97%
9-Trophy:55.26%Top-9:95.24%4n9:100.00%
10-Trophy:53.47%Top-10:95.21%4n10:100.00%
11-Trophy:51.68%Top-11:95.28%4n11:100.00%
12-Trophy:50.08%Top-12:95.71%4n12:100.00%
13-Trophy:49.05%Top-13:96.11%4n13:100.00%

These are measures of the specific Stearns chart in the example, and not for Stearns charts in general. It is likely that other runs of the Stearns generator with the same parameters would produce charts with other accuracy characteristics. It seems likely that those measures would not vary markedly from what is depicted here.

The most important measures of this chart, according to its proposed usage, will be its ability to accurately assign trophies for first through fourth place and to select the four representatives to the district races.

Measures through 13 are included for comparison to the measures of the 10-round partial chart below.

## Reference: Analysis of 10-Round Stearns

The chart for this alternative is made using the first 10 rounds of the 13-Round Stearns chart above. As would be expected, the accuracy measures fall off slightly with the reduction in Rounds.

This Stearns fragment may not satisfy all of the requirements for a Stearns Method Chart. In particular, it may be inferior to a properly generated 10-Round Stearns Chart. It is used in this study because it should have many of the same performance characteristics as its 13-Round parent, and so will introduce fewer variables into the comparisons.

Like all Stearns Charts, all the cars participate regularly throughout the competition. Each car races 10 times.

Accuracy measures: 10-Round 28-Car Stearns
TrophyRepresentationScreening
1-Trophy:90.30%Top-1:90.30%
2-Trophy:82.60%Top-2:91.15%
3-Trophy:75.50%Top-3:90.67%
4-Trophy:69.30%Top-4:90.85%4n4:90.85%
5-Trophy:64.08%Top-5:91.92%4n5:97.38%
6-Trophy:60.18%Top-6:92.75%4n6:99.38%
7-Trophy:57.04%Top-7:93.47%4n7:99.88%
8-Trophy:54.69%Top-8:93.55%4n8:99.95%
9-Trophy:52.43%Top-9:94.24%4n9: 99.97%
10-Trophy:50.44%Top-10:94.61%4n10: 100.00%
11-Trophy:48.84%Top-11:94.81%4n11: 100.00%
12-Trophy:47.40%Top-12:95.40%4n12: 100.00%
13-Trophy:46.18%Top-13:95.58%4n13: 100.00%

These are measures of the specific 10-Round Stearns chart fragment in the example, and not for Stearns charts or fragments in general. See additional comments under the 13-Round Stearns chart accuracy measures.

The most important measures of this chart, according to its proposed usage, will be its ability to include the top 4 cars in its selection of the Top N, 4n6, 4n7, etc.

## Reference: Analysis of 2-Round Young and Pope

The chart for this alternative is made using the publicly available Young and Pope (Partial Perfect N) Chart Generator, which was created by Cory Young and Stan Pope. Accuracy measures are comparable to the 10-Round Stearns chart fragment.

Chart Generation parameters used were:

• Number of lanes: 4
• Number of cars: 28
• Number of rounds: 2
• Heat Ordering Options: all "Medium"

2-Round 28-Car Young and Pope
Round 1 Round 2
151016 284915
26111719261816
371218251612
48131981575
5914202322220
243232114211311
2872725182826
11152026101797
131722282762624
61015211620253
141823151242
25424221923286
121621271118108
151924215221412
22263920271917
81217239131824
101419257111622
2627131721264
91686310228
182227516231513
41131273814
212820182652523
232741029127
2024172428511
121911971464
21252813201210
6135318251715
172416142212119

Like the Stearns Charts, all the cars participate regularly throughout the competition. Each car races 8 times.

Accuracy measures: 2-Round 28-Car Young and Pope
TrophyRepresentationScreening
1-Trophy:90.70%Top-1:90.70%
2-Trophy:82.30%Top-2:90.00%
3-Trophy:76.30%Top-3:92.27%
4-Trophy:71.93%Top-4:92.72%4n4:92.72%
5-Trophy:68.08%Top-5:93.38%4n5:98.03%
6-Trophy:65.18%Top-6:94.03%4n6:99.45%
7-Trophy:62.57%Top-7:94.64%4n7:99.90%
8-Trophy:60.36%Top-8:95.11%4n8:100.00%
9-Trophy:58.70%Top-9:95.33%4n9:100.00%
10-Trophy:57.13%Top-10:95.60%4n10:100.00%
11-Trophy:55.61%Top-11:95.98%4n11:100.00%
12-Trophy:54.27%Top-12:96.20%4n12:100.00%
13-Trophy:53.22%Top-13:96.61%4n13:100.00%

The most important measures of this chart, according to its proposed usage, will be its ability to include the top 4 cars in its selection of the Top N, 4n6, 4n7, etc.

## Reference: Analysis of Perfect 13-4 (1)

This is a Perfect-13 Chart. It was produced using the publicly available Perfect-N Chart Generator, which was created by Cory Young and Stan Pope.

P13-4 (1)
12410
23511
34612
45713
5681
6792
78103
89114
910125
1011136
111217
121328
13139

Accuracy measures: P13-4 (1)
TrophyRepresentationScreening
1-Trophy:90.90%Top-1:90.90%
2-Trophy:84.85%Top-2:92.90%
3-Trophy:80.37%Top-3:93.97%
4-Trophy:76.88%Top-4:94.68%4n4:94.68%

The most important measures of this chart, according to its proposed usage, will be its ability to accurately assign trophies for first through fourth place and to select the four representatives to the district races.

## Reference: Analysis of Perfect 7-3 (1)

This is a Perfect-7 Chart. It was produced using the publicly available Perfect-N Chart Generator, which was created by Cory Young and Stan Pope.

P7-3 (1)
124
235
346
457
561
672
713

Accuracy measures: P7-3 (1)
TrophyRepresentationScreening
1-Trophy:90.60%Top-1:90.60%
2-Trophy:85.65%Top-2:94.45%
3-Trophy:82.87%Top-3:95.43%
4-Trophy:80.80%Top-4:96.35%4n4:96.35%

The most important measures of this chart, according to its proposed usage, will be its ability to accurately assign trophies for first through fourth place and to select the four representatives to the district races.

## Reference: Analysis of Complementary-Perfect 7-3 (2)

This is a Complementary Perfect-7 Chart. It was produced using the publicly available Perfect-N Chart Generator, which was created by Cory Young and Stan Pope.

Note that each head to head matchup occurs twice, in alternating lanes.

CP7-3 (2)
124
653
175
346
672
431
216
457
235
764
327
561
713
542

Accuracy measures: CP7-3 (2)
TrophyRepresentationScreening
1-Trophy:95.20%Top-1:95.20%
2-Trophy:92.25%Top-2:96.75%
3-Trophy:89.93%Top-3: 97.13%
4-Trophy:87.90%Top-4: 97.40%4n4:97.40%

The most important measures of this chart, according to its proposed usage, will be its ability to accurately assign trophies for first through fourth place and to select the four representatives to the district races.

## Acknowledgements

This study has benefitted, in large measure, from frequent counsel, ideas and evaluation software provided by Cory Young. His patient, thoughtful responses to my inquiries have helped bring the project to a swift completion.

Latest update:
4/8/2001: Added Case 5 using Partial Perfect N as Screening Method
1/6/98: New simulations using 1/5/98 version of DerbySim